IAM Roles for Service Accounts IRSA across multi-cloud environments Lab

By /

Implementing IAM Roles for Service Accounts (IRSA) across multi-cloud environments (e.g., authenticating GKE, AKS, or on-premises Kubernetes workloads to AWS) requires decoupling the AWS IAM OIDC federation mechanism from the Amazon EKS mutating admission webhook.

In a native EKS environment, the EKS control plane automatically configures the cluster’s OIDC discovery endpoint and runs a webhook that intercepts Pod creation to inject AWS environment variables and the Kubelet-projected OIDC token volume.

For multi-cloud workloads, you must manually architect these three components:

  1. Expose the external Kubernetes cluster’s Service Account Issuer discovery endpoint publicly.
  2. Register the external cluster’s OIDC issuer URL as an Identity Provider in AWS IAM.
  3. Manually construct the Pod specification with a projected Service Account token volume and explicit AWS SDK environment variables.

Step 1: Retrieve the External Cluster’s OIDC Issuer URL

Your external Kubernetes cluster must have Service Account Token Volume Projection and OIDC discovery enabled, and the JSON Web Key Set (JWKS) endpoint must be publicly accessible to AWS STS.

For Google Kubernetes Engine (GKE), you can retrieve the active OIDC issuer URL using the Google Cloud CLI:

Bash
# Retrieve K8s OIDC Issuer for a GKE cluster
export ISSUER_URL=$(gcloud container clusters describe gke-cluster \
    --region us-central1 \
    --format="value(workloadIdentityConfig.workloadPool)")

# For AKS or custom clusters, this is typically a public HTTPS URL 
# ending in /.well-known/openid-configuration
echo $ISSUER_URL

Step 2: Register the OIDC Provider in AWS

AWS IAM must trust the external K8s cluster’s identity provider. You must fetch the TLS certificate thumbprint of the external K8s API server’s domain and register it.

Bash
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
export ISSUER_HOST=$(echo $ISSUER_URL | awk -F/ '{print $3}')

# Fetch the top-level CA thumbprint for the OIDC endpoint
export THUMBPRINT=$(echo | openssl s_client -servername $ISSUER_HOST -showcerts -connect $ISSUER_HOST:443 2>/dev/null | openssl x509 -in /dev/stdin -fingerprint -noout | awk -F= '{print tolower($2)}' | tr -d ':')

# Create the OIDC Identity Provider in AWS IAM
aws iam create-open-id-connect-provider \
    --url "https://$ISSUER_URL" \
    --client-id-list "sts.amazonaws.com" \
    --thumbprint-list $THUMBPRINT

Step 3: Create the AWS IAM Role and Trust Policy

Create an IAM role that explicitly trusts the external OIDC provider and restricts assumption to a specific K8s namespace and service account name (e.g., default namespace, multicloud-sa service account).

Bash
export OIDC_PROVIDER_ARN="arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/$ISSUER_URL"

cat <<EOF > multicloud-trust-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "$OIDC_PROVIDER_ARN"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "$ISSUER_URL:aud": "sts.amazonaws.com",
          "$ISSUER_URL:sub": "system:serviceaccount:default:multicloud-sa"
        }
      }
    }
  ]
}
EOF

# Create Role
ROLE_ARN=$(aws iam create-role \
    --role-name MultiCloudIRSARole \
    --assume-role-policy-document file://multicloud-trust-policy.json \
    --query 'Role.Arn' \
    --output text)

# Attach a testing policy (e.g., S3 ReadOnly)
aws iam attach-role-policy \
    --role-name MultiCloudIRSARole \
    --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

Step 4: Configure the Kubernetes Workload (The Manual Injection)

Because the external cluster lacks the EKS pod identity mutating webhook, you must manually recreate what the webhook does. This involves defining a ServiceAccount and explicitly configuring the Pod to project an OIDC token and set the exact environment variables expected by the AWS SDKs (AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE).

Bash
cat <<EOF > multicloud-pod.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: multicloud-sa
  namespace: default
---
apiVersion: v1
kind: Pod
metadata:
  name: aws-cli-multicloud
  namespace: default
spec:
  serviceAccountName: multicloud-sa
  containers:
  - name: aws-cli
    image: amazon/aws-cli:latest
    command: ["sleep", "3600"]
    env:
    # 1. Provide the IAM Role ARN
    - name: AWS_ROLE_ARN
      value: "$ROLE_ARN"
    # 2. Point the SDK to the projected token path
    - name: AWS_WEB_IDENTITY_TOKEN_FILE
      value: "/var/run/secrets/aws-iam-token/token"
    volumeMounts:
    - name: aws-iam-token
      mountPath: /var/run/secrets/aws-iam-token
      readOnly: true
  volumes:
  # 3. Request a projected service account token tailored for AWS STS
  - name: aws-iam-token
    projected:
      sources:
      - serviceAccountToken:
          audience: sts.amazonaws.com
          expirationSeconds: 86400
          path: token
EOF

# Deploy to the external K8s cluster
kubectl apply -f multicloud-pod.yaml

Step 5: Verify Federation Architecture

Once the Kubelet creates the Pod, it requests a JWT from the external K8s API server with the audience sts.amazonaws.com. It mounts this JWT into the container. When the aws CLI executes, the underlying AWS SDK reads the token and calls sts:AssumeRoleWithWebIdentity against AWS, exchanging the K8s JWT for temporary AWS credentials.

Verify this credential exchange within the external pod:

Bash
# Verify the injected K8s JWT exists
kubectl exec -it aws-cli-multicloud -- cat /var/run/secrets/aws-iam-token/token

# Verify AWS identity resolution via the exchanged token
kubectl exec -it aws-cli-multicloud -- aws sts get-caller-identity

# Execute an authorized AWS API call from K8s
kubectl exec -it aws-cli-multicloud -- aws s3 ls

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top