Kubernetes ServiceAccount Controller

In the world of Kubernetes, humans have “User Accounts” (managed by Google/AWS/LDAP), but robots and applications need their own ID cards too. These ID cards are called ServiceAccounts.

The ServiceAccount Controller is the manager that makes sure every “room” (Namespace) in your cluster has a default ID card available. It also helps manage the secure tokens (passwords) that your Pods use to talk to the API Server. Basically, it ensures your Pods have an identity so they aren’t strangers in the cluster.

Think of the ServiceAccount Controller as the “Office Security Desk” for employees (Pods).

• The Pod: A new employee joining a department (Namespace).
• ServiceAccount: The employee ID badge.
• The Controller: The security officer who ensures every department has a stash of blank ID badges (the default ServiceAccount) and issues secure, temporary access cards (Tokens) so employees can open doors (API calls). If you don’t bring your own ID, they give you the “Visitor” (default) badge.

Key Characteristics

• “Since Kubernetes 1.24, creating a ServiceAccount does NOT automatically create a Secret object anymore.”
• “If you don’t specify a serviceAccountName in your Pod, it automatically gets the default one.”
• “This controller ensures that if you delete the default SA, it gets recreated immediately.”

Feature	Description
Primary Goal	Manage lifecycle of ServiceAccounts (SAs) and their tokens.
Default Behavior	Ensures a default SA exists in every active Namespace.
Token Handling	Legacy: Auto-created Secrets. Modern (v1.24+): Ephemeral TokenRequest (no secrets).
Safety Feature	Automount: Controls if the token is automatically injected into Pods.
Component	Runs inside the kube-controller-manager (KCM).

The ServiceAccount Controller is a core control loop within the kube-controller-manager. Its primary responsibility is to manage the ServiceAccount resources within the cluster. It ensures that the default ServiceAccount exists in every Namespace.

Historically, this controller was also responsible for creating a long-lived Secret (containing a JWT token) for every ServiceAccount. However, in modern Kubernetes (v1.24+), this behavior has changed. The controller now primarily relies on the TokenRequest API to provide short-lived, rotatable tokens that are projected directly into Pods as volumes, rather than storing them as static Secrets in etcd. This improves security by reducing the attack surface of stolen static credentials.

–

• The “Default” Account:
  • When you create a Namespace my-app, Kubernetes automatically creates a ServiceAccount named default.
  • If you launch a Pod without mentioning an ID, it uses this default account.
  • Warning: Usually, the default account has no permissions (it can’t do anything), which is good for security.
• Image Pull Secrets:
  • If you need to pull images from a private registry (like a private Docker Hub), you can attach the credentials to the ServiceAccount. The controller ensures any Pod using that account automatically gets those credentials.

DevSecOps Architect Level

• Lifecycle Management:
  • Namespace Creation: The controller watches for new Namespaces and instantaneously creates a ServiceAccount named default.
  • Namespace Deletion: When a Namespace is terminating, the controller assists in cleaning up the SAs.
  • Resurrection: If you manually kubectl delete sa default, the controller sees this loop and recreates it instantly.
• The “Token Controller” Split (Critical for v1.24+):
  • Pre-1.24: The ServiceAccount Controller observed SA creation and triggered the creation of a Secret of type kubernetes.io/service-account-token.
  • Post-1.24: This automatic secret creation is disabled.
  • Implication: If you need a static token (e.g., for an external CI/CD tool to talk to K8s), you must manually create a Secret and annotate it with kubernetes.io/service-account.name: <sa-name>. The Token Controller (a separate loop) will then populate it.
• Root CA Injection:
  • The controller (via the Admission plugin) ensures that the cluster’s Root CA certificate (ca.crt) is mounted into the Pod at /var/run/secrets/kubernetes.io/serviceaccount/. This allows the Pod to verify it is actually talking to the real API Server and not a hacker.

Additional Details

• The Admission Controller Interaction:
  • The ServiceAccount Controller manages the SA object itself.
  • The ServiceAccount Admission Controller (a plugin in the API Server) is what actually modifies your Pod to add the volume mount for the token. They work in tandem.
• Automounting Logic:
  • You can disable token mounting at two levels:
    1. ServiceAccount Level: automountServiceAccountToken: false (Applies to all pods using this SA).
    2. Pod Level: automountServiceAccountToken: false (Applies to just this Pod).
  • Logic: The Pod setting overrides the SA setting.
• Security Gap:
  • If you bind the cluster-admin role to the default ServiceAccount in the default namespace (a common rookie mistake), every pod you launch becomes a super-admin. Never do this!

Key Components

1. ServiceAccount Object: The resource itself (kind: ServiceAccount).
2. TokenRequest API: The backend API used to fetch time-bound tokens.
3. Projected Volume: The method used to inject the token file into the container.

Use Cases

• Pod Identity: Allowing a Pod to talk to the API server (e.g., a Jenkins Agent needing to spin up new pods).
• Workload Identity: Integrating with Cloud Providers (AWS IRSA, Google Workload Identity). The Cloud Provider trusts the Kubernetes ServiceAccount token.
• Image Pulling: Storing imagePullSecrets centrally so you don’t have to put them in every Pod definition.

Best Practices

• Disable Automount by Default: Most apps (Nginx, Redis) don’t need to talk to the K8s API. Set automountServiceAccountToken: false to reduce the attack surface.
• Dedicated SAs: Never use default. Create a specific SA for each application (e.g., web-sa, db-sa).
• Clean up Unused SAs: Use tools to find SAs that are not used by any Pods.

Common Issues

1. “Secret Not Found” (v1.24+):
   • Problem: You created an SA and are looking for its secret, but kubectl get sa my-sa -o yaml shows empty secrets.
   • Solution: Use kubectl create token my-sa to get a token, or manually create a Secret object if you really need a long-lived one.
2. Pod Stuck in ContainerCreating:
   • Problem: The ServiceAccount specified in the Pod does not exist.
   • Solution: The Admission Controller rejects the Pod creation, or it hangs waiting for the SA.
3. Token Expiration:
   • Problem: Bound tokens expire (usually after 1 hour). If your app caches the token and doesn’t reload it from the disk, it will start getting 401 Unauthorized errors.
   • Solution: Ensure your app re-reads the token file from /var/run/secrets/... periodically.

Limitations

• Namespace Scoped: A ServiceAccount in Namespace A cannot be used by a Pod in Namespace B.
• Credential Rotation: Rotating the signing keys for ServiceAccounts is a complex cluster-admin operation that triggers a restart of all pods to get new tokens.

• Service Accounts: https://kubernetes.io/docs/concepts/security/service-accounts/
• Managing Service Accounts: https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/