Kubernetes ServiceAccount

Welcome back to another exciting topic! When we work with Kubernetes, we as human developers or administrators use our own user accounts to log in and give commands. But what about the applications, scripts, and automation bots running inside the cluster? How do they prove who they are?

This is where ServiceAccounts come into the picture. A ServiceAccount is basically an identity card given to your Pods (your running applications) so they can securely talk to the Kubernetes API server and access the resources they need. It’s super important to understand this because it forms the absolute foundation of your cluster’s security!

Introduction & The Basics

Imagine your Kubernetes cluster is a highly secure corporate office building.

• Human Users: These are the employees who have standard HR-issued ID badges to enter the building.
• Pods (Applications): These are the delivery robots moving around inside the office.
• ServiceAccount: This is the special “Robot ID Badge” attached to the delivery bot. Whenever the robot needs to open a restricted door (like accessing a database secret or listing other pods), it scans this badge. Without the badge, the building’s security system (the API Server) will simply deny access!

Quick Reference

• Crucial Note: Since Kubernetes 1.24, creating a ServiceAccount does NOT automatically create a Secret object anymore.
• Default Behavior: If you don’t specify a serviceAccountName in your Pod, it automatically gets the default one.
• Auto-Healing: The ServiceAccount controller ensures that if you delete the default SA, it gets recreated immediately.

Feature	Description	Key Detail to Remember
Namespace Scope	SAs live inside a specific Namespace.	You cannot share an SA across different Namespaces.
Default SA	Automatically created in every Namespace.	Do not use it for your custom apps! Create a dedicated one.
Token Projection	How the password reaches the Pod.	Injected securely as a volume at /var/run/secrets/...
RBAC Binding	Giving permissions to the SA.	SAs have no power until you bind them to a Role/ClusterRole.

---

The Default ServiceAccount

When you create a new Namespace (e.g., my-app), Kubernetes automatically creates a ServiceAccount named default. If you launch a Pod without explicitly mentioning an ID, it automatically assumes this default account.

The ServiceAccount Controller manages its lifecycle: if you manually delete the default SA (kubectl delete sa default), the controller sees this and recreates it instantly. Usually, the default account has no permissions (it can’t do anything), which is good for baseline security.

Security Gap Warning: If you bind the cluster-admin role to the default ServiceAccount in the default namespace (a common rookie mistake), every pod you launch becomes a super-admin. Never do this!

How ServiceAccounts Work Under the Hood

A ServiceAccount provides an identity for processes that run in a Pod. Let’s break down the basic workflow so it is super easy to understand:

1. The Identity: You create a ServiceAccount object using a simple YAML file.
2. The Attachment: You tell your Pod to use this ServiceAccount by mentioning serviceAccountName: my-custom-sa in the Pod’s YAML spec.
3. The Magic (Admission Controller): When the Pod is starting up, the ServiceAccount Admission Plugin intercepts the creation and automatically mounts a tiny file containing a secure token into the Pod’s file system.
4. The Action: The application inside the Pod reads this token file and includes it in its HTTP headers when it asks the Kubernetes API server to do something.

How Tokens Work: The v1.24+ Security Upgrade

Modern Kubernetes (v1.24 onwards) has dramatically improved the security of ServiceAccounts by moving away from long-lived, static secrets stored in etcd.

• Pre-1.24: The ServiceAccount Controller observed SA creation and triggered the automatic creation of a permanent Secret.
• Post-1.24: This automatic secret creation is disabled. Instead, Kubernetes uses the TokenRequest API to fetch time-bound, ephemeral JSON Web Tokens (JWTs) that are mounted directly into the Pod.

If a token is somehow compromised today, it will expire very quickly, significantly reducing your security risk. If you absolutely need a static token (e.g., for an external CI/CD tool to talk to K8s), you must manually create a Secret and annotate it. The Token Controller (a separate loop) will then populate it.

Token Expiration and Rotation

The Kube-API server acts as the OIDC (OpenID Connect—an identity layer that verifies the client) issuer for these tokens, signing them with a private key. The kubelet handles token rotation proactively. If a token is valid for 1 hour, the kubelet requests a new one and updates the file in the Pod’s volume when it reaches 80% of its lifetime, or if it’s older than 24 hours.

Developer Note: Your application code must be written to re-read the token from the disk periodically. Legacy applications that cache the token in memory on startup will crash with a 401 error once the ephemeral token rotates!

The Authentication Flow (The “Token”)

When a Pod starts with a ServiceAccount, the Kubelet mounts a Projected Volume at /var/run/secrets/kubernetes.io/serviceaccount/.

• token: A signed JSON Web Token (JWT).
• ca.crt: The certificate to verify the API server.
• namespace: A text file containing the Pod’s namespace.

Pro-tip: Modern tokens are “bound” to the Pod’s UID and expire (default 1 hour). If the Pod is deleted, the token is instantly invalidated.

Authorization via RBAC

Identity is useless without permissions. You must link your ServiceAccount to a Role (namespaced) or ClusterRole (cluster-wide) using a Binding.

Component	Scope	Purpose
Role	Namespace	Defines what can be done (get, list, watch) on which resources.
RoleBinding	Namespace	Grants the Role to the ServiceAccount in a specific namespace.
ClusterRole	Global	Defines permissions for the entire cluster (e.g., list all Nodes).
ClusterRoleBinding	Global	Grants permissions across all namespaces.

Cross-Namespace Power: While a ServiceAccount object is strictly tied to the Namespace it is created in (Namespace A), you can grant it permissions in another Namespace (Namespace B). You do this by creating a RoleBinding in Namespace B that references the ServiceAccount from Namespace A.

DevSecOps Architect Level

In a production-grade DevSecOps environment, you do not use ServiceAccounts just for internal Kubernetes API access. You use them as a bridge to grant your K8s workloads access to external cloud services securely, without ever hardcoding passwords.

Modern Kubernetes clusters expose an OIDC Discovery Endpoint. External cloud providers hit this public endpoint to verify that the JWT token presented by your Pod was genuinely signed by your cluster.

Production Tools & Integrations:

• AWS IAM Roles for Service Accounts (IRSA): You can link a K8s SA to an AWS IAM Role. The K8s token is trusted by AWS STS to issue temporary AWS credentials. Official AWS IRSA Docs
• Google Cloud Workload Identity: Maps K8s ServiceAccounts to Google Cloud IAM Service Accounts. Official GCP Workload Identity Docs
• HashiCorp Vault K8s Auth: Allows Pods to log into Vault using their SA JWT token to fetch dynamic database credentials. Official Vault Kubernetes Auth Docs

Additional Details

1. Key Components
   • ServiceAccount Object: The K8s resource (kind: ServiceAccount).
   • ServiceAccount Admission Plugin: The component that intercepts Pod creation to inject the token volume.
   • TokenRequest API: The mechanism that mints the short-lived JWTs.
   • RBAC Bindings: The K8s resources that attach actual permissions to the SA.
2. Key Characteristics
   • Namespace-Bound: Tied strictly to the Namespace it is created in.
   • Lightweight: Costs almost no resources to create.
   • Automatically Managed: The cluster creates and cleans up default SAs automatically.
3. Use Case
   • A CI/CD pipeline agent (like a GitLab Runner Pod) needing permissions to deploy other K8s resources.
   • A monitoring agent (like Prometheus) needing to list all running Pods to scrape their metrics.
   • An application needing to authenticate to an external cloud database without storing a hardcoded connection string.
4. Benefits
   • No Hardcoded Passwords: Everything is dynamic and cryptographic.
   • Granular Security: You can give exactly the right amount of access to each individual microservice.
   • Auditing: Because every app uses its own SA, your K8s audit logs will show exactly which application performed an action, rather than a generic “admin” user.
5. Best Practices
   • One SA per Application: Never share a ServiceAccount between your Frontend app and your Database app.
   • Opt-in Token Mounting: Always set automountServiceAccountToken: false at the SA or Pod level, unless the application genuinely requires K8s API access.
   • Least Privilege: When binding a Role to an SA, only grant the exact verbs (get, list) and resources (pods, services) needed. Never grant * (wildcard) access.
6. Technical Challenges
   • Application Caching: Legacy applications that read the token file once on startup and keep it in memory will break when the ephemeral token rotates.
7. Limitations
   • You cannot easily use a ServiceAccount from one cluster to authenticate against a completely different, independent Kubernetes cluster without complex federation setups.
8. Common Issues
   • “Forbidden” Errors: The developer attached the SA to the Pod, but forgot to create the RoleBinding.
   • Empty Secrets: Looking for the SA secret using kubectl get secret in modern K8s versions and finding nothing (because they are now ephemeral, not stored in etcd as Secrets).
9. Problems and Solutions
   • Problem: I need to pull a private container image, but I don’t want to paste my Docker credentials into every single Pod YAML.
   • Solution: Store the credentials in a K8s Secret, and link that secret to your ServiceAccount using imagePullSecrets. Any Pod using that SA will automatically inherit the ability to pull the private image!

• https://kubernetes.io/docs/concepts/security/service-accounts/
• https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/

Conclusion

Mastering ServiceAccounts is a non-negotiable skill for any DevSecOps engineer. By treating every application as a unique identity and enforcing the principle of least privilege, you create a robust, secure environment where a compromised application is heavily restricted from doing further damage. Keep practicing, and you will be a Kubernetes security guru in no time!