4. Registries and The Secure Supply Chain

A Docker Registry is a secure warehouse where your “Gold Images” are stored, versioned, and protected before they go to the production servers.

Imagine your code is vital medicine.

• Unverified Public Image (e.g., docker run mongo): This is like buying a random bottle of pills from a street vendor. It has a label, but you don’t know who made it, if it’s expired, or if it’s been tampered with. It might cure you, or it might poison you.
• Your Secure Registry: This is a Verified Pharmacy. Every medicine bottle here has:
  • A Batch Number (Version Tag): You know exactly when and how it was made.
  • Tested for Purity (Vulnerability Scan): A lab has confirmed it contains no harmful substances (CVEs).
  • A Safety Seal (Digital Signature): The pharmacist (Kubernetes) checks the seal. If it’s broken, they refuse to dispense it to the patient (Production Server).

As an Architect, your job is to ensure the “medicine” stays sterile from the Lab (CI/CD) to the Patient (Production).

How images move safely from your laptop to the Cloud.

1. Docker Hub vs. Private Registries: Why enterprises use Harbor, ECR, or Artifactory.
2. Tagging Strategy: Never rely on :latest. Use Semantic Versioning (myapp:1.2.3) or Git Commit SHAs.
3. Immutability: Once a version is pushed, it should never be changed.

—

4.1. Registry Architecture: Public vs. Private

Think of a Docker Registry like a Warehouse for your code.

• Public Registry (Public Library): Anyone can walk in and borrow a book. It’s free and has millions of books, but if you leave your personal diary (secret code) there, anyone can read it.
• Private Registry (Bank Vault): You rent a secure vault. Only people with a key (password/token) can enter. You pay for the security and maintenance.
• Self-Hosted Registry (Bunker): You build the vault yourself inside your own basement. You have total control, but if the roof leaks (server crashes), you have to fix it yourself.

A Docker Registry is the server application that stores and distributes Docker images. Choosing the right one is the foundation of your supply chain security.

—

4.1.1. Public Registries (The “Open Market”)

Think of a Public Registry (like Docker Hub) as a massive Public Library.

1. Access: It is open 24/7, free to enter, and anyone can walk in and borrow a book (Image).
2. Content: It houses millions of standard books like “How to cook” (Nginx), “Learn Math” (Python), or “Basic History” (Ubuntu).
3. The Rule: You borrow books from there, but you never leave your personal diary (Source Code) on the table. If you do, anyone can read your secrets.

—

These are massive, shared libraries hosted on the public internet. They are the default place where open-source software lives.

• Examples: Docker Hub (docker.io), http://Quay.io (quay.io), Google Container Registry (gcr.io – public images).

1. Pros:
   1. Instant Access: You don’t need to install an OS from a CD/ISO anymore. You just type FROM ubuntu:22.04, and you have a working server in 5 seconds.
   2. Community Support: Every major software (Node.js, Postgres, Redis) publishes their “Official Image” here first.
2. Cons (The Architect’s Headache):
   1. Rate Limiting (The “429” Error): Docker Hub enforces a strict “Fair Use” policy.
   2. Anonymous Users: Limited to 100 pulls per 6 hours.
   3. Authenticated Free Users: Limited to 200 pulls per 6 hours.
   4. Impact: If your production cluster tries to auto-scale and pull 500 images at once, Docker Hub blocks you. Your scaling fails, and your website goes down.
3. Typosquatting (The Security Trap): Hackers upload malicious images with names that look almost like the real ones.
   1. Real: official/nginx
   2. Fake: oficial/nginx (Note the missing ‘f’).
   3. Result: If a developer makes a typo, they download malware that mines cryptocurrency on your server.
4. Accidental Exposure: If you run docker push my-app while logged into Docker Hub, your proprietary code becomes public instantly.

—

DevSecOps Architect Perspective

The “Pull-Through Cache” Pattern (Mandatory for Enterprise)

Never let your production servers pull directly from Docker Hub. It is unreliable and insecure for enterprise scale.

The Solution: Set up a Proxy (Pull-Through Cache) using a private registry tool (like Harbor, AWS ECR, or Nexus).

The Workflow:

1. Request: Your Production Server asks your internal Private Registry for ubuntu.
2. Check: The Private Registry looks in its local storage.
   • Scenario A (Found): It delivers the image locally at Gigabit LAN speeds. (Fast & Free).
   • Scenario B (Missing): It goes to Docker Hub once, downloads the image, scans it, saves it, and then delivers it to your server.
3. Benefit:
   • You only hit Docker Hub 1 time (to fill the cache).
   • You never hit Rate Limits.
   • If the internet goes down, you can still deploy because you have a local copy.

—

Use Case

—

—

Technical Challenges

• The “Tag Mutability” Risk:
  • In a Public Registry, the author of library/postgres:14 can delete the image and upload a new, broken version with the same tag 10 minutes later.
  • Architect’s Fix: Always pin your base images to a specific SHA Digest (e.g., FROM postgres@sha256:abc123…) or mirror them to your private registry where you control the changes.

—

Cheat Sheet

—

4.1.2. Managed Cloud Registries (The “SaaS Vault”)

The “Rented Bank Vault” Think of this as “Registry as a Service.”

Instead of building a safe in your own house (Self-Hosted), you rent a high-security box inside a massive bank (Cloud Provider like AWS or Google).

You don’t worry about the locks, the guards, or the building collapsing. The bank handles all the maintenance and security. You just show your ID card (IAM Role) to get in.

These are fully managed services provided by cloud giants. You pay them to store and secure your Docker images. They take care of the underlying storage, backups, and patching.

Tools:

• AWS: Elastic Container Registry (ECR)
• Azure: Azure Container Registry (ACR)
• Google: Google Artifact Registry (GAR)

1. Pros:
   1. Zero Maintenance (SLA): No OS patching, no hard drive monitoring. The provider guarantees 99.9% uptime.
   2. Native Security:
      1. Encryption: Images are encrypted at rest (e.g., using AWS KMS keys).
      2. Scanning: Most trigger an automatic vulnerability scan (using engines like Trivy or Clair) the moment an image is uploaded.
   3. Integration (The “Killer Feature”): It connects natively to the cloud ecosystem using IAM (Identity & Access Management). You don’t need “usernames and passwords.”
2. Cons:
   1. Cost: You pay for storage (per GB) and data transfer (Egress). If you pull a huge image 1000 times to a server outside the cloud, the bill will shock you.
   2. Vendor Lock-in: Moving 50TB of images from AWS to Azure is slow, expensive, and technically painful.

—

DevSecOps Architect Perspective

As an Architect, you choose Cloud Registries for Identity Security and Resilience.

1. The “Zero Credentials” Strategy (OIDC Federation)
   1. You create a “Service Account,” generate a long-lived password, and save it in Jenkins secrets.
      1. Risk: If that secret leaks, hackers own your registry.
   2. The Architect’s Way (IAM Roles): We use OpenID Connect (OIDC).
      1. Request: Your GitHub Action (Build Job) says: “Hey AWS, I am the ‘Build-Job’ repository. Can I have a temporary key?”
      2. Verify: AWS checks the trust policy.
      3. Grant: AWS gives a temporary token valid for only 15 minutes.
      4. Result: No static passwords exist anywhere. Even if a hacker intercepts the token, it expires before they can use it.
2. Cross-Region Replication (Disaster Recovery) The Scenario: What happens if the AWS us-east-1 (Virginia) region goes offline due to a hurricane?
   1. You enable Cross-Region Replication on your registry.
   2. As soon as you push an image to us-east-1, AWS automatically copies it to eu-west-1 (London).
   3. Your Kubernetes cluster in London keeps pulling images locally without interruption. Zero downtime.

—

Use Case: The “Private Link” Architecture

Scenario: A Banking Application running on AWS.

• Requirement: The bank’s servers (EC2/EKS) must never access the public internet, not even to pull images.
• Solution: Use VPC Endpoints (PrivateLink).
• How it works: The connection between your server and the ECR Registry travels through the AWS internal backbone network, not the public internet. It is faster, more secure, and compliant with banking regulations.

—

—

Technical Challenges

—

• AWS ECR: docs.aws.amazon.com/AmazonECR/latest/userguide/
• Azure ACR: learn.microsoft.com/en-us/azure/container-registry/
• Google Artifact Registry: cloud.google.com/artifact-registry/docs

—

Cheat Sheet (Managed Registries)

—

4.1.3. Self-Hosted Private Registries (The “On-Premises Fort”)

The “Underground Bunker” Think of this as building your own high-security bunker.

You don’t trust the public library (Docker Hub) or the rented bank vault (Cloud). You buy your own steel, concrete, and servers. You build the vault inside your own basement.

The Trade-off: You have total control and total privacy. But if the light bulb breaks or the roof leaks, you have to fix it. There is no landlord to call.

You download registry software (like Harbor) and install it on your own servers (Linux or Kubernetes) inside your private data center. You own the hardware, the network cables, and the hard drives.

1. Tools:
   1. Harbor: (The CNCF Standard – Highly Recommended). Open-source, widely used, and feature-rich.
   2. JFrog Artifactory: The enterprise giant, supports generic binaries too (Maven, NPM).
   3. Sonatype Nexus: Another popular choice for mixed artifact storage.

1. Pros:
   1. Data Sovereignty (Total Privacy): Your code never touches the public internet. This is mandatory for Banks, Military, and Hospitals (GDPR/HIPAA compliance).
   2. Speed (LAN vs WAN): Since the server is next to your build agents on the Local Area Network (LAN), pulling a 2GB image takes seconds (Gigabit speeds), unlike pulling from the Cloud which depends on internet bandwidth.
   3. Cost Control: No “per GB” monthly bill from Amazon. You pay for the hardware once.
2. Cons (The “Hidden” Costs):
   1. High Operational Overhead: “With great power comes great responsibility.” You are responsible for:
      • Backups: If the disk fails and you have no backup, the code is gone forever.
      • Patching: You must update the OS and Registry software to fix security holes.
      • High Availability (HA): If your single server crashes, no one can deploy code. You need to architect complex clusters with load balancers.

—

—

Technical Challenges

—

Cheat Sheet: Registry Selection

4.2. The Tagging Strategy: The End of :latest

Tagging is the process of giving your Docker image a specific, unchangeable version name so you always know which “batch” of code you are running.

Think of Docker Tags like Batch Numbers on Medicines.

1. The Bad Way (:latest): You buy a bottle just labeled “Medicine”. You don’t know if it was made yesterday or 3 years ago. If you buy a second bottle tomorrow, it might have completely different pills inside. This is dangerous.
2. The Good Way (SemVer): You buy a bottle labeled “Paracetamol-Batch-2025-V1” (myapp:1.0.1). You know exactly what is inside.
3. The Seal (Immutability): If someone tries to open the bottle and swap the pills (overwrite the tag), the seal breaks. The Pharmacist (Kubernetes) checks the seal and refuses to sell it.

Tagging is the process of applying a label to a specific version of your image. It is the only way to distinguish “Code V1” from “Code V2”.

The :latest Trap

Using :latest is the 1^st cause of “it works on my machine but breaks in production.”

1. It is a Pointer, not a Version. It simply points to the last image pushed.
2. The Scenario: You deploy myapp:latest on Friday. It works. On Saturday, a junior dev pushes broken code to myapp:latest. On Sunday, your server restarts, pulls the “new” :latest, and crashes. You have no way to rollback because you don’t know what the previous version was.

The Gold Standards

1. Semantic Versioning (SemVer): The Human Standard.
   1. Format: vMajor.Minor.Patch (e.g., v2.1.0).
   2. Major: Breaking changes (API changes).
   3. Minor: New features (Backward compatible).
   4. Patch: Bug fixes.
2. Git Commit SHA: The Machine Standard.
   1. Format: First 7 chars of Git Hash (e.g., sha-8db2a1).
   2. Why: It creates a perfect “Audit Trail.” If sha-8db2a1 crashes, you go to GitHub, look up commit 8db2a1, and you see exactly which line of code caused it.
3. Multi-Tagging: The Professional Way.
   1. You tag the same image with both. One for humans to read, one for the machine to audit.

# Tag for Humans (Release Version)
docker tag myapp:build myrepo/myapp:v1.2.3

# Tag for Audit (Code Traceability)
docker tag myapp:build myrepo/myapp:sha-8db2a1

# Push both
docker push myrepo/myapp:v1.2.3
docker push myrepo/myapp:sha-8db2a1

—

DevSecOps Architect Perspective

As an Architect, you don’t trust developers to “remember” not to overwrite tags. You enforce it.

1. Tag Immutability (The Registry Lock)
   1. The Risk: A developer fixes a bug and pushes v1.0.0 again. Now v1.0.0 means two different things. This destroys consistency.
   2. The Fix: Enable “Tag Immutability” in AWS ECR or Harbor.
   3. The Effect: If v1.0.0 exists, the registry rejects any push trying to overwrite it. The developer is forced to increment the version to v1.0.1.
2. Digest Pinning (Zero Trust / The “Fingerprint”)
   1. The Concept: A Tag (v1.0) can theoretically be moved. A Digest (sha256:a8f9…) is a mathematical calculation of the image content. It cannot be faked or moved.
   2. The Strategy: In high-security Production Kubernetes manifests, we do not use tags. We use Digests.
      • Risky: image: myapp:v1.2.3
      • Secure: image: myapp@sha256:5b8f…
   3. Result: It ensures that the code running is bit-for-bit identical to the code you tested.
3. Binary Authorization (Trusting the Signature)
   1. We move from “Trusting the Tag” to “Trusting the Signature”.
      1. Tools: Cosign (Sigstore).
      2. Workflow:
         1. Sign: CI Pipeline signs the image: cosign sign –key private.key myapp:v1.0.
         2. Verify: Kubernetes Admission Controller (Kyverno) checks: “Does this image have a valid signature?”
         3. Block: If the image was created by a hacker (unsigned), Kubernetes deletes the Pod immediately.

—

—

Technical Challenges

—

• Docker Tagging: docs.docker.com/engine/reference/commandline/tag/
• AWS ECR Immutability: docs.aws.amazon.com/AmazonECR/latest/userguide/image-tag-mutability.html
• Cosign (Sigstore): docs.sigstore.dev/cosign/overview/

—

Cheat Sheet: Tagging Strategy

4.3. Immutability and Content Trust

Immutability ensures that once a version of your app is saved, it can never be changed; Content Trust ensures that the app you are downloading is actually the one your team created and signed.

Imagine you are sending a secret letter (Code) to a friend (Server).

• Immutability (Permanent Ink): You write the letter in permanent ink on numbered pages. If you make a mistake on Page 5, you cannot simply erase it and rewrite it. You must create a completely new Page 6. This guarantees that “Page 5” always reads the same way, forever.
• Content Trust (Wax Seal): You place a wax seal with your unique ring on the envelope.
  • If the friend receives the letter and the seal is broken (tampered with) or looks fake (wrong signature), they refuse to open it.
  • This guarantees the letter actually came from you, not an imposter.

—

Securing the Image: The Risk: Tag Overwriting

• The Problem: By default, Docker tags are mutable (changeable).
  • Scenario: You push myapp:v1.0 (Good Code). Later, a hacker (or a buggy pipeline) pushes a malicious image and also tags it myapp:v1.0.
  • Result: Docker simply moves the label v1.0 to the bad image. Your production server pulls v1.0 thinking it is the safe version, but runs the malware instead.
• The Fix: We need Immutability.

Mechanism 1: Tag Immutability (The Registry Lock)

• A configuration setting in enterprise registries (AWS ECR, Harbor, Azure ACR).
• You tell the registry: “If a tag named v1.0 already exists, REJECT any attempt to overwrite it.”

Mechanism 2: Content Trust (Digital Signatures)

• Cryptographically signing the container image.
  1. Sign: When the developer (or CI/CD) pushes code, they sign it with a Private Key.
  2. Verify: When the server (Kubernetes) tries to pull/run it, it checks the signature against the Public Key.
  3. Block: No signature? No run.

—

DevSecOps Architect Perspective

4.3.1. Tag Immutability vs. Digest Pinning

Tag Immutability is a good “Governance” policy, but Digest Pinning is the “Zero Trust” technical standard.

• Tag Immutability (Registry Level): Good. Prevents humans from making mistakes.
• Digest Pinning (Manifest Level): Best.
  • Concept: Every image has a unique SHA256 Digest (e.g., sha256:8db2…). This is a mathematical hash of the binary content.
  • Strategy: In your Kubernetes YAML, do not write image: myapp:v1.2.3. Instead, write image: myapp@sha256:8db2….
  • Why: Even if the registry is hacked and the tag is moved, the Digest cannot change. It is physically impossible to swap the image content without changing the hash.

4.3.2. Signing with Cosign (The Modern Standard)

The industry has moved from Docker Content Trust (Notary v1) to Sigstore Cosign.

• Why Architects Prefer Cosign:
  • No Heavy Server: Unlike Notary, it doesn’t require managing complex infrastructure.
  • OIDC Integration: It supports “Keyless Signing” using your existing identity (GitHub Actions, Google, OIDC).
  • Simplicity: It stores the signature in the registry itself (as a separate artifact alongside the image), making it easy to manage.

The Architect’s Workflow:

1. CI Build: Pipeline builds the image.
2. Sign: Pipeline signs the image: cosign sign –key cosign.key myrepo/app:v1.
3. Admission Control: Install Kyverno or OPA Gatekeeper in the Kubernetes cluster.
   • Policy: “Block any Pod start request unless the image has a valid signature from ‘MyCompany’.”

—

—

Technical Challenges

—

• AWS ECR Immutability: docs.aws.amazon.com/AmazonECR/latest/userguide/image-tag-mutability.html
• Cosign (Sigstore): docs.sigstore.dev/cosign/overview/
• Harbor Tag Immutability: goharbor.io/docs/2.0.0/administration/configure-tag-immutability/
• Docker Content Trust: docs.docker.com/engine/security/trust/

—

Cheat Sheet: Trust Strategy

4.4. Lab: The Secure Push Workflow

The “Sterile Lab” Protocol Think of this workflow like a Pharmaceutical Lab.

• The Amateur Way: You mix chemicals in a beaker and immediately sell it to patients. If it’s poisonous, people die.
• The Architect’s Way:
  1. Synthesize (Build): Create the medicine in a sterile environment.
  2. Lab Test (Scan): Run it through a spectrometer to check for toxins (Vulnerabilities). If toxins are found, destroy the batch.
  3. Label (Tag): Give it a precise batch number (Version).
  4. Seal (Sign): Put a tamper-proof seal on the bottle.
  5. Distribute (Push): Only then ship it to the pharmacy (Registry).

—

Practical Lab: Manual Walkthrough

Prerequisite: You need Docker installed and a Docker Hub account.

Step 1: Authenticate Safely (The Key Card)

Never type your password in the terminal. It gets saved in your shell history (~/.bash_history) where hackers can read it.

• Architect’s Fix: Use a Personal Access Token (PAT).# Create a file named 'token.txt' with your PAT inside. # The '--password-stdin' flag tells Docker to read from the pipe, not the keyboard. cat token.txt | docker login --username <your_hub_user> --password-stdin

Step 2: Build with Precision (The Batch Number)

Never build without a version. We simulate a release here.

export VERSION=v1.0.5
export IMAGE=myrepo/secure-app

# Build and tag immediately
docker build -t $IMAGE:$VERSION .

Step 3: The Security Gate (The Lab Test)

This is the most critical step. We scan the image locally before it ever leaves our laptop.

• Tool: Trivy (running as a container to scan another container).
• The Logic: We use –exit-code 1. This tells Trivy: “If you find a CRITICAL bug, crash the script (return Error 1).”docker run --rm \ -v /var/run/docker.sock:/var/run/docker.sock \ aquasec/trivy image \ --severity CRITICAL \ --exit-code 1 \ $IMAGE:$VERSION # Check the result manually if [ $? -eq 0 ]; then echo "✅ Image is Clean. Proceeding..." else echo "❌ CRITICAL Vulnerabilities found! Push Aborted." exit 1 fi

Step 4: The Push (Shipping)

Only if Step 3 passed do we upload the image.

docker push $IMAGE:$VERSION

Step 5: The Signature (The Wax Seal) – Advanced

Now that the image is in the registry, we sign it to prove ownership.

• Tool: Cosign.# Sign the uploaded image with your private key cosign sign --key cosign.key $IMAGE:$VERSION

—

DevSecOps Architect Perspective

Why this specific order matters:

• Scan BEFORE Push: Many juniors push first, then scan in the registry.
  • The Risk: Once you push, the image is in the cloud. Even if you delete it later, a hacker might have already pulled it.
  • The Rule: Shift Left. Scan locally. If it’s dirty, it never leaves your machine.
• Exit Codes: Automation depends on “Exit Codes.”
  • Exit Code 0 = Success (Green).
  • Exit Code 1 = Failure (Red).
  • By using –exit-code 1 in Trivy, we allow Jenkins/GitLab CI to automatically stop the pipeline without human intervention.

—

Technical Challenges

Cheat Sheet (The Workflow Script)

Copy this script to automate your push.

#!/bin/bash
# Architect's Secure Push Script

IMAGE_NAME="myrepo/webapp"
TAG="v1.2.0"

# 1. Build
echo "🔨 Building..."
docker build -t $IMAGE_NAME:$TAG .

# 2. Scan (The Gate)
echo "🔍 Scanning..."
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
    aquasec/trivy image --exit-code 1 --severity CRITICAL $IMAGE_NAME:$TAG

# 3. Check Exit Code
if [ $? -ne 0 ]; then
    echo "❌ Security Check Failed. Fix vulnerabilities before pushing."
    exit 1
fi

# 4. Push
echo "🚀 Security Check Passed. Pushing..."
docker push $IMAGE_NAME:$TAG
echo "✅ Done."