DevSecOps CI CD Pipeline
This document provides a quick glance at DevSecOps CI/CD Pipeline Read more to deepen your understanding of how to automate security within your delivery pipeline.
Before reading this document, please read DevOps Essentials, DevSecOps Essentials and Git
DevSecOps Pipeline is like a high-tech assembly line for software. While a traditional CI/CD pipeline focuses on speed (getting code to production fast), DevSecOps adds “security sensors” at every stage. Instead of checking for security bugs at the very end, we check for them the moment a developer writes a line of code. This ensures that the software is not just fast and functional, but also “secure by design.”
1. DevSecOps CI/CD Foundations
Building the culture of automation and security integration.
| Topic | Short Notes | Official Doc |
| What is CI/CD | Simple: Automated way to test and ship code. Architect: CI ensures code integrity via frequent merges & automated builds. CD automates the path to production, reducing “Mean Time to Recovery” (MTTR). | Martin Fowler CI |
| CI vs CD vs CD | Simple: CI = Testing; Continuous Delivery = Ready to ship; Continuous Deployment = Auto-shipped. Architect: Delivery requires a “human trigger” for production; Deployment relies on 100% automated health checks. | Red Hat CI/CD |
| Why Manual Fails | Simple: Humans make mistakes. Architect: Manual steps lead to Configuration Drift where environments (Dev/Prod) become different over time, causing “it works on my machine” bugs. | Google SRE Book |
| DevSecOps Mindset | Simple: Security is everyone’s job. Architect: Moving from “Security as a Gate” to “Security as Code.” It involves automating compliance and security testing within the developer’s existing workflow. | DevSecOps.org |
| Immutable Infrastructure | Simple: Never patch a running server; replace it. Architect: Instead of updating software on a live server, you build a new Image (AMI/Docker) and replace the old one. This ensures consistency. | HashiCorp Intro |
2. Shift-Left Code and Dependency Security
Finding vulnerabilities before the code ever leaves the developer’s machine.
| Topic | Detailed Notes | Official Doc |
| Shift-Left Security | Simple: Find bugs early when they are cheap to fix. Architect: Integrating security tools into IDEs and PRs. Reducing the feedback loop from weeks to minutes. | OWASP Shift Left |
| SAST (Static Analysis) | Simple: Checking the recipe without cooking the food. Architect: Scans source code for patterns (SQLi, Hardcoded Keys). High false-positives require fine-tuning of rulesets. | SonarQube |
| SCA (Dependency Sec) | Simple: Checking if your “store-bought” ingredients are expired. Architect: Analyzing Third-Party libraries (NPM, Maven) against CVE databases. Essential because 80% of modern code is open-source. | Snyk |
| Secret Scanning | Simple: Don’t leave your house keys in the mailbox. Architect: Using regex and entropy checks to find AWS keys or API tokens in Git history. Tools: trufflehog, gitleaks. | GitHub Secrets |
| Pre-Commit Hooks | New (Missing): Automatically runs linting and basic security checks before a developer can even push code to the server. | Pre-Commit.com |
3. Application, Runtime and Infra Security
Protecting the environment where the code lives.
| Topic | Detailed Notes | Official Doc |
| DAST (Dynamic Analysis) | Simple: Testing the app while it’s running. Architect: “Black-box” testing that probes the API/UI for vulnerabilities like XSS or Auth bypass that SAST might miss. | OWASP ZAP |
| Security Gates | Simple: If the code is “sick,” don’t let it pass. Architect: Programmatic thresholds (e.g., “Fail build if Critical CVE > 0”). Integrated into the CI pipeline (Jenkins/GitHub Actions). | SonarQube Gates |
| IaC Security | Simple: Checking the blueprint of your data center. Architect: Scanning Terraform/CloudFormation for misconfigurations (e.g., S3 buckets open to the public). | Checkov |
| Container Security | New (Missing): Scanning Docker layers for vulnerabilities and ensuring “Distroless” or minimal base images are used to reduce attack surface. | Docker Security |
| Runtime Protection | New (Missing): Using RASP (Runtime Application Self-Protection) to detect attacks on the application while it’s actively serving users. | Trend Micro RASP |
4. Software Supply Chain Security
Ensuring the “factory” and the “parts” haven’t been tampered with.
| Topic | Detailed Notes | Official Doc |
| Supply Chain (SLSA) | Simple: Knowing exactly where your code came from. Architect: A framework for supply-chain levels (1-4) to ensure build integrity and prevent unauthorized code injection. | SLSA.dev |
| SBOM (Missing) | Simple: A detailed “List of Ingredients” for your software. Architect: Software Bill of Materials. A machine-readable inventory of all components in your software. Crucial for rapid response to new CVEs. | CISA SBOM |
| Artifact Attestation | Simple: Putting a digital “wax seal” on your build. Architect: Signing artifacts using tools like Cosign. Verifying that the image running in Prod is exactly what was built in CI. | Sigstore |
| Pinned Dependencies | Simple: Use a specific version, not “latest.” Architect: Using Commit SHAs instead of mutable tags (like :latest) to prevent “poisoned” updates from breaking the build. | GitHub Hardening |
5. CI/CD Reliability and Quality Engineering
Ensuring the pipeline itself is a stable, high-quality product.
| Topic | Detailed Notes | Official Doc |
| Pipeline SRE | Simple: The pipeline must not break. Architect: Treating the CI/CD pipeline as a Production service. Implementing monitoring, alerting, and high availability for runners. | Google SRE |
| Flaky Test Management | Simple: Fix tests that fail “sometimes” for no reason. Architect: Implementing “Quarantine” for flaky tests so they don’t block the pipeline while being investigated. | Martin Fowler Flaky |
| Chaos Engineering | New (Missing): Intentionally breaking things in the pipeline to see if the system recovers. Architect: Using tools like Litmus or Gremlin to test pipeline resilience. | Principles of Chaos |
| Observability (CI) | New (Missing): Tracking “Pipeline Traceability.” Understanding exactly which step in a 20-minute pipeline is the bottleneck. | OpenTelemetry CI |
6. Delivery Safety and Blast-Radius Control
Releasing features safely without crashing the whole system.
| Topic | Detailed Notes | Official Doc |
| Canary Deployments | Simple: Test the new version on 5% of users first. Architect: Using Service Meshes (Istio/Linkerd) to route a small slice of traffic to the new version and monitoring error rates. | Kubernetes Canary |
| Feature Toggles | Simple: A “Light Switch” to turn features on/off without redeploying. Architect: Decouples “Deployment” from “Release.” Allows dark-launching features for testing in Prod. | LaunchDarkly Guide |
| GitOps (Missing) | Simple: Use Git as the “Single Source of Truth” for Infrastructure. Architect: Using tools like ArgoCD or Flux. The cluster pulls the state from Git, ensuring no manual changes (No kubectl edit). | GitOps Guide |
| Automated Rollback | Simple: Hit the “Undo” button automatically if things go wrong. Architect: Integration of Prometheus/Datadog alerts with the deployment tool to auto-revert if 5xx errors spike. | ArgoCD Rollback |
7. Platform Engineering and Scaling
Standardizing the developer experience.
| Topic | Detailed Notes | Official Doc |
| Internal Dev Platform | Simple: A “Self-Service” portal for developers to get tools. Architect: Platform teams build the “Golden Path” a pre-approved, secure way to deploy apps, so devs don’t have to worry about YAML. | PlatformEngineering.org |
| Policy as Code (PaC) | New (Missing): Writing “Laws” for your cloud that the pipeline enforces. Architect: Using OPA (Open Policy Agent) to ensure no one creates an unencrypted database, even if they have the permissions. | Open Policy Agent |
| Reusable Workflows | Simple: Write the pipeline code once, use it for 100 apps. Architect: Centralized YAML templates in GitHub/GitLab to ensure every team follows the same security scanning steps. | GitHub Reusable |
8. Governance, Compliance and Legal
Keeping the lawyers and auditors happy.
| Topic | Detailed Notes | Official Doc |
| Compliance as Code | Simple: Automatically prove you are following the rules. Architect: Mapping pipeline checks (SAST/DAST/Audit logs) to specific compliance controls (SOC2/PCI-DSS/GDPR). | Chef InSpec |
| License Compliance | Simple: Make sure you aren’t using “illegal” code. Architect: Using SCA tools to detect GPL or other restrictive licenses that might force your company to open-source its private code. | FOSSA |
| Audit Traceability | Simple: A permanent record of “Who, What, When.” Architect: Ensuring non-repudiable logs of every production change, linked to a specific Jira ticket and Peer Review. | ISO 27001 |
9. Business and Human Context
The “People” side of DevOps.
| Topic | Detailed Notes | Official Doc |
| Break-Glass Access | Simple: Emergency key for the server. Architect: A process where developers get temporary elevated access (Privileged Access Management) that triggers intense logging and alerts. | Google SRE Incident |
| Blame-Free Postmortems | Simple: Don’t point fingers; fix the system. Architect: Focusing on why the process allowed a mistake, rather than who made it. Essential for a healthy DevSecOps culture. | Etsy Postmortems |
| Cognitive Load | New (Missing): Ensuring pipelines aren’t too complex for devs to understand. Architect: Following “Team Topologies” to ensure developers focus on business logic while Platform teams handle the “plumbing.” | Team Topologies |
10. Metrics, Cost and Furure
Measuring success and looking ahead.
| Topic | Detailed Notes | Official Doc |
| DORA Metrics | Simple: The 4 keys to high performance (Speed & Stability). Architect: Deployment Frequency, Lead Time for Changes, Change Failure Rate, and Time to Restore Service. | DORA Research |
| FinOps in CI/CD | Simple: Don’t waste money on cloud runners. Architect: Monitoring the cost of CI runs. Deleting “stale” preview environments and optimizing build cache to save money. | FinOps Foundation |
| AI-Driven Security | Simple: AI finding bugs humans miss. Architect: Using LLMs to explain vulnerabilities to developers and suggest “Auto-remediation” code fixes. | Google Cloud AI |
11. Advanced Cloud Native Security Kubernetes
| Topic | Detailed Notes (Architect Level) | Official Doc |
| Admission Controllers | Architect: Using Validating Admission Webhooks (like Kyverno or OPA Gatekeeper) to block non-compliant pods before they ever start. | Kubernetes Docs |
| Runtime Security (eBPF) | Architect: Deep kernel-level visibility. Detecting “drift” in running containers (e.g., a process starting a shell in a container). Tools: Falco, Tetragon. | Falco.org |
| Service Mesh Security | Architect: Implementing mTLS (Mutual TLS) by default for all service-to-service communication. Preventing lateral movement of attackers. | Istio Security |
| Secret Management (External) | Architect: Moving away from K8s Secrets (base64) to external providers like HashiCorp Vault or AWS Secrets Manager with External Secrets Operator. | External Secrets |
12. The Secure Build Infrastructure
Most courses teach how to use Jenkins/GitHub Actions, but not how to SECURE the CI server itself.
| Topic | Detailed Notes (Architect Level) | Official Doc |
| Ephemeral Build Runners | Architect: Never use “Persistent” runners. Spin up a fresh Docker container or VM for every job and destroy it immediately to prevent cross-job contamination. | GitHub Self-Hosted |
| Network Isolation | Architect: Placing CI runners in private VPCs with no inbound internet access. Using NAT Gateways or Private Links for controlled outbound traffic. | AWS VPC Best Practices |
| Hardened Images (CIS) | Architect: Building runners using CIS Benchmarks or STIG-hardened base images. Disabling unnecessary services and ports. | CIS Benchmarks |
13. Vulnerability Management and Remediation
| Topic | Detailed Notes (Architect Level) | Official Doc |
| Vulnerability Prioritization | Architect: Using VEX (Vulnerability Exploitability eXchange) and EPSS scores to decide what to fix. Don’t fix a “Critical” if the code isn’t reachable. | CISA VEX |
| ASPM (New Category) | Architect: Application Security Posture Management. Consolidating findings from SAST, DAST, and SCA into one single dashboard. | Gartner ASPM Intro |
| Auto-Remediation | Architect: Creating “Fixer” pipelines. If a minor CVE is found, the pipeline automatically opens a PR with the updated version for the developer. | Snyk Fix |
14. Secret Rotation and Zero Trust
Moving from “Static Secrets” to “Dynamic Identity.”
| Topic | Detailed Notes (Architect Level) | Official Doc |
| Dynamic Credentials | Architect: Instead of a long-lived AWS IAM User key, use OIDC (OpenID Connect). GitHub Actions “logs in” to AWS using a temporary 1-hour token. | GitHub OIDC AWS |
| Secrets Rotation | Architect: Automating the rotation of DB passwords every 30 days without manual intervention or downtime using Lambda/CloudWatch. | AWS Secrets Rotation |