Skip to content
About Me
----

Auto Scaling Groups (ASG) & Launch Templates: Production Architecture

Architect Key Takeaways

Master Amazon EC2 Auto Scaling: Launch Templates versioning, dynamic scaling policies (Target Tracking, Step, Scheduled), Spot/On-Demand mixed fleets, and Lifecycle Hooks.

Auto Scaling Groups (ASG) & Launch Templates: Production Architecture

Section titled β€œAuto Scaling Groups (ASG) & Launch Templates: Production Architecture”

In legacy IT operations, engineers treated virtual machines like β€œPets”: giving them unique hostnames, logging into them manually via SSH to install patches, and panicking when a server crashed.

Modern cloud engineering treats compute nodes as β€œCattle”: identical, ephemeral, and managed entirely by software automation. If an instance experiences high memory pressure or fails a health check, you don’t troubleshoot it liveβ€”you let the system terminate it and provision an identical replacement immediately.

Amazon EC2 Auto Scaling Groups (ASG) combined with Launch Templates form the cornerstone of this resilient, immutable cloud compute architecture.


Before an Auto Scaling Group can provision an EC2 instance, it requires a specification defining the Amazon Machine Image (AMI), instance type, key pair, security groups, EBS volumes, and User Data bootstrap scripts.

LAUNCH CONFIGURATIONS (DEPRECATED LEGACY) LAUNCH TEMPLATES (MODERN STANDARD)
β€’ Immutable (Cannot be modified after creation) β€’ Fully Versioned ($Default, $Latest, v1, v2)
β€’ No Parameter Inheritance β€’ Template Inheritance & Parameter Subsets
β€’ No Spot / On-Demand Mixed Fleets β€’ Full Support for Spot & Mixed Instance Fleets
β€’ No T2/T3 Unlimited Mode β€’ Supports IMDSv2, Nitro Enclaves, ARM Graviton

Production Terraform Launch Template with IMDSv2 Hardening:

Section titled β€œProduction Terraform Launch Template with IMDSv2 Hardening:”
resource "aws_launch_template" "app_template" {
name_prefix = "corp-prod-web-template-"
image_id = "ami-0c55b159cbfafe1f0" # Golden AMI pre-scanned by CIS benchmarks
instance_type = "t3.medium"
# DevSecOps Hardening: Enforce IMDSv2 to prevent SSRF credential theft
metadata_options {
http_endpoint = "enabled"
http_tokens = "required" # Blocks legacy IMDSv1
http_put_response_hop_limit = 1
}
monitoring {
enabled = true # Detailed 1-minute CloudWatch monitoring
}
network_interfaces {
associate_public_ip_address = false
security_groups = [var.app_security_group_id]
}
user_data = base64encode(<<-EOF
#!/bin/bash
echo "Starting production service..."
systemctl start app.service
EOF
)
tag_specifications {
resource_type = "instance"
tags = {
Environment = "Production"
ManagedBy = "Terraform"
}
}
}

2. Dynamic Scaling Policies: Thermostat vs. Step Scaling

Section titled β€œ2. Dynamic Scaling Policies: Thermostat vs. Step Scaling”

Auto Scaling Groups use CloudWatch Alarms to trigger dynamic scaling adjustments:

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Amazon CloudWatch Metric / Alarmβ”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β–Ό β–Ό β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Target Tracking β”‚ β”‚ Step Scaling β”‚ β”‚Scheduled Scaling β”‚
β”‚ e.g. Maintain β”‚ β”‚ +2 instances if β”‚ β”‚ Spin up 20 nodes β”‚
β”‚ CPU at exact 60% β”‚ β”‚ CPU > 70%; +4 if β”‚ β”‚ at 9:00 AM every β”‚
β”‚ like a Thermostatβ”‚ β”‚ CPU hits > 90% β”‚ β”‚ Black Friday β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
  1. Target Tracking Scaling: Functions like a household thermostat. You define a target metric (e.g., β€œMaintain average fleet CPU utilization at exactly 60%”). The ASG automatically computes the mathematical delta and scales in or out continuously to maintain that balance.
  2. Step Scaling: Increases or decreases capacity based on graduated metric thresholds (e.g., add 1 node if CPU is between 70–80%, add 3 nodes if CPU is between 80–90%, and add 6 nodes if CPU exceeds 90%).
  3. Scheduled Scaling: Adjusts capacity based on predictable calendar events (e.g., scale up at 8:30 AM before daily business trading begins, scale down at 8:00 PM to save costs).

When an instance scales out (launches) or scales in (terminates), AWS provides an intercept window through Lifecycle Hooks:

[ Instance Launch Initiated ] ──> [ Pending:Wait ] ──> (Run config / Pull secrets) ──> [ InService ]
β”‚
[ Instance Terminate Initiated ] ──> [ Terminating:Wait ] ──> (Drain sessions / Take snapshot) ──> [ Terminated ]
  1. Graceful Connection Draining: Pauses termination for up to 300 seconds, allowing in-flight HTTP requests and database transactions to finish cleanly before terminating the host.
  2. Security Incident Forensics: When Amazon GuardDuty flags an EC2 instance as compromised, a Lambda hook automatically intercepts the termination, detaches the infected root EBS volume, takes an immutable forensic snapshot for security analysis, and then dumps memory artifacts to an air-gapped S3 bucket before destroying the host.

Incident: Auto Scaling Group Stuck in Launch Failure Loop

Section titled β€œIncident: Auto Scaling Group Stuck in Launch Failure Loop”

Symptom: CloudWatch shows instance count is 0. ASG Activity History displays Launching a new EC2 instance: status: Failed.

Triage Protocol:

  1. Inspect ASG Activity History: Navigate to EC2 Console -> Auto Scaling Groups -> Activity tab. AWS displays the exact error message:
    • Client.UnauthorizedOperation: The IAM Role or Instance Profile referenced in the Launch Template does not exist or lacks trust permissions.
    • VolumeTypeNotAvailableInZone: The requested EBS volume type or IOPS configuration is not supported in that specific AZ.
    • InsufficientInstanceCapacity: AWS temporarily lacks physical hardware capacity for that instance size in that AZ.
  2. Remediation for Capacity Exhaustion: Modify the Launch Template to support Mixed Instance Types (e.g., allow the ASG to fall back to m6i.large, m5.large, or m6a.large if m6g.large is temporarily sold out in that AZ).

1. Why is combining Spot Instances with On-Demand Instances in an Auto Scaling Group considered an enterprise best practice?

Answer: Spot Instances offer up to a 90% discount compared to On-Demand pricing. By configuring an ASG with an On-Demand base capacity (e.g., 20% On-Demand for baseline stability) and fulfilling the remaining 80% with diversified Spot Instances across multiple families (m5, m6i, m6a), organizations achieve massive cost reductions while maintaining resilience against Spot reclamation.

2. What is the difference between ASG Health Checks and ELB Health Checks?

Answer:

  • EC2 Status Checks (Default): The ASG monitors hardware hypervisor health and OS kernel responsiveness. If an instance’s web server crashes (e.g., Nginx stops running) but the underlying OS is still alive, the default EC2 status check reports Healthy, leaving users hitting broken 502 errors!
  • ELB Health Checks: The ASG monitors the Application Load Balancer’s target group health. If the web server returns 5xx errors or stops responding on port 80/443, the ELB marks the target Unhealthy, and the ASG automatically terminates the broken instance and launches a fresh one.