User Data Scripts in EC2 Instances

A Complete Guide to Automating Instance Configuration at Launch

What is User Data?

User Data is a feature provided by Amazon EC2 that allows you to run scripts automatically when an instance launches. These scripts can be used to configure the instance, install software, or perform any other setup tasks.

Key Characteristics

Runs only during the first boot cycle of an instance
Limited to 16 KB of data (in raw form)
Executed with root/administrator privileges
Can be provided as plain text or base64-encoded
Output is logged to /var/log/cloud-init-output.log (Linux)

How User Data Works

Execution Flow

EC2 instance is launched with User Data
Cloud-init service detects and processes the User Data
Scripts are executed during the boot process
Results are logged for verification
Instance becomes available for use

Types of User Data Scripts

Shell Scripts

The most common type, starting with #!/bin/bash for Linux instances.

#!/bin/bash
yum update -y
yum install -y httpd
systemctl start httpd
systemctl enable httpd

Cloud-Init Directives

YAML-formatted cloud-init configuration for more complex setups.

#cloud-config
packages:
  - httpd
  - php
runcmd:
  - systemctl start httpd
  - systemctl enable httpd

Windows PowerShell

For Windows instances, using PowerShell scripts.


Install-WindowsFeature -Name Web-Server
New-Item -Path C:\inetpub\wwwroot\index.html -Value "Hello World"

User Data Execution Flow

Adding User Data to Instances

During Instance Launch

Start the EC2 instance launch wizard
Configure instance details
Expand the "Advanced Details" section
Enter your script in the "User data" text area
Complete the launch process

Using AWS CLI

# Launch with user data from file
aws ec2 run-instances \
  --image-id ami-12345678 \
  --instance-type t2.micro \
  --key-name MyKeyPair \
  --user-data file://my-script.sh

# Launch with inline user data
aws ec2 run-instances \
  --image-id ami-12345678 \
  --instance-type t2.micro \
  --key-name MyKeyPair \
  --user-data "#!/bin/bash
yum update -y
yum install -y httpd
systemctl start httpd"

Modifying User Data

For Stopped Instances

User data can be modified for stopped instances:

Stop the instance
Select the instance in the EC2 console
Choose Actions > Instance Settings > Edit user data
Modify the script and save
Start the instance

Important Limitations

Modified user data runs only on first boot after modification
Cannot modify user data for running instances
User data scripts don't run automatically when restarting an instance
For persistent configuration, consider other methods like Systems Manager

Using AWS CLI

# Modify user data for stopped instance
aws ec2 modify-instance-attribute \
  --instance-id i-1234567890abcdef0 \
  --attribute userData \
  --value file://new-script.sh

Viewing User Data & Logs

Viewing User Data

From the EC2 console:

Select the instance
Choose Actions > Instance Settings > View/Edit user data

From the instance (Linux):

# View user data
curl http://169.254.169.254/latest/user-data

# View base64-encoded user data
curl -s http://169.254.169.254/latest/user-data | base64 --decode

Viewing Execution Logs

For Linux instances:

# View cloud-init output log
cat /var/log/cloud-init-output.log

# View cloud-init log
cat /var/log/cloud-init.log

For Windows instances:

# View EC2Launch logs
Get-Content C:\ProgramData\Amazon\EC2-Windows\Launch\Log\EC2Launch.log

# View EC2Launch v2 logs
Get-Content C:\ProgramData\Amazon\EC2Launch\log\agent.log

Cloud-Init Directives

Cloud-init is a multi-distribution package that handles early initialization of a cloud instance. It provides advanced configuration options through YAML directives.

Common Directives

#cloud-config
# Update package lists
package_update: true

# Install packages
packages:
  - nginx
  - python3
  - git

# Create users
users:
  - name: devuser
    groups: sudo
    shell: /bin/bash
    sudo: ['ALL=(ALL) NOPASSWD:ALL']
    ssh-authorized-keys:
      - ssh-rsa AAAAB3NzaC1yc2E...

# Write files
write_files:
  - path: /var/www/html/index.html
    content: |
      
      
      
        Hello World!
      
      

# Run commands
runcmd:
  - systemctl start nginx
  - systemctl enable nginx

Multi-part User Data

MIME multi-part archives allow you to include multiple scripts or configuration files in a single user data payload.

Example Multi-part Script

Content-Type: multipart/mixed; boundary="//"
MIME-Version: 1.0

--//
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config.txt"

#cloud-config
package_update: true
packages:
  - httpd
  - php

--//
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="userdata.txt"

#!/bin/bash
echo "Hello World" > /var/www/html/index.html
systemctl start httpd
systemctl enable httpd

--//--

Bootstrapping with User Data

Downloading & Running Scripts

#!/bin/bash
# Download bootstrap script from S3
aws s3 cp s3://my-bucket/bootstrap.sh /tmp/
chmod +x /tmp/bootstrap.sh
/tmp/bootstrap.sh

# Or using curl/wget
curl -s https://my-website.com/bootstrap.sh | bash

Using CloudFormation Init

#!/bin/bash
# Install AWS CFN tools
yum install -y aws-cfn-bootstrap

# Signal success or failure to CloudFormation
/opt/aws/bin/cfn-signal -e $? \
  --stack ${AWS::StackName} \
  --resource WebServerInstance \
  --region ${AWS::Region}

Handling Errors & Debugging

Error Handling

#!/bin/bash
# Exit on error
set -e

# Error handling function
handle_error() {
  local exit_code=$?
  echo "Error occurred on line $1, exit code: $exit_code"
  # Send notification or log to external service
  aws sns publish \
    --topic-arn arn:aws:sns:region:account:topic \
    --message "Instance bootstrap failed"
  exit $exit_code
}

# Set trap for errors
trap 'handle_error $LINENO' ERR

# Your commands here
yum update -y
yum install -y httpd
systemctl start httpd

Debugging Tips

Add verbose logging with set -x
Write logs to a file: exec > >(tee /var/log/user-data.log)
Test scripts locally before using in User Data
Use wait commands for dependent services
Check cloud-init logs for errors

AWS Customer Examples

Netflix

Netflix uses User Data scripts to bootstrap their EC2 instances with their custom Spinnaker deployment platform. This allows them to quickly deploy and configure thousands of instances with consistent configurations.

OpenAI (ChatGPT)

OpenAI leverages User Data scripts to set up GPU-optimized instances for AI model training and inference. Their scripts configure CUDA drivers, ML frameworks, and monitoring tools automatically.

Amazon Prime Video

Prime Video uses User Data to configure streaming servers, setting up content delivery networks and caching layers automatically when scaling up during high-demand periods.

Common Use Cases

Web Server Setup

#!/bin/bash # Install and configure NGINX yum update -y yum install -y nginx systemctl start nginx systemctl enable nginx # Create custom index page cat > /usr/share/nginx/html/index.html << 'EOF' Welcome to my website

Hello from EC2!

This server was configured automatically.

EOF

Database Configuration

#!/bin/bash
# Install and configure MySQL
yum update -y
yum install -y mysql-server
systemctl start mysqld
systemctl enable mysqld

# Secure MySQL installation
mysql_secure_installation << EOF

y
MySecurePassword
MySecurePassword
y
y
y
y
EOF

# Create database and user
mysql -u root -pMySecurePassword << EOF
CREATE DATABASE myapp;
CREATE USER 'appuser'@'%' IDENTIFIED BY 'AppPassword123';
GRANT ALL PRIVILEGES ON myapp.* TO 'appuser'@'%';
FLUSH PRIVILEGES;
EOF

Auto Scaling Group Examples

Dynamic Configuration

#!/bin/bash
# Get instance metadata
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
REGION=$(curl -s http://169.254.169.254/latest/meta-data/placement/region)

# Register with load balancer
aws elbv2 register-targets \
  --target-group-arn arn:aws:elasticloadbalancing:$REGION:123456789012:targetgroup/my-targets/73e2d6bc24d8a067 \
  --targets Id=$INSTANCE_ID \
  --region $REGION

# Get configuration from Parameter Store
CONFIG=$(aws ssm get-parameter \
I'll provide the rest of the code for the user_data.html file, continuing from where it left off:

```html
  --name /myapp/config \
  --with-decryption \
  --region $REGION \
  --query Parameter.Value \
  --output text)

# Apply configuration
echo "$CONFIG" > /etc/myapp/config.json

Application Deployment

Deploying a Node.js App

#!/bin/bash
# Install Node.js
curl -sL https://rpm.nodesource.com/setup_14.x | bash -
yum install -y nodejs

# Create app directory
mkdir -p /var/www/myapp
cd /var/www/myapp

# Clone application from Git
git clone https://github.com/myuser/myapp.git .

# Install dependencies
npm install

# Setup process manager
npm install -g pm2
pm2 start app.js
pm2 startup
pm2 save

# Setup nginx as reverse proxy
cat > /etc/nginx/conf.d/myapp.conf << 'EOF'
server {
    listen 80;
    server_name _;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}
EOF

# Restart nginx
systemctl restart nginx

Advantages of User Data Scripts

Automation & Consistency

Automates instance configuration without manual intervention
Ensures consistent setup across multiple instances
Reduces human error in configuration
Enables infrastructure as code practices
Simplifies deployment processes

Flexibility & Integration

Works with both Linux and Windows instances
Supports multiple script types (shell, PowerShell, cloud-init)
Integrates with other AWS services (S3, CloudFormation, etc.)
Can be used with custom AMIs or marketplace AMIs
Enables dynamic configuration based on instance metadata

Operational Benefits

Reduces time to deploy and configure instances
Simplifies scaling operations
Enables self-healing infrastructure
Facilitates blue/green deployments
Supports immutable infrastructure patterns

Limitations of User Data Scripts

Technical Constraints

Limited to 16 KB in size (raw, uncompressed)
Runs only during the first boot cycle
Cannot be automatically triggered on instance restart
No built-in error recovery mechanisms
Limited visibility into execution status

Operational Challenges

Debugging can be difficult without proper logging
Cannot modify user data for running instances
Long-running scripts can delay instance availability
No native versioning or change tracking
Scripts may fail silently without proper error handling

Alternative Considerations

For complex configurations, consider AWS Systems Manager
For ongoing management, use AWS Config or Chef/Puppet
For larger deployments, consider custom AMIs
For sensitive data, use AWS Secrets Manager instead
For complex orchestration, use AWS CloudFormation

User Data vs. Other Configuration Methods

Feature	User Data	Custom AMIs	Systems Manager	CloudFormation
Execution Timing	First boot only	Pre-configured	Any time	Stack creation/update
Size Limitation	16 KB	None	None	51,200 bytes
Reusability	Medium	High	High	High
Maintenance	Simple	Complex	Medium	Medium
Ongoing Management	No	No	Yes	Yes

Test Your Knowledge

1. What is the maximum size limit for EC2 User Data scripts?

A) 8 KB

B) 16 KB

C) 32 KB

D) 64 KB

2. When does a User Data script execute?

A) Only during the first boot cycle of an instance

B) Every time an instance is started or restarted

C) On a schedule defined in the script

D) Whenever manually triggered through the AWS console

3. Which service processes User Data scripts on EC2 Linux instances?

A) AWS Systems Manager

B) EC2Config

C) cloud-init

D) AWS CloudFormation

4. Can you modify User Data for a running EC2 instance?

A) Yes, at any time

B) Yes, but only through the AWS CLI

C) No, the instance must be stopped first

D) No, User Data can never be modified after launch