How to Build an Automated AWS Resource Monitor: Never Pay for Forgotten Resources Again

How to Build an Automated AWS Resource Monitor: Never Pay for Forgotten Resources Again
Have you ever received an unexpectedly high AWS bill because you forgot to shut down a test instance or database? You're not alone. According to recent studies, up to 35% of cloud spending goes to waste due to forgotten or unused resources.
In this comprehensive guide, I'll show you how to build an automated AWS resource monitoring system that sends you hourly email notifications about all running resources in your account. This simple automation has saved me hundreds of dollars and countless hours of manual monitoring.
Why You Need Automated AWS Resource Monitoring
The Problem
- Forgotten test instances running for days or weeks
- Development databases left running after hours
- Load balancers with no attached instances
- EBS volumes not attached to any instances
- Lambda functions with high invocation costs
The Solution
An automated system that:
- Monitors all AWS resources across your account
- Sends detailed email reports every hour
- Shows exact uptime and costs for each resource
- Helps you identify and eliminate waste immediately
Prerequisites
Before we start, make sure you have:
- An AWS account with appropriate permissions
- Basic knowledge of AWS services (Lambda, SES, CloudWatch)
- Python programming fundamentals
- AWS CLI configured (optional but recommended)
Step 1: Set Up Amazon SES (Simple Email Service)
Amazon SES will handle sending our notification emails.
Navigate to Amazon SES Console
- Open the AWS Management Console
- Search for "SES" and select "Simple Email Service"
- Go to "Email Addresses" in the SES console
- Click "Verify a New Email Address"
- Enter your email and click "Verify This Email Address"
- Check your email and click the verification link
Move Out of SES Sandbox (if needed)
If you're in the SES sandbox, you can only send emails to verified addresses. For production use, submit a request to move out of the sandbox.
Step 2: Create an IAM Role for Lambda
Your Lambda function needs permissions to access AWS resources and send emails.
- Go to IAM > Roles Click Create Role
Select Lambda as trusted entity
- Attach these policies:
AmazonEC2ReadOnlyAccess
AmazonRDSReadOnlyAccess
AWSLambda_ReadOnlyAccess
AmazonSESFullAccess (or restricted access)
Custom inline policy to access cloudwatch:Describe, logs:Describe (optional)**
Step 3: Write the Lambda Function
- Write a Lambda function in Python that:
- Uses boto3 to query EC2, RDS, and Lambda
- Calculates their running time
- Sends a report via SES
Sample Code (Python):
import boto3
from datetime import datetime, timezone
import os
ses = boto3.client('ses')
ec2 = boto3.client('ec2')
rds = boto3.client('rds')
lambda_client = boto3.client('lambda')
TO_EMAIL = "your-verified-email@example.com"
FROM_EMAIL = "your-verified-email@example.com"
def get_ec2_instances():
instances = []
reservations = ec2.describe_instances(Filters=[
{'Name': 'instance-state-name', 'Values': ['running']}
])['Reservations']
for res in reservations:
for inst in res['Instances']:
launch_time = inst['LaunchTime']
uptime = datetime.now(timezone.utc) - launch_time
instances.append(f"EC2 ID: {inst['InstanceId']}, Launched: {launch_time}, Uptime: {str(uptime).split('.')[0]}")
return instances
def get_rds_instances():
instances = []
dbs = rds.describe_db_instances()['DBInstances']
for db in dbs:
if db['DBInstanceStatus'] == 'available':
launch_time = db['InstanceCreateTime']
uptime = datetime.now(timezone.utc) - launch_time
instances.append(f"RDS ID: {db['DBInstanceIdentifier']}, Launched: {launch_time}, Uptime: {str(uptime).split('.')[0]}")
return instances
def get_lambda_functions():
functions = []
response = lambda_client.list_functions()
for fn in response['Functions']:
functions.append(f"Lambda: {fn['FunctionName']}, Created: {fn['LastModified']}")
return functions
def lambda_handler(event, context):
ec2s = get_ec2_instances()
rds_dbs = get_rds_instances()
lambdas = get_lambda_functions()
message = "\n\n".join([
"Running EC2 Instances:\n" + "\n".join(ec2s) if ec2s else "No running EC2 instances",
"Running RDS Instances:\n" + "\n".join(rds_dbs) if rds_dbs else "No running RDS instances",
"Lambda Functions:\n" + "\n".join(lambdas) if lambdas else "No Lambda functions"
])
subject = "AWS Resource Report - Hourly Summary"
ses.send_email(
Source=FROM_EMAIL,
Destination={'ToAddresses': [TO_EMAIL]},
Message={
'Subject': {'Data': subject},
'Body': {'Text': {'Data': message}}
}
)
return {
'statusCode': 200,
'body': 'Email sent successfully!'
}
Here's the detailed Python code if you want to use for your Lambda function:
import boto3
import json
from datetime import datetime, timezone
from botocore.exceptions import ClientError
def lambda_handler(event, context):
# Initialize AWS clients
ec2 = boto3.client('ec2')
rds = boto3.client('rds')
lambda_client = boto3.client('lambda')
s3 = boto3.client('s3')
ses = boto3.client('ses')
# Configuration
SENDER_EMAIL = "your-verified-email@example.com"
RECIPIENT_EMAIL = "your-email@example.com"
resources = []
try:
# Get EC2 instances
ec2_response = ec2.describe_instances()
for reservation in ec2_response['Reservations']:
for instance in reservation['Instances']:
if instance['State']['Name'] == 'running':
launch_time = instance['LaunchTime']
uptime = calculate_uptime(launch_time)
resources.append({
'Type': 'EC2 Instance',
'ID': instance['InstanceId'],
'Name': get_instance_name(instance),
'State': instance['State']['Name'],
'Instance Type': instance['InstanceType'],
'Launch Time': launch_time.strftime('%Y-%m-%d %H:%M:%S UTC'),
'Uptime': uptime
})
# Get RDS instances
rds_response = rds.describe_db_instances()
for db in rds_response['DBInstances']:
if db['DBInstanceStatus'] == 'available':
create_time = db['InstanceCreateTime']
uptime = calculate_uptime(create_time)
resources.append({
'Type': 'RDS Instance',
'ID': db['DBInstanceIdentifier'],
'Name': db['DBInstanceIdentifier'],
'State': db['DBInstanceStatus'],
'Instance Type': db['DBInstanceClass'],
'Launch Time': create_time.strftime('%Y-%m-%d %H:%M:%S UTC'),
'Uptime': uptime
})
# Get Lambda functions
lambda_response = lambda_client.list_functions()
for function in lambda_response['Functions']:
last_modified = datetime.fromisoformat(function['LastModified'].replace('Z', '+00:00'))
uptime = calculate_uptime(last_modified)
resources.append({
'Type': 'Lambda Function',
'ID': function['FunctionName'],
'Name': function['FunctionName'],
'State': 'Active',
'Instance Type': f"Runtime: {function['Runtime']}",
'Launch Time': last_modified.strftime('%Y-%m-%d %H:%M:%S UTC'),
'Uptime': uptime
})
# Get EBS volumes
volumes_response = ec2.describe_volumes()
for volume in volumes_response['Volumes']:
if volume['State'] == 'available': # Unattached volumes
create_time = volume['CreateTime']
uptime = calculate_uptime(create_time)
resources.append({
'Type': 'EBS Volume (Unattached)',
'ID': volume['VolumeId'],
'Name': volume['VolumeId'],
'State': volume['State'],
'Instance Type': f"{volume['Size']}GB {volume['VolumeType']}",
'Launch Time': create_time.strftime('%Y-%m-%d %H:%M:%S UTC'),
'Uptime': uptime
})
# Send email notification
send_email_notification(ses, resources, SENDER_EMAIL, RECIPIENT_EMAIL)
return {
'statusCode': 200,
'body': json.dumps(f'Successfully monitored {len(resources)} resources')
}
except Exception as e:
print(f"Error: {str(e)}")
return {
'statusCode': 500,
'body': json.dumps(f'Error: {str(e)}')
}
def get_instance_name(instance):
"""Extract instance name from tags"""
if 'Tags' in instance:
for tag in instance['Tags']:
if tag['Key'] == 'Name':
return tag['Value']
return 'No Name'
def calculate_uptime(start_time):
"""Calculate uptime in a human-readable format"""
if isinstance(start_time, str):
start_time = datetime.fromisoformat(start_time.replace('Z', '+00:00'))
now = datetime.now(timezone.utc)
delta = now - start_time
days = delta.days
hours, remainder = divmod(delta.seconds, 3600)
minutes, _ = divmod(remainder, 60)
if days > 0:
return f"{days}d {hours}h {minutes}m"
elif hours > 0:
return f"{hours}h {minutes}m"
else:
return f"{minutes}m"
def send_email_notification(ses, resources, sender, recipient):
"""Send email notification with resource details"""
if not resources:
subject = "AWS Resource Monitor - No Running Resources"
body = """
<html>
<body>
<h2>AWS Resource Monitor Report</h2>
<p>Good news! No resources are currently running in your AWS account.</p>
<p>Report generated at: {}</p>
</body>
</html>
""".format(datetime.now().strftime('%Y-%m-%d %H:%M:%S UTC'))
else:
subject = f"AWS Resource Monitor - {len(resources)} Resources Running"
# Create HTML table
table_rows = ""
for resource in resources:
table_rows += f"""
<tr>
<td>{resource['Type']}</td>
<td>{resource['ID']}</td>
<td>{resource['Name']}</td>
<td>{resource['State']}</td>
<td>{resource['Instance Type']}</td>
<td>{resource['Launch Time']}</td>
<td><strong>{resource['Uptime']}</strong></td>
</tr>
"""
body = f"""
<html>
<body>
<h2>AWS Resource Monitor Report</h2>
<p>You have <strong>{len(resources)}</strong> resources currently running in your AWS account.</p>
<table border="1" style="border-collapse: collapse; width: 100%;">
<tr style="background-color: #f2f2f2;">
<th>Type</th>
<th>ID</th>
<th>Name</th>
<th>State</th>
<th>Instance Type</th>
<th>Launch Time</th>
<th>Uptime</th>
</tr>
{table_rows}
</table>
<p><em>Report generated at: {datetime.now().strftime('%Y-%m-%d %H:%M:%S UTC')}</em></p>
<p>💡 <strong>Tip:</strong> Review resources with long uptimes to avoid unnecessary charges!</p>
</body>
</html>
"""
try:
ses.send_email(
Source=sender,
Destination={'ToAddresses': [recipient]},
Message={
'Subject': {'Data': subject},
'Body': {'Html': {'Data': body}}
}
)
print(f"Email sent successfully to {recipient}")
except ClientError as e:
print(f"Error sending email: {e}")
raise
Step 4: Create and Configure the Lambda Function
- Go to Lambda Console → Create function
- Choose Python 3.x
- Set the IAM role you created
- Copy the Python code from Step 3
- Paste the code above in the editor
Deploy the Code
- Update the email addresses in the configuration section
- Click "Deploy"
Step 5: Test the Setup
Test the Lambda Function
Check Email Delivery
- Verify you receive the email notification
- Check that all resources are listed correctly
- Confirm uptime calculations are accurate
Advanced Customizations
Adding More Resource Types
You can extend the function to monitor additional AWS services:
# Add ELB monitoring
elb = boto3.client('elbv2')
elb_response = elb.describe_load_balancers()
for lb in elb_response['LoadBalancers']:
if lb['State']['Code'] == 'active':
# Add load balancer to resources list
pass
# Add CloudWatch alarms
cloudwatch = boto3.client('cloudwatch')
alarms = cloudwatch.describe_alarms()
# Process alarms...
Cost Estimation
Add cost estimation to your reports:
def estimate_hourly_cost(resource_type, instance_type, region='us-east-1'):
# Simple cost estimation logic
pricing = {
't2.micro': 0.0116,
't2.small': 0.023,
't2.medium': 0.046,
# Add more instance types
}
return pricing.get(instance_type, 0.0)
Security Best Practices
Use Least Privilege Access
- Grant only necessary permissions
- Use resource-based policies where possible
- Regularly review and audit permissions
Enable Encryption
- Enable encryption for Lambda environment variables
- Use encrypted email transmission
- Consider encrypting sensitive data in reports
Monitor and Audit
- Set up CloudWatch alarms for Lambda errors
- Monitor unauthorized access attempts
- Track resource access patterns
Troubleshooting Common Issues
Email Not Received
- Check SES verification status
- Verify IAM permissions for SES
- Check spam folder
- Ensure you're not in SES sandbox for external emails
Lambda Function Timeout
- Increase timeout to 5 minutes
- Optimize code for better performance
- Consider using pagination for large numbers of resources
Missing Resources
- Check IAM permissions for specific services
- Verify region configuration
- Ensure resources are in the expected state
Conclusion
You now have a comprehensive AWS resource monitoring system that will help you avoid unexpected cloud charges and maintain better visibility into your AWS resources. This automation typically pays for itself within the first month by preventing just one forgotten resource from running unnecessarily.
The Lambda function costs pennies to run, while the insights it provides can save hundreds of dollars. Remember to regularly review and update your monitoring system as your AWS usage evolves.