Automating EBS Snapshots with Python and Boto3

Introduction:
As an AWS user, creating regular snapshots of your EBS volumes is crucial for data protection and disaster recovery. However, manually creating snapshots can be time-consuming and prone to errors. In this blog post, we’ll explore how to automate the process of creating EBS snapshots using Python and the Boto3 library.

Why Use a Custom Python Script?
While AWS offers backup services like AWS Backup, there are several reasons why you might choose to use a custom Python script for creating EBS snapshots:

- Flexibility and Customisation:
  Using a Python script allows you to have full control over the snapshot creation process. You can customise the script to fit your specific requirements, such as filtering volumes based on tags, defining custom scheduling frequencies, and integrating with other AWS services or tools.
- Cost Optimisation:
  With a custom script, you have fine-grained control over which volumes are included in the snapshot process. By selectively choosing volumes based on tags or other criteria, you can optimise costs by creating snapshots only for the volumes that truly need to be backed up.
- Integration with Existing Workflows:
  A Python script can be easily integrated into your existing automation workflows or CI/CD pipelines. You can trigger the script programmatically, pass dynamic parameters, and incorporate it into your overall infrastructure management processes.
- Learning and Skill Development:
  Writing your own snapshot automation script provides an opportunity to learn and deepen your understanding of AWS services, Python programming, and infrastructure automation. It allows you to develop valuable skills that can be applied to other areas of your work.

Prerequisites:
Before we dive into the code, make sure you have the following prerequisites:

An AWS account with the necessary permissions to create snapshots.
Python is installed on your machine.
The Boto3 library installed (pip install boto3).

Step 1: Set Up the Script
Let’s start by creating a new Python file and importing the required libraries:

import boto3
import schedule
import time

We’ll use the boto3 library to interact with AWS services, schedule to define the snapshot schedule, and time for adding delays.

Step 2: Define the Snapshot Creation Function
Next, we’ll define a function called create_volume_snapshots that takes the AWS region and optional tags as parameters:

def create_volume_snapshots(region, tags=None):
    """
    Creates snapshots for EC2 volumes in the specified region, filtered by tags (optional).

    Args:
        region (str): The AWS region where the volumes are located.
        tags (dict, optional): A dictionary of tags to filter the volumes. Defaults to None.
    """
    ec2 = boto3.resource('ec2', region_name=region)
    filters = []
    if tags:
        for key, value in tags.items():
            filters.append({'Name': f'tag:{key}', 'Values': [value]})
    volumes = ec2.volumes.filter(Filters=filters)
    for volume in volumes:
        try:
            snapshot = volume.create_snapshot()
            print(f"Created snapshot: {snapshot.id} for volume {volume.id}")
        except Exception as e:
            print(f"Error creating snapshot for volume {volume.id}: {str(e)}")

This function does the following:

Creates an EC2 resource object for the specified region.
Builds a list of filters based on the provided tags.
Retrieves the volumes in the specified region, filtered by tags (if provided).
Iterates over each volume and creates a snapshot.
Prints the snapshot ID and volume ID upon successful creation.
Handles any exceptions and prints an error message if a snapshot creation fails.

Step 3: Configure the Script
Now, let’s configure the script with the desired AWS region and tags:

region = 'us-east-1'
tags = {'env': 'prod'}

Replace 'us-east-1' with the appropriate AWS region where your EC2 instances are located. Modify the tags dictionary to match the tags of the volumes you want to create snapshots for.

Step 4: Schedule the Snapshot Creation
To automate the snapshot creation process, we’ll use the schedule library to define the desired frequency:

schedule.every().day.at("00:00").do(create_volume_snapshots, region=region, tags=tags)

This line schedules the create_volume_snapshots function to run every day at midnight (00:00). Adjust the schedule according to your requirements.

Step 5: Run the Script
Finally, we’ll add a loop to keep the script running and execute the scheduled tasks:

while True:
    schedule.run_pending()
    time.sleep(1)

The script will continuously check for pending scheduled tasks and execute them when the scheduled time arrives. The time.sleep(1) statement adds a 1-second delay between each check to avoid excessive CPU usage.

For those interested in implementing or adapting this script, here’s the complete code:

import boto3
import schedule
import time


def create_volume_snapshots(region, tags=None):
    """
    Creates snapshots for EC2 volumes in the specified region, filtered by tags (optional).

    Args:
        region (str): The AWS region where the volumes are located.
        tags (dict, optional): A dictionary of tags to filter the volumes. Defaults to None.
    """
    # Create an EC2 resource object for the specified region
    ec2 = boto3.resource('ec2', region_name=region)

    # Create a list to store the filters
    filters = []

    # If tags are provided, add them to the filters list
    if tags:
        for key, value in tags.items():
            filters.append({'Name': f'tag:{key}', 'Values': [value]})

    # Get all volumes in the specified region, filtered by tags (if provided)
    volumes = ec2.volumes.filter(Filters=filters)

    # Iterate over each volume
    for volume in volumes:
        try:
            # Create a snapshot of the volume
            snapshot = volume.create_snapshot()
            print(f"Created snapshot: {snapshot.id} for volume {volume.id}")
        except Exception as e:
            # If an exception occurs, print an error message with the volume ID and exception details
            print(f"Error creating snapshot for volume {volume.id}: {str(e)}")


# Specify the AWS region where your EC2 instance is located
region = 'us-east-1'

# Optional: Specify tags to filter volumes (e.g., {'Environment': 'Production'})
tags = {'env': 'prod'}


# Schedule the script to run once a day at midnight
schedule.every().day.at("00:00").do(create_volume_snapshots, region=region, tags=tags)

# Run the scheduled tasks indefinitely
while True:
    # Check if any scheduled tasks are pending and run them
    schedule.run_pending()

    # Pause the script for 1 second before checking for pending tasks again
    time.sleep(1)

Conclusion:
With this Python script, you can automate the process of creating EBS snapshots based on specific tags and schedule them to run at your desired frequency. By leveraging the power of Boto3 and the schedule library, you can ensure regular and consistent snapshot creation without manual intervention.

Remember to review and adjust the script according to your specific requirements, such as the AWS region, tags, and scheduling frequency. Additionally, make sure to monitor the snapshot creation process and set up appropriate monitoring and alerting mechanisms to stay informed about the status of your snapshots.

By automating EBS snapshots, you can enhance your data protection strategy, minimize the risk of data loss, and save time and effort in managing your AWS infrastructure.

Posted in AWS, Python by Petru Porumbescu

Petru Porumbescu

Petru Porumbescu

Residence:

Age:

Licenses & Certifications:

English

Romanian

Skills:

Automating EBS Snapshots with Python and Boto3