The AWS IAM console helps to highlight which keys are old, but if you have dozens of users, or multiple AWS accounts, it is still boring doing it manually. So, I wrote some code to doing it automatically leveraging AWS Lambda - since it has a generous free-tier, this check is free (however, your mileage may vary).

Image by Randall Munroe, xkcd.com

Setting up the permissions

Of course, we want to follow the principle of least privilege: the Lambda function will have access only to the minimum data necessary to perform its task. Thus, we need to create a dedicated role over the IAM Console. AWS Guide to create roles for AWS services

Our custom role needs to have the managed policy AWSLambdaBasicExecutionRole, needed to execute a Lambda function. Other than this, we create a custom inline policy with this permissions:

• iam:ListUsers, to know which users have access to the account. If you want to check only a subset of users, like filtering by department, you can use the Resource field to limit the access.

• iam:ListAccessKeys, to read access keys of the users. Of course, you can limit here as well which users the Lambda has access to.

• ses:SendEmail, to send the notification emails. Once again, you can (and should!) restrict the ARN to which it has access to.

And that are all the permissions we need!

The generated policy should look like this, more or less:

 1{
 2    "Version": "2012-10-17",
 3    "Statement": [
 4        {
 5            "Sid": "VisualEditor0",
 6            "Effect": "Allow",
 7            "Action": [
 8                "ses:SendEmail",
 9                "iam:ListAccessKeys"
10            ],
11            "Resource": [
12                "arn:aws:iam::<ACCOUNT_ID>:user/*",
13                "arn:aws:ses:eu-central-1:<ACCOUNT_ID>:identity/*"
14            ]
15        },
16        {
17            "Sid": "VisualEditor1",
18            "Effect": "Allow",
19            "Action": "iam:ListUsers",
20            "Resource": "*"
21        }
22    ]
23}

Setting up SES

To send the notification email, we use AWS Simple Email Service.

Before using it, you need to move out of the sandbox mode, or to verify domains you want to send emails to.

After that, you don’t have to do anything else, SES will be used by the Lambda code.

Setting up Lambda

You can now create an AWS Lambda function. I’ve written the code that you find below in Python, since I find it is the fastest way to put in production such a simple script. However, you can use any of the supported languages.

You need to assign the role we created before as an execution role. As memory, 128 MB is more than enough. About the timeout, it’s up to how big your company is. More or less, it can check 5/10 users every second. You should test it and see if it goes in timeout.

Lambda Code

Following there is the code to perform the task. To read it better, you can find it also on this Gitlab’s snippet.

  1from collections import defaultdict
  2from datetime import datetime, timezone
  3import logging
  4
  5import boto3
  6from botocore.exceptions import ClientError
  7
  8
  9# How many days before sending alerts about the key age?
 10ALERT_AFTER_N_DAYS = 100
 11# How ofter we have set the cron to run the Lambda?
 12SEND_EVERY_N_DAYS = 3
 13# Who send the email?
 14SES_SENDER_EMAIL_ADDRESS = '[email protected]'
 15# Where did we setup SES?
 16SES_REGION_NAME = 'eu-west-1'
 17
 18iam_client = boto3.client('iam')
 19ses_client = boto3.client('ses', region_name=SES_REGION_NAME)
 20
 21# Helper function to choose if a key owner should be notified today
 22def is_key_interesting(key):
 23    # If the key is inactive, it is not interesting
 24    if key['Status'] != 'Active':
 25        return False
 26    
 27    elapsed_days = (datetime.now(timezone.utc) - key['CreateDate']).days
 28    
 29    # If the key is newer than ALERT_AFTER_N_DAYS, we don't need to notify the
 30    # owner
 31    if elapsed_days < ALERT_AFTER_N_DAYS:
 32        return False
 33    
 34    return True
 35    
 36# Helper to send the notification to the user. We need the receiver email, 
 37# the keys we want to notify the user about, and on which account we are
 38def send_notification(email, keys, account_id):
 39    email_text = f'''Dear {keys[0]['UserName']},
 40this is an automatic reminder to rotate your AWS Access Keys at least every {ALERT_AFTER_N_DAYS} days.
 41
 42At the moment, you have {len(keys)} key(s) on the account {account_id} that have been created more than {ALERT_AFTER_N_DAYS} days ago:
 43'''
 44    for key in keys:
 45        email_text += f"- {key['AccessKeyId']} was created on {key['CreateDate']} ({(datetime.now(timezone.utc) - key['CreateDate']).days} days ago)\n"
 46    
 47    email_text += f"""
 48To learn how to rotate your AWS Access Key, please read the official guide at https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_RotateAccessKey
 49If you have any question, please don't hesitate to contact the Support Team at [email protected].
 50
 51This automatic reminder will be sent again in {SEND_EVERY_N_DAYS} days, if the key(s) will not be rotated.
 52
 53Regards,
 54Your lovely Support Team
 55"""
 56    
 57    try:
 58        ses_response = ses_client.send_email(
 59            Destination={'ToAddresses': [email]},
 60            Message={
 61                'Body': {'Html': {'Charset': 'UTF-8', 'Data': email_text}},
 62                'Subject': {'Charset': 'UTF-8',
 63                            'Data': f'Remember to rotate your AWS Keys on account {account_id}!'}
 64            },
 65            Source=SES_SENDER_EMAIL_ADDRESS
 66        )
 67    except ClientError as e:
 68        logging.error(e.response['Error']['Message'])
 69    else:
 70        logging.info(f'Notification email sent successfully to {email}! Message ID: {ses_response["MessageId"]}')
 71
 72def lambda_handler(event, context):
 73    users = []
 74    is_truncated = True
 75    marker = None
 76    
 77    # We retrieve all users associated to the AWS Account.  
 78    # Results are paginated, so we go on until we have them all
 79    while is_truncated:
 80        # This strange syntax is here because `list_users` doesn't accept an 
 81        # invalid Marker argument, so we specify it only if it is not None
 82        response = iam_client.list_users(**{k: v for k, v in (dict(Marker=marker)).items() if v is not None})
 83        users.extend(response['Users'])
 84        is_truncated = response['IsTruncated']
 85        marker = response.get('Marker', None)
 86    
 87    # Probably in this list you have bots, or users you want to filter out
 88    # You can filter them by associated tags, or as I do here, just filter out 
 89    # all the accounts that haven't logged in the web console at least once
 90    # (probably they aren't users)
 91    filtered_users = list(filter(lambda u: u.get('PasswordLastUsed'), users))
 92    
 93    interesting_keys = []
 94    
 95    # For every user, we want to retrieve the related access keys
 96    for user in filtered_users:
 97        response = iam_client.list_access_keys(UserName=user['UserName'])
 98        access_keys = response['AccessKeyMetadata']
 99        
100        # We are interested only in Active keys, older than
101        # ALERT_AFTER_N_DAYS days
102        interesting_keys.extend(list(filter(lambda k: is_key_interesting(k), access_keys)))
103    
104    # We group the keys by owner, so we send no more than one notification for every user
105    interesting_keys_grouped_by_user = defaultdict(list)
106    for key in interesting_keys:
107        interesting_keys_grouped_by_user[key['UserName']].append(key)
108
109    for user in interesting_keys_grouped_by_user.values():
110        # In our AWS account the username is always a valid email. 
111        # However, you can recover the email from IAM tags, if you have them
112        # or from other lookups
113        # We also get the account id from the Lambda context, but you can 
114        # also specify any id you want here, it's only used in the email 
115        # sent to the users to let them know on which account they should
116        # check
117        send_notification(user[0]['UserName'], user, context.invoked_function_arn.split(":")[4])

Schedule your Lambda

You can schedule your Lambda to run thanks to CloudWatch Events. You can use a schedule expression such rate(3 days) to run the email every 3 days. Lambda will add necessary permissions to the role we created before to invoke the Lambda. If you need any help, AWS covers you with a dedicated tutorial!

Conclusions

This is just an idea on how to create a little script, leveraging AWS Lambda and AWS SES, to keep your AWS account safe. There are, of course, plenty of possible improvements! And remember to check the logs, sometimes ;-)

If you have hundreds or thousands of users, the function will go in timeout: there are different solutions you can implement, as using tags on users to know when you have lasted checked them, or checking a different group of users every hour, leveraging the PathPrefix argument of list_users.

Also, in my example it’s simple to know to whom send the notification email - but what if your users don’t have their email as username? You can use tags, and set their contact email there. Or, you maybe have to implement a lookup somewhere else.

We could also send a daily report to admins: since users usually ignore automatic emails, admins can intervene if too many reports have been ignored. Or, we can forcibly delete keys after some time - although this could break production code, so I wouldn’t really do it - or maybe yes, it’s time developers learn to have a good secrets’ hygiene.

And you? How do you check your users rotate their access keys?

For any comment, feedback, critic, suggestion on how to improve my English, leave a comment below, or drop an email at [email protected].

Ciao,
R.