Email notifications in MWAA (AWS Airflow)

Setting up MWAA in the AWS console is fairly straightforward. However, what is not very obvious is how to enable email notifications for task/DAG failures. While not obvious, it is fairly straightforward. This post will show you how to set your email provider in MWAA.

As a bonus, I will also show how one can use their Gmail/GSuite account securely and without exposing one's password.

Update settings

It's easy to miss the following panel when setting up your MWAA cluster initially. However, one can go back and add configuration options later. Go back and edit your cluster and find the following panel.

screenshot 1


Let's go ahead and click "Add custom configuration value"

Add the following variables and your specific values.

screenshot 2

Once you have added and saved these custom configuration values, Airflow notifications will start getting delivered as expected.

Bonus

If you do not have an email provider or a dedicated email account for notifications, you can use your own account. Since these settings can be viewed by anyone who has access to the AWS console, you don't want to use your own password.

Create & use app passwords

Important: To create an app password, you need 2-Step Verification on your Google Account.

  1. Go to your Google Account
  2. Select Security.
  3. Under "Signing in to Google," select 2-Step Verification.
  4. At the bottom of the page, select App passwords.
  5. Enter a name that helps you remember where you’ll use the app password. For example, type Airflow
  6. Select Generate.
  7. To enter the app password, follow the instructions on your screen. The app password is the 16-character code that generates on your device.
  8. Select Done.

You can now use this password in the Airflow SMTP Password setting.

As a general rule, I do not recommend using a Google app password for production.
There are a number of issues with this approach. It's best to use this for troubleshooting during setup or for your MWAA local runner instance.

  • If you send notifications to a group, chances are that you will not receive notifications since it is your account sending the emails.
  • Even though it is an application password, you are still exposing a password that can be used to access your account.
  • Depending on the volume of emails sent, these notifications may affect your account.