EmailOperator in Apache Airflow

The EmailOperator in Apache Airflow is used to send emails as part of your workflow. When using Gmail as the email service, you'll need to configure SMTP settings properly to ensure successful email delivery. Below are the steps to set up and use the EmailOperator with Gmail in Airflow.

Step 1: Install Required Libraries

Make sure you have the required libraries installed. You may need to install the apache-airflow-providers-smtp provider if it's not already installed:

bash
pip install apache-airflow-providers-smtp

Step 2: Configure Airflow to Use Gmail

You need to configure your Airflow instance to send emails using Gmail's SMTP server. This can be done in the airflow.cfg configuration file or by setting environment variables.

Option A: Using airflow.cfg

1. Locate the airflow.cfg file in your Airflow home directory (usually ~/airflow/).
2. Update the [email] section with your Gmail SMTP settings:

ini
[email]
email_backend = airflow.utils.email.send_email_smtp

[smtp]
smtp_host = smtp.gmail.com
smtp_starttls = True
smtp_ssl = False
smtp_user = your_email@gmail.com  # Your Gmail address
smtp_password = your_app_password  # Your Gmail app password
smtp_port = 587
smtp_mail_from = your_email@gmail.com  # Your Gmail address

Option B: Using Environment Variables

Alternatively, you can set these values using environment variables:

bash
export AIRFLOW__EMAIL__EMAIL_BACKEND=airflow.utils.email.send_email_smtp
export AIRFLOW__SMTP__SMTP_HOST=smtp.gmail.com
export AIRFLOW__SMTP__SMTP_STARTTLS=True
export AIRFLOW__SMTP__SMTP_SSL=False
export AIRFLOW__SMTP__SMTP_USER=your_email@gmail.com
export AIRFLOW__SMTP__SMTP_PASSWORD=your_app_password
export AIRFLOW__SMTP__SMTP_PORT=587
export AIRFLOW__SMTP__SMTP_MAIL_FROM=your_email@gmail.com

Step 3: Create an App Password (Optional)

If you have two-factor authentication (2FA) enabled on your Google account, you will need to create an App Password to use with Airflow. Follow these steps:

1. Go to your Google Account settings.
2. Navigate to Security > Signing in to Google > App passwords.
3. Create a new App Password for "Mail" and "Other" (you can name it "Airflow").
4. Use this App Password in the smtp_password configuration.

Step 4: Use EmailOperator in a DAG

Now that you've configured your SMTP settings, you can use the EmailOperator in your Airflow DAG.

Example DAG

Here's a simple example of how to use the EmailOperator:

python
from airflow import DAG
from airflow.operators.email_operator import EmailOperator
from airflow.operators.dummy_operator import DummyOperator
from datetime import datetime

default_args = {
    'owner': 'airflow',
    'start_date': datetime(2024, 10, 1),
}

with DAG('email_example_dag', default_args=default_args, schedule_interval='@daily') as dag:
    start_task = DummyOperator(task_id='start')

    send_email = EmailOperator(
        task_id='send_email',
        to='recipient_email@example.com',  # Recipient's email address
        subject='Airflow Email Test',
        html_content='<h3>This is a test email sent from Airflow using Gmail!</h3>',
    )

    end_task = DummyOperator(task_id='end')

    start_task >> send_email >> end_task

Explanation

1. DAG Definition: The DAG is named email_example_dag and is scheduled to run daily.
2. Tasks:
   • start_task: A dummy task representing the start of the DAG.
   • send_email: The EmailOperator sends an email to the specified recipient. You can customize the subject and the HTML content of the email.
   • end_task: Another dummy task representing the end of the DAG.

Notes

• HTML Content: The html_content parameter allows you to send HTML formatted emails. You can also use the text_content parameter for plain text emails.
• Multiple Recipients: You can send emails to multiple recipients by providing a comma-separated string in the to parameter.
• Testing: Make sure to test the email sending functionality by running your DAG manually and checking if the email is received.

This setup allows you to send emails from your Airflow workflows using Gmail effectively.

Post Reference: Vikram Aristocratic Elfin Share