AWS › AWS Glue & ETL

How does AWS Glue support job scheduling and triggering ETL workflows automatically based on events or a defined schedule?

Intermediate

Answer

Glue supports scheduling jobs to run at specific times using cron based schedules, as well as event based triggers that automatically start a job when a specific condition occurs, such as another job completing successfully, letting you build multi step ETL pipelines where jobs run in a defined sequence or in response to new data arriving, without needing a separate external orchestration tool for straightforward pipeline needs.

aws glue create-trigger --name daily-etl-trigger --type SCHEDULED --schedule 'cron(0 2 * * ? *)' --actions '[{"JobName":"my-etl-job"}]'

Real-world example A company schedules its Glue ETL job to automatically run every night at two in the morning, processing the previous day's transaction data and loading fresh results into their data warehouse before business hours begin.

Common follow-ups: How do you chain multiple Glue jobs together using triggers based on job completion?;When should you use Glue Workflows instead of simple triggers for more complex pipelines?

AWS Step Functions;AWS Batch