Repolode
Submit
aws-samples

aws-samples/aws-glue-samples

Code samples, tutorials and utilities for building ETL jobs and data integration pipelines on AWS Glue.

View on GitHub
Stars
1,540
Forks
835
Open issues
48
Last commit
yesterday

What is it?

This repository collects sample code, tutorials and utilities for AWS Glue, Amazon's serverless data integration service used to discover, prepare, move and integrate data for analytics and machine learning. It covers Glue for Apache Spark ETL scripts, Glue Studio notebooks, the Glue Data Catalog, crawlers, Glue Data Quality, and Glue for Ray.

Beyond ETL examples, it includes operational utilities such as a Hive metastore migration tool, a Glue resource sync utility between accounts or regions, a Glue job version deprecation checker, and a Spark history server Dockerfile. It also provides examples for custom connectors that plug third-party data stores into the Glue Spark runtime, along with local validation tests.

The repository also ships Markdown-based agent skills for AI coding assistants that help upgrade older Glue jobs and migrate dev endpoints or Glue for Ray jobs to Amazon EKS. The code is licensed under MIT-0.

Key features

  • 01Sample Glue for Spark ETL scripts for joining, cleaning and converting data.
  • 02Examples for open table formats Apache Hudi, Delta Lake and Apache Iceberg on Glue.
  • 03Tutorials covering Glue Data Catalog partition indexes, crawlers and Glue Data Quality.
  • 04Utilities for Hive metastore migration, resource sync between accounts, and deprecation checks.
  • 05Custom connector development guides and local validation tests for the Glue Spark runtime.
  • 06Agent skills that help migrate older Glue jobs to Glue 4.0 or to Amazon EKS.

Use cases

  • Learning how to write and deploy AWS Glue ETL jobs with PySpark.
  • Migrating a Hive metastore or old Glue 0.9/1.0 jobs to current Glue versions.
  • Building CDC or slowly changing dimension pipelines into Iceberg, Delta or S3 data lakes.
  • Connecting a data source that Glue does not support natively through a custom connector.

Things to know

  • The samples run on AWS Glue, so you need an AWS account and the service is billed by usage.

Topics

#aws-glue#etl#spark#data-integration#data-lake#apache-iceberg#serverless#aws

FAQ

What is aws-glue-samples?
Code samples, tutorials and utilities for building ETL jobs and data integration pipelines on AWS Glue. This repository collects sample code, tutorials and utilities for AWS Glue, Amazon's serverless data integration service used to discover, prepare, move and integrate data for analytics and machine learning. It covers Glue for Apache Spark ETL scripts, Glue Studio notebooks, the Glue Data Catalog, crawlers, Glue Data Quality, and Glue for Ray.
Is aws-glue-samples free to use? What license does it have?
aws-glue-samples is open source under the MIT-0 license. Check the license terms before using it in commercial products.
Is aws-glue-samples still maintained?
Actively maintained. The last commit to aws-glue-samples was on Nov 6, 2026.
What can I use aws-glue-samples for?
Learning how to write and deploy AWS Glue ETL jobs with PySpark. Migrating a Hive metastore or old Glue 0.9/1.0 jobs to current Glue versions. Building CDC or slowly changing dimension pipelines into Iceberg, Delta or S3 data lakes. Connecting a data source that Glue does not support natively through a custom connector.
Where can I get aws-glue-samples?
The source code and installation instructions are on GitHub: https://github.com/aws-samples/aws-glue-samples

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects