buoyant-data
/
oxbow
1
0
Fork 0
Code Pull Requests Releases Activity
oxbow/lambdas/glue-create
History
R Tyler Croy bb5caee2f7 Introduce glue-create to handle initial table creation with AWS Athena 2024-04-21 23:07:14 +00:00
..
src Introduce glue-create to handle initial table creation with AWS Athena 2024-04-21 23:07:14 +00:00
.gitignore Introduce glue-create to handle initial table creation with AWS Athena 2024-04-21 23:07:14 +00:00
Cargo.toml Introduce glue-create to handle initial table creation with AWS Athena 2024-04-21 23:07:14 +00:00
README.adoc Introduce glue-create to handle initial table creation with AWS Athena 2024-04-21 23:07:14 +00:00

README.adoc

<html lang="en"> <head> </head>

Glue Create

This is a simple Lambda which receives S3 bucket notifications when a new Delta Lake table is created and creates the appropriate Glue table entry in the data catalog.

The function exists to handle some of the custom renaming logic to accommodate the two-layer hierarchy in the AWS Glue Data Catalog (database and table name).

Table of Contents

Environment Variables

Name

Default Value

Notes

RUST_LOG

error

Set the log level, e.g. info, warn, error. Can be scoped to specific modules, i.e. oxbow=debug

UNWRAP_SNS_ENVELOPE

null

Should only be used if S3 Event Notifications are first going to SNS and then routing to SQS for Oxbow
Last updated 2024-05-23 16:51:11 UTC
</html>