rtyler
/
delta-s3-loader
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
An AWS Lambda for automatically loading JSON files as they're created into Delta tables
10 Commits 1 Branch 0 Tags 92 KiB
Rust 100%
Go to file
R Tyler Croy fb0582e4ca Make lambda an optional build target 
This will help for use-cases where running AWS Lambda is cost-prohibitive
because the volume of inbound events is so high
 		2021-05-21 19:40:59 -07:00
data Add some properly structured test data and a destination audit_log delta table 2021-04-21 10:30:35 -07:00
src Make lambda an optional build target 2021-05-21 19:40:59 -07:00
.gitignore Initial commit 2021-04-14 20:00:27 -07:00
Cargo.lock Make lambda an optional build target 2021-05-21 19:40:59 -07:00
Cargo.toml Make lambda an optional build target 2021-05-21 19:40:59 -07:00
LICENSE.txt Initial commit 2021-04-14 20:00:27 -07:00
README.adoc Returning to schema inference work 2021-04-17 14:41:18 -07:00
config.yml Tinkering with wiring through the events into a processing/matching loop 2021-04-21 11:05:19 -07:00

README.adoc

<html lang="en"> <head> </head>

Delta S3 Loader

This AWS Lambda serves a singular purpose: bring JSON files from an S3 bucket into Delta Lake. This can be highly useful for legacy or external processes which rely on uploading JSON to an S3 bucket and cannot be properly updated to write directly to Delta Lake.

Related work

Last updated 2024-05-10 08:07:44 UTC
</html>