rtyler
/
delta-s3-loader
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
An AWS Lambda for automatically loading JSON files as they're created into Delta tables
9 Commits 1 Branch 0 Tags 92 KiB
Rust 100%
Go to file
R Tyler Croy 3b598da87f Ensure that the MatchedFile information gets carried out of match_record 
Since the function is already doing some parsing, might as well carry out the
important data.
 		2021-04-21 14:06:24 -07:00
data Add some properly structured test data and a destination audit_log delta table 2021-04-21 10:30:35 -07:00
src Ensure that the MatchedFile information gets carried out of match_record 2021-04-21 14:06:24 -07:00
.gitignore Initial commit 2021-04-14 20:00:27 -07:00
Cargo.lock Working on the means of actually configuring the desired behavior 2021-04-17 15:37:47 -07:00
Cargo.toml Working on the means of actually configuring the desired behavior 2021-04-17 15:37:47 -07:00
LICENSE.txt Initial commit 2021-04-14 20:00:27 -07:00
README.adoc Returning to schema inference work 2021-04-17 14:41:18 -07:00
config.yml Tinkering with wiring through the events into a processing/matching loop 2021-04-21 11:05:19 -07:00

README.adoc

<html lang="en"> <head> </head>

Delta S3 Loader

This AWS Lambda serves a singular purpose: bring JSON files from an S3 bucket into Delta Lake. This can be highly useful for legacy or external processes which rely on uploading JSON to an S3 bucket and cannot be properly updated to write directly to Delta Lake.

Related work

Last updated 2024-05-10 02:50:09 UTC
</html>