rtyler
/
delta-s3-loader
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Update the README to make it clear that this isn't going to be lambda only

This commit is contained in:
R Tyler Croy 2021-06-10 10:59:15 -07:00
parent fb0582e4ca
commit 88264d83c9
2 changed files with 9 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -1 +1,2 @@
/target
*.swp

14
README.adoc
View File

@ -1,14 +1,16 @@
= Delta S3 Loader
This link:https://aws.amazon.com/lambda/[AWS Lambda] serves a singular purpose:
bring JSON files from an S3 bucket into Delta Lake. This can be highly useful
for legacy or external processes which rely on uploading JSON to an S3 bucket
and cannot be properly updated to write directly to link:https://delta.io[Delta
Lake].
Delta S3 Loader is a project to quickly and cheaply bring JSON files added to
S3 buckets into Delta Lake. This can be highly useful for legacy or external
processes which rely on uploading JSON to an S3 bucket and cannot be properly
updated to write directly to link:https://delta.io[Delta Lake].
Delta S3 Loader can be built into a stnadalone binary or an
link:https://aws.amazon.com/lambda/[AWS Lambda].
== Related work
link:https://docs.databricks.com/spark/latest/structured-streaming/auto-loader.html[S3 Auto-loader]
link:https://docs.databricks.com/spark/latest/structured-streaming/auto-loader.html[S3 Auto-loader in Apache Spark]