Commit Graph

12 Commits

Author SHA1 Message Date
R Tyler Croy 67950fcb6e Upgrade the deltalake dependency to a released crate 2023-05-23 21:54:02 -07:00
R Tyler Croy aef49a9cb4 Add the `license` key to Cargo.toml 2023-04-01 22:03:35 -07:00
R Tyler Croy 0cdb91c8b4 Update the deployment to optimize yesterday's partition 2023-03-30 23:41:07 -07:00
R Tyler Croy 45ec8d5d5d Introduce the OPTIMIZE_DS environment variable to scope optimize
The only supported value at the moment is "yesterday" which will only optimize
the `ds` partition for the prior day UTC.
2023-03-30 23:04:52 -07:00
R Tyler Croy d815329c9f Pretty up the readme on GitHub a bit 2023-03-27 12:46:52 -07:00
R Tyler Croy 79263750c7 Correct the AWS_S3_LOCKING_PROVIDER environment variable which is required by delta-rs
This ensures that Delta uses DynamoDB for its locking, thereby allowing safe
concurrent writers to the same delta table
2023-03-27 07:57:25 -07:00
R Tyler Croy d77dff4779 Add commercial licensing details 2023-03-26 22:53:10 -07:00
R Tyler Croy 939989c0a2 Update the README for some details about using the function 2023-03-26 22:45:09 -07:00
R Tyler Croy 65d34abb9c Complete the necessary terraform code for all the necessary resources to run
This terraform code doesn't actually provision a bucket, the function assumes
that a bucket is defined elsewhere
2023-03-26 22:27:43 -07:00
R Tyler Croy 3ca550827d Introduce a set of terraform to provision the function 2023-03-26 21:45:02 -07:00
R Tyler Croy c899b47377 Enable the most optimized release profile possible
This changes the bootstrap.zip from 9.8M to 4.4M which I suppose is good :)
2023-03-26 21:18:50 -07:00
R Tyler Croy d9a2c6290e Initial commit of an optimization function 2023-03-26 21:14:43 -07:00