Commit Graph

17 Commits

Author SHA1 Message Date
gautham acharya 1e19cf3f20 fix: include .venv in .gitignore 2024-03-08 16:54:38 -08:00
Nikolay Ulmasov cc4bc8f54c add CI for docs
Signed-off-by: Nikolay Ulmasov <ulmasov@hotmail.com>
2023-11-04 08:39:35 -07:00
Niko a67140c6b1
docs: convert docs to use mkdocs (#1731)
# Description
Completed the outstanding tasks in #1708

Also changed theme from readthedocs to mkdocs - both are built-in but
latter looks sleeker

# Related Issue(s)
closes #1708

---------

Co-authored-by: Robert Pack <42610831+roeap@users.noreply.github.com>
Co-authored-by: R. Tyler Croy <rtyler@brokenco.de>
2023-10-23 19:02:02 +00:00
Robert Pack 9fef678bc4
feat: handle path encoding in serde and encode partition values in file names 2023-09-24 10:34:14 +02:00
Ian Alexander Joiner c4ba34e4ad
Update versions to 0.6.0 and update changelog (#1023)
# Description
The description of the main changes of your pull request

# Related Issue(s)
<!---
For example:

- closes #106
--->
Closes #971 
# Documentation

<!---
Share links to useful documentation
--->
2022-12-19 08:39:55 -08:00
Ian Alexander Joiner c1f5a7e949
Update `.gitignore` and add/remove `Cargo.lock` when appropriate (#1001)
# Description
1. Add to `.gitignore` so that all `Cargo.lock` files except for the
ones in the listed binary crates are ignored.
2. Add `Cargo.lock` to the proofs crate since it is binary.
3. Remove `Cargo.lock` from the root directory.

# Related Issue(s)
Closes https://github.com/delta-io/delta-rs/issues/1000
2022-12-07 13:26:54 -08:00
Robert Pack 17999d24a5
Adopt `ObjectStore` (#761)
* feat: update datafusion and object store

* feat: adopt ObjectStore

* chore: add integration test feature

* refactor: object store as ground truth for table uri

* refactor: make creating builder infallible

* fix: fix python build

* fix: datafusion tests

* chore: flatten directories

* fix: allow configuring non https connections

* feat: draft integration set scaffolding

* refactor: clean up storage errors

* refactor: remove StorageError

* test: port tests from object_store crate

* test: include local StorageIntegration

* ci: update build scripts

* chore: add exempts for ruby build

* chore: cleanup / consolidate tests

* fix: repair s3 clock client integration

* test: migrate s3 repair tests

* fix: s3 common importa

* cleanup

* test: datafusion integration tests

* fix: checkpoint integration tests

* fix: environemtn during integration tests

* fix: write non empty test fiules to work with azure listing

* fix: use correct storage backend in azure comit tests

* fix: py3.7 tests

* fix: parquet2 merge errors

* fix: add missing rustls feature gates

* fix: add missing dev dependency

* chore: pr comments

* Apply suggestions from code review

Co-authored-by: Will Jones <willjones127@gmail.com>

* fix: doctest

Co-authored-by: Will Jones <willjones127@gmail.com>
2022-08-30 23:09:29 +02:00
Robert Pack 422fbef8ea
fix: read vacuumed delta log without _last_checkpoint (#643)
* feat: read vacuumed delta log without _last_checkpoint

* chore: fmt

* fix: reading metadta of files thay may have been removed

* fix: handle errors in local store list

* fix: re-add size to local list

* chore: clippy

* ci: move cache behind toolstack

* fix: allow not found when searching latest checkpoint
2022-06-21 07:57:07 +02:00
Graeme Cliffe d890928d17
Python API - delta.appendOnly enforcement (#590)
* Add appendOnly check and test

* Add .coverage to .gitignore

* Readd metadata test, fix test

* Whitespace

* Remove unneeded concat

* Shorten condition - is this less clear?

* Fix configuration type annotation

* Move code to func, test existing config

* Fix formatting

* Add error string match, remove config to rely on existing config

* Fix linting for function

* Rename function

* Add version, add mode to error

* Add mode/data_store_type itertools product

* Sort imports
2022-05-03 18:35:05 -07:00
Will Jones 346f51a277
[Python] Initial PyArrow writer (#566)
* Initial writer implementation

* Add basic partitioning support

* Update docs and link to other projects

* Add Pandas support

* Test writer stats and partitioning

* Test statistics

* Enforce protocol version

* Add experimental to docstring

* Need tying extensions for checking now

* Add nipick ignore for typing_extensions
2022-03-20 15:47:06 -07:00
Will Jones 3c83a1df0e
[Python] Create PyArrow dataset fragments from delta log (#525) 2022-01-05 19:51:30 -08:00
roeap 7fb8289be5
materialize tables in pyhton via native storage backend (#463) 2021-10-21 08:05:44 -07:00
Conner Murphy 03c930e7f3
added pub get configuration method for DeltaTableMetaData and DeltaTable (#356) 2021-08-06 22:46:58 -07:00
Serge Smertin 414e971d24
FIX #249 Rename camelCased fields to snake_case (#256) 2021-05-24 21:17:03 +00:00
QP Hou 033d690807
tla+ spec for dynamodb lock (#182)
Spec is model checked without pause and crash.

Adding crash and pause to the modeling requires enabling timer in the
spec (i.e. setting TIME_TICK_UNIT > 0). This will fail the model check
because lock with expiration cannot guarantee correctness without use of
fencing token.
2021-04-13 08:55:57 -07:00
R Tyler Croy d85e465c6c Add a little README
I don't have enough details to add just yet, but this is at least a start
2020-04-28 21:38:40 -07:00
Qingping Hou 06b48c9706 initial commit 2020-04-27 00:41:48 -07:00