From 379e3cb8fa55884050de4fa8ac68b7e156c1bbdb Mon Sep 17 00:00:00 2001 From: R Tyler Croy Date: Wed, 26 Jul 2023 11:41:47 -0700 Subject: [PATCH] Why re-export? --- _posts/2023-07-26-rust-re-export.md | 71 +++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 _posts/2023-07-26-rust-re-export.md diff --git a/_posts/2023-07-26-rust-re-export.md b/_posts/2023-07-26-rust-re-export.md new file mode 100644 index 0000000..62869b6 --- /dev/null +++ b/_posts/2023-07-26-rust-re-export.md @@ -0,0 +1,71 @@ +--- +layout: post +title: "Why we re-export symbols from other libraries in Rust" +tags: +- rust +- opinion +--- + +Dependency management in the Rust ecosystem is _fairly_ mature from my perspective, with [crates.io](https://crates.io), Cargo, and some cultural norms around semantic versions, I feel safer with dependencies in Rust than I have in previous toolchains. It's far from perfect however, and [this question](https://mastodon.social/@davidpdrsn/110780897434598935) helps highlight one of the quirks of how Rust dependency management does or does not work, depending on your perspective: + +> What is it that makes Rust users want libraries to re-export stuff from other +libraries? +> +> I often get requests for axum to re-export stuff from hyper, time, or other +common crates. Why? Just “cargo add hyper” and you’re good to go. Hyper is in +your crate graph regardless. +> +> I also often get feature requests for the few types axum does re-export so it +does confuse some. That’s why I’m reluctant to just re-export everything. + +I started writing up a reply in Mastodon but then I noticed that my words were +approaching the 500 character limit and perhaps this topic wasn't +microbloggable! I help maintain the [deltalake](https://crates.io/deltalake) +package for Rust and we **do** re-export a number of libraries, such as +[arrow](https://crates.io/arrow) of which I am a strong supporter. + +The biggest motivation for re-exporting is to preserve ABI compatibility in our +interfaces. For some crates your transitive dependencies may be masked entirely +from the end-user, for example if I pull in the `regex` crate I'm typically +just using it for regular expressions inside my crate and not exposing an +interface which takes a `regex::Regex`. The ABI is safe from transitive version +changes of that crate. If however my crate exposes an API which is dependent on +a transitive dependency then I can have problems with version mismatches. Such +is the case with `arrow` in [delta-rs](https://github.com/delta-io/delta-rs), +which exposes `arrow_array::RecordBatch`. There is a much larger chance of ABI +incompatibilties between a transitive version of arrow needed by the +`deltalake` crate and what the consuming project may specify. This is +exacerbated in our case because _another_ transitive dependency of `deltalake` +specifies a dependency on `arrow`: [datafusion](https://crates.io/datafusion). + +That means that the user, `deltalake`, and `datafusion` all have to agree on +the same version of `arrow` for types to properly interoperate between API +calls. + +But it gets worse! + +The Rust community generally seems to follow semantic versioning, but that +doesn't mean anything about the releases, just the version numbers used for +them. I can make major breaking API changes every one of my 0.x.x releases, or +in the case of `arrow` and `datafusion` I can just increment the major version +every release. + +By re-exporting symbols from those two crates, downstream users of the +`deltalake` packagew ill have a stable `RecordBatch` type ABI to work with for +every release, and can _largely_ ignore non-API breaking changes such as +struct layout changes, etc. + +I am still mixed on whether _all_ types from other crates exposed in my APIs +should be exported. I think there is benefit to doing so for faster moving +dependencies. In essence: + +* `arrow`, moving fast, better user experience to re-export +* `url`, moves slow, very mature, not really needed to re-export. + + +The judgement call I am typically making is whether this would make my life +easier as a downstream consumer of the crate. It's not that much effort of +maintenance burden to `pub use` something in a crate if that's convenient. + + +