Migrate rustfix to the cargo repo
This migrates the `rustfix` crate from https://github.com/rust-lang/rustfix/ to the cargo repo. The cargo team has been responsible for the client-side of `cargo fix`, and it can make it easier to maintain with all our tooling and tracking here. This crate is used by some external parties (like the compiler), so it will need to be maintained like an "ecosystem" package, but hopefully there shouldn't be any outside requirements (I haven't seen any in several years).
After merging, I'll follow up with some things to address in the future, such as:
- Migrating issues from the other repo.
- Opening new issues for some cleanup tasks, such as adding documentation, fixing the `#[ignore]` annotations, fixing testing on windows, maybe migrating the test code to use different dependencies, various code cleanup.
- Archiving the repo.
e58b84d changed the shape of response of cargo credential protocol trait,
so credential plugins crates effectively depend on `cargo-credential@0.4.0`.
However, `cargo@0.74.0` still depends on`cargo-credential@0.3.0`.
They must depends on the same major version of `cargo-credential`
otherwise incompatible.
This PR
* bumps the version to `cargo-credential-wincred@0.4.2`
* bumps the version to `cargo-credential-macos-keychain@0.4.2`
* bumps the version to `cargo-credential-li@0.4.2`
See rust-lang#13004 for more.
This adds a garbage collector which will remove old files from cargo's
global cache.
A general overview of the changes here:
- `cargo::core::global_cache_tracker` contains the `GlobalCacheTracker`
which handles the interface to a sqlite database which stores
timestamps of the last time a file was used.
- `DeferredGlobalLastUse` is a type that implements an optimization for
collecting last-use timestamps so that they can be flushed to disk all
at once.
- `cargo::core::gc` contains the `Gc` type which is the interface for
performing garbage collection. It coordinates with the
`GlobalCacheTracker` for determining what to delete.
- Garbage collection can either be automatic or manual. The automatic
garbage collection supports some config options for defining when
it runs and how much it deletes.
- Manual garbage collection can be performed via options to `cargo
clean`.
- `cargo clean` uses the new package cache locking system to coordinate
access to the package cache to prevent interference with other cargo
commands running concurrently.
chore(deps): update rust crate gix to 0.55.2
[](https://renovatebot.com)
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
| [gix](https://togithub.com/Byron/gitoxide) | workspace.dependencies | minor | `0.54.1` -> `0.55.2` |
---
### Release Notes
<details>
<summary>Byron/gitoxide (gix)</summary>
### [`v0.55.2`](https://togithub.com/Byron/gitoxide/releases/tag/gix-v0.55.2): gix v0.55.2
[Compare Source](https://togithub.com/Byron/gitoxide/compare/gix-v0.55.1...gix-v0.55.2)
##### Bug Fixes
- bump `gix-transport` version to prevent it from being picked up.
`gix-transport` v0.37.1 could accidentally be picked up by older, incompatible,
`gix` versions which now fail to build.
Thus v0.37.1 is now yanked and replaced with v0.38.0 along with a new
release of `gix` to go with it.
##### Commit Statistics
- 2 commits contributed to the release.
- 1 commit was understood as [conventional](https://www.conventionalcommits.org).
- 0 issues like '(#ID)' were seen in commit messages
##### Commit Details
<csr-read-only-do-not-edit/>
<details><summary>view details</summary>
- **Uncategorized**
- Prepare changelogs prior to release ([`12b5caf`](https://togithub.com/Byron/gitoxide/commit/12b5caf))
- Bump `gix-transport` version to prevent it from being picked up. ([`8011c73`](https://togithub.com/Byron/gitoxide/commit/8011c73))
</details>
### [`v0.55.1`](https://togithub.com/Byron/gitoxide/releases/tag/gix-v0.55.1): gix v0.55.1
[Compare Source](https://togithub.com/Byron/gitoxide/compare/gix-v0.55.0...gix-v0.55.1)
##### New Features
- Add `take_data()` to all primitive object types.
That is the new, most direct way to obtain its data which otherwise
is immovable.
- Add `detach()` and `detached()` too all object types.
That way, the detachment API is symmetric.
It's required to overcome the `Drop` implementation of each of these types
which prevents moving data out of the object (easily).
##### Commit Statistics
- 2 commits contributed to the release.
- 2 commits were understood as [conventional](https://www.conventionalcommits.org).
- 0 issues like '(#ID)' were seen in commit messages
##### Commit Details
<csr-read-only-do-not-edit/>
<details><summary>view details</summary>
- **Uncategorized**
- Add `take_data()` to all primitive object types. ([`5732303`](https://togithub.com/Byron/gitoxide/commit/5732303))
- Add `detach()` and `detached()` too all object types. ([`88f2e6c`](https://togithub.com/Byron/gitoxide/commit/88f2e6c))
</details>
### [`v0.55.0`](https://togithub.com/Byron/gitoxide/releases/tag/gix-v0.55.0): gix v0.55.0
[Compare Source](https://togithub.com/Byron/gitoxide/compare/gix-v0.54.1...gix-v0.55.0)
This release contains a complete rewrite of the internal url parsing logic, the public interface stays mostly the same however. Gitoxide will now be
more correct, interpreting more urls the same way Git does. Improvements include the added support for ssh aliases (`github:byron/gitoxide` has previously
been parsed as local path), adjustments around the interpretation of colons in file names (previously we disallowed colons that were not followed up
with a slash character) and some smaller changes that bring the interpretation of file urls more in line with Git's implementation. Additionally, the
error types have been adjusted to print a more comprehensive message by default, making sure they stay helpful even when bubbled up through multiple abstraction
layers.
There are still many (edge) cases in Git's url parsing implementation which are not handled correctly by Gitoxide. If you notice any such deviation please
open a new issue to help us making Gitoxide even more correct.
##### Other
- <csr-id-f478a3722f0be35c109ea60b79cd4ac6e607480b/> inform about the absence of strict hash verification and strict object creation.
Those are present in `git2` and enabled by default, and `gitoxde` definitely
wants to do the same at some point.
##### New Features
- add `Repository::head_tree()` to more easily obtain the current tree.
- Add `Repository::has_object()` as a high-level alternative.
Previously, one would have to call `repo.objects.contains()`, which
is fine, but this method is necessary for symmetry of the API
and one shouldn't have to drop down a level to do this.
This method also knows empty trees as special case.
- add `Object::try_into_blob()` and `into_blob()` and `Repository::empty_blob()`
This way it's easier to assert that an object is actually a blob.
- add `Repository::index_or_empty()`.
This is useful if a missing index should mean it's empty.
##### Commit Statistics
- 14 commits contributed to the release over the course of 16 calendar days.
- 17 days passed between releases.
- 5 commits were understood as [conventional](https://www.conventionalcommits.org).
- 0 issues like '(#ID)' were seen in commit messages
##### Thanks Clippy
[Clippy](https://togithub.com/rust-lang/rust-clippy) helped 1 time to make code idiomatic.
##### Commit Details
<csr-read-only-do-not-edit/>
<details><summary>view details</summary>
- **Uncategorized**
- Release gix-hash v0.13.1, gix-features v0.36.0, gix-actor v0.28.0, gix-object v0.38.0, gix-glob v0.14.0, gix-attributes v0.20.0, gix-command v0.2.10, gix-filter v0.6.0, gix-fs v0.8.0, gix-commitgraph v0.22.0, gix-revwalk v0.9.0, gix-traverse v0.34.0, gix-worktree-stream v0.6.0, gix-archive v0.6.0, gix-tempfile v11.0.0, gix-lock v11.0.0, gix-ref v0.38.0, gix-config v0.31.0, gix-url v0.25.0, gix-credentials v0.21.0, gix-diff v0.37.0, gix-discover v0.26.0, gix-ignore v0.9.0, gix-index v0.26.0, gix-mailmap v0.20.0, gix-negotiate v0.9.0, gix-pack v0.44.0, gix-odb v0.54.0, gix-pathspec v0.4.0, gix-packetline v0.16.7, gix-transport v0.37.0, gix-protocol v0.41.0, gix-revision v0.23.0, gix-refspec v0.19.0, gix-worktree v0.27.0, gix-status v0.2.0, gix-submodule v0.5.0, gix-worktree-state v0.4.0, gix v0.55.0, safety bump 37 crates ([`68e5432`](https://togithub.com/Byron/gitoxide/commit/68e5432))
- Prepare changelogs prior to release ([`1347a54`](https://togithub.com/Byron/gitoxide/commit/1347a54))
- Merge branch 'improvements' ([`429e7b2`](https://togithub.com/Byron/gitoxide/commit/429e7b2))
- Inform about the absence of strict hash verification and strict object creation. ([`f478a37`](https://togithub.com/Byron/gitoxide/commit/f478a37))
- Add `Repository::head_tree()` to more easily obtain the current tree. ([`c79a7da`](https://togithub.com/Byron/gitoxide/commit/c79a7da))
- Add `Repository::has_object()` as a high-level alternative. ([`787a9aa`](https://togithub.com/Byron/gitoxide/commit/787a9aa))
- Add `Object::try_into_blob()` and `into_blob()` and `Repository::empty_blob()` ([`3cec935`](https://togithub.com/Byron/gitoxide/commit/3cec935))
- Thanks clippy ([`345712d`](https://togithub.com/Byron/gitoxide/commit/345712d))
- Merge branch 'reset' ([`b842691`](https://togithub.com/Byron/gitoxide/commit/b842691))
- Trust Ctime again ([`f929d42`](https://togithub.com/Byron/gitoxide/commit/f929d42))
- Add `Repository::index_or_empty()`. ([`7d9ecdd`](https://togithub.com/Byron/gitoxide/commit/7d9ecdd))
- Adapt to changes in `gix-status` ([`54fb7c2`](https://togithub.com/Byron/gitoxide/commit/54fb7c2))
- Merge branch 'gix-url-parse-rewrite' ([`a12e4a8`](https://togithub.com/Byron/gitoxide/commit/a12e4a8))
- Update changelogs ([`4349353`](https://togithub.com/Byron/gitoxide/commit/4349353))
</details>
</details>
---
### Configuration
📅 **Schedule**: Branch creation - "before 5am on the first day of the month" (UTC), Automerge - At any time (no schedule defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/rust-lang/cargo).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zMS41IiwidXBkYXRlZEluVmVyIjoiMzcuMzEuNSIsInRhcmdldEJyYW5jaCI6Im1hc3RlciJ9-->
Do not call it "Downgrading" when difference is only build metadata
### What does this PR try to resolve?
When a `cargo update --precise` changes a dependency between 2 versions which differ only in build metadata, Cargo prints a log referring to it as "Updating" or "Downgrading" the dependency, depending on a comparison between the build metadatas.
This is usually not meaningful, given that build metadata is often stuff like git commit hashes, which are not meaningfully ordered.
```console
Updating crates.io index
Downgrading foo v0.0.1+43ef4fe -> v0.0.1+2c65d16
Updating bar v0.0.2+bc17664 -> v0.0.2+c144a98
```
~~This PR changes to the word "Switching" when the version major, minor, patch, and pre-release value are not being changed.~~
This PR uses the word "Updating" when the version major, minor, patch, and pre-release value are unchanged, regardless of whether the build metadata is going up or down.
### How should we test and review this PR?
- `cargo test`
- `cargo build --release`
- `/path/to/cargo/target/release/cargo add tonic_datastore_v1`
- `/path/to/cargo/target/release/cargo update -p tonic_datastore_v1 --precise 0.1.0+3562b6cb3`
- `/path/to/cargo/target/release/cargo update -p tonic_datastore_v1 --precise 0.1.0+ee9e8e4e6`
Before:
<img src="https://github.com/rust-lang/cargo/assets/1940490/93e377e7-928e-4cec-aff6-451166ef7c81" width="500">
~~After:~~
<img src="https://github.com/rust-lang/cargo/assets/1940490/bb71459e-469a-4e09-bb8a-4083f34bce79" width="500">
After:
<img src="https://github.com/rust-lang/cargo/assets/1940490/8804e2fe-d0de-4c9e-b463-a5742daf9446" width="500">
chore: Specify all of toml_edit's features
This is preventing us from being able to update toml/toml_edit independent of each other since the new versions are both breaing changes, so by updating one, we are getting the features enabled for us by the wrong version.
We are already getting `anstream` through `clap`, so this is no extra
cost and let's us drop some dependencies.
The `anstream` implementation is also orders of magnitude faster (last I
benchmarked)
Traditionally, cargo has disabled clap's styled help output. My assumed
reason is that cargo mixes custom help output with auto-generated and
you couldn't previously make it all styled to match. Clap 4.2 allowed
users to pass in strings styled using ANSI escape codes, allowing us to
pass in styled text that matches clap, unblocking this. In clap
4.4.1, clap gained the ability for the user to override the style,
allowing us to choose the styling as we wish.
In this PR, I decided to use the new 4.4.1 feature to style clap's
output to match the rest of cargo's output. Alternatively, we could use
a more subdue style that clap uses by default. That subdued style was
mostly chosen to be app theme neurtral (since we didn't have theming
support yet) and there were problems with our style and no one stepped
up to fix them (cargo has a style we can match to instead).
I decided to *not* style `Arg::help` messages because
- It might be distracting to have the descriptions lit up like a
christmas tree
- It is a lot more work
The one exception I made was for `--list` since it is for a
psuedo-command (`...`) and I wanted to intentionally draw attention to
it.
As of serde 1.0.172, `serde_derive` ships a binary blog for Linux x64
for faster build times. This blob is not yet reproducible to ensure
that the safety of it. See serde-rs/serde#2538
This is not a judgement on serde or on dtolnay but just a precaution to
buy us more time as the community works through this since the beta cut
is coming up. rust-1.72 branch is unaffected.
cargo-credential-gnome-secret: dynamically load libsecret
Building `cargo-credential-gnome-secret` currently requires the `libsecret` development libraries to be installed and findable via `pkg-config`. This is often an extra step for users and complicates CI builds.
This loads the required functions from `libsecret` dynamically using `libloading` which uses `dlopen` internally.
Closes#12503
Testing this requires manually installing the credential provider on a system with libsecret set up. I tested it on Arch Linux.
Update pretty_env_logger to 0.5
This updates pretty_env_logger from 0.4.0 to 0.5.0. The primary motivation is to drop some dependencies like `atty`.
https://github.com/seanmonstar/pretty-env-logger/compare/v0.4.0...v0.5.0
I think the main change is updating to env_logger 0.7 to 0.10 which syncs with what is normally used by cargo.
fix(package): Recognize and normalize `cargo.toml`
### What does this PR try to resolve?
This solution is a blend of conservative and easy
- Normalizes `cargo.toml` to `Cargo.toml` on publish
- Ensuring we publish the `prepare_for_publish` version and include `Cargo.toml.orig`
- Avoids dealing with trying to push existing users to `Cargo.toml`
- All other cases of `Cargo.toml` are warnings
- We could either normalize or turn this into an error in the future
- When helping users with case previously, we've only handle the `cargo.toml` case
- We already should have a fallback in case a manifest isn't detected
- I didn't want to put in too much effort to make the code more complicated to handle this
As a side effect, if a Linux user has `cargo.toml` and `Cargo.toml`, we'll only put one of them in the `.crate` file. We can extend this out to also include a warning for portability for case insensitive filesystems but I left that for after rust-lang/cargo#12235.
### How should we test and review this PR?
A PR at a time will show how the behavior changed as the source was edited
This does add a direct dependency on `unicase` to help keep case-insensitive comparisons easy / clear and to avoid riskier areas for bugs like writing an appropriate `Hash` implementation. `unicase` is an existing transitive dependency of cargo.
### Additional information
Fixes#12384
[Discussion on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/.60cargo.2Etoml.60.20on.20case.20insensitive.20filesystems)
To keep things simple, especially in getting a `Hash` implementation
correct, I'm leveraging `unicase` for case-insensitive
comparisons which is an existing dependency and I've been using for
years on other projects.
This also opens the door for us to add cross-platform compatibility
hazard warnings about multiple paths that would write to the same
location on a case insensitive file system. I held off on that because
I assume we would want #12235 first.
This does mean we can't test the "no manifest" case anymore because the
one case (no pun intended) I knew of for hitting it is now gone.
No code changes are required for the current uses of indexmap.
I also updated `toml_edit` to get its update to `indexmap 2`, and
`gix-hashtable` to align them all on `hashbrown 0.14`.
fix(embedded): Don't create an intermediate manifest
### What does this PR try to resolve?
More immediately, this is to unblock rust-lang/rust#112601
More generally, this gets us away from hackily writing out an out-of-line manifest from an embedded manifest. To parse the manifest, we have to write it out so our regular manifest
loading code could handle it. This updates the manifest parsing code to
handle it.
This doesn't mean this will work everywhere in all cases though. For
example, ephemeral workspaces parses a manifest from the SourceId and
these won't have valid SourceIds.
As a consequence, `Cargo.lock` and `CARGO_TARGET_DIR` are changing from being next to
the temp manifest to being next to the script. This still isn't the
desired behavior but stepping stones.
### How should we test and review this PR?
A Commit at a time
### Additional information
In production code, this does not conflict with #12255 (due to #12262) but in test code, it does.
To parse the manifest, we have to write it out so our regular manifest
loading code could handle it. This updates the manifest parsing code to
handle it.
This doesn't mean this will work everywhere in all cases though. For
example, ephemeral workspaces parses a manifest from the SourceId and
these won't have valid SourceIds.
As a consequence, `Cargo.lock` and `CARGO_TARGET_DIR` are changing from being next to
the temp manifest to being next to the script. This still isn't the
desired behavior but stepping stones.
This also exposes the fact that we didn't disable `autobins` like the
documentation says we should.
The hope is this will result in more resilient comment handling, being
more consistent with rustdoc.
I also hoped for less code but `syn` is doing less than I had expected,
requiring us to copy code over from other parts of rust. It seems every
proc macro has to do this but there is no guide to it, so they all do it
differently, only covering the cases they thought to test for.
Note that this still won't support `include_str!()`.
Previously, fetches and clones would routinely fail with a panic
that indicated that pack-negotiation can't take longer than 1 round
with our previous `Naive` approach.
With this version of `gitoxide` there is now faithful support for both
the `consecutive` and the `skipping` algorithm and multiple rounds of
negotiations, which should make all clones and fetches possible.
The implementation hinges on passing information about the kind of clone
and fetch to the `fetch()` method, which then configures the fetch accordingly.
Note that it doesn't differentiate between initial clones and fetches as
the shallow-ness of the repository is maintained nonetheless.
Update windows-sys
This updates the windows-sys dependency from 0.45 to 0.48. This shouldn't add or remove any duplicate dependencies (since there are other dependencies still using 0.45 and 0.42). The intent is to move it along the direction towards unifying in the future (though it seems like a moving target that will be difficult to ever hit).
This also bumps the home crate version. I think it should be OK to make the migration from winapi to windows-sys a patch version, though there seems to be some issues with the way windows-sys works that could introduce some build-time problems in some situations (such as those encountered in https://github.com/rust-lang/rust/pull/108665 and https://github.com/rust-lang/rust/pull/106610). However, I don't expect too much of an issue.
This is a short-term option until we can have a better solution for
globbing. This does not update `benches/` to support which has a README
in there preventing globbing; this seems low-churn enough not to find a
solution for it.
On the next sync-up with rust-lang/rust, we'll need to update 4e46301258/src/bootstrap/tool.rs (L588-L603)Fixes#11988
This is primarily for the release process of rust-lang/rust.
Note that in rustc-worksace-hack[1] it enable http2 via libnghttp2,
cargo probably needs to enable it to compile in rust-lang/rust.
[1]: 992d154f3a/src/tools/rustc-workspace-hack/Cargo.toml (L77)
Co-authored-by: Scott Schafer <schaferjscott@gmail.com>
Co-authored-by: Eric Huss <eric@huss.org>
Some dependencies in `resolver-tests` do not have any license
information. This prevent it from being a member when integrating in
rust-lang/rust. Will figure it out after.
Co-authored-by: Scott Schafer <schaferjscott@gmail.com>
Co-authored-by: Eric Huss <eric@huss.org>
The `git2` implementation can leverage that `git2` provides a consistent
view on the objects to be index, so it looks like 33% of the time is spent
receiving objects, and the rest of the time is used resolving them.
In `gitoxide`, there are two distinct phases and these are exposed by the way
we obtain progress for two separate phases. We have to do some math to renormalize
those to a single, continuous progress by mapping the values for 'amount of objects'
to the first half and second half of the progress bar respectively.
This has the advantage of having the first phase (receiving objects) end at 50%
and the second phase (resolving deltas) at 100%.
chore: Update base64
This removes one of cargo's duplicate dependencies as found by #11761.
`base64` is a bit of a controversial crate right now. It is going through large API changes, making it not as ergonomic for basic cases, which has ticked off a number of people. I kept it for now because its elsewhere in our dependency tree.
Byron already updated `prodash` to use the latest `parking_lot`
Remaining duplicates
- `hex` is blocked on `crypto-hash` which seems to no longer be maintained
- `hashbrown` is blocked on `indexmap` (updated in master) and `imara-diff`
- `humantime`, `env_logger`, `hermit-abi` are present from the optional `pretty_env_logger` dependency (why are we using optional deps? #6348)
- `windows-sys` is held back by `schannel`, `tempfile`, and `mio`
This allows to use `gitoxide` for all fetch operations, boosting performance
for fetching the `crates.io` index by a factor of 2.2x, while being consistent
on all platforms.
For trying it, nightly builds of `cargo` can specify `-Zgitoxide=fetch`.
It's also possible to set the `__CARGO_USE_GITOXIDE_INSTEAD_OF_GIT2=1` environment
variable (value matters), which is when `-Zgitoxide=none` can be used
to use `git2` instead.
Limitations
-----------
Note that what follows are current shortcomings that will be addressed in future PRs.
- it's likely that authentication around the `ssh` protocol will work differently in practice
as it uses the `ssh` program.
- clones from `file://` based crates indices will need the `git` binary to serve the locatl repository.
- the progress bar shown when fetching doesn't work like the orgiinal, but should already feel 'faster'.
renovate[bot]
Weihang Lo
bors
Eric Huss
Josh Stone
Ed Page
David Calavera
David Tolnay
Sebastian Thiel
Henry Chen
Arlo Siemsen
Jacob Finkelman
Deadbeef
Arlo Siemsen
abhiram
dependabot[bot]
klensy
WANG Rui
Scott Schafer
Luca Barbato
QiangHeisenberg
Andreas Hollmann
hzlinyiyu
