Add reasons to all ignored tests.
This adds a reason string to all `#[ignore]` attributes. This will be displayed when running the test (since 1.61), which can help quickly see and identify why tests are being ignored. It looks roughly like:
```
test basic ... ignored, requires nightly, CARGO_RUN_BUILD_STD_TESTS must be set
test build::simple_terminal_width ... ignored, --diagnostic-width is stabilized in 1.64
test check_cfg::features_with_cargo_check ... ignored, --check-cfg is unstable
test plugins::panic_abort_plugins ... ignored, requires rustc_private
```
In writing the contrib documentation for functional vs ui tests, I
realized that as we work to make snapbox work for the functional tests,
we'll need distinct `Assert` objects since we'll want to elide a lot
more content in functional tests. I'm making room for this by
qualifying the existing asserts as being for "ui".
Improve integration of the http server introduced by the http-registry feature.
Now the same HTTP server is used for serving downloads, the index, and
the API.
This makes it easier to write tests that deal with authentication and
http registries.
Git lets users define the default update/checkout strategy for a submodule
by setting the `submodule.<name>.update` key in `.gitmodules` file.
If the update strategy is `none`, the submodule will be skipped during
update. It will not be fetched and checked out:
1. *foo* is a big git repo
```
/tmp $ git init foo
Initialized empty Git repository in /tmp/foo/.git/
/tmp $ dd if=/dev/zero of=foo/big bs=1000M count=1
1+0 records in
1+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.482087 s, 2.2 GB/s
/tmp $ git -C foo add big
/tmp $ git -C foo commit -m 'I am big'
[main (root-commit) 84fb533] I am big
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 big
```
2. *bar* is a repo with a big submodule with `update=none`
```
/tmp $ git init bar
Initialized empty Git repository in /tmp/bar/.git/
/tmp $ git -C bar submodule add file:///tmp/foo foo
Cloning into '/tmp/bar/foo'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 1 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), 995.50 KiB | 338.00 KiB/s, done.
/tmp $ git -C bar config --file .gitmodules submodule.foo.update none
/tmp $ cat bar/.gitmodules
[submodule "foo"]
path = foo
url = file:///tmp/foo
update = none
/tmp $ git -C bar commit --all -m 'I have a big submodule with update=none'
[main (root-commit) 6c355ea] I have a big submodule not updated by default
2 files changed, 4 insertions(+)
create mode 100644 .gitmodules
create mode 160000 foo
```
3. *baz* is a clone of *bar*, notice *foo* submodule gets skipped
```
/tmp $ git clone --recurse-submodules file:///tmp/bar baz
Cloning into 'baz'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (3/3), done.
Submodule 'foo' (file:///tmp/foo) registered for path 'foo'
Skipping submodule 'foo'
/tmp $ git -C baz submodule update --init
Skipping submodule 'foo'
/tmp $
```
Cargo, on the other hand, ignores the submodule update strategy set in
`.gitmodules` properties when updating dependencies. Such behavior can
be considered against the wish of the crate publisher.
4. *bar* is now a lib with a big submodule with update disabled
```
/tmp $ cargo init --lib bar
Created library package
/tmp $ git -C bar add .
/tmp $ git -C bar commit -m 'I am a lib with a big submodule but update=none'
[main eb07cf7] I am a lib with a big submodule but update=none
3 files changed, 18 insertions(+)
create mode 100644 .gitignore
create mode 100644 Cargo.toml
create mode 100644 src/lib.rs
/tmp $
```
5. *qux* depends on *bar*, notice *bar*'s submodules are fetched
```
/tmp $ cargo init qux && cd qux
Created binary (application) package
/tmp/qux $ echo -e '[dependencies.bar]\ngit = "file:///tmp/bar"' >> Cargo.toml
/tmp/qux $ time cargo update
Updating git repository `file:///tmp/bar`
Updating git submodule `file:///tmp/foo`
real 0m22.182s
user 0m20.402s
sys 0m1.714s
/tmp/qux $
```
Fix it by checking if a Git repository submodule should be updated when
cargo processes dependencies.
6. With the change applied, submodules with `update=none` are skipped
```
/tmp/qux $ cargo cache -a > /dev/null
/tmp/qux $ time ~/src/cargo/target/debug/cargo update
Updating git repository `file:///tmp/bar`
Skipping git submodule `file:///tmp/foo`
real 0m0.029s
user 0m0.021s
sys 0m0.008s
/tmp/qux $
```
Fixes#4247.
Signed-off-by: Jakub Sitnicki <jakub@cloudflare.com>
When running tests in the `rust-lang/cargo` repo, `file!` is relative to
the crate root and tests are run relative to the crate root and
everything is fine.
When running tests in the `rust-lang/rust` repo, `file!` is relative to
the workspace root and tests are run relative to the crate root and
there is much sadness.
If we are compiling relative to the crate root, we could make the path
absolute and everything would be dandy but this needs to happen at
compile time. Didn't see a way to do this.
We could stop using `curr_dir` but that makes the tests a bit noisier
with more overhead for creating a new tests from an existing case.
Since we can reasonly know what all roots will be used for `file!`, we
can just hard code-in support for those two roots. Much happiness
ensues as everything works with this surgical hack.
This is something the existing test infrastructure supports, so I
figured I'd make it mirror it for snapbox. I'm mixed.
- It reads more like what a user would type, making it easier to run a
test locally or take a manual test case and automate it
- It can make it harder to parse the arguments when scanning tests
- Without using a crate like `shlex`, the syntax support is unclear
This was written for `cargo_add.rs`, based on `snapbox`. This allows
creating a test from a known reproduction case or easily debugging on an
existing test case.
Mark .cargo/git and .cargo/registry as cache dirs
Fixes#10457
### What does this PR try to resolve?
As we previously discussed in #10457 this marks `~/.cargo/git` and `~/.cargo/registry` as not to be included in backups, and not subject to content indexing. These directories can be fairly large and frequently changed, and should only contain content that can be re-downloaded if necessary.
### How should we test and review this PR?
I did two manual tests:
1. Using the binary built from this tree, run `cargo update` in a source tree that has a git dependency, and observe that afterwards, there is a `CACHEDIR.TAG` in `~/.cargo/git`.
2. Similarly, run `cargo update` and observe that there's a `CACHEDIR.TAG` in `~/.cargo/registry`.
(I ran this on Linux. This code should also trigger OS-specific behavior on macOS and Windows that's the same as is currently applied to `target/`.)
I added some test assertions.
- Add `ProcessBuilder::output` and ProcessBuilder::status`, which are
unopinionated version of `exec_*` (won't error out when exitcode > 0)
- Add `ProcessBuilder::retry_with_argfile` to enable trying with argfile
when hitting the "command line too big" error.
Registry functions return Poll to enable parallel fetching of index data
Adds `Poll` as a return type for several registry functions to enable parallel fetching of crate metadata with a future http-based registry.
Work is scheduled by calling the `query` and related functions, then waited on with `block_until_ready`.
This PR is based on the draft PR started by eh2406 here [#8985](https://github.com/rust-lang/cargo/pull/8985).
r? `@Eh2406`
cc `@alexcrichton`
cc `@jonhoo`
Implement "artifact dependencies" (RFC-3028)
Tracking issue: #9096
#### Tasks
* [x] -Z unstable option called `bindeps`
* ✅ **(config)** allow parsing of newly introduced 'artifact' fields
* [x] into `TomlManifest`
* [x] into `Manifest`
- [x] ~~abort~~ warn if artifacts are used on stable
* ✅ **resolver** : assure artifact dependencies are part of the resolution process and unified into the dependency tree
* 🔬**compiler**: make it understand 'artifact' dependencies and pass new environment variables to the crate being build
* [x] `lib=false` should not be considered a rust library for the dependent, in unit and possibly resolve graph
* [x] assure profile settings are applied correctly
* [x] target overrides work
* [x] `target = "target"` in build deps
* [x] other targets on build deps
* [x] other targets on non-build deps
* [x] 'no-cross doc tests' seems like a constraint we should apply as well maybe
* [x] more confidence with `resolver = "2"`
* [x] assure artifact placement is correct (bin and various forms of lib)
* ✅ **serialization**: rewriting manifests (i.e. for publishing) does not discard artifact information
* [x] publishing keeps `artifact` and `lib` values
* **Other cargo subcommands**
* [x] `cargo metadata`
* leave unchanged
* [x] artifacts work with `cargo check`
* [x] artifacts work with rustdoc, such that it doesn't document them unless `lib=true`
* [x] `cargo tree` maybe?
* [x] `cargo clean` should clean artifacts - even though it's more complex, ultimately it deletes the `target` directory.
* [x] artifacts work with `cargo test` (and dev-dependencies)
* [x] doctests
* [x] try `reproducible` repository as well.
* 🧪 **tests** for more subtle RFC constraints
- [x] build scripts cannot access artifact environment variables at compile time, only at runtime)
- [x] assure 'examples' which also support crate-type fields like [[lib]] won't become artifacts themselves.
- [x] assure `--out-dir` does not leak artifacts - tested manually, it seemed to niche to add a test.
- [x] try `target="foo"` in artifact and assure it sees a decent error message
- [x] Assure RFC 3176 _doesn't_ work
* 🧹cleanup and finalization
- [x] assure no `TODO(ST)` markers are left in code
- [x] assure no tests are ignored
- [x] use `resolver = "1"` once to assert everything also works with the previous resolver, but leave it on "2".
#### Implementation and review notes
- artifacts and unstable options are only checked when transforming them from `TomlManifest` to `Manifest`, discarding artifact information if the unstable flag is not set. Nowhere else in code will the CLI options be checked again.
- `If no binaries are specified, all the binaries in the package will be built and made available.` - this should only refer to `[[bin]]` targets, not examples for instance.
- artifact binaries won't be uplifted, hence won't be present outside of their output directory
- ❗️We don't know how [package links](00e925f61f/src/cargo/core/compiler/unit_dependencies.rs (L380)) will affect artifacts for build dependencies. Should probably be thought through.
- ❗️The location of artifacts is only tested roughly to avoid having to deal with different output names on the four platforms that seem to matter (gnu, macos, windows msvc, windows gnu).
- `cargo tree` doesn't handle artifacts specifically, and it might be interesting to make clear if an artifact is only an artifact, or both artifact and dependency.
- Most error and warning messages can probably be more cargo-matic.
#### Questions
* Does `cargo` without the feature enabled have to complain about the "artifact" field in a dependency, like it does right now? It doesn't look like machinery for that exists in `do_read_manifest()`.
- ✔️It warns now
* Should parsing of artifact values, like "bin" be case sensitive?
- ✔️ It's case sensitive now, which should help with serde roundtripping.
#### Review Progress
* [x] address Josh's review notes one by one
* [x] introduce `IsArtifact` (see [this answer](https://github.com/rust-lang/cargo/pull/9992#discussion_r774928102)) (76cd48a2d62d74e043a1a482199c5bb920f50311)
* [x] prefer uplifting artifact deps that were written into the `deps` directory, but prefer to do that in [this PR instead](https://github.com/rust-lang/cargo/pull/9992)
* [x] add extra-tests as described in Josh's comment: " features get unified between a Rust library and a binary, and one that confirms features don't get unified between a Rust library and a binary for a different target?"
* [x] Make target-based artifact splitting work by porting parts of RFC-3176
* [x] test-support/cross-compile
* [x] namespace separation
* [x] re-create RFC-3176 what's not in RFC-3028, namely multidep support and all related tests
* [x] Address Eh2406 's review comments
* [x] Address Eric's comments
* [x] Unstable features need to be documented in unstable.md.
* [x] sort out [`target_data`](https://github.com/rust-lang/cargo/pull/9992#discussion_r787316229)
* [x] figure out [cargo metadata](https://github.com/rust-lang/cargo/pull/9992#discussion_r787320923)
* [x] See if target-data can work without an [index format update](https://github.com/rust-lang/cargo/pull/9992#issuecomment-1021697124).
* [x] truncate comments at 80-90 lines and remove unused methods, remove -Z unstable-options, use `cfg!(target_env = "msvc")`
* [x] add missing doc comments to newly added methods and funtions
* [x] simplify method parameters and inline some functions
* [x] add test and extend test to show additional issues
* [x] assure current set of tests works consistently, also on windows
Sponsored by [Profian](https://www.profian.com)
Pietro Albini
Weihang Lo
bors
Eric Huss
Ed Page
Scott Schafer
Arlo Siemsen
Jakub Sitnicki
Yuki Okushi
Koichi ITO
klensy
Martin Pool
Weihang Lo
Lucas Kent
Loïc BRANSTETT
Jacob Finkelman