Commit Graph

17362 Commits

Author SHA1 Message Date
bors 7e9c2ef30f Auto merge of #13775 - epage:incomplete-dep, r=weihanglo
fix(toml)!: Disallow source-less dependencies

### What does this PR try to resolve?

This is part of #13629 addressing “dependency without path, version, git, workspace specified”.

This turns deps like
```toml
foo = { optional = true }
```
from `version="*"` deps with a warning into errors. This breaking change was deemed acceptable because this behavior has been considered a bug from the beginning.
We have gotten little to no feedback about people wanting this behavior.

This improves our forwards-compatibility story as we can add new dependency sources and they won't be considered a wildcard registry dependency on older cargo.

### How should we test and review this PR?

I removed the `cargo_add` test because it is redundant.

We no longer get the “unused key” warnings because those warnings get deferred to after this error gets reported.

### Additional information
2024-04-18 22:02:05 +00:00
Ed Page 0a5c7d987f fix(toml): Don't crash on parse errors that point to multi-byte character
Fixes #13772
2024-04-18 16:57:22 -05:00
Ed Page d1f02476c2 fix(toml): Deprecate underscore fields 2024-04-18 16:15:34 -05:00
Ed Page 868662c480 test(toml): Show underscore field behavior 2024-04-18 14:57:12 -05:00
Ed Page 208d10d06d fix(toml): Switch the duplicate table warning to an unused key warning 2024-04-18 14:49:50 -05:00
Ed Page a4a3254bc3 refactor(toml): Pull conditional into deprecation fn 2024-04-18 14:45:45 -05:00
Ed Page 42939d9eff refactor(toml): Clarify function name 2024-04-18 14:45:37 -05:00
Ed Page 265f93e913 test(toml): Be consistent on expected output formatting 2024-04-18 14:45:27 -05:00
Ed Page d210644a1b test(toml): Switch from build to check for these bad_config 2024-04-18 14:40:04 -05:00
Ed Page 8520ec63b6 test(toml): Centralize underscore field tests 2024-04-18 14:38:05 -05:00
Ed Page 3fc52d53b1 test(toml): Make test names scale to more scenarios 2024-04-18 14:35:39 -05:00
Ed Page f1caef11c4 test: Move off of deprecated manifest fields 2024-04-18 14:11:31 -05:00
bors 2956296659 Auto merge of #13769 - epage:msrv-config, r=weihanglo
fix(msrv): Put MSRV-aware resolver behind a config

### What does this PR try to resolve?
This is a part of #13540 which is a party of #9930.

The config is `resolver.something-like-precedence` with values:
- `something-like-maximum` (default)
- `something-like-rust-version`

This is punting on the actual config schema so we can implement
`package.resolver` and `edition = "2024"` support as we want the
MSRV-aware resolver available without `cargo_features`.

### How should we test and review this PR?

One of the included test cases shows a bug with `cargo install`.  Resolving that will be tracked in #9930

### Additional information
2024-04-18 18:48:45 +00:00
Ed Page cf23e4b538 fix(toml)!: Disallow source-less dependencies
This is part of #13629

This turns deps like
```toml
foo = { optional = true }
```
from `version="*"` deps with a warning into errors.
This breaking change was deemed acceptable because this behavior has
been considered a bug from the beginning.
We have gotten little to no feedback about people wanting this behavior.

This improves our forwards-compatibility story as we can add new
dependency sources and they won't be considered a wildcard registry
dependency on older cargo.
2024-04-18 12:55:27 -05:00
Scott Schafer 20b3734b40
refactor: Improve lint tests 2024-04-18 11:13:15 -06:00
bors 39b8f1702e Auto merge of #13771 - epage:rust-version, r=weihanglo
fix(msrv): Error, rather than panic, on rust-version 'x'

### What does this PR try to resolve?

Fixes #13768

### How should we test and review this PR?

### Additional information
2024-04-18 01:52:33 +00:00
bors 5afc53a3c7 Auto merge of #13770 - arlosi:cred-trim-newlines, r=epage
fix(credential): trim newlines in tokens from stdin

### What does this PR try to resolve?

`cargo login` when using a credential provider other than `cargo:token` does not automatically trim whitespace from tokens.

This can lead to extra whitespace being included in the pasted token value (usually a trailing newline) that makes the token invalid.

### How should we test and review this PR?

First commit adds a test showing the problematic behavior. Second commit fixes it.
2024-04-17 23:12:14 +00:00
Arlo Siemsen 6207f93087 fix(credential): trim newlines in token from stdin for credential providers 2024-04-17 17:39:46 -05:00
Ed Page 6d8d3b6420 fix(schemas): Allow parsing pre-release with X 2024-04-17 16:49:22 -05:00
Ed Page 6f22e9dbee test(schemas): Add PartialVersion unit tests 2024-04-17 16:47:51 -05:00
Ed Page 3a2cc82789 test(msrv): Migrate most parse tests to unit tests 2024-04-17 16:47:29 -05:00
Ed Page 675d67d093 fix(msrv): Error, rather than panic, on rust-version 'x'
Fixes #13768
2024-04-17 15:10:18 -05:00
Ed Page af9288f1b6 test(msrv): Show current parse behavior with X 2024-04-17 15:05:58 -05:00
Arlo Siemsen a498391686 show buggy behavior of not trimming newlines in new credential process test 2024-04-17 12:50:32 -05:00
Ed Page 18a5940046 fix(msrv): Put MSRV-aware resolver behind a config
This is a part of #13540 which is a party of #9930.

The config is `resolver.something-like-precedence` with values:
- `something-like-maximum` (default)
- `something-like-rust-version`

This is punting on the actual config schema so we can implement
`package.resolver` and `edition = "2024"` support as we want the
MSRV-aware resolver available without `cargo_features`.
2024-04-17 12:35:14 -05:00
Ed Page 7af4bcf397 test(msrv): Show config on stable 2024-04-17 12:35:05 -05:00
Ed Page 34afc4f1ff test(msrv): Prep for config to be added 2024-04-17 11:44:02 -05:00
Ed Page 226b53ef31 test(resolver): Verify some more msrv cases 2024-04-17 11:35:18 -05:00
bors 852a31615d Auto merge of #13767 - epage:msrv-test, r=ehuss
test(msrv): Re-organize MSRV tests

### What does this PR try to resolve?

- Merge test cases
- Clarify names
- Focus on `cargo tree`, rather than `cargo check` (faster and more to
  what we are testing)

### How should we test and review this PR?

### Additional information
2024-04-17 15:32:37 +00:00
bors 3a9b59f213 Auto merge of #13764 - epage:install-lock, r=ehuss
feat(install): Including Locking message

### What does this PR try to resolve?

This extends #13561 to include `cargo install`, like #13759 did for `cargo update`.

As we switch to MSRV-aware resolver, this will help users work out why
MSRV-aware resolving isn't helping them.

This will also make it more obvious if we breaking things when
developing the MSRV-aware resolver.

### How should we test and review this PR?

### Additional information

This still leaves `cargo publish` and a couple other misc situations that I'm intentionally avoiding because
- They hit some weird cases that can confuse the user (e.g. causing `cargo install --locked` to show that 1 package is being added) and we can't distinguish these cases too well from where this is happening
- The value is lower
2024-04-17 14:59:32 +00:00
Ed Page 3a3dda38d9 test(msrv): Reorganize MSRV tests
- Merge test cases
- Clarify names
- Focus on `cargo tree`, rather than `cargo check` (faster and more to
  what we are testing)
2024-04-17 09:32:44 -05:00
Ed Page d6643254b5 test(msrv): Show regular MSRV resolve case 2024-04-17 09:32:41 -05:00
Ed Page 04f84e461b test(msrv): Group bad rust-version tests 2024-04-17 09:32:36 -05:00
bors 6f06fe908a Auto merge of #13747 - epage:deprecated, r=weihanglo
fix(toml): Error on `[project]` in Edition 2024

### What does this PR try to resolve?

`[package]` was added in 86b2a2a432 in the pre-1.0 days but `[project]` was never removed nor did we warn on its use until recently in #11135.  So likely we can't remove it completely but we can remove it in Edition 2024+.

This includes `cargo fix` support which is hard coded directly into the `cargo fix` command.

This is part of
- #13629
- rust-lang/rust#123754

While we haven't signed off on everything in #13629, I figured this (and a couple other changes) are not very controversial.

### How should we test and review this PR?

Per commit.  Tests are added to show the behavior changes, including in `cargo fix`.

### Additional information
2024-04-16 18:47:44 +00:00
Ed Page 1d0c6ebd8f feat(install): Including Locking message
As we switch to MSRV-aware resolver, this will help users work out why
MSRV-aware resolving isn't helping them.

This will also make it more obvious if we breaking things when
developing the MSRV-aware resolver.
2024-04-16 13:35:25 -05:00
Ed Page c38673c665 refactor(resolve): Make it easier to customize around the resolve call 2024-04-16 12:28:27 -05:00
Ed Page ea27ec1f96 docs: Clarify why we aren't printing Locking in some cases 2024-04-16 12:08:57 -05:00
bors b9d913e532 Auto merge of #13759 - epage:more-lock, r=weihanglo
feat(update): Include a Locking message

### What does this PR try to resolve?

This extends #13561 to `cargo update`.  I previously left it out because the locking message was redundant.  However the `Locking` message has been extended in #13754 to include the resolving policy which can be a useful point of interest (e.g. "why does `cargo update` do nothing? Oh, `-Zminimal-versions` is enabled").

I still left out the message for `--precise` because the user is overriding all of that.

I'd still like to extend all of this to `cargo install` (and maybe all resolves) but that is taking more investigation.

### How should we test and review this PR?

### Additional information
2024-04-16 02:35:06 +00:00
bors add150cb04 Auto merge of #13760 - rust-lang:renovate/crate-gix-vulnerability, r=epage
chore(deps): update rust crate gix to 0.62.0 [security]

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [gix](https://togithub.com/Byron/gitoxide) | workspace.dependencies | minor | `0.61.0` -> `0.62.0` |

### GitHub Vulnerability Alerts

#### [GHSA-98p4-xjmm-8mfh](https://togithub.com/Byron/gitoxide/security/advisories/GHSA-98p4-xjmm-8mfh)

### Summary

`gix-transport` does not check the username part of a URL for text that the external `ssh` program would interpret as an option. A specially crafted clone URL can smuggle options to SSH. The possibilities are syntactically limited, but if a malicious clone URL is used by an application whose current working directory contains a malicious file, arbitrary code execution occurs.

### Details

This is related to the patched vulnerability https://github.com/advisories/GHSA-rrjw-j4m2-mf34, but appears less severe due to a greater attack complexity. Since [https://github.com/Byron/gitoxide/pull/1032](https://togithub.com/Byron/gitoxide/pull/1032), `gix-transport` checks the host and path portions of a URL for text that has a `-` in a position that will cause `ssh` to interpret part of all of the URL as an option argument. But it does not check the non-mandatory username portion of the URL.

As in Git, when an address is a URL of the form `ssh://username@hostname/path`, or when it takes the special form `username@hostname:dirs/repo`, this is treated as an SSH URL. `gix-transport` will replace some characters in `username` with their `%`-based URL encodings, but otherwise passes `username@hostname` as an argument to the external `ssh` command. This happens even if `username` begins with a hyphen. In that case, `ssh` treats that argument as an option argument, and attempts to interpret and honor it as a sequence of one or more options possibly followed by an operand for the last option.

This is harder to exploit than GHSA-rrjw-j4m2-mf34, because the possibilities are constrained by:

- The difficulty of forming an option argument `ssh` accepts, given that characters such as `=`, `/`, and `\`, are URL-encoded, `:` is removed, and the argument passed to `ssh` contains the ``@`` sign and subsequent host identifier, which in an effective attack must be parseable as a suffix of the operand passed to the last option.

  The inability to include a literal `=` prevents the use of `-oNAME=VALUE` (e.g., `-oProxyCommand=payload`). The inability to include a literal `/` or `\` prevents smuggling in a path operand residing outside the current working directory, incuding on Windows. (Although a `~` character may be smuggled in, `ssh` does not perform its own tilde expansion, so it does not form an absolute path.)

- The difficulty, or perhaps impossibility, of completing a connection (other than when arbitrary code execution has been achieved). This complicates or altogether prevents the use of options such as `-A` and `-X` together with a connection to a real but malicious server. The reason a connection cannot generally be completed when exploiting this vulnerability is that, because the argument `gix-transport` intends as a URL is treated as an option argument, `ssh` treats the subsequent non-option argument `git-upload-pack` as the host instead of the command, but it is not a valid host name.

  Although `ssh` supports aliases for hosts, even if `git-upload-pack` could be made an alias, that is made difficult by the URL-encoding transformation.

However, an attacker who is able to cause a specially named `ssh` configuration file to be placed in the current working directory can smuggle in an `-F` option referencing the file, and this allows arbitrary command execution.

This scenario is especially plausible because programs that operate on git repositories are often run in untrusted git repositories, sometimes even to operate on another repository. Situations where this is likely, such that an attacker could predict or arrange it, may for some applications include a malicious repository with a malicious submodule configuration.

Other avenues of exploitation exist, but appear to be less severe. For example, the `-E` option can be smuggled to create or append to a file in the current directory (or its target, if it is a symlink). There may also be other significant ways to exploit this that have not yet been discovered, or that would arise with new options in future versions of `ssh`.

### PoC

To reproduce the known case that facilitates arbitrary code execution, first create a file in the current directory named `configfile@example.com`, of the form

```text
ProxyCommand payload
```

where `payload` is a command with an observable side effect. On Unix-like systems, this could be `date | tee vulnerable` or an `xdg-open`, `open`, or other command command to launch a graphical application. On Windows, this could be the name of a graphical application already in the search path, such as `calc.exe`.

(Although the syntax permitted in the value of `ProxyCommand` may vary by platform, this is not limited to running commands in the current directory. That limitation only applies to paths directly smuggled in the username, not to the contents of a separate malicious configuration file. Arbitrary other settings may be specified in `configfile@example.com` as well.)

Then run:

```sh
gix clone 'ssh://-Fconfigfile@example.com/abc'
```

Or:

```sh
gix clone -- '-Fconfigfile@example.com:abc/def'
```

(The `--` is required to ensure that `gix` is really passing the argument as a URL for use in `gix-transport`, rather than interpreting it as an option itself, which would not necessarily be a vulnerability.)

In either case, the payload specified in `configfile@example.com` runs, and its side effect can be observed.

Other cases may likewise be produced, in either of the above two forms of SSH addresses. For example, to create or append to the file `errors@example.com`, or to create or append to its target if it is a symlink:

```sh
gix clone 'ssh://-Eerrors@example.com/abc'
```

```sh
gix clone -- '-Eerrors@example.com:abc/def'
```

### Impact

As in https://github.com/advisories/GHSA-rrjw-j4m2-mf34, this would typically require user interaction to trigger an attempt to clone or otherwise connect using the malicious URL. Furthermore, known means of exploiting this vulnerability to execute arbitrary commands require further preparatory steps to establish a specially named file in the current directory. The impact is therefore expected to be lesser, though it is difficult to predict it with certainty because it is not known exactly what scenarios will arise when using the `gix-transport` library.

Users who use applications that make use of `gix-transport` are potentially vulnerable, especially:

- On repositories with submodules that are automatically added, depending how the application manages submodules.
- When operating on other repositories from inside an untrusted repository.
- When reviewing contributions from untrusted developers by checking out a branch from an untrusted fork and performing clones from that location.

---

### Release Notes

<details>
<summary>Byron/gitoxide (gix)</summary>

### [`v0.62.0`](https://togithub.com/Byron/gitoxide/releases/tag/gix-v0.62.0): gix v0.62

[Compare Source](https://togithub.com/Byron/gitoxide/compare/gix-v0.61.1...gix-v0.62.0)

Please note that this release contains a security fix originally implemented in `gix-transport` via [this PR](https://togithub.com/Byron/gitoxide/pull/1342) which prevents `ssh` options to be smuggled into the `ssh` command-line invocation with a username provided to a clone or fetch URL.

Details can be found [in the advisory](https://togithub.com/Byron/gitoxide/security/advisories/GHSA-98p4-xjmm-8mfh).

##### Bug Fixes

-   `into_index_worktree_iter()` now takes an iterator, instead of a Vec.
    This makes the API more consistent, and one can pass `None`
    as well.
-   show submodules in status independently of their active state.
    Even inactive submodules are shown in the status by `git status`,
    so `gix` should do the same.

    First observed in [https://github.com/helix-editor/helix/pull/5645#issuecomment-2016798212](https://togithub.com/helix-editor/helix/pull/5645#issuecomment-2016798212)
-   forward `curl` rustls feature from `gix-transport` to avoid `curl` in `gix`.
    This removes the `curl` dependency just for configuring it, and removes
    a hazard which became evident with reqwest.

##### Bug Fixes (BREAKING)

-   Make `topo` more similar to `Ancestors`, but also rename `Ancestors` to `Simple`

##### Commit Statistics

-   16 commits contributed to the release over the course of 20 calendar days.
-   22 days passed between releases.
-   4 commits were understood as [conventional](https://www.conventionalcommits.org/).
-   1 unique issue was worked on: [https://github.com/Byron/gitoxide/issues/1328](https://togithub.com/Byron/gitoxide/issues/1328)

##### Thanks Clippy

[Clippy](https://togithub.com/rust-lang/rust-clippy) helped 1 time to make code idiomatic.

##### Commit Details

-   **[https://github.com/Byron/gitoxide/issues/1328](https://togithub.com/Byron/gitoxide/issues/1328)**
    -   Forward `curl` rustls feature from `gix-transport` to avoid `curl` in `gix`. (98cfbec512)
-   **Uncategorized**
    -   Prepare changelogs prior to release (57552717f4)
    -   Merge pull request [https://github.com/Byron/gitoxide/pull/1341](https://togithub.com/Byron/gitoxide/pull/1341) from szepeviktor/typos (55f379bc47)
    -   Fix typos (f72ecce45b)
    -   Merge branch 'add-topo-walk' (b590a9d2b6)
    -   Adapt to changes in `gix-traverse` (1cfeb11f1f)
    -   Make `topo` more similar to `Ancestors`, but also rename `Ancestors` to `Simple` (2a9c178326)
    -   Adapt to changes in `gix-traverse` (6154bf3a34)
    -   Thanks clippy (7f6bee5452)
    -   Merge branch 'status' (45edd2ea66)
    -   `into_index_worktree_iter()` now takes an iterator, instead of a Vec. (18b2921aaa)
    -   Show submodules in status independently of their active state. (719ced8a79)
    -   Make it easier to discover `is_path_excluded()` in documentation (c13632959e)
    -   Adapt to changes in `gix-index` (1e1fce11a9)
    -   Merge branch 'patch-1' (9e9c653a83)
    -   Remove dep reqwest from gix (e3eedd8b53)

### [`v0.61.1`](https://togithub.com/Byron/gitoxide/releases/tag/gix-v0.61.1): gix v0.61.1

[Compare Source](https://togithub.com/Byron/gitoxide/compare/gix-v0.61.0...gix-v0.61.1)

This release also updates `reqwest` to v0.12, bringing hyper 1.0 and a more recent `rustls` version.

##### Bug Fixes

-   missing closing backtick in gix lib documentation

##### Commit Statistics

-   7 commits contributed to the release over the course of 2 calendar days.
-   3 days passed between releases.
-   1 commit was understood as [conventional](https://www.conventionalcommits.org).
-   0 issues like '(#ID)' were seen in commit messages

##### Commit Details

<csr-read-only-do-not-edit/>

<details><summary>view details</summary>

-   **Uncategorized**
    -   Prepare changelogs prior to release ([`7018a92`](https://togithub.com/Byron/gitoxide/commit/7018a92))
    -   Merge branch 'patch-1' ([`8fde62b`](https://togithub.com/Byron/gitoxide/commit/8fde62b))
    -   Turn`curl` into a workspace package ([`adee500`](https://togithub.com/Byron/gitoxide/commit/adee500))
    -   Make reqwest a workspace package ([`369cf1b`](https://togithub.com/Byron/gitoxide/commit/369cf1b))
    -   Merge pull request [#&#8203;1325](https://togithub.com/Byron/gitoxide/issues/1325) from kdelorey/fix/simple-docs-formatting ([`3b34699`](https://togithub.com/Byron/gitoxide/commit/3b34699))
    -   Fixed opening of backtick in documentation. ([`f1bc4cd`](https://togithub.com/Byron/gitoxide/commit/f1bc4cd))
    -   Missing closing backtick in gix lib documentation ([`e1fec3c`](https://togithub.com/Byron/gitoxide/commit/e1fec3c))

</details>

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/rust-lang/cargo).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yOTMuMCIsInVwZGF0ZWRJblZlciI6IjM3LjI5My4wIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIiwibGFiZWxzIjpbXX0=-->
2024-04-16 02:03:30 +00:00
bors f8c4af5ec2 Auto merge of #13761 - weihanglo:pkgidspec, r=epage
test(schemas): Ensure tests cover the correct case

Also, I am preparing an experiment of unidiff patch, which will introduce one more error kind.
2024-04-16 01:32:39 +00:00
Weihang Lo eacdfd2917
test: fix tests to assert unxpected querystrring error 2024-04-15 20:25:01 -04:00
Weihang Lo 99b4a49530
test: assert error kind for pkgidspec parsing 2024-04-15 20:25:01 -04:00
renovate[bot] 1a111ab753 chore(deps): update rust crate gix to 0.62.0 [security] 2024-04-15 21:30:05 +00:00
Ed Page cbd9def9dc feat(fix): Migrate from project to package on Edition 2024 2024-04-15 13:33:28 -05:00
Ed Page 98298d4966 feat(fix): Report manifest migrations 2024-04-15 13:33:27 -05:00
Ed Page d5606b2278 test(fix): Show current project/package behavior 2024-04-15 13:32:17 -05:00
Ed Page 1ec1455747 refactor(fix): Reload the workspace
This opens the door for fixing the workspace
2024-04-15 13:32:17 -05:00
Ed Page ca699d264a refactor(fix): Flatten workspace loading
This misses out on features that shouldn't be relevant to fix, like
avoid-dev-deps.

However, this prepares the way for workspace re-loading.
2024-04-15 13:32:17 -05:00
Ed Page 9fb428bd05 refactor(fix): Move workspace loading closer to use 2024-04-15 13:32:17 -05:00
Ed Page 2f4d3df54d fix(toml): Error on use of [project] on 2024 Edition 2024-04-15 13:32:17 -05:00