Move `Limit` out of `rustc_hir`
It feels rather out of place in `rustc_hir`, so this PR moves it to `rustc_data_structures`. Part of hopefully moving attributes from `rustc_hir` to their own crate.
~~This also lets `rustc_type_ir` use it~~ actually rustc_type_ir needs to compile on stable and `rustc_data_structures` uses like 30 features 😢
Resolve: more preperation work for parallelizing the import resolution loop
This is basically rust-lang/rust#158845 but we do not:
- actually use `par_slice` because:
- we do not migrate `CmRefCell` to use `RwLocks` (yet) because of perf reasons.
The resolution loop is now in place instead of recollecting indeterminate imports.
r? @petrochenkov
Currently it is all still single threaded, but almost everything needed to make the loop use multithreaded code is done in this commit.
For performance reasons we do the resolution now in place instead of recollecting the indeterminate imports ([see benchmark pr](https://github.com/rust-lang/rust/pull/159387) and pr of this commit).
Populating external resolution tables now use `Once` to ensure only 1 thread builds the table and all others wait if they need it, this has no impact on single threaded behaviour.
`Cache(Ref)Cells` are not tackled yet because they are not needed here (and require extra work).
`Interned` does pointer equality/hashing, which is valid in two cases.
- The values are guaranteed to be unique (e.g. via interning, or
construction). This is how `rustc_middle` uses `Interned`.
- The type has "identity" and different values should be considered
distinct even if they are identical. This is how `rustc_resolve` uses
`Interned`.
PR 137202 tried to clarify things by adding a `T: Hash` constraint to
`Interned<'a, T>`. This constraint isn't actually used, because
`Interned` is hashed based on pointer value, not contents. But it was
intended to communicate the idea that a type stored in `Interned` is
actually interned, which is likely to be done with hashing. Panicking
impls of `Hash` were added for the relevant `rustc_resolve` types to
work around the fact that it doesn't use hashing-based interning.
In my opinion PR 137202 didn't improve things. The `T: Hash` constraint
is only aimed at the interning case, and even for that case it's not
quite right because you could use a `BTreeMap` to intern instead of a
`HashMap`.
This commit does several things.
- Removes the `T: Hash` constraint and the `Hash` impls for
`rustc_resolve` types added in PR 137202.
- Improves the comments on `Interned` to cover the non-interning cases.
- Removes the `PartialOrd`/`Ord` impls on `Interned` because (a) they're
not used, and (b) their meaning is unclear for the "identity" case.
- Improves the documentation in `rustc_resolve` to explain how
`Interned` usage is valid there.
thin-vec versions before 0.2.16 have a use-after-free / double-free in
`IntoIter::drop` and `ThinVec::clear` when an element's `Drop` panics
(RUSTSEC-2026-0103). The requirement now sits past the affected range.
Add a strategy FnMut to inject behavior into the flush cycle
This PR adds the interface needed to avoid forking FileEncoder in https://github.com/rust-lang/rust/pull/154724.
This adds a FnMut used as a strategy/observer to see the bytes being flushed to disk. For https://github.com/rust-lang/rust/pull/154724, that would be used to calculate the metadata hash on the raw metadata bytes.
The 'static hack is pretty ugly, but otherwise, the FileEncoder lifetime goes exceptionally viral and requires some very unfortunate workarounds to avoid self referential lifetimes. That's a bit of a timebomb if the wrong FileEncoder client tries to use the new API, but the worst case outcome is a wild goose chase of lifetime chasing.
Full disclosure: I used Claude Code to generate much of the PR. I have fully reviewed the code and stand behind it.
r? @cjgillot
The in-memory dep graph kept for `-Zquery-dep-graph` is built by
`GraphEncoder::record`, which pushed each node using `try_lock` and
silently dropped the node when the lock was contended. Single-threaded
the only contention is a query forced re-entrantly from within
`with_retained_dep_graph`, so dropping was harmless. Under the parallel
frontend several encoder threads record nodes at the same time, so a
contended `try_lock` dropped nodes and edges at random, leaving the
retained graph nondeterministic and making the dep-graph ui tests, which
assert on its contents, fail intermittently.
Remove that reentrancy in `check_paths`: read the graph into owned
results while the lock is held, then emit the diagnostics afterwards.
The only query forced inside the old closure was `def_path_str` (for the
error messages), which can create dep nodes and re-enter the encoder;
doing the emission after the lock is released means `record` never
re-enters while the lock is held. `record` can then block on the lock
unconditionally instead of using `try_lock`, so concurrent encoders
never drop a node or edge.
This only affects compilation with `-Zquery-dep-graph`; with the flag
off the retained graph is absent and nothing changes.
The type's been stable for over 6 weeks now, so let's use it! It's better for cases like this one where it's stored in a data structure.
Probably won't be materially faster, but does make the variant slightly smaller and lets some more things be `Copy`.
Specifically:
- `HashStable` -> `StableHash` (trait)
- `HashStable` -> `StableHash` (derive)
- `HashStable_NoContext` -> `StableHash_NoContext` (derive)
Note: there are some names in `compiler/rustc_macros/src/hash_stable.rs`
that are still to be renamed, e.g. `HashStableMode`.
Part of MCP 983.
Simplify `HashStable`
This PR:
- Simplifies the `HashStable` trait, by moving its generic parameter from the trait to its single method.
- Eliminates the need for the non-obvious `derive(HashStable)`/`derive(HashStable_Generic)` distinction.
- Reduces the need for, and clarifies, the non-obvious `derive(HashStable)`/`derive(HashStable_NoContext)` distinction.
r? @fee1-dead
Thanks to the `HashStable` trait being simplified,
`HashStable_NoContext` is only needed when a (near) perfect derive is
required. In practice, this means it's only needed for types with a
generic `<I: Interner>` parameter.
This commit replaces `derive(HashStable_NoContext)` with
`derive(HashStable)` for all types that don't have `<I: Interner>`.
`std::hash::Hash` looks like this:
```
pub trait Hash {
fn hash<H>(&self, state: &mut H)
where H: Hasher;
...
}
```
The method is generic.
In contrast, `HashStable` looks like this:
```
pub trait HashStable<Hcx> {
fn hash_stable(&self, hcx: &mut Hcx, hasher: &mut StableHasher);
}
```
and impls look like this (in crates upstream of `rustc_middle`):
```
impl<Hcx: HashStableContext> HashStable<Hcx> for Path {
fn hash_stable(&self, hcx: &mut Hcx, hasher: &mut StableHasher) {
...
}
}
```
or this (in `rustc_middle` and crates downstream of `rustc_middle`):
```
impl<'tcx> HashStable<StableHashingContext<'tcx>> for rustc_feature::Features {
fn hash_stable(&self, hcx: &mut StableHashingContext<'tcx>, hasher: &mut StableHasher) {
...
}
}
```
Differences to `std::hash::Hash`:
- The trait is generic, rather than the method.
- The way impls are written depends their position in the crate graph.
- This explains why we have both `derive(HashStable)` and
`derive(HashStable_Generic)`. The former is for the
downstream-of-`rustc_middle` case, the latter is for the upstream of
`rustc_middle` case.
Why the differences? It all boils down to `HashStable` and
`HashStableContext` being in different crates. But the previous commit
fixed that, which means `HashStable` can be simplified to this, with a
generic method:
```
pub trait HashStable {
fn hash_stable<Hcx: HashStableContext>(&self, hcx: &mut Hcx, hasher: &mut StableHasher);
}
```
and all impls look like this:
```
impl HashStable for Path {
fn hash_stable<Hcx: HashStableContext>(&self, hcx: &mut Hcx, hasher: &mut StableHasher) {
...
}
}
```
Other consequences:
- `derive(HashStable_Generic)` is no longer needed; `derive(HashStable)`
can be used instead.
- In this commit, `derive(HashStable_Generic` is made a synonym of
`derive(HashStable)`. The next commit will remove this synonym,
because it's a change that touches many lines.
- `#[stable_hash_generic]` is no longer needed (for `newtype_index`);
`#[stable_hash]` can be used instead.
- `#[stable_hash_no_context]` was already a synonym of
`#[stable_hash_generic]`, so it's also removed in favour of just
`#[stable_hash]`.
- The difference between `derive(HashStable)` and
`derive(HashStable_NoContext)` now comes down to the difference
between `synstructure::AddBounds::Generics` and
`synstructure::AddBounds::Fields`, which is basically "vanilla derive"
vs "(near) perfect derive".
- I have improved the comments on `HashStableMode` to better
explaining this subtle difference.
- `rustc_middle/src/ich/impls_syntax.rs` is no longer needed; the
relevant impls can be defined in the crate that defines the relevant
type.
- Occurrences of `for<'a> HashStable<StableHashingContext<'a>>` are
replaced with with `HashStable`, hooray.
- The commit adds a `HashStableContext::hashing_controls` method, which
is no big deal. (It's necessary for `AdtDefData::hash_stable`, which
calls `hashing_controls` and used to have an `hcx` that was a
concrete `StableHashingContext` but now has an `hcx` that is just
`Hcx: HashStableContext`.)
Overall this is a big simplification, removing a lot of confusing
complexity in stable hashing traits.
This puts it in the same crate as the `HashStable` and `ToStableHasher`
traits. This requires introducing three types `RawSpan`, `RawDefId` and
`RawDefPathHash` to work around the fact that `rustc_data_structures`
is upstream of `rustc_span` and so doesn't have access to `Span`,
`DefId`, and `DefPathHash`. This is a bit ugly but is worth it because
moving `HashStableContext` enables big cleanups across many crates in
subsequent commits.
Replace `ShardedHashMap` method `insert` with debug-checked `insert_unique`
Currently every use of `ShardedHashMap::insert` checks that it won't evict an old value due to unique key. I haven't found any issue related to that faulty condition, so I thought of replacing it with `ShardedHashMap::insert_unique` which doesn't check for this condition unless `debug_assertions` are enabled. This might improve the performance.
r? @petrochenkov
Replace the hacky macro with a generic function and a new
`FlatMapInPlaceVec` trait. More verbose but more readable and typical.
LLM disclosure: I asked Claude Code to critique this file and it
suggested the generic function + trait idea. I implemented the idea
entirely by hand.
compiler: update hashbrown to 0.17
See library's rust-lang/rust#155154 for the bug fixes this brings.
This PR also updates `indexmap` in the compiler as a direct dependent.
`HashStable::hash_stable` takes a `&mut Hcx`. In contrast,
`ToStableHashKey::to_stable_hash_key` takes a `&Hcx`. But there are
some places where the latter calls the former, and due to the mismatch a
`clone` call is required to get a mutable `StableHashingContext`.
This commit changes `to_stable_hash_key` to instead take a `&mut Hcx`.
This eliminates the mismatch, the need for the clones, and the need for
the `Clone` impls.
Take first task group for further execution
Continuing from https://github.com/rust-lang/rust/pull/153768#discussion_r2938828363.
I thought that storing a first group of tasks for immediate execution instead of pushing and immediately poping it from rayon's local task queue in par_slice would avoid overwhelming work stealing potentially blocking the original thread. So I've implemented this change.
8 threads benchmarks:
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>baseline~~9</b></th><td colspan="2"><b>new~take-first-group~1</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>hyper</b>:check</td><td align="right">0.1110s</td><td align="right">0.1086s</td><td align="right">💚 -2.13%</td></tr><tr><td>🟣 <b>hyper</b>:check:initial</td><td align="right">0.1314s</td><td align="right">0.1298s</td><td align="right">💚 -1.23%</td></tr><tr><td>🟣 <b>hyper</b>:check:unchanged</td><td align="right">0.0771s</td><td align="right">0.0755s</td><td align="right">💚 -2.14%</td></tr><tr><td>🟣 <b>clap</b>:check</td><td align="right">0.3787s</td><td align="right">0.3757s</td><td align="right"> -0.80%</td></tr><tr><td>🟣 <b>clap</b>:check:initial</td><td align="right">0.4680s</td><td align="right">0.4564s</td><td align="right">💚 -2.48%</td></tr><tr><td>🟣 <b>clap</b>:check:unchanged</td><td align="right">0.2337s</td><td align="right">0.2301s</td><td align="right">💚 -1.52%</td></tr><tr><td>🟣 <b>syn</b>:check</td><td align="right">0.4321s</td><td align="right">0.4265s</td><td align="right">💚 -1.31%</td></tr><tr><td>🟣 <b>syn</b>:check:initial</td><td align="right">0.5586s</td><td align="right">0.5401s</td><td align="right">💚 -3.31%</td></tr><tr><td>🟣 <b>syn</b>:check:unchanged</td><td align="right">0.3434s</td><td align="right">0.3429s</td><td align="right"> -0.14%</td></tr><tr><td>🟣 <b>regex</b>:check</td><td align="right">0.2755s</td><td align="right">0.2661s</td><td align="right">💚 -3.40%</td></tr><tr><td>🟣 <b>regex</b>:check:initial</td><td align="right">0.3350s</td><td align="right">0.3347s</td><td align="right"> -0.11%</td></tr><tr><td>🟣 <b>regex</b>:check:unchanged</td><td align="right">0.1851s</td><td align="right">0.1832s</td><td align="right">💚 -1.01%</td></tr><tr><td>Total</td><td align="right">3.5296s</td><td align="right">3.4695s</td><td align="right">💚 -1.70%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9837s</td><td align="right">💚 -1.63%</td></tr></table>
The `HashStable` and `ToStableHashKey` traits both have a type parameter
that is sometimes called `CTX` and sometimes called `HCX`. (In practice
this type parameter is always instantiated as `StableHashingContext`.)
Similarly, variables with these types are sometimes called `ctx` and
sometimes called `hcx`. This inconsistency has bugged me for some time.
The `HCX`/`hcx` form is more informative (the `H`/`h` indicates what
type of context it is) and it matches other cases like `tcx`, `dcx`,
`icx`.
Also, RFC 430 says that type parameters should have names that are
"concise UpperCamelCase, usually single uppercase letter: T". In this
case `H` feels insufficient, and `Hcx` feels better.
Therefore, this commit changes the code to use `Hcx`/`hcx` everywhere.
Folkert de Vries