rustc_target/asm: add LoongArch LSX/LASX inline asm register support
Add support for LoongArch LSX and LASX registers in inline assembly by introducing the `vreg` and `xreg` register classes, along with their associated vector types and operand modifiers. The new register classes are gated behind the `asm_experimental_reg` feature. Also model the overlap between FPU, LSX, and LASX registers so register conflict checking works correctly for aliased registers.
Rename `align` to `default_align` on `Scalar` and `Primitive`
To emphasize that just because you see a `Scalar(I32)` that doesn't really tell you anything about the alignment it has -- one should be looking at the type (well, the place) for that.
(The `size` method doesn't really have that problem -- i32 is always 4 bytes, no matter the context -- so is left untouched.)
No actual layout or behaviour changes in *this* PR. Just the rename and some comments.
r? @workingjubilee
Move per-function debuginfo methods from codegen to builder methods
In cg_clif the debuginfo for the function we are currently codegening is stored in `FunctionDebugContext`, which is stored in the `FunctionCx`, not in the per-cgu `DebugContext`.
Introduce aarch64-unknown-linux-pauthtest target
This target enables Pointer Authentication Code (PAC) support in Rust on AArch64
ELF-based Linux systems. It uses the `aarch64-unknown-linux-pauthtest` LLVM
target and a pointer-authentication-enabled sysroot with a custom musl as a
reference libc implementation. Dynamic linking is required, with a dynamic
linker acting as the ELF interpreter that can resolve pauth relocations and
enforce pointer authentication constraints.
### Supported features include:
* authentication of signed function pointers for extern "C" calls (corresponds
to LLVM's `-fptrauth-calls`)
* signing of return addresses before spilling to the stack and authentication
after restoring for non-leaf functions (corresponds to `-fptrauth-returns`)
* trapping on authentication failure when the FPAC feature is not present
(corresponds to `-fptrauth-auth-traps`)
* signing of init/fini array entries using the LLVM-defined pointer
authentication scheme (corresponds to `-fptrauth-init-fini` and
`-fptrauth-init-fini-address-discrimination`)
* non-ABI-affecting indirect control-flow hardening features as implemented in
LLVM (corresponds to `-faarch64-jump-table-hardening` and
`-fptrauth-indirect-gotos`)
* signed ELF GOT entries (gated behind `-Z ptrauth-elf-got`, off by default)
Existing compiler support, such as enabling branch authentication instructions
(i.e.: `-Z branch-protection`) provide limited functionality, mainly signing
return addresses (`pac-ret`). The new target goes further by enabling ABI-level
pointer authentication support.
This target does not define a new ABI; it builds on the existing C/C++ language
ABI with pointer authentication support added. However, different authentication
features, encoded in the signing schema, are not ABI-compatible with one
another.
### Useful links:
* Earlier PR: https://github.com/rust-lang/rust/pull/154759
* Part of: https://github.com/rust-lang/rust/issues/148640
* Project goal: https://rust-lang.github.io/rust-project-goals/2026/aarch64_pointer_authentication_pauthtest.html
* Clang pointer authentication documentation:
https://clang.llvm.org/docs/PointerAuthentication.html
* LLVM pointer authentication documentation:
https://llvm.org/docs/PointerAuth.html
* PAuth ABI Extension to ELF for the AArch64 architecture:
https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst
### Tier 3 check list
> - A tier 3 target must have a designated developer or developers (the "target
> maintainers") on record to be CCed when issues arise regarding the target.
> (The mechanism to track and CC such developers may evolve over time.)
I pledge to do my best maintaining it.
> - Targets must use naming consistent with any existing targets; for instance, a
> target for the same CPU or OS as an existing Rust target should use the same
> name for that CPU or OS. Targets should normally use the same names and
> naming conventions as used elsewhere in the broader ecosystem beyond Rust
> (such as in other toolchains), unless they have a very good reason to
> diverge. Changing the name of a target can be highly disruptive, especially
> once the target reaches a higher tier, so getting the name right is important
> even for a tier 3 target.
The name chosen for the target is `aarch64-unknown-linux-pauthtest` which
mirrors the [LLVM target naming](https://github.com/llvm/llvm-project/blob/main/llvm/unittests/TargetParser/TripleTest.cpp#L1407).
> - Target names should not introduce undue confusion or ambiguity unless
> absolutely necessary to maintain ecosystem compatibility. For example, if
> the name of the target makes people extremely likely to form incorrect
> beliefs about what it targets, the name should be changed or augmented to
> disambiguate it.
There should be no confusion, the name follows naming convention and is
descriptive.
> - If possible, use only letters, numbers, dashes and underscores for the name.
> Periods (`.`) are known to cause issues in Cargo.
Letters, numbers and dashes only.
> - Tier 3 targets may have unusual requirements to build or use, but must not
> create legal issues or impose onerous legal terms for the Rust project or for
> Rust developers or users.
The target requires system `clang` and `lld` available as well as custom libc
([musl](https://github.com/access-softek/musl) based) and sysroot, provided [through the build scripts](https://github.com/access-softek/pauth-toolchain-build-scripts/tree/master).
> - The target must not introduce license incompatibilities.
There are no license implications.
> - Anything added to the Rust repository must be under the standard Rust
> license (`MIT OR Apache-2.0`).
Understood.
> - The target must not cause the Rust tools or libraries built for any other
> host (even when supporting cross-compilation to the target) to depend
> on any new dependency less permissive than the Rust licensing policy. This
> applies whether the dependency is a Rust crate that would require adding
> new license exceptions (as specified by the `tidy` tool in the
> rust-lang/rust repository), or whether the dependency is a native library
> or binary. In other words, the introduction of the target must not cause a
> user installing or running a version of Rust or the Rust tools to be
> subject to any new license requirements.
There are no new dependencies or requirements.
> - Compiling, linking, and emitting functional binaries, libraries, or other
> code for the target (whether hosted on the target itself or cross-compiling
> from another target) must not depend on proprietary (non-FOSS) libraries.
> Host tools built for the target itself may depend on the ordinary runtime
> libraries supplied by the platform and commonly used by other applications
> built for the target, but those libraries must not be required for code
> generation for the target; cross-compilation to the target must not require
> such libraries at all. For instance, `rustc` built for the target may
> depend on a common proprietary C runtime library or console output library,
> but must not depend on a proprietary code generation library or code
> optimization library. Rust's license permits such combinations, but the
> Rust project has no interest in maintaining such combinations within the
> scope of Rust itself, even at tier 3.
The target only relies on open source tools.
> - "onerous" here is an intentionally subjective term. At a minimum, "onerous"
> legal/licensing terms include but are *not* limited to: non-disclosure
> requirements, non-compete requirements, contributor license agreements
> (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms,
> requirements conditional on the employer or employment of any particular
> Rust developers, revocable terms, any requirements that create liability
> for the Rust project or its developers or users, or any requirements that
> adversely affect the livelihood or prospects of the Rust project or its
> developers or users.
No such terms present.
> - Neither this policy nor any decisions made regarding targets shall create any
> binding agreement or estoppel by any party. If any member of an approving
> Rust team serves as one of the maintainers of a target, or has any legal or
> employment requirement (explicit or implicit) that might affect their
> decisions regarding a target, they must recuse themselves from any approval
> decisions regarding the target's tier status, though they may otherwise
> participate in discussions.
Understood.
> - This requirement does not prevent part or all of this policy from being
> cited in an explicit contract or work agreement (e.g. to implement or
> maintain support for a target). This requirement exists to ensure that a
> developer or team responsible for reviewing and approving a target does not
> face any legal threats or obligations that would prevent them from freely
> exercising their judgment in such approval, even if such judgment involves
> subjective matters or goes beyond the letter of these requirements.
Understood.
> - Tier 3 targets should attempt to implement as much of the standard libraries
> as possible and appropriate (`core` for most targets, `alloc` for targets
> that can support dynamic memory allocation, `std` for targets with an
> operating system or equivalent layer of system-provided functionality), but
> may leave some code unimplemented (either unavailable or stubbed out as
> appropriate), whether because the target makes it impossible to implement or
> challenging to implement. The authors of pull requests are not obligated to
> avoid calling any portions of the standard library on the basis of a tier 3
> target not implementing those portions.
`aarch64-unknown-linux-pauthtest target` has std library support, moreover all
`library` tests pass for the target.
> - The target must provide documentation for the Rust community explaining how
> to build for the target, using cross-compilation if possible. If the target
> supports running binaries, or running tests (even if they do not pass), the
> documentation must explain how to run such binaries or tests for the target,
> using emulation if possible or dedicated hardware if necessary.
Platform support document covers building instructions.
> - Tier 3 targets must not impose burden on the authors of pull requests, or
> other developers in the community, to maintain the target. In particular,
> do not post comments (automated or manual) on a PR that derail or suggest a
> block on the PR based on a tier 3 target. Do not send automated messages or
> notifications (via any medium, including via `@`) to a PR author or others
> involved with a PR regarding a tier 3 target, unless they have opted into
> such messages.
Understood.
> - Backlinks such as those generated by the issue/PR tracker when linking to
> an issue or PR are not considered a violation of this policy, within
> reason. However, such messages (even on a separate repository) must not
> generate notifications to anyone involved with a PR who has not requested
> such notifications.
Understood.
> - Patches adding or updating tier 3 targets must not break any existing tier 2
> or tier 1 target, and must not knowingly break another tier 3 target without
> approval of either the compiler team or the maintainers of the other tier 3
> target.
Understood.
> - In particular, this may come up when working on closely related targets,
> such as variations of the same architecture with different features. Avoid
> introducing unconditional uses of features that another variation of the
> target may not have; use conditional compilation or runtime detection, as
> appropriate, to let each target run code supported by that target.
Understood.
> - Tier 3 targets must be able to produce assembly using at least one of
> rustc's supported backends from any host target. (Having support in a fork
> of the backend is not sufficient, it must be upstream.)
It is expected that the target should be able to compile binaries on any systems
that are capable of compiling `aarch64` code.
To emphasize that just because you see a `Scalar(I32)` that doesn't really tell you anything about the alignment it has -- one should be looking at the type (well, the place) for that.
No actual layout or behaviour changes in *this* PR.
cg_LLVM: Stop needing an alloca for volatile loads
This ended up also being reimplementing it to not use `load` of the `llvm_type`, since without doing that everything blew up horribly. cc the zulip conversation [#t-opsem > Defining volatile splitting @ 💬](https://rust-lang.zulipchat.com/#narrow/channel/136281-t-opsem/topic/Defining.20volatile.20splitting/near/597451615).
And while I'm here, improve the tests to check that the unaligned ones are actually unaligned, since `unaligned_volatile_load::<u8>` doesn't actually test anything.
r? @nikic
cc @RalfJung
MCP tracking issue rust-lang/rust#153250
Add support for LoongArch LSX and LASX registers in inline assembly by
introducing the `vreg` and `xreg` register classes, along with their
associated vector types and operand modifiers. The new register classes
are gated behind the `asm_experimental_reg` feature. Also model the overlap
between FPU, LSX, and LASX registers so register conflict checking works
correctly for aliased registers.
And while I'm here, improve the tests to check that the unaligned ones are actually unaligned, since `unaligned_volatile_load::<u8>` doesn't actually test anything.
When emitting a `memcpy` for a scalable vector, the size computed by
rustc (`num_vectors * element_count * element_ty`), since
rust-lang/rust#157915, needs to be multiplied by `vscale`.
Add support for inline assembly for the amdgpu backend (the
amdgcn-amd-amdhsa target).
Add register classes for `vgpr` (vector general purpose register) and
`sgpr` (scalar general purpose register).
The LLVM backend supports two more classes, `reg`, which is either VGPR
or SGPR, up to the compiler to decide. As instructions often rely on a
register being either a VGPR or SGPR for the assembly to be valid, reg
doesn’t seem that useful (I struggled to write correct tests for it), so
I didn’t end up adding it.
The fourth register class is AGPRs, which only exist on some hardware
versions (not the consumer ones) and they have restricted ways to write
and read from them, which makes it hard to write a Rust variable into
them. They could be used inside assembly blocks, but I didn’t add them
as Rust register class.
There are a few change affecting general inline assembly code, that is
`InlineAsmReg::name()` now returns a `Cow` instead of a `&'static str`.
Because amdgpu has many registers, 256 VGPRs plus combinations of 2 or 4
VGPRs, and I didn’t want to list hundreds of static strings, the amdgpu
reg stores the register number(s) and a non-static String is generated
at runtime for the register name.
Similar for register classes and supported_types.
Vectors of 64-bit types are supported by the LLVM backend, but omitted
here to make the code simpler. There is currently no systematic support
in LLVM of which vectors of 64-bit types are supported. Also, they are
likely seldomly unused, vectors of 16- and 32-bit types are important.
asm! support for the Xtensa architecture
This implements the asm! support for Xtensa. We've been using this code for a few years in [our fork](https://github.com/esp-rs/rust) and it's been working well. I finally found some time to clean it up a bit and start the upstreaming process. This should be one of the final PRs for Xtensa support on the Rust side (minus bug fixes of course). After this, we're mostly just waiting on the LLVM upstreaming which is going well. This PR doesn't cover all possible asm options for Xtensa, but the base ISA plus a few extras that are used in Espressif chips.
r? Amanieu
This is an UnordMap internally. Iteration order for the work product map
should not matter aside from the place it is serialized where sorting by
WorkProductId is sufficient.
Use a `ArrayVec` in `CastTarget`
This commit switches a fixed-size list of `[Option<Reg>; 8]` to instead
holding `ArrayVec<Reg, 8>` in the `CastTarget` type used when
calculating ABIs. This is inspired by [discussion on Zulip][link] where
I'm hoping to in the near future extend the usage of this to possibly
beyond 8 elements for a new WebAssembly ABI taking advantage of
multi-value. For now though this mostly just switches to
array/slice-like idioms of accessors rather than dealing with
`Option<Reg>` as the unit.
[link]: https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Using.20.60ArgAbi.3A.3Amake_direct_deprecated.60/with/598607139
This commit switches a fixed-size list of `[Option<Reg>; 8]` to instead
holding `ArrayVec<Reg, 8>` in the `CastTarget` type used when
calculating ABIs. This is inspired by [discussion on Zulip][link] where
I'm hoping to in the near future extend the usage of this to possibly
beyond 8 elements for a new WebAssembly ABI taking advantage of
multi-value. For now though this mostly just switches to
array/slice-like idioms of accessors rather than dealing with
`Option<Reg>` as the unit.
[link]: https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Using.20.60ArgAbi.3A.3Amake_direct_deprecated.60/with/598607139
Stop needing an alloca for `catch_unwind`
Turns out these were all making `OperandValue::Immediate`s already -- the intrinsic always returns a primitive scalar -- so pretty easy to handle.
cc https://github.com/rust-lang/rust/issues/153250https://github.com/rust-lang/compiler-team/issues/970
While I was looking at it, I also "rustified" the intrinsic signature a bit: returning a `bool` and taking a generic pointee and `unsafe fn`s cleans up the call in `std` a bit without making the implementation in the backend any harder.
r? dianqk
Turns out these were all making `OperandValue::Immediate`s already -- the intrinsic always returns a primitive scalar -- so pretty easy to handle.
While I was looking at it, I also "rustified" the intrinsic signature a bit: returning a `bool` and taking a generic pointee and `unsafe fn`s cleans up the call in `std` a bit without making the implementation in the backend any harder.
Add Swift function call ABI
Adds an unstable `extern "Swift"` ABI behind the `abi_swift` feature gate, mapping to LLVM's `swiftcc` calling convention. This is only allowed (a) for `is_darwin_like` targets, since the [ABI is only stable for those platforms](https://www.swift.org/blog/abi-stability-and-more/) and (b) with the LLVM backend, since the other backends don't support it.
Current approaches to interoperability with Swift lower to Objective-C (or require a Swift stub exposing a C ABI), but that is an optional mapping on the Swift side that some newer Apple frameworks omit. It would be great to be able to more directly/natively be able to call into Swift code directly via its stable API (on Apple platforms at least).
Reimplements rust-lang/rust#64582 on top of current main. The main objection to the previous PR seemed to be that it needed an RFC, but there was pushback (which seems sensible to me) that an RFC could be deferred until stabilization.
I think this needs a tracking issue? Would be happy to write one up if/when there is a consensus that this will be merged.
Adds an unstable `extern "Swift"` ABI behind the `abi_swift` feature
gate, mapping to LLVM's `swiftcc` calling convention. Cranelift and
GCC backends fall back to the platform default since they have no
equivalent.
change the type of the argument of `drop_in_place` lang item to `&mut _`
We used to special case `core::ptr::drop_in_place` when computing LLVM argument attributes with this hack:
https://github.com/rust-lang/rust/blob/db5e2dc248fe5bb26f70d7baec46a3bca9fa3e1d/compiler/rustc_ty_utils/src/abi.rs#L383-L392
This is because even though `drop_in_place` takes a `*mut T` it is semantically a `&mut T` (remember how `&mut Self` is passed to `Drop::drop`). This is apparently relevant for perf.
This PR replaces this hack with a simpler solution -- it makes `drop_in_place` a thin wrapper around newly added `core::ptr::drop_glue`, which is the actual lang item and takes a `&mut T`:
https://github.com/rust-lang/rust/blob/d2563d5003bbecff1efc40c1f5673ceec603825b/library/core/src/ptr/mod.rs#L810-L833
------
The rest of the PR is blessing tests and cleaning up things which are not necessary after this change.
One thing that is a bit awkward is that now that `drop_glue` is the actual lang item, a lot of the comments referring to `drop_in_place` are outdated. Should I try fixing that?
I've also changed `async_drop_in_place` to take a `&mut T`, and it simplified the code handling it a bit. (since it's unstable we don't need to introduce a wrapper)
-------
cc @RalfJung
Closes https://github.com/rust-lang/rust/issues/154274
Add rlib digest to identify Rust object files
This adds a metadata entry to `rlib` archives that lists which members are Rust object files instead of relying on the filename heuristic in `looks_like_rust_object.file`. I also added a fallback to the old behavior for `rlibs` built by older compilers.
Part of https://github.com/rust-lang/rust/issues/138243.
Jacob Pratt