A handful of changes to get the regressed behavior tests running again.
- Replace `decorateBlockOffsets` with a recursive function `decorateLayout`
that walks arrays, vectors, structs, unions, optionals, and error unions,
emitting `ArrayStride` and member `Offset` decorations at every
level. Previously we weren't handling nested types.
- Restrict `Block` decoration to struct types with
`uniform`, `push_constant`, `storage_buffer` storage classes.
Previously we decorated through every pointer, contaminating the cached struct
type so the same shape used as a stack local also picked up `Block`.
- Lower `ptr_slice_ptr_ptr` and `ptr_slice_len_ptr`
- No longer emit a redundant `**T` typed `OpVariable` for function parameters.
Logical addressing also forbids such variables.
- Eliminate dead code from invocation globals unreachable from any entry point.
Reverting the workaround in `lib/std/start.zig`.
Execution modes (e.g. `LocalSize`, `OriginUpperLeft`) were previously set
via `gpu.executionMode()`, which used inline assembly to emit `OpExecutionMode`.
The SPIR-V assembler now rejects this instruction and retrieves execution mode information
from function cc, deleting `gpu.executionMode()` entirely.
Two new `spirv_task` and `spirv_mesh` calling conventions are also added
and `PackedCallingConvention.unpack()` now takes a trailing data slice.
Once the switch condition is been shifted up so the minimum value is 0,
it makes sense to treat its type as the unsigned version of its original
type.
Fixes#35651
Closes#35240Fixes#35238Fixes#35259
Supported types are:
- `OpTypeSampler`
- `OpTypeImage`
- `OpTypeSampledImage`
- `OpTypeRuntimeArray` with indexing and `.len` field
The SPIR-V backend is bit-rotted so behavior tests no longer pass (compiler crashes).
However I've verified the new added tests are passing.
Some of these changes are kind of hacky, and arguably indicate that we
should deal with field alignment in CBE a bit differently, but for now
these changes do manage to fix *some* bugs.
This solves the problems demonstrated in #35552. However, when trying to
actually build that example, GCC emits these errors from the generated C
code in `compiler_rt.c`:
```
/tmp/ccgboCZq.s: Assembler messages:
/tmp/ccgboCZq.s:29321: Error: bad expression -- `bl #__udivmodsi4'
/tmp/ccgboCZq.s:29344: Error: bad expression -- `bl #__divmodsi4'
/tmp/ccgboCZq.s:30200: Error: bad expression -- `bl #__udivmoddi4'
/tmp/ccgboCZq.s:30263: Error: bad expression -- `bl #__divmoddi4'
```
This looks to be an unrelated bug, which I am not going to attempt to
tackle for now. The static assertions passing indicates that we're now
emitting at least semi-reasonable type definitions in this example, so
the bug actually tracked by #35552 is fixed.
Resolves: https://codeberg.org/ziglang/zig/issues/35552
This started as a small diff to `Elf2` to make it support the LLVM
backend, but as I was writing it, I felt that I was just writing logic
shared by every linker implementation. The object file emitted by the
LLVM backend is a totally normal link input as far as the linker
implementations are concerned---so, why not treat it as one?
The `zcu_object_basename` field is removed from `link.File`, instead
moved to `llvm.Object`. Logic in `link.Queue` avoids calling `prelink`
when using the LLVM backend, because it knows we will receive another
link input later. In `Compilation.flush`, after LLVM emits the ZCU
object, we call `link.runPrelinkTask` to process that input, and then
perform the deferred `prelink` call. This means that linker
implementations which integrate with prelink don't need to be aware of
LLVM whatsoever! (Aside from perhaps checking `use_llvm` to know if they
are going to receive any ZCU tasks.)
The `MachO` and `Lld` linker implementations still have specific
handling for the ZCU object file from LLVM, because they do not
currently integrate with `prelink`---but the `Coff`, `Elf`, `Wasm`, and
`Elf2` implementations no longer need to handle this case specially.
Note that the self-hosted linkers currently do not support incremental
compilation with the LLVM backend---this is an existing issue, but I
thought I'd mention it here because I wasn't previously aware of this
bug. That is tracked by https://codeberg.org/ziglang/zig/issues/32053.
All public codegen and linker APIs now use the following error set:
Allocator.Error || Io.Cancelable || error{AlreadyReported}
This is defined as `link.Error` and aliased as `codegen.Error`.
The compiler "backend" (including both codegen and linker) has a fairly
limited set of failure modes. Most of them are as follows:
* Bad inline assembly (codegen)
* Output file I/O error (link)
* Symbol with no definition or multiple definitions (link)
* Relocation error, e.g. overflow (link)
* Unimplemented/unsupported feature (codegen/link)
* `error.OutOfMemory` (codegen/link)
* `error.Canceled` (codegen/link)
The last two cases are special, because they are possible across most of
the compiler codebase and have fixed code paths for handling---e.g.
`error.OutOfMemory` almost always calls `Compilation.setAllocFailure`,
and `error.Canceled` should typically propagate all the way up the call
stack.
However, all of the other cases should follow the same general pattern:
the codegen/linker implementation should mark an error on `Compilation`
somehow (either through `link_diags` or `failed_codegen`), and return
`error.AlreadyReported`. This error code indicates that an operation
failed, but that the caller does not need to take action to recover,
because the failure has already been recorded in a way which will be
expressed to the user. For instance, the codegen error set used to
contain `error.Overflow`, but this case should have already been
reported to the user, because implementation-agnostic code cannot know
how best to express the failure to the user.
The `error.AlreadyReported` error code encompasses what was previously
represented by multiple errors, including `error.CodegenFail`,
`error.LinkFailure`, and `error.AnalysisFail`. It is not necessary to
differentiate these cases. (Not to be confused with the usage of
`error.AnalysisFail` in the compiler *frontend*, i.e. `Sema`, which is
unchanged by this diff.)
Because the backend error set is now very small, it is easy to
exhaustively `switch` on, and also many functions can be given concrete
error sets. There are no longer different error sets for different
operations. (As an exception, I have not tackled `link.File.open` in
this diff, which has a big inferred error set where I think most errors
are reported to the user with an `else => |e|` case.)
I have also changed how `link.MappedFile`, our abstraction for handling
the awkward "moving things around" aspect of incremental linking, does
error handling. The public API of this abstraction now uses the
following small error set:
Allocator.Error || Io.Cancelable || error{MappedFileIo}
The first two cases are self-explanatory. Then, `error.MappedFileIo` is
a catch-all error code encompassing that an error was encountered when
accessing the underlying `Io.File`. This error may have occurred when
attempting to flush the file to disk, or to change its size, etc. In
this case, the *specific* error---which was actually returned from the
`Io.File` API---is available in the `MappedFile.io_err.?` field. Linker
implementations which use `MappedFile` (currently `Elf2` and `Coff`)
should eventually handle `error.MappedFileIo` by reporting an error in
`Compilation.link_diags`, including that specific I/O error in the error
message.
The rationale here is essentially that error codes returned from
functions exist for control flow purposes, and a linker implementation
should not care about the distinction between file I/O error codes (e.g.
`error.DiskQuota` vs `error.InputOutput`). The only reason it needs the
concrete error is to expose it to the user. Therefore, by wrapping these
failure modes under `error.MappedFileIo`, we keep the error set actually
used by the linker implementation as small as possible, which encourages
avoiding patterns like `else => |e| diags.fail(...)` (which is
potentially dangerous since it may prevent `error.OutOfMemory` or
`error.Canceled` from propagating correctly). It additionally helps
linker implementations maintain more precise error messages---for
instance, if `Elf2` encounters `error.NoSpaceLeft` writing to the output
file, the error will now read "failed to write output file: NoSpaceLeft"
instead of something more generic like "prelink failed: NoSpaceLeft".
Finally, while working on these refactors, I was able to eliminate those
pesky `src_loc: Zcu.LazySrcLoc` parameters from the codegen and link
logic. These parameters did not make sense, because at this point in the
compiler pipeline, we do not have precise source location
information---so these parameters were always passed as just the
location of the function declaration, or as some placeholder
(`.unneeded` or the top of `lib/std/std.zig`) if there wasn't an
appropriate function definition. The only logic which actually *uses*
these source locations is codegen implementations' `fail` functions. Per
my earlier list of failure modes, those functions are called in two main
cases:
* Bad inline assembly
* Unimplemented/unsupported feature
The first case should be moved to the compiler frontend (tracked by
https://github.com/ziglang/zig/issues/10761), while the second case is
essentially a compiler TODO so it is permissible for it to have
imprecise error reporting. Therefore, codegen implementations' `fail`
functions now just call `navSrcLoc` themselves when necessary, which
(once we make improvements to inline assembly) will only happen when
there is a deficiency in the codegen implementation.
I was careful to make `Elf2` and `Coff` error reporting work well in
this diff, by using no `else` case other than `else => |e| return e`
when `switch`ing on errors and by always handling `error.MappedFileIo`
by unwrapping `MappedFile.io_err.?`. I also added concrete error sets to
almost every function in `Elf2`. (That was, um, actually why I started
this diff, because it was annoying me that I wouldn't get as many
compile errors at once...)
Sorry for the mega-commit, this diff got a little out of control.
The main thing here is a complete rework of how Elf2 handles the symbol
table. I messed around with the design for a while and landed on
something which is fairly memory-efficient (in particular the overhead
for STB_LOCAL symbols is as low as possible) and fulfils some of the
more awkward constraints of the ELF format. The main such constraint is
that all STB_LOCAL symbols in a symbol table are required to appear
before any STB_GLOBAL/STB_WEAK symbols. This is further complicated by
the fact that when producing a DSO, symbols with STV_HIDDEN or
STV_INTERNAL visibility are required to have STB_LOCAL binding in the
symbol table, even though they are global symbols from the perspective
of the link editor. Plus, when combining multiple symbols with the same
name, the resulting visibility is the strictest of all of the inputs, so
it is possible at any point in compilation to discover an extern/export
symbol which forces an existing STB_GLOBAL symbol to become STB_LOCAL
and therefore requires it to move to an earlier symtab index. Dealing
with all of this was quite awkward.
But I got there! I also implemented a lot of features in the process. I
don't remember everything perfectly, but here's a vague list:
* Multiple definitions of and/or unresolved references to symbols are
now combined correctly in all cases
* `.bss` sections from inputs are correctly lowered (we don't actually
emit a `.bss` section of our own yet, but I was able to put that data
into the `.data` section so that the functionality is correct)
* Relocations in link inputs are now always processed (previously they
would be silently ignored in most cases)
* Linker errors are triggered if a supported input section has a
relocation which targets an unsupported input section (previously
the unsupported section's symbol was dropped and associated
relocations would be silently ignored)
* When linking a static executable, an error is emitted if a required
symbol (i.e. an undefined reference with strong linkage) was never
defined
* Duplicate symbol errors now work correctly
* When emitting a relocatable, the offsets of relocation entries are now
correct (previously the offsets written were relative to a symbol
rather than a section, meaning that e.g. almost all text relocations
were just in a single function)
The changes in all of the other linkers and codegen backends are some
added type-safety at the codegen-linker API boundary. There are now
distinct `u32`-backed types for identifying an "atom" (the thing we're
codegenning) and a "symbol" (the thing which a relocation targets).
Linker implementations can use a couple of private helper functions to
convert between this implementation-agnostic type and their specific
type; for instance, `Elf2` can convert between a `Symbol.Id` and a
`link.File.SymbolId` with `Symbol.Id.fromTypeErased` and
`Symbol.Id.toTypeErased`. I didn't implement this nicely for any other
linker, so right now there's a lot of `@enumFromInt`/`@intFromEnum`
sprinkled all over the place, particularly with the legacy ELF and
Mach-O linkers.
I tested that I could still perform incremental updates to the Zig
compiler using this commit. In terms of the new behaviors, the most
interesting stuff is symbol and relocation resolution, so I ran a few
tests involving building a "Hello World" binary in various different
ways:
* `build-exe` correctly succeeds
* `build-exe -fno-compiler-rt` correctly reports undefined symbols
* `build-obj` linked with `build-exe` correctly succeeds
* `build-obj` linked with `build-exe -fno-compiler-rt` correctly reports
undefined symbols
* `build-obj -fcompiler-rt` linked with `build-exe -fno-compiler-rt`
correctly succeeds
* `build-obj -fcompiler-rt` linked with `build-exe` correctly succeeds
(the compiler-rt symbols are weak so the global symbols are
arbitrarily resolved to one of the two implementations)
I also manually verified with `readelf` that symbol tables were always
ordered correctly (before this PR, `readelf -s` would usually emit
warnings about incorrectly-ordered symtabs!), and verified that various
visibility attributes worked as expected.
No actual test coverage is added due to the current lack of a useful
linker test harness. Once a good test harness is available I will be
willing to write some tests.
This PR merges the functionality of the `getLastOrNull` method into `getLast`, which improves consistency as its
based on methods like `front`, `back`, and `peek` in the `Deque` and `PriorityQueue` containers.
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/32008
Reviewed-by: Andrew Kelley <andrew@ziglang.org>
As of LLVM 22, most backends can (finally) handle legalization of these types as
necessary, so we don't need to do it ourselves anymore.
closes https://github.com/ziglang/zig/issues/23674
Alex Rønne Petersen