This patch also fixes a bug where 'tac' would print a vague error on
some inputs:
$ seq 10000 | ./src/tac-prev > /dev/full
tac-prev: write error
$ seq 10000 | ./src/tac > /dev/full
tac: write error: No space left on device
In this case ferror (stdout) is true, but errno has been set back to
zero by a successful fclose (stdout) call.
* src/tac.c (output): Call write_error() if fwrite fails.
* tests/misc/io-errors.sh: Check that 'tac' prints a detailed write
error.
* NEWS: Mention the improvement.
* tests/misc/io-errors.sh: Support checkout for a specific error
in commands that don't run indefinitely. Currently all the explicitly
listed commands output a specific error and do not need to be tagged.
A good reference for the concepts used here is:
https://mazzo.li/posts/fast-pipes.html
We don't consider huge pages or busy loops here,
but use vmsplice(), and splice() to get significant speedups:
i7-5600U-laptop $ taskset 1 yes | taskset 2 pv > /dev/null
... [4.98GiB/s]
i7-5600U-laptop $ taskset 1 src/yes | taskset 2 pv > /dev/null
... [34.1GiB/s]
IBM,9043-MRX $ taskset 1 yes | taskset 2 pv > /dev/null
... [11.6GiB/s]
IBM,9043-MRX $ taskset 1 src/yes | taskset 2 pv > /dev/null
... [175GiB/s]
Also throughput to file (on BTRFS) was seen to increase significantly.
With a Fedora 43 laptop improving from 690MiB/s to 1.1GiB/s.
* bootstrap.conf: Ensure sys/uio.h is present.
This was an existing transitive dependency.
* m4/jm-macros.m4: Define HAVE_SPLICE appropriately.
We assume vmsplice() is available if splice() is as they
were introduced at the same time to Linux and glibc.
* src/yes.c (repeat_pattern): A new function to efficiently
duplicate a pattern in a buffer with memcpy calls that double in size.
This also makes the setup for the existing write() path more efficient.
(pipe_splice_size): A new function to increase the kernel pipe buffer
if possible, and use an appropriately sized buffer based on that (25%).
(splice_write): A new function to call vmplice() when outputting
to a pipe, and also splice() if outputting to a non-pipe.
* tests/misc/yes.sh: Verify the non-pipe output case,
(main): Adjust to always calling write on the minimal buffer first,
then trying vmsplice(), then falling back to write from bigger buffer.
and the vmsplice() fallback to write() case.
* NEWS: Mention the improvement.
* tests/misc/warning-errors.sh: Skip commands which have been built with
sanitizers, since standard error will not be closed and checked for
errors.
Reported by Bruno Haible.
* tests/misc/warning-errors.sh: Expect 'wc' and 'cksum' to exit
successfully if there is not an optimized 'wc -l' implementation or
CRC32 implementation.
Reported by Bruno Haible.
* NEWS: Mention the improvement.
* src/id.c (print_stuff): Call fflush for each listed user to check for
write errors.
* tests/misc/io-errors.sh: Add an invocation of 'id'.
* NEWS: Mention the improvement.
* src/groups.c (main): Call fflush for each listed user to check for
write errors.
* tests/misc/io-errors.sh: Add an invocation of 'groups'.
* tests/misc/warning-errors.sh: Add a new test to ensure
failure to write warnings is diagnosed in the exit status.
* tests/local.mk: Reference the new test.
* tests/df/no-mtab-status-masked-proc.sh: Tweak unshare check.
* tests/local.mk: Reference new test.
* tests/misc/user.sh: Add new test using unshare -U, to verify
that whoami and logname diagnose failure correctly.
https://github.com/coreutils/coreutils/pull/195
* tests/misc/coreutils.sh: Remove the check for non-empty coreutils.h,
as the existing `print_ver_ coreutils` is sufficient to determine
support for single binary mode.
https://github.com/coreutils/coreutils/pull/189
* tests/misc/read-errors.sh: Use getlimits_ determined EIO error string,
rather than inferring the string from bash's output.
See https://bugs.gnu.org/80353
* NEWS: Mention the improvement.
* src/nl.c: Include mcel.h.
(DEFAULT_SECTION_DELIMITERS): Resize to fit 2 multi-byte characters.
(section_del_len): New variable.
(check_section): Compare against section_del_len instead of 2.
(main): Support multi-byte characters for the -d option.
* tests/nl/multibyte.sh: New file.
* tests/nl/nl.sh: New file, moved from tests/misc/nl.sh.
* tests/local.mk (all_tests): Add the new test. Adjust the existing
tests file name.
* cfg.mk (exclude_file_name_regexp--sc_space_tab): Adjust Adjust the
existing tests file name.
This is a more standard mechanism to disable markup.
* src/system.h (oputs_): Logic change to honor TERM=dumb,
rather than HELP_NO_MARKUP=something.
* doc/coreutils.texi: Adjust the description for --help.
* man/local.mk: Ensure TERM is set to something,
so that man pages have links included.
* man/viewman: Just honor users $TERM.
* tests/misc/getopt_vs_usage.sh: Remove env var complication,
as TERM is unset automatically.
* tests/misc/usage_vs_refs.sh: Likewise.
* NEWS: Adjust the change in behavior note.
* tests/misc/usage_vs_refs.sh: A new test to correlation options
listed in --help, and anchors referenced in the texinfo (html) manual.
* tests/local.mk: Add the new test.
* src/ls.c (oputs): A new function that wraps puts(),
but also highlights the --option-text portion, and
adds links to the appropriate part of the online manual.
(usage): Call oputs() rather than puts().
* doc/coreutils.texi (--help): Document new HELP_NO_MARKUP env var,
which can be used in the edge case one wants to suppress ansi escapes.
* tests/misc/getopt_vs_usage.sh: Use HELP_NO_MARKUP to ensure the
test continues to pass.
* tests/misc/sync.sh: Check 'sync fifo' with no arguments, -d, and -f
separately. Only perform the check for -f if RUN_EXPENSIVE_TESTS=yes
since it may be expensive.
* tests/misc/coreutils.sh: Be more flexibile by using `command -v`
to identify the binary to be executed, rather than linking
directly to the built binary.
Setup
$ ln -nsf src/coreutils foo
Before
$ ./foo; echo $?
foo: unknown program ‘foo’
Try './foo --help' for more information.
1
./foo --version; echo $?
coreutils (GNU coreutils) 9.9.172-01993
0
After
$ ./foo; echo $?
coreutils: unknown program 'foo'
1
$ ./foo --version; echo $?
coreutils: unknown program 'foo'
1
* src/coreutils.c (main): Don't process options if
we don't know they're intended for the multi-call binary.
Otherwise `foo --version` would return true, even though
foo was symlinked to the multi-call binary, but not supported.
* tests/misc/coreutils.sh: Add test cases.
* NEWS: Mention the change in behavior.
* tests/misc/read-errors.sh: Ensure each utility calls a
read system call as requested. Not doing so (say by avoiding
reading from a directory) is a layering violation.
Also ensure we diagnose the particular error encountered.
* src/paste.c (collapse_escapes): This is the central --delimiters
parsing function, so adjust to handle multi-byte chars with
mcel_scanz(). Populate a delimiters length array to support
characters of differing lengths.
(paste_serial): Use the delimiters length array to output
the appropriate delimiter.
(paste_parallel): Likewise.
* tests/paste/multi-byte.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
* src/cksum.c (output_file, digest_check): Check if standard output has
it's error flag set after printing.
* tests/misc/write-errors.sh: Add a test case that would previously run
forever.
* NEWS: Mention the improvement. Reorder alphabetically.
oech3