Compare commits

160 Commits

Author SHA1 Message Date
renovate[bot] e4462b038e chore(deps): update rust crate divan to v5.0.1 2026-06-27 11:15:35 +02:00
renovate[bot] 30bbc169ac chore(deps): update actions/cache action to v6 2026-06-24 09:11:15 +02:00
renovate[bot] ff6b8d46a7 chore(deps): update rust crate divan to v5 2026-06-24 07:53:41 +02:00
oech3 dc9ca179f3 cmp: use .map_err 2026-06-10 15:12:36 +02:00
oech3 f29e96cdba cmp.rs: simplify by .ok_or 2026-06-10 10:21:25 +02:00
renovate[bot] a46dae68b1 chore(deps): update rust crate regex to v1.12.4 2026-06-10 07:20:17 +02:00
renovate[bot] 1a8d7f96a6 chore(deps): update codecov/codecov-action action to v7 (#240)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-06-07 09:08:29 +02:00
renovate[bot] 53599ccd40 chore(deps): update rust crate libfuzzer-sys to v0.4.13 2026-06-05 07:24:05 +02:00
renovate[bot] 9bc64f03ed chore(deps): update rust crate chrono to v0.4.45 2026-06-05 07:17:21 +02:00
Marc Herbert d266f9b90e CI: remove echo "/opt/homebrew/opt/gpatch/libexec/gnubin" >> "$GITHUB_PATH" (#234)
This is not needed now that tests automatically look for `gpatch` on mac
since commit 1254f146f8 ("tests: validate "patch" and "ed" commands
once, print meaningful messages (#226)")

The fewer PATH changes, the better.

Signed-off-by: Marc Herbert <Marc.Herbert@gmail.com>
2026-06-03 14:13:10 +02:00
Marc Herbert ec3428b48f tests: fix "gpatch --version" error message on mac (#235)
When we fail to find `gpatch`, don't say that we failed to find `patch`.

Cosmetic fix to commit 1254f146f8 ("tests: validate "patch" and "ed"
commands once, print meaningful messages (#226)")

This is an extremely minor fix because the error message already printed
"gpatch validation failed, no such file or directory" even before this
commit.

Signed-off-by: Marc Herbert <Marc.Herbert@gmail.com>
2026-06-03 14:11:51 +02:00
oech3 58da229c09 support prefixed names (#231) 2026-06-03 14:11:29 +02:00
Sylvestre Ledru 250f935efe Add CONTRIBUTING.md pointing to the review guidelines 2026-05-30 10:07:54 +02:00
Marc Herbert 1254f146f8 tests: validate "patch" and "ed" commands once, print meaningful messages (#226)
macOS' /usr/bin/patch and GNU patch have very subtle incompatibilities
that cause only some "more advanced" tests to fail in obscure and very
time-consuming ways - while other tests pass. In some cases (depending
on test threads racing), the lack of newlines in some test data even
causes the whole test suite to stall.

This fix runs `patch -version` (only once), makes sure the output starts
with "GNU patch" and shows a meaningful assert message when not. It also
looks for `gpatch` instead of `patch` on macOS and shows a meaningful
assert message if either is missing.

Fixes: #225

This also provides faster and better feedback when `ed` is missing (see
#39) and implements a portable and basic check.

Last but not least, this new code is generic enough to support the
validation of any other test dependency in the future.
2026-05-24 17:08:52 +02:00
renovate[bot] c1943c5abb chore(deps): update rust crate divan to v4.7.0 2026-05-23 07:09:07 +02:00
Gunter Schmidt d33aca1fff cmp Feat: change data type for 'bytes' limit and 'ignore initial' to u64 (#183)
* feat: u64 for --bytes and --ignore-initial

fix: bumped up tempfile to "3.26.0"

The variables for --bytes, --ignore-initial and line count where size 'usize',
thus limiting the readable bytes on 32-bit systems.
GNU cmp is compiled with LFS (Large File Support) and allows i64 values.

This is now all u64, which works also on 32-bit systems with Rust.
There is no reason to implement a 32-bit barrier for 32 bit machines.

Additionally the --bytes limit can be set to 'u128' using the feature
"cmp_bytes_limit_128_bit".

The performance impact would be negligible, as there only few calculations
each time a full block is read from the file.


---------

Co-authored-by: Gunter Schmidt <gsgit@beadsoft.de>
Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
2026-05-14 23:13:02 +02:00
renovate[bot] 649179069c chore(deps): update dawidd6/action-download-artifact action to v21 2026-05-14 10:29:01 +02:00
Daniel Hofstetter 9fe96ed5e9 Make compare_test_results.py executable 2026-05-14 10:25:48 +02:00
pocopepe 2c47ea9f04 feat: add gnu testsuite tracking workflows 2026-05-14 09:42:26 +02:00
Daniel Hofstetter 9a7a727da4 Fix end of some files 2026-05-14 09:41:36 +02:00
Daniel Hofstetter a24b0c391e ci: add .pre-commit-config.yaml 2026-05-14 09:41:36 +02:00
xtqqczze 3f2c8678da actions: add security audit workflow 2026-05-14 09:41:25 +02:00
Marc Herbert d73fa831b0 tests: fix "old" names in generated patch files
Fixes #223. Very simple reproduction

```
cd diffutils
mkdir a
touch a/alef  a/alefn  a/alef_  a/alefx  a/alefr  a/fuzz.file
cargo test
```
 => fail

https://www.gnu.org/software/diffutils/manual/html_node/Multiple-Patches.html
states that the "old" file name has precedence over the "new" filename.

I hit this problem because some other (and unfortunately: unknown for
now) test issue left bogus `a/alef*` file(s) behind in my workspace. I
didn't bother cleaning them up because I assumed some test would keep
recreating them and that cost me a lot of time.

This issue seems to have existed since the very first commit.
Interestingly, there as a previous attempt in 2024 to fix this in commit
a3a372ff36 ! So I was apparently not the only affected. BUT that
fix was immediately reverted by commit ba7cb0aef9 in the same
PR. Admittedly, that fix seemed somewhat off-topic in
https://github.com/uutils/diffutils/pull/33. So here it is again.
2026-05-13 14:02:35 +02:00
renovate[bot] be90f75e68 chore(deps): update rust crate assert_cmd to v2.2.2 2026-05-12 09:13:23 +02:00
oech3 259e51d0d4 *.yml: dedup env: 2026-05-09 16:56:20 +02:00
xtqqczze da98437b08 chore: add COPYRIGHT file and update license references
- move copyright information from LICENSE-* files to COPYRIGHT
- use some improved wording used by the Rust project
2026-05-09 16:48:09 +02:00
xtqqczze 54db7b0b3e Add SECURITY.md
Copied from https://github.com/uutils/coreutils/blob/5e974797bd8050c2d425a706670254ad0323404d/SECURITY.md

Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
Co-authored-by: Daniel Hofstetter <daniel.hofstetter@42dh.com>
2026-05-09 16:44:14 +02:00
renovate[bot] c811142a6c chore(deps): update rust crate divan to v4.6.0 2026-04-28 16:05:06 +02:00
pocopepe 4043bb1928 skip --help args in fuzz_cmp_args since it causes process exit 2026-04-21 10:25:34 +02:00
pocopepe d11f672d29 fix: fuzz targets missing target dir and silent CI failures 2026-04-21 10:25:34 +02:00
oech3 37abce4eab Add CI for wasm32 (#218)
Co-authored-by: oech3 <>
2026-04-19 11:05:03 +02:00
viju a340afb6d1 fix build failure on wasm32-wasip1 target (#215)
Co-authored-by: viju <pocopepe@vijus-MacBook-Air.local>
2026-04-18 19:32:32 +02:00
renovate[bot] 18c5533b82 chore(deps): update rust crate assert_cmd to v2.2.1 2026-04-17 16:42:59 +02:00
renovate[bot] 34f3935b71 chore(deps): update rust crate divan to v4.5.0 2026-04-17 13:34:41 +02:00
renovate[bot] 904efda150 chore(deps): update softprops/action-gh-release action to v3 2026-04-12 10:20:51 +02:00
renovate[bot] af3e010b26 chore(deps): update rust crate rand to v0.10.1 2026-04-11 15:27:05 +02:00
xtqqczze 0001b2036e chore(deps): update rust crates 2026-04-11 08:06:26 +02:00
renovate[bot] 8aa2a2cb7c chore(deps): update codecov/codecov-action action to v6 2026-03-27 07:25:18 +01:00
Kevin Burke 23890b6c94 fix: follow redirects when fetching gnulib init.sh in upstream test suite (#202)
The gnulib gitweb server returns a 302 redirect, but curl was called
without -L so it saved the HTML redirect page instead of init.sh.
This caused all 33 GNU upstream tests to fail in CI since the init.sh
fetch was introduced in c1b66e4.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 07:51:26 +01:00
renovate[bot] 5fc37c7c73 chore(deps): update rust crate itoa to v1.0.18 2026-03-20 06:05:18 +01:00
renovate[bot] f4895861db chore(deps): update rust crate divan to v4.4.1 2026-03-12 14:01:06 +01:00
renovate[bot] 25cad28b99 chore(deps): update rust crate divan to v4.4.0 2026-03-12 11:21:24 +01:00
renovate[bot] 454f5436ce chore(deps): update rust crate tempfile to v3.27.0 2026-03-11 06:13:15 +01:00
renovate[bot] 2efd4e17fa chore(deps): update rust crate assert_cmd to v2.2.0 2026-03-11 06:10:01 +01:00
Ryuji Yasukochi 9dcca24fb0 fix: match GNU error format for unrecognized options (#180)
* fix: match GNU error format for unrecognized options

Use single quotes and remove colon to match GNU diff/cmp output:
`unrecognized option '--foobar'` instead of `unrecognized option: "--foobar"`

Also use `contains` instead of `starts_with` in the integration test
to handle the command prefix (e.g. `cmp: unrecognized option ...`).

Follow-up to #178 / #179.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: apply cargo fmt formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 11:48:58 +01:00
oech3 5660d0eafb Cargo.toml: Simplify profiles 2026-03-08 21:53:27 +01:00
renovate[bot] c624dc489d chore(deps): update moonrepo/setup-rust action to v1 2026-03-08 14:03:47 +01:00
oech3 bdf449eaf2 Publish binary from main (#163) 2026-03-07 22:04:19 +01:00
oech3 f8248801a9 fuzzing.yml: Avoid non reusable cache generation (#170) 2026-03-07 18:39:56 +01:00
Aster Boese 357c99038f cmp: fix 32-bit usize overflow in test (#173)
Fixes https://github.com/uutils/diffutils/issues/172
2026-03-07 18:35:57 +01:00
codspeed-hq[bot] 59e130aa22 Add CodSpeed performance benchmarking workflow and badge (#189)
Co-authored-by: codspeed-hq[bot] <117304815+codspeed-hq[bot]@users.noreply.github.com>
2026-03-07 15:12:18 +01:00
Gunter Schmidt 54c8b7aeb9 feat: Divan Benchmark (#185)
* feat: Criterion Benchmark

* fix: Replaced Criterion with codspeed drop-in replacement

* feat: uses Divan instead of Criterion

* changed file num lines to file size in kb

---------

Co-authored-by: Gunter Schmidt <gsgit@beadsoft.de>
2026-03-07 14:55:10 +01:00
Ryuji Yasukochi 6f082c6572 fix: rename "Unknown option" to "unrecognized option" for diff and cmp (#179) 2026-02-28 13:43:56 +01:00
renovate[bot] 34db0ade7c chore(deps): update rust crate tempfile to v3.26.0 2026-02-24 10:29:37 +01:00
renovate[bot] d3d0b0c966 chore(deps): update rust crate chrono to v0.4.44 2026-02-23 14:00:09 +01:00
renovate[bot] 87e0aa2828 chore(deps): update rust crate predicates to v3.1.4 2026-02-12 06:27:45 +01:00
renovate[bot] 9f419c31ea chore(deps): update rust crate libfuzzer-sys to v0.4.12 2026-02-10 23:23:31 +01:00
renovate[bot] 95883b462b chore(deps): update rust crate tempfile to v3.25.0 2026-02-10 06:08:40 +01:00
Daniel Hofstetter f20af97a09 Merge pull request #166 from uutils/renovate/regex-1.x-lockfile
chore(deps): update rust crate regex to v1.12.3
2026-02-03 17:06:50 +01:00
renovate[bot] b9b7ea8d2b chore(deps): update rust crate regex to v1.12.3 2026-02-03 14:47:31 +00:00
Sylvestre Ledru 47798b4b2c Merge pull request #164 from oech3/patch-2
Use preinstalled rust, disable incremental build
2026-01-25 17:22:11 +01:00
oech3 445e1ea02f Use preinstalled rust, disable incremental build 2026-01-25 11:58:26 +09:00
renovate[bot] e2fb192d52 chore(deps): update rust crate chrono to v0.4.43 2026-01-15 06:13:15 +01:00
renovate[bot] a1d18a0c09 chore(deps): update rust crate assert_cmd to v2.1.2 2026-01-09 19:40:57 +01:00
Sylvestre Ledru 5dd2e9d30c cmp: stop allocating for byte printing (#153)
This makes verbose comparison of 37MB completely different files 2.34x
faster than our own baseline, putting our cmp at almost 6x faster than
GNU cmp (/opt/homebrew/bin/cmp) on my M4 Pro Mac. The output remains
identical to that of GNU cmp. Mostly equal and smaller files do not
regress.

Benchmark 1: ./bin/baseline/diffutils cmp -lb t/huge t/eguh
  Time (mean ± σ):      1.669 s ±  0.011 s    [User: 1.594 s, System: 0.073 s]
  Range (min … max):    1.654 s …  1.689 s    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark 2: ./target/release/diffutils cmp -lb t/huge t/eguh
  Time (mean ± σ):     714.2 ms ±   4.1 ms    [User: 629.3 ms, System: 82.7 ms]
  Range (min … max):   707.2 ms … 721.5 ms    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark 3: /opt/homebrew/bin/cmp -lb t/huge t/eguh
  Time (mean ± σ):      4.213 s ±  0.050 s    [User: 4.128 s, System: 0.081 s]
  Range (min … max):    4.160 s …  4.316 s    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark 4: /usr/bin/cmp -lb t/huge t/eguh
  Time (mean ± σ):      3.892 s ±  0.048 s    [User: 3.819 s, System: 0.070 s]
  Range (min … max):    3.808 s …  3.976 s    10 runs

  Warning: Ignoring non-zero exit code.

Summary
  ./target/release/diffutils cmp -lb t/huge t/eguh ran
    2.34 ± 0.02 times faster than ./bin/baseline/diffutils cmp -lb t/huge t/eguh
    5.45 ± 0.07 times faster than /usr/bin/cmp -lb t/huge t/eguh
    5.90 ± 0.08 times faster than /opt/homebrew/bin/cmp -lb t/huge t/eguh
2026-01-08 23:33:42 +01:00
Gustavo Noronha Silva e00ff6b108 cmp: stop allocating for byte printing
This makes verbose comparison of 37MB completely different files 2.34x
faster than our own baseline, putting our cmp at almost 6x faster than
GNU cmp (/opt/homebrew/bin/cmp) on my M4 Pro Mac. The output remains
identical to that of GNU cmp. Mostly equal and smaller files do not
regress.

Benchmark 1: ./bin/baseline/diffutils cmp -lb t/huge t/eguh
  Time (mean ± σ):      1.669 s ±  0.011 s    [User: 1.594 s, System: 0.073 s]
  Range (min … max):    1.654 s …  1.689 s    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark 2: ./target/release/diffutils cmp -lb t/huge t/eguh
  Time (mean ± σ):     714.2 ms ±   4.1 ms    [User: 629.3 ms, System: 82.7 ms]
  Range (min … max):   707.2 ms … 721.5 ms    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark 3: /opt/homebrew/bin/cmp -lb t/huge t/eguh
  Time (mean ± σ):      4.213 s ±  0.050 s    [User: 4.128 s, System: 0.081 s]
  Range (min … max):    4.160 s …  4.316 s    10 runs

  Warning: Ignoring non-zero exit code.

Benchmark 4: /usr/bin/cmp -lb t/huge t/eguh
  Time (mean ± σ):      3.892 s ±  0.048 s    [User: 3.819 s, System: 0.070 s]
  Range (min … max):    3.808 s …  3.976 s    10 runs

  Warning: Ignoring non-zero exit code.

Summary
  ./target/release/diffutils cmp -lb t/huge t/eguh ran
    2.34 ± 0.02 times faster than ./bin/baseline/diffutils cmp -lb t/huge t/eguh
    5.45 ± 0.07 times faster than /usr/bin/cmp -lb t/huge t/eguh
    5.90 ± 0.08 times faster than /opt/homebrew/bin/cmp -lb t/huge t/eguh
2026-01-02 11:21:48 -03:00
Sylvestre Ledru c38fe5f2e5 Merge pull request #158 from uutils/cargo-dist-3
cargo-dist: remove duplicate info
2026-01-02 10:51:12 +01:00
Sylvestre Ledru 44565de705 cargo-dist: refresh & remove duplicate info 2026-01-02 10:43:28 +01:00
Sylvestre Ledru 8c8c1db5c6 Merge pull request #157 from uutils/cargo-dist-2
update cargo dist conf
2026-01-01 20:33:26 +01:00
Sylvestre Ledru 3e02493701 ride along: ship fuzz/Cargo.lock 2026-01-01 20:03:49 +01:00
Sylvestre Ledru c6e8b46d21 update cargo dist 2026-01-01 20:03:47 +01:00
Sylvestre Ledru 125fc298c5 Merge pull request #156 from uutils/0.5.0
prepare version 0.5.0
2026-01-01 19:36:05 +01:00
Sylvestre Ledru 16673cf466 prepare version 0.5.0 2026-01-01 18:40:17 +01:00
Sylvestre Ledru f66ad85757 Merge pull request #152 from kov/cmp_fast_path
cmp_fast_path test improvements
2026-01-01 18:32:30 +01:00
Sylvestre Ledru de9bf94d01 Merge pull request #154 from uutils/renovate/itoa-1.x-lockfile
chore(deps): update rust crate itoa to v1.0.17
2025-12-27 11:17:32 +01:00
renovate[bot] 940a0e00b6 chore(deps): update rust crate itoa to v1.0.17 2025-12-27 09:24:33 +00:00
Gustavo Noronha Silva 7ddc6c6c4b Make cmp_fast_path more robust
On my Mac I see this test fail quite consistently. This change makes it
more resilient in systems with slower startup times, while still
allowing faster systems to finish as soon as possible.
2025-12-26 14:52:16 -03:00
Gustavo Noronha Silva 8997ac06b8 Use specific locale for cmp_fast_path test
The test was failing in the regular MacOS terminal due to it defaulting
to LC_ALL=C. Best to standardize like the other tests that check for
locale-dependent output.
2025-12-26 14:52:12 -03:00
Daniel Hofstetter df90e37566 Merge pull request #151 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.24.0
2025-12-24 07:13:44 +01:00
renovate[bot] f25cad8497 chore(deps): update rust crate tempfile to v3.24.0 2025-12-24 01:55:24 +00:00
LunarEclipse a09dcac41d Fix compilation for 32-bit targets 2025-12-22 00:04:48 +01:00
Daniel Hofstetter 98bf765a98 Merge pull request #150 from uutils/renovate/itoa-1.x-lockfile
chore(deps): update rust crate itoa to v1.0.16
2025-12-21 10:14:13 +01:00
renovate[bot] 1e1e968027 chore(deps): update rust crate itoa to v1.0.16 2025-12-21 05:08:26 +00:00
Daniel Hofstetter 4eee9cefa0 Merge pull request #147 from uutils/renovate/actions-cache-5.x
chore(deps): update actions/cache action to v5
2025-12-12 07:14:43 +01:00
renovate[bot] 67589b9331 chore(deps): update actions/cache action to v5 2025-12-12 02:51:06 +00:00
Daniel Hofstetter 83f6d2db7c tests: fix deprecation warnings from assert_cmd 2025-11-02 23:28:41 +01:00
Daniel Hofstetter b193ea0c43 Merge pull request #141 from uutils/renovate/assert_cmd-2.x-lockfile
chore(deps): update rust crate assert_cmd to v2.1.1
2025-10-29 16:52:48 +01:00
renovate[bot] 5f2ba7a84c chore(deps): update rust crate assert_cmd to v2.1.1 2025-10-29 15:19:26 +00:00
E 15473edcd7 Cargo.toml sync 2 profiles with other uutils 2025-10-29 16:18:18 +01:00
renovate[bot] 30b6bd2523 chore(deps): update rust crate assert_cmd to v2.1.0 2025-10-29 10:21:47 +01:00
renovate[bot] 590a4b405e chore(deps): update rust crate regex to v1.12.2 2025-10-13 21:31:41 +02:00
Daniel Hofstetter 418596138e Merge pull request #135 from uutils/renovate/regex-1.x-lockfile
chore(deps): update rust crate regex to v1.12.1
2025-10-11 07:17:39 +02:00
renovate[bot] eadc8c3dc5 chore(deps): update rust crate regex to v1.12.1 2025-10-11 00:35:41 +00:00
renovate[bot] 2806ec2029 chore(deps): update rust crate unicode-width to v0.2.2 2025-10-09 13:25:15 +02:00
renovate[bot] dbd60416e6 chore(deps): update rust crate regex to v1.11.3 2025-10-09 13:13:45 +02:00
Olivier Tilloy 392b8fa07b cargo dist: update to a recent release 2025-09-24 19:18:39 +02:00
Olivier Tilloy 44645e5428 Merge pull request #131 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.23.0
2025-09-23 12:55:07 +02:00
renovate[bot] 644a794067 chore(deps): update rust crate tempfile to v3.23.0 2025-09-23 10:46:28 +00:00
Olivier Tilloy 19b79efd76 Merge pull request #130 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.22.0
2025-09-09 21:57:23 +02:00
renovate[bot] 3380bab935 chore(deps): update rust crate tempfile to v3.22.0 2025-09-09 17:31:06 +00:00
Olivier Tilloy a0a05eeba9 Merge pull request #129 from uutils/renovate/chrono-0.x-lockfile
fix(deps): update rust crate chrono to v0.4.42
2025-09-08 12:56:03 +02:00
renovate[bot] 611e380266 fix(deps): update rust crate chrono to v0.4.42 2025-09-08 10:46:32 +00:00
Daniel Hofstetter af0dc993b8 Merge pull request #128 from uutils/renovate/regex-1.x-lockfile
fix(deps): update rust crate regex to v1.11.2
2025-08-24 18:49:09 +02:00
renovate[bot] a680c4f467 fix(deps): update rust crate regex to v1.11.2 2025-08-24 16:38:04 +00:00
Daniel Hofstetter 0b604f67aa Merge pull request #127 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.21.0
2025-08-20 08:00:37 +02:00
renovate[bot] cc67cbcc59 chore(deps): update rust crate tempfile to v3.21.0 2025-08-20 01:43:49 +00:00
Olivier Tilloy a95ca0062f Merge pull request #125 from cakebaker/clippy_fix_warnings
clippy: fix warnings from `unnecessary_unwrap` lint
2025-08-11 13:09:06 +02:00
Daniel Hofstetter b59d9be943 clippy: fix warnings from unnecessary_unwrap lint 2025-08-08 11:48:58 +02:00
Olivier Tilloy 3654b82a6d Merge pull request #123 from cakebaker/clippy_fix_warnings
clippy: fix warnings
2025-06-27 11:28:09 +02:00
Daniel Hofstetter 7df02399ba clippy: fix warning from ptr_arg lint 2025-06-27 10:52:37 +02:00
Daniel Hofstetter 03fe614087 clippy: fix warnings from useless_format lint 2025-06-27 10:50:06 +02:00
Daniel Hofstetter 8261d790f4 clippy: fix warnings from uninlined_format_args 2025-06-27 10:45:40 +02:00
Olivier Tilloy 168dae3aee Merge pull request #122 from uutils/renovate/unicode-width-0.x-lockfile
fix(deps): update rust crate unicode-width to v0.2.1
2025-06-10 06:41:24 +02:00
renovate[bot] c7d4140fa3 fix(deps): update rust crate unicode-width to v0.2.1 2025-06-09 23:03:42 +00:00
Sylvestre Ledru dee3bc1d66 Merge pull request #117 from sami-daniel/main
Create the side-by-side option (-y) feature for the diff command (Incomplete)
2025-06-03 14:01:38 +02:00
Sami Daniel fce0881e27 Merge branch 'main' into main 2025-06-02 22:37:12 -03:00
Sami Daniel (Tsoi) 45b3072534 Configure CI fuzzer for fuzz_side
Configuring CI to run fuzz from fuzz_side
2025-06-02 22:33:21 -03:00
renovate[bot] a3e57c950e chore(deps): update rust crate tempfile to v3.20.0 2025-06-02 22:33:21 -03:00
Sami Daniel (Tsoi) 1ef6923b7d Add side by side diff (partial)
Create the diff -y utility, this time introducing tests and changes focused
    mainly on the construction of the utility and issues related to alignment
    and response tabulation. New parameters were introduced such as the size
    of the total width of the output in the parameters. A new calculation was
    introduced to determine the size of the output columns and the maximum
    total column size. The tab and spacing mechanism has the same behavior
     as the original diff, with tabs and spaces formatted in the same way.

    - Introducing tests for the diff 'main' function
    - Introducing fuzzing for side diff utility
    - Introducing tests for internal mechanisms
    - Modular functions that allow consistent changes across the entire project
2025-06-02 22:33:04 -03:00
renovate[bot] dff98a2969 fix(deps): update rust crate chrono to v0.4.41 2025-06-02 22:33:04 -03:00
Sami Daniel 8105420bb4 Create the side-by-side option (-y) feature for the diff command (Incomplete).
- Create the function, in the utils package, limited_string that allows you to truncate a string based on a
delimiter (May break the encoding of the character where it was cut)

- Create tests for limited_string function

- Add support for -y and --side-by-side flags that enables diff output for side-by-side mode

- Create implementation of the diff -y (SideBySide) command, base command for sdiff, using the crate
diff as engine. Currently it does not fully represent GNU diff -y, some flags (|, (, ), , /) could
not be developed due to the limitation of the engine we currently use (crate diff), which did not
allow perform logic around it. Only the use of '<' and '>' were enabled.

- Create tests for SideBySide implementation
2025-06-02 22:32:11 -03:00
Daniel Hofstetter 5b791e8bf6 Merge pull request #119 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.20.0
2025-05-12 07:17:30 +02:00
renovate[bot] b31df0b5e8 chore(deps): update rust crate tempfile to v3.20.0 2025-05-11 22:12:26 +00:00
Olivier Tilloy c02273c827 Merge pull request #118 from uutils/renovate/chrono-0.x-lockfile
fix(deps): update rust crate chrono to v0.4.41
2025-04-29 17:58:44 +02:00
renovate[bot] 199c7f169c fix(deps): update rust crate chrono to v0.4.41 2025-04-29 15:45:04 +00:00
Olivier Tilloy 978390c14d Merge pull request #116 from uutils/renovate/assert_cmd-2.x-lockfile
chore(deps): update rust crate assert_cmd to v2.0.17
2025-04-16 22:04:28 +02:00
renovate[bot] 87ccc8e4c2 chore(deps): update rust crate assert_cmd to v2.0.17 2025-04-16 19:55:52 +00:00
Olivier Tilloy 9bc53486df Merge pull request #115 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.19.1
2025-03-20 06:47:26 +01:00
renovate[bot] 0d7e4d82ae chore(deps): update rust crate tempfile to v3.19.1 2025-03-19 22:53:36 +00:00
Olivier Tilloy 360bff50ed Merge pull request #114 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.19.0
2025-03-14 06:28:33 +01:00
renovate[bot] 26ee98dfaa chore(deps): update rust crate tempfile to v3.19.0 2025-03-14 02:47:05 +00:00
Olivier Tilloy 009d64acd2 Merge pull request #113 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.18.0
2025-03-07 06:42:32 +01:00
renovate[bot] b53d4f427c chore(deps): update rust crate tempfile to v3.18.0 2025-03-06 23:24:15 +00:00
Olivier Tilloy 8448fd8068 Merge pull request #111 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.17.1
2025-03-04 12:08:10 +01:00
renovate[bot] d573c3ae1d chore(deps): update rust crate tempfile to v3.17.1 2025-03-04 10:51:49 +00:00
Olivier Tilloy ca1c4c3618 Merge pull request #110 from uutils/renovate/predicates-3.x-lockfile
chore(deps): update rust crate predicates to v3.1.3
2025-03-04 11:50:11 +01:00
renovate[bot] ba1cac3c20 chore(deps): update rust crate predicates to v3.1.3 2025-03-04 10:40:57 +00:00
Olivier Tilloy 949cccebd4 Merge pull request #109 from uutils/renovate/chrono-0.x-lockfile
fix(deps): update rust crate chrono to v0.4.40
2025-03-04 11:39:52 +01:00
renovate[bot] bbdfa1b765 fix(deps): update rust crate chrono to v0.4.40 2025-03-04 10:29:12 +00:00
Olivier Tilloy 2f1a89173a Merge pull request #108 from uutils/renovate/itoa-1.x-lockfile
fix(deps): update rust crate itoa to v1.0.15
2025-03-04 11:27:34 +01:00
renovate[bot] f9553984f4 fix(deps): update rust crate itoa to v1.0.15 2025-03-04 10:09:06 +00:00
Olivier Tilloy 59920040f6 Merge pull request #112 from oSoMoN/ci-macos-use-gpatch
ci: make sure gpatch is actually being used for tests on MacOS
2025-03-04 11:07:02 +01:00
Olivier Tilloy 44c195c0b2 ci: make sure gpatch is actually being used for tests on MacOS 2025-03-04 10:58:14 +01:00
Daniel Hofstetter fdc69921e6 Merge pull request #107 from uutils/renovate/codecov-codecov-action-5.x
chore(deps): update codecov/codecov-action action to v5
2024-11-16 09:52:40 +01:00
Daniel Hofstetter 4ff2d6b182 ci: fix deprecated codecov argument 2024-11-16 09:42:50 +01:00
renovate[bot] 39e092488b chore(deps): update codecov/codecov-action action to v5 2024-11-16 07:49:05 +00:00
Olivier Tilloy dcd3dfd6e0 Merge pull request #106 from cakebaker/ci_fix_cargo_features_option
ci: remove `CARGO_FEATURES_OPTION`
2024-11-08 16:50:55 +01:00
Daniel Hofstetter 90bed40046 ci: remove CARGO_FEATURES_OPTION 2024-11-08 09:48:11 +01:00
Sylvestre Ledru 1575aec22c Merge pull request #105 from cakebaker/fix_unused_import_on_windows
Fix "unused import" warning on Windows
2024-11-08 09:36:06 +01:00
Daniel Hofstetter 4f2f869021 Fix "unused import" warning on Windows 2024-11-08 09:06:33 +01:00
Daniel Hofstetter 3101aa1aff Merge pull request #104 from uutils/renovate/tempfile-3.x-lockfile
chore(deps): update rust crate tempfile to v3.14.0
2024-11-08 07:09:59 +01:00
renovate[bot] 14b062251f chore(deps): update rust crate tempfile to v3.14.0 2024-11-08 03:47:37 +00:00
Daniel Hofstetter 0e11811ce1 Merge pull request #102 from uutils/renovate/regex-1.x-lockfile
fix(deps): update rust crate regex to v1.11.1
2024-10-24 17:14:14 +02:00
renovate[bot] 3de1930bbe fix(deps): update rust crate regex to v1.11.1 2024-10-24 15:05:12 +00:00
Sylvestre Ledru 889e7bb7cc Merge pull request #101 from cakebaker/fix_clippy_warnings
Fix clippy warnings
2024-10-18 09:53:12 +02:00
Daniel Hofstetter 1910cbfe58 Fix warnings from needless_borrow lint 2024-10-18 09:10:03 +02:00
Daniel Hofstetter c70cc1921c Fix warnings from write_with_newline lint 2024-10-18 09:08:07 +02:00
Olivier Tilloy 933230e103 Merge pull request #100 from kov/cmp-mem
cmp: print verbose diffs as we find them
2024-10-08 14:07:56 +02:00
Gustavo Noronha Silva a316262603 cmp: print verbose diffs as we find them
Before this change, we would first find all changes so we could obtain
the largest offset we will report and use that to set up the padding.

Now we use the file sizes to estimate the largest possible offset.
Not only does this allow us to print earlier, reduces memory usage, as
we do not store diffs to report later, but it also fixes a case in
which our output was different to GNU cmp's - because it also seems
to estimate based on size.

Memory usage drops by a factor of 1000(!), without losing performance
while comparing 2 binaries of hundreds of MBs:

Before:

 Maximum resident set size (kbytes): 2489260

 Benchmark 1: ../target/release/diffutils \
 cmp -l -b /usr/lib64/chromium-browser/chromium-browser /usr/lib64/firefox/libxul.so
   Time (mean ± σ):     14.466 s ±  0.166 s    [User: 12.367 s, System: 2.012 s]
   Range (min … max):   14.350 s … 14.914 s    10 runs

After:

 Maximum resident set size (kbytes): 2636

 Benchmark 1: ../target/release/diffutils \
 cmp -l -b /usr/lib64/chromium-browser/chromium-browser /usr/lib64/firefox/libxul.so
   Time (mean ± σ):     13.724 s ±  0.038 s    [User: 12.263 s, System: 1.372 s]
   Range (min … max):   13.667 s … 13.793 s    10 runs
2024-10-08 08:28:06 -03:00
Sylvestre Ledru 0bf04b4395 README.md: be explicit with the list of tools (#99) 2024-10-02 12:00:43 +00:00
41 changed files with 4266 additions and 711 deletions
+100
View File
@@ -0,0 +1,100 @@
name: GnuComment
on:
workflow_run:
workflows: ["GnuTests"]
types:
- completed
permissions: {}
jobs:
post-comment:
permissions:
actions: read # to list workflow runs artifacts
pull-requests: write # to comment on pr
runs-on: ubuntu-latest
if: >
github.event.workflow_run.event == 'pull_request'
steps:
- name: 'Download artifact'
uses: actions/github-script@v9
with:
script: |
// List all artifacts from GnuTests
var artifacts = await github.rest.actions.listWorkflowRunArtifacts({
owner: context.repo.owner,
repo: context.repo.repo,
run_id: ${{ github.event.workflow_run.id }},
});
// Download the "comment" artifact, which contains a PR number (NR) and result.txt
var matchArtifact = artifacts.data.artifacts.filter((artifact) => {
return artifact.name == "comment"
})[0];
if (!matchArtifact) {
console.log('No comment artifact found');
return;
}
var download = await github.rest.actions.downloadArtifact({
owner: context.repo.owner,
repo: context.repo.repo,
artifact_id: matchArtifact.id,
archive_format: 'zip',
});
var fs = require('fs');
fs.writeFileSync('${{ github.workspace }}/comment.zip', Buffer.from(download.data));
- run: unzip comment.zip || echo "Failed to unzip comment artifact"
- name: 'Comment on PR'
uses: actions/github-script@v9
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
var fs = require('fs');
// Check if files exist
if (!fs.existsSync('./NR')) {
console.log('No NR file found, skipping comment');
return;
}
if (!fs.existsSync('./result.txt')) {
console.log('No result.txt file found, skipping comment');
return;
}
var issue_number = Number(fs.readFileSync('./NR'));
var content = fs.readFileSync('./result.txt');
if (content.toString().trim().length > 7) { // 7 because we have backquote + \n
// Update existing comment if present, otherwise create a new one
var marker = '<!-- gnu-tests-bot -->';
var body = marker + '\nGNU diffutils testsuite comparison:\n```\n' + content + '```';
var comments = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue_number,
});
var existing = comments.data.filter(c => c.body.includes(marker))[0];
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body: body,
});
} else {
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: issue_number,
body: body,
});
}
} else {
console.log('Comment content too short, skipping');
}
+231
View File
@@ -0,0 +1,231 @@
name: GnuTests
# Run GNU diffutils testsuite against the Rust diffutils implementation
# and compare results against the main branch to catch regressions
on:
pull_request:
push:
branches:
- '*'
permissions:
contents: write # Publish diffutils instead of discarding
# End the current execution if there is a new changeset in the PR
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
env:
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
TEST_FULL_SUMMARY_FILE: 'diffutils-gnu-full-result.json'
jobs:
native:
name: Run GNU diffutils testsuite
runs-on: ubuntu-24.04
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
persist-credentials: false
- uses: dtolnay/rust-toolchain@master
with:
toolchain: stable
- uses: Swatinem/rust-cache@v2
### Build
- name: Build Rust diffutils binary
shell: bash
run: |
## Build Rust diffutils binary
cargo build --config=profile.release.strip=true --profile=release
zstd -19 target/release/diffutils -o diffutils-x86_64-unknown-linux-gnu.zst
- name: Publish latest commit
uses: softprops/action-gh-release@v3
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
with:
tag_name: latest-commit
body: |
commit: ${{ github.sha }}
draft: false
prerelease: true
files: |
diffutils-x86_64-unknown-linux-gnu.zst
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
### Run tests
- name: Run GNU diffutils testsuite
shell: bash
run: |
## Run GNU diffutils testsuite
./tests/run-upstream-testsuite.sh release || true
env:
TERM: xterm
- name: Upload full json results
uses: actions/upload-artifact@v4
with:
name: diffutils-gnu-full-result
path: tests/test-results.json
if-no-files-found: warn
aggregate:
needs: [native]
permissions:
actions: read
contents: read
pull-requests: read
name: Aggregate GNU test results
runs-on: ubuntu-24.04
steps:
- name: Initialize workflow variables
id: vars
shell: bash
run: |
## VARs setup
outputs() { step_id="${{ github.action }}"; for var in "$@" ; do echo steps.${step_id}.outputs.${var}="${!var}"; echo "${var}=${!var}" >> $GITHUB_OUTPUT; done; }
TEST_SUMMARY_FILE='diffutils-gnu-result.json'
outputs TEST_SUMMARY_FILE
- name: Checkout code
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Retrieve reference artifacts
uses: dawidd6/action-download-artifact@v21
continue-on-error: true
with:
workflow: GnuTests.yml
branch: "${{ env.DEFAULT_BRANCH }}"
workflow_conclusion: completed
path: "reference"
if_no_artifact_found: warn
- name: Download full json results
uses: actions/download-artifact@v4
with:
name: diffutils-gnu-full-result
path: results
- name: Extract/summarize testing info
id: summary
shell: bash
run: |
## Extract/summarize testing info
outputs() { step_id="${{ github.action }}"; for var in "$@" ; do echo steps.${step_id}.outputs.${var}="${!var}"; echo "${var}=${!var}" >> $GITHUB_OUTPUT; done; }
RESULT_FILE="results/test-results.json"
if [[ ! -f "$RESULT_FILE" ]]; then
echo "::error ::Missing test results at $RESULT_FILE"
exit 1
fi
TOTAL=$(jq '[.tests[]] | length' "$RESULT_FILE")
PASS=$(jq '[.tests[] | select(.result=="PASS")] | length' "$RESULT_FILE")
FAIL=$(jq '[.tests[] | select(.result=="FAIL")] | length' "$RESULT_FILE")
SKIP=$(jq '[.tests[] | select(.result=="SKIP")] | length' "$RESULT_FILE")
ERROR=0
output="GNU diffutils tests summary = TOTAL: $TOTAL / PASS: $PASS / FAIL: $FAIL / SKIP: $SKIP"
echo "${output}"
if [[ "$FAIL" -gt 0 ]]; then
echo "::warning ::${output}"
fi
jq -n \
--arg date "$(date --rfc-email)" \
--arg sha "$GITHUB_SHA" \
--arg total "$TOTAL" \
--arg pass "$PASS" \
--arg skip "$SKIP" \
--arg fail "$FAIL" \
--arg error "$ERROR" \
'{($date): { sha: $sha, total: $total, pass: $pass, skip: $skip, fail: $fail, error: $error }}' > '${{ steps.vars.outputs.TEST_SUMMARY_FILE }}'
HASH=$(sha1sum '${{ steps.vars.outputs.TEST_SUMMARY_FILE }}' | cut --delim=" " -f 1)
outputs HASH TOTAL PASS FAIL SKIP
- name: Upload SHA1/ID of 'test-summary'
uses: actions/upload-artifact@v4
with:
name: "${{ steps.summary.outputs.HASH }}"
path: "${{ steps.vars.outputs.TEST_SUMMARY_FILE }}"
- name: Upload test results summary
uses: actions/upload-artifact@v4
with:
name: test-summary
path: "${{ steps.vars.outputs.TEST_SUMMARY_FILE }}"
- name: Compare test failures VS reference
shell: bash
run: |
## Compare test failures VS reference
REF_SUMMARY_FILE='reference/diffutils-gnu-full-result/test-results.json'
CURRENT_SUMMARY_FILE="results/test-results.json"
IGNORE_INTERMITTENT=".github/workflows/ignore-intermittent.txt"
COMMENT_DIR="reference/comment"
mkdir -p ${COMMENT_DIR}
echo ${{ github.event.number }} > ${COMMENT_DIR}/NR
COMMENT_LOG="${COMMENT_DIR}/result.txt"
COMPARISON_RESULT=0
if test -f "${CURRENT_SUMMARY_FILE}"; then
if test -f "${REF_SUMMARY_FILE}"; then
echo "Reference summary SHA1/ID: $(sha1sum -- "${REF_SUMMARY_FILE}")"
echo "Current summary SHA1/ID: $(sha1sum -- "${CURRENT_SUMMARY_FILE}")"
python3 util/compare_test_results.py \
--ignore-file "${IGNORE_INTERMITTENT}" \
--output "${COMMENT_LOG}" \
"${CURRENT_SUMMARY_FILE}" "${REF_SUMMARY_FILE}"
COMPARISON_RESULT=$?
else
echo "::warning ::Skipping test comparison; no prior reference summary is available at '${REF_SUMMARY_FILE}'."
fi
else
echo "::error ::Failed to find summary of test results (missing '${CURRENT_SUMMARY_FILE}'); failing early"
exit 1
fi
if [ ${COMPARISON_RESULT} -eq 1 ]; then
echo "::error ::Found new non-intermittent test failures"
exit 1
else
echo "::notice ::No new test failures detected"
fi
- name: Upload comparison log (for GnuComment workflow)
if: success() || failure()
uses: actions/upload-artifact@v4
with:
name: comment
path: reference/comment/
- name: Report test results
if: success() || failure()
shell: bash
run: |
## Report final results
echo "::notice ::GNU diffutils testsuite results:"
echo "::notice :: Total tests: ${{ steps.summary.outputs.TOTAL }}"
echo "::notice :: Passed: ${{ steps.summary.outputs.PASS }}"
echo "::notice :: Failed: ${{ steps.summary.outputs.FAIL }}"
echo "::notice :: Skipped: ${{ steps.summary.outputs.SKIP }}"
if [[ "${{ steps.summary.outputs.FAIL }}" -gt 0 ]]; then
PASS_RATE=$(( ${{ steps.summary.outputs.PASS }} * 100 / (${{ steps.summary.outputs.PASS }} + ${{ steps.summary.outputs.FAIL }}) ))
echo "::notice :: Pass rate: ${PASS_RATE}%"
fi
+15
View File
@@ -0,0 +1,15 @@
name: Security audit
on:
schedule:
- cron: "0 0 * * *"
jobs:
audit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
persist-credentials: false
- uses: rustsec/audit-check@v2
with:
token: ${{ secrets.GITHUB_TOKEN }}
+10 -42
View File
@@ -4,6 +4,7 @@ name: Basic CI
env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: 0
jobs:
check:
@@ -15,7 +16,6 @@ jobs:
os: [ubuntu-latest, macOS-latest, windows-latest]
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- run: cargo check
test:
@@ -27,10 +27,10 @@ jobs:
os: [ubuntu-latest, macOS-latest, windows-latest]
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- name: install GNU patch on MacOS
if: runner.os == 'macOS'
run: brew install gpatch
run: |
brew install gpatch
- name: set up PATH on Windows
# Needed to use GNU's patch.exe instead of Strawberry Perl patch
if: runner.os == 'Windows'
@@ -42,8 +42,6 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- run: rustup component add rustfmt
- run: cargo fmt --all -- --check
clippy:
@@ -55,29 +53,12 @@ jobs:
os: [ubuntu-latest, macOS-latest, windows-latest]
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- run: rustup component add clippy
- run: cargo clippy -- -D warnings
gnu-testsuite:
name: GNU test suite
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@stable
- run: cargo build --release
# do not fail, the report is merely informative (at least until all tests pass reliably)
- run: ./tests/run-upstream-testsuite.sh release || true
env:
TERM: xterm
- uses: actions/upload-artifact@v4
with:
name: test-results.json
path: tests/test-results.json
- run: ./tests/print-test-results.sh tests/test-results.json
coverage:
name: Code Coverage
env:
RUSTC_BOOTSTRAP: 1
runs-on: ${{ matrix.job.os }}
strategy:
fail-fast: false
@@ -94,34 +75,22 @@ jobs:
run: |
## VARs setup
outputs() { step_id="vars"; for var in "$@" ; do echo steps.${step_id}.outputs.${var}="${!var}"; echo "${var}=${!var}" >> $GITHUB_OUTPUT; done; }
# toolchain
TOOLCHAIN="nightly" ## default to "nightly" toolchain (required for certain required unstable compiler flags) ## !maint: refactor when stable channel has needed support
# * specify gnu-type TOOLCHAIN for windows; `grcov` requires gnu-style code coverage data files
case ${{ matrix.job.os }} in windows-*) TOOLCHAIN="$TOOLCHAIN-x86_64-pc-windows-gnu" ;; esac;
# * use requested TOOLCHAIN if specified
if [ -n "${{ matrix.job.toolchain }}" ]; then TOOLCHAIN="${{ matrix.job.toolchain }}" ; fi
outputs TOOLCHAIN
# target-specific options
# * CARGO_FEATURES_OPTION
CARGO_FEATURES_OPTION='--all -- --check' ; ## default to '--all-features' for code coverage
# * CODECOV_FLAGS
CODECOV_FLAGS=$( echo "${{ matrix.job.os }}" | sed 's/[^[:alnum:]]/_/g' )
outputs CODECOV_FLAGS
- name: rust toolchain ~ install
uses: dtolnay/rust-toolchain@nightly
- run: rustup component add llvm-tools-preview
- name: install GNU patch on MacOS
if: runner.os == 'macOS'
run: brew install gpatch
run: |
brew install gpatch
- name: set up PATH on Windows
# Needed to use GNU's patch.exe instead of Strawberry Perl patch
if: runner.os == 'Windows'
run: echo "C:\Program Files\Git\usr\bin" >> $env:GITHUB_PATH
- name: Test
run: cargo test ${{ steps.vars.outputs.CARGO_FEATURES_OPTION }} --no-fail-fast
run: cargo test --all-features --no-fail-fast
env:
CARGO_INCREMENTAL: "0"
RUSTC_WRAPPER: ""
RUSTFLAGS: "-Cinstrument-coverage -Zcoverage-options=branch -Ccodegen-units=1 -Copt-level=0 -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort"
RUSTDOCFLAGS: "-Cpanic=abort"
@@ -158,12 +127,11 @@ jobs:
grcov . --output-type lcov --output-path "${COVERAGE_REPORT_FILE}" --binary-path "${COVERAGE_REPORT_DIR}" --branch
echo "report=${COVERAGE_REPORT_FILE}" >> $GITHUB_OUTPUT
- name: Upload coverage results (to Codecov.io)
uses: codecov/codecov-action@v4
uses: codecov/codecov-action@v7
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ${{ steps.coverage.outputs.report }}
files: ${{ steps.coverage.outputs.report }}
## flags: IntegrationTests, UnitTests, ${{ steps.vars.outputs.CODECOV_FLAGS }}
flags: ${{ steps.vars.outputs.CODECOV_FLAGS }}
name: codecov-umbrella
fail_ci_if_error: false
+37
View File
@@ -0,0 +1,37 @@
name: CodSpeed
on:
push:
branches:
- "main"
pull_request:
# `workflow_dispatch` allows CodSpeed to trigger backtest
# performance analysis in order to generate initial data.
workflow_dispatch:
permissions:
contents: read
id-token: write
jobs:
codspeed:
name: Run benchmarks
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup rust toolchain, cache and cargo-codspeed binary
uses: moonrepo/setup-rust@v1
with:
channel: stable
cache-target: release
bins: cargo-codspeed
- name: Build the benchmark target(s)
run: cargo codspeed build -m simulation
- name: Run the benchmarks
uses: CodSpeedHQ/action@v4
with:
mode: simulation
run: cargo codspeed run
+14 -9
View File
@@ -2,6 +2,10 @@ name: Fuzzing
# spell-checker:ignore fuzzer
env:
CARGO_INCREMENTAL: 0
RUSTC_BOOTSTRAP: 1
on:
pull_request:
push:
@@ -21,15 +25,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@nightly
- name: Install `cargo-fuzz`
run: cargo install cargo-fuzz
run: |
cargo install cargo-fuzz --locked
- uses: Swatinem/rust-cache@v2
with:
shared-key: "cargo-fuzz-cache-key"
cache-directories: "fuzz/target"
- name: Run `cargo-fuzz build`
run: cargo +nightly fuzz build
run: cargo fuzz build
fuzz-run:
needs: fuzz-build
@@ -46,28 +50,29 @@ jobs:
- { name: fuzz_ed, should_pass: true }
- { name: fuzz_normal, should_pass: true }
- { name: fuzz_patch, should_pass: true }
- { name: fuzz_side, should_pass: true }
steps:
- uses: actions/checkout@v4
- uses: dtolnay/rust-toolchain@nightly
- name: Install `cargo-fuzz`
run: cargo install cargo-fuzz
run: |
cargo install cargo-fuzz --locked
- uses: Swatinem/rust-cache@v2
with:
shared-key: "cargo-fuzz-cache-key"
cache-directories: "fuzz/target"
- name: Restore Cached Corpus
uses: actions/cache/restore@v4
uses: actions/cache/restore@v6
with:
key: corpus-cache-${{ matrix.test-target.name }}
path: |
fuzz/corpus/${{ matrix.test-target.name }}
- name: Run ${{ matrix.test-target.name }} for XX seconds
shell: bash
continue-on-error: ${{ !matrix.test-target.name.should_pass }}
continue-on-error: ${{ !matrix.test-target.should_pass }}
run: |
cargo +nightly fuzz run ${{ matrix.test-target.name }} -- -max_total_time=${{ env.RUN_FOR }} -detect_leaks=0
cargo fuzz run ${{ matrix.test-target.name }} -- -max_total_time=${{ env.RUN_FOR }} -detect_leaks=0
- name: Save Corpus Cache
uses: actions/cache/save@v4
uses: actions/cache/save@v6
with:
key: corpus-cache-${{ matrix.test-target.name }}
path: |
+84 -59
View File
@@ -1,10 +1,12 @@
# This file was autogenerated by dist: https://axodotdev.github.io/cargo-dist
#
# Copyright 2022-2024, axodotdev
# SPDX-License-Identifier: MIT or Apache-2.0
#
# CI that:
#
# * checks for a Git Tag that looks like a release
# * builds artifacts with cargo-dist (archives, installers, hashes)
# * builds artifacts with dist (archives, installers, hashes)
# * uploads those artifacts to temporary workflow zip
# * on success, uploads the artifacts to a GitHub Release
#
@@ -12,9 +14,8 @@
# title/body based on your changelogs.
name: Release
permissions:
contents: write
"contents": "write"
# This task will run whenever you push a git tag that looks like a version
# like "1.0.0", "v0.1.0-prerelease.1", "my-app/0.1.0", "releases/v1.0.0", etc.
@@ -23,10 +24,10 @@ permissions:
# must be a Cargo-style SemVer Version (must have at least major.minor.patch).
#
# If PACKAGE_NAME is specified, then the announcement will be for that
# package (erroring out if it doesn't have the given version or isn't cargo-dist-able).
# package (erroring out if it doesn't have the given version or isn't dist-able).
#
# If PACKAGE_NAME isn't specified, then the announcement will be for all
# (cargo-dist-able) packages in the workspace with that version (this mode is
# (dist-able) packages in the workspace with that version (this mode is
# intended for workspaces with only one dist-able package, or with all dist-able
# packages versioned/released in lockstep).
#
@@ -38,15 +39,15 @@ permissions:
# If there's a prerelease-style suffix to the version, then the release(s)
# will be marked as a prerelease.
on:
pull_request:
push:
tags:
- '**[0-9]+.[0-9]+.[0-9]+*'
pull_request:
jobs:
# Run 'cargo dist plan' (or host) to determine what tasks we need to do
# Run 'dist plan' (or host) to determine what tasks we need to do
plan:
runs-on: ubuntu-latest
runs-on: "ubuntu-22.04"
outputs:
val: ${{ steps.plan.outputs.manifest }}
tag: ${{ !github.event.pull_request && github.ref_name || '' }}
@@ -57,12 +58,18 @@ jobs:
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install cargo-dist
- name: Install dist
# we specify bash to get pipefail; it guards against the `curl` command
# failing. otherwise `sh` won't catch that `curl` returned non-0
shell: bash
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.13.3/cargo-dist-installer.sh | sh"
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.30.3/cargo-dist-installer.sh | sh"
- name: Cache dist
uses: actions/upload-artifact@v4
with:
name: cargo-dist-cache
path: ~/.cargo/bin/dist
# sure would be cool if github gave us proper conditionals...
# so here's a doubly-nested ternary-via-truthiness to try to provide the best possible
# functionality based on whether this is a pull_request, and whether it's from a fork.
@@ -70,8 +77,8 @@ jobs:
# but also really annoying to build CI around when it needs secrets to work right.)
- id: plan
run: |
cargo dist ${{ (!github.event.pull_request && format('host --steps=create --tag={0}', github.ref_name)) || 'plan' }} --output-format=json > plan-dist-manifest.json
echo "cargo dist ran successfully"
dist ${{ (!github.event.pull_request && format('host --steps=create --tag={0}', github.ref_name)) || 'plan' }} --output-format=json > plan-dist-manifest.json
echo "dist ran successfully"
cat plan-dist-manifest.json
echo "manifest=$(jq -c "." plan-dist-manifest.json)" >> "$GITHUB_OUTPUT"
- name: "Upload dist-manifest.json"
@@ -89,18 +96,19 @@ jobs:
if: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix.include != null && (needs.plan.outputs.publishing == 'true' || fromJson(needs.plan.outputs.val).ci.github.pr_run_mode == 'upload') }}
strategy:
fail-fast: false
# Target platforms/runners are computed by cargo-dist in create-release.
# Target platforms/runners are computed by dist in create-release.
# Each member of the matrix has the following arguments:
#
# - runner: the github runner
# - dist-args: cli flags to pass to cargo dist
# - install-dist: expression to run to install cargo-dist on the runner
# - dist-args: cli flags to pass to dist
# - install-dist: expression to run to install dist on the runner
#
# Typically there will be:
# - 1 "global" task that builds universal installers
# - N "local" tasks that build each platform's binaries and platform-specific installers
matrix: ${{ fromJson(needs.plan.outputs.val).ci.github.artifacts_matrix }}
runs-on: ${{ matrix.runner }}
container: ${{ matrix.container && matrix.container.image || null }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BUILD_MANIFEST_NAME: target/distrib/${{ join(matrix.targets, '-') }}-dist-manifest.json
@@ -110,12 +118,17 @@ jobs:
git config --global core.longpaths true
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- uses: swatinem/rust-cache@v2
with:
key: ${{ join(matrix.targets, '-') }}
- name: Install cargo-dist
run: ${{ matrix.install_dist }}
- name: Install Rust non-interactively if not already installed
if: ${{ matrix.container }}
run: |
if ! command -v cargo > /dev/null 2>&1; then
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
echo "$HOME/.cargo/bin" >> $GITHUB_PATH
fi
- name: Install dist
run: ${{ matrix.install_dist.run }}
# Get the dist-manifest
- name: Fetch local artifacts
uses: actions/download-artifact@v4
@@ -129,8 +142,8 @@ jobs:
- name: Build artifacts
run: |
# Actually do builds and make zips and whatnot
cargo dist build ${{ needs.plan.outputs.tag-flag }} --print=linkage --output-format=json ${{ matrix.dist_args }} > dist-manifest.json
echo "cargo dist ran successfully"
dist build ${{ needs.plan.outputs.tag-flag }} --print=linkage --output-format=json ${{ matrix.dist_args }} > dist-manifest.json
echo "dist ran successfully"
- id: cargo-dist
name: Post-build
# We force bash here just because github makes it really hard to get values up
@@ -140,7 +153,7 @@ jobs:
run: |
# Parse out what we just built and upload it to scratch storage
echo "paths<<EOF" >> "$GITHUB_OUTPUT"
jq --raw-output ".upload_files[]" dist-manifest.json >> "$GITHUB_OUTPUT"
dist print-upload-files-from-manifest --manifest dist-manifest.json >> "$GITHUB_OUTPUT"
echo "EOF" >> "$GITHUB_OUTPUT"
cp dist-manifest.json "$BUILD_MANIFEST_NAME"
@@ -157,17 +170,21 @@ jobs:
needs:
- plan
- build-local-artifacts
runs-on: "ubuntu-20.04"
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BUILD_MANIFEST_NAME: target/distrib/global-dist-manifest.json
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install cargo-dist
shell: bash
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.13.3/cargo-dist-installer.sh | sh"
- name: Install cached dist
uses: actions/download-artifact@v4
with:
name: cargo-dist-cache
path: ~/.cargo/bin/
- run: chmod +x ~/.cargo/bin/dist
# Get all the local artifacts for the global tasks to use (for e.g. checksums)
- name: Fetch local artifacts
uses: actions/download-artifact@v4
@@ -178,8 +195,8 @@ jobs:
- id: cargo-dist
shell: bash
run: |
cargo dist build ${{ needs.plan.outputs.tag-flag }} --output-format=json "--artifacts=global" > dist-manifest.json
echo "cargo dist ran successfully"
dist build ${{ needs.plan.outputs.tag-flag }} --output-format=json "--artifacts=global" > dist-manifest.json
echo "dist ran successfully"
# Parse out what we just built and upload it to scratch storage
echo "paths<<EOF" >> "$GITHUB_OUTPUT"
@@ -200,19 +217,24 @@ jobs:
- plan
- build-local-artifacts
- build-global-artifacts
# Only run if we're "publishing", and only if local and global didn't fail (skipped is fine)
if: ${{ always() && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }}
# Only run if we're "publishing", and only if plan, local and global didn't fail (skipped is fine)
if: ${{ always() && needs.plan.result == 'success' && needs.plan.outputs.publishing == 'true' && (needs.build-global-artifacts.result == 'skipped' || needs.build-global-artifacts.result == 'success') && (needs.build-local-artifacts.result == 'skipped' || needs.build-local-artifacts.result == 'success') }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
runs-on: "ubuntu-20.04"
runs-on: "ubuntu-22.04"
outputs:
val: ${{ steps.host.outputs.manifest }}
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
submodules: recursive
- name: Install cargo-dist
run: "curl --proto '=https' --tlsv1.2 -LsSf https://github.com/axodotdev/cargo-dist/releases/download/v0.13.3/cargo-dist-installer.sh | sh"
- name: Install cached dist
uses: actions/download-artifact@v4
with:
name: cargo-dist-cache
path: ~/.cargo/bin/
- run: chmod +x ~/.cargo/bin/dist
# Fetch artifacts from scratch-storage
- name: Fetch artifacts
uses: actions/download-artifact@v4
@@ -220,11 +242,10 @@ jobs:
pattern: artifacts-*
path: target/distrib/
merge-multiple: true
# This is a harmless no-op for GitHub Releases, hosting for that happens in "announce"
- id: host
shell: bash
run: |
cargo dist host ${{ needs.plan.outputs.tag-flag }} --steps=upload --steps=release --output-format=json > dist-manifest.json
dist host ${{ needs.plan.outputs.tag-flag }} --steps=upload --steps=release --output-format=json > dist-manifest.json
echo "artifacts uploaded and released successfully"
cat dist-manifest.json
echo "manifest=$(jq -c "." dist-manifest.json)" >> "$GITHUB_OUTPUT"
@@ -234,23 +255,7 @@ jobs:
# Overwrite the previous copy
name: artifacts-dist-manifest
path: dist-manifest.json
# Create a GitHub Release while uploading all files to it
announce:
needs:
- plan
- host
# use "always() && ..." to allow us to wait for all publish jobs while
# still allowing individual publish jobs to skip themselves (for prereleases).
# "host" however must run to completion, no skipping allowed!
if: ${{ always() && needs.host.result == 'success' }}
runs-on: "ubuntu-20.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
# Create a GitHub Release while uploading all files to it
- name: "Download GitHub Artifacts"
uses: actions/download-artifact@v4
with:
@@ -262,10 +267,30 @@ jobs:
# Remove the granular manifests
rm -f artifacts/*-dist-manifest.json
- name: Create GitHub Release
uses: ncipollo/release-action@v1
env:
PRERELEASE_FLAG: "${{ fromJson(steps.host.outputs.manifest).announcement_is_prerelease && '--prerelease' || '' }}"
ANNOUNCEMENT_TITLE: "${{ fromJson(steps.host.outputs.manifest).announcement_title }}"
ANNOUNCEMENT_BODY: "${{ fromJson(steps.host.outputs.manifest).announcement_github_body }}"
RELEASE_COMMIT: "${{ github.sha }}"
run: |
# Write and read notes from a file to avoid quoting breaking things
echo "$ANNOUNCEMENT_BODY" > $RUNNER_TEMP/notes.txt
gh release create "${{ needs.plan.outputs.tag }}" --target "$RELEASE_COMMIT" $PRERELEASE_FLAG --title "$ANNOUNCEMENT_TITLE" --notes-file "$RUNNER_TEMP/notes.txt" artifacts/*
announce:
needs:
- plan
- host
# use "always() && ..." to allow us to wait for all publish jobs while
# still allowing individual publish jobs to skip themselves (for prereleases).
# "host" however must run to completion, no skipping allowed!
if: ${{ always() && needs.host.result == 'success' }}
runs-on: "ubuntu-22.04"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
tag: ${{ needs.plan.outputs.tag }}
name: ${{ fromJson(needs.host.outputs.val).announcement_title }}
body: ${{ fromJson(needs.host.outputs.val).announcement_github_body }}
prerelease: ${{ fromJson(needs.host.outputs.val).announcement_is_prerelease }}
artifacts: "artifacts/*"
persist-credentials: false
submodules: recursive
+28
View File
@@ -0,0 +1,28 @@
# spell-checker:ignore wasip
name: WASI
on:
pull_request:
push:
branches:
- main
permissions:
contents: read
# End the current execution if there is a new changeset in the PR.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
test_wasi:
name: Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
with:
targets: wasm32-wasip1
- name: check
run: cargo check --target wasm32-wasip1
+48
View File
@@ -0,0 +1,48 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
exclude: ^tests/fixtures/
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: check-added-large-files
- id: check-executables-have-shebangs
- id: check-json
exclude: '\.vscode/(cSpell|extensions)\.json' # cSpell.json and extensions.json use comments
- id: check-shebang-scripts-are-executable
exclude: '.+\.rs' # would be triggered by #![some_attribute]
- id: check-symlinks
- id: check-toml
- id: check-yaml
args: [ --allow-multiple-documents ]
- id: destroyed-symlinks
- id: end-of-file-fixer
- id: mixed-line-ending
args: [ --fix=lf ]
- id: trailing-whitespace
- repo: local
hooks:
- id: rust-linting
name: Rust linting
description: Run cargo fmt on files included in the commit.
entry: cargo +stable fmt --
pass_filenames: true
types: [file, rust]
language: system
- id: rust-clippy
name: Rust clippy
description: Run cargo clippy on files included in the commit.
entry: cargo +stable clippy --workspace --all-targets --all-features -- -D warnings
pass_filenames: false
types: [file, rust]
language: system
- id: cspell
name: Code spell checker (cspell)
description: Run cspell to check for spelling errors (if available).
entry: bash -c 'if command -v cspell >/dev/null 2>&1; then cspell --no-must-find-files -- "$@"; else echo "cspell not found, skipping spell check"; exit 0; fi' --
pass_filenames: true
language: system
ci:
skip: [rust-linting, rust-clippy, cspell]
+32
View File
@@ -0,0 +1,32 @@
# Contributing to diffutils
Hi! Welcome to uutils/diffutils, and thanks for wanting to contribute!
This project follows the shared conventions of the [uutils](https://github.com/uutils)
organization. Before opening a pull request, please read:
- Our **[Review Guidelines](https://uutils.github.io/reviews/)** — what we expect
from a pull request and how reviews are carried out.
- Our community's [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md), if present.
Finally, feel free to join our [Discord](https://discord.gg/wQVJbvJ)!
> [!WARNING]
> uutils is original code and cannot contain any code from GNU or other
> strongly-licensed (GPL/LGPL) implementations. We **cannot** accept changes
> based on the GNU source code, and you **must not link** to it either. You may
> look at permissively-licensed implementations (MIT/BSD) and read the GNU
> *manuals* — never the GNU *source*.
## In short
- Discuss non-trivial changes in an issue **before** writing the code.
- Keep pull requests **small, self-contained, and descriptively titled**
(e.g. `diffutils: fix ...`).
- Make sure CI passes: tests are green, `rustfmt` is satisfied, and there are
no `clippy` warnings.
- Add tests for new behavior; don't let coverage regress.
- Write small, atomic commits annotated with the component you touched.
See the [Review Guidelines](https://uutils.github.io/reviews/) for the full
details.
+8
View File
@@ -0,0 +1,8 @@
Copyright (c) Michael Howell
Copyright (c) uutils developers
Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
<LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
option. All files in the project carrying such notice may not be
copied, modified, or distributed except according to those terms.
Generated
+704 -201
View File
File diff suppressed because it is too large Load Diff
+22 -17
View File
@@ -1,6 +1,6 @@
[package]
name = "diffutils"
version = "0.4.2"
version = "0.5.0"
edition = "2021"
description = "A CLI app for generating diff files"
license = "MIT OR Apache-2.0"
@@ -23,25 +23,30 @@ same-file = "1.0.6"
unicode-width = "0.2.0"
[dev-dependencies]
pretty_assertions = "1.4.0"
assert_cmd = "2.0.14"
divan = { version = "5.0.0", package = "codspeed-divan-compat" }
pretty_assertions = "1.4.0"
predicates = "3.1.0"
tempfile = "3.10.1"
rand = "0.10.0"
tempfile = "3.26.0"
# The profile that 'cargo dist' will build with
[profile.release]
lto = "thin"
codegen-units = 1
panic = "abort"
# alias profile for 'dist'
[profile.dist]
inherits = "release"
lto = "thin"
# Config for 'cargo dist'
[workspace.metadata.dist]
# The preferred cargo-dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.13.3"
# CI backends to support
ci = ["github"]
# The installers to generate for each app
installers = []
# Target platforms to build apps for (Rust target-triple syntax)
targets = ["aarch64-apple-darwin", "x86_64-apple-darwin", "x86_64-unknown-linux-gnu", "x86_64-pc-windows-msvc"]
# Publish jobs to run in CI
pr-run-mode = "plan"
[[bench]]
name = "bench_diffutils"
path = "benches/bench-diffutils.rs"
harness = false
[features]
# default = ["feat_bench_not_diff"]
# Turn bench for diffutils cmp off
feat_bench_not_cmp = []
# Turn bench for diffutils diff off
feat_bench_not_diff = []
-3
View File
@@ -1,6 +1,3 @@
Copyright (c) Michael Howell
Copyright (c) uutils developers
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
-3
View File
@@ -1,6 +1,3 @@
Copyright (c) Michael Howell
Copyright (c) uutils developers
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
+6 -1
View File
@@ -2,6 +2,7 @@
[![Discord](https://img.shields.io/badge/discord-join-7289DA.svg?logo=discord&longCache=true&style=flat)](https://discord.gg/wQVJbvJ)
[![License](http://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/uutils/diffutils/blob/main/LICENSE)
[![dependency status](https://deps.rs/repo/github/uutils/diffutils/status.svg)](https://deps.rs/repo/github/uutils/diffutils)
[![CodSpeed](https://img.shields.io/endpoint?url=https://codspeed.io/badge.json)](https://codspeed.io/uutils/diffutils?utm_source=badge)
[![CodeCov](https://codecov.io/gh/uutils/diffutils/branch/main/graph/badge.svg)](https://codecov.io/gh/uutils/diffutils)
@@ -53,4 +54,8 @@ $ cargo run -- -u fruits_old.txt fruits_new.txt
## License
diffutils is licensed under the MIT and Apache Licenses - see the `LICENSE-MIT` or `LICENSE-APACHE` files for details
This project is distributed under the terms of both the MIT license and the
Apache License (Version 2.0).
See [LICENSE-APACHE](LICENSE-APACHE), [LICENSE-MIT](LICENSE-MIT), and
[COPYRIGHT](COPYRIGHT) for details.
+44
View File
@@ -0,0 +1,44 @@
# Security Policy
## Supported Versions
We provide security updates only for the latest released version of `uutils/diffutils`.
Older versions may not receive patches.
If you are using a version packaged by your Linux distribution, please check with your distribution maintainers for their update policy.
---
## Reporting a Vulnerability
**Do not open public GitHub issues for security vulnerabilities.**
This prevents accidental disclosure before a fix is available.
Instead, please use the following method:
- **Email:** [sylvestre@debian.org](mailto:Sylvestre@debian.org)
- **Encryption (optional):** You may encrypt your report using our PGP key:
Fingerprint: B60D B599 4D39 BEC4 D1A9 5CCF 7E65 28DA 752F 1BE1
---
### What to Include in Your Report
To help us investigate and resolve the issue quickly, please include as much detail as possible:
- **Type of issue:** e.g. privilege escalation, information disclosure.
- **Location in the source:** file path, commit hash, branch, or tag.
- **Steps to reproduce:** exact commands, test cases, or scripts.
- **Special configuration:** any flags, environment variables, or system setup required.
- **Affected systems:** OS/distribution and version(s) where the issue occurs.
- **Impact:** your assessment of the potential severity (DoS, RCE, data leak, etc.).
---
## Disclosure Policy
We follow a **Coordinated Vulnerability Disclosure (CVD)** process:
1. We will acknowledge receipt of your report within **10 days**.
2. We will investigate, reproduce, and assess the issue.
3. We will provide a timeline for developing and releasing a fix.
4. Once a fix is available, we will publish a GitHub Security Advisory.
5. You will be credited in the advisory unless you request anonymity.
+377
View File
@@ -0,0 +1,377 @@
// This file is part of the uutils diffutils package.
//
// For the full copyright and license information, please view the LICENSE-*
// files that was distributed with this source code.
//! Benches for all utils in diffutils.
//!
//! There is a file generator included to create files of different sizes for comparison. \
//! Set the TEMP_DIR const to keep the files. df_to_ files have small changes in them, search for '#'. \
//! File generation up to 1 GB is really fast, Benchmarking above 100 MB takes very long.
/// Generate test files with these sizes in KB.
const FILE_SIZE_KILO_BYTES: [u64; 4] = [100, 1 * MB, 10 * MB, 25 * MB];
// const FILE_SIZE_KILO_BYTES: [u64; 3] = [100, 1 * MB, 5 * MB];
// Empty String to use TempDir (files will be removed after test) or specify dir to keep generated files
const TEMP_DIR: &str = "";
const NUM_DIFF: u64 = 4;
// just for FILE_SIZE_KILO_BYTES
const MB: u64 = 1_000;
const CHANGE_CHAR: u8 = b'#';
#[cfg(not(feature = "feat_bench_not_cmp"))]
mod diffutils_cmp {
use std::hint::black_box;
use diffutilslib::cmp;
use divan::Bencher;
use crate::{binary, prepare::*, FILE_SIZE_KILO_BYTES};
#[divan::bench(args = FILE_SIZE_KILO_BYTES)]
fn cmp_compare_files_equal(bencher: Bencher, kb: u64) {
let (from, to) = get_context().get_test_files_equal(kb);
let cmd = format!("cmp {from} {to}");
let opts = str_to_options(&cmd).into_iter().peekable();
let params = cmp::parse_params(opts).unwrap();
bencher
// .with_inputs(|| prepare::cmp_params_identical_testfiles(lines))
.with_inputs(|| params.clone())
.bench_refs(|params| black_box(cmp::cmp(&params).unwrap()));
}
// bench the actual compare; cmp exits on first difference
#[divan::bench(args = FILE_SIZE_KILO_BYTES)]
fn cmp_compare_files_different(bencher: Bencher, bytes: u64) {
let (from, to) = get_context().get_test_files_different(bytes);
let cmd = format!("cmp {from} {to} -s");
let opts = str_to_options(&cmd).into_iter().peekable();
let params = cmp::parse_params(opts).unwrap();
bencher
// .with_inputs(|| prepare::cmp_params_identical_testfiles(lines))
.with_inputs(|| params.clone())
.bench_refs(|params| black_box(cmp::cmp(&params).unwrap()));
}
// bench original GNU cmp
#[divan::bench(args = FILE_SIZE_KILO_BYTES)]
fn cmd_cmp_gnu_equal(bencher: Bencher, bytes: u64) {
let (from, to) = get_context().get_test_files_equal(bytes);
let args_str = format!("{from} {to}");
bencher
// .with_inputs(|| prepare::cmp_params_identical_testfiles(lines))
.with_inputs(|| args_str.clone())
.bench_refs(|cmd_args| binary::bench_binary("cmp", cmd_args));
}
// bench the compiled release version
#[divan::bench(args = FILE_SIZE_KILO_BYTES)]
fn cmd_cmp_release_equal(bencher: Bencher, bytes: u64) {
let (from, to) = get_context().get_test_files_equal(bytes);
let args_str = format!("cmp {from} {to}");
bencher
// .with_inputs(|| prepare::cmp_params_identical_testfiles(lines))
.with_inputs(|| args_str.clone())
.bench_refs(|cmd_args| binary::bench_binary("target/release/diffutils", cmd_args));
}
}
#[cfg(not(feature = "feat_bench_not_diff"))]
mod diffutils_diff {
// use std::hint::black_box;
use crate::{binary, prepare::*, FILE_SIZE_KILO_BYTES};
// use diffutilslib::params;
use divan::Bencher;
// bench the actual compare
// TODO diff does not have a diff function
// #[divan::bench(args = [100_000,10_000])]
// fn diff_compare_files(bencher: Bencher, bytes: u64) {
// let (from, to) = gen_testfiles(lines, 0, "id");
// let cmd = format!("cmp {from} {to}");
// let opts = str_to_options(&cmd).into_iter().peekable();
// let params = params::parse_params(opts).unwrap();
//
// bencher
// // .with_inputs(|| prepare::cmp_params_identical_testfiles(lines))
// .with_inputs(|| params.clone())
// .bench_refs(|params| diff::diff(&params).unwrap());
// }
// bench original GNU diff
#[divan::bench(args = FILE_SIZE_KILO_BYTES)]
fn cmd_diff_gnu_equal(bencher: Bencher, bytes: u64) {
let (from, to) = get_context().get_test_files_equal(bytes);
let args_str = format!("{from} {to}");
bencher
// .with_inputs(|| prepare::cmp_params_identical_testfiles(lines))
.with_inputs(|| args_str.clone())
.bench_refs(|cmd_args| binary::bench_binary("diff", cmd_args));
}
// bench the compiled release version
#[divan::bench(args = FILE_SIZE_KILO_BYTES)]
fn cmd_diff_release_equal(bencher: Bencher, bytes: u64) {
let (from, to) = get_context().get_test_files_equal(bytes);
let args_str = format!("diff {from} {to}");
bencher
// .with_inputs(|| prepare::cmp_params_identical_testfiles(lines))
.with_inputs(|| args_str.clone())
.bench_refs(|cmd_args| binary::bench_binary("target/release/diffutils", cmd_args));
}
}
mod parser {
use std::hint::black_box;
use diffutilslib::{cmp, params};
use divan::Bencher;
use crate::prepare::str_to_options;
// bench the time it takes to parse the command line arguments
#[divan::bench]
fn cmp_parser(bencher: Bencher) {
let cmd = "cmd file_1.txt file_2.txt -bl n10M --ignore-initial=100KiB:1MiB";
let args = str_to_options(&cmd).into_iter().peekable();
bencher
.with_inputs(|| args.clone())
.bench_values(|data| black_box(cmp::parse_params(data)));
}
// // test the impact on the benchmark if not converting the cmd to Vec<OsString> (doubles for parse)
// #[divan::bench]
// fn cmp_parser_no_prepare() {
// let cmd = "cmd file_1.txt file_2.txt -bl n10M --ignore-initial=100KiB:1MiB";
// let args = str_to_options(&cmd).into_iter().peekable();
// let _ = cmp::parse_params(args);
// }
// bench the time it takes to parse the command line arguments
#[divan::bench]
fn diff_parser(bencher: Bencher) {
let cmd = "diff file_1.txt file_2.txt -s --brief --expand-tabs --width=100";
let args = str_to_options(&cmd).into_iter().peekable();
bencher
.with_inputs(|| args.clone())
.bench_values(|data| black_box(params::parse_params(data)));
}
}
mod prepare {
use std::{
ffi::OsString,
fs::{self, File},
io::{BufWriter, Write},
path::Path,
sync::OnceLock,
};
use rand::RngExt;
use tempfile::TempDir;
use crate::{CHANGE_CHAR, FILE_SIZE_KILO_BYTES, NUM_DIFF, TEMP_DIR};
// file lines and .txt will be added
const FROM_FILE: &str = "from_file";
const TO_FILE: &str = "to_file";
const LINE_LENGTH: usize = 60;
/// Contains test data (file names) which only needs to be created once.
#[derive(Debug, Default)]
pub struct BenchContext {
pub tmp_dir: Option<TempDir>,
pub dir: String,
pub files_equal: Vec<(String, String)>,
pub files_different: Vec<(String, String)>,
}
impl BenchContext {
pub fn get_path(&self) -> &Path {
match &self.tmp_dir {
Some(tmp) => tmp.path(),
None => Path::new(&self.dir),
}
}
pub fn get_test_files_equal(&self, kb: u64) -> &(String, String) {
let p = FILE_SIZE_KILO_BYTES.iter().position(|f| *f == kb).unwrap();
&self.files_equal[p]
}
#[allow(unused)]
pub fn get_test_files_different(&self, kb: u64) -> &(String, String) {
let p = FILE_SIZE_KILO_BYTES.iter().position(|f| *f == kb).unwrap();
&self.files_different[p]
}
}
// Since each bench function is separate in Divan it is more difficult to dynamically create test data.
// This keeps the TempDir alive until the program exits and generates the files only once.
static SHARED_CONTEXT: OnceLock<BenchContext> = OnceLock::new();
/// Creates the test files once and provides them to all tests.
pub fn get_context() -> &'static BenchContext {
SHARED_CONTEXT.get_or_init(|| {
let mut ctx = BenchContext::default();
if TEMP_DIR.is_empty() {
let tmp_dir = TempDir::new().expect("Failed to create temp dir");
ctx.tmp_dir = Some(tmp_dir);
} else {
// uses current directory, the generated files are kept
let path = Path::new(TEMP_DIR);
if !path.exists() {
fs::create_dir_all(path).expect("Path {path} could not be created");
}
ctx.dir = TEMP_DIR.to_string();
};
// generate test bytes
for kb in FILE_SIZE_KILO_BYTES {
let f = generate_test_files_bytes(ctx.get_path(), kb * 1000, 0, "eq")
.expect("generate_test_files failed");
ctx.files_equal.push(f);
let f = generate_test_files_bytes(ctx.get_path(), kb * 1000, NUM_DIFF, "df")
.expect("generate_test_files failed");
ctx.files_different.push(f);
}
ctx
})
}
pub fn str_to_options(opt: &str) -> Vec<OsString> {
let s: Vec<OsString> = opt
.split(" ")
.into_iter()
.filter(|s| !s.is_empty())
.map(|s| OsString::from(s))
.collect();
s
}
/// Generates two test files for comparison with <bytes> size.
///
/// Each line consists of 10 words with 5 letters, giving a line length of 60 bytes.
/// If num_differences is set, '#' will be inserted between the first two words of a line,
/// evenly spaced in the file. 1 will add the change in the last line, so the comparison takes longest.
fn generate_test_files_bytes(
dir: &Path,
bytes: u64,
num_differences: u64,
id: &str,
) -> std::io::Result<(String, String)> {
let id = if id.is_empty() {
"".to_string()
} else {
format!("{id}_")
};
let f1 = format!("{id}{FROM_FILE}_{bytes}.txt");
let f2 = format!("{id}{TO_FILE}_{bytes}.txt");
let from_path = dir.join(f1);
let to_path = dir.join(f2);
generate_file_bytes(&from_path, &to_path, bytes, num_differences)?;
Ok((
from_path.to_string_lossy().to_string(),
to_path.to_string_lossy().to_string(),
))
}
fn generate_file_bytes(
from_name: &Path,
to_name: &Path,
bytes: u64,
num_differences: u64,
) -> std::io::Result<()> {
let file_from = File::create(from_name)?;
let file_to = File::create(to_name)?;
// for int division, lines will be smaller than requested bytes
let n_lines = bytes / LINE_LENGTH as u64;
let change_every_n_lines = if num_differences == 0 {
0
} else {
let c = n_lines / num_differences;
if c == 0 {
1
} else {
c
}
};
// Use a larger 128KB buffer for massive files
let mut writer_from = BufWriter::with_capacity(128 * 1024, file_from);
let mut writer_to = BufWriter::with_capacity(128 * 1024, file_to);
let mut rng = rand::rng();
// Each line: (5 chars * 10 words) + 9 spaces + 1 newline = 60 bytes
let mut line_buffer = [b' '; 60];
line_buffer[59] = b'\n'; // Set the newline once at the end
for i in (0..n_lines).rev() {
// Fill only the letter positions, skipping spaces and the newline
for word_idx in 0..10 {
let start = word_idx * 6; // Each word + space block is 6 bytes
for i in 0..5 {
line_buffer[start + i] = rng.random_range(b'a'..b'z' + 1);
}
}
// Write the raw bytes directly to both files
writer_from.write_all(&line_buffer)?;
// make changes in the file
if num_differences == 0 {
writer_to.write_all(&line_buffer)?;
} else {
if i % change_every_n_lines == 0 && n_lines - i > 2 {
line_buffer[5] = CHANGE_CHAR;
}
writer_to.write_all(&line_buffer)?;
line_buffer[5] = b' ';
}
}
// create last line
let missing = (bytes - n_lines as u64 * LINE_LENGTH as u64) as usize;
if missing > 0 {
for word_idx in 0..10 {
let start = word_idx * 6; // Each word + space block is 6 bytes
for i in 0..5 {
line_buffer[start + i] = rng.random_range(b'a'..b'z' + 1);
}
}
line_buffer[missing - 1] = b'\n';
writer_from.write_all(&line_buffer[0..missing])?;
writer_to.write_all(&line_buffer[0..missing])?;
}
writer_from.flush()?;
writer_to.flush()?;
Ok(())
}
}
mod binary {
use std::process::Command;
use crate::prepare::str_to_options;
pub fn bench_binary(program: &str, cmd_args: &str) -> std::process::ExitStatus {
let args = str_to_options(cmd_args);
Command::new(program)
.args(args)
.status()
.expect("Failed to execute binary")
}
}
fn main() {
// Run registered benchmarks.
divan::main();
}
+13
View File
@@ -0,0 +1,13 @@
[workspace]
members = ["cargo:."]
# Config for 'dist'
[dist]
# The preferred dist version to use in CI (Cargo.toml SemVer syntax)
cargo-dist-version = "0.30.3"
# CI backends to support
ci = "github"
# The installers to generate for each app
installers = []
# Target platforms to build apps for (Rust target-triple syntax)
targets = ["aarch64-apple-darwin", "aarch64-unknown-linux-gnu", "x86_64-apple-darwin", "x86_64-unknown-linux-gnu", "x86_64-pc-windows-msvc"]
+447
View File
@@ -0,0 +1,447 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "aho-corasick"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301"
dependencies = [
"memchr",
]
[[package]]
name = "android_system_properties"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "819e7219dbd41043ac279b19830f2efc897156490d7fd6ea916720117ee66311"
dependencies = [
"libc",
]
[[package]]
name = "arbitrary"
version = "1.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d036a3c4ab069c7b410a2ce876bd74808d2d0888a82667669f8e783a898bf1"
[[package]]
name = "autocfg"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "bumpalo"
version = "3.20.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5d20789868f4b01b2f2caec9f5c4e0213b41e3e5702a50157d699ae31ced2fcb"
[[package]]
name = "cc"
version = "1.2.60"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "43c5703da9466b66a946814e1adf53ea2c90f10063b86290cc9eb67ce3478a20"
dependencies = [
"find-msvc-tools",
"jobserver",
"libc",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "chrono"
version = "0.4.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c673075a2e0e5f4a1dde27ce9dee1ea4558c7ffe648f576438a20ca1d2acc4b0"
dependencies = [
"iana-time-zone",
"js-sys",
"num-traits",
"wasm-bindgen",
"windows-link",
]
[[package]]
name = "core-foundation-sys"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "diff"
version = "0.1.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "56254986775e3233ffa9c4d7d3faaf6d36a2c09d30b20687e9f88bc8bafc16c8"
[[package]]
name = "diffutils"
version = "0.5.0"
dependencies = [
"chrono",
"diff",
"itoa",
"regex",
"same-file",
"unicode-width",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5baebc0774151f905a1a2cc41989300b1e6fbb29aff0ceffa1064fdd3088d582"
[[package]]
name = "getrandom"
version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "899def5c37c4fd7b2664648c28120ecec138e4d395b459e5ca34f9cce2dd77fd"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"wasip2",
]
[[package]]
name = "iana-time-zone"
version = "0.1.65"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e31bc9ad994ba00e440a8aa5c9ef0ec67d5cb5e5cb0cc7f8b744a35b389cc470"
dependencies = [
"android_system_properties",
"core-foundation-sys",
"iana-time-zone-haiku",
"js-sys",
"log",
"wasm-bindgen",
"windows-core",
]
[[package]]
name = "iana-time-zone-haiku"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f31827a206f56af32e590ba56d5d2d085f558508192593743f16b2306495269f"
dependencies = [
"cc",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "jobserver"
version = "0.1.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9afb3de4395d6b3e67a780b6de64b51c978ecf11cb9a462c66be7d4ca9039d33"
dependencies = [
"getrandom",
"libc",
]
[[package]]
name = "js-sys"
version = "0.3.94"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2e04e2ef80ce82e13552136fabeef8a5ed1f985a96805761cbb9a2c34e7664d9"
dependencies = [
"once_cell",
"wasm-bindgen",
]
[[package]]
name = "libc"
version = "0.2.184"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "48f5d2a454e16a5ea0f4ced81bd44e4cfc7bd3a507b61887c99fd3538b28e4af"
[[package]]
name = "libfuzzer-sys"
version = "0.4.13"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a9fd2f41a1cba099f79a0b6b6c35656cf7c03351a7bae8ff0f28f25270f929d2"
dependencies = [
"arbitrary",
"cc",
]
[[package]]
name = "log"
version = "0.4.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897"
[[package]]
name = "memchr"
version = "2.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8ca58f447f06ed17d5fc4043ce1b10dd205e060fb3ce5b979b8ed8e59ff3f79"
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "r-efi"
version = "5.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
[[package]]
name = "regex"
version = "1.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e10754a14b9137dd7b1e3e5b0493cc9171fdd105e0ab477f51b72e7f3ac0e276"
dependencies = [
"aho-corasick",
"memchr",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "regex-automata"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dc897dd8d9e8bd1ed8cdad82b5966c3e0ecae09fb1907d58efaa013543185d0a"
[[package]]
name = "rustversion"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]]
name = "same-file"
version = "1.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502"
dependencies = [
"winapi-util",
]
[[package]]
name = "shlex"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-width"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254"
[[package]]
name = "unified-diff-fuzz"
version = "0.0.0"
dependencies = [
"diffutils",
"libfuzzer-sys",
]
[[package]]
name = "wasip2"
version = "1.0.2+wasi-0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9517f9239f02c069db75e65f174b3da828fe5f5b945c4dd26bd25d89c03ebcf5"
dependencies = [
"wit-bindgen",
]
[[package]]
name = "wasm-bindgen"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0551fc1bb415591e3372d0bc4780db7e587d84e2a7e79da121051c5c4b89d0b0"
dependencies = [
"cfg-if",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7fbdf9a35adf44786aecd5ff89b4563a90325f9da0923236f6104e603c7e86be"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dca9693ef2bab6d4e6707234500350d8dad079eb508dca05530c85dc3a529ff2"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39129a682a6d2d841b6c429d0c51e5cb0ed1a03829d8b3d1e69a011e62cb3d3b"
dependencies = [
"unicode-ident",
]
[[package]]
name = "winapi-util"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22"
dependencies = [
"windows-sys",
]
[[package]]
name = "windows-core"
version = "0.62.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb"
dependencies = [
"windows-implement",
"windows-interface",
"windows-link",
"windows-result",
"windows-strings",
]
[[package]]
name = "windows-implement"
version = "0.60.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "windows-interface"
version = "0.59.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-result"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-strings"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "wit-bindgen"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7249219f66ced02969388cf2bb044a09756a083d0fab1e566056b04d9fbcaa5"
+5 -1
View File
@@ -47,4 +47,8 @@ path = "fuzz_targets/fuzz_ed.rs"
test = false
doc = false
[[bin]]
name = "fuzz_side"
path = "fuzz_targets/fuzz_side.rs"
test = false
doc = false
+2 -2
View File
@@ -4,7 +4,7 @@ extern crate libfuzzer_sys;
use diffutilslib::cmp::{self, Cmp};
use std::ffi::OsString;
use std::fs::File;
use std::fs::{self, File};
use std::io::Write;
fn os(s: &str) -> OsString {
@@ -18,7 +18,7 @@ fuzz_target!(|x: (Vec<u8>, Vec<u8>)| {
.peekable();
let (from, to) = x;
fs::create_dir_all("target").unwrap();
File::create("target/fuzz.cmp.a")
.unwrap()
.write_all(&from)
+3
View File
@@ -11,6 +11,9 @@ fn os(s: &str) -> OsString {
}
fuzz_target!(|x: Vec<OsString>| -> Corpus {
if x.iter().any(|a| a == "--help") {
return Corpus::Reject;
}
if x.len() > 6 {
// Make sure we try to parse an option when we get longer args. x[0] will be
// the executable name.
+1
View File
@@ -38,6 +38,7 @@ fuzz_target!(|x: (Vec<u8>, Vec<u8>)| {
} else {
return;
}
fs::create_dir_all("target").unwrap();
let diff = diff_w(&from, &to, "target/fuzz.file").unwrap();
File::create("target/fuzz.file.original")
.unwrap()
+1
View File
@@ -23,6 +23,7 @@ fuzz_target!(|x: (Vec<u8>, Vec<u8>)| {
return
}*/
let diff = normal_diff::diff(&from, &to, &Params::default());
fs::create_dir_all("target").unwrap();
File::create("target/fuzz.file.original")
.unwrap()
.write_all(&from)
+5 -3
View File
@@ -21,15 +21,17 @@ fuzz_target!(|x: (Vec<u8>, Vec<u8>, u8)| {
} else {
return
}*/
fs::create_dir_all("target").unwrap();
let patched = "target/fuzz.file";
let diff = unified_diff::diff(
&from,
&to,
&Params {
from: "a/fuzz.file".into(),
to: "target/fuzz.file".into(),
from: patched.into(),
to: patched.into(),
context_count: context as usize,
..Default::default()
}
},
);
File::create("target/fuzz.file.original")
.unwrap()
+43
View File
@@ -0,0 +1,43 @@
#![no_main]
#[macro_use]
extern crate libfuzzer_sys;
use diffutilslib::side_diff;
use diffutilslib::params::Params;
use std::fs::{self, File};
use std::io::Write;
fuzz_target!(|x: (Vec<u8>, Vec<u8>, /* usize, usize */ bool)| {
let (original, new, /* width, tabsize, */ expand) = x;
// if width == 0 || tabsize == 0 {
// return;
// }
let params = Params {
// width,
// tabsize,
expand_tabs: expand,
..Default::default()
};
fs::create_dir_all("target").unwrap();
let mut output_buf = vec![];
side_diff::diff(&original, &new, &mut output_buf, &params);
File::create("target/fuzz.file.original")
.unwrap()
.write_all(&original)
.unwrap();
File::create("target/fuzz.file.new")
.unwrap()
.write_all(&new)
.unwrap();
File::create("target/fuzz.file")
.unwrap()
.write_all(&original)
.unwrap();
File::create("target/fuzz.diff")
.unwrap()
.write_all(&output_buf)
.unwrap();
});
+243 -225
View File
@@ -9,36 +9,41 @@ use std::ffi::OsString;
use std::io::{BufRead, BufReader, BufWriter, Read, Write};
use std::iter::Peekable;
use std::process::ExitCode;
use std::{fs, io};
use std::{cmp, fs, io};
#[cfg(not(target_os = "windows"))]
#[cfg(unix)]
use std::os::fd::{AsRawFd, FromRawFd};
#[cfg(not(target_os = "windows"))]
#[cfg(unix)]
use std::os::unix::fs::MetadataExt;
#[cfg(target_os = "windows")]
use std::os::windows::fs::MetadataExt;
/// for --bytes, so really large number limits can be expressed, like 1Y.
pub type BytesLimitU64 = u64;
// ignore initial is currently limited to u64, as take(skip) is used.
pub type SkipU64 = u64;
#[derive(Clone, Debug, Default, Eq, PartialEq)]
pub struct Params {
executable: OsString,
from: OsString,
to: OsString,
print_bytes: bool,
skip_a: Option<usize>,
skip_b: Option<usize>,
max_bytes: Option<usize>,
skip_a: Option<SkipU64>,
skip_b: Option<SkipU64>,
max_bytes: Option<BytesLimitU64>,
verbose: bool,
quiet: bool,
}
#[inline]
fn usage_string(executable: &str) -> String {
format!("Usage: {} <from> <to>", executable)
format!("Usage: {executable} <from> <to>")
}
#[cfg(not(target_os = "windows"))]
#[cfg(unix)]
fn is_stdout_dev_null() -> bool {
let Ok(dev_null) = fs::metadata("/dev/null") else {
return false;
@@ -60,23 +65,25 @@ fn is_stdout_dev_null() -> bool {
is_dev_null
}
#[cfg(not(any(unix, target_os = "windows")))]
fn is_stdout_dev_null() -> bool {
false
}
pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Result<Params, String> {
let Some(executable) = opts.next() else {
return Err("Usage: <exe> <from> <to>".to_string());
};
let executable = opts.next().ok_or("Usage: <exe> <from> <to>".to_string())?;
let executable_str = executable.to_string_lossy().to_string();
let parse_skip = |param: &str, skip_desc: &str| -> Result<usize, String> {
let parse_skip = |param: &str, skip_desc: &str| -> Result<SkipU64, String> {
let suffix_start = param
.find(|b: char| !b.is_ascii_digit())
.unwrap_or(param.len());
let mut num = match param[..suffix_start].parse::<usize>() {
let mut num = match param[..suffix_start].parse::<SkipU64>() {
Ok(num) => num,
Err(e) if *e.kind() == std::num::IntErrorKind::PosOverflow => usize::MAX,
Err(e) if *e.kind() == std::num::IntErrorKind::PosOverflow => SkipU64::MAX,
Err(_) => {
return Err(format!(
"{}: invalid --ignore-initial value '{}'",
executable_str, skip_desc
"{executable_str}: invalid --ignore-initial value '{skip_desc}'"
))
}
};
@@ -84,7 +91,7 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
if suffix_start != param.len() {
// Note that GNU cmp advertises supporting up to Y, but fails if you try
// to actually use anything beyond E.
let multiplier: usize = match &param[suffix_start..] {
let multiplier: SkipU64 = match &param[suffix_start..] {
"kB" => 1_000,
"K" => 1_024,
"MB" => 1_000_000,
@@ -97,21 +104,21 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
"P" => 1_125_899_906_842_624,
"EB" => 1_000_000_000_000_000_000,
"E" => 1_152_921_504_606_846_976,
"ZB" => usize::MAX, // 1_000_000_000_000_000_000_000,
"Z" => usize::MAX, // 1_180_591_620_717_411_303_424,
"YB" => usize::MAX, // 1_000_000_000_000_000_000_000_000,
"Y" => usize::MAX, // 1_208_925_819_614_629_174_706_176,
// TODO setting usize:MAX does not mimic GNU cmp behavior, it should be an error.
"ZB" => SkipU64::MAX, // 1_000_000_000_000_000_000_000,
"Z" => SkipU64::MAX, // 1_180_591_620_717_411_303_424,
"YB" => SkipU64::MAX, // 1_000_000_000_000_000_000_000_000,
"Y" => SkipU64::MAX, // 1_208_925_819_614_629_174_706_176,
_ => {
return Err(format!(
"{}: invalid --ignore-initial value '{}'",
executable_str, skip_desc
"{executable_str}: invalid --ignore-initial value '{skip_desc}'"
));
}
};
num = match num.overflowing_mul(multiplier) {
(n, false) => n,
_ => usize::MAX,
_ => SkipU64::MAX,
}
}
@@ -165,13 +172,13 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
let (_, arg) = param_str.split_once('=').unwrap();
arg.to_string()
};
let max_bytes = match max_bytes.parse::<usize>() {
let max_bytes = match max_bytes.parse::<BytesLimitU64>() {
Ok(num) => num,
Err(e) if *e.kind() == std::num::IntErrorKind::PosOverflow => usize::MAX,
// TODO limit to MAX is dangerous, this should become an error like in GNU cmp.
Err(e) if *e.kind() == std::num::IntErrorKind::PosOverflow => BytesLimitU64::MAX,
Err(_) => {
return Err(format!(
"{}: invalid --bytes value '{}'",
executable_str, max_bytes
"{executable_str}: invalid --bytes value '{max_bytes}'"
))
}
};
@@ -210,7 +217,7 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
std::process::exit(0);
}
if param_str.starts_with('-') {
return Err(format!("Unknown option: {:?}", param));
return Err(format!("unrecognized option '{}'", param.to_string_lossy()));
}
if from.is_none() {
from = Some(param);
@@ -226,7 +233,7 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
}
// Do as GNU cmp, and completely disable printing if we are
// outputing to /dev/null.
// outputting to /dev/null.
#[cfg(not(target_os = "windows"))]
if is_stdout_dev_null() {
params.quiet = true;
@@ -236,8 +243,7 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
if params.quiet && params.verbose {
return Err(format!(
"{}: options -l and -s are incompatible",
executable_str
"{executable_str}: options -l and -s are incompatible"
));
}
@@ -279,32 +285,21 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
fn prepare_reader(
path: &OsString,
skip: &Option<usize>,
skip: &Option<SkipU64>,
params: &Params,
) -> Result<Box<dyn BufRead>, String> {
let mut reader: Box<dyn BufRead> = if path == "-" {
Box::new(BufReader::new(io::stdin()))
} else {
match fs::File::open(path) {
Ok(file) => Box::new(BufReader::new(file)),
Err(e) => {
return Err(format_failure_to_read_input_file(
&params.executable,
path,
&e,
));
}
}
let file = fs::File::open(path)
.map_err(|e| format_failure_to_read_input_file(&params.executable, path, &e))?;
Box::new(BufReader::new(file))
};
if let Some(skip) = skip {
if let Err(e) = io::copy(&mut reader.by_ref().take(*skip as u64), &mut io::sink()) {
return Err(format_failure_to_read_input_file(
&params.executable,
path,
&e,
));
}
// cast as u64 must remain, because value of IgnInit data type could be changed.
io::copy(&mut reader.by_ref().take(*skip), &mut io::sink())
.map_err(|e| format_failure_to_read_input_file(&params.executable, path, &e))?;
}
Ok(reader)
@@ -320,33 +315,44 @@ pub fn cmp(params: &Params) -> Result<Cmp, String> {
let mut from = prepare_reader(&params.from, &params.skip_a, params)?;
let mut to = prepare_reader(&params.to, &params.skip_b, params)?;
let mut at_byte = 1;
let mut at_line = 1;
let mut offset_width = params.max_bytes.unwrap_or(BytesLimitU64::MAX);
if let (Ok(a_meta), Ok(b_meta)) = (fs::metadata(&params.from), fs::metadata(&params.to)) {
#[cfg(not(target_os = "windows"))]
let (a_size, b_size) = (a_meta.len(), b_meta.len());
#[cfg(target_os = "windows")]
let (a_size, b_size) = (a_meta.file_size(), b_meta.file_size());
// If the files have different sizes, we already know they are not identical. If we have not
// been asked to show even the first difference, we can quit early.
if params.quiet && a_size != b_size {
return Ok(Cmp::Different);
}
let smaller = cmp::min(a_size, b_size) as BytesLimitU64;
offset_width = cmp::min(smaller, offset_width);
}
let offset_width = 1 + offset_width.checked_ilog10().unwrap_or(1) as usize;
// Capacity calc: at_byte width + 2 x 3-byte octal numbers + 2 x 4-byte value + 4 spaces
let mut output = Vec::<u8>::with_capacity(offset_width + 3 * 2 + 4 * 2 + 4);
let mut at_byte: BytesLimitU64 = 1;
let mut at_line: u64 = 1;
let mut start_of_line = true;
let mut verbose_diffs = vec![];
let mut stdout = BufWriter::new(io::stdout().lock());
let mut compare = Cmp::Equal;
loop {
// Fill up our buffers.
let from_buf = match from.fill_buf() {
Ok(buf) => buf,
Err(e) => {
return Err(format_failure_to_read_input_file(
&params.executable,
&params.from,
&e,
));
}
};
let from_buf = from
.fill_buf()
.map_err(|e| format_failure_to_read_input_file(&params.executable, &params.from, &e))?;
let to_buf = match to.fill_buf() {
Ok(buf) => buf,
Err(e) => {
return Err(format_failure_to_read_input_file(
&params.executable,
&params.to,
&e,
));
}
};
let to_buf = to
.fill_buf()
.map_err(|e| format_failure_to_read_input_file(&params.executable, &params.to, &e))?;
// Check for EOF conditions.
if from_buf.is_empty() && to_buf.is_empty() {
@@ -360,10 +366,6 @@ pub fn cmp(params: &Params) -> Result<Cmp, String> {
&params.to.to_string_lossy()
};
if params.verbose {
report_verbose_diffs(verbose_diffs, params)?;
}
report_eof(at_byte, at_line, start_of_line, eof_on, params);
return Ok(Cmp::Different);
}
@@ -374,8 +376,8 @@ pub fn cmp(params: &Params) -> Result<Cmp, String> {
if from_buf[..consumed] == to_buf[..consumed] {
let last = from_buf[..consumed].last().unwrap();
at_byte += consumed;
at_line += from_buf[..consumed].iter().filter(|&c| *c == b'\n').count();
at_byte += consumed as BytesLimitU64;
at_line += (from_buf[..consumed].iter().filter(|&c| *c == b'\n').count()) as u64;
start_of_line = *last == b'\n';
@@ -395,8 +397,24 @@ pub fn cmp(params: &Params) -> Result<Cmp, String> {
// first one runs out.
for (&from_byte, &to_byte) in from_buf.iter().zip(to_buf.iter()) {
if from_byte != to_byte {
compare = Cmp::Different;
if params.verbose {
verbose_diffs.push((at_byte, from_byte, to_byte));
format_verbose_difference(
from_byte,
to_byte,
at_byte,
offset_width,
&mut output,
params,
)?;
stdout.write_all(output.as_slice()).map_err(|e| {
format!(
"{}: error printing output: {e}",
params.executable.to_string_lossy()
)
})?;
output.clear();
} else {
report_difference(from_byte, to_byte, at_byte, at_line, params);
return Ok(Cmp::Different);
@@ -422,12 +440,7 @@ pub fn cmp(params: &Params) -> Result<Cmp, String> {
to.consume(consumed);
}
if params.verbose && !verbose_diffs.is_empty() {
report_verbose_diffs(verbose_diffs, params)?;
return Ok(Cmp::Different);
}
Ok(Cmp::Equal)
Ok(compare)
}
// Exit codes are documented at
@@ -450,21 +463,6 @@ pub fn main(opts: Peekable<ArgsOs>) -> ExitCode {
return ExitCode::SUCCESS;
}
// If the files have different sizes, we already know they are not identical. If we have not
// been asked to show even the first difference, we can quit early.
if params.quiet {
if let (Ok(a_meta), Ok(b_meta)) = (fs::metadata(&params.from), fs::metadata(&params.to)) {
#[cfg(not(target_os = "windows"))]
if a_meta.size() != b_meta.size() {
return ExitCode::from(1);
}
#[cfg(target_os = "windows")]
if a_meta.file_size() != b_meta.file_size() {
return ExitCode::from(1);
}
}
}
match cmp(&params) {
Ok(Cmp::Equal) => ExitCode::SUCCESS,
Ok(Cmp::Different) => ExitCode::from(1),
@@ -477,12 +475,6 @@ pub fn main(opts: Peekable<ArgsOs>) -> ExitCode {
}
}
#[inline]
fn is_ascii_printable(byte: u8) -> bool {
let c = byte as char;
c.is_ascii() && !c.is_ascii_control()
}
#[inline]
fn format_octal(byte: u8, buf: &mut [u8; 3]) -> &str {
*buf = [b' ', b' ', b'0'];
@@ -502,137 +494,149 @@ fn format_octal(byte: u8, buf: &mut [u8; 3]) -> &str {
}
#[inline]
fn format_byte(byte: u8) -> String {
let mut byte = byte;
let mut quoted = vec![];
if !is_ascii_printable(byte) {
if byte >= 128 {
quoted.push(b'M');
quoted.push(b'-');
byte -= 128;
fn write_visible_byte(output: &mut Vec<u8>, byte: u8) -> usize {
match byte {
// Control characters: ^@, ^A, ..., ^_
0..=31 => {
output.push(b'^');
output.push(byte + 64);
2
}
if byte < 32 {
quoted.push(b'^');
byte += 64;
} else if byte == 127 {
quoted.push(b'^');
byte = b'?';
// Printable ASCII (space through ~)
32..=126 => {
output.push(byte);
1
}
// DEL: ^?
127 => {
output.extend_from_slice(b"^?");
2
}
// High bytes with control equivalents: M-^@, M-^A, ..., M-^_
128..=159 => {
output.push(b'M');
output.push(b'-');
output.push(b'^');
output.push(byte - 64);
4
}
// High bytes: M-<space>, M-!, ..., M-~
160..=254 => {
output.push(b'M');
output.push(b'-');
output.push(byte - 128);
3
}
// Byte 255: M-^?
255 => {
output.extend_from_slice(b"M-^?");
4
}
assert!((byte as char).is_ascii());
}
}
quoted.push(byte);
/// Writes a byte in visible form with right-padding to 4 spaces.
#[inline]
fn write_visible_byte_padded(output: &mut Vec<u8>, byte: u8) {
const SPACES: &[u8] = b" ";
const WIDTH: usize = SPACES.len();
// SAFETY: the checks and shifts we do above match what cat and GNU
let display_width = write_visible_byte(output, byte);
// Add right-padding spaces
let padding = WIDTH.saturating_sub(display_width);
output.extend_from_slice(&SPACES[..padding]);
}
/// Formats a byte as a visible string (for non-performance-critical path)
#[inline]
fn format_visible_byte(byte: u8) -> String {
let mut result = Vec::with_capacity(4);
write_visible_byte(&mut result, byte);
// SAFETY: the checks and shifts in write_visible_byte match what cat and GNU
// cmp do to ensure characters fall inside the ascii range.
unsafe { String::from_utf8_unchecked(quoted) }
unsafe { String::from_utf8_unchecked(result) }
}
// This function has been optimized to not use the Rust fmt system, which
// leads to a massive speed up when processing large files: cuts the time
// for comparing 2 ~36MB completely different files in half on an M1 Max.
fn report_verbose_diffs(diffs: Vec<(usize, u8, u8)>, params: &Params) -> Result<(), String> {
#[inline]
fn format_verbose_difference(
from_byte: u8,
to_byte: u8,
at_byte: BytesLimitU64,
offset_width: usize,
output: &mut Vec<u8>,
params: &Params,
) -> Result<(), String> {
assert!(!params.quiet);
let mut stdout = BufWriter::new(io::stdout().lock());
if let Some((offset, _, _)) = diffs.last() {
// Obtain the width of the first column from the last byte offset.
let width = format!("{}", offset).len();
let mut at_byte_buf = itoa::Buffer::new();
let mut from_oct = [0u8; 3]; // for octal conversions
let mut to_oct = [0u8; 3];
let mut at_byte_buf = itoa::Buffer::new();
let mut from_oct = [0u8; 3]; // for octal conversions
let mut to_oct = [0u8; 3];
if params.print_bytes {
// "{:>width$} {:>3o} {:4} {:>3o} {}",
let at_byte_str = at_byte_buf.format(at_byte);
let at_byte_padding = offset_width.saturating_sub(at_byte_str.len());
// Capacity calc: at_byte width + 2 x 3-byte octal numbers + 4-byte value + up to 2 byte value + 4 spaces
let mut output = Vec::<u8>::with_capacity(width + 3 * 2 + 4 + 2 + 4);
if params.print_bytes {
for (at_byte, from_byte, to_byte) in diffs {
output.clear();
// "{:>width$} {:>3o} {:4} {:>3o} {}",
let at_byte_str = at_byte_buf.format(at_byte);
let at_byte_padding = width - at_byte_str.len();
for _ in 0..at_byte_padding {
output.push(b' ')
}
output.extend_from_slice(at_byte_str.as_bytes());
output.push(b' ');
output.extend_from_slice(format_octal(from_byte, &mut from_oct).as_bytes());
output.push(b' ');
let from_byte_str = format_byte(from_byte);
let from_byte_padding = 4 - from_byte_str.len();
output.extend_from_slice(from_byte_str.as_bytes());
for _ in 0..from_byte_padding {
output.push(b' ')
}
output.push(b' ');
output.extend_from_slice(format_octal(to_byte, &mut to_oct).as_bytes());
output.push(b' ');
output.extend_from_slice(format_byte(to_byte).as_bytes());
output.push(b'\n');
stdout.write_all(output.as_slice()).map_err(|e| {
format!(
"{}: error printing output: {e}",
params.executable.to_string_lossy()
)
})?;
}
} else {
for (at_byte, from_byte, to_byte) in diffs {
output.clear();
// "{:>width$} {:>3o} {:>3o}"
let at_byte_str = at_byte_buf.format(at_byte);
let at_byte_padding = width - at_byte_str.len();
for _ in 0..at_byte_padding {
output.push(b' ')
}
output.extend_from_slice(at_byte_str.as_bytes());
output.push(b' ');
output.extend_from_slice(format_octal(from_byte, &mut from_oct).as_bytes());
output.push(b' ');
output.extend_from_slice(format_octal(to_byte, &mut to_oct).as_bytes());
output.push(b'\n');
stdout.write_all(output.as_slice()).map_err(|e| {
format!(
"{}: error printing output: {e}",
params.executable.to_string_lossy()
)
})?;
}
for _ in 0..at_byte_padding {
output.push(b' ')
}
output.extend_from_slice(at_byte_str.as_bytes());
output.push(b' ');
output.extend_from_slice(format_octal(from_byte, &mut from_oct).as_bytes());
output.push(b' ');
write_visible_byte_padded(output, from_byte);
output.push(b' ');
output.extend_from_slice(format_octal(to_byte, &mut to_oct).as_bytes());
output.push(b' ');
write_visible_byte(output, to_byte);
output.push(b'\n');
} else {
// "{:>width$} {:>3o} {:>3o}"
let at_byte_str = at_byte_buf.format(at_byte);
let at_byte_padding = offset_width - at_byte_str.len();
for _ in 0..at_byte_padding {
output.push(b' ')
}
output.extend_from_slice(at_byte_str.as_bytes());
output.push(b' ');
output.extend_from_slice(format_octal(from_byte, &mut from_oct).as_bytes());
output.push(b' ');
output.extend_from_slice(format_octal(to_byte, &mut to_oct).as_bytes());
output.push(b'\n');
}
Ok(())
}
#[inline]
fn report_eof(at_byte: usize, at_line: usize, start_of_line: bool, eof_on: &str, params: &Params) {
fn report_eof(
at_byte: BytesLimitU64,
at_line: u64,
start_of_line: bool,
eof_on: &str,
params: &Params,
) {
if params.quiet {
return;
}
@@ -684,7 +688,13 @@ fn is_posix_locale() -> bool {
}
#[inline]
fn report_difference(from_byte: u8, to_byte: u8, at_byte: usize, at_line: usize, params: &Params) {
fn report_difference(
from_byte: u8,
to_byte: u8,
at_byte: BytesLimitU64,
at_line: u64,
params: &Params,
) {
if params.quiet {
return;
}
@@ -706,9 +716,9 @@ fn report_difference(from_byte: u8, to_byte: u8, at_byte: usize, at_line: usize,
print!(
" is {:>3o} {:char_width$} {:>3o} {:char_width$}",
from_byte,
format_byte(from_byte),
format_visible_byte(from_byte),
to_byte,
format_byte(to_byte)
format_visible_byte(to_byte)
);
}
println!();
@@ -781,7 +791,7 @@ mod tests {
from: os("foo"),
to: os("bar"),
skip_a: Some(1),
skip_b: Some(usize::MAX),
skip_b: Some(SkipU64::MAX),
..Default::default()
}),
parse_params(
@@ -959,7 +969,7 @@ mod tests {
executable: os("cmp"),
from: os("foo"),
to: os("bar"),
max_bytes: Some(usize::MAX),
max_bytes: Some(BytesLimitU64::MAX),
..Default::default()
}),
parse_params(
@@ -976,6 +986,7 @@ mod tests {
);
// Failure case
// TODO This is actually fine in GNU cmp. --bytes does not have a unit parser yet.
assert_eq!(
Err("cmp: invalid --bytes value '1K'".to_string()),
parse_params(
@@ -1021,8 +1032,8 @@ mod tests {
executable: os("cmp"),
from: os("foo"),
to: os("bar"),
skip_a: Some(usize::MAX),
skip_b: Some(usize::MAX),
skip_a: Some(SkipU64::MAX),
skip_b: Some(SkipU64::MAX),
..Default::default()
}),
parse_params(
@@ -1062,6 +1073,9 @@ mod tests {
from: os("foo"),
to: os("bar"),
skip_a: Some(1_000_000_000),
#[cfg(target_pointer_width = "32")]
skip_b: Some((2_147_483_647.5 * 2.0) as usize),
#[cfg(target_pointer_width = "64")]
skip_b: Some(1_152_921_504_606_846_976 * 2),
..Default::default()
}),
@@ -1093,8 +1107,12 @@ mod tests {
.enumerate()
{
let values = [
1_000usize.checked_pow((i + 1) as u32).unwrap_or(usize::MAX),
1024usize.checked_pow((i + 1) as u32).unwrap_or(usize::MAX),
(1_000 as SkipU64)
.checked_pow((i + 1) as u32)
.unwrap_or(SkipU64::MAX),
(1024 as SkipU64)
.checked_pow((i + 1) as u32)
.unwrap_or(SkipU64::MAX),
];
for (j, v) in values.iter().enumerate() {
assert_eq!(
+23 -16
View File
@@ -381,6 +381,9 @@ pub fn diff(expected: &[u8], actual: &[u8], params: &Params) -> Vec<u8> {
mod tests {
use super::*;
use pretty_assertions::assert_eq;
use crate::utils::testcmds::PATCH_CMD;
#[test]
fn test_permutations() {
// test all possible six-line files.
@@ -394,7 +397,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"b\n" })
@@ -429,12 +431,13 @@ mod tests {
}
// This test diff is intentionally reversed.
// We want it to turn the alef into bet.
let patched = &format!("{target}/alef");
let diff = diff(
&alef,
&bet,
&Params {
from: "a/alef".into(),
to: (&format!("{target}/alef")).into(),
from: patched.into(),
to: patched.into(),
context_count: 2,
..Default::default()
},
@@ -449,7 +452,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.arg("--context")
.stdin(File::open(format!("{target}/ab.diff")).unwrap())
@@ -481,7 +485,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"\n" } else { b"b\n" }).unwrap();
@@ -510,12 +513,13 @@ mod tests {
}
// This test diff is intentionally reversed.
// We want it to turn the alef into bet.
let patched = &format!("{target}/alef_");
let diff = diff(
&alef,
&bet,
&Params {
from: "a/alef_".into(),
to: (&format!("{target}/alef_")).into(),
from: patched.into(),
to: patched.into(),
context_count: 2,
..Default::default()
},
@@ -530,7 +534,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.arg("--context")
.stdin(File::open(format!("{target}/ab_.diff")).unwrap())
@@ -562,7 +567,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"" }).unwrap();
@@ -594,12 +598,13 @@ mod tests {
};
// This test diff is intentionally reversed.
// We want it to turn the alef into bet.
let patched = &format!("{target}/alefx");
let diff = diff(
&alef,
&bet,
&Params {
from: "a/alefx".into(),
to: (&format!("{target}/alefx")).into(),
from: patched.into(),
to: patched.into(),
context_count: 2,
..Default::default()
},
@@ -614,7 +619,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.arg("--context")
.stdin(File::open(format!("{target}/abx.diff")).unwrap())
@@ -646,7 +652,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"f\n" })
@@ -681,12 +686,13 @@ mod tests {
}
// This test diff is intentionally reversed.
// We want it to turn the alef into bet.
let alefr_path = &format!("{target}/alefr");
let diff = diff(
&alef,
&bet,
&Params {
from: "a/alefr".into(),
to: (&format!("{target}/alefr")).into(),
from: alefr_path.into(),
to: alefr_path.into(),
context_count: 2,
..Default::default()
},
@@ -701,7 +707,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.arg("--context")
.stdin(File::open(format!("{target}/abr.diff")).unwrap())
+6 -2
View File
@@ -5,11 +5,11 @@
use crate::params::{parse_params, Format};
use crate::utils::report_failure_to_read_input_file;
use crate::{context_diff, ed_diff, normal_diff, unified_diff};
use crate::{context_diff, ed_diff, normal_diff, side_diff, unified_diff};
use std::env::ArgsOs;
use std::ffi::OsString;
use std::fs;
use std::io::{self, Read, Write};
use std::io::{self, stdout, Read, Write};
use std::iter::Peekable;
use std::process::{exit, ExitCode};
@@ -79,6 +79,10 @@ pub fn main(opts: Peekable<ArgsOs>) -> ExitCode {
eprintln!("{error}");
exit(2);
}),
Format::SideBySide => {
let mut output = stdout().lock();
side_diff::diff(&from_content, &to_content, &mut output, &params)
}
};
if params.brief && !result.is_empty() {
println!(
+9 -6
View File
@@ -162,6 +162,9 @@ pub fn diff(expected: &[u8], actual: &[u8], params: &Params) -> Result<Vec<u8>,
mod tests {
use super::*;
use pretty_assertions::assert_eq;
use crate::utils::testcmds::ED_CMD;
pub fn diff_w(expected: &[u8], actual: &[u8], filename: &str) -> Result<Vec<u8>, DiffError> {
let mut output = diff(expected, actual, &Params::default())?;
writeln!(&mut output, "w {filename}").unwrap();
@@ -237,8 +240,8 @@ mod tests {
let _ = fb;
#[cfg(not(windows))] // there's no ed on windows
{
use std::process::Command;
let output = Command::new("ed")
let output = ED_CMD
.new()
.arg(format!("{target}/alef"))
.stdin(File::open(format!("{target}/ab.ed")).unwrap())
.output()
@@ -311,8 +314,8 @@ mod tests {
let _ = fb;
#[cfg(not(windows))] // there's no ed on windows
{
use std::process::Command;
let output = Command::new("ed")
let output = ED_CMD
.new()
.arg(format!("{target}/alef_"))
.stdin(File::open(format!("{target}/ab_.ed")).unwrap())
.output()
@@ -391,8 +394,8 @@ mod tests {
let _ = fb;
#[cfg(not(windows))] // there's no ed on windows
{
use std::process::Command;
let output = Command::new("ed")
let output = ED_CMD
.new()
.arg(format!("{target}/alefr"))
.stdin(File::open(format!("{target}/abr.ed")).unwrap())
.output()
+2
View File
@@ -4,6 +4,7 @@ pub mod ed_diff;
pub mod macros;
pub mod normal_diff;
pub mod params;
pub mod side_diff;
pub mod unified_diff;
pub mod utils;
@@ -11,4 +12,5 @@ pub mod utils;
pub use context_diff::diff as context_diff;
pub use ed_diff::diff as ed_diff;
pub use normal_diff::diff as normal_diff;
pub use side_diff::diff as side_by_side_diff;
pub use unified_diff::diff as unified_diff;
+12 -7
View File
@@ -18,6 +18,7 @@ mod ed_diff;
mod macros;
mod normal_diff;
mod params;
mod side_diff;
mod unified_diff;
mod utils;
@@ -57,7 +58,7 @@ fn main() -> ExitCode {
let exe_path = binary_path(&mut args);
let exe_name = name(&exe_path);
let util_name = if exe_name == "diffutils" {
let util_name = if exe_name.as_encoded_bytes().ends_with(b"diffutils") {
// Discard the item we peeked.
let _ = args.next();
@@ -68,13 +69,17 @@ fn main() -> ExitCode {
OsString::from(exe_name)
};
match util_name.to_str() {
Some("diff") => diff::main(args),
Some("cmp") => cmp::main(args),
Some(name) => {
eprintln!("{}: utility not supported", name);
match util_name.as_encoded_bytes() {
name if name.ends_with(b"diff") => diff::main(args),
name if name.ends_with(b"cmp") => cmp::main(args),
name => {
use std::io::{stderr, Write as _};
let _ = writeln!(
stderr(),
"{}: utility not supported",
String::from_utf8_lossy(name)
);
ExitCode::from(2)
}
None => second_arg_error(exe_name),
}
}
+10 -8
View File
@@ -215,6 +215,8 @@ mod tests {
use super::*;
use pretty_assertions::assert_eq;
use crate::utils::testcmds::PATCH_CMD;
#[test]
fn test_basic() {
let mut a = Vec::new();
@@ -239,7 +241,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"b\n" })
@@ -285,7 +286,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.arg(format!("{target}/alef"))
.stdin(File::open(format!("{target}/ab.diff")).unwrap())
@@ -318,7 +320,6 @@ mod tests {
for &g in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"b\n" })
@@ -377,7 +378,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.arg("--normal")
.arg(format!("{target}/alefn"))
@@ -411,7 +413,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"\n" } else { b"b\n" }).unwrap();
@@ -451,7 +452,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.arg(format!("{target}/alef_"))
.stdin(File::open(format!("{target}/ab_.diff")).unwrap())
@@ -483,7 +485,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"f\n" })
@@ -529,7 +530,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.arg(format!("{target}/alefr"))
.stdin(File::open(format!("{target}/abr.diff")).unwrap())
+61 -26
View File
@@ -11,6 +11,7 @@ pub enum Format {
Unified,
Context,
Ed,
SideBySide,
}
#[derive(Clone, Debug, Eq, PartialEq)]
@@ -24,6 +25,7 @@ pub struct Params {
pub brief: bool,
pub expand_tabs: bool,
pub tabsize: usize,
pub width: usize,
}
impl Default for Params {
@@ -38,6 +40,7 @@ impl Default for Params {
brief: false,
expand_tabs: false,
tabsize: 8,
width: 130,
}
}
}
@@ -57,6 +60,7 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
let mut format = None;
let mut context = None;
let tabsize_re = Regex::new(r"^--tabsize=(?<num>\d+)$").unwrap();
let width_re = Regex::new(r"--width=(?P<long>\d+)$").unwrap();
while let Some(param) = opts.next() {
let next_param = opts.peek();
if param == "--" {
@@ -101,6 +105,34 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
format = Some(Format::Ed);
continue;
}
if param == "-y" || param == "--side-by-side" {
if format.is_some() && format != Some(Format::SideBySide) {
return Err("Conflicting output style option".to_string());
}
format = Some(Format::SideBySide);
continue;
}
if width_re.is_match(param.to_string_lossy().as_ref()) {
let param = param.into_string().unwrap();
let width_str: &str = width_re
.captures(param.as_str())
.unwrap()
.name("long")
.unwrap()
.as_str();
params.width = match width_str.parse::<usize>() {
Ok(num) => {
if num == 0 {
return Err("invalid width «0»".to_string());
}
num
}
Err(_) => return Err(format!("invalid width «{width_str}»")),
};
continue;
}
if tabsize_re.is_match(param.to_string_lossy().as_ref()) {
// Because param matches the regular expression,
// it is safe to assume it is valid UTF-8.
@@ -112,9 +144,16 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
.unwrap()
.as_str();
params.tabsize = match tabsize_str.parse::<usize>() {
Ok(num) => num,
Ok(num) => {
if num == 0 {
return Err("invalid tabsize «0»".to_string());
}
num
}
Err(_) => return Err(format!("invalid tabsize «{tabsize_str}»")),
};
continue;
}
match match_context_diff_params(&param, next_param, format) {
@@ -156,7 +195,7 @@ pub fn parse_params<I: Iterator<Item = OsString>>(mut opts: Peekable<I>) -> Resu
Err(error) => return Err(error),
}
if param.to_string_lossy().starts_with('-') {
return Err(format!("Unknown option: {:?}", param));
return Err(format!("unrecognized option '{}'", param.to_string_lossy()));
}
if from.is_none() {
from = Some(param);
@@ -240,17 +279,15 @@ fn match_context_diff_params(
context_count = Some(numvalue.as_str().parse::<usize>().unwrap());
}
}
if param == "-C" && next_param.is_some() {
match next_param.unwrap().to_string_lossy().parse::<usize>() {
Ok(context_size) => {
context_count = Some(context_size);
next_param_consumed = true;
}
Err(_) => {
return Err(format!(
"invalid context length '{}'",
next_param.unwrap().to_string_lossy()
))
if param == "-C" {
if let Some(p) = next_param {
let size_str = p.to_string_lossy();
match size_str.parse::<usize>() {
Ok(context_size) => {
context_count = Some(context_size);
next_param_consumed = true;
}
Err(_) => return Err(format!("invalid context length '{size_str}'")),
}
}
}
@@ -286,17 +323,15 @@ fn match_unified_diff_params(
context_count = Some(numvalue.as_str().parse::<usize>().unwrap());
}
}
if param == "-U" && next_param.is_some() {
match next_param.unwrap().to_string_lossy().parse::<usize>() {
Ok(context_size) => {
context_count = Some(context_size);
next_param_consumed = true;
}
Err(_) => {
return Err(format!(
"invalid context length '{}'",
next_param.unwrap().to_string_lossy()
))
if param == "-U" {
if let Some(p) = next_param {
let size_str = p.to_string_lossy();
match size_str.parse::<usize>() {
Ok(context_size) => {
context_count = Some(context_size);
next_param_consumed = true;
}
Err(_) => return Err(format!("invalid context length '{size_str}'")),
}
}
}
@@ -704,11 +739,11 @@ mod tests {
executable: os("diff"),
from: os("foo"),
to: os("bar"),
tabsize: 0,
tabsize: 1,
..Default::default()
}),
parse_params(
[os("diff"), os("--tabsize=0"), os("foo"), os("bar")]
[os("diff"), os("--tabsize=1"), os("foo"), os("bar")]
.iter()
.cloned()
.peekable()
+1263
View File
File diff suppressed because it is too large Load Diff
+29 -20
View File
@@ -408,6 +408,8 @@ mod tests {
use super::*;
use pretty_assertions::assert_eq;
use crate::utils::testcmds::PATCH_CMD;
#[test]
fn test_permutations() {
let target = "target/unified-diff/";
@@ -421,7 +423,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"b\n" })
@@ -456,12 +457,13 @@ mod tests {
}
// This test diff is intentionally reversed.
// We want it to turn the alef into bet.
let patched = &format!("{target}/alef");
let diff = diff(
&alef,
&bet,
&Params {
from: "a/alef".into(),
to: (&format!("{target}/alef")).into(),
from: patched.into(),
to: patched.into(),
context_count: 2,
..Default::default()
},
@@ -492,7 +494,10 @@ mod tests {
.unwrap_or_else(|_| String::from("[Invalid UTF-8]"))
);
let output = Command::new("patch")
use crate::utils::testcmds::PATCH_CMD;
let output = PATCH_CMD
.new()
.arg("-p0")
.stdin(File::open(format!("{target}/ab.diff")).unwrap())
.output()
@@ -524,7 +529,6 @@ mod tests {
for &g in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"b\n" })
@@ -572,12 +576,13 @@ mod tests {
}
// This test diff is intentionally reversed.
// We want it to turn the alef into bet.
let patched = &format!("{target}/alefn");
let diff = diff(
&alef,
&bet,
&Params {
from: "a/alefn".into(),
to: (&format!("{target}/alefn")).into(),
from: patched.into(),
to: patched.into(),
context_count: 2,
..Default::default()
},
@@ -592,7 +597,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.stdin(File::open(format!("{target}/abn.diff")).unwrap())
.output()
@@ -625,7 +631,6 @@ mod tests {
for &g in &[0, 1, 2, 3] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"\n" } else { b"b\n" }).unwrap();
@@ -668,12 +673,13 @@ mod tests {
}
// This test diff is intentionally reversed.
// We want it to turn the alef into bet.
let patched = &format!("{target}/alef_");
let diff = diff(
&alef,
&bet,
&Params {
from: "a/alef_".into(),
to: (&format!("{target}/alef_")).into(),
from: patched.into(),
to: patched.into(),
context_count: 2,
..Default::default()
},
@@ -688,7 +694,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.stdin(File::open(format!("{target}/ab_.diff")).unwrap())
.output()
@@ -720,7 +727,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"" }).unwrap();
@@ -749,12 +755,13 @@ mod tests {
}
// This test diff is intentionally reversed.
// We want it to turn the alef into bet.
let patched = &format!("{target}/alefx");
let diff = diff(
&alef,
&bet,
&Params {
from: "a/alefx".into(),
to: (&format!("{target}/alefx")).into(),
from: patched.into(),
to: patched.into(),
context_count: 2,
..Default::default()
},
@@ -769,7 +776,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.stdin(File::open(format!("{target}/abx.diff")).unwrap())
.output()
@@ -800,7 +808,6 @@ mod tests {
for &f in &[0, 1, 2] {
use std::fs::{self, File};
use std::io::Write;
use std::process::Command;
let mut alef = Vec::new();
let mut bet = Vec::new();
alef.write_all(if a == 0 { b"a\n" } else { b"f\n" })
@@ -835,12 +842,13 @@ mod tests {
}
// This test diff is intentionally reversed.
// We want it to turn the alef into bet.
let patched = &format!("{target}/alefr");
let diff = diff(
&alef,
&bet,
&Params {
from: "a/alefr".into(),
to: (&format!("{target}/alefr")).into(),
from: patched.into(),
to: patched.into(),
context_count: 2,
..Default::default()
},
@@ -855,7 +863,8 @@ mod tests {
fb.write_all(&bet[..]).unwrap();
let _ = fa;
let _ = fb;
let output = Command::new("patch")
let output = PATCH_CMD
.new()
.arg("-p0")
.stdin(File::open(format!("{target}/abr.diff")).unwrap())
.output()
+94 -2
View File
@@ -3,9 +3,8 @@
// For the full copyright and license information, please view the LICENSE-*
// files that was distributed with this source code.
use std::{ffi::OsString, io::Write};
use regex::Regex;
use std::{ffi::OsString, io::Write};
use unicode_width::UnicodeWidthStr;
/// Replace tabs by spaces in the input line.
@@ -99,6 +98,99 @@ pub fn report_failure_to_read_input_file(
);
}
#[cfg(test)]
pub mod testcmds {
// Command construction wrapper that provides some validation and non-obscure, "fail fast"
// feedback and error messages.
use std::any::Any;
use std::io::Write;
use std::panic::catch_unwind;
use std::process::{Command, Stdio};
use std::sync::LazyLock;
pub struct CmdFactory {
cmd: &'static str,
validated_once: LazyLock<Result<(), String>>,
validate: fn(&CmdFactory) -> (),
}
impl CmdFactory {
pub fn new(&self) -> Command {
match &*self.validated_once {
Ok(()) => Command::new(self.cmd),
Err(errmsg) => panic!(
"'{}' validation failed in earlier thread/test: {}",
self.cmd, errmsg
),
}
}
// "self" is not compatible with static initialization
fn try_catch_validate(cmd: &CmdFactory) -> Result<(), String> {
// Note catch_unwind() does _not_ hide error messages, stack traces, etc.
catch_unwind(|| {
let _ = (cmd.validate)(cmd);
})
.map_err(find_panic_message)
}
}
fn find_panic_message(payload: Box<dyn Any + Send>) -> String {
// https://github.com/rust-lang/rust/blob/1.95.0/library/std/src/panicking.rs#L771
if let Some(&s) = payload.downcast_ref::<&'static str>() {
String::from(s)
} else if let Some(s) = payload.downcast_ref::<String>() {
s.clone()
} else {
format!(
"Unusual panic payload type {:?}, look for the first thread/test that failed",
payload.type_id(),
)
}
}
pub static PATCH_CMD: CmdFactory = CmdFactory {
cmd: if cfg!(target_os = "macos") {
"gpatch" // brew install gpatch
} else {
"patch"
},
validated_once: LazyLock::new(|| CmdFactory::try_catch_validate(&PATCH_CMD)),
validate: (|myself| {
let output = Command::new(myself.cmd)
.arg("--version")
.output()
.expect(format!("`{} --version` failed", myself.cmd).as_str());
// Non-GNU versions have subtle differences. When some newlines are missing in some test
// patches, the macOS version can even stall the whole test run.
assert!(output.stdout.starts_with(b"GNU patch"));
assert!(output.status.success());
}),
};
pub static ED_CMD: CmdFactory = CmdFactory {
cmd: "ed",
validated_once: LazyLock::new(|| CmdFactory::try_catch_validate(&ED_CMD)),
validate: (|myself| {
let mut child = Command::new(myself.cmd)
.arg("!echo hello_ed")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.expect("Failed to start 'ed' command");
let mut stdin = child.stdin.take().unwrap();
writeln!(stdin, "1p\nq").expect("Failed to send command to 'ed'");
let output = child
.wait_with_output()
.expect("Failed to read 'ed' stdout");
assert_eq!(String::from_utf8_lossy(&output.stdout), "9\nhello_ed\n");
}),
};
}
#[cfg(test)]
mod tests {
use super::*;
+75 -57
View File
@@ -3,9 +3,11 @@
// For the full copyright and license information, please view the LICENSE-*
// files that was distributed with this source code.
use assert_cmd::cmd::Command;
use assert_cmd::cargo::cargo_bin_cmd;
use predicates::prelude::*;
use std::fs::{File, OpenOptions};
use std::fs::File;
#[cfg(not(windows))]
use std::fs::OpenOptions;
use std::io::Write;
use tempfile::{tempdir, NamedTempFile};
@@ -15,14 +17,14 @@ mod common {
#[test]
fn unknown_param() -> Result<(), Box<dyn std::error::Error>> {
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("patch");
cmd.assert()
.code(predicate::eq(2))
.failure()
.stderr(predicate::eq("patch: utility not supported\n"));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.assert()
.code(predicate::eq(0))
.success()
@@ -30,14 +32,14 @@ mod common {
"Expected utility name as second argument, got nothing.\n",
));
for subcmd in ["diff", "cmp"] {
let mut cmd = Command::cargo_bin("diffutils")?;
for subcmd in ["diff", "cmp", "uu-diff", "uucmp"] {
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg(subcmd);
cmd.arg("--foobar");
cmd.assert()
.code(predicate::eq(2))
.failure()
.stderr(predicate::str::starts_with("Unknown option: \"--foobar\""));
.stderr(predicate::str::contains("unrecognized option '--foobar'"));
}
Ok(())
}
@@ -56,7 +58,7 @@ mod common {
let error_message = "The system cannot find the file specified.";
for subcmd in ["diff", "cmp"] {
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg(subcmd);
cmd.arg(&nopath).arg(file.path());
cmd.assert()
@@ -67,7 +69,7 @@ mod common {
&nopath.as_os_str().to_string_lossy()
)));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg(subcmd);
cmd.arg(file.path()).arg(&nopath);
cmd.assert()
@@ -79,7 +81,7 @@ mod common {
)));
}
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
cmd.arg(&nopath).arg(&nopath);
cmd.assert().code(predicate::eq(2)).failure().stderr(
@@ -103,7 +105,7 @@ mod diff {
fn no_differences() -> Result<(), Box<dyn std::error::Error>> {
let file = NamedTempFile::new()?;
for option in ["", "-u", "-c", "-e"] {
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
if !option.is_empty() {
cmd.arg(option);
@@ -123,7 +125,7 @@ mod diff {
let mut file1 = NamedTempFile::new()?;
file1.write_all("foo\n".as_bytes())?;
for option in ["", "-u", "-c", "-e"] {
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
if !option.is_empty() {
cmd.arg(option);
@@ -142,7 +144,7 @@ mod diff {
let mut file2 = NamedTempFile::new()?;
file2.write_all("foo\n".as_bytes())?;
for option in ["", "-u", "-c", "-e"] {
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
if !option.is_empty() {
cmd.arg(option);
@@ -167,7 +169,7 @@ mod diff {
let mut file2 = NamedTempFile::new()?;
file2.write_all("bar\n".as_bytes())?;
for option in ["", "-u", "-c", "-e"] {
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
if !option.is_empty() {
cmd.arg(option);
@@ -188,7 +190,7 @@ mod diff {
let mut file2 = NamedTempFile::new()?;
file2.write_all("bar\n".as_bytes())?;
for option in ["", "-u", "-c", "-e"] {
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
if !option.is_empty() {
cmd.arg(option);
@@ -212,7 +214,7 @@ mod diff {
file1.write_all("foo".as_bytes())?;
let mut file2 = NamedTempFile::new()?;
file2.write_all("bar".as_bytes())?;
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
cmd.arg("-e").arg(file1.path()).arg(file2.path());
cmd.assert()
@@ -229,7 +231,7 @@ mod diff {
let mut file2 = NamedTempFile::new()?;
file2.write_all("bar\n".as_bytes())?;
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
cmd.arg("-u")
.arg(file1.path())
@@ -246,7 +248,7 @@ mod diff {
)
);
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
cmd.arg("-u")
.arg("-")
@@ -263,7 +265,7 @@ mod diff {
)
);
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
cmd.arg("-u").arg("-").arg("-");
cmd.assert()
@@ -273,7 +275,7 @@ mod diff {
#[cfg(unix)]
{
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
cmd.arg("-u")
.arg(file1.path())
@@ -309,7 +311,7 @@ mod diff {
let mut da = File::create(&da_path).unwrap();
da.write_all(b"da\n").unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
cmd.arg("-u").arg(&directory).arg(&a_path);
cmd.assert().code(predicate::eq(1)).failure();
@@ -324,7 +326,7 @@ mod diff {
)
);
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("diff");
cmd.arg("-u").arg(&a_path).arg(&directory);
cmd.assert().code(predicate::eq(1)).failure();
@@ -348,7 +350,7 @@ mod cmp {
#[test]
fn cmp_incompatible_params() -> Result<(), Box<dyn std::error::Error>> {
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg("-l");
cmd.arg("-s");
@@ -371,7 +373,7 @@ mod cmp {
let mut a = File::create(&a_path).unwrap();
a.write_all(b"a\n").unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg(&a_path);
cmd.write_stdin("a\n");
@@ -381,7 +383,7 @@ mod cmp {
.stderr(predicate::str::is_empty())
.stdout(predicate::str::is_empty());
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg(&a_path);
@@ -407,7 +409,7 @@ mod cmp {
let mut b = File::create(&b_path).unwrap();
b.write_all(b"a\n").unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg(&a_path).arg(&b_path);
cmd.assert()
@@ -430,7 +432,7 @@ mod cmp {
let b_path = tmp_dir.path().join("b");
let _ = File::create(&b_path).unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg(&a_path).arg(&b_path);
cmd.assert()
@@ -454,7 +456,7 @@ mod cmp {
let mut b = File::create(&b_path).unwrap();
b.write_all(b"bcd\n").unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg(&a_path).arg(&b_path);
@@ -463,7 +465,7 @@ mod cmp {
.failure()
.stdout(predicate::str::ends_with(" differ: char 1, line 1\n"));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg("-b");
@@ -476,7 +478,7 @@ mod cmp {
" differ: byte 1, line 1 is 141 a 142 b\n",
));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg("-l");
@@ -487,7 +489,7 @@ mod cmp {
.stderr(predicate::str::is_empty())
.stdout(predicate::eq("1 141 142\n2 142 143\n3 143 144\n"));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg("-l");
@@ -516,7 +518,7 @@ mod cmp {
let mut b = File::create(&b_path).unwrap();
b.write_all(b"abc\ndef\ng").unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg(&a_path).arg(&b_path);
@@ -526,7 +528,7 @@ mod cmp {
.stderr(predicate::str::is_empty())
.stdout(predicate::str::ends_with(" differ: char 8, line 2\n"));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg("-b");
@@ -539,7 +541,7 @@ mod cmp {
" differ: byte 8, line 2 is 147 g 12 ^J\n",
));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg("-l");
@@ -551,7 +553,7 @@ mod cmp {
.stderr(predicate::str::contains(" EOF on"))
.stderr(predicate::str::ends_with(" after byte 8\n"));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg("-b");
@@ -579,7 +581,7 @@ mod cmp {
let mut b = File::create(&b_path).unwrap();
b.write_all(b"abcdefghijkl\n").unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg("-l");
cmd.arg("-b");
@@ -592,7 +594,7 @@ mod cmp {
.stderr(predicate::str::is_empty())
.stdout(predicate::str::is_empty());
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg("-l");
cmd.arg("-b");
@@ -605,7 +607,7 @@ mod cmp {
.stderr(predicate::str::is_empty())
.stdout(predicate::eq("4 40 144 d\n"));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg("-l");
cmd.arg("-b");
@@ -616,7 +618,7 @@ mod cmp {
.code(predicate::eq(1))
.failure()
.stderr(predicate::str::is_empty())
.stdout(predicate::eq("4 40 144 d\n8 40 150 h\n"));
.stdout(predicate::eq(" 4 40 144 d\n 8 40 150 h\n"));
Ok(())
}
@@ -632,7 +634,7 @@ mod cmp {
let mut b = File::create(&b_path).unwrap();
b.write_all(b"###abc\n").unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg("-i");
@@ -645,7 +647,7 @@ mod cmp {
.stdout(predicate::str::is_empty());
// Positional skips should be ignored
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg("-i");
@@ -659,7 +661,7 @@ mod cmp {
.stdout(predicate::str::is_empty());
// Single positional argument should only affect first file.
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg(&a_path).arg(&b_path);
@@ -670,7 +672,7 @@ mod cmp {
.stderr(predicate::str::is_empty())
.stdout(predicate::str::ends_with(" differ: char 1, line 1\n"));
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.env("LC_ALL", "C");
cmd.arg("cmp");
cmd.arg(&a_path).arg(&b_path);
@@ -691,15 +693,15 @@ mod cmp {
let a_path = tmp_dir.path().join("a");
let mut a = File::create(&a_path).unwrap();
write!(a, "{}c\n", "a".repeat(1024)).unwrap();
writeln!(a, "{}c", "a".repeat(1024)).unwrap();
a.flush().unwrap();
let b_path = tmp_dir.path().join("b");
let mut b = File::create(&b_path).unwrap();
write!(b, "{}c\n", "b".repeat(1024)).unwrap();
writeln!(b, "{}c", "b".repeat(1024)).unwrap();
b.flush().unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg("--ignore-initial=1K");
cmd.arg(&a_path).arg(&b_path);
@@ -724,7 +726,7 @@ mod cmp {
let mut b = File::create(&b_path).unwrap();
b.write_all(b"abcdefghijkl\n").unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg("-l");
cmd.arg("-b");
@@ -737,7 +739,7 @@ mod cmp {
.stderr(predicate::str::is_empty())
.stdout(predicate::str::is_empty());
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg("-b");
cmd.arg("-i");
@@ -770,7 +772,7 @@ mod cmp {
let mut b = File::create(&b_path).unwrap();
b.write_all(&bytes).unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg("-l");
cmd.arg("-b");
@@ -815,7 +817,7 @@ mod cmp {
let dev_null = OpenOptions::new().write(true).open("/dev/null").unwrap();
let mut child = std::process::Command::new(assert_cmd::cargo::cargo_bin("diffutils"))
let mut child = std::process::Command::new(assert_cmd::cargo::cargo_bin!("diffutils"))
.arg("cmp")
.arg(&a_path)
.arg(&b_path)
@@ -823,12 +825,27 @@ mod cmp {
.spawn()
.unwrap();
std::thread::sleep(std::time::Duration::from_millis(100));
assert_eq!(child.try_wait().unwrap().unwrap().code(), Some(1));
// Bound the runtime to a very short time that still allows for some resource
// constraint to slow it down while also allowing very fast systems to exit as
// early as possible.
const MAX_TRIES: u8 = 50;
for tries in 0..=MAX_TRIES {
if tries == MAX_TRIES {
panic!("cmp took too long to run, /dev/null optimization probably not working")
}
match child.try_wait() {
Ok(Some(status)) => {
assert_eq!(status.code(), Some(1));
break;
}
Ok(None) => (),
Err(e) => panic!("{e:#?}"),
}
std::thread::sleep(std::time::Duration::from_millis(10));
}
// Two stdins should be equal
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg("-");
cmd.arg("-");
@@ -851,17 +868,18 @@ mod cmp {
let a_path = tmp_dir.path().join("a");
let mut a = File::create(&a_path).unwrap();
a.write_all(&bytes).unwrap();
a.write_all(bytes).unwrap();
a.write_all(b"A").unwrap();
let b_path = tmp_dir.path().join("b");
let mut b = File::create(&b_path).unwrap();
b.write_all(&bytes).unwrap();
b.write_all(bytes).unwrap();
b.write_all(b"B").unwrap();
let mut cmd = Command::cargo_bin("diffutils")?;
let mut cmd = cargo_bin_cmd!("diffutils");
cmd.arg("cmp");
cmd.arg(&a_path).arg(&b_path);
cmd.env("LC_ALL", "en_US");
cmd.assert()
.code(predicate::eq(1))
.failure()
+1 -1
View File
@@ -62,7 +62,7 @@ cd ../tests
# Fetch tests/init.sh from the gnulib repository (needed since
# https://git.savannah.gnu.org/cgit/diffutils.git/commit/tests?id=1d2456f539)
curl -s "$gitserver/gitweb/?p=gnulib.git;a=blob_plain;f=tests/init.sh;hb=HEAD" -o init.sh
curl -sL "$gitserver/gitweb/?p=gnulib.git;a=blob_plain;f=tests/init.sh;hb=HEAD" -o init.sh
if [[ -n "$TESTS" ]]
then
+158
View File
@@ -0,0 +1,158 @@
#!/usr/bin/env python3
"""
Compare the current GNU test results to the last results gathered from the main branch to
highlight if a PR is making the results better/worse.
Don't exit with error code if all failing tests are in the ignore-intermittent.txt list.
"""
import json
import sys
import argparse
from pathlib import Path
def load_ignore_list(ignore_file):
"""Load list of intermittent test names to ignore from file."""
ignore_set = set()
if ignore_file and Path(ignore_file).exists():
with open(ignore_file, "r") as f:
for line in f:
line = line.strip()
if line and not line.startswith("#"):
ignore_set.add(line)
return ignore_set
def extract_test_results(json_data):
"""Extract test results from a diffutils test-results.json.
Note: unlike sed, diffutils JSON has no 'summary' object results are
computed from the 'tests' array using the 'result' and 'test' fields.
"""
tests = json_data.get("tests", [])
passed = sum(1 for t in tests if t.get("result") == "PASS")
failed = sum(1 for t in tests if t.get("result") == "FAIL")
skipped = sum(1 for t in tests if t.get("result") == "SKIP")
summary = {"total": len(tests), "passed": passed, "failed": failed, "skipped": skipped}
failed_tests = [t["test"] for t in tests if t.get("result") == "FAIL"]
return summary, failed_tests
def compare_results(current_file, reference_file, ignore_file=None, output_file=None):
"""Compare current results with reference results."""
ignore_set = load_ignore_list(ignore_file)
try:
with open(current_file, "r") as f:
current_data = json.load(f)
current_summary, current_failed = extract_test_results(current_data)
except Exception as e:
print(f"Error loading current results: {e}")
return 1
try:
with open(reference_file, "r") as f:
reference_data = json.load(f)
reference_summary, reference_failed = extract_test_results(reference_data)
except Exception as e:
print(f"Error loading reference results: {e}")
return 1
# Calculate differences
pass_diff = int(current_summary.get("passed", 0)) - int(reference_summary.get("passed", 0))
fail_diff = int(current_summary.get("failed", 0)) - int(reference_summary.get("failed", 0))
total_diff = int(current_summary.get("total", 0)) - int(reference_summary.get("total", 0))
# Find new failures and improvements
current_failed_set = set(current_failed)
reference_failed_set = set(reference_failed)
new_failures = current_failed_set - reference_failed_set
improvements = reference_failed_set - current_failed_set
# Filter out intermittent failures
non_intermittent_new_failures = new_failures - ignore_set
# Check if results are identical (no changes)
no_changes = (
pass_diff == 0
and fail_diff == 0
and total_diff == 0
and not new_failures
and not improvements
)
# If no changes, write empty output to prevent comment posting
if no_changes:
if output_file:
with open(output_file, "w") as f:
f.write("")
return 0
# Prepare output message
output_lines = []
output_lines.append("Test results comparison:")
output_lines.append(
f" Current: TOTAL: {current_summary.get('total', 0)} / PASSED: {current_summary.get('passed', 0)} / FAILED: {current_summary.get('failed', 0)} / SKIPPED: {current_summary.get('skipped', 0)}"
)
output_lines.append(
f" Reference: TOTAL: {reference_summary.get('total', 0)} / PASSED: {reference_summary.get('passed', 0)} / FAILED: {reference_summary.get('failed', 0)} / SKIPPED: {reference_summary.get('skipped', 0)}"
)
output_lines.append("")
if pass_diff != 0 or fail_diff != 0 or total_diff != 0:
output_lines.append("Changes from main branch:")
output_lines.append(f" TOTAL: {total_diff:+d}")
output_lines.append(f" PASSED: {pass_diff:+d}")
output_lines.append(f" FAILED: {fail_diff:+d}")
output_lines.append("")
if new_failures:
output_lines.append(f"New test failures ({len(new_failures)}):")
for test in sorted(new_failures):
if test in ignore_set:
output_lines.append(f" - {test} (intermittent)")
else:
output_lines.append(f" - {test}")
output_lines.append("")
if improvements:
output_lines.append(f"Test improvements ({len(improvements)}):")
for test in sorted(improvements):
output_lines.append(f" + {test}")
output_lines.append("")
output_text = "\n".join(output_lines)
if output_file:
with open(output_file, "w") as f:
f.write(output_text)
else:
print(output_text)
if non_intermittent_new_failures:
print(
f"ERROR: Found {len(non_intermittent_new_failures)} new non-intermittent test failures"
)
return 1
return 0
def main():
parser = argparse.ArgumentParser(description="Compare GNU diffutils test results")
parser.add_argument("current", help="Current test results JSON file")
parser.add_argument("reference", help="Reference test results JSON file")
parser.add_argument(
"--ignore-file", help="File containing intermittent test names to ignore"
)
parser.add_argument("--output", help="Output file for comparison results")
args = parser.parse_args()
return compare_results(args.current, args.reference, args.ignore_file, args.output)
if __name__ == "__main__":
sys.exit(main())