I sampled > 10 random issues. Most of them involve symlinks (these are always fun, ha?). quite a few non-UTF-8 handling issues. Some of the issues seem ... kind of basic, even surprising. E.g... discarding errors seems like a easy "don't do that" kind of a bug.
Don't want to be too harsh, as it's always easy to judge, but it does seem like not at level of maturity required. Which is a check-and-egg problem too.
BTW. What's on my mind immediately is how feasible it is to do some kind of a file-system-centring fuzzing-like suite, to run new and original implementations side by side to detect mismatches automatically.
I find it very disturbing that Ubuntu shipped this in LTS. Yes, the found issues have presumably been fixed, but the amount of very basic issues that were found casts a bad look on this project.
I find it very disturbing that Ubuntu shipped this in LTS.
I'm not so sure. It was shipped first in 25.10, has had a security audit (and addressed issues) and continues to use GNU coreutils for commands with known showstoppers.
An LTS's stability is part "does this work", and part "how costly is this to support for 5+ years".
GNU coreutils has no LTS release, which means Canonical defacto provide that LTS for 5+ years.
Since Canonical are already funding development of rust-coreutils, I can see why they might want to avoid those coreutils maintenance costs too. That will allow them to spend effort in other places.
had a security audit (and addressed issues) and continues to use GNU coreutils for commands with known showstoppers.
The security audit found so many issues that the uutils team were unable to fix them all before the next LTS deadline They were only able to fix ~half. And Ubuntu☨ decided to ship anyway.
Yes, uutils has done tons of good work (writing a test suite from scratch as GNU coreutils has little tests AFAIU), is good citizen as it collaborates with GNU coreutils. I think long term uutils is going to be a success.
But going ahead with the LTS realize looks like a C-level (uutils adoption is happening because the VP of engineering IIRC) want to tick off a checkbox to show 'results' now, consequences be damned. Treating Ubuntu user's like their QA. More bad press is likely to come out of this decision, let's hope the peanunt gallery doesn't point the finger at the uutils project or Rust. I wish they would have dropped uutils from the LTS release and continue their approach of shipping it on non-LTS releases and target the next LTS release instead.
☨ I want to emphasize that the party behaving irresponsibly here is Ubuntu, not the uutils maintainer, who is doing the best they can.
How did that pass code review in the first place? TOCTOU is a pretty basic thing to be aware when writing core system utilities. <rhetoric:inflammatory>Maybe rust makes you more blind to certain race conditions because the compiler handles a lot of them for you.</rhetoric:inflammatory>
The sooner rust-coreutils is default, the sooner they can remove coreutils from future LTSs. It's easier to maintain just one.
Further, when rust-coreutils gets more mature, they might even choose to make coreutils an empty package that depends on rust-coreutils. That sounds controversial today, but in 5 years, maybe not so.
Few distributions have the burden of 15 years maintenance.
Why would it be easier to backport fixes to a rather new Rust project, than a very mature C project? One of them is still changing quickly, making backporting much harder than coreutils which probably has a new release every 3 years.
It's easier to backport fixes to one project, so if they're already committing to rust-coreutils, the sooner they can drop LTS support, the easier it is.
That statistic (# of issues found) is meaningless to me without further information. Ubuntu commissioned a security firm to search for issues; that they found some (mostly minor) isn't too surprising.
To assess it I would need to know things like the size and complexity of the code being analyzed. Or the number of issues found in some other existing implementation.
(To be clear, If the issues had not been addressed I would have a whole different level of concern, but since they WERE addressed the remaining concern is "we found that many, think how many more we didn't find yet".)
The issues are not mostly minor, they betray a problematic level of knowledge on the part of the developers about how to properly work with Unix filesystem APIs.
It’s almost like it would be beneficial to refer to the GNU implementations to see how they work. But they are totally not doing that, of course, because MIT.
Interesting that they assigned a CVE to that one. I agree that it is a serious issue that should be fixed (and assigning a CVE may accomplish that), but I fail to see how someone could exploit this.
You'd probably first have to cause an error on the backup medium, wait for the backup script to run, wait untill the original is deleted (or cause a failure on the source), and then you didn't accomplish anything extra, other than perhaps the admin not noticing the backup failed. If the tool notices the error the backup would've still failed, and the data would still be lost...
I can't help but feel that uutils/coreutils is going about this the wrong way.
They're attempting a breadth-first conversion where they have dozens of partially-implemented replacements for the GNU coreutils, but this introduces a lot of compatibility risk because the GNU coreutils are mostly unspecified and untested. And the code they've got does not fill me with confidence, it's full of unwrap() which leads to bug reports like https://github.com/uutils/coreutils/issues/5371 plus they're doing the "hackathon tech demo" thing of mixing tests into the implementation source file.
Ideally there'd be more of a focus on a small batch of tools, getting those rock solid, and then moving on to the next batch. It's not like there's an urgent need to prioritize migrating in one big go, the system runs fine with a mixture of uutils and GNU.
It's not unusual for Rust because a lot of Rust packages are hackathon tech demos, but if you're writing code for production use then you want separate _test.rs files so that:
code coverage metrics can be accurately calculated,
you don't mix test-only dependencies into the normal build, and
you can iterate on the test suite without having to rebuild the system-under-test on each iteration.
Like, it's not a fatal flaw, but it's sort of a "hmm really?" that I am not super excited to see in code with the specific role of POSIX core functionality.
All of the things you list are and always have been solved by Cargo. Modules are not necessarily files in Rust. Tests are usually compiled only with a specfic build configuration, so their sources do not influence coverage metrics. Dev-only dependencies are clearly separated from regular dependencies in Cargo.toml. Rust translation units are crates, not files.
To avoid rebuilding a system under test, Cargo provides integration tests, which are external to a crate.
It's not unusual for Rust because a lot of Rust packages are hackathon tech demos
This is a pretty unsubstantiated claim, so I'm going to ignore it.
code coverage metrics can be accurately calculated
This works perfectly fine with inline tests
you don't mix test-only dependencies into the normal build
test-dependencies is a different section in Cargo.toml. This is a build configuration issue.
you can iterate on the test suite without having to rebuild the system-under-test on each iteration.
The places where you deploy inline tests will most likely require a rebuild of the module anyways or you will not run into large issues.
Here's the thing: Rust has very strong visibility and isolation rules. In particular, any internal state needs to be behind a privacy boundary. Testing units of that kind often benefits from being able to peek inside. Rust often chooses to not e.g. manipulate visibility boundaries to spy into the system under the test, but rather write tests inside that visibility boundary. And that's what in-module tests are often used for.
All test forms in Rust exist for a reason. I'll give you that knowing which one to choose when needs a bit of thinking, but just going and calling a project a hackathon project because they use in-module tests is not par for the course. And forgive me if I'm a bit salty if it happens to be my home course.
Ideally there'd be more of a focus on a small batch of tools, getting those rock solid, and then moving on to the next batch. It's not like there's an urgent need to prioritize migrating in one big go, the system runs fine with a mixture of uutils and GNU.
I think it's difficult to set priorities on volunteer contributions in OSS. I would find it difficult to reject a PR with a 90% implementation of ls until cp was 100% done. After all, many early adopters would be happy with 90% done, and there's value in getting early adopters actually using tools, in part to identify the remaining gap.
Distributions such as Ubuntu are well placed to select the subset that are sufficiently compatible.
Off-topic/tangential: does anybody know what is up with Discord only showing "HTML content omitted because you are logged in or using a modern mobile device." ? Sure, I am on mobile with JS turned off; but the entire discussion displays just fine in HTML if I request the desktop site.
It is clear from the outcome that Discord directorship forbids supporting plain HTML on mobile[1]. So my best guess is that this message is by employees who thinks that is bullshit, doing their best to tell the users how to get around that. If so: thank you, lonely user-minded Discord employees <3
[1] Let's not dignify such behaviour by calling it 'leading'.
I sampled > 10 random issues. Most of them involve symlinks (these are always fun, ha?). quite a few non-UTF-8 handling issues. Some of the issues seem ... kind of basic, even surprising. E.g... discarding errors seems like a easy "don't do that" kind of a bug.
Don't want to be too harsh, as it's always easy to judge, but it does seem like not at level of maturity required. Which is a check-and-egg problem too.
BTW. What's on my mind immediately is how feasible it is to do some kind of a file-system-centring fuzzing-like suite, to run new and original implementations side by side to detect mismatches automatically.
I find it very disturbing that Ubuntu shipped this in LTS. Yes, the found issues have presumably been fixed, but the amount of very basic issues that were found casts a bad look on this project.
I'm not so sure. It was shipped first in 25.10, has had a security audit (and addressed issues) and continues to use GNU coreutils for commands with known showstoppers.
An LTS's stability is part "does this work", and part "how costly is this to support for 5+ years".
GNU coreutils has no LTS release, which means Canonical defacto provide that LTS for 5+ years.
Since Canonical are already funding development of rust-coreutils, I can see why they might want to avoid those coreutils maintenance costs too. That will allow them to spend effort in other places.
The security audit found so many issues that the uutils team were unable to fix them all before the next LTS deadline They were only able to fix ~half. And Ubuntu☨ decided to ship anyway.
As ~mxey mentions the issues found reveal the code has a long way to go to reach maturity. f/e take a look at https://github.com/uutils/coreutils/issues/10011. The bug is that cp creates the files with the broad permissions only to reduce them later! Similar problem with mkfifo on https://github.com/uutils/coreutils/issues/10020 !!
Yes, uutils has done tons of good work (writing a test suite from scratch as GNU coreutils has little tests AFAIU), is good citizen as it collaborates with GNU coreutils. I think long term uutils is going to be a success.
But going ahead with the LTS realize looks like a C-level (uutils adoption is happening because the VP of engineering IIRC) want to tick off a checkbox to show 'results' now, consequences be damned. Treating Ubuntu user's like their QA. More bad press is likely to come out of this decision, let's hope the peanunt gallery doesn't point the finger at the uutils project or Rust. I wish they would have dropped uutils from the LTS release and continue their approach of shipping it on non-LTS releases and target the next LTS release instead.
☨ I want to emphasize that the party behaving irresponsibly here is Ubuntu, not the uutils maintainer, who is doing the best they can.
Ubuntu 26.04 uses GNU coreutils'
cp.Re the mkfifo issue, the security audit mentions this:
I interpret "production branch" as being "Ubuntu's fork that is shipped".
I tried to check it had been included in Ubuntu's fork but I'm too unfamiliar with launchpad and Ubuntu branches to know where to look.
How did that pass code review in the first place? TOCTOU is a pretty basic thing to be aware when writing core system utilities.
<rhetoric:inflammatory>Maybe rust makes you more blind to certain race conditions because the compiler handles a lot of them for you.</rhetoric:inflammatory>I don't think there's much backporting need in coreutils, and virtually every other distro also uses coreutils.
Also, GNU coreutils is still part of Ubuntu, just not enabled by default, except for cp/mv.
It seems easier to backport necessary fixes to cp and mv, than to backport necessary fixes to all of coreutils.
coreutils still exists as a package in Ubuntu 26.04, with all the programs. Unless they exempt a package in the main archive from security support, they will have to fix it.
Also, they have already been primarily tracking Debian's package, and you can see how little there is going on with it.
The sooner rust-coreutils is default, the sooner they can remove coreutils from future LTSs. It's easier to maintain just one.
Further, when rust-coreutils gets more mature, they might even choose to make coreutils an empty package that depends on rust-coreutils. That sounds controversial today, but in 5 years, maybe not so.
Few distributions have the burden of 15 years maintenance.
Every other distro uses GNU coreutils, but few provide near 15 years of support.
Red Hat do, but last I knew, they funded the GNU Coreutils maintainer, and so they might have in-house experience to do it.
Yes, perhaps with the experience provided this LTS, then by next LTS they will be able to fully remove it.
Why would it be easier to backport fixes to a rather new Rust project, than a very mature C project? One of them is still changing quickly, making backporting much harder than coreutils which probably has a new release every 3 years.
It's easier to backport fixes to one project, so if they're already committing to rust-coreutils, the sooner they can drop LTS support, the easier it is.
Ok, then we were just talking past each other. I meant they shouldn’t have put the Rust version into LTS at all and only have the GNU coreutils.
That statistic (# of issues found) is meaningless to me without further information. Ubuntu commissioned a security firm to search for issues; that they found some (mostly minor) isn't too surprising.
To assess it I would need to know things like the size and complexity of the code being analyzed. Or the number of issues found in some other existing implementation.
(To be clear, If the issues had not been addressed I would have a whole different level of concern, but since they WERE addressed the remaining concern is "we found that many, think how many more we didn't find yet".)
The issues are not mostly minor, they betray a problematic level of knowledge on the part of the developers about how to properly work with Unix filesystem APIs.
It’s almost like it would be beneficial to refer to the GNU implementations to see how they work. But they are totally not doing that, of course, because MIT.
Interesting that they assigned a CVE to that one. I agree that it is a serious issue that should be fixed (and assigning a CVE may accomplish that), but I fail to see how someone could exploit this.
You'd probably first have to cause an error on the backup medium, wait for the backup script to run, wait untill the original is deleted (or cause a failure on the source), and then you didn't accomplish anything extra, other than perhaps the admin not noticing the backup failed. If the tool notices the error the backup would've still failed, and the data would still be lost...
Oh. Right. I forgot CVEs are kind of supposed to be for security issues
I can't help but feel that
uutils/coreutilsis going about this the wrong way.They're attempting a breadth-first conversion where they have dozens of partially-implemented replacements for the GNU coreutils, but this introduces a lot of compatibility risk because the GNU coreutils are mostly unspecified and untested. And the code they've got does not fill me with confidence, it's full of
unwrap()which leads to bug reports like https://github.com/uutils/coreutils/issues/5371 plus they're doing the "hackathon tech demo" thing of mixing tests into the implementation source file.Ideally there'd be more of a focus on a small batch of tools, getting those rock solid, and then moving on to the next batch. It's not like there's an urgent need to prioritize migrating in one big go, the system runs fine with a mixture of uutils and GNU.
This is not unusual in Rust. Unit tests are usually in the source file where the code lives. This has multiple reasons, among them visibility.
It's not unusual for Rust because a lot of Rust packages are hackathon tech demos, but if you're writing code for production use then you want separate
_test.rsfiles so that:Like, it's not a fatal flaw, but it's sort of a "hmm really?" that I am not super excited to see in code with the specific role of POSIX core functionality.
All of the things you list are and always have been solved by Cargo. Modules are not necessarily files in Rust. Tests are usually compiled only with a specfic build configuration, so their sources do not influence coverage metrics. Dev-only dependencies are clearly separated from regular dependencies in Cargo.toml. Rust translation units are crates, not files.
To avoid rebuilding a system under test, Cargo provides integration tests, which are external to a crate.
This is a pretty unsubstantiated claim, so I'm going to ignore it.
This works perfectly fine with inline tests
test-dependencies is a different section in Cargo.toml. This is a build configuration issue.
The places where you deploy inline tests will most likely require a rebuild of the module anyways or you will not run into large issues.
Here's the thing: Rust has very strong visibility and isolation rules. In particular, any internal state needs to be behind a privacy boundary. Testing units of that kind often benefits from being able to peek inside. Rust often chooses to not e.g. manipulate visibility boundaries to spy into the system under the test, but rather write tests inside that visibility boundary. And that's what in-module tests are often used for.
All test forms in Rust exist for a reason. I'll give you that knowing which one to choose when needs a bit of thinking, but just going and calling a project a hackathon project because they use in-module tests is not par for the course. And forgive me if I'm a bit salty if it happens to be my home course.
You can have dev-only dependencies that don't show up in the normal build but are only visible in the test modules.
I think it's difficult to set priorities on volunteer contributions in OSS. I would find it difficult to reject a PR with a 90% implementation of
lsuntilcpwas 100% done. After all, many early adopters would be happy with 90% done, and there's value in getting early adopters actually using tools, in part to identify the remaining gap.Distributions such as Ubuntu are well placed to select the subset that are sufficiently compatible.
For comparison: The Debian security tracker for GNU coreutils
Surprisingly few issues for such a new project.
I still wouldn't want to rely on it in production though.
The project goes back to 2014. Wit development picking up on 2021. It is not a new project.
Off-topic/tangential: does anybody know what is up with Discord only showing "HTML content omitted because you are logged in or using a modern mobile device." ? Sure, I am on mobile with JS turned off; but the entire discussion displays just fine in HTML if I request the desktop site.
It is clear from the outcome that Discord directorship forbids supporting plain HTML on mobile[1]. So my best guess is that this message is by employees who thinks that is bullshit, doing their best to tell the users how to get around that. If so: thank you, lonely user-minded Discord employees <3
[1] Let's not dignify such behaviour by calling it 'leading'.