Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It still lacks basic functionality like a JSON parser, regex, directory walker, rnd generator and cli arg parsing.

I won't mention lack of date/time lib because that's complex and changes often.

That's why projects end up with 100s of crates, sometimes 1000s.

This might not be a well received fact in Rust community, but it's a fact nonetheless.



JSON support is provided by serde_json, which is owned by dtolnay, a longtime member of the Rust standard library team. Regex is provided by the regex crate, owned by burntsushi, a longtime member of the Rust standard library team. Directory walking is provided by walkdir, also owned by burntsushi. RNG is provided by the rand crate, owned by the rust-random organization, also full of longtime Rust contributors. CLI argument parsing is provided by the clap crate, owned by epage, the lead developer of Cargo. These aren't randos, these are all de-facto first party libraries. You can trust these developers as much as you trust any random owner of any random Debian package.


Ok, so I created an empty cargo project and added serde_json, regex, walkdir and rand. This added 28 crates, several of these from the same authors.

If the absence of these features caused 1000s of dependencies, then where are the remaining 972?

When I look at a project at work then what inflates the dependency tree is a combination of a whole webserver application stack plus SDKs consisting of dozens of crates. Those then pull in an async runtime or two, different HTTP clients, dozens of crypto crates and so on.

The crate count is a poor metric anyway since some subtree of dependencies is often provided by a single organization.

And I find it quite questionable that everything that's needed for an enterprise grade webserver stack should be part of the standard library, not even Java has that. Relatedly, cryptographers have failed to come up with a proven set of primitives, what's standard changes every few years.


28 crates for 3 of the most common functionalities.

And it would have been 0 crates in .NET or Go. Even after adding a web server.


> 3 of the most common functionalities.

Four. Though you initially asked for five features, so let me add lexopt, which brings the number up to 29.

> That's why projects end up with 100s of crates, sometimes 1000s.

If your argument is "out of 1000s of dependencies 29 could be easily removed" then it does sound a lot less of a deceive change when it comes to supply chain security.

And even getting those 29 right is hard. For example people do want regular expressions with lookaround assertions, but most implementations suffer from runtime blowups (resulting in ReDoS attacks) and improving on that is a fairly recent research[0], so this is hardly a trivial and settled thing to implement. So often there's a tradeoff between choosing more powerful regular expressions and DoS-resistant ones, not one standard.

[0] https://systemf.epfl.ch/blog/re2-lookbehinds/


My personal opinion on each of those:

JSON: there are a ton of different ways to do serialization and deserialization, each with their own tradeoffs, and serde (the most popular) is far from universally agreed upon. The same goes for JSON specifically, there are many different serialization formats with different tradeoffs.

regex: Owned by the rust-lang organization already. You get the benefits of trust (if you trust std, you trust the rust-lang organization anyway), but without the issues of being in std (backwards compatibility and bloat). The only problem I see with that is that BurntSushi is still the owner of the package and as such can still publish new versions on his own (AFAIK crates.io currently requires at least one user owner, but that's something that can be solved by improving crates.io permissions).

walkdir: It has been been postponed, due to the complexity of WalkDir, but may be added in the future. I agree this should be in std. https://web.archive.org/web/20260820171531/https://github.co...

RNG: rand is still evolving, with breaking changes half a year ago. Preferred generators tend to change over time, so I don't see those getting into std (remember, it has to be maintained forever!). I could see the interface (traits) getting into std, but I see no advantage to it: it's maintained by rust-random, but even if you wanted it to be maintained by the same authors as the Rust language (let's say you trust them more than rust-random), you could just move it back to rust-lang-nursery or rust-lang.

CLI arg parsing: not simple at all! The community's preferred solution has 70kLOC (with support for so many features), but there's also argh, pico-args, and others, each with their own tradeoffs.

> That's why projects end up with 100s of crates, sometimes 1000s.

Also because libraries are split up in many crates. For example, regex is split in 3 crates: regex, regex-syntax and regex-automata, and depends on other crates from the same author, such as aho-corasick.

All that said, there are many crates, such as algorithms like aho-corasick, that I think could me moved into the rust-lang org, like regex was.

See also: https://home.expurple.me/posts/a-big-standard-library-is-ove...


On the topic of RNG, the functionality of the getrandom crate is getting ready to be exposed from libstd: https://github.com/rust-lang/rust/pull/157168


A big issue here is the complete lack of namespacing. You have these sub crates with zero ability to know whether they are related to a given organization/author/project.

I mean look at the people defending the lack of namespacing:

https://samsieber.tech/posts/2020/09/registry-structure-infl...

One of the most obvious problems with the lack of namespacing is that if you have a group of crates belonging together, you have to reserve them all at once otherwise an automated script could detect your package and add common suffixes like -sys and take the name even though you got the non sufficed name.

You cannot add name spacing by just prefixing everything with your preferred prefix, because anyone can publish under that prefix.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: