Autark: Rethinking build systems – Integrate, Don’t Outsource

Posted by adamansky@reddit | programming | View on Reddit | 11 comments

[-]

Revolutionary_Ad7262@reddit

Rethinking build systems – Integrate, Don’t Outsource

A.k.a do as any other sane build system except those from C/C++ community

[-]

Yeah, at this point I suspect people are more wondering about what it is about C/C++ that seems to make them immune to adopting something more along the veins of npm/go/cargo/uv/etc/etc, because there must have been tons of people thinking "I wish I could have $OTHERLANG build system but for C/C++".

Like, for all the complaints about the js/npm ecosystem, it's still massively successful and enables people to pull in absolutely trivial dependencies—they obviously don't think it's too much of a hassle with the dependency vs writing is-even themselves (or even just copypasting the original).

[-]

PotentialBat34@reddit

I saw a tweet yesterday blaming Rust because "packages you download off of cargo can be harmful for your program"

The truth is, the old guard who once rebelled against their "Write in C" masters and replaced it with "C with Classes" 3 decades ago became the very thing they swore to destroy. They became the status quo, and started actively resisting innovation.

[-]

syklemil@reddit

Yeah, there's some echo there of the old "but how can you trust open source software?", and some real supply chain security issues, which can be worked on (e.g. cargo vet, SBOMs, signatures, etc, etc).

But I'm still pretty sure that the shape of the C/C++ ecosystem is more about the actual lack of convenience in pulling in more dependencies, as opposed to any really deep SCS stance.

[-]

segv@reddit

I feel like it is a "fractal of death by thousand papercuts" type of situation, including what you listed.

The overall experience is janky because the tooling is janky. The tooling is janky because it has to deal with homegrown stuff like dependency management based on copypasting code ("single file header only library" is still a selling point), and with the language treating each translation unit as a separate universe that relies on hopes and prayers that the linker will figure it out and so on and so forth.. y'all know the drill.

[-]

edgmnt_net@reddit

I think it relates to how you model the build target, at least partly. Typical Unix tools written in C probe a lot of stuff and tend to be fairly accepting of various dependencies (even if not explicit) at build time. More modern ecosystems seem to use more fixed targets and dependencies (pinned, sometimes including the toolchain). This is one of the reasons why you need a whole autoconf setup / configure script for the former, while in Go you just run the compiler and be done with it (even cross-compiling is easy via GOARCH, partly because it's just one setting). Also look at more modern build systems like Meson and see that they still deal with such complexities to a great extent.

In turn this means that C packages tend to adapt to the local environment. Which means they don't even specify deps (it's the job of the OS or user to provide them), so there's nothing to fetch and install.

Not saying that's the only reason, though. Obviously there's a lot of historical baggage too.

[-]

syklemil@reddit

Typical Unix tools written in C probe a lot of stuff and tend to be fairly accepting of various dependencies (even if not explicit) at build time. More modern ecosystems seem to use more fixed targets and dependencies (pinned, sometimes including the toolchain).

Eh, it's also entirely possible to have some fairly trivial-for-users feature-flag system, like in cargo.

Non-explicit dependencies also sounds like a supply chain security nightmare. :)

Which means they don't even specify deps (it's the job of the OS or user to provide them), so there's nothing to fetch and install.

Yeah, I remember my first interactions with that from some attempts to install unconventional software a couple of decades ago; even if the build system doesn't fetch && install dependencies, it'd be real nice to have them actually listed out the way they'd be in other build systems, and it'd be nice to get a "$foo not present" or "$bar is wrong version" early.

I think a lot of us have had the realisation that building native software isn't actually difficult after trying out languages like Go or Rust, it was just C.

[-]

edgmnt_net@reddit

Non-explicit dependencies also sounds like a supply chain security nightmare. :)

Not in the case of C, because you're not fetching them automatically. Yes, it is a security and reproducibility nightmare when a build system is allowed to pick up remote dependencies without any control whatsoever. But compatibility and version pins are orthogonal concerns, IMO.

For Linux, distros are effectively setting up exact dependency sets. It's not package maintainers.

Now if I'm to think about it, this might avoid some troubles with transitive dependencies. If you have tight dependency ranges (single pinned version at worst), one version bump can cascade through a whole lot of packages. However, this is, to some extent, trading off assurance because you don't really know if things work together. But luckily, it is the distro ecosystem that's making up for it, because they're the ones picking exact versions and doing a lot of testing on a larger scale.

The alternative is to have vendors specify all dependencies, but then you quickly run into software distribution models based on static linking, which raise different problems and tradeoffs. Because there's practically little chance independent vendors will happen to align on very specific versions of dependencies, so you kinda have to forego dynamic linking.

[-]

International_Cell_3@reddit

about what it is about C/C++ that seems to make them immune to adopting something more along the veins of npm/go/cargo/uv/etc/etc,

Well these are all package managers that try their darndest not to be build systems.

Something that go, python, javascript, and Rust all have that C and C++ don't are precisely defined import semantics. This goes by a few names in PL circles but the gist is that modern languages define exactly what a "unit" of work (compilation, compile + interpret, etc - it depends) for the language implementation and how those units refer to each other (import, use keywords, etc).

C and C++ have no such semantics, and the various bullshit you see in the wild make it next to impossible to package it all together.

Metabuild systems in C/C++ are the standard and not bad. Package management is just kinda fucked because no one can agree on what it even means, let alone how it should work or what tools should be standard.

[-]

Halkcyon@reddit

(or even just copypasting the original)

Also called "vendoring" which is largely seen as a Bad Thing™️ in dependency management.

[-]

syklemil@reddit

Some of those libraries I'd be willing to argue don't really meet any threshold of originality, and so should be uncopyrightable.

But yeah, I really don't want to imagine the egg on someone's face if they vendor is-even and is-even then turns out to have some CVE that they remain vulnerable to.