of course the toml crate has serde on it
Discussion
Loading...
cargofuckyourself
[morpheus.jpg] what if i told you
you could build an extensible task system
outside the build system
the reason i'm thinking about this is because:
(1) this is also the exact thing that pip needs for parallel cacheable package finding
(2) the one thing i was always annoyed about from pants (and had spent quite some time attempting to fix) was how it stuffed like 300 separate individual great ideas into the single monolithic cli
especially pants demonstrates a very powerful request-response model with a cyclic task dependency graph. as pants grew bigger and added more functionality, the reliance on a persistent daemon became self-reinforcing
pants would previously be able to make use of a ./pants shell script at the root of the repo (i still have the muscle memory). it now employs a scie jump, which is a system for bootstrapping executables that can be thought of as a generalization of pex zipapps
the point is that even the monolithic pants is now assembled from a specification. and i think furthermore that such an assembly process can be a component of hooking up toolchains that don't want to talk to each other
~65 more replies (not shown)
consider autoconf. the thing i've wanted to hack into it for ages is parallel test evaluation. the part i hadn't considered was how the result of a test can often be cached globally. when is this not possible? this can be answered if tests are able to specify distinct inputs i.e. dependencies. autoconf sadly does not have a task system
cargo's problems are legion:
(1) build scripts can't do IPC to other build scripts
(2) build scripts can't fetch resources the way they can other rust code
(3) build scripts don't have any shared output space where one package can compute an output that's shared with others, even in the same dep graph
the way packages can and do work around this is to simply write to a known filesystem path as a primitive form of IPC. this is highly problematic
pip's case is interesting too:
(1) it has a specific fetch-parse-cache process it needs to perform before it can do any resolve work
(2) it needs to be portable, so it can't add its own native code
(3) it has builds and fetches that can occur in parallel
(4) it wants to avoid being a build system like cargo, but it ends up propagating build provenance through its outputs
the pants project used to have a set of binary bootstrap scripts for things like gcc (that's the one i maintained for twitter's c++ code)
the reason this arose was:
(1) i need to build zstd if not available
(2) i need to build curl if not available
(3) i need to build make if not available
there are distinct contexts separating all of these! a bootstrap make doesn't need to have guile extensions. a bootstrap curl doesn't need to have zstd built in. a bootstrap zstd doesn't need multithreading
@hipsterelectron cargo
at least it's not cmake
@hipsterelectron there’s always merde (nope, he started facet, that’s right)
https://github.com/bearcove/merde