of course the toml crate has serde on it
Replies:
15
one other curious thought from the pip case: there may be information that should be globally available, but is highly inefficient in distinct file outputs--it needs to be synchronized into some shared state (like a sqlite db). this is how the available versions for a python package can be made queryable
that's notable because it represents a different kind of state and a different type of dependency than other tasks
in the case of querying python indices, it represents the world state at that time
(importantly, the "world state" paradigm can also be applied to other forms of global mutable state like the filesystem. this is one problem with spack externals -- they don't have any concept of invalidation)
so where does it end?
if we want to use this bootstrap zstd, we have:
(1) use pkg-config to find one matching the desired version spec
(2) bootstrap a known version from source
spack is a massive piece of machinery to resolve a dependency graph with versions of c and c++ and python etc. it's not worth anyone's time to recreate from scratch. but spack itself has bootstrap needs (it needs python)
it would be impossible to reproduce spack-style resolves without spack itself. but we could wrap spack, after bootstrapping its dependencies
omg we could bootstrap a rust toolchain. that'd be so fun and flirty
if we can bootstrap python, we can bootstrap pip
guix has already gone down the reproducibility path with gnu mes. we won. final boss defeated
pants dug itself into a bit of a corner. in fact it's about to lose one of the most powerful aspects of the rule graph because scanning for internal (recursive) calls poses a performance problem, so the recursive calls need to perform the type-based resolution by hand https://github.com/pantsbuild/pants/issues/19730
i had proposed defining rules in webassembly or even jvm bytecode before, because one of the biggest points of feedback from twitter engineers was "we don't understand this complex python system, we write scala code". this was one of the most reasonable and honest bits of feedback i've ever received
how do you architect a system that crosses ecosystems?
build tools have largely converged upon process executions as the shared unit of work and the filesystem as IPC. this is the most successful and portable FFI i am aware of
process executions are also memory-safe by default, from the OS's segmentation. there are portable synchronization tools for the filesystem alone, and higher-performance ones depending upon the platform. there's a standard resource discovery mechanism, and a standard name assignment mechanism
