The UNIX system has been in wide use for over 20 years, and has helped to define many areas of computing.
Post
Replies:
15
Boosts:
16
In particular, we devise a graph-based optimization algorithm that can leverage an intuitive graph representation of dataflow- and system-related information, and automatically carry out co-scheduling of task and data placement.
it's that fucking easy
Efficient data management in HPC workflows requires the development of not only optimized scheduling strategies for data placement, but also proper assignment of computation resources to tasks.
ok they assume DAG structure and whip out the linear programming which sounds great for HPC workloads like weather simulations but is as usual trying to solve a harder problem and not seeking to also provide strong memory isolation and/or persistence guarantees
LLNL was easily the best job i've ever had and now i'm wondering if this is more of a candidate for phd study than parse scheduling
i feel that using the dependency graph to schedule the setup/shutdown of a task-specific memory and resource environment for isolated process execution is a really unique approach and really worth pursuing
and that the fractal zip format is not only a great meme but indeed an ideal form of hemoglobin for the i/o circulatory system
like one issue with a bare page is that it has no inherent semantics except the ones we impart to it by manipulating the PAT from the cpu and those aren't in the page itself. meanwhile i have personally invented so many semantics for the humble zip file it has a whole CVE and associated propaganda campaign against it https://alpha-omega.dev/wp-content/uploads/sites/22/2025/10/ao_wp_102725a.pdf
These formats were created a long time ago, 36 and 46 years ago, respectively.
they're scared of the old magic
Archive formats are more accurately described as a series of instructions that occur to a filesystem than a neat set of files bundled together.
i keep forgetting how thick the prose is on this piece of art lmao. "instructions that occur to a filesystem" is reaching so hard and it still sounds awesome
Both ZIPs and tar archives support having additional file “entries” appended to the archive without removing the previous entries of the same name, and still produce
a valid archive.
literally just glazing
Implementations need to process these instructions consistently to be safe from confusion attacks.
oh better watch out it's misty's psyduck oooooo
Archive formats support many features beyond mapping file paths to data
yeah don't forget how they also provide a structured index of mapped regions and colocated metadata that naturally maps to a file path
Archives support many of the same features that filesystems do, such as creation and modification times, permissions, ownership, and links between different filesystem entries
yeah what if we did make this into a filesystem
Supporting these features, which are often platform-specific, means that implementations need to handle each in their respective platform-specific manner, leading to differences and complexity.
translation: yeah symlinks are represented by just storing another file path as the data entry but windows makes symlinks require administrator permissions
For example, if the xz-utils project had a fully reproducible build, the attack used by the malicious maintainer could have been detected if projects were checking the reproducibility of
artifacts from the source code.
THE MALICIOUS MAINTAINER!
which reminds me that macrokernel isolation is the better way to address poettering shamelessly dlopen()ing shared libs because "uh i want to catch and respond to errors"
Software packages ideally should be byte-for-byte reproducible,
unless it's the rust compiler
but many features of archive formats make reproducing package archives difficult
"reproducing package archives" means you have to erase metadata in the code you ship to all your users because otherwise i'm going to continuously neg you until you give up maintainership
compression (stream names!)
this one is really trying
OMG THE EVIL SCARY AUTOCONF TAR COMMAND LINE
