i feel that the grammar of a programming language is among the least appropriate of all possible facets of its behavior to start off with. why on earth would i care about your preferred tokens to represent concepts which have not yet been defined
Post
Replies:
0
if i wanted to make a language for the macrokernel i would have to decide to understand a lot more than i do now about what the hell a kernel is and especially boot logic. and then i'd have to learn about disk persistence.
i think as an implementation language for managing memory and cpu structures there definitely could be better frontends. and i think once i feel more comfortable about the hardware semantics (particularly x86 cpu + ssd nvme) then i will want to start gernerating structures that translate the macrokernel user API in my head to match the requirements from the hardware
like an interlocking web of formal models dancing together in the memory page prairie is actually what i see in my head when i think of the end goal
look up this paper on "model checking c source code for embedded systems" https://link.springer.com/article/10.1007/s10009-009-0106-5
- buy it for $40! thanks!
two just horrifying suggestions to purchase below that:
- A Model Checker Collection for the Model Checking Contest Using Docker and Machine Learning
- Finding software vulnerabilities in large C projects via bounded model checking
never sure how to take lines like these https://sci-hub.st/10.1007/s10009-009-0106-5
The disadvantage is that all specific knowledge of the C code and the underlying hardware has to be used in the abstraction process as the general purpose model checkers are not aware of these peculiarities.
i assumed everyone doing this sort of thing was the author of the C code they're checking and that everything of course has to be specialized to the particular CPU. i don't know what anyone would expect to get out of model checking otherwise
actually delighted to hear not only that gcc tends to be the de facto here but that CIL is compatible. CIL sounds sick
If the GCC compiler supports the chosen microcontroller, the adjustments are less costly since many of the C code model checkers use the GCC compiler or a compatible framework such as CIL for preprocessing.
yet another seemingly-legit ARPANET paper that seems like it was made to be found discarded in an abandoned laboratory https://people.mpi-sws.org/~gummadi/teaching/sp07/sys_seminar/arpanet.pdf
Attempts at computer networks have been made in the past
"but they weren't evil enough for our purposes".
dude is absolutely crashing out about "load sharing", claiming it will never be worth the cost, and computer programs are incompatible, etc. given that i know that worked for parallel scala compiles, it seemed confusing until the next section
Data Sharing:
The program is sent to a remote computer where a large data bank exists.
This type of operation will be particularly useful where data files are too large to be duplicated economically.
so our boy lawrence g. roberts totally predicted bazel cloud builds and github actions.
Access to this data base will be required simply to make an inquiry or may involve executing a complex program using the data base.
mysterious access control mechanisms? potential surveillance? it gets better:
This type of use is particularly important to the military for command and control, information, retrieval, logistics and war gaming applications.
In these cases, one command would send a program to be executed at another center where the data base existed.
i really never know if they're just saying intentionally ridiculous shit
note how he distinguishes "send a program" -- clearly an RPC call, which were definitely around at the time
fidonet did a ton of load sharing on a per-file basis, including some really interesting locality-based queueing
Program Sharing: Data is sent to a program located at a remote computer and the answer is returned. Software of particular efficiency or capability exists on certain machines.
literally this is all google tech lmao it's like he's salivating over this
The use of specialized programs at remote facilities makes possible large gains in performance.
Perhaps even more important is the potential saving in reprogramming effort.
ridiculous shit
yeah and then he mentions three separate times how scientists can use it to do new science together. it seems important that scientists are on it at all
and i found all that about DARPA salivating over people never writing their own programs again because of:
- the seL4 paper which got best paper https://web.archive.org/web/20110219113850/http://www.ok-labs.com/releases/release/open-kernel-labs-paper-on-formal-verification-wins-top-prize-at-prestigious
still think this paper is terrible. it keeps saying it made compromises for verifiability and wildly overstates the guarantees
turns out that conference SOSP is literally the (ACM) conference whose first year was when a big ARPANET thing was unveiled https://en.wikipedia.org/wiki/Symposium_on_Operating_Systems_Principles
there's these unstructured notes from the fucking pentagon lmao https://web.archive.org/web/20150405055923/https://web.stanford.edu/dept/SUL/library/extra4/sloan/mousesite/EngelbartPapers/B1_F20_CompuMtg.html
and finally this is IETF at its best https://en.wikipedia.org/wiki/Shared_resource choose the most generic possible term that sounds like page cache, but it specifically means network share
oh this is a great tidbit
Before 2023, SOSP was held every other year, alternating with the conference on Operating Systems Design and Implementation (OSDI);
- starting 2024, SOSP began to be held every year.
lots of weird things like this
oh then i found this guy who was at the arpanet conference http://royalsocietypublishing.org/rsbm/article-pdf/doi/10.1098/rsbm.2002.0006/911101/rsbm.2002.0006.pdf
so like this guy is easily off the charts evil imho. this is him saying he was smarter and braver than alan turing:
My few contacts with Turing were not encouraging. I wanted to talk to him about the remarkable results of his paper ‘On computable numbers’. Reading this paper I had found numerous errors in the formal specification of the universal computer. Some were trivial but others were quite subtle and I was not sure that my solutions were correct. When I came to this point, Turing became more and more agitated, until I could see that no sensible discussion was possible. Clearly he felt the errors to be irrelevant and my drawing attention to them rather foolish.
then he mysteriously advises on "cryptography" from the late 80s until he finally fucked off this planet
Retirement did not by any means imply inactivity. For the next 15 years Davies practised
as a consultant in security engineering for the financial and media industries. This was at a
time when systems based on cryptographic and similar techniques were coming into wide use
both for cash cards and pay television.
omg
Davies was very much an engineer rather than a scientist, and he was always on the lookout
for topics in which his ingenuity and insight could be deployed.
guy who steals people's ideas
He was early in the field of civil uses of cryptography and always had a healthy scepticism of claims to perfection.
terrifying
Mathematical proof of the security of a system struck him as dubious, because it is much easier to prove resistance to attacks one has thought about than it is to prove resistance to attacks one has not thought about.
guy who knows how cryptography works
omg the EROS author is literally validating all my ideas about the macrokernel love that for me
Systems programs are strongly driven by bulk I/O
performance.
he keeps talking about multiple aliasing being problematic and of course this is why i decided to never share anything and instead have layered i/o queues
In systems code, the effect of representation and data placement can be extreme. Bonwick et al. discuss some of these effects [5], noting that the performance of system-level benchmarks can change by 50% through careful management of cache residency and collisions.
but how do you manage something "carefully" if all the interfaces allow for is urgency???
This tends to penalize the performance of automatic storage reclamation strategies. To make matters more interesting, there are caches.
yeah it rly annoys me how the filesystem has its own caches and the kernel has its own caches but there's this assumption that persistence is always the final destiny of all writes
It follows that user-managed storage is a requirement, but perhaps not in fully general form.
that's exactly it!
The facts say otherwise. The annual cost to operate a large banking data center today is $150,000 per square foot. It is by far the most expensive real estate in the world, and more than one third of that cost is the cost of cooling the data center.
2006!!!!
wait shit he had a point:
The general rule of thumb is that power
is proportional to V2F: the square of the voltage times the frequency. Most of this power is wasted as heat. To a system’s programmer, the cost of doubling the clock rate is $50,000 per square foot per machine room.
do i detect an IETF hater???
Raising the clock rate decidedly isn’t free, and walking into the network distribution closet at your business or school will quickly convince you that current power usage is excessive.
oh oops he glazes up tcp/ip immediately after. this is "Programming Language Challenges in Systems Codes" by jonathan shapiro
on the internet:
Large block processing costs are dominated by memory bandwidth, not software overheads.
that makes sense. the difficulty with fitting network i/o into my beautiful symphony of data locality is that the network is "necessary global" in some sense, and can't do multi-level queueing or w/e because you can't dictate to network resources how fast or slow to send data to you!
As Blackwell discusses [4], processing overhead on smaller packets is necessarily much higher.
hmmmm
vaguely interesting microsoft research paper https://research.cs.wisc.edu/areas/os/Seminar/schedules/papers/Deconstructing_Process_Isolation_final.pdf
A software isolated process is a collection of memory pages and a language safety mechanism that ensures that code in a process cannot access another process’s pages. A SIP replaces hardware memory protection with static verification of program safety.
DEEPLY suspicious to hear "replaces hardware memory protection" coming from microsoft lmao
They rely on verifying code’s safe behavior to prevent it from accessing another process’s (or the kernel’s) instructions or data.
LMAO
However, language safety offers important benefits not provided by hardware process protection, for example, detecting in-process errors such buffer overruns.
literally nothing in this paper makes any sense
just read a liedtke paper for the first time https://cgi.cse.unsw.edu.au/~cs9242/19/papers/Liedtke_93.pdf i think this guy is crazy for still trying to make ipc faster but this was actually cool to read. should have thought to learn that context first before hating on all the modern microkernel stuff =\
and he completely blew my fucking mind with this lmao:
5.3.5 Direct Process Switch
For a remote procedure call it is natural to switch the flow of control directly to the called thread, donating the current timeslice to it (as also LRPC does).
This is also the most efficient method, since it only involves changing stack pointer and address space.
i don't think i would ever have thought of that myself and i can see why all-consuming focus on a hopeless task can actually get you places sometimes if you don't half-ass it
guy seems cool
ipc performance is not only determined by the kernel algorithms, but also by the user/kernel interface. It is important to support typical usage and permit compilers to optimize code.
clearly we agree on the important things??? lol
Since there are no compilers (as far as we
know) which permit interfaces to be specified at register level and basic block sequences to be optimized by programmer supplied usage information, we had to use hand coding for the critical ipc related parts.
see i love this guy lmao
oh amoeba is so cool lmao https://dl.acm.org/doi/abs/10.1145/54289.54291
6. THE FAST AMOEBA FILE SERVER
Like the Amoeba communication primitives, the Amoeba file server, called the bullet server was designed for extremely high performance.
you're allowed to say stuff like this if you can back it up. let's see:
In particular, the decrease in the cost of disk and RAM memories over the past decade has allowed to use a radically different design than is used in UNIX and most other operating systems. In particular, we have abandoned the idea of storing files as a collection of fixed size disk blocks.
HELL yes i win again
All files are stored contiguously, both on the disk and in the server's (16 MB) main memory
16 mb lmao
The bullet server is an immutable file store, with as principal operations READ-FILE and CREATE-FILE.
this is how pants works and how my shared memory ipc worked, it's cool
(For garbage collection purposes there is also a DELETE-FILE operation.)
love this!
the cache kernel is sick. closest thing to the macrokernel i've found so far https://dl.acm.org/doi/10.1145/504390.504414 research sponsored by ARPA wish ARPA did more locality-centric memory motion stuff
SPIN kernel rox my sox!!! https://www.cs.cornell.edu/people/egs/papers/spin-tr94-03-03.pdf they're literally just saying "yeah so turns out applications have highly structured resource dependencies and you can just ask them for that shit"
In terms of memory resources, multimedia applications use large amounts of data (audio and video streams) with access patterns that interact poorly with locality-based page replacement algorithms [Anderson 93, Nakajima et al. 92]. Application-specific virtual memory management policies can solve this problem.
yes!!!!!!! but they go deeper:
High-level information about media
direction, edit cuts, and temporal constraints are directly relevant to page replacement decisions.
yes!!!!!!!!!
When presenting a video stream, for example, an application can sequentially prefetch video frames directly from disk into memory-resident buffers. Information about synchronization between media streams can also be specified to prevent unnecessary replacement of pages that are interdependent.
literally the application knows what they want lmao
Filesystem performance can benefit from application-specific information in several ways.
TRUTHNUKE
The application can provide hints about future usage to the filesystem to help it schedule disk traffic [Gibson et al. 92]. This can result in
more effective prefetching policies and lower buffer cache miss rates.
amazing
An effective prefetching policy can also remove virtual memory remapping operations from the critical path, since disk blocks are already mapped into the application address space when they are needed.
i think this is prob what i'm doing
In addition, the application can inform the kernel about how it will use the buffer cache, so that the kernel can make informed decisions about physical memory allocation [Stonebraker 81]
y e s
Extensible interprocess communication
An extensible IPC interface enables applications and servers to define their own semantics for interprocess communication enabling the best tradeoff between performance and functionality.
of course but also yes!!!!!!!!
Some systems rely on “little languages” to safely extend the operating system interface through the use of interpreted code that runs in the kernel [Lee et al. 94, Mogul et al. 87, Yuhara et al. 94].
i think it's a cute idea but it shouldn't be code it should be data describing a set of access patterns for an isolated application process
These systems suffer from three
problems. First, the languages, being little, make the expression of arbitrary control and data structures cumbersome, and therefore limit the range of possible extensions.
this is why you never make your own language for a specific problem and then force people to use it!!!!
Second, the interface between the language’s programming environment and the rest of the system is generally narrow, making system integration difficult.
great to hear how bazel and nix were by no means the first to make this mistake
a professor i follow on here who has been way more annoying on here recently and i didn't know why......anyway happened to find a paper of his from last year and he's just doing literal LLM slop now. RIP in peace