I mean ideally it'd be sendfile but like. it appears that this is actually very complicated to implement for reasons that are currently going over my head
there appears to be at least a proposal for something more generic...
oh so basically it's just splice? I can just use splice? might have to write a wrapper around it or something but if it's that easy (it's probably not) that should be epic. time to look at sendfile and copy_file_range source code yahoo....
> function with several nested typedefs
> look inside
memcpy
file_copy_range is also a spicy splice. how does it differ from sendfile?
@dragon all of these work by moving around references to pages in kernel memory and if they were ever a valid optimization it was predicated upon performance considerations from back in the 90s that made copies really slow. i found this page to be very helpful regarding the concept of the page cache https://blog.pr4tt.com/2016/02/02/BSD-virtual-memory/ and particularly this dissertation on the uvm virtual memory system describes it thusly http://chuck.cranor.org/p/diss.pdf
One major problem with traditional operating systems is that they require unnecessary data copies. Data copies are expensive for a number of reasons. First, the bandwidth of main memory is limited. Each copy made of the data is effected by this. Second, to copy data, the CPU must move it word-by-word from the source buffer to the destination. While the CPU is busy moving data it is unavailable to the user application. Finally, data copying often flushes useful information out of the cache. since the CPU accesses main memory through the cache, the process of copying data fills the cache with the data being copied — displacing potentially useful data that was resident in the cache before the copy.
@dragon i think this absolutely should not be done anymore and that modern CPUs have completely upended the optimal scenarios for high-performance dataflow
@hipsterelectron wait what should absolutely not be done anymore? moving around page references in kernel memory?
@dragon yes! imho it completely subverts the entire purpose of virtual memory and furthermore destroys any sense of locality. essentially all i/o is performed globally and this is a huge security and performance problem. this specific issue is why i decided to turn my focus to operating system design after learning about the page cache from the copy dot fail vuln
@dragon imho every single process should have its own local view of i/o and there should be a precise set of operations by which data can be transferred across domains. in particular i believe this necessitates transactional semantics which can be layered on top of POSIX without any breakage
@dragon the more interesting and difficult part is how that then relates to persistence. i believe persistence and IPC are two wholly distinct concepts