i have learned just moments ago that the adobe portable document format is yet another rebranding of the same tarball slop we have been tossing around for half a century
22
Replies
5
Boosts
https://hackers.town/@zwol/114158568811964745 over a year ago, @zwol immediately identified the PDF as the epitome of design that optimizes for the derivative with not a moment for the integral
that tweet from yesteryear reached out to answer my lingering question: "did they really call it a production-quality document format and then stuff everything of note at the end?"
yes! because stuffing extra data after the original document the domain experts signed off on is where you can add special features
@hipsterelectron what do you mean by optimizing for the derivative rather than the integral?
@dragon it's a really really pretentious way to say it but i'm referring to control theory and the canonical PID controller used in that field
@dragon i am specifically thinking about how this category of architecture (stuffing shit at the end of a stream without any cross-referencing) feeds the microsoft/google pathology of shipping a half-assed feature quickly (e.g. an incremental edit, which is nonstandard but only occurs with special software, which isn't fully-specified) that further imperils the ability to extract structured data from documents that people put an immense degree of effort and willpower into creating
@dragon i'm thinking of lawyers and mathematicians
@dragon and HTML is not better, because to do anything declarative you need CSS properties, and CSS properties means using a whole fucking w3c web browser
and my impetus here is actually not about rendering a document to pixels, but about indexing its information
@dragon if you read the alt-text for OP you will see my valiant attempt to represent the structure of the screenshotted document excerpt with that old rusty knife, unadorned ASCII whitespace
@dragon not only are there 2-3 ways to even represent text mapped to a pdf, but you absolutely are not allowed to just edit the text by hand. this is something i learned from the tesseract project (which google acquired from someone who gave a shit), in which tesseract is the only program that can modify the text layer (and it doesn't accept human input)
@dragon this is a violent statement https://github.com/tesseract-ocr/tesseract#about
Tesseract 4 adds a new neural net (LSTM) based OCR engine which is focused on line recognition, but also still supports the legacy Tesseract OCR engine of Tesseract 3 which works by recognizing character patterns.
- line recognition: "we can use all the half-assed 'research' in 'computer vision' to draw boxes" (it sucks at drawing boxes)
- recognizing character patterns: this is (and i cannot possibly stress this enough) --
THE ONLY FIELD OF 'AI' THAT WORKS!
...because OCR has had 70 years to percolate!
@dragon OCR uses "heuristic" algorithms. that's what fuckboy statisticians call "a model based on scientific theory"
@dragon so my next thought was: ok, how do we get the document structure out of the web browser? because this "pdf" thing is a recipe for undecidability
and it's a good question!
@dragon this is spurred by (what else) emacs org-mode. because org-mode actually has an answer to this!
@dragon i have a very fond memory of receiving my first github issue a decade ago, when someone complained that i relicensed org-info-js from LGPL to the full GPL. they were mad because my changes were actually really fucking sick
@dragon i do not have actual content for this https://amass.energy/ but check that clean js console. try those keybindings with ?
i need to remove that link to w3.org at the bottom left though, because that is an evil domain
@hipsterelectron How do you get the keybinds to work without JS?
Editing a pdf in the first place is a horrendous idea that should never be undertaken but obviously it's completely necessary if you want to have forms or signatures or addendae because of the complexity brought along by having so many programs that can export pdfs and rarely having access to source files
@dragon it is js!!!!! it was my first js project before i engaged in the whole coffeescript thing. coffeescript is love coffeescript is life. i used electron when it was stil called atom-shell and was part of an actually fucking groundbreaking text editor that was easily going to be another emacs
@hipsterelectron I loved atom I was so disappointed when Microsoft killed it!!!! It was genuinely such a pleasure to use!!!!!
@dragon almost shed a tear i did shed a tear thank you for affirming that immensely emotional process
there's this delectable bit of circumlocution right after the screenshot:
A cross-reference section for an incremental update shall contain entries only for objects that have been changed, replaced, or deleted. Deleted objects shall be left unchanged in the file, but shall be marked as deleted by means of their cross-reference entries.
they don't even garbage-collect. you can in fact append an operational transform over the document contents with absolutely zero global indexing and the viewer won't detect a thing until their reader comes upon your change!
@hipsterelectron i did understand this bit and lol that sucks so bad
The added trailer shall contain all the entries except the Prev entry (if present) from the previous trailer, whether modified or not. In addition, the added trailer dictionary shall contain a Prev entry giving the location of the previous cross-reference section (see Table 15). Each trailer shall be terminated by its own end-of-file (
%%EOF) marker.
that's right: IT'S YET ANOTHER TURING MACHINE!!!
