spent an excessive number of hours learning about just enough information theory to make sense of why jarek duda's tANS (table-based asymmetric numeral system) actually works because unlike yann collet at fb he seems to be trying very hard to write papers that explain things. there's a clear improvement over time commensurate with a language deficit and it's why i decided to dig in more because to me this is very obviously trying to do things right:
The modern data compression is mainly based on two approaches to entropy coding: Huffman (HC) and arithmetic/range coding (AC). The former is much faster, but approximates probabilities with powers of 2, usually leading to relatively low compression rates. The latter uses nearly exact probabilities - easily approaching theoretical compression rate limit (Shannon entropy), but at cost of much larger computational cost.
