Discussion
it turns out that a soviet mathematician stanning for king kolmogorov and shitting on shannon for 42 pages is exactly what i was looking for https://scispace.com/pdf/mathematical-problems-in-the-shannon-theory-of-optimal-cnf4ox90w8.pdf
In his fundamental work, Shannon introduced quantities which enabled him to formulate an answer to the problem he raised. The principal one of these is a quantity called information.
(The somewhat belated paper of Fleischer [34] reflects the insufficient contact between Soviet and American investigators in this field.)
From this point of view the very popular axiomatic definition of entropy, which was introduced at first by Shannon in [79]
"axiomatic" i.e. "i made it up" dr. manhattan shit. not coherent, nor could it be said to conform to other logical structures
The point is that not all the axioms which enter into the definition of entropy are equally natural (for example, the formula of conditional entropy), and also that, as Kolmogorov notes, it is not a priori clear whether there exists a unique one-dimensional numerical characteristic which adequately reflects the properties of such a complicated phenomenon that information turns out to be in a diverse real situation.
This is explained (see [21]) by the fact that information density (in the probability sense of convergence) depends continuously on the distribution (with the metric of convergence in quadratic mean); but from the convergence in probability of a sequence of integrands it does not follow that the sequence of their integrals converges, unless these integrals are uniformly convergent.
this is a CLASSIC mistake!
emphasis NOT mine:
That is, the information between the input and output messages cannot be greater than the information between the input and output signals.
and this is the result of specifically defining a unidirectional information flow as a markov chain A -> c -> B, where A and B are independent random variables, in which their paired interactions (i'm simplifying) to send/recv with the channel c introduce a conditional probability (and therefore, if the sequences converge appropriately) conditional information!
the markov chain concept makes a hell of a lot more sense than shannon's bullshit at least. a markov whatever is just a graph
Install bonfire.cafe
Get the full app experience
