Discussion
it turns out that a soviet mathematician stanning for king kolmogorov and shitting on shannon for 42 pages is exactly what i was looking for https://scispace.com/pdf/mathematical-problems-in-the-shannon-theory-of-optimal-cnf4ox90w8.pdf
In his fundamental work, Shannon introduced quantities which enabled him to formulate an answer to the problem he raised. The principal one of these is a quantity called information.
(The somewhat belated paper of Fleischer [34] reflects the insufficient contact between Soviet and American investigators in this field.)
From this point of view the very popular axiomatic definition of entropy, which was introduced at first by Shannon in [79]
"axiomatic" i.e. "i made it up" dr. manhattan shit. not coherent, nor could it be said to conform to other logical structures
The point is that not all the axioms which enter into the definition of entropy are equally natural (for example, the formula of conditional entropy), and also that, as Kolmogorov notes, it is not a priori clear whether there exists a unique one-dimensional numerical characteristic which adequately reflects the properties of such a complicated phenomenon that information turns out to be in a diverse real situation.
This is explained (see [21]) by the fact that information density (in the probability sense of convergence) depends continuously on the distribution (with the metric of convergence in quadratic mean); but from the convergence in probability of a sequence of integrands it does not follow that the sequence of their integrals converges, unless these integrals are uniformly convergent.
this is a CLASSIC mistake!
emphasis NOT mine:
That is, the information between the input and output messages cannot be greater than the information between the input and output signals.
and this is the result of specifically defining a unidirectional information flow as a markov chain A -> c -> B, where A and B are independent random variables, in which their paired interactions (i'm simplifying) to send/recv with the channel c introduce a conditional probability (and therefore, if the sequences converge appropriately) conditional information!
the markov chain concept makes a hell of a lot more sense than shannon's bullshit at least. a markov whatever is just a graph
and lmfao now he's mentioning shannon actually proved the converse of the "shannon theorem". fucking lmao
and apparently king kolmogorov was the first known author to call that shit out too.
It seems that it is not yet sufficiently widely used in information theory literature, since in some papers facts which are special cases of this result are still proved by more unwieldy ad hoc methods.
this is every single thing i have ever wanted to hear in my whole life
As for the statement of the Shannon theorem about sufficient conditions for the possibility of transmission, the subject becomes more complicated.
The most vulnerable aspect of Shannon's theorem, from the point of view of practical applicability, is the fact that the coding and decoding methods, whose existence is guaranteed by the fulfillment of condition (28),
(that's the converse shannon theorem)
are obviously unavoidably connected with the coding of a large quantity of information as a single unit, and are therefore inevitably complex.
(The case of infinite information, for which a theory can also be developed [19] is not considered here.)
i FUCKING love this guy
this is how he concludes the paper:
8.4. The author does not understand the reasons for the appearance of the concepts of entropy and information in certain game-theoretic constructions (see [47]). The perspectives of this direction are not clear.
brutal. that goes to https://www.princeton.edu/~wbialek/rome/refs/kelly_56.pdf
In its most general formulation this problem seems to have but one solution.
great start. let's go. bell labs do your worst
A cost function must be defined on pairs of symbols which tells how bad it is to receive a certain symbol when a specified signal is transmitted.
The utility theory of Von Neumann2 shows us one way
to obtain such a cost function.
looked it up and yes it indeed only contains static games, i.e. a game where you pretend no one has any memory of your behavior. boring!
but then i looked up this vile man von neumann and found this ridiculous sentence midway through that explained everything to me:
He then went to the University of Göttingen on a grant from the Rockefeller Foundation to study mathematics under David Hilbert.[42]
oh, so he was the chosen fuckboy ordained by rockefeller to succeed the fascist hilbert
In 1926, he graduated as a chemical engineer from ETH Zurich and simultaneously passed his final examinations summa cum laude for his Ph.D. in mathematics (with minors in experimental physics and chemistry) at the University of Budapest.[40][41]
one might naively think [40][41] implies that the statement was backed up by encyclopedic sources. but this is von neumann, so it's:
but then i learned something extraordinary
so there's this 2013 text released by cambridge (who very clearly has skin in this game) about "the creation of game theory": https://www.cambridge.org/core/books/abs/von-neumann-morgenstern-and-the-creation-of-game-theory/wrestling-with-complexity/EB3F15910B58D9FD7522D5DFD9D57727 @davidgerard did you know this shit????
i'm just going to quote the chapter intro verbatim:
Rockefeller Years
When Morgenstern entered the University of Vienna, the Laura Spelman Rockefeller Memorial had been in existence for several years. Named after John D. Rockefeller's widow, its endowment ran more than $70 million and it had become the main source of Rockefeller support for the social sciences before being absorbed into the Rockefeller Foundation in 1929. The Memorial's favourable disposition towards social science research during the 1920s was attributable largely to its director, Beardsley Ruml. He arrived at Rockefeller with a doctorate in experimental psychology from the University of Chicago and was interested in the construction of a less “abstract and remote” social science that would be of immediate use to the “social engineer”.
Ruml established a system of grants, placing emphasis on empirical research and preferring universities that also emphasised teaching. In the United States, the universities of Chicago, Harvard, Columbia, and Iowa State, and the National Bureau of Economic Research were amongst the main beneficiaries. In England, money was given initially to the London School of Economics and later to Oxford. The Cambridge of Keynes and Robinson, on the other hand, showed little interest in shifting focus in response to Rockefeller, and was passed over. In Continental Europe, Rockefeller tended not to support universities directly, in the belief that European social science was still highly speculative, the universities were often torn by internecine strife, and the risks of wasting money through bureaucracy were high. Consequently, the Foundation devoted its efforts to a programme of fellowships intended to free future teachers “from the traditional conceptual thinking and a priori generalizations of the present generation of teachers” and to the funding of institutions that were independent of universities.
Install bonfire.cafe
Get the full app experience
