Post by @jonny@neuromatch.social

I read halfway through this looking for a "what did you actually do and what are you talking about" section, found the "here are 1,000 papers about individual neurons" and was like oh no is this the methods. And I just finished and think those are in fact the methods. I can't find any links to code or anything.

Anyway, if you want an example of why if you're a scientist you need to be getting very transparent with your methods, here's a paper that looks soooo fancy but as far as I can tell none of the numbers may actually be real.

Also reading this was very funny as a neuroscientist. I don't think these kinds of folks have read a single neuro paper.

https://already-clean.goodfire.com/

Goodfire

Neural Networks Are Already Clean

A working hypothesis about neurons, generalization, and the ecology of circuits.

0 Boosts

0 Likes

Manic Pixie Dream Gremlin

@gildilinie@beige.party 3 hr. ago

@jonny a computer scientist what the fuck am I reading.

David Chisnall (*Now with 50% more sarcasm!*)

@david_chisnall@infosec.exchange 2 hr. ago

@gildilinie @jonny

Also a computer scientist and, as the author of the fourth most-downloaded paper in the ACM Digital Library, I can claim some experience with publishing in our discipline.

I gave up a few sections in. I have absolutely no idea what the paper is trying to show nor how they are trying to show it. Or even what ‘clean’ means in the context of the paper.

Will Berard 🫳🎤🫶

@MrBerard@mastodon.acm.org 1 hr. ago

@david_chisnall @gildilinie @jonny

I reckon the mistake is reading this as a 'paper', this is an overgrown blog post is what it is.

Or, as Bender calls them, "paper-shaped object" - but this one doesn't even meet the bar. The lack of in-text citations is a big tell.

Manic Pixie Dream Gremlin

@gildilinie@beige.party 1 hr. ago

@david_chisnall @jonny 'clean' means "not messy" which means "not opaque" which means "interpretable" ... see his jab at his colleagues who have "proofs", so basically "please believe me this is a real sub-discipline and not just labelling data by dowsing rod"

David Chisnall (*Now with 50% more sarcasm!*)

@david_chisnall@infosec.exchange 1 hr. ago

@gildilinie @jonny

Oh, like all the papers that say ‘we found the neurone in model X that controls output Y!’ And pretend that this means interpretable because obviously once you have found one neurone you can understand how the 14 layers before it interact with it?

1+ more replies

Adrian

@yo8rzz@mastodon.radio 3 hr. ago

@jonny Pseudoscience. Comparing neurons (living cells) with floating point arrays used in large matrix calculations is clear evidence of snake oil salesmanship, severe lack of education in both biology and computer science or outright fraud. Total garbage, not even worth spending 30 seconds reading this.