Discussion
Loading...

Post

  • About
  • Code of conduct
  • Privacy
  • Users
  • Instances
  • About Bonfire
Ed Summers
@edsu@social.coop  ·  activity timestamp 3 months ago

This is a really excellent talk at #openrepos2025 by Ben Zhao about the limits of and misconceptions about LLMs, that is tailored for the #libraries #archives and #digitalpreservation crowd. He focuses in on how they work, and don't work -- and bridges that to a discussion of the impacts that the technology is having on the larger web (bots).

It gives me no measure of good vibes to know that a big crowd of people in my profession got to hear this for a keynote about AI.

https://www.youtube.com/watch?v=JicpcYwQe3w

  • Copy link
  • Flag this post
  • Block
bjb :devuannew: :emacs:
@bjb@fosstodon.org replied  ·  activity timestamp 3 months ago
@edsu
Another indication that LLM tech is reaching/has reached it's limit in capability is that the efforts to make it work "better" are using exponentially more compute - instead of just asking one question and getting an answer, we have to generate (say) 6 questions, get 6 answers, then check the 6 answers and choose one. So instead of 1 question, 18 questions plus an evaluation/comparison. Yikes.
And it is still not great at finding the right answer.
  • Copy link
  • Flag this comment
  • Block
Ed Summers
@edsu@social.coop replied  ·  activity timestamp 3 months ago
@bjb yes, he talks about that, and why it won't work
  • Copy link
  • Flag this comment
  • Block
bjb :devuannew: :emacs:
@bjb@fosstodon.org replied  ·  activity timestamp 3 months ago
@edsu He gives the example of how they are trying to improve the LLM approach (that's where I got the numbers of 6+6+6+1), but kind of glosses over the exponential-growth-of-compute-needed-for-incremental-improvement. To me, that part underlines why this approach ultimately is not going to bear fruit.
  • Copy link
  • Flag this comment
  • Block
bjb :devuannew: :emacs:
@bjb@fosstodon.org replied  ·  activity timestamp 3 months ago
@edsu
I found Prof. Zhao, tenured prof at U Chicago, described the current state of LLM research very well.
If you want to point out a snippet to stakeholders who do not have a lot of time, I recommend:
30:51 -> 34:58.
  • Copy link
  • Flag this comment
  • Block
bjb :devuannew: :emacs:
@bjb@fosstodon.org replied  ·  activity timestamp 3 months ago
@edsu
And by "you" I do not necessarily mean Ed Summers who started the thread, I mean all the thread readers.
  • Copy link
  • Flag this comment
  • Block
paulwalk
@paulwalk@mastodon.social replied  ·  activity timestamp 3 months ago
@edsu it was brilliant - I told the organisers in the feedback form that this single presentation made the trip to Chicago worthwhile. I’ve been sharing the link to slides & recording far and wide!
  • Copy link
  • Flag this comment
  • Block
Ed Summers
@edsu@social.coop replied  ·  activity timestamp 3 months ago
@paulwalk nice! it was great to hear from the applause that it was well received
  • Copy link
  • Flag this comment
  • Block
Ed Summers
@edsu@social.coop replied  ·  activity timestamp 3 months ago

PS. thanks @mjgiarlo for dropping it in the $work Slack so those of us who weren't there could benefit as well.

  • Copy link
  • Flag this comment
  • Block
Log in

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances
Bonfire social · 1.0.0-rc.3.13 no JS en
Automatic federation enabled
  • Explore
  • About
  • Members
  • Code of Conduct
Home
Login