Discussion
Moving from Training to Inference, we have Peng Wu from Meta.
The trends here match the trends in training! Heterogeneous hardware, dsitributed inference, concern over numerics and determinism -- all on the rise!
Also seeing deep consolidation in LLM serving platforms.
PyTorch has a dominant position with AI researchers. 16k papers published in the past year using PyTorch.
Community involvement also remains high! 90k stars / 25k forks on github. This leads to 14k commits made to PyTorch core over the past year
What's coolest is that there's over 1000 contributors to the codebase over the past 12 months!
The follow on effect is that 400k projects were added to github over the past twelve months that use PyTorch
Now: trends in PyTorch for Training with Edward Yang from Meta.
It's been a wild year! DeepSeek R1 was this past January, proving out the value of mixture of experts, reinforcement learning, and the shift from pre-training to post-training inference.
We also saw a huge rise in coding agents, multimodal deployment, and AI at the edge.
What does the modern stack look like for post-training RL?
Training, Inference, and Rollouts all happen under one roof, all with their own orchestration needs.
One solution to all this: Monarch (from Meta?)
The PyTorch team is also trying to solve specific problems! Like: numerics-sensitive models that make compilers sussy.
The solution here is region-based inductors with in-code annotations.
A space for Bonfire maintainers and contributors to communicate
