#Tag
Inspired by a talk I had with @BajoranEngineer at #PyTorchCon, I've jotted down some thoughts about #Python as a scripting engine for apps.
https://phildini.dev/python-in-every-app
Shares appreciated! Commentary welcome, but if you're a jerk I'll block you 😇
@freakboy3742 @glyph @brettcannon this is why I was asking about built python ✨
Also included: a thought on how @conda monetizes this 😅
Moving from Training to Inference, we have Peng Wu from Meta.
The trends here match the trends in training! Heterogeneous hardware, dsitributed inference, concern over numerics and determinism -- all on the rise!
Also seeing deep consolidation in LLM serving platforms.
PyTorch Compiler is a control point for all of these. Compiler can apply LLM-specific and CUDA-specific optimizations to code.
The PyTorch team is also trying to solve specific problems! Like: numerics-sensitive models that make compilers sussy.
The solution here is region-based inductors with in-code annotations.
Moving from Training to Inference, we have Peng Wu from Meta.
The trends here match the trends in training! Heterogeneous hardware, dsitributed inference, concern over numerics and determinism -- all on the rise!
Also seeing deep consolidation in LLM serving platforms.
What does the modern stack look like for post-training RL?
Training, Inference, and Rollouts all happen under one roof, all with their own orchestration needs.
One solution to all this: Monarch (from Meta?)
The PyTorch team is also trying to solve specific problems! Like: numerics-sensitive models that make compilers sussy.
The solution here is region-based inductors with in-code annotations.
It's been a wild year! DeepSeek R1 was this past January, proving out the value of mixture of experts, reinforcement learning, and the shift from pre-training to post-training inference.
We also saw a huge rise in coding agents, multimodal deployment, and AI at the edge.
What does the modern stack look like for post-training RL?
Training, Inference, and Rollouts all happen under one roof, all with their own orchestration needs.
One solution to all this: Monarch (from Meta?)
Now: trends in PyTorch for Training with Edward Yang from Meta.
It's been a wild year! DeepSeek R1 was this past January, proving out the value of mixture of experts, reinforcement learning, and the shift from pre-training to post-training inference.
We also saw a huge rise in coding agents, multimodal deployment, and AI at the edge.
What's coolest is that there's over 1000 contributors to the codebase over the past 12 months!
The follow on effect is that 400k projects were added to github over the past twelve months that use PyTorch
Now: trends in PyTorch for Training with Edward Yang from Meta.
PyTorch has a dominant position with AI researchers. 16k papers published in the past year using PyTorch.
Community involvement also remains high! 90k stars / 25k forks on github. This leads to 14k commits made to PyTorch core over the past year
What's coolest is that there's over 1000 contributors to the codebase over the past 12 months!
The follow on effect is that 400k projects were added to github over the past twelve months that use PyTorch
Next Keynote speaker! Alban Desmaison (and others!) for a #PyTorch Technical Deep Dive.
This is the PyTorch Core Maintainers talk ✨
PyTorch has a dominant position with AI researchers. 16k papers published in the past year using PyTorch.
Community involvement also remains high! 90k stars / 25k forks on github. This leads to 14k commits made to PyTorch core over the past year
Next Keynote speaker! Alban Desmaison (and others!) for a #PyTorch Technical Deep Dive.
This is the PyTorch Core Maintainers talk ✨
Next speaker: Anush Elangovan from AMD talking about Empowering Developers with AMD ROCm™ Everywhere.
ROCm is a full-stack approach for open development in machine learning and LLMs.
Three pillars:
- Ci/CD for streamlined dev
- OSS projects for first-class support
- one-click deploy of jupyter noteboooks to AMD GPUs
Next speaker: Anush Elangovan from AMD talking about Empowering Developers with AMD ROCm™ Everywhere.
Wrapping up, we need more hardware-efficient AI training.
"Hardware should be part of the constraint of algorithm design"
The true bottleneck right now is memory. Memory is a bigger bottleneck than compute.
Lots of free lunches still exist for efficiency! Being more hardware-aware will produce better results soon.
Neural networks need to be able to understand mathematical reasoning -- LEAN / LeanDojo are critical steps in this path.
This team took LEAN and built an LLM out of it, they're hoping to Open Source soon
Wrapping up, we need more hardware-efficient AI training.
"Hardware should be part of the constraint of algorithm design"
The true bottleneck right now is memory. Memory is a bigger bottleneck than compute.
Beyond modeling phenomena, these neural operators are able to do hardware designed at the quantum gate level.
(I absolutely do not know enough about the science here, but it looks cool!)
Neural networks need to be able to understand mathematical reasoning -- LEAN / LeanDojo are critical steps in this path.
This team took LEAN and built an LLM out of it, they're hoping to Open Source soon
They've completed a weather forecasting model that is 45,000 times faster than current models (not sure how measured).
The model is Apache-licensed #PyTorch code.
Beyond modeling phenomena, these neural operators are able to do hardware designed at the quantum gate level.
(I absolutely do not know enough about the science here, but it looks cool!)
NeuralOperator is the group this work is based on, check them out on github. There will also be a poster at #PyTorchCon
Now on to success stories.
They've completed a weather forecasting model that is 45,000 times faster than current models (not sure how measured).
The model is Apache-licensed #PyTorch code.
The team has developed Physics-Informed Neural Operators (PINO).
Can more accurately predict phenomena like fluid dynamics (very cool demo that's hard to capture in text)
NeuralOperator is the group this work is based on, check them out on github. There will also be a poster at #PyTorchCon
Now on to success stories.
We need new neural operators that can handle physical, mathematical domains better.
Using lower-res translations and models considered harmful! Think: Hurricanes
#PyTorchCon TorchCon
The team has developed Physics-Informed Neural Operators (PINO).
Can more accurately predict phenomena like fluid dynamics (very cool demo that's hard to capture in text)
Mathematical equations govern the world at all scales, and there's a need for physical understanding, which is outside the language-based domain of modern LLMs
We need new neural operators that can handle physical, mathematical domains better.
Using lower-res translations and models considered harmful! Think: Hurricanes
#PyTorchCon TorchCon
A space for Bonfire maintainers and contributors to communicate
