A few devs on my team are convinced that their recent usage of AI has made them more productive than ever.
See if you can guess when the company began paying for AI subscriptions for the team.
---
February is low mainly because it's in progress.
Discussion
Loading...
@jamie AI usage is like a placebo effect. Everyone believes they've been “cured” now. 😂
@jaredwhite I really like the idea of comparing it to a placebo. It's easy to *feel* productive when using AI coding tools. It's a very similar dopamine hit to when you're really in the zone and I think a lot of folks are having trouble telling the difference.
But if they were more productive with it, January would've dominated previous months.
@jamie I can't think of a single product or service or tool which has noticeably improved in the past 12 months that can be attributed to gains due to AI. Not a single one.
"I'm more productive than ever!"
My dude, you're not even as productive as you were in October.
I know PRs per month isn't the ideal metric for measuring productivity. But it's a metric that AI should be absolutely crushing.
The top red line in the graph is yours truly. I rarely use any AI tooling for development. If AI made these folks faster they should be stomping me, but that's not what's happening.
It's easy to feel so much more productive when you "got a lot done" without really doing anything. It's one reason executives think everything is so easy. Don't use vibes. Measure.
@jamie any sense of what the contributing factors are (or are not)?
in my case, i'm still seeing a tremendous amount of variety in terms of what "using AI tooling" means for a team—everything from people still prompting "You are a senior software engineer..." to mammoth PRs to attempts at project/spec-driven development—so it's hard to normalize anything.
in my own case, i think i'm literally my own bottleneck. coding happens faster, but it still takes the same amount of effort to figure out what needs to be written.
@toddsundsted Yeah, it definitely seems to vary. The team's been working on putting together a place to share their prompts.
And I think that's a big part of where their time is going. Talking about AI seems to be a sort of minigame that folks are spending a lot of time on.
Your last point is also huge. Typing isn't where we spend the bulk of our time. We have a lot of legacy code and big clients with diverse and sometimes conflicting requirements. Validating change is critical.
@toddsundsted Even if AI-generated code is speeding up the writing of the code, the team still needs to understand it. I've been very vocal with questions like "Who is an expert in the code that nobody wrote?" "Who understands why we're using a table schema that nobody chose?".
It's like how, as the team has grown, we've had to take over a few of each other's in-progress PRs. Every time, that's been harder than writing it from scratch even though the code's mostly there.
@jamie @toddsundsted I can attest that I and my team have become lots more productive: it’s visible in shipped product. But it means working together on the strategies for coding with AI, sharing commands, learnings, etc. Likewise building tools to leverage AI to understand the code it or others wrote. I posted earlier about the Mermaid tool I built. I have the AI draw me sequence diagrams of flows in the code, ask questions, etc. AFAICT it’s even better for understanding code than writing it.
1 more replies (not shown)