Post

@mttaggart I wanted to write almost the same post, as this

> I let this thing into my brain, and now it is always there. For any new potential project, there is a voice in my head telling me how much easier it would be to let the model do it. How much faster it would be to simply describe the objective in a prompt and let go.

is what I'm strugling with now, but you wrote it perfectly

@mttaggart Debates aside, your thorough post was also a charm to read this gem.

"Do I think LinkedIn is the digital River Styx, where damned souls clamber over each other and claw at the boat passing overhead in the dim hope of salvation from those who have escaped the shambling horde?"

Also happy to read it works with Discourse.

@mttaggart
This might the best, most nuanced writeup I have read.

@mttaggart I feel very similar and have gone down a similar path recently. For me I've found it does very little wrong for the apps I am trying to make and I am more eager to press 2 because I enjoy the outcome and testing that outcome more then have control over the code.

I've been calling myself a self loathing AI user but I'm in the process of building 5 or 6 apps and tools I have wanted over the years but never found the effort to do.

@mttaggart "the best way to be pickled is to stay in the brine" - Gerald M. Weinberg

We are all in the brine now. Does the brine evaporate fast enough, or does it thicken?

@mttaggart thanks for writing this!

@mttaggart thanks for sharing your experience using genAI. I too struggle with it. The tech in itself may be empowering some, while at the same time corroding others to gain / keep knowledge, art & craft. The power unbalance, copyright theft, impact on labour (incl data cleaners/trainers) & climate, land & energy use makes it hard to embrace genAI as just a new technology. Currently IMHO using it is not proportional to the negative impacts it has. The Q: would it be possible to fix these?

@mttaggart

Thanks for posting this, it was very insightful.

The human factors issues you raise are important. They strike me as similar to issues with autopilot systems in aviation. If the human-in-the-loop is left with nothing more than tediously approving automated functionality then at least two problems emerge: 1) the pilots lose valuable skills that they may need in life-or-death situations, and 2) the pilots are lulled into stupor as they mindlessly monitor a stream of automated behaviors and fail to recognize and react to problems that are rare events.

The aviation industry's response has been to require manual piloting periodically to maintain proficiency, along with perhaps other interventions that I'm unfamiliar with. I'm not sure what this would look like in practice for software development. Maybe requiring developers to manually generate some features? Maybe intentionally inserting mentally-engaging tasks in the human-in-the-loop code review process?

@mttaggart Very well written and considered, thanks!

@mttaggart thanks, Matt. This is a pragmatic approach, very similar to my own style, taste and experience using Claude Code (my best creation using the tool so far has been a rust project with SSR).

I viscerally feel why this was hard to write and hard to post. There are many gifted people on this platform whose ideological opposition is a luxury most of us can’t afford. I find myself wanting to say something more nuanced in the conversations around its use and often I just don’t because I know it won’t be well received.

And like you’ve done here, what I’ve been doing isn’t really vibe coding. When you actually have experience and can read code and write good tests that is something else entirely. But yes, the skill required is the skill it atrophies. Bizarro land, but here we are. Anyway, I appreciate your willingness to share here. These are the kind of conversations that people are avoiding.

I really appreciate all the replies and support on this one. It was hard to write. I do want to call out two points that aren't being discussed, and that I felt pretty strongly about:

1. Open source is in trouble, and maintainers need help. Generative code is the help that showed up. What is the expectation here?
2. "The tool requires expertise to validate, but its use diminishes expertise and stunts its growth." What does "responsible use" look like that prevents this obvious and pervasive harm?

@mttaggart That's all pretty consistent with my experience. I've spent some time learning the tools and the underlying theory, and from earlier work I also know a bit about chips and the stack between chat and chips. On the one hand there are some major limits to how well the average person can describe the problems the want to solve. On the other I think the people building their thinking on "it's a bubble" miss that the tech is not fundamentally expensive to maintain once built.

@mttaggart
Thanks for the write-up. An interesting if tricky read. I find myself in a similar place. If we don’t become knowledgeable about these tools, able to map out failure modes and boundaries, and identify how they should be delimited and defanged, we effectively cede the floor to evangelists who won’t know or care.

@mttaggart

Wow, thanks for taking the time to write out your experience so completely. I think I’d have a similar complex reaction.

Recently developed a large multi threaded python program implementing a complex PID controlled system with lots of realtime IO from sensors and encoders and to actuators along with weak test coverage. Kids interacted with it at a science museum.

1/n

@mttaggart Great write up. I agree that there's a lot of nuance. Those solely using AI as a means for dividing the population into "ethical" and "unethical" groups, while stating "AI is a bubble", are not going to create the change they want to see. Balanced takes that insist on using tools as safely (and, ideally, as ethically) as possible are how you cross the divide.

@mttaggart Great. Sounds so familiar. I just tried repeating a git process which I knew worked because I did it yesterday using Claude as a help. Today Claude was completely off target. I had to correct it on major things many times. Was a bit shocked but....shows that one can learn from using it ....how else could I have corrected it the second time?

@mttaggart That was a thin line to walk. Nicely done. Very even keeled.

I’ve experienced the echos of what you wrote in my day job too. My team has decided it’s a tool, it will be used, and it will never be trusted. But writing SOC PowerShell scripts and KQL alerts isn’t the same as an ERP. I do not envy that team.

As for the place of agentics in a future world, I look to the automobile and the airplane as relevant comparisons. The modern world could not exist without either. I don’t think the next future can exist without neural nets and other agentics. The jury is still out for me on LLMs.

But the automobile and the airplane have wrecked this planet environmentally, though ther inventors knew not that would happen. We know better now. Has our species matured enough to not make the same mistake? Perhaps. But not everyone has. And they turned on the hard sell big time trying to get filthy rich from so called “AI” hoping no one would question it.

Fortunately (?) it’s starting to look like the economics are not sustainable. As evidence I posit that Nadella may not have a job for much longer.

Perhaps we’ll collectively get a moment to pause, reassess and reset. Less hype and a more considered approach to this tech is gravely needed. That worked after the Internet bubble of the 1990s burst. It’ll work here if we can get the more even keeled among us to take charge. May that happen soon.

@mttaggart

Thanks for this post. I've been an extreme skeptic of LLMs but seeing increasingly promising results for agentic coding. I'm not sure I'm on board with using it as regular practice but increasingly seeing the need to experiment with it to better understand how it'll impact my job, and provide better informed opinions on it.

@mttaggart I have read only the self-flagellation so far and can I just say: oof.

my own co-skeptic feeling here is that I am deeply sympathetic to what you’re trying to do here and also I am furious with your employer (or maybe just the ecosystem more generally) effectively forcing you to take a bunch of risks with this

@mttaggart
Just finished reading it. Thanks for giving me cognitive dissonance on a Monday!
Most of the questions I would have asked, you answered. It was thought provoking, and so you get a long post of thoughts, sorry in advance.

It feels like you really discounted how much your existing skills and expertise helped the process. Though I guess to mention them repeatedly would feel like self aggrandizing or something similar.
I’m glad it worked, I’m not glad that it worked. I hate that it exists, I don’t hate that it exists. I keep thinking back to the car analogy. I know that fossil fuels are destroying the planet, I still have to drive to work (I live in a place where cycling isn’t viable, and transit is almost nonexistent).
Using AI to code. It is a different set of skills (that humans are inherently not great at), it is change, and you have to pay the bills in a world where that change is already happening whether you like it or not. We didn’t start the fire. You didn’t make the systems that we live in, and living off the grid does not solve any problems (at all). We live in a society. I don’t enjoy quoting the Joker, and I think it is overused by people who use it as a cop out, an absolution from responsibility, but we do live in a society and we should do so with a level of responsibility.
Recognizing the harms and still having to use the things, it hurts. Ignorance truly is bliss. I do think we owe it to ourselves to suffer with that recognition and use it as a driving force, to drive the efficient and practical use of harmful tech, to reduce the damage inflicted. Or maybe that’s just my catholic upbringing.

You did a good job.

@mttaggart my criteria for using llms for code generation at work:
1. Internal only tool
2. Doesn't involve new ideas, just involves implementing well known design patterns
3. Doesn't directly affect anything critical
4. I could do it, and have a detailed idea of how I would implement it
5. I have a good understanding of the necessary tests and edge cases that would verify the generated code
6. I don't have the time available to set aside for implementing it in the next 6 months

@mttaggart

“I nevertheless recognize the societal and environmental harms posed by these tools. I want them to unexist. I even recognize the cognitive hazards to which I expose myself in their use (more on that later). I do not want to use them. And yet, I must understand them. If that damns me in your eyes, so be it.”

My friend, I think many of us are in exactly the same place. I hate AI (1) with every fiber of my being, but in order to secure it, I must understand it, and the best way for me to do that is to use it. 😢

(1) Generative AI, Agentic AI, etc. as opposed to traditional ML/AI

@mttaggart something I have been thinking recently, and which chimes a bit with your ultimate conclusion, is that I think of AI users a lot like smokers. 1/2

@mttaggart This fits what I’ve seen at $dayjob recently where talented and experienced people manage to sometimes get good use out of these tools (although with fewer ethical doubts than you describe). I’m mostly worried about problems caused by folks who don’t care or don’t know any better.
Successful use cases will also make it more difficult to argue against LLM use for those of us who don’t want to use them due to ethical reasons. I’m not looking forward to that.

@mttaggart ah. That’s what the vaguetoot was about.

@mttaggart Nice post. Yeah, the tipping point from these coding assistant creating slop, to being usable is a fairly recent thing. I'm not a coder, I'm a security engineer. So I'm used to handing over trust to a tool or SaaS service. Guardrails and layered controls are the key.

I think the skills we're learning right with how to make coding assistant write good code is a marketable skill. I feel like were back in the early days of the cloud learning a new skill that's cutting edge.

Anywho, just wanted to say I enjoyed reading your blog post. I too am struggling with all the complexities and externalities of AI.