Sketchnote of "The Agentic AI World is Already Here... Are You Ready?" by Martin Hynie at Agile Testing Days 2025 Sketch figure with speech bubble: "I have no idea what's going on on the machine learning side of our company... got transferred and ChatGPT came out" How do we even test that? "Let's throw ChatGPT at it and see what it does" - terrifying Come up with a testing strategy - but for what? What are the sorts of biases that come into it? The LLM would defer information Built a risk map Guardrails were often too restrictive "Just give us a test plan" - but we're just playing around People want to see something to increased their confidence Insight, circled like a cloud: We're often rather an uncertainty coach - offering a sense of calm" We depended on prompt engineering to augment capabilities LLM gave horrible response - shut down quickly Built tool to evaluate LLM Partnered up with legal - worked surprisingly well - worries LLM made something up, misleading Insight, circled like a cloud: "Hallucination" is such a misnomer - it's statistical interpretation of the prompt, confabulation Used automation to detect those conversations a human needs to look at Discovered another team building the same solution... you need to advocate for your work Insight, framed: Ask better questions. Design for observability. Split systems into the slides. Sketchnote author: @lisihocke@mastodon.social