What Does a Good Spec File Look Like?

Most legacy government systems exist in a state of profound documentation poverty. The knowledge lives in the heads of retiring employees, in COBOL comments from 1987, in binders that may or may not reflect current behavior. Against this baseline, the question of what makes a “good” spec file takes on different dimensions than it might in greenfield development.

Common Elements

Any spec worth writing answers the fundamental question: what are we building and why? Beyond that, good specs share a few specific characteristics:

Clear success criteria. Not just features, but how you’ll know the thing works. This matters especially when AI agents are generating implementations—they need something concrete to validate against.

Constraints and boundaries. What’s out of scope. What technologies or patterns to use or avoid. Performance requirements. AI tools are prone to scope creep and assumption-making without explicit boundaries.

Examples of expected behavior. Concrete inputs and outputs, edge cases, error states. These serve as both specification and implicit test cases.

Context about the broader system. How this piece fits into what exists. AI assistants lack awareness of surrounding code and architectural decisions unless you tell them.

The SpecOps Context

When modernizing legacy government systems, specs serve a different purpose than typical development documentation. They’re not just implementation guides—they are artifacts that preserve institutional knowledge. This changes what “good” looks like.

A SpecOps specification document must work for multiple audiences simultaneously: domain experts who verify that the spec captures policy intent, software developers and AI coding agents who need precision to generate correct implementations, and future humans who need to understand why the system behaves a certain way years from now—possibly after everyone currently involved has moved on.

That last audience is the one most spec formats neglect entirely.

Three States, Not One

Legacy system specs can’t just describe “what the system does.” They need to distinguish between:

1. Current system behavior—what the legacy code actually does today, bugs and all
2. Current policy requirements—what the system should do according to governing statutes and regulations
3. Technical constraints—what the system cannot do regardless of policy, due to missing integrations or platform limitations

These three things can be in alignment or tension at any moment. And that alignment can shift over time without the code changing—a policy update tomorrow can transform compliant behavior into a violation.

Known Deviation Patterns

Consider the example of a benefits system that should verify income against a state tax agency records, but the legacy system only captures self-reported income because the integration with the tax agency was never built. A good spec would make this explicit:

Policy requirement: Per [directive], applicant income must be verified against tax agency records prior to benefit approval.

Current implementation: Self-reported income only. Applicant provides income information on Form X.

Deviation reason: No interface to tax agency income verification service exists. Integration requested in 2019, not funded.

Modernization note: Modern implementation should include tax agency income verification integration.

This surfaces the gap, documents why it exists, and gives the modernization effort clear direction—without pretending the legacy system does something it doesn’t.

Explicit Ambiguity as a Feature

There’s something that seems almost radical about a methodology that says write down what you don’t know. Traditional documentation can project false confidence. It often describes how things should work and quietly omits the messy parts.

A spec that explicitly marks areas of tension or uncertainty is more honest, more useful for risk assessment, and a better starting point for modernization. It’s an invitation for future clarification rather than a false endpoint.

A spec with unresolved tension is better than no reviewable documentation at all. 

Policy Grounding

Government system specs need explicit links to authorizing statutes, regulations, or directives. Not just “these items are excluded from income calculations” but “per 42 USC § 1382a, the following items are excluded from income calculations”

This is the why that survives personnel turnover. It’s what allows future teams to evaluate whether behavior that was correct five years ago still aligns with current policy.

Decision Records

When domain experts verify a spec, they make judgment calls—especially where legacy behavior diverges from current policy understanding. Those decisions need to be captured in the spec, not in a separate document that gets lost.

The spec becomes the repository of institutional reasoning, not just institutional behavior.

Accessible or Precise?

The SpecOps approach says that specs should be “readable by domain experts while detailed enough to guide implementation.” This is genuinely hard.

Options include stratified specs (plain-language summaries with expandable technical detail), executable specs (written as tests that are simultaneously human-readable and machine-verifiable), or annotated specs (a single verbose document where technical precision is explained inline).

Given that the spec is meant to be the source of truth that outlasts implementations, keeping everything in one artifact—even at the cost of verbosity—reduces the risk of layers drifting apart over time.

The Road Ahead

We’re still in early days. Questions remain open:

• How granular should policy references be?
• What’s the right way to represent known deviations?
• How should specs age—versioning, or is git history enough?
• What level of detail helps AI agents versus adding noise?

These will get answered empirically as more agencies adopt the approach. The methodology will evolve. The important thing is to start—to surface questions that were previously invisible, to give future teams something to interrogate rather than nothing at all.

Because the knowledge is what matters. Everything else is implementation details.

#AI #artificialIntelligence #ChatGPT #governmentServices #legacySystems #systemModernization

Three Projects I’m Proud of in 2025

As the year winds down, I wanted to share three projects that I’m particularly proud of working on in 2025, each focusing on how AI is changing the way we build and deliver government digital services.

What’s kind of surprising to me is that all of this work happened in the second half of the year. That’s less a testament to my productivity and more a reflection of how quickly this space is changing. The tools and capabilities that made these projects feasible for me to do simply weren’t there six months earlier.

Designing for Delegation

This year I spent a lot of time thinking about what happens when AI agents can act on people’s behalf. Instead of citizens navigating complex government forms and processes themselves, what if they could delegate those tasks to an AI agent that handles the complexity for them?

I wrote about this idea in the post Designing for Delegation and built delegation.design as a resource for the design patterns government teams will need as this shift happens. The key takeaway for me is that we need to stop thinking only about designing traditional interfaces and start designing delegation relationships. We need to start considering how people establish boundaries, how they stay informed about what’s happening, and how they maintain control.

This is sort of related to another post I wrote with my colleague Dan Munz on the end of the Civic Tech’s interface era. People’s expectations for how they can interact with their government are changing. So we must change the way we design and build government digital services.

ATLAS ATO Accelerator

Getting software through the federal Authority to Operate process is painful. I started exploring how well-structured Terraform code could serve as compliance documentation. Focusing not just on infrastructure-as-code, but on compliance-as-code.

The ATLAS ATO Accelerator project takes this further by creating AI-readable instructions for FedRAMP compliance. The idea is that AI coding agents can use these instructions to generate infrastructure that’s compliant by default, dramatically reducing the time and effort agencies spend on security documentation.

SpecOps

Legacy system modernization is hard, and most AI-assisted approaches focus narrowly on translating old code into new code. But when AI translates code it doesn’t understand, there’s a high probability that errors will get baked into the new system.

The SpecOps approach flips this traditional understanding of legacy system modernization around: the software specification – not the software code – is the source of truth. Legacy modernizations should begin by extracting knowledge and documenting how a system is supposed to work. Then, that specification can be used to guide modernization via spec-driven development. The spec generated through SpecOps outlasts any particular technology stack, so when today’s modern code becomes tomorrow’s legacy system, agencies still have the authoritative system documentation.

All three projects share a common thread: using AI to make government technology work better, whether that’s improving citizen experiences, accelerating compliance, or modernizing legacy systems.

I’m excited to keep building on these ideas in 2026.

#AI #artificialIntelligence #ChatGPT #government #llm #technology

What Does a Good Spec File Look Like?

Most legacy government systems exist in a state of profound documentation poverty. The knowledge lives in the heads of retiring employees, in COBOL comments from 1987, in binders that may or may not reflect current behavior. Against this baseline, the question of what makes a “good” spec file takes on different dimensions than it might in greenfield development.

Common Elements

Any spec worth writing answers the fundamental question: what are we building and why? Beyond that, good specs share a few specific characteristics:

Clear success criteria. Not just features, but how you’ll know the thing works. This matters especially when AI agents are generating implementations—they need something concrete to validate against.

Constraints and boundaries. What’s out of scope. What technologies or patterns to use or avoid. Performance requirements. AI tools are prone to scope creep and assumption-making without explicit boundaries.

Examples of expected behavior. Concrete inputs and outputs, edge cases, error states. These serve as both specification and implicit test cases.

Context about the broader system. How this piece fits into what exists. AI assistants lack awareness of surrounding code and architectural decisions unless you tell them.

The SpecOps Context

When modernizing legacy government systems, specs serve a different purpose than typical development documentation. They’re not just implementation guides—they are artifacts that preserve institutional knowledge. This changes what “good” looks like.

A SpecOps specification document must work for multiple audiences simultaneously: domain experts who verify that the spec captures policy intent, software developers and AI coding agents who need precision to generate correct implementations, and future humans who need to understand why the system behaves a certain way years from now—possibly after everyone currently involved has moved on.

That last audience is the one most spec formats neglect entirely.

Three States, Not One

Legacy system specs can’t just describe “what the system does.” They need to distinguish between:

1. Current system behavior—what the legacy code actually does today, bugs and all
2. Current policy requirements—what the system should do according to governing statutes and regulations
3. Technical constraints—what the system cannot do regardless of policy, due to missing integrations or platform limitations

These three things can be in alignment or tension at any moment. And that alignment can shift over time without the code changing—a policy update tomorrow can transform compliant behavior into a violation.

Known Deviation Patterns

Consider the example of a benefits system that should verify income against a state tax agency records, but the legacy system only captures self-reported income because the integration with the tax agency was never built. A good spec would make this explicit:

Policy requirement: Per [directive], applicant income must be verified against tax agency records prior to benefit approval.

Current implementation: Self-reported income only. Applicant provides income information on Form X.

Deviation reason: No interface to tax agency income verification service exists. Integration requested in 2019, not funded.

Modernization note: Modern implementation should include tax agency income verification integration.

This surfaces the gap, documents why it exists, and gives the modernization effort clear direction—without pretending the legacy system does something it doesn’t.

Explicit Ambiguity as a Feature

There’s something that seems almost radical about a methodology that says write down what you don’t know. Traditional documentation can project false confidence. It often describes how things should work and quietly omits the messy parts.

A spec that explicitly marks areas of tension or uncertainty is more honest, more useful for risk assessment, and a better starting point for modernization. It’s an invitation for future clarification rather than a false endpoint.

A spec with unresolved tension is better than no reviewable documentation at all. 

Policy Grounding

Government system specs need explicit links to authorizing statutes, regulations, or directives. Not just “these items are excluded from income calculations” but “per 42 USC § 1382a, the following items are excluded from income calculations”

This is the why that survives personnel turnover. It’s what allows future teams to evaluate whether behavior that was correct five years ago still aligns with current policy.

Decision Records

When domain experts verify a spec, they make judgment calls—especially where legacy behavior diverges from current policy understanding. Those decisions need to be captured in the spec, not in a separate document that gets lost.

The spec becomes the repository of institutional reasoning, not just institutional behavior.

Accessible or Precise?

The SpecOps approach says that specs should be “readable by domain experts while detailed enough to guide implementation.” This is genuinely hard.

Options include stratified specs (plain-language summaries with expandable technical detail), executable specs (written as tests that are simultaneously human-readable and machine-verifiable), or annotated specs (a single verbose document where technical precision is explained inline).

Given that the spec is meant to be the source of truth that outlasts implementations, keeping everything in one artifact—even at the cost of verbosity—reduces the risk of layers drifting apart over time.

The Road Ahead

We’re still in early days. Questions remain open:

• How granular should policy references be?
• What’s the right way to represent known deviations?
• How should specs age—versioning, or is git history enough?
• What level of detail helps AI agents versus adding noise?

These will get answered empirically as more agencies adopt the approach. The methodology will evolve. The important thing is to start—to surface questions that were previously invisible, to give future teams something to interrogate rather than nothing at all.

Because the knowledge is what matters. Everything else is implementation details.

#AI #artificialIntelligence #ChatGPT #governmentServices #legacySystems #systemModernization

Maybe We Shouldn’t Call Them AI “Agents”

Beware of pretty faces that you find. A pretty face can hide an evil mind.
– Johnny Rivers, Secret Agent Man

As artificial intelligence capabilities expand into government service delivery, it’s worth pausing to think carefully about the language we’re using. The terms “agentic services” and “agentic AI” have gained significant traction in the tech industry, and for good reason — it captures something important about AI systems that can act autonomously. I myself am as guilty as anyone of using this term frequently. But for those of us working in government contexts, there are some considerations worth keeping in mind.

The “Agent” Problem in Government

In government, the word “agent” carries particular connotations. FBI agents. Border patrol agents. IRS agents. These are enforcement and investigative roles. When citizens hear “government agent,” they often think of authority, compliance, and oversight — not helpful service delivery.

This isn’t an insurmountable problem, but it’s worth being aware of. The language we choose shapes how citizens perceive and respond to new service models. If we’re trying to build trust in AI-enabled services, starting with terminology that might trigger concerns about surveillance or enforcement may not be ideal.

(And yes, for a certain generation, The Matrix movies didn’t exactly help the cultural perception of “agents” either. 😅)

What the term “agents” might obscure

There’s a deeper consideration beyond just the word “agent” itself. Calling these services “agentic” can make them sound radically new — a complete departure enabled by cutting-edge AI. But that framing might obscure an important reality.

Delegation-based government services aren’t new. They’ve existed for decades, and are extremely common today.

Tax preparers handle filing returns on behalf of clients. Immigration attorneys navigate visa applications. Customs brokers manage import/export documentation for businesses. Permit expediters guide building approval processes. Benefits navigators help people apply for disability or veterans services.

These are all delegation relationships. Citizens hand over complex, high-stakes government interactions to trusted specialists who handle the administrative burden on their behalf. AI doesn’t enable this service delivery paradigm, but it does potentially make it more scalable and affordable.

Why Words Matter

Thinking about these services as “delegation-based” rather than simply “agentic” opens up useful design questions.

When you frame it as delegation, you can look to existing delegation relationships for guidance. What makes someone comfortable delegating their tax filing to a CPA? What trust factors matter when hiring an immigration attorney? These aren’t abstract questions — there are decades of real-world answers.

The language of delegation also centers the citizen experience more clearly. It’s not about what the AI can do autonomously; it’s about what citizens are willing to hand over and under what conditions. That subtle shift in framing can lead to different design choices around transparency, control and oversight.

Moving Forward

This isn’t a call to abandon the term “agentic services” entirely. It’s widely used in industry, and there’s value in using common language when talking with technology partners and vendors.

But maybe for internal discussions, policy development, and especially citizen-facing communications, it might be worth experimenting with terms like “delegation-based services” or similar language. It acknowledges continuity with existing practices, avoids potentially problematic associations with “government agents,” and keeps the focus on what citizens are actually doing: choosing to delegate burdensome tasks while maintaining appropriate oversight and accountability.

The technology may be new, but the underlying service delivery paradigm isn’t. Our language should reflect that.

Note – this post originally appeared on GovLoop.

#agent #AI #artificialIntelligence #ChatGPT #serviceDelivery

AI Agents aren’t as radical as they sound

When we talk about AI agents handling government services, it can feel totally far-fetched and radically new, like something from a distant future. But delegation of government services is already happening all around us.

People routinely delegate high-stakes government interactions to:

• Tax preparers for filing returns
• Immigration attorneys for visa applications
• Permit expediters for building approvals
• Benefits navigators for disability claims
• Customs brokers for import/export documentation

To think about the potential for AI agents, I created this visualization to better understand why people delegate these tasks. Mapping services by administrative burden and frequency reveals a clear pattern: the upper-left quadrant of this chart (high burden, low frequency) is where delegation makes most sense. For these kinds of services, citizens don’t build up expertise because these tasks are relatively rare – this is where specialists who can be delegated to thrive.

The color coding of the chart highlights something important: many of these prime delegation candidates are critical or high stakes transactions. If they are not completed properly, there are potentially significant consequences. These are exactly the kinds of tasks people already pay professionals to handle.

This existing delegation landscape can inform how we design for AI agents.

What makes someone comfortable handing sensitive financial data to a tax preparer or personal information to an immigration attorney? These delegation-based services have the following characteristics:

• Professional credentials and accountability
• Transparent process (you see the forms before submission)
• Ability to override decisions
• Clear recourse when mistakes happen
• Legal liability

These things aren’t just nice-to-haves. They are the foundation of trust in delegation relationships. As we design AI agent systems for government services, we need equivalent trust mechanisms. These mechanisms won’t look identical to the real world delegation-based relationships we have today (i.e., no personal relationship with the professional offering support, different accountability structures), but they must address the same fundamental human needs for transparency, control, and recourse.

The future of government AI agents isn’t about inventing something entirely new. It’s about understanding what already works in delegation relationships and translating those trust factors into AI systems.

#AI #AIGovernance #artificialIntelligence #ChatGPT #civictech #DigitalGovernment #GovTech #machineLearning #ServiceDesign #technology

#OpenAI Has a Fix For Hallucinations, But You Really Won't Like It. Via @sciencealert #AI #ArtificialIntelligence 💻 🤖 🧠 #LLM #ChatGPT

OpenAI Has a Fix For Hallucina...

#OpenAI Has a Fix For Hallucinations, But You Really Won't Like It. Via @sciencealert #AI #ArtificialIntelligence 💻 🤖 🧠 #LLM #ChatGPT

OpenAI Has a Fix For Hallucina...

The Third Wave of Government Disruption

When printed telephone directories first started including blue pages for government offices in the 1950s and 60s, they created a new expectation: citizens should be able to reach their government by phone. The Internet revolution of the 1990s raised these expectations exponentially—if you could bank online and shop on Amazon, why couldn’t you renew your license or apply for benefits with the same ease?

Now, with 34% of U.S. adults having used ChatGPT—roughly double the share since 2023—we’re witnessing the third major wave of technology-driven transformation in how citizens expect to interact with their government. And once again, we’re watching the same pattern unfold: rapid consumer adoption creating new expectations, followed by delayed government adaptation, followed (potentially) by a long period of playing catch-up.

The difference this time? The stakes are higher, the pace is faster, and the consequences of falling behind may be more severe than ever.

Three Waves of Technological Disruption

Each wave of technological change outlined above has followed a similar trajectory, but with accelerating speed:

The telephone era unfolded over decades. Telephone adoption began in the late 1870s as an expensive luxury for the wealthy, with monthly costs of $20-40 (equivalent to $500-1,000 today). It took until the mid-20th century for phones to become commonplace in households. Governments had time to establish call centers and phone-based services without fundamentally redesigning how it operated. The pace was manageable—measured in decades, not in years or months.

The Internet era compressed this timeline to years. Internet users exploded from 45 million in 1996 to 407 million by 2000—a ninefold increase in just four years. Citizens who could accomplish complex tasks online in minutes naturally expected similar efficiency from their government. But while private companies were redesigning their entire business models around digital capabilities, governments largely treated the Internet as a new channel for existing processes.

The AI era is compressing change to months. Generative AI has been adopted at a faster pace than PCs or the internet, with breakthroughs moving from laboratory to widespread deployment in timeframes that would have seemed impossible just a few years ago.

The Structural Challenge: Democracy vs. Speed

As I’ve written about extensively before, governments aren’t slow at technology adoption by accident—they’re designed that way. The very features that are intended to make democratic government more trustworthy and accountable also make it structurally unsuited for rapid technological change.

The classic example is government procurement. The average technology buying cycle for government is 22 months compared to 6-7 months in the private sector. These delays aren’t the result of bureaucratic incompetence—they’re the deliberate result of requirements designed to help ensure fairness, transparency, and accountability. Public bid posting periods, vendor diversity requirements, the acquisition of performance bonds, and detailed financial scrutiny all represent important values imbued in public procurement processes. But they can also add months to timelines in a world where technology solutions can have shorter development cycles than government procurement processes.

The same pattern can be seen across government operations. Budget processes designed to prevent waste and enable legislative oversight create “use it or lose it” dynamics that discourage efficiency innovations. Civil service systems meant to prevent patronage and ensure merit-based hiring create lengthy processes that struggle to compete for scarce technical talent against private companies that can hire faster and pay more.

These aren’t bugs—they’re features. The transparency requirements, deliberative processes, and risk aversion that can slow down government technology adoption exist to uphold fundamental values and principles. The design of these processes is deliberate. The problem is that these principles are increasingly in tension with the pace of technological change.

The Compounding Crisis

This structural mismatch becomes more problematic with each technological wave because the pace of change keeps accelerating while government processes remain largely constant. What was a manageable gap during the telephone era became a significant lag during the Internet era and is becoming an existential challenge in the AI era.

The Internet wave provides a sobering lesson in the cost of delayed adaptation. Despite clear evidence throughout the 1990s that digital services were transforming how people expected to interact with institutions, most governments were slow to recognize that the rapid evolution of the Internet was changing people’s expectations for how they communicated and interacted with their government. Two decades later, we’re still playing catch-up, retrofitting digital services onto processes designed for paper-based workflows, and struggling to make basic websites and online services accessible.

The consequences aren’t just about inefficiency, they are about the loss of public trust. When citizens can accomplish complex tasks seamlessly with private companies but struggle with basic government services, the contrast erodes confidence in government competence and accountability.

The Stakes Are Higher

The AI wave presents an even greater challenge because it doesn’t just change how governments deliver services—it potentially changes how governments make decisions. Unlike previous technological waves that primarily affected operational efficiency, AI touches the core of democratic governance: the exercise of judgment and discretion in applying laws and policies to individual circumstances.

The stakes couldn’t be higher. As an example, when Spain implemented an algorithmic system to assess domestic violence risk, the software became “so woven into law enforcement that it is hard to know where its recommendations end and human decision-making begins.” Tragically, people assessed as low-risk still became victims of violence.

This use of AI in decision making processes highlights a troubling pattern identified by researchers like Virginia Eubanks in her analysis of Pennsylvania’s Allegheny Family Screening Tool. While AI systems are meant to “support, not supplant, human decision-making,” in practice “the algorithm seems to be training the intake workers.” Staff begin to defer to algorithmic judgments, believing the model is less fallible than human screeners.

The “human-in-the-loop” approach—where people supposedly maintain oversight of AI decisions—may not be sufficient protection against the human tendency to cede authority to software. When New York City’s AI chatbot tells businesses they can take workers’ tips and that landlords can discriminate based on source of income, both illegal, it demonstrates how AI systems can undermine the rule of law even in seemingly routine interactions.

The acceleration of AI adoption in government is happening precisely in contexts where lives hang in the balance—decisions about protection from violence, child welfare, emergency response, and access to vital resources. Unlike the more gradual telephone and Internet adoption cycles that gave governments some time—limited as it was—to learn and adapt, AI deployment can sometimes happen without proper safeguards, training, or accountability mechanisms in place.

Getting It Right This Time

The lesson from previous technological waves is clear: the cost of delayed or unorganized adaptation grows exponentially. Governments that fell behind during the Internet era spent decades and billions of dollars trying to catch up, often with mixed results. With AI moving even faster and touching more fundamental aspects of governance, the penalty for falling behind again could be severe.

But speed without safeguards is equally dangerous. The challenge isn’t choosing between moving fast and maintaining accountability—it’s developing the capacity to do both simultaneously. This means building safeguards into the adoption process from the start, not retrofitting them later. It means creating review mechanisms that can operate at the speed of technology development, not the traditional pace of government oversight.

The solution requires adapting democratic processes for technological speed without abandoning democratic values. This means creating “fast lanes” for certain types of technological adoption while maintaining rigorous oversight. It means developing rapid-response teams for AI evaluation that include technical experts, legal reviewers, and community representatives. It means investing in government workforce development so staff can properly assess and oversee AI systems rather than simply defer to them.

Most importantly, it means recognizing that the structural challenges governments face with technology adoption aren’t bugs in the system—they’re features designed to serve important functions. The transparency requirements, deliberative processes, and accountability mechanisms that slow government down exist for a reason. The question isn’t how to eliminate these constraints, but how to redesign them so they can operate effectively when technological change happens faster than traditional democratic processes were designed to accommodate.

As this historical pattern of technology adoption has advanced, governments have played catch-up before, each time with higher stakes and less time to adapt. Given the pace and implications of AI adoption in government services, we can’t afford to play catch up again.

#technology#AI #ChatGPT #artificialIntelligence #business #government #GenAI #Procurement

A depressing fable about how ChatGPT is corroding trust in scholarship

In preparation for next week’s keynote on generative AI and the crisis of trust, I picked up a book about trust by a philosopher, who I’ve decided not to name, when I saw it in the Tate bookshop earlier today. It began with a quote from bell hooks which caught my attention:

Trust is both a personal and a political endeavour, an affirmation of our shared humanity and our collective potential for growth and transformation. By embracing trust, by fostering connections, grounded in love and compassion, we have the power to not only change our own lives but also to reshape the world around us…

I wanted to post it on my blog, so I immediately looked for a citation. I could find no result for the exact quote but Google returned this site at the top of the list, where I found nearly the same quote:

In the end, trust is both a personal and a political endeavor, an affirmation of our shared humanity and our collective potential for growth and transformation. By embracing trust, by fostering connections grounded in love and compassion, we have the power to not only change our own lives but also to reshape the world around us, one relationship at a time.

The problem is that this site hosts imagined responses by philosophers to the question ‘what is trust?’ produced by ChatGPT. These (genuinely quite interesting) LLM outputs were posted in April 2023, only to feature in a book published in 2024. I can find no other source for the quote the author includes, other than this nearly exact quote produced by ChatGPT.

The most obvious explanation here is that they decided they want to start the book with a quote from bell hooks. They then typed in ‘bell hooks and trust’ which returns the site above as its second result. They didn’t read the introduction which explains the exercise with the LLM and instead copy & pasted the ChatGPT output into his book, without checking for the source of the citation.

The irony being that I now don’t trust the rest of the book. A philosopher writing a book about trust begins the book with such lazy scholarship that I now struggle to trust them. I hope I’m wrong. But without wishing to personalise things, I’m tempted to use this an example in next week’s keynote. It illustrate how LLMs are contributing to an environment in which lazy scholarship, cherry picking a quote from a google search, becomes much riskier given the circulation of synthetic content.

#AI #artificialIntelligence #ChatGPT #generativeAI #PascalGielen #scholarship #technology #trust #writing

For anyone wondering why #gravy has been trending: AI hucksters are trawling through your social media posts for training data and trends.

And you know what can gum up the gears of an automated sentence generator?

Posts that use the word gravy out of context.

#auspol #ChatGPT #AI #LLM

For anyone wondering why #gravy has been trending: AI hucksters are trawling through your social media posts for training data and trends.

And you know what can gum up the gears of an automated sentence generator?

Posts that use the word gravy out of context.

#auspol #ChatGPT #AI #LLM

Une extension indispensable signée Next pour lutter contre la désinformation massive des IA génératives

Bonjour,

Au cours de nos revues sur l’IA, un point a vite été constaté : les IA génératives (ChatGPT et consorts) produisent énormément de contenus erronés (appelés hallucinations), ce qui peut mener à de la désinformation. Nous considérons d’ailleurs ces outils comme mauvais pour s’informer sur un sujet ou suivre l’actualité, comme dit dans les tests effectués en Mars 2025 pour savoir si on était plus efficace ou non avec une IA générative.

Si les modèles de langages utilisés ont évolué depuis, cela ne corrige pas le problème. C’est tout le contraire même, c’est de pire en pire comme constaté et partagé par OpenAI et évoqué dans cet article écrit en Mai.

On a aussi eu récemment l’audition de Luc Julia (concepteur de Siri) au Sénat affirmant que les IA ne sont pas fiables, sont de plus en plus mauvaises et qu’il est impératif de relire ce qu’elles génèrent. Ci-dessous l’extrait vidéo :

https://youtu.be/M0CrfxSqP4Q

Le plus simple serait donc de ne pas utiliser ces outils pour s’informer ou d’être très attentifs à ce qui est pondu et de vérifier ce qui est dit systématiquement.

Le problème, c’est que le contenu généré par IA gangrène le web et ne se limite plus à ces outils… On peut donc y être confronté sans forcément s’en rendre compte.

Pour prendre un exemple récent, le 12 Juin dernier un Boeing 787 s’est écrasé en Inde faisant de nombreuses victimes. Google AI Overview qui est intégrée dans le moteur de recherche Google (aux US du moins), est une IA proposant une réponse positionnée avant les résultats de recherche traditionnels. Eh bien, grâce à l’IA, le crash est devenu celui d’un Airbus A330 !

Il y a encore plus vicieux, aujourd’hui des sites sont entièrement générés par l’IA et se présentent à tort comme des sites d’information. Il est donc plus difficile de déceler le vrai du faux, surtout que certains articles sont référencés dans Google Actualités.

Heureusement, il existe des solutions pour tenter d’y voir plus clair. En dehors du fait que je déconseille fortement l’utilisation directe de l’IA pour faire de la recherche d’informations, je conseille également d’installer sur vos navigateurs l’excellente extension « Alerte sur les sites GenAI » développée par Next et disponible sur Firefox (et dérivés comme Librewolf) et Chromium (et ceux basés dessus comme Chrome ou Vivaldi).

Cette extension a pour but d’alerter si vous visitez une site généré en partie ou totalement par de l’IA. Lorsque c’est le cas, une pop-up comme ci-dessous s’affiche :

Cela permet de sensibiliser quant à la véracité du contenu et également de rappeler que bientôt, en Europe, les sites présentant du contenu généré par une IA auront l’obligation de l’afficher clairement. Une fois la pop-up fermée, le site est accessible et vous pouvez accéder au contenu si vous le souhaitez. Aucun blocage donc, juste un avertissement pour savoir dans quoi on embarque.

Personnellement, j’aime beaucoup cette extension ! Dès que je vois ce bandeau, je ne cherche pas à comprendre et je vais directement chercher l’information ailleurs vu que je n’accorde aucune confiance au contenu généré par IA et que je ne souhaite pas m’attarder davantage sur un site recourant à cela.

Et ce qui me rassure, c’est que pour le moment, je n’ai vu ce pop-up que sur des liens que l’on m’a envoyé donc mes lectures habituelles semblent bien être réalisées par des êtres humains !

Bonne découverte et utilisation 🙂

#AI #AIOverview #ChatGPT #Chromium #Copilot#Firefox #Gemini #IA #Next

A depressing fable about how ChatGPT is corroding trust in scholarship

In preparation for next week’s keynote on generative AI and the crisis of trust, I picked up a book about trust by a philosopher, who I’ve decided not to name, when I saw it in the Tate bookshop earlier today. It began with a quote from bell hooks which caught my attention:

Trust is both a personal and a political endeavour, an affirmation of our shared humanity and our collective potential for growth and transformation. By embracing trust, by fostering connections, grounded in love and compassion, we have the power to not only change our own lives but also to reshape the world around us…

I wanted to post it on my blog, so I immediately looked for a citation. I could find no result for the exact quote but Google returned this site at the top of the list, where I found nearly the same quote:

In the end, trust is both a personal and a political endeavor, an affirmation of our shared humanity and our collective potential for growth and transformation. By embracing trust, by fostering connections grounded in love and compassion, we have the power to not only change our own lives but also to reshape the world around us, one relationship at a time.

The problem is that this site hosts imagined responses by philosophers to the question ‘what is trust?’ produced by ChatGPT. These (genuinely quite interesting) LLM outputs were posted in April 2023, only to feature in a book published in 2024. I can find no other source for the quote the author includes, other than this nearly exact quote produced by ChatGPT.

The most obvious explanation here is that they decided they want to start the book with a quote from bell hooks. They then typed in ‘bell hooks and trust’ which returns the site above as its second result. They didn’t read the introduction which explains the exercise with the LLM and instead copy & pasted the ChatGPT output into his book, without checking for the source of the citation.

The irony being that I now don’t trust the rest of the book. A philosopher writing a book about trust begins the book with such lazy scholarship that I now struggle to trust them. I hope I’m wrong. But without wishing to personalise things, I’m tempted to use this an example in next week’s keynote. It illustrate how LLMs are contributing to an environment in which lazy scholarship, cherry picking a quote from a google search, becomes much riskier given the circulation of synthetic content.

#AI #artificialIntelligence #ChatGPT #generativeAI #PascalGielen #scholarship #technology #trust #writing