AI-Powered Automation: Taking ATO Modernization Beyond the Bottleneck

A recent article in U.S. Cybersecurity Magazine, “The ATO Bottleneck: Rethinking Responsibility and Enabling Automation,” makes a compelling case for transforming the federal Authority to Operate (ATO) process. The authors argue that the current manual, documentation-heavy approach—which can take months or even years—must evolve into an automated, continuous compliance model. Their vision: cut ATO cycles by 40-60% through Secure-by-Design principles, DevSecOps integration, and real-time evidence generation.

The Case for Automation

The article identifies several critical problems with traditional ATO processes:

• Static documentation that becomes outdated as systems evolve
• Manual evidence gathering that consumes months of effort
• Reactive compliance where security is retrofitted rather than built-in
• Knowledge silos that bottleneck approvals with specialized expertise

Their solution emphasizes automation tools that integrate with DevSecOps pipelines, continuous monitoring platforms that generate live compliance dashboards, and Secure-by-Design practices that embed security controls directly into infrastructure code from day one. The goal: transform ATO from a bureaucratic gate into a dynamic trust mechanism.

AI Takes It Further

While the article champions automation for validating and collecting evidence, AI-powered approaches like the ATLAS ATO Accelerator extend this vision by also automating compliant code generation. Here’s how AI enhances the automation framework:

Knowledge Democratization: The article emphasizes training programs to teach developers RMF and DevSecOps principles. The ATLAS approach encodes this expert knowledge into AI agent instruction files, making compliance guidance accessible at the moment of code creation—no deep NIST expertise required.

Generative Compliance: Rather than just scanning existing code for violations, the ATLAS approach uses AI to generate infrastructure that’s compliant from the start. The approach focuses on generating Infrastructure as Code artifacts with compliance controls built in, and clearly document, significantly reducing the time required to gather and document artifacts when the time comes.

Adaptive Guidance: As requirements evolve—new NIST revisions, emerging threats, updated frameworks—AI agent instructions (which are art the heart of the ATLAS approach) can be updated once and propagate consistently across all projects, addressing the article’s concern about “shifting interpretations” and “changing requirements.”

From Automation to Intelligence

The authors are right to point out that automation tools can “cut ATO cycles by an estimated 40-60% range.” AI-guided generation has the potential to push this further by preventing compliance gaps before they occur. When security controls, proper documentation patterns, and ATO readiness are embedded into the IaC generation process itself, teams spend less time on rework and more time on innovation.

The article’s vision of continuous ATO, living documentation, and Secure-by-Design is the right direction. AI-powered tools like those documented in the ATLAS approach demonstrate how to operationalize that vision—not just automating the compliance process, but making secure, ATO-ready infrastructure the natural default output of modern development workflows.

Read the full article: The ATO Bottleneck: Rethinking Responsibility and Enabling Automation

#AI #artificialIntelligence #ATO #business #ChatGPT #government #llm #technology

AI-Powered Automation: Taking ATO Modernization Beyond the Bottleneck

A recent article in U.S. Cybersecurity Magazine, “The ATO Bottleneck: Rethinking Responsibility and Enabling Automation,” makes a compelling case for transforming the federal Authority to Operate (ATO) process. The authors argue that the current manual, documentation-heavy approach—which can take months or even years—must evolve into an automated, continuous compliance model. Their vision: cut ATO cycles by 40-60% through Secure-by-Design principles, DevSecOps integration, and real-time evidence generation.

The Case for Automation

The article identifies several critical problems with traditional ATO processes:

• Static documentation that becomes outdated as systems evolve
• Manual evidence gathering that consumes months of effort
• Reactive compliance where security is retrofitted rather than built-in
• Knowledge silos that bottleneck approvals with specialized expertise

Their solution emphasizes automation tools that integrate with DevSecOps pipelines, continuous monitoring platforms that generate live compliance dashboards, and Secure-by-Design practices that embed security controls directly into infrastructure code from day one. The goal: transform ATO from a bureaucratic gate into a dynamic trust mechanism.

AI Takes It Further

While the article champions automation for validating and collecting evidence, AI-powered approaches like the ATLAS ATO Accelerator extend this vision by also automating compliant code generation. Here’s how AI enhances the automation framework:

Knowledge Democratization: The article emphasizes training programs to teach developers RMF and DevSecOps principles. The ATLAS approach encodes this expert knowledge into AI agent instruction files, making compliance guidance accessible at the moment of code creation—no deep NIST expertise required.

Generative Compliance: Rather than just scanning existing code for violations, the ATLAS approach uses AI to generate infrastructure that’s compliant from the start. The approach focuses on generating Infrastructure as Code artifacts with compliance controls built in, and clearly document, significantly reducing the time required to gather and document artifacts when the time comes.

Adaptive Guidance: As requirements evolve—new NIST revisions, emerging threats, updated frameworks—AI agent instructions (which are art the heart of the ATLAS approach) can be updated once and propagate consistently across all projects, addressing the article’s concern about “shifting interpretations” and “changing requirements.”

From Automation to Intelligence

The authors are right to point out that automation tools can “cut ATO cycles by an estimated 40-60% range.” AI-guided generation has the potential to push this further by preventing compliance gaps before they occur. When security controls, proper documentation patterns, and ATO readiness are embedded into the IaC generation process itself, teams spend less time on rework and more time on innovation.

The article’s vision of continuous ATO, living documentation, and Secure-by-Design is the right direction. AI-powered tools like those documented in the ATLAS approach demonstrate how to operationalize that vision—not just automating the compliance process, but making secure, ATO-ready infrastructure the natural default output of modern development workflows.

Read the full article: The ATO Bottleneck: Rethinking Responsibility and Enabling Automation

#AI #artificialIntelligence #ATO #business #ChatGPT #government #llm #technology

What Does a Good Spec File Look Like?

Most legacy government systems exist in a state of profound documentation poverty. The knowledge lives in the heads of retiring employees, in COBOL comments from 1987, in binders that may or may not reflect current behavior. Against this baseline, the question of what makes a “good” spec file takes on different dimensions than it might in greenfield development.

Common Elements

Any spec worth writing answers the fundamental question: what are we building and why? Beyond that, good specs share a few specific characteristics:

Clear success criteria. Not just features, but how you’ll know the thing works. This matters especially when AI agents are generating implementations—they need something concrete to validate against.

Constraints and boundaries. What’s out of scope. What technologies or patterns to use or avoid. Performance requirements. AI tools are prone to scope creep and assumption-making without explicit boundaries.

Examples of expected behavior. Concrete inputs and outputs, edge cases, error states. These serve as both specification and implicit test cases.

Context about the broader system. How this piece fits into what exists. AI assistants lack awareness of surrounding code and architectural decisions unless you tell them.

The SpecOps Context

When modernizing legacy government systems, specs serve a different purpose than typical development documentation. They’re not just implementation guides—they are artifacts that preserve institutional knowledge. This changes what “good” looks like.

A SpecOps specification document must work for multiple audiences simultaneously: domain experts who verify that the spec captures policy intent, software developers and AI coding agents who need precision to generate correct implementations, and future humans who need to understand why the system behaves a certain way years from now—possibly after everyone currently involved has moved on.

That last audience is the one most spec formats neglect entirely.

Three States, Not One

Legacy system specs can’t just describe “what the system does.” They need to distinguish between:

1. Current system behavior—what the legacy code actually does today, bugs and all
2. Current policy requirements—what the system should do according to governing statutes and regulations
3. Technical constraints—what the system cannot do regardless of policy, due to missing integrations or platform limitations

These three things can be in alignment or tension at any moment. And that alignment can shift over time without the code changing—a policy update tomorrow can transform compliant behavior into a violation.

Known Deviation Patterns

Consider the example of a benefits system that should verify income against a state tax agency records, but the legacy system only captures self-reported income because the integration with the tax agency was never built. A good spec would make this explicit:

Policy requirement: Per [directive], applicant income must be verified against tax agency records prior to benefit approval.

Current implementation: Self-reported income only. Applicant provides income information on Form X.

Deviation reason: No interface to tax agency income verification service exists. Integration requested in 2019, not funded.

Modernization note: Modern implementation should include tax agency income verification integration.

This surfaces the gap, documents why it exists, and gives the modernization effort clear direction—without pretending the legacy system does something it doesn’t.

Explicit Ambiguity as a Feature

There’s something that seems almost radical about a methodology that says write down what you don’t know. Traditional documentation can project false confidence. It often describes how things should work and quietly omits the messy parts.

A spec that explicitly marks areas of tension or uncertainty is more honest, more useful for risk assessment, and a better starting point for modernization. It’s an invitation for future clarification rather than a false endpoint.

A spec with unresolved tension is better than no reviewable documentation at all. 

Policy Grounding

Government system specs need explicit links to authorizing statutes, regulations, or directives. Not just “these items are excluded from income calculations” but “per 42 USC § 1382a, the following items are excluded from income calculations”

This is the why that survives personnel turnover. It’s what allows future teams to evaluate whether behavior that was correct five years ago still aligns with current policy.

Decision Records

When domain experts verify a spec, they make judgment calls—especially where legacy behavior diverges from current policy understanding. Those decisions need to be captured in the spec, not in a separate document that gets lost.

The spec becomes the repository of institutional reasoning, not just institutional behavior.

Accessible or Precise?

The SpecOps approach says that specs should be “readable by domain experts while detailed enough to guide implementation.” This is genuinely hard.

Options include stratified specs (plain-language summaries with expandable technical detail), executable specs (written as tests that are simultaneously human-readable and machine-verifiable), or annotated specs (a single verbose document where technical precision is explained inline).

Given that the spec is meant to be the source of truth that outlasts implementations, keeping everything in one artifact—even at the cost of verbosity—reduces the risk of layers drifting apart over time.

The Road Ahead

We’re still in early days. Questions remain open:

• How granular should policy references be?
• What’s the right way to represent known deviations?
• How should specs age—versioning, or is git history enough?
• What level of detail helps AI agents versus adding noise?

These will get answered empirically as more agencies adopt the approach. The methodology will evolve. The important thing is to start—to surface questions that were previously invisible, to give future teams something to interrogate rather than nothing at all.

Because the knowledge is what matters. Everything else is implementation details.

#AI #artificialIntelligence #ChatGPT #governmentServices #legacySystems #systemModernization

Three Projects I’m Proud of in 2025

As the year winds down, I wanted to share three projects that I’m particularly proud of working on in 2025, each focusing on how AI is changing the way we build and deliver government digital services.

What’s kind of surprising to me is that all of this work happened in the second half of the year. That’s less a testament to my productivity and more a reflection of how quickly this space is changing. The tools and capabilities that made these projects feasible for me to do simply weren’t there six months earlier.

Designing for Delegation

This year I spent a lot of time thinking about what happens when AI agents can act on people’s behalf. Instead of citizens navigating complex government forms and processes themselves, what if they could delegate those tasks to an AI agent that handles the complexity for them?

I wrote about this idea in the post Designing for Delegation and built delegation.design as a resource for the design patterns government teams will need as this shift happens. The key takeaway for me is that we need to stop thinking only about designing traditional interfaces and start designing delegation relationships. We need to start considering how people establish boundaries, how they stay informed about what’s happening, and how they maintain control.

This is sort of related to another post I wrote with my colleague Dan Munz on the end of the Civic Tech’s interface era. People’s expectations for how they can interact with their government are changing. So we must change the way we design and build government digital services.

ATLAS ATO Accelerator

Getting software through the federal Authority to Operate process is painful. I started exploring how well-structured Terraform code could serve as compliance documentation. Focusing not just on infrastructure-as-code, but on compliance-as-code.

The ATLAS ATO Accelerator project takes this further by creating AI-readable instructions for FedRAMP compliance. The idea is that AI coding agents can use these instructions to generate infrastructure that’s compliant by default, dramatically reducing the time and effort agencies spend on security documentation.

SpecOps

Legacy system modernization is hard, and most AI-assisted approaches focus narrowly on translating old code into new code. But when AI translates code it doesn’t understand, there’s a high probability that errors will get baked into the new system.

The SpecOps approach flips this traditional understanding of legacy system modernization around: the software specification – not the software code – is the source of truth. Legacy modernizations should begin by extracting knowledge and documenting how a system is supposed to work. Then, that specification can be used to guide modernization via spec-driven development. The spec generated through SpecOps outlasts any particular technology stack, so when today’s modern code becomes tomorrow’s legacy system, agencies still have the authoritative system documentation.

All three projects share a common thread: using AI to make government technology work better, whether that’s improving citizen experiences, accelerating compliance, or modernizing legacy systems.

I’m excited to keep building on these ideas in 2026.

#AI #artificialIntelligence #ChatGPT #government #llm #technology

What Does a Good Spec File Look Like?

Most legacy government systems exist in a state of profound documentation poverty. The knowledge lives in the heads of retiring employees, in COBOL comments from 1987, in binders that may or may not reflect current behavior. Against this baseline, the question of what makes a “good” spec file takes on different dimensions than it might in greenfield development.

Common Elements

Any spec worth writing answers the fundamental question: what are we building and why? Beyond that, good specs share a few specific characteristics:

Clear success criteria. Not just features, but how you’ll know the thing works. This matters especially when AI agents are generating implementations—they need something concrete to validate against.

Constraints and boundaries. What’s out of scope. What technologies or patterns to use or avoid. Performance requirements. AI tools are prone to scope creep and assumption-making without explicit boundaries.

Examples of expected behavior. Concrete inputs and outputs, edge cases, error states. These serve as both specification and implicit test cases.

Context about the broader system. How this piece fits into what exists. AI assistants lack awareness of surrounding code and architectural decisions unless you tell them.

The SpecOps Context

When modernizing legacy government systems, specs serve a different purpose than typical development documentation. They’re not just implementation guides—they are artifacts that preserve institutional knowledge. This changes what “good” looks like.

A SpecOps specification document must work for multiple audiences simultaneously: domain experts who verify that the spec captures policy intent, software developers and AI coding agents who need precision to generate correct implementations, and future humans who need to understand why the system behaves a certain way years from now—possibly after everyone currently involved has moved on.

That last audience is the one most spec formats neglect entirely.

Three States, Not One

Legacy system specs can’t just describe “what the system does.” They need to distinguish between:

1. Current system behavior—what the legacy code actually does today, bugs and all
2. Current policy requirements—what the system should do according to governing statutes and regulations
3. Technical constraints—what the system cannot do regardless of policy, due to missing integrations or platform limitations

These three things can be in alignment or tension at any moment. And that alignment can shift over time without the code changing—a policy update tomorrow can transform compliant behavior into a violation.

Known Deviation Patterns

Consider the example of a benefits system that should verify income against a state tax agency records, but the legacy system only captures self-reported income because the integration with the tax agency was never built. A good spec would make this explicit:

Policy requirement: Per [directive], applicant income must be verified against tax agency records prior to benefit approval.

Current implementation: Self-reported income only. Applicant provides income information on Form X.

Deviation reason: No interface to tax agency income verification service exists. Integration requested in 2019, not funded.

Modernization note: Modern implementation should include tax agency income verification integration.

This surfaces the gap, documents why it exists, and gives the modernization effort clear direction—without pretending the legacy system does something it doesn’t.

Explicit Ambiguity as a Feature

There’s something that seems almost radical about a methodology that says write down what you don’t know. Traditional documentation can project false confidence. It often describes how things should work and quietly omits the messy parts.

A spec that explicitly marks areas of tension or uncertainty is more honest, more useful for risk assessment, and a better starting point for modernization. It’s an invitation for future clarification rather than a false endpoint.

A spec with unresolved tension is better than no reviewable documentation at all. 

Policy Grounding

Government system specs need explicit links to authorizing statutes, regulations, or directives. Not just “these items are excluded from income calculations” but “per 42 USC § 1382a, the following items are excluded from income calculations”

This is the why that survives personnel turnover. It’s what allows future teams to evaluate whether behavior that was correct five years ago still aligns with current policy.

Decision Records

When domain experts verify a spec, they make judgment calls—especially where legacy behavior diverges from current policy understanding. Those decisions need to be captured in the spec, not in a separate document that gets lost.

The spec becomes the repository of institutional reasoning, not just institutional behavior.

Accessible or Precise?

The SpecOps approach says that specs should be “readable by domain experts while detailed enough to guide implementation.” This is genuinely hard.

Options include stratified specs (plain-language summaries with expandable technical detail), executable specs (written as tests that are simultaneously human-readable and machine-verifiable), or annotated specs (a single verbose document where technical precision is explained inline).

Given that the spec is meant to be the source of truth that outlasts implementations, keeping everything in one artifact—even at the cost of verbosity—reduces the risk of layers drifting apart over time.

The Road Ahead

We’re still in early days. Questions remain open:

• How granular should policy references be?
• What’s the right way to represent known deviations?
• How should specs age—versioning, or is git history enough?
• What level of detail helps AI agents versus adding noise?

These will get answered empirically as more agencies adopt the approach. The methodology will evolve. The important thing is to start—to surface questions that were previously invisible, to give future teams something to interrogate rather than nothing at all.

Because the knowledge is what matters. Everything else is implementation details.

#AI #artificialIntelligence #ChatGPT #governmentServices #legacySystems #systemModernization

The Future is Ahead of Schedule

MCP Apps and the Acceleration of Just-in-Time Interfaces

In August, Dan Munz and I wrote about the end of civic tech’s interface era, arguing that the rise of AI-generated, just-in-time interfaces would fundamentally change how civic technologists think about designing government services. We acknowledged that these ideas were still mostly theoretical—”this is still an idea that lies in the future,” we wrote. “But the future is getting here very quickly.”

That’s starting to look like a pretty significant understatement.

Just three months later, the organiztions behind the Model Context Protocol (MCP) — the open standard that connects AI assistants to data sources and tools — have announced MCP Apps, a formal extension for delivering interactive user interfaces through the MCP protocol. What we described in our earlier post as an emerging concept is now being standardized by Anthropic, OpenAI, and the MCP community. The timeline from theoretical possibility to a formal specification to guide production implementations wasn’t years or even months. It was weeks.

And we’d better get usef to it – this is what change looks like in the AI era.

From Concept to Standard in Record Time

When we initialy wrote about just-in-time interfaces, we pointed to early experiments and proof-of-concepts: Shopify’s internal prototyping with generative AI, Google’s Stitch and Opal projects, AWS’s explorations with PartyRock. These seemed like interesting signals, but they were scattered efforts using different approaches and solving similar problems in ways that were not obviously compatible.

MCP Apps changes seems poised to change that. It provides a standardized way for AI tools to deliver interactive interfaces — not as a speculative idea, but as a specification that developers can start to implement today. The extension enables AI-powered tools that can present rich, interactive interfaces while maintaining the security, auditability, and consistency that production systems will require.

The design is deliberately lean, starting with HTML-based interfaces delivered through sandboxed iframes. But the implications reach further. As the team beghins this effort notes, this is starting to look like “an agentic app runtime: a foundation for novel interactions between AI models, users, and applications.”

This matters for government digital services because it validates the core thesis of our earlier post: the constraints that forced civic designers to build one interface for everyone is eroding faster than most people anticipated. Certainly faster than we did.

The Infrastructure and the Ingredients

MCP Apps provides the delivery mechanism — a standardized way to serve interactive interfaces through AI systems. The specification itself is deliberately lean, focusing on core infrastructure: HTML templates delivered through sandboxed iframes, JSON-RPC protocols for communication, and multiple layers of security (iframe sandboxing, predeclared templates, auditable messaging, and user consent requirements).

What MCP Apps doesn’t specify is what makes those interfaces good or appropriate for government use. That’s where the foundational work civic technologists have already done becomes critical.

When we wrote about just-in-time interfaces in August, we noted that Shopify’s generative UI prototyping works in part because their design system is built on tokens—named variables that store key aspects of design systems like colors, spacing, and typography. We noted that “tokens aren’t sufficient to make just-in-time UIs a reality, but they probably are foundational.”

MCP Apps now provides the plumbing. But the quality of AI-generated government interfaces will still depend on having the right ingredients: well-structured design systems, clear interaction principles, and encoded policy logic. The U.S. Web Design System, VA.gov Design System, National Cancer Institute Design System, and other design systems used in government use tokens. That existing infrastructure positions government agencies to potentially benefit from MCP Apps when the time comes to experiment with dynamic interfaces — not because MCP Apps requires tokens, but because tokenized design systems can give AI something coherent to work with when generating interfaces.

The architectural decisions in MCP Apps demonstrate another principle that veteran civic technologists will recognize: building on proven patterns rather than inventing everything from scratch. Using MCP’s existing JSON-RPC protocol means developers can use familiar tools. Prioritizing security from the start means it won’t need to be retrofitted later. These are the kinds of decisions that distinguish serious infrastructure from interesting experiments—and exactly the kinds of decisions that government technology teams need to see before they’ll trust a new approach for delivering citizen-facing services.

What This Means for Civic Designers

The rapid standardization of interactive interfaces in AI systems has immediate implications for how civic designers should think about their work.

First, it underscores that the shift from fixed, multitenant interfaces to adaptive, context-specific experiences isn’t just theoretically possible — it’s actively being built. The expertise that civic designers have developed around creating design systems, documenting interaction patterns, and encoding policy logic won’t becoming obsolete. it will becoming more valuable because it provides the neccesary ingredients that AI systems will use to generate appropriate interfaces.

Second, it underscores the importance of getting the upstream architecture right. As we wrote in August, expertise in civic tech will move upstream — from implementation to architecture, from specific solutions to systemic standards. MCP Apps makes this more concrete. The work of defining interaction principles, building component libraries, and establishing visual identity standards becomes foundational to building great experiences, not nice-to-haves.

Third, it highlights the compressed timeline that government agencies are now facing. In previous waves of technological change, governments had years to observe how the private sector adopted new approaches before deciding whether (and how) to follow suit. The telephone era unfolded over decades. The Internet era compressed change to years. The AI era is compressing change to months. MCP Apps emerged from theoretical concept to production standard in less time than it typically takes a government agency to complete a procurement cycle for new software.

This mismatch between the pace of technological change and the pace of government adoption isn’t new – but the gap is widening at an accelerating rate.

The Infrastructure We Need Now

If just-in-time interfaces are moving from concept to production this quickly, what should government digital services teams be doing now to prepare?

The answer isn’t to rush into production deployments of AI-generated interfaces. The better approach is to strengthen the foundations that make such deployments viable when the time is right.

That means investing in design systems that use tokens and are built with the assumption that they’ll need to support dynamic interface generation. It means continuing the hard work of encoding policy logic in formats that AI systems can understand—efforts like the Digital Benefits Network’s Rules as Code community of practice aren’t just preparing for a possible future, they’re building essential infrastructure for a future that’s arriving ahead of schedule.

It also means rethinking how government agencies approach risk and experimentation. The traditional model of waiting until a technology is fully mature before considering adoption doesn’t work when the maturity cycle has compressed from years to months. Agencies need to develop the capacity to experiment safely and learn quickly—running controlled pilots, establishing clear evaluation criteria, and building the organizational muscle to rapidly deploy what works while quickly abandoning what doesn’t.

Acceleration Requires New Muscles

Perhaps the most important takeaway from the rapid emergence of MCP Apps isn’t about the technology itself. It’s about the pace of change in the AI era and what that means for how government organizations operate.

Three months ago, we described just-in-time interfaces as lying in the future. Today, there’s a formal specification proposal for delivering them. The team behind the MCP protocol has built an early access SDK to demonstrate the patterns, and projects like MCP-UI are already implementing support. The cycle of innovation, standardization, and adoption that once took years now happens in weeks and months — even if we’re still in the early stages of this particular evolution.

This creates genuine challenges for government organizations whose processes and decision-making structures were designed for a different era. But it also creates opportunities. Agencies that have invested in the right foundations — strong design systems, encoded policy logic, clear interaction principles — are positioned to benefit from these rapid advances. Those that haven’t will find themselves further behind with each passing month.

The future we wrote about in August isn’t coming. It’s here, and it arrived faster than even we expected. Government digital services will need to adapt to just-in-time interfaces.

The challenge for those of us working in and with governments is whether these organizations can develop the capacity to adapt at the speed that technological change now demands. Because if three months taught us anything, it’s that the next three months will bring changes we haven’t yet imagined.

#artificialIntelligence #governmentServices #justInTimeInterfaces #modelContextProtocol #userExperience

Infrastructure as Code for AI: The Rapid Evolution of Agent Instructions

Over the past year, we’ve seen the world of AI coding agents evolve at a pace that seems crazy fast. A process that started as simple markdown files to guide AI assistants has rapidly matured into sophisticated orchestration frameworks. It’s a transformation that took infrastructure automation roughly a decade – from shell scripts to Terraform – but for AI agent instructions, it’s happened in months.

From notes to infrastructure

When AI coding agents first emerged, developers quickly realized they needed a way to communicate project-specific standards and conventions. The solution was elegantly simple: markdown files in your repository that any AI tool could read. An AGENTS.md file became the standard way to explain your coding conventions, project structure, and workflow preferences to an AI coding agent – essentially documentation that both humans and machines could understand.

But as with most things in technology, simplicity gave way to more complexity 😅. We soon saw tool-specific variants of this approach emerge – CLAUDE.md, GEMINI.md, .github/copilot-instructions.md – each serving the same basic purpose but tailored to specific platforms. The principle remained consistent: give AI coding agents written instructions about how your project works.

A two-tiered strategy

What’s particularly interesting is how a pattern emerged for managing these instructions at different levels of specificity. We can think of it as two distinct layers:

High-level project instructions provide the constitution of your codebase – broad guidance that’s always relevant. The overall architecture, general coding standards and conventions, and common workflows live here.

Specialized, task-specific instructions offer detailed, narrowly-focused guidance pulled into context only when needed. These might be domain-specific instruction files for generating compliance documentation, or granular files for formatting particular types of components.

The key insight about this approach is that it’s about managing the finite context windows of AI coding agents. Developers keep essential, high-level guidance always available while bringing in detailed instructions just-in-time, only when the agent needs them for a specific task.

Claude Skills and custom Copilot Spaces are examples of this narrowly-focused approach, though they differ in implementation. Skills are platform-integrated and can activate automatically when relevant. Custom Copilot instructions live in your repository, version-controlled alongside your code. But they follow the same pattern: specialized knowledge, activated when needed.

Rise of the Meta-Frameworks

Here’s where things get really interesting. New projects like Superpowers and Amplifier represent a new category – meta-frameworks for instruction management. In simplest terms, they are orchestration platforms for AI coding instructions.

These tools solve second-order problems. The first-order problem was: “How do I give AI the right instructions?” The answer: AGENTS.md, Skills, custom instructions. The second-order problem is: “Now I have many instruction sources – how do I manage, combine, and optimize them?” The answer: Superpowers, Amplifier, and probably a bunch of other projects under development right now.

These frameworks aggregate multiple instruction sources, prioritize what matters for current context, compose instructions intelligently to stay within context limits, and provide version control and management for instruction sets across teams and across organizations.

They are, in essence, infrastructure as code for AI agent instructions.

The speed of change

What stands out most to me about this evolution is how fast it’s happening. The transition from writing shell scripts to using Terraform to manage cloud infrastructure took years – arguably a decade or more before infrastructure as code became standard practice in many organizations. But for AI agent instructions, we’ve gone from basic markdown files to sophisticated orchestration frameworks in a matter of just months.

We’re watching an entire infrastructure layer emerge in real-time. The move from manual instruction crafting to instruction management systems. From static files to dynamic, context-aware composition. From individual projects to organization-wide instruction governance.

Implications for government software development

For those of us who work on government technology projects, this rapid evolution presents both opportunity and challenge. Government agencies are just beginning to experiment with AI coding tools, and many are still getting their arms around how to safely and effectively incorporate them into their development practices.

The good news is that the industry appears to be converging on patterns that could work well within government constraints. Version-controlled instruction files fit naturally with existing government practices around documentation and change management. The ability to create organization-wide instruction standards aligns well with the need for standards across agencies and programs.

But the speed of change is the challenge. Government technology adoption typically moves deliberately, with good reason – there are legitimate concerns about security, privacy, and the need for deeper scrutiny of new tools. The challenge is figuring out how to move quickly enough to benefit from these new capabilities while maintaining the appropriate level of oversight and management.

A new vector for government collaboration

For many years, advocates have pushed for government agencies to share open source software code – and many agencies and states do publish their code under open source licenses. It makes sense on paper – different agencies and states have common needs and functions, so sharing code should be cheaper and easier than each jurisdiction buying or building custom software independently.

Despite some notable successes, widespread code sharing between governments has proven much harder than many people – including myself – initially thought. The challenges are real: different technology stacks, varying regulatory requirements, distinct operational contexts, and the simple reality that code built for one jurisdiction’s specific needs rarely transfers easily or cleanly to another’s environment.

But AI agent instructions could provide a different path to the same destination.

I’m thinking about the Atlas ATO Accelerator project I started recently – a specialized instruction set designed to help AI agents generate infrastructure-as-code documents that meet NIST compliance requirements. This isn’t meant to be general-purpose guidance. It’s narrowly focused on a problem that’s nearly universal across government: producing compliant technical documentation for security authorization processes.

Here’s what makes this interesting: an instruction set is fundamentally more portable than code. A well-crafted AGENTS.md file that helps an AI generate compliant security documentation can work just as effectively on different technology stacks, different cloud providers, and different jurisdictional requirements. The agent instructions encode knowledge and patterns, not specific implementation details.

This opens up new possibilities for government collaboration. States could develop and share instruction sets for common functions – eligibility determination logic for public benefits, tax calculation patterns, procurement document generation. Federal agencies could create specialized instructions for working with government-specific frameworks and compliance requirements. Cities could share instruction sets for common civic functions like permit processing or 311 service requests.

The barriers to sharing are lower. A different government agency or office doesn’t need to adopt an entire codebase, with all its dependencies and technical debt. This approach pivots from sharing code to sharing knowledge about how to approach a problem, encoded in a way that helps AI agents generate appropriate solutions for different contexts.

It’s a fundamentally different model. Instead of “here’s working code you can deploy,” it’s “here’s a way to think about this problem, packaged in a way that helps AI generate code that works for your environment.”

What governments can learn from this evolution

There are a few lessons I think government technology leaders should draw from watching this space evolve so rapidly:

Start experimenting now. The patterns are still emerging, but waiting for them to fully mature means falling further behind. Set up safe environments where teams can experiment with AI coding agents and different instruction approaches.

Invest in government-specific instruction sets. Unlike open source code, which has proven difficult to share across jurisdictions, instruction sets for AI agents may be inherently more portable. Government agencies should start to think about developing specialized instructions for common government functions – from compliance documentation to eligibility determination – that can be shared and adapted across agencies and jurisdictions.

Think about oversight early. The rapid emergence of meta-frameworks like Amplifier suggests that instruction governance will become critical as adoption scales. Government agencies should begin thinking now about how instruction sets will be managed, reviewed, and controlled across programs and teams.

Recognize the infrastructure layer. AI agent instructions are more than just helpful documentation – they’re becoming a critical piece of development infrastructure. They deserve the same attention and investment as other infrastructure components like CI/CD pipelines or deployment platforms.

Consider the context window as a constraint. Just as government developers learned to work within memory, processing, and bandwidth constraints, they’ll need to learn to work within AI context window constraints. The two-tier instruction strategy offers a proven pattern for doing this effectively.

Prepare for the next evolution. If the past year has taught us anything, it’s that this space will continue to change very quickly. Whatever instruction management approaches agencies adopt should be flexible enough to evolve as the ecosystem continues to mature.

Looking ahead

Standing back from the details, what we’re really watching is the emergence of a new layer in the software development stack. Just as we moved from manually configuring servers to infrastructure as code, we’re now moving from manually instructing AI to structured, governed, version-controlled instruction management systems.

As AI becomes more integral to how we build software, the instructions we give these tools become a form of infrastructure themselves – something that needs to be managed, versioned, and reviewed with the same scrutiny we apply to other foundational infrastructure components.

For government agencies, this evolution carries particular significance. Traditional open source code sharing has often fallen short of expectations due to the friction of different technical contexts and requirements. But instruction sets for AI agents may finally provide the more portable, adaptable approach to sharing knowledge and best practices across jurisdictions that advocates have long sought.

A new critical infrastructure layer is forming. Government needs to be part of shaping it.

#AI #artificialIntelligence #government #serviceDelivery

Maybe We Shouldn’t Call Them AI “Agents”

Beware of pretty faces that you find. A pretty face can hide an evil mind.
– Johnny Rivers, Secret Agent Man

As artificial intelligence capabilities expand into government service delivery, it’s worth pausing to think carefully about the language we’re using. The terms “agentic services” and “agentic AI” have gained significant traction in the tech industry, and for good reason — it captures something important about AI systems that can act autonomously. I myself am as guilty as anyone of using this term frequently. But for those of us working in government contexts, there are some considerations worth keeping in mind.

The “Agent” Problem in Government

In government, the word “agent” carries particular connotations. FBI agents. Border patrol agents. IRS agents. These are enforcement and investigative roles. When citizens hear “government agent,” they often think of authority, compliance, and oversight — not helpful service delivery.

This isn’t an insurmountable problem, but it’s worth being aware of. The language we choose shapes how citizens perceive and respond to new service models. If we’re trying to build trust in AI-enabled services, starting with terminology that might trigger concerns about surveillance or enforcement may not be ideal.

(And yes, for a certain generation, The Matrix movies didn’t exactly help the cultural perception of “agents” either. 😅)

What the term “agents” might obscure

There’s a deeper consideration beyond just the word “agent” itself. Calling these services “agentic” can make them sound radically new — a complete departure enabled by cutting-edge AI. But that framing might obscure an important reality.

Delegation-based government services aren’t new. They’ve existed for decades, and are extremely common today.

Tax preparers handle filing returns on behalf of clients. Immigration attorneys navigate visa applications. Customs brokers manage import/export documentation for businesses. Permit expediters guide building approval processes. Benefits navigators help people apply for disability or veterans services.

These are all delegation relationships. Citizens hand over complex, high-stakes government interactions to trusted specialists who handle the administrative burden on their behalf. AI doesn’t enable this service delivery paradigm, but it does potentially make it more scalable and affordable.

Why Words Matter

Thinking about these services as “delegation-based” rather than simply “agentic” opens up useful design questions.

When you frame it as delegation, you can look to existing delegation relationships for guidance. What makes someone comfortable delegating their tax filing to a CPA? What trust factors matter when hiring an immigration attorney? These aren’t abstract questions — there are decades of real-world answers.

The language of delegation also centers the citizen experience more clearly. It’s not about what the AI can do autonomously; it’s about what citizens are willing to hand over and under what conditions. That subtle shift in framing can lead to different design choices around transparency, control and oversight.

Moving Forward

This isn’t a call to abandon the term “agentic services” entirely. It’s widely used in industry, and there’s value in using common language when talking with technology partners and vendors.

But maybe for internal discussions, policy development, and especially citizen-facing communications, it might be worth experimenting with terms like “delegation-based services” or similar language. It acknowledges continuity with existing practices, avoids potentially problematic associations with “government agents,” and keeps the focus on what citizens are actually doing: choosing to delegate burdensome tasks while maintaining appropriate oversight and accountability.

The technology may be new, but the underlying service delivery paradigm isn’t. Our language should reflect that.

Note – this post originally appeared on GovLoop.

#agent #AI #artificialIntelligence #ChatGPT #serviceDelivery

AI Agents aren’t as radical as they sound

When we talk about AI agents handling government services, it can feel totally far-fetched and radically new, like something from a distant future. But delegation of government services is already happening all around us.

People routinely delegate high-stakes government interactions to:

• Tax preparers for filing returns
• Immigration attorneys for visa applications
• Permit expediters for building approvals
• Benefits navigators for disability claims
• Customs brokers for import/export documentation

To think about the potential for AI agents, I created this visualization to better understand why people delegate these tasks. Mapping services by administrative burden and frequency reveals a clear pattern: the upper-left quadrant of this chart (high burden, low frequency) is where delegation makes most sense. For these kinds of services, citizens don’t build up expertise because these tasks are relatively rare – this is where specialists who can be delegated to thrive.

The color coding of the chart highlights something important: many of these prime delegation candidates are critical or high stakes transactions. If they are not completed properly, there are potentially significant consequences. These are exactly the kinds of tasks people already pay professionals to handle.

This existing delegation landscape can inform how we design for AI agents.

What makes someone comfortable handing sensitive financial data to a tax preparer or personal information to an immigration attorney? These delegation-based services have the following characteristics:

• Professional credentials and accountability
• Transparent process (you see the forms before submission)
• Ability to override decisions
• Clear recourse when mistakes happen
• Legal liability

These things aren’t just nice-to-haves. They are the foundation of trust in delegation relationships. As we design AI agent systems for government services, we need equivalent trust mechanisms. These mechanisms won’t look identical to the real world delegation-based relationships we have today (i.e., no personal relationship with the professional offering support, different accountability structures), but they must address the same fundamental human needs for transparency, control, and recourse.

The future of government AI agents isn’t about inventing something entirely new. It’s about understanding what already works in delegation relationships and translating those trust factors into AI systems.

#AI #AIGovernance #artificialIntelligence #ChatGPT #civictech #DigitalGovernment #GovTech #machineLearning #ServiceDesign #technology

AI instructions as platform infrastructure

In previous posts on the Ad Hoc LLC website, I’ve talked about how platforms reduce complexity by providing reusable building blocks and how AI-assisted code generation is transforming custom development. I think that there is an emerging opportunity that sits at the intersection of these two trends that warrants some attention: treating AI coding agent instructions as a core component of platform infrastructure itself.

From documentation to executable knowledge

Platform teams invest significant effort in creating documentation that helps application developers understand and use platform services. This documentation typically includes architecture diagrams, API references, code samples, best practices, security guidelines, and deployment instructions.

While this documentation is essential, it still places a burden on developers to read, interpret, and correctly apply this information. Even with excellent documentation, developers must synthesize information across multiple documents, translate examples to their specific use case, and navigate the gap between understanding and implementation.

This cognitive load, while reduced compared to starting from scratch, can still create friction in the development process. Developers need to spend time deciphering documentation rather than solving their unique business problems.

AI instructions as infrastructure

What if platform teams could encode their knowledge directly into AI coding agent instructions that developers could use immediately? Instead of providing documentation that developers must interpret, platforms would provide machine-readable instructions that AI agents can execute directly.

This represents a fundamental shift in how platform knowledge is packaged and delivered. In this model, AI coding agent instructions become a first-class component of platform infrastructure, alongside the other building blocks I’ve discussed in the past. Just as platforms provide deployment pipelines and monitoring services, they would also provide curated AI instructions that embody institutional knowledge about how to build on the platform.

Platform-provided AI instructions would codify the collective knowledge of the platform team, including how to organize code within the platform’s architecture, required security controls and their implementation patterns, how to connect to platform APIs and shared services, coding standards and testing approaches, and agency-specific business domain knowledge.

The developer experience transformed

For application developers, this would transform the experience of starting a new project. Instead of reading through multiple documentation pages, finding and adapting code examples, and setting up basic scaffolding manually, developers would provide their AI coding assistant with the platform’s instruction set, describe their specific application needs, and focus immediately on unique business logic.

The AI agent, armed with platform-specific instructions, would generate code that already incorporates correct platform integration patterns, required security controls, proper configuration for deployment, and compliance with platform governance.

This approach dramatically reduces cognitive load on product teams in several ways. Developers don’t need to mentally process and translate documentation—the AI agent handles the interpretation and application of platform knowledge. Every project starts with the correct patterns and practices baked in. New team members can become productive more quickly, as the AI agent acts as an expert guide. And teams can spend their mental energy on the aspects of their application that make it unique, rather than on boilerplate integration with the platform.

Implementation considerations

For platform teams looking to adopt this approach, several factors are important.

• AI instructions should be versioned alongside platform releases, ensuring developers use guidance that matches their platform version.
• Platform teams should validate that instructions generate code that actually works correctly on the platform, treating instructions as code that requires testing. Instructions will improve over time based on feedback from developers and analysis of what the AI agents produce.

AI instructions can’t (and shouldn’t) replace documentation entirely. They would complement it by making knowledge executable while documentation remains important for deeper understanding. And in secure government environments, access to detailed platform instructions may need to be controlled, ensuring only authorized developers can use them.

The broader impact

This evolution of platforms has implications beyond individual developer productivity. New agencies or teams adopting a platform can become productive faster when AI instructions encode platform expertise. Institutional knowledge about platform best practices becomes more durable when encoded in AI instructions rather than living only in documentation or team members’ heads. Organizations can achieve greater consistency across applications when the same AI instructions guide initial implementation.

When AI-generated scaffolding already incorporates required security controls and compliance patterns, the path to ATO becomes even faster. And platform teams can support more application teams without proportionally increasing support burden, as AI instructions scale knowledge delivery.

A natural evolution

AI coding agent instructions represent a natural next step in platform evolution. Platforms have always been about encoding solutions to common problems so teams don’t reinvent the wheel. Documentation and code samples were the first generation of this encoded knowledge. AI instructions are the next generation—knowledge that isn’t just readable but directly executable.

For government agencies navigating the dual mandate to adopt commercial solutions and leverage AI capabilities, this approach offers a practical path forward. Platform foundations provide the commercial infrastructure, traditional documentation ensures understanding and oversight, and AI instructions dramatically accelerate the development of agency-specific applications on that foundation.

The result is a platform that doesn’t just provide services but actively helps developers use those services correctly and efficiently. It’s an approach that makes the right way not just the easiest way, but increasingly the automatic way—freeing government product teams to focus on what matters most: delivering value to the people who depend on government digital services.

#AI #artificialIntelligence #llm #technology

The Third Wave of Government Disruption

When printed telephone directories first started including blue pages for government offices in the 1950s and 60s, they created a new expectation: citizens should be able to reach their government by phone. The Internet revolution of the 1990s raised these expectations exponentially—if you could bank online and shop on Amazon, why couldn’t you renew your license or apply for benefits with the same ease?

Now, with 34% of U.S. adults having used ChatGPT—roughly double the share since 2023—we’re witnessing the third major wave of technology-driven transformation in how citizens expect to interact with their government. And once again, we’re watching the same pattern unfold: rapid consumer adoption creating new expectations, followed by delayed government adaptation, followed (potentially) by a long period of playing catch-up.

The difference this time? The stakes are higher, the pace is faster, and the consequences of falling behind may be more severe than ever.

Three Waves of Technological Disruption

Each wave of technological change outlined above has followed a similar trajectory, but with accelerating speed:

The telephone era unfolded over decades. Telephone adoption began in the late 1870s as an expensive luxury for the wealthy, with monthly costs of $20-40 (equivalent to $500-1,000 today). It took until the mid-20th century for phones to become commonplace in households. Governments had time to establish call centers and phone-based services without fundamentally redesigning how it operated. The pace was manageable—measured in decades, not in years or months.

The Internet era compressed this timeline to years. Internet users exploded from 45 million in 1996 to 407 million by 2000—a ninefold increase in just four years. Citizens who could accomplish complex tasks online in minutes naturally expected similar efficiency from their government. But while private companies were redesigning their entire business models around digital capabilities, governments largely treated the Internet as a new channel for existing processes.

The AI era is compressing change to months. Generative AI has been adopted at a faster pace than PCs or the internet, with breakthroughs moving from laboratory to widespread deployment in timeframes that would have seemed impossible just a few years ago.

The Structural Challenge: Democracy vs. Speed

As I’ve written about extensively before, governments aren’t slow at technology adoption by accident—they’re designed that way. The very features that are intended to make democratic government more trustworthy and accountable also make it structurally unsuited for rapid technological change.

The classic example is government procurement. The average technology buying cycle for government is 22 months compared to 6-7 months in the private sector. These delays aren’t the result of bureaucratic incompetence—they’re the deliberate result of requirements designed to help ensure fairness, transparency, and accountability. Public bid posting periods, vendor diversity requirements, the acquisition of performance bonds, and detailed financial scrutiny all represent important values imbued in public procurement processes. But they can also add months to timelines in a world where technology solutions can have shorter development cycles than government procurement processes.

The same pattern can be seen across government operations. Budget processes designed to prevent waste and enable legislative oversight create “use it or lose it” dynamics that discourage efficiency innovations. Civil service systems meant to prevent patronage and ensure merit-based hiring create lengthy processes that struggle to compete for scarce technical talent against private companies that can hire faster and pay more.

These aren’t bugs—they’re features. The transparency requirements, deliberative processes, and risk aversion that can slow down government technology adoption exist to uphold fundamental values and principles. The design of these processes is deliberate. The problem is that these principles are increasingly in tension with the pace of technological change.

The Compounding Crisis

This structural mismatch becomes more problematic with each technological wave because the pace of change keeps accelerating while government processes remain largely constant. What was a manageable gap during the telephone era became a significant lag during the Internet era and is becoming an existential challenge in the AI era.

The Internet wave provides a sobering lesson in the cost of delayed adaptation. Despite clear evidence throughout the 1990s that digital services were transforming how people expected to interact with institutions, most governments were slow to recognize that the rapid evolution of the Internet was changing people’s expectations for how they communicated and interacted with their government. Two decades later, we’re still playing catch-up, retrofitting digital services onto processes designed for paper-based workflows, and struggling to make basic websites and online services accessible.

The consequences aren’t just about inefficiency, they are about the loss of public trust. When citizens can accomplish complex tasks seamlessly with private companies but struggle with basic government services, the contrast erodes confidence in government competence and accountability.

The Stakes Are Higher

The AI wave presents an even greater challenge because it doesn’t just change how governments deliver services—it potentially changes how governments make decisions. Unlike previous technological waves that primarily affected operational efficiency, AI touches the core of democratic governance: the exercise of judgment and discretion in applying laws and policies to individual circumstances.

The stakes couldn’t be higher. As an example, when Spain implemented an algorithmic system to assess domestic violence risk, the software became “so woven into law enforcement that it is hard to know where its recommendations end and human decision-making begins.” Tragically, people assessed as low-risk still became victims of violence.

This use of AI in decision making processes highlights a troubling pattern identified by researchers like Virginia Eubanks in her analysis of Pennsylvania’s Allegheny Family Screening Tool. While AI systems are meant to “support, not supplant, human decision-making,” in practice “the algorithm seems to be training the intake workers.” Staff begin to defer to algorithmic judgments, believing the model is less fallible than human screeners.

The “human-in-the-loop” approach—where people supposedly maintain oversight of AI decisions—may not be sufficient protection against the human tendency to cede authority to software. When New York City’s AI chatbot tells businesses they can take workers’ tips and that landlords can discriminate based on source of income, both illegal, it demonstrates how AI systems can undermine the rule of law even in seemingly routine interactions.

The acceleration of AI adoption in government is happening precisely in contexts where lives hang in the balance—decisions about protection from violence, child welfare, emergency response, and access to vital resources. Unlike the more gradual telephone and Internet adoption cycles that gave governments some time—limited as it was—to learn and adapt, AI deployment can sometimes happen without proper safeguards, training, or accountability mechanisms in place.

Getting It Right This Time

The lesson from previous technological waves is clear: the cost of delayed or unorganized adaptation grows exponentially. Governments that fell behind during the Internet era spent decades and billions of dollars trying to catch up, often with mixed results. With AI moving even faster and touching more fundamental aspects of governance, the penalty for falling behind again could be severe.

But speed without safeguards is equally dangerous. The challenge isn’t choosing between moving fast and maintaining accountability—it’s developing the capacity to do both simultaneously. This means building safeguards into the adoption process from the start, not retrofitting them later. It means creating review mechanisms that can operate at the speed of technology development, not the traditional pace of government oversight.

The solution requires adapting democratic processes for technological speed without abandoning democratic values. This means creating “fast lanes” for certain types of technological adoption while maintaining rigorous oversight. It means developing rapid-response teams for AI evaluation that include technical experts, legal reviewers, and community representatives. It means investing in government workforce development so staff can properly assess and oversee AI systems rather than simply defer to them.

Most importantly, it means recognizing that the structural challenges governments face with technology adoption aren’t bugs in the system—they’re features designed to serve important functions. The transparency requirements, deliberative processes, and accountability mechanisms that slow government down exist for a reason. The question isn’t how to eliminate these constraints, but how to redesign them so they can operate effectively when technological change happens faster than traditional democratic processes were designed to accommodate.

As this historical pattern of technology adoption has advanced, governments have played catch-up before, each time with higher stakes and less time to adapt. Given the pace and implications of AI adoption in government services, we can’t afford to play catch up again.

#technology#AI #ChatGPT #artificialIntelligence #business #government #GenAI #Procurement

A depressing fable about how ChatGPT is corroding trust in scholarship

In preparation for next week’s keynote on generative AI and the crisis of trust, I picked up a book about trust by a philosopher, who I’ve decided not to name, when I saw it in the Tate bookshop earlier today. It began with a quote from bell hooks which caught my attention:

Trust is both a personal and a political endeavour, an affirmation of our shared humanity and our collective potential for growth and transformation. By embracing trust, by fostering connections, grounded in love and compassion, we have the power to not only change our own lives but also to reshape the world around us…

I wanted to post it on my blog, so I immediately looked for a citation. I could find no result for the exact quote but Google returned this site at the top of the list, where I found nearly the same quote:

In the end, trust is both a personal and a political endeavor, an affirmation of our shared humanity and our collective potential for growth and transformation. By embracing trust, by fostering connections grounded in love and compassion, we have the power to not only change our own lives but also to reshape the world around us, one relationship at a time.

The problem is that this site hosts imagined responses by philosophers to the question ‘what is trust?’ produced by ChatGPT. These (genuinely quite interesting) LLM outputs were posted in April 2023, only to feature in a book published in 2024. I can find no other source for the quote the author includes, other than this nearly exact quote produced by ChatGPT.

The most obvious explanation here is that they decided they want to start the book with a quote from bell hooks. They then typed in ‘bell hooks and trust’ which returns the site above as its second result. They didn’t read the introduction which explains the exercise with the LLM and instead copy & pasted the ChatGPT output into his book, without checking for the source of the citation.

The irony being that I now don’t trust the rest of the book. A philosopher writing a book about trust begins the book with such lazy scholarship that I now struggle to trust them. I hope I’m wrong. But without wishing to personalise things, I’m tempted to use this an example in next week’s keynote. It illustrate how LLMs are contributing to an environment in which lazy scholarship, cherry picking a quote from a google search, becomes much riskier given the circulation of synthetic content.

#AI #artificialIntelligence #ChatGPT #generativeAI #PascalGielen #scholarship #technology #trust #writing

A depressing fable about how ChatGPT is corroding trust in scholarship

In preparation for next week’s keynote on generative AI and the crisis of trust, I picked up a book about trust by a philosopher, who I’ve decided not to name, when I saw it in the Tate bookshop earlier today. It began with a quote from bell hooks which caught my attention:

Trust is both a personal and a political endeavour, an affirmation of our shared humanity and our collective potential for growth and transformation. By embracing trust, by fostering connections, grounded in love and compassion, we have the power to not only change our own lives but also to reshape the world around us…

I wanted to post it on my blog, so I immediately looked for a citation. I could find no result for the exact quote but Google returned this site at the top of the list, where I found nearly the same quote:

In the end, trust is both a personal and a political endeavor, an affirmation of our shared humanity and our collective potential for growth and transformation. By embracing trust, by fostering connections grounded in love and compassion, we have the power to not only change our own lives but also to reshape the world around us, one relationship at a time.

The problem is that this site hosts imagined responses by philosophers to the question ‘what is trust?’ produced by ChatGPT. These (genuinely quite interesting) LLM outputs were posted in April 2023, only to feature in a book published in 2024. I can find no other source for the quote the author includes, other than this nearly exact quote produced by ChatGPT.

The most obvious explanation here is that they decided they want to start the book with a quote from bell hooks. They then typed in ‘bell hooks and trust’ which returns the site above as its second result. They didn’t read the introduction which explains the exercise with the LLM and instead copy & pasted the ChatGPT output into his book, without checking for the source of the citation.

The irony being that I now don’t trust the rest of the book. A philosopher writing a book about trust begins the book with such lazy scholarship that I now struggle to trust them. I hope I’m wrong. But without wishing to personalise things, I’m tempted to use this an example in next week’s keynote. It illustrate how LLMs are contributing to an environment in which lazy scholarship, cherry picking a quote from a google search, becomes much riskier given the circulation of synthetic content.

#AI #artificialIntelligence #ChatGPT #generativeAI #PascalGielen #scholarship #technology #trust #writing