Before You Paste Anything Into a Chatbot: AI Questions for Legislative Bodies

At the last few conferences I have spoken at, I have expressed a concern that we aren’t giving enough consideration to the effect of introducing AI technologies into legislatures with respect to confidentiality and business continuity.

That is why I published my Six Commitments for AI in Legislative Drafting a few weeks ago. We need to really think about this. The commitments are what I believe software should promise a drafting office. This post looks at the other side: the questions the office should ask itself, and its vendors, to know whether those promises are being kept. Along the way, I’ll point out which commitment each set of questions connects to.

AI is now in almost every tool a legislature uses, whether anyone chose it or not. Before we argue about whether AI can draft a good bill, there’s a more basic set of questions to ask: what happens to us if the service disappears, what AI is already inside the software we bought, where do our words go, and who can see them?

I’m not the only one asking. The Inter-Parliamentary Union has published Guidelines for AI in Parliaments, covering data governance, security and risk management. NCSL reported last year that AI use among legislative staff had doubled while the policies to guide it hadn’t kept pace, and it has been surveying legislatures on their use of AI ever since. The U.S. House now tells its staff not to put sensitive data into chatbots, still allows AI to help draft a bill, and leaves it to each office to write its own policy.

But in all of that, I haven’t found anyone who has pulled these concerns together into a single list of questions a legislature can actually work through. And two of them, what happens when the AI goes away and what AI is already inside the software you own, hardly come up at all. So that’s where I’ll start.

Today, I sat down and worked through those questions one at a time, the way I was taught to approach any systems problem: break the big problem into smaller ones and stick to the facts. A nation’s laws are its most important documents, and the drafts, instructions and deliberations behind them deserve the same care. What follows is a summary of what I found. I’m an engineer, not a lawyer, so where the law comes up I’ve framed things as questions to ponder rather than answers I have.

When the AI goes away

Legislatures don’t get to pause. Bills have deadlines, sessions end on fixed dates, and the work has to get done. So the continuity question matters a great deal: what happens when the AI you depend on isn’t there?

Ordinary outages and capacity limits are the obvious risk. Friday night’s FAA ground stop showed what happens when the technology fails. Earlier this year there was a less obvious incident. On June 12, 2026, a US government export-control directive led Anthropic to suspend access to two of its newest models with almost no notice. Access was restored on July 1. This was a compliance action rather than a technical outage.

There are other risks too:

  • Models retire. Anthropic’s API promises only 60 days’ notice. This means a model you rely on can stop answering.
  • Replacements behave differently. Whatever validation you did on the old model may not hold for the new one. I’m pretty sure that the AI vendors won’t take your legislative session’s work cycle into account when changing models.
  • Connectivity. Most AI needs the internet, which is exactly what a disaster or cyber incident tends to take away. Will enough of your system still function if the legislative body is cut off from the cloud infrastructure AI builds on?
  • Skills fade. If nobody does the work by hand anymore, the manual fallback quietly stops working.

The test I’d suggest for any drafting office is simple: if the AI were unavailable during a session, would the bills still go out? If the answer isn’t a confident yes, you need a manual path, ideally a second source, and the authoritative text, meaning the bills, amendments and decisions, kept in your own systems rather than inside an AI tool.

This is why two of my commitments are that the core works without AI (Commitment 2) and that there’s no dependence on a single vendor (Commitment 5). If the AI goes away, drafting, amending and publishing should keep working, and moving to another model should be a configuration change, not a crisis. Keeping the authoritative text in your own systems is also what my sixth commitment, authoritative data first, depends on.

The AI you didn’t choose

Most of the AI a legislature uses won’t arrive as a chatbot someone signed up for. It will arrive inside the software you already have.

The names are everywhere: Microsoft 365 Copilot, Gemini in Google Workspace, Apple Intelligence, Salesforce Agentforce, SAP Joule, ServiceNow Now Assist, Adobe’s AI Assistant, Atlassian Rovo, Slack AI and Notion AI. On the legal side there’s Lexis+ with Protégé, CoCounsel inside Westlaw, and Harvey. The names change often. Zoom, for example, retired its AI Companion brand this year.

Very few of these vendors build their own large models. Most call a model from one of a handful of providers. Microsoft 365 Copilot now uses both OpenAI and Anthropic models, and Apple is using Google’s Gemini for the new Siri. That means everything in this post about continuity, training, residency and privilege applies all the way down the chain, not just to the application you bought.

These changes can arrive quietly. When Anthropic became a subprocessor for Microsoft 365 Copilot in January 2026, it came through a routine terms update.

So how do you tell whether an application uses AI under the hood? In my experience, using the product won’t reliably tell you. The paperwork will:

  1. Read the subprocessor list. Look for OpenAI, Anthropic, Google, Azure OpenAI or AWS Bedrock. This is the best single check.
  2. Check the AI terms, privacy policy and release notes for phrases like “large language models” or “third-party AI providers.”
  3. Look in the admin console for AI settings.
  4. Watch for the visual cues: the sparkle icons which symbolize AI, “Copilot” or “Assist” labels, “Summarize” buttons, and even “may contain mistakes” disclaimers.
  5. Ask the vendor, and write it into the contract.

Regulations are starting to help. The EU AI Act’s transparency obligations took effect on August 2, 2026, so people in the EU must be told when they’re interacting with AI. But those rules generally don’t require naming which model is behind it.

My first commitment is that the office stays in charge. That’s hard to do when you don’t know the AI is there. Each of the checks above is really a way of finding out whether that commitment holds for the software you already have, and the same commitments belong in the contract with every vendor, not just the ones that sell AI by name.

It’s the contract, not the price tag

The most important dividing line isn’t between AI providers. It’s between consumer accounts and business accounts. I don’t think most people realize there is a difference.

Consumer accounts (the free and personal paid plans of Claude, ChatGPT and Gemini) generally run on consumer terms. Since late 2025, that has meant your conversations may be used to train future models unless you go into the settings and turn it off. With Claude, choosing to allow training also stretches retention from 30 days to as long as five years. Even with training turned off, there are exceptions, such as conversations you rate or ones flagged by safety systems. So, don’t answer the question when the AI asks you how it’s doing.

Business accounts (Claude Team and Enterprise, ChatGPT Business and Enterprise, Gemini for Workspace, and the APIs) run on commercial terms. Training on your data is off by default and the contract says so. Your organization, not each individual, controls retention and access.

What surprised me is that price you pay tells you nothing. A $200 a month personal plan is still a consumer account. So when a staffer pastes a draft into their personal chatbot, that draft may be governed by the weaker consumer terms, whatever they paid for it.

This is exactly why my third commitment is that confidentiality comes first: drafts stay private until they’re introduced, with zero data retention and no training on them without written consent. A consumer account can’t make that promise.

Where your information actually goes

When you type a prompt, it doesn’t stay on your computer. It travels to a data center, where the model runs on the AI provider’s servers. Unless someone has pinned a region, that can be wherever there’s capacity, which may be in another country.

This matters if you have data residency rules, and there’s a distinction I think many people don’t realize. Data residency is about where things are stored. Inference residency is about where the model actually reads your information and writes its answer. The two aren’t the same. A vendor can promise to store your data within your jurisdiction while still processing it somewhere else.

Even “in-region” processing has gaps. Routing, extracting text from uploaded files, metadata and logs may still leave the region. Tools like web search and the connectors, like MCP servers, that AI applications attach to fall under someone else’s terms entirely.

The options available today differ a lot by deployment path. Consumer apps generally offer no choice of region at all. Cloud platforms such as AWS Bedrock and Google Vertex AI offer regional processing for some models, including in the EU. So the same model can satisfy a residency requirement when it’s deployed one way and fail it when it’s deployed another. If your rules say data must be processed within your country, ask about inference, not just storage.

My fourth commitment, that processing stays within your boundaries, is meant to cover this. The office chooses where the AI runs, and if that can’t be guaranteed, the AI does nothing.

Privilege: questions for the lawyers

This is where I tread carefully, because I’m not a lawyer. But legislative counsel offices and their clients often rely on attorney-client privilege, so this is worth raising.

In February 2026, in United States v. Heppner, a federal judge in New York ruled that a defendant’s conversations with a consumer AI chatbot were protected by neither attorney-client privilege nor the work product doctrine. As I read the commentary, the reasoning was roughly that the AI isn’t a lawyer, the consumer terms didn’t promise confidentiality, and handing the results to your lawyer afterward doesn’t make them privileged. Some have warned that pasting already-privileged material into a public chatbot could put that privilege at risk.

Other questions seem unsettled. A different court reasoned that AI programs are “tools, not persons” when it came to work product. Business accounts with contractual confidentiality may be on more solid ground, though that hasn’t been tested much.

So the questions I would ask are these:

  • Can privileged material ever go into an AI tool? Under what account type, and on whose direction?
  • Does our business AI contract give us the confidentiality we would need to argue privilege survives?
  • What should staff do, and not do, with personal AI accounts?

Are your prompts public records?

There’s one more question I hadn’t considered until I read NCSL’s coverage of how legislative staff are using AI: are the prompts we type, and the answers we get back, public records? Many legislatures are covered by records or open-government laws, and some are exempt from them. Either way, a chat history may be the best record of how a bill came to be written, and it may be sitting on a vendor’s servers under someone else’s retention policy.

Again, I’m not a lawyer, so I’ll leave this as questions:

  • Are AI prompts and outputs records under our rules, and who decides?
  • If they are, where are they kept, and for how long? Inside the AI tool, or in our own systems?
  • How does that square with keeping drafts confidential until a bill is introduced?

The questions to ask, and who decides

None of this is a reason to avoid AI. I use it, and I think it can be genuinely useful in legislative work. But we must approach it the way we would approach any other critical system, with facts rather than enthusiasm or fear.

If I had to boil it down to a short list for a legislature or drafting office, it would be these questions:

  1. If the AI disappeared tomorrow, could we still do our work?
  2. Which of the products we already own have AI inside, and from whom?
  3. Are we using business accounts, under commercial terms, for anything sensitive? Do we know what staff do with personal accounts?
  4. Where is our data stored, AND where is it actually processed?
  5. What have the lawyers decided about privilege and AI?
  6. Are our prompts and AI outputs public records, and how will we keep them?

If those questions sound familiar, they should. They’re my six commitments seen from the legislature’s side of the table. The commitments say what the software should promise, and these questions are how you check.

The last point I’d make is about who answers these questions. It shouldn’t be each individual drafter deciding on their own which chatbot to paste a bill into. These are deliberate decisions for the organization as a whole, meaning the drafting office or even higher, to write down. As I said in my commitments, the drafting office decides. Always. Your laws are too important for anything less.

Comments

Leave a Reply