Somewhere in a law firm or legal department right now, a lawyer is about to act on an AI output they cannot verify, cannot trace, and could not reproduce if asked to. Two colleagues may have asked the system the same compliance question and received different answers — without either of them knowing.
This is not a hypothetical. It is the default state of most AI deployments in legal today.
This article explains why black-box AI creates unacceptable risk for legal, compliance, and risk teams, and what a governed alternative looks like in practice.
Black-box AI refers to AI systems — typically large language model (LLM) assistants like general-purpose chatbots — that generate outputs through probabilistic reasoning. These systems train on vast quantities of text and produce responses that are statistically likely given the input. They are not guaranteed to be consistent, traceable, or correct.
For many use cases, this is acceptable. For legal work, it is not.
Legal teams operate in an environment where the same set of facts must produce the same answer — regardless of who asks the question, when they ask it, or how they phrase it. A compliance workflow that routes a high-risk transaction to senior review cannot function if the routing logic shifts based on subtle variations in the query. An employment policy tool that advises employees on their rights cannot be deployed if its outputs are unpredictable.
The core issue is not that general-purpose AI is unintelligent. It is ungoverned.
Ungoverned AI introduces three specific failure modes into legal workflows.
Hallucination
Large language models can produce confident, well-structured, completely incorrect answers. In a legal context, a hallucinated case citation, a fabricated regulatory threshold, or an incorrect statement about jurisdiction-specific requirements can expose an organisation to significant liability. The error may not surface until damage is already done.
Inconsistency
LLMs generate responses probabilistically. The same question asked at different times, by different users, or with slightly different phrasing can yield materially different answers. This is incompatible with any workflow that requires uniformity — equal treatment in HR decisions, consistent application of contractual standards, or standardised risk scoring across a portfolio of matters.
Non-auditability
When a decision faces challenge — by a regulator, a counterparty, or an employee — a legal team must explain how they reached it. “The AI said so” is not an answer. Black-box systems cannot produce an audit trail that shows which rules applied, what logic the system followed, and what it knew at the time of the decision.
The pressure to adopt AI is genuine, and the tools available have improved rapidly. But the legal industry has reached a critical inflection point: AI adoption is accelerating faster than AI governance frameworks are developing.
Many legal teams have deployed general-purpose AI tools — or face executive pressure to do so — without answering fundamental questions: What happens when the AI is wrong? Who is accountable for an AI-generated decision that turns out to be incorrect? How do we demonstrate compliance to a regulator who asks us to show our work?
Regulators across multiple jurisdictions are moving from guidance to enforcement on AI explainability and auditability.
EU
The EU AI Act became fully applicable in August 2026 for most operators. Under Article 13, high-risk AI systems must be transparent enough to allow deployers to interpret and use outputs appropriately. Article 12 mandates automatic logging of events throughout the system’s lifecycle to ensure full traceability. Fines for non-compliance reach €35 million or 7% of global annual turnover.
UK
The UK’s cross-sector AI framework establishes transparency and explainability as core regulatory principles. Existing sector regulators — including the ICO, FCA, and MHRA — apply these principles now. The Data (Use and Access) Act 2025 introduced new algorithmic accountability obligations. A comprehensive AI Bill expected in 2026 would make these principles legally binding across sectors.
US
No single federal AI law exists, but enforcement is active. The FTC’s Operation AI Comply has brought multiple cases against companies making deceptive AI claims. The SEC’s AI task force focuses on how companies govern and disclose their AI systems. Colorado’s AI Act requires deployers of high-risk AI systems to document decision-making processes and conduct impact assessments, with effect from June 2026. New York’s RAISE Act, signed in December 2025, adds transparency and safety requirements for large AI model developers operating in the state.
Legal teams that have built workflows on black-box AI will face a significant remediation challenge as these requirements mature and enforcement intensifies.
Governed AI — sometimes called deterministic AI — works differently from general-purpose LLM assistants. Rather than generating answers probabilistically, it encodes the expertise of legal and compliance professionals into rule-based systems. Those systems apply the same logic consistently, every time, to every user.
A governed AI system for legal work has four key characteristics.
Predictable outputs
The same set of facts produces the same output, every time. A transaction that triggers a reporting obligation will always trigger that obligation — regardless of who submits it or when.
Traceable logic
Every decision traces back to the rules that produced it. The system shows exactly which logic applied, in what sequence, and on what basis — producing a complete audit trail.
Human expertise at the centre
The rules encoded in the system reflect the expert judgment of the lawyers and compliance professionals who built it. The AI does not replace that expertise. It scales it. One senior lawyer’s knowledge of a complex regulatory framework becomes available — consistently and accurately — to thousands of users across an organisation.
Generative AI as a tool, not an oracle
Governed systems can use generative AI for specific tasks where it adds value: extracting data from unstructured documents, drafting initial contract language, summarising lengthy materials. But those outputs feed into deterministic workflows, where rules govern what happens next. The AI assists. It does not decide.
Consider a large organisation managing business traveller compliance across multiple jurisdictions. Each trip may trigger different tax, immigration, and employment obligations depending on the traveller’s home country, destination, duration, and the nature of their activities.
A general-purpose AI assistant can describe the general framework for business traveller compliance. It cannot reliably apply that framework to a specific traveller’s specific itinerary and produce a governed, auditable recommendation.
A deterministic system can. Built on the expertise of employment and immigration lawyers who know exactly which rules apply in which circumstances, it asks the right questions, applies the right logic, routes edge cases to the right human reviewer, and produces a complete record of every decision.
The distinction that matters for legal teams: not whether AI is involved, but whether the AI is governed.
Is governed AI less capable than general-purpose AI?
No — it is differently capable. Governed AI does not attempt to answer every question about everything. It applies expert legal and compliance logic to the specific workflows it handles, and it does so with a level of consistency and auditability that general-purpose AI cannot match. For use cases where accuracy and accountability matter most, governed AI is the stronger tool.
How long does it take to build a governed AI workflow?
With a modern no-code automation platform, legal and compliance professionals can build governed workflows without engineering support. A prototype can go live in days. A production-grade solution typically takes weeks, not months. The expert knowledge that powers the system comes from the legal team, not from a vendor — so the system reflects the organisation’s actual standards and risk appetite.
What happens when the rules change?
Governed systems encode explicit logic rather than trained weights, so updating the rules is straightforward. When a regulation changes, the team updates the relevant rule and the system immediately applies the new logic. There is no need to retrain a model or wait for a new version.
Neota Logic builds governed legal AI services, so your organization can deploy AI with confidence.
