Two Clocks: Testing AI-Generated Software in Regulated Industries
I run a software testing company, so I have a direct interest in what AI does to testing work. The honest answer is that a large part of that work is going away, and I don’t think anyone in this business is served by pretending otherwise. But going away isn’t the whole story, and the part that stays is more interesting than the part that leaves. To explain why, I need to start a long way from testing AI-generated code.
The Two Clocks
Every new technology runs on two clocks. The first clock is the technology itself: what it can do and how fast people adopt it wherever nothing stands in their way. AI coding assistants run on this clock. A developer who accepts a function written by an AI answers to a code review and not much else, which is why AI coding went from novelty to default in about two years.
The second clock is institutional. It’s set by regulators, courts, standards bodies, insurers, professional licensing, procurement rules, and plain organizational habit. It moves when a regulation is rewritten, when a court rules, when a standard is revised, or when the people making the decisions retire and are replaced by people who grew up with the new tool.
The second clock is slow on purpose. It’s tempting to treat institutional lag as bureaucracy that will eventually catch up, but most of it is a deliberate encoding of the cost of being wrong. When a bad software release means a patient gets the wrong dose or a bank misstates its books, we’ve collectively decided that change should move slowly and that a person should be accountable for it.
That decision doesn’t get revisited every time a model gets better. The slow clock will still be running at its own pace when AI is twice as capable as it is today.
This Has Happened Before
The pattern isn’t new, and the history is worth a moment because it shows how long the gap between the two clocks can last.
Electric motors were commercially available in the 1880s. Factory productivity didn’t respond until the 1920s. The reason, documented in Paul David’s work on the dynamo, was that factories had been built around a central steam engine and a system of shafts and belts, and the benefit of electricity only showed up once plants were rebuilt around individual motors. That was forty years of lag with no regulator involved, just the time it takes for organizations to reorganize around a new capability.
In 2016, Geoffrey Hinton said we should stop training radiologists because deep learning would outperform them within five years. The models did get very good at reading images. Ten years later, radiologists are in short supply.
What changed was the shape of the job. The machine reads, and the radiologist signs, integrates the finding into the patient’s care, and carries the liability. The task moved to the machine and the institution kept the role.
Legal document review is the case closest to my own business. For decades, law firms billed junior associates and offshore review vendors by the hour to read documents in discovery. Technology-assisted review could do that work by 2008, and a federal court accepted it in 2012.
Over the following decade, the hourly review business collapsed. The firms that repositioned to managed review, with a lawyer accountable for the defensibility of the process and the privilege calls, did fine. The vendors selling only hours mostly didn’t.
And commercial aircraft have been able to land themselves since the 1960s. Every airliner still has two pilots in the cockpit. The pilot’s job became monitoring rather than flying, and sixty years later the institution hasn’t moved.
Two Clocks, Two Gaps
Two clocks running at different speeds produce two different gaps, and they behave very differently as businesses.
The first gap is between what AI can do today and what it will be able to do. Business in this gap means humans doing work the machine cannot yet do. It’s real revenue, but every model release closes some of it, and it closes fastest exactly where the work is well defined. I’d measure the shelf life of a business built here in months.
The second gap is between what AI can do and what institutions will accept as sufficient. Business in this gap means making a new capability acceptable: producing the evidence, defining the method, providing the independence and the accountable signature that a regulator or a customer requires. This gap closes when the institution moves, which is to say in steps, over years.
It took me a while to see the next part. The two gaps respond to AI progress in opposite directions. Faster AI shrinks the first gap. It widens the second, because the further capability runs ahead of what institutions will accept, the more work there is in getting the two to meet.
The same trend that is eliminating one kind of testing business is creating demand for another. The question for anyone in this industry is which clock their business is running on, and I think most of us have been answering that question wrong.
The two clocks collide, and the collisions are the steps in Figure 1. The fast clock produces change faster than the institution can review it, so review gets thinner, the evidence behind it gets weaker, and defects that would once have been caught accumulate until one of them surfaces in public.
When that happens, the institution doesn’t catch up gradually. It reacts. Enron produced Sarbanes-Oxley. The 737 MAX produced tighter oversight of the certification work the FAA had delegated.
A rule written after a failure is written in a hurry, adds human review rather than removing it, and applies to everyone, including the people who were careful. Occasionally the pressure runs the other way, and a shock forces the institution forward, as pandemic waivers did for telemedicine. In both directions, the speed of the fast clock decides when the slow clock’s next move arrives, and nobody gets to choose the timing.
Figure 1. AI capability advances on the fast clock; institutional acceptance moves in steps on the slow clock. The shaded region between them is the acceptance gap, and it widens as capability accelerates.
That framing is broad, and I will narrow it through the rest of this article: first to software, then to what my company has learned running on both, and finally to what that means for your team.
Where the Clocks Collide
Software is unusual in that both clocks now run inside a single release cycle. Writing code sits on the fast clock. Nothing outside the team gates the decision to accept what an AI produced, and in most shops that decision has already been made. Verifying the code, in any regulated environment, sits on the slow clock, because someone outside the team decides whether the evidence is good enough: a validation reviewer, an internal auditor, a regulator, a customer’s procurement office, or, in the case of accessibility, a court.
So, code arrives at the release gate at the pace of the fast clock, and the gate itself runs at the pace of the slow one. If you run engineering or QA in a bank or a medical device company, this is your daily reality, and it gets worse with every model release. Developers are producing far more code than they were a year ago. The amount of code in the product that no person fully understands is going up.
The questions asked at the gate haven’t changed in decades: what was this supposed to do, how do we know it does that, who checked, and can we show our work. What has changed is where the answers come from.
When people wrote code by hand, most of those answers were produced as a side effect of the writing. The developer knew what the function was for because someone had spent an hour thinking about it before typing. Generated code arrives without that history attached, and the answers the gate wants have to be produced deliberately, as their own body of work.
The same dynamic means the first serious failure of AI-generated code in a regulated product will produce exactly the kind of step described in the previous section, and the review it mandates will land on every team in the industry. The teams that come through it are the ones whose evidence already exists when the new rule arrives.
What We Learned Running on Both Clocks
We didn’t arrive at this view by theorizing about it. QualityLogic started where most testing organizations started, and I think the sequence is worth describing because each step taught us something about the two clocks that we couldn’t have learned any other way.
We began by using AI to automate the processes we already had: generating test cases from requirements, writing automation code, and maintaining regression suites. It worked, and the mechanics turned out to be the easy part. What it exposed was that every one of those processes had quietly depended on a person who already knew what correct looked like. Remove that person from the loop and the automation reproduced the ambiguity in the requirements at much higher speed.
In most of the environments we work in, the authoritative statement of what the software is supposed to do was never written down anywhere a machine could read it. The bottleneck moved from executing tests to knowing what to test against.
Next, we built processes designed around AI from the start rather than retrofitted. This is where we learned that verification can keep pace with generation, but only under two conditions. The statement of intended behavior must exist before the code is generated, not be inferred from it afterward. And the evidence must be captured as the work happens, because reconstructing it after the fact at this volume isn’t possible.
Where both conditions held, we ran verification at generation speed. Where they didn’t, we produced a great deal of fast, confident output that was wrong in ways that took an experienced person to notice.
The third lesson came from our regulated customers, and it’s the one that reorganized the company. None of the above matters if the reviewer, auditor, or regulator won’t accept the result. Acceptance turns on things the fast clock doesn’t measure: whether the verification was independent of the development, whether the method is defensible, whether the evidence is complete, and whether a qualified person is accountable for the conclusion.
A test suite that a model wrote to check code the same model wrote isn’t independent verification, however high the pass rate. Institutions in finance and aviation worked that out a long time ago, and nothing about better models changes it.
Ground Truth at the Speed of Generation
Those three lessons define what we do now.
The first part is establishing ground truth. In most organizations we walk into, the intended behavior of the system lives in people’s heads, in old tickets, and in the undocumented behavior of the legacy code everything else depends on. For forty years our work has been walking into exactly that situation and producing a testable definition of correct: the requirements, the acceptance criteria, and the cases that prove them.
We used to do it, so our own engineers had something to test against. We do it now because it has become the scarce input for AI-speed development. Generated code and generated tests can share the same wrong assumption, and the only defense is a reference point that came from somewhere else. The thing we’ve always done turned out to be the thing this new environment needs most.
The second part is velocity. With ground truth in place, verification can run at the speed of generation, and we build the process, so the evidence accumulates as the work happens: what was required, what was checked, what passed, what was found, and how it was resolved. That’s what the speed buys you.
The release reaches the gate with the slow clock’s questions already answered, and nobody had to stop the fast clock to answer them. Put plainly, we get your verification running at the pace of your code generation, with evidence your compliance function will accept.
Assurance in Regulated Industries
We’ve chosen to concentrate on finance and healthcare because that’s where the slow clock is hardest and the gap between the two clocks is widest. Not every institutional constraint is equally durable. Habit and preference give way under price pressure within a few years, standards and contracts move on their revision cycles, and law and liability move only through rulemaking or the courts, usually with years of notice. Finance and healthcare sit at the hard end of that range, which is why positions built there last.
Figure 2. Not all institutional constraints are equally durable. Regulated industries sit at the hard end of the range, where the slow clock moves only through rulemaking or the courts.
We’re not starting from zero here, and the clearest evidence is accessibility. Accessibility conformance is a legal obligation under the ADA, Section 508, and the European Accessibility Act. Automated scanners find a portion of the problems. A conformance claim that will hold up still requires testing with assistive technology by people who know how to do it, and a written attestation that someone stands behind.
That’s an assurance business running on a legally driven slow clock, and we’ve operated it for years. What we’re doing now is extending the same model to AI-assisted verification more broadly.
I want to be direct about what isn’t settled. What acceptable AI-assisted verification looks like under FDA design controls, or inside a bank’s model risk governance, hasn’t been defined, because the regulators and standards bodies haven’t defined it. Anyone who tells you they have the final answer is selling something.
History says the institution will shape the form as it accepts the technology, the way electronic records came with audit trail requirements and technology-assisted review came with sampling protocols. That process is underway for AI in software and it will take years. We intend to be part of it, working alongside our customers and the reviewers and regulators they answer to. The direction is clear even though the final form isn’t, and the four things that have survived every previous version of this transition (independence, method, evidence, and accountability) are the things we’re building to.
QualityLogic has been through client-server, the web, mobile, cloud, and agile, and each one changed what testing looked like. None of them removed the need for someone to say, with evidence, that the software was fit to release. This transition is faster than the others, and it’s changing more, but I don’t think that requirement goes away either. It moves, and we’re moving with it.
What This Means for Your Team
I said at the start that a large part of testing work is going away, and I meant our work as much as yours. If you lead engineering or QA in a bank, an insurer, or a medical device company, you’re already living with the consequences, and they’re not evenly distributed.
Your developers are on the fast clock. Your release gate is on the slow one. Your QA team is being asked to run on both at once, usually with a budget that assumes AI made the whole problem cheaper. It didn’t. It made one part of the problem cheaper and the other part larger.
Three questions will tell you which of your team’s work sits on which clock.
First, for each thing your team does, would a regulator, an auditor, or a plaintiff’s attorney care who did it? If the answer is no, that work is on the fast clock whatever it feels like from inside, and you should plan for it to be automated, by you or by someone else. If the answer is yes, the execution can still be automated, but the accountability can’t, and the people who carry it need to be the people who understand your product and your regulator.
Second, where does your definition of correct live? If it’s in the heads of a few senior people, in old tickets, and in the behavior of code nobody wants to touch, then generated tests will inherit whatever those sources got wrong, and you have a ground truth problem before you have a velocity problem.
Third, when a release reaches the gate, can you produce the evidence without stopping? If answering the gate’s questions means a week of reconstruction after the fact, your verification isn’t running at the speed of your development, and the gap will widen with every model release.
Most teams we talk to can answer the first question quickly, struggle with the second, and have never been asked the third. That’s the shape of the work ahead. The question is less how many people you need than which work moves to the machine, which work stays with people, and what those people need that they don’t have today.
We publish what we’re learning about this every month in the State of AI QA Newsletter, and if you want to follow how the two clocks play out across the industry, that’s the place to do it. If you would rather talk about your own team, we’ve spent two years sorting our own work onto these clocks, and we would be glad to compare notes on yours.
