I tested a traditional NLP chatbot, and a wrong answer was usually obvious. The model missed the intent, gave the wrong canned reply, or said it did not know when its confidence was too low. Quality assurance was still work, but it was work you could pin down.
That is no longer the main problem with large language models. An LLM can answer in slightly different words when it is right, and it can sound sure when it is inventing a fact. Guardrails, retrieval, and other context-engineering techniques reduce that risk. They do not give you back a test that simply checks whether the output matches a stored sentence.
That gap matters in regulated industry. The last person to say the answer is good still has to be a human.
What exact match could do in a bank chatbot
At Standard Chartered I worked on Stacy, the bank’s enterprise virtual assistant in Hong Kong. The system used traditional NLP models. At first I tested it one question at a time. If a customer asked how to apply for an ABC card, the expected answer was the application steps. Ask the same thing again, and you got the same wording.
That stability made automation possible. I ran the FAQ-style cases through exact match: seed questions as the input, the expected model answer as the output. For those intents, it cut the time spent clicking through the chatbot one prompt at a time.
It did not replace every check. Some answers still had to be tested by a person, because the risk was not a slightly different sentence. It was whether the system returned the right account balance, or the right ATM location. Those cases sit next to a backend, a customer, and a consequence. Exact match on a FAQ script cannot carry that load.
I am not describing an incident. I am describing the bar. In a bank, giving one customer another customer’s balance is not a wording issue. It is a failure you cannot afford to miss.
Why that method breaks on an LLM
A generative model does not retrieve a stored reply the way a traditional intent system often did. It produces language. Even when retrieval, guardrails, and a tight prompt keep it inside the right policy, the same correct answer can come back in a new order, with a new connector, or with a slightly different level of detail.
Exact match then fails in both directions. A good paraphrase looks like a miss, so the suite becomes noisy and people stop trusting it. A fluent but wrong sentence can still look close enough if you start relaxing the match, which is the last thing you want when the bar is 100% correctness.
That is why I do not treat RAG, guardrails, or context engineering as a replacement for QA. They are ways to reduce hallucination and restrict behaviour. They do not restore a deterministic string you can tick off in a spreadsheet.
I wrote earlier that AI adoption is still a translation problem — professionals need to know how an answer can be checked, and what happens if it is wrong. QA is that check. If the check cannot tell a paraphrase from an error, it is not doing the job.
LLM-as-a-judge is useful. It is not a sign-off.
The usual next idea is to let another model judge the output: does this answer match the expected meaning, even if the words moved?
I think that is a reasonable way to run a large suite. It is not a reasonable last control when the ask is 100% correctness.
The judge is also an LLM. It can hallucinate a pass. In a regulated process, a false pass is worse than a slow test, because it looks like evidence. If the question is a balance, a policy clause, a claims fact, or anything else where the wrong detail harms a customer, “the judge said it was fine” is not enough.
That is not an argument against using a judge at all. It is an argument against treating the judge as the person who signed.
The mix I would use
The pattern I would use in regulated work is mixed, and the human does not go back to typing every seed question by hand.
Let the LLM-as-a-judge run the full set of tests. For each case, show the actual output next to the expected result, with a screenshot and an audit-trail log so a reviewer can see what was asked, what came back, and what the judge decided. The human’s job is to go through those results, flag the ones that are wrong, and sign off only if they think the run is good.
That saves the time that used to sit in conducting the tests one by one. It does not save the time that sits in accountability. In a bank, an insurer, or a healthcare setting, that second part is the point.
I would not sample a handful of cases and call the rest done. If the bar is low error, the reviewer reads the pack. The judge is there so that pack exists. The signature is still human.
This is a recommendation from the NLP testing I actually ran, and from how LLMs fail. It is not a production LLM QA system I shipped at the bank.
What this is for
The chatbot story is from banking because that is where I did the work. The QA problem is not a banking specialty.
Anywhere an invented fact, a swapped identity, or a fluent misstatement is unacceptable, exact match will not hold the line on its own, and a second model will not hold it either. The useful split is the same: automate the running of the tests, keep a person on the results, and make the evidence easy enough to review that sign-off is real work rather than a ceremony.
That is also why I still care about designing the workflow, not only the model. I took an LLM engineering course so I could design a solution and a roadmap around a business need. QA is part of that design. If you cannot check the output in a way a regulated team can stand behind, you do not yet have a system. You have a demo that talks.
Where in your organisation would you still refuse to let an LLM-as-a-judge be the last check?
