For years, game development has operated around a fairly predictable sequence: version a feature, stabilize it, hand it over to QA, find problems, fix them, and repeat.
At SuperCity, Playkot’s 15-year-old city-building game, AI is beginning to change that sequence. Instead of simply making existing checks faster, the QA team is using AI to move testing earlier in the production pipeline – in some cases, before a feature has even been assembled in the game.
For Nikita, QA Engineer at SuperCity, that shift began from a much more basic problem. He was a manual tester who did not know how to program, but saw a growing opportunity to automate repetitive work.
“I had some basic understanding of how programming languages work, but I had never written automated tests myself,” Nikita says. “Two years ago, I started using AI for the thing it was already quite good at – helping me write my first autotests. It worked really well in our production process.”
Those first experiments gradually evolved into something much larger.
Today, most of SuperCity’s QA team can version scripts with AI assistance. Some of those scripts are combined with LLM-based reasoning, internal documentation and agentic workflows. And instead of spending hours manually comparing tables, localization files and in-game content, QA engineers are increasingly supervising systems that perform the first layers of verification themselves.
The starting point: hundreds of lines, dozens of checks
One of the clearest examples comes from SuperCity’s expeditions.
Expeditions are large live events containing quests, cutscenes, dialogue, configuration data and localized text across multiple languages. Testing the narrative layer used to be highly manual.
Once an expedition had been assembled and was stable enough to test, QA would play through it, skip through quests, read the text in quests and cutscenes, compare entries against documentation and localization files, and look for missing strings, incorrect keys or discrepancies between different versions.
The source material itself could consist of hundreds of rows in large spreadsheets.
“You would literally sit there and look through hundreds and hundreds of lines to make sure nothing had shifted, the texts were correct and everything matched,” Nikita says. “Doing that with your eyes was painful.”
QA would identify technical discrepancies, create bugs and try to localize their causes. Afterwards, the narrative team would conduct another review focused on issues that required narrative expertise. The checks alone could consume roughly a working day for QA and another working day for narrative specialists.
The team first attacked the problem with scripts.
With AI assistance, Nikita began creating automated checks that could compare source documentation with production files and highlight obvious mismatches. But deterministic comparison had its own limitation: a script could tell whether two strings differed, but it could not necessarily understand whether the difference actually mattered.
That led to the next step – adding an AI reasoning layer on top. The resulting system does more than compare text character by character. The LLM can evaluate the content semantically, examine translations and help distinguish meaningful inconsistencies from harmless differences.
The process that once required substantial manual review from two disciplines can now be reduced to roughly one and a half to two hours: run the scripts, review the resulting log, investigate the remaining anomalies and create bugs where necessary.
AI works better when the process already knows what “correct” means
A common assumption about LLM adoption is that a team can simply give a model its documentation and ask it to automate the workflow.
SuperCity found almost the opposite to be true.
The better the existing QA process, the easier it became to automate. Before AI, much of the team’s testing expertise existed inside the heads of experienced people: what to check in an expedition, how quests, objects, rewards, configs and other parts of the feature were connected, which defects tended to repeat, and where problems were most likely to appear.
To turn that into a system an AI agent could work with, the team had to formalize it.
Bug reports became valuable historical context. Repeated defects were analyzed. Their causes were documented. Existing scripts captured specific validation rules and dependencies between data fields, and defined what should be considered a valid or invalid state.
Over time, that created a knowledge layer that AI could consume.
Nikita says simply uploading a feature description and a few configuration files is rarely enough. A model needs documentation, examples, files from previous runs, known errors and the validation rules encoded in existing tests.
“If parts of the puzzle are missing, the model starts inventing things,” he says. “The more detailed and accurate the context is, and the more sides of the system it describes, broader and more accurate context helps it the more LLM understands how different parts of the feature work together and make fewer incorrect assumptions.
There is also an iterative layer.
The engineer runs what the model has produced, examines where it misunderstood the system, gives feedback and improves the surrounding instructions and documentation.
In other words, AI adoption did not eliminate the need for mature QA knowledge.
It forced the team to externalize that knowledge.
From one enthusiast to a team capability
That change also affected who could participate in automation.
At the time, SuperCity’s QA team was mostly doing manual testing, and scripting was not yet a regular part of the workflow. Today, Nikita says scripting has become a normal part of the workflow across the team. When new approaches proved useful, knowledge was converted into documentation and manuals and shared with colleagues. As tools changed, those guides were rewritten and the process evolved with them.
Eventually, writing a script for a new feature became less of a specialist activity and more of an expected QA option.
At the most recent launch, Nikita says automated scripts covered roughly 20% of the testing surface but detected around half of all bugs found during testing. He notes that this was an unusually strong result: typically, automated checks account for closer to 30% of the bugs the team finds.
That does not mean manual testing disappears. It means QA engineers can spend less time manually repeating deterministic checks and more time on cases that actually require human judgment.
A game designer, for example, previously might receive an expedition version only to discover technical problems that made proper gameplay review difficult. Resolving those issues and reaching a usable state could cost a day or two.
The team now runs a set of automated checks first. In roughly two hours, it can hand the designer a version where many of those structural problems have already been removed.
The designer can then focus on what actually requires design judgment: how the game feels, whether the balance works and whether the experience is enjoyable.
The bigger shift: finding bugs before there is a version
For Nikita, the most important change may not be automation itself.
It is when testing begins.
The team is currently working on a new archaeological expedition. Historically, meaningful QA could begin only after the expedition was assembled and sufficiently stabilized.
With the new workflow, some checks can begin as soon as a game designer has filled in the documentation.
An agent can analyze that documentation, compare data and detect inconsistencies before a developer has implemented the feature. That changes the economics of a bug. Fixing an incorrect value in documentation may take minutes. Discovering the same mistake after implementation may require investigation, changes, reversioning, retesting and coordination across several specialists.
“The stage at which we start finding serious bugs has moved dramatically earlier,” Nikita says. “The difference in the cost of fixing an error in documentation and fixing the consequences of that same error in an already assembled feature is enormous.” This turns AI from a testing accelerator into something closer to a shift-left mechanism: quality assurance begins upstream, before production work compounds an early mistake.
Human in the loop – but at a higher level
As agents become more capable, the human role changes. A year or two ago, every newly generated script required careful manual validation. The team created test datasets, deliberately inserted known errors and checked whether the automation detected the right problems. That validation still happens whenever a new tool, test or workflow is introduced. But mature systems do not need to be manually revalidated from scratch every time they run.
Nikita can now ask an agent to execute established scripts, examine the output, validate potential errors and prepare an initial testing report. His work increasingly starts one level above: reading that report, spotting suspicious conclusions and deciding where deeper investigation is required.
The principle is straightforward: automate the process, not the accountability. “If something is new, we test it,” he says. “Human intervention is needed to create something new, set up a new architecture or establish a new process. Once something has been tested many times, we know how to verify it and the verification itself has been automated, you do not need a human to repeat the same work forever.”
And there is one layer he does not expect AI to take away. Responsibility.
“No matter how AI develops, there will still be someone accountable when something goes wrong,” Nikita says. “Humans in the loop will be here for a long time.” QA becomes less about checking and more about designing quality systems For a manual tester, this creates a surprisingly different job.
Nikita describes many traditional QA tasks as cognitively routine rather than technically difficult: look at a value in a table, look at the corresponding value in the game, compare them – then repeat the same operation hundreds of times.
AI and automation remove more of that layer. What remains is deciding what should be checked, how a system can fail, what context an agent needs, how the result should be validated and how the next part of the workflow can be improved.
“It became much more interesting,” Nikita says. “Instead of spending time on cognitive routine, you are constantly thinking about something new: how can I make this faster, how can I make it better? You start expanding your expertise because now you actually have time to understand more things.”
That may ultimately be the more important story behind AI in QA. At SuperCity, the technology did not simply make manual testing faster. It allowed a manual QA engineer who had never written automated tests to begin versioning them, helped turn individual expertise into reusable team knowledge, moved some testing from the finished playable version to the documentation stage, and gradually shifted the engineer’s role from executing checks to architecting the systems that perform them.
The machine can run the test. The harder questions – what should be tested, what “correct” means, and who is responsible for the result – still belong to the engineer.
