Autonomous driving is one of the most advanced applications of artificial intelligence in the real world. Behind every safe manoeuvre lies a complex stack of AI systems responsible for perception, localisation, and decision-making under uncertainty.
In this interview, we speak with Avraam Tolmidis, an Engineering Manager with over 10 years of experience in autonomous driving and embedded systems. Avraam has led teams tackling one of the hardest AI challenges in the field: centimetre-level urban localisation. As part of Germany’s @city initiative, his team built systems capable of accurately understanding vehicle position and surroundings in highly complex urban environments.
He shares insights into the technical and leadership challenges of building real-world AI systems that move beyond controlled demos and toward scalable deployment.
One of your most successful projects is @city. Please give an overview of this project and your role in it.
@city was a large German government-backed initiative focused on automated driving in urban environments, which we participated in as a team during my tenure at ZF. Different automotive companies contributed to different scenarios. The scenario that we had to demo was especially demanding: the vehicle had to approach a multi-directional intersection with traffic from all four sides, obey traffic lights, make an autonomous left turn, and end up correctly positioned in the target lane.
I was leading the localisation and mapping team, which handled a significant portion of the project. Our team processed sensor and map data, constructed a representation of the vehicle’s surrounding environment, and localised our vehicle and other traffic participants. This output was provided to the subsequent decision-making and planning modules.
Besides managing my team, I coordinated with other units to adequately resource our scope and resolve planning conflicts, support team unblocking efforts, and assist with key technical decisions. Additionally, I contributed to the initial localisation direction, method of environment modelling, team preparation to work with HD maps before standard maturity, and the validation methodology to determine system accuracy for urban autonomous driving.
What were the main technical challenges in the @city project you knew you’d have to solve from the start?
From the beginning, there were three big challenges.
The first was localisation accuracy. For the scenario to work safely, it was not enough for the vehicle to know roughly where it was. It had to know its position with very high precision. When you approach a complex intersection and then execute a left turn, even relatively small errors can have a significant impact: you may stop in the wrong place, position poorly in the junction, or fail to land correctly in the target lane. We knew we needed something in the range of roughly 10 to 12 centimetres worst-case accuracy.
The second challenge was environmental modelling. It was not enough to localise the vehicle itself. We also needed a representation of the environment that was useful for the downstream planning and decision teams. That meant it had to be expressive enough to support complex manoeuvres, but also efficient enough in terms of compute and easy to query in real time.
The third challenge was the use of HD maps. At that time, HD maps were clearly important for this type of precise urban localisation, but the ecosystem was still immature. We were going to receive OpenDRIVE-compatible map data for the demonstration track, but we had not worked with that format before. At the same time, the industry-standard path through ADASIS HD maps was not yet available, as ADASIS v3 was still under development, and there were no usable provider maps. So we knew very early that we would have to solve not just localisation but also the practical challenge of obtaining, parsing, interpreting, and integrating HD map data before the standards and tooling were really there.
How did you approach building the localisation system during the project, given that industry standards were not mature yet?
We took a pragmatic approach instead of waiting for the industry to develop around us.
What helped was that this was not our first thought process about urban localisation. In an earlier project with an external partner that I personally led, we had already built a prototype to explore how localisation would work in an urban environment. That work was incredibly valuable, as it revealed the limitations of the more straightforward approaches and first indications that standard Kalman filtering would not be robust or precise enough for our needs. So, by the time @city started, we had already made up our minds that a particle-filter-based approach, although more complex, was better. This gave us a more precise technical starting point.
In parallel, we had to solve the map side. Since the standards were not yet mature and HD map availability was limited, I helped the team find a practical path forward. I had previously met another team inside the company that was working on a cloud mapping initiative and had already mapped the area around their location. I reached out to them and was able to make several things available to our team: parsing software, tests, support, and actual OpenDRIVE map material that we could use for development. That accelerated us significantly because we did not have to build everything from scratch.
You hit 7 cm worst-case accuracy. How did you achieve that, and what drove that level of precision?
The first thing to say is that the manoeuvre drove the precision target. A left turn in an urban environment calls for far more stringent localisation than in simpler highway-style automation. The vehicle has to stop in the correct place relative to the traffic lights and other cars, position itself correctly within the lane as it approaches the junction, make the turn, and end up near the centre of the target lane. If localisation is off by too much, everything else in the stack is immediately under pressure.
The requirement for the scenario to be successful was that localisation be off by no more than 10-12 centimetres in the worst case. We ultimately achieved around 7 centimetres, and that was due to a combination of factors.
The first was the use of HD maps. Without HD maps and given the sensor and hardware constraints at that time, previous work had already shown how difficult this would be. The highly detailed lane-level information in the map gave a much stronger frame of reference. The second was the sensing combination, particularly the use of lidar with the rest of the sensing stack. This allowed us to obtain much better-quality environmental cues against which to align the map.
The third was the notion of localisation itself. A particle-filter-based approach was used because it better handles the uncertainties and complexities of an urban environment than simpler filtering methods. The fourth was the rigour of validation. It was not just tuning until it looked good. We had already established a means of comparing against high-accuracy differential GPS as ground truth using structured test cases on a mapped test track. This gave the team confidence that the numbers were real and not just optimistic internal estimates.
How did you validate that the system was ready for real urban traffic conditions?
We had to be serious about validation because relying on happy-path demos is very easy and can delude oneself with such systems. A critical opportunity was a prior project: I was aligned with another internal team that had access to a company test track outside the company. They also created an HD map of the test track, including landmarks and lane markers, making it a good validation environment.
This gave us differential GPS with an absolute accuracy of about 2 cm. From that, we could set up well-structured test cases to expose, in addition to average behaviour, failure modes and limitations of the previous concept. It also highlighted several hardware constraints.
We reused and extended that testing framework for @city. Hence, we could quantitatively test localisation rather than subjectively assessing it. We tested whether the system was accurate enough for the manoeuvres we cared about, not just whether the car appeared to drive correctly in a demo.
Beyond measurement, readiness was also validated by ensuring that the consuming teams could consume the output. A localisation system can only be considered ready when the planning and decision layers can rely on it stably, in a timely manner, and in an understandable way. So, in that sense, validation also came down to performance, interfaces, and how well the environmental model fed the rest of the stack.
How do you see the localisation problem in autonomous driving evolving? What’s still unsolved?
I think localisation has improved a lot, but I would say the core challenge is still not fully solved in the general case.
In controlled environments, or in well-mapped areas with strong infrastructure and favourable sensing conditions, very good localisation is absolutely possible. But what remains difficult is delivering that same level of robustness across all the variations you get in the real world: changing weather, temporary roadworks, degraded lane markings, unexpected objects, map drift over time, GNSS limitations in dense urban environments, and the long tail of unusual road layouts.
The next big question is scalability. Excellent localisation over the demo area or an ODOD is one thing; updating accurate HD maps economically over large areas is another. It is still an open systems question of how much prior map knowledge is leveraged versus how much is inferred online from perception. When I worked at @city, HD maps were needed to achieve the level of accuracy we aimed for. I still consider maps very important, but the balance among mapping, online perception, and prediction has yet to be established.
What remains an open question is not a particular instance or case where exact localisation has been achieved. The more challenging, open question is how this is done robustly, economically, and at scale in messy real-world conditions.
What’s the gap between what autonomous driving demos show and what is actually needed for real-world deployment?
The most significant gap is between a successful demo and a deployable product.
A demo can be very significant. It can demonstrate that a concept works or that a vehicle can be maneuvered through a tricky maneuver. It can also show that an architecture has real potential. This is why projects like @city matter: they push the technology into genuinely challenging scenarios.
However, the standards are quite different when deploying in the real world. Success in one defined setup will not be good enough. The system has to work reliably under all kinds of conditions: various road layouts, different traffic behaviours, changing light, bad weather, degraded sensors, out-of-date maps, and all the other nasty details that go with real driving. It has to be done safely, repeatedly, and at scale.
And then come the costs. Usually, this is underestimated. You can show the state-of-the-art with a stack of sensors and computation in demos or research. However, real-life deployment comes down to the economically viable performance-to-cost ratio for a product and its end user. This is even more the case in complex urban scenarios, where safe autonomous performance usually depends on a demanding mix of sensors, compute, and high-quality maps, which, while they work in practice, have scale-related aspects at the commercial level.
For me, demos are significant steps, but they are only steps: they are proof of concept; they cannot, in any case, prove that something is available for a large-scale application. The real work starts after the demos: from here onwards, the system must be made reliable, safe, and affordable enough to operate outside controlled circumstances.
What does good engineering leadership look like to you, and how did @city shape that view?
To me, good engineering leadership is about creating an environment where a team can do good work. That begins with the basics: the right people, clear priorities, sufficient focus, good ownership, and healthy collaboration. However, that is not enough for truly ambitious projects. An excellent engineering leader helps the team navigate ambiguity by removing impediments, supporting difficult decisions, and providing clarity when the way forward is unclear.
That was true in @city. It was not the kind of project where leadership meant just tracking milestones and asking for updates. The team was working through technical uncertainty, immature standards, cross-team dependencies, and real delivery pressure. In that environment, leadership had to be active. Sometimes that meant making sure the right people were focused on the right parts of the problem.
Sometimes it meant helping the team think through technical direction. For example, around the localisation concept, the environmental model, or exposing an efficient interface to downstream teams. Sometimes it meant unblocking the team in practical ways. One example was helping secure approval to use C++, where it was the better engineering choice, even though the wider group had been working only in C at the time. Another was connecting the team with an internal group that already had useful map-parsing capabilities, so we could move faster rather than build everything from scratch. The third example is to ensure we collaborate closely with that group so we both benefit.
What @city reinforced for me is that good leadership is both supportive and hands-on, but not controlling. You do not micromanage, but you also do not stay distant. You understand the problem space well enough to ask good questions, support good decisions, and help the team keep momentum when things get difficult.
