This article was co-authored by Augustin Ador, lead data scientist at Dataiku, and Radoslaw Jurga, R&D engineer at UAVIA.
It’s a technology that goes by many names: unmanned aerial systems, remotely piloted aircraft, miniature pilotless aircraft, or simply drones. But regardless of what you call them, these little flying robots are everywhere and are quickly moving from a fun-to-have consumer good to a necessary and useful piece of industrial technology.
Unmanned autonomous vehicles can go wherever humans can’t and can speed up or augment a huge variety of industrial processes. By this point, we’ve all seen drones’ ability to take hi-res aerial video and photographs but, as technology advances, these machines are able to do a lot more than add some “wow” factor to the average travel vlog.
We’re now creating tools that allow these machines to make decisions and parse visual data on their own, sending the relevant information back to their on-the-ground handlers and avoiding risks along the way. The potential here is vast — soon, drones could be assisting with disaster recovery, surveying inaccessible terrain, assessing the safety of damaged buildings, helping farmers keep track of their crops and livestock, and providing real-time aerial inspections and inventories of shipping yards and manufacturing plants. If used correctly, they can create smarter and more efficient industrial settings and make work safer for their human operators.
Industrial Drones Face Unique Challenges
Drones form just a small part of the growing ecosystem of technologies that are using automation, machine learning (ML), and data exchange to enhance industrial spaces. This massive shift is known in some circles as “Industry 4.0.” But despite the high-minded, futuristic name, there are a lot of painfully practical hurdles we must overcome before the fourth industrial revolution can become a complete reality.
In order to get to the smarter, safer, and more efficient industrial spaces that Industry 4.0 promises, we need excellent data analysis and a lot of collaboration — between companies, technologies, and even pieces of hardware themselves in the internet of things.
As mentioned above, drone technology can implement AI and ML so that the machines can make decisions, assess risks, and send useful information back to the ground. But to do this effectively, the technology must first overcome challenges related to object detection.
In order for a drone to assess areas in real time, it must be able to identify and detect objects quickly and accurately. When the machine is hovering in the sky, the objects it needs to detect — like a shipping container, a box, or a sprouting crop — may only take up a few pixels on its internal camera, which makes accurate object detection even more difficult. And much of this computing needs to be done on the drone itself to avoid computational bottlenecks that could get in the way of real-time insights.
Improving Image Detection With AI
Running an ML model on live video for edge computing on a tiny piece of machinery that may or may not be hundreds of feet off the ground is a tall order. Its solution requires both state-of-the-art data analysis and a purpose-built ML pipeline.
Only when we combined the AI capabilities of Dataiku with drone hardware and data management systems from UAVIA were we able to create a solution. UAVIA’s Robotics Platform allows users to remotely control fleets of industrial drones while processing, analyzing, and sharing the data they gather. In this collaboration, Dataiku further enhanced UAVIA’s robust AI capabilities with its all-in-one platform for enterprise AI and ML.
This project deployed Dataiku’s platform, which allowed drones using UAVIA’s DroneOS embedded intelligence to use AI in order to detect, inspect, and track objects of interest through time.
The data pipeline starts with the careful choice of a method for object detection. Many current models of object detection like R-CNN can be incredibly accurate, but because they consist of two stages (region proposal and classification), they just aren’t fast enough to react to real-life scenarios like analyzing live video yet. And in the particular use case of aerial inspections, speed is of the essence and can weigh out a slight loss of accuracy. Using a single shot detector, or SSD, we are able to eliminate the region proposal stage and find a compromise between inference speed and accuracy.
Altitude, too, can present an issue — even from only 40 feet in the air, the objects detected are so small that the ML model leaves key objects undetected and has a few too many false identifications. By implementing a real-time detection tracker, which assigns a unique ID to detected objects from one frame of video to the next, we’re able to eliminate much of the “clutter” in the drone’s field of view. This allows the model to smoothly track the movement of objects between frames, rather than analyzing each frame individually. This greatly reduces the risk of false positives and makes the drone’s analysis of its surroundings more efficient.
But even with working object detection and tracking, there were more hurdles to jump. In order for this solution to be scalable in industrial contexts, the drones need to have ML capabilities for edge computing right out of the box. Deploying ML models usually requires heavy manual adjustments to adapt memory size and speed, but with the right combination of data analysis and drone technology, it’s possible to fully automate the process.
For further accessibility, there also needs to be a useful abstraction layer above the technology to provide on-site operators with critical information that can help them make decisions and optimize worksite performance. Both Dataiku and UAVIA strive to make data accessible to all levels of decision makers. Dataiku’s flexible, collaborative platform can tackle the entire data pipeline and pair well with UAVIA’s easy-to-use drone control applications.
Beyond all of the technical specifics, what really solved this problem was the spirit of collaboration and cooperation, which will be even more critical as we march toward Industry 4.0. It was only by combining the best in machine learning and the best in drone tech that we moved closer to actually seeing the full potential of unmanned autonomous vehicles.
