Generative AI and ChatGPT have been the stars of the tech show over the last couple of years. Storming into the mainstream and hitting headlines with the creation of personal Pixar avatars and cheating on university essays, there aren’t many who haven’t heard of or tried generative AI in some capacity.
Recent developments in AI systems able to generate pictures (DALL-E2) or text (ChatGPT) have taken society by surprise. But under all of the fun and time-saving hacks, there has been an undercurrent of ‘what does this mean for the future of humankind?’. Suddenly, this dystopian-sci-fi story we believed to be hundreds of years away could be fast approaching, with many fearing humans will become obsolete in the process.
Of course, the debate about AI has two sides to it: some people predict singularity to happen in the next decade and there are ones who claim that the current state of generative models is to merely mimic all the art and text that already has been created by humans. As such, the argument follows that AI is simply missing the creative spark that only humans have.
But putting the question of whether current generative AI models are creative or not to one side for now, let’s take a step back and instead ask: why did such great AI advances happen in the image, text, and music generation institution? And what potential does AI hold beyond large language models (LLMs)?
Images and text: why is this the generative AI starting point?
One of the reasons that researchers may have targeted those applications is because they believed that this type of image and text generation can resonate better with the general population. Most people can relate and understand pictures and text but not as many can appreciate the intricacies of more complex tasks such as architectural or engineering design. Another reason is that there was plenty of good quality, well-structured data available to train those models – the internet, together with digital libraries and museums, are full of pictures and text that is easily and freely available.
Three main ingredients make AI training possible and successful – access to data representing the problem domain, the ML algorithms needed to process that data, and the computing power to run those algorithms and crunch the data.
The last decade has seen a dramatic increase in compute power availability due to advancements in GPGPU (general purpose graphic processing units) led mostly by Nvidia. Cloud computing (AWS, Azure, Google Cloud) provides access to a scalable way of deploying computing power. The introduction of transformer-based Neural Networks (NNs) enabled an effective way of using large volumes of data in training, and as mentioned before, text and image data was already available on the internet.
The buzz around ChatGPT-like models makes people forget, however, that the first time modern AI really impressed by dethroning a human in something highly complex and intellectual, was when AlphaGo beat Go World Champion in 2017. So how did the researchers from DeepMind train AI to outclass the best human in a very complex game?
How can AI outperform humans?
There is some data on Go games available but nowhere near as much as would be needed to train an AI to beat a world champion! There is however one big advantage that researchers working on AlphaGo had over the ones working on ChatGPT – Go is a board game, with very well-defined rules, and objective winning conditions, and it is very easy to simulate on a computer.
Looking ahead to generative AI such as ChatGPT’s capabilities, there is a key lesson to be learned here: the key aspect of AlphaGo’s success was the ability to simulate the game which allowed the training process to generate the data it needed on demand. Simulating Go is very easy – the rules are simple and the winning conditions are very well defined. The success of AlphaGo suggests that if we can simulate a given process, we can use that simulation to generate data to train an AI system to operate in this domain better than any human.
What does the future of AI look like beyond LLMs?
In terms of the use and potential that AI holds, we’re at a really interesting inflection point. As we start to move beyond the hype that OpenAI brought with ChatGPT, innovators are using AI to solve some of the biggest problems we are facing today such as the climate crisis.
Engineering is an industry that has been using AI for a while and this is only accelerating. AI can be used for multi-physics simulation to create iterations of engineering designs. In Go, there are game rules and winning conditions therefore, we want AI to find the optimal game strategy. In engineering there are also rules: the laws of physics, manufacturing constraints, and a winning condition – an optimal design that fulfills the requirements. With that, we would want AI to find that optimal design.
Recent data suggested that the possible development of AI that can outperform humans on every task was given a 50 percent odds of happening by 2047. Where AI can outperform humans is in its ability to remove human bias and create designs from scratch. In doing so, AI can create optimal designs at the source level – without generations of human bias and knowledge, the designs created can bring about change that is simply beyond human capability.
Moreover, ML can use the data generated and make improvements in the speed of simulation which becomes a positive feedback loop. The more data we have, the better we can be at generating more data and pushing the limits of unexplored engineering designs.
Enter Large Engineering Models (LEMs)
In motor engineering and design specifically, over the coming years, it will be possible to accumulate enough diverse motor data that we will be able to train models of a scale similar to Large Language Models (ChatGPT, DALL-E2 – like models), we call those new models Large Engineering Models (LEMs).
They will unlock for engineering and motor design what ChatGPT and DALL-E2 have achieved for text and image generation, with a major difference that LLMs output is like human output, while LEMs output will far supersede human engineers’ output, just like Alpha-Go superseded human Go champion.
