The biggest moments in AI in 2021

Battling a deadly virus, launching a vaccine in record time, unsticking the supply chain glut and getting addicted to Tiktok dance moves are just some of the examples of how Artificial Intelligence (AI) played a role in our lives this year. As 2021 races to a finish, here is a quick recap of the biggest moments in AI.

– The GPT-3 API ecosystem: Large Transformers and language models were all the rave in 2020, but they progressed so fast that now OpenAI has over 300 companies building apps by mid 2021 focussed on GPT-3–powered search, conversation, text completion, and other advanced AI features, using its API.

– AI2’s Macaw: While GPT-3 was awesome, as it can support a wide variety of tasks, it was also expensive to build, train and run. This year, Allen Institute for AI (AI2) open sourced Macaw, a generative QA system that outperformed GPT-3 by over 10%, Even though Macaw is an order of magnitude smaller (11 billion vs. 175 billion parameters) Macaw is publicly available for free. Thanks to @Adam Cheyer for bringing this to my attention

– Deepmind AlphaFold 2: Using Deep Learning to aid fundamental discovery, the AlphaFold 2 was most likely the technological breakthrough of the year. This deep learning model increased the accuracy and reliability in predicting the ways proteins fold, which could lead to targeted drugs that bind to specific parts of molecules, and more personalized medicine breakthroughs. Just last week, this AI model predicted SARS-CoV-2 omicron variant might not evade antibody neutralization (Yay!)

– Github Copilot: In June, Github launched Copilot, a new AI pair programmer that helps you write better code. GitHub Copilot draws context from the code you’re working on, suggesting whole lines or entire functions. It was developed in collaboration with Open AI (based on Codex) and is more just an auto-complete. If writing code gives you superpowers, now you have an AI superpower helping you gain your superpowers!

– Multimodal learning: If the 2020s, gave us large language models and meta-learning, 2021 combined vision and text together. OpenAI’s CLIP and DALL-E models have shown just how robust the combination of language modeling and vision can be. Real world application beyond content generation including robotics, will have the potential to put AI into the physical world.

– Open-Ended Learning: Google’s DeepMind published a paper on this and I thought it was game-changing. Going beyond Reinforcement Learning (RL) that built amazing systems like the Alpha Go, this paper describes the ability to build agents that are capable of learning several new tasks, without needing human interaction data; this has applications beyond just gaming.

– Self-Supervised Learning : Data curation, labeling and training take up a lot of cycles in doing AI, and are often the reason whether a project succeeds or fails. For AI systems to be really integrate into the real world, these systems need to learn like humans do. They have to be able to directly learn from whatever they are given – text, images or other types of data, without relying on the labelled datasets to teach them how to recognize objects in a photo, interpret a block of text, or perform any of the countless other tasks that we ask it to. Facebook’s SEER enables this for computer vision today, and is definitely a game changer.

– Graph Neural Networks (GNN): GNNs grew in popularity this year, but still has ways to go. I’ve long maintained that Graphs are a better data representation that captures the richness and dimensions in the data and captures context better. But computing on graphs had been hard, especially for Deep Learning. GNNs solve that problem, more to come here in 2022.

– Generative AI – Generative AI is how you create new content by utilizing existing text, audio files, or images information. Generative AI algorithms detect the underlying pattern related to the input and produce similar content. In fact, by 2025, Gartner predicts that generative AI will account for 10% of all data produced, which may be problematic in the era of fake news!

I’m sure there is a lot more, but this is what I thought were the big movers and shakers in Research. The moments in Applied AI were even more fascinating.

– Drug Discovery & Life Sciences: Thanks to the spotlight landed by COVID-19, data science in life sciences and drug discovery is the hottest trend in AI. Obviously this is an early indicator based on VC investments, but is also highlight more smart people are choosing to work on these problems.

– Tiktok’s ‘For You’ Algorithm: The addictive app powered by its video recommender system won the hearts of a billion people, and is a classic result of pairing mathematical prowess and an acute understanding of the human condition. The inner workings were “leaked” and was covered in a NYTimes article last week. It will blow your mind.

– Trouble at Facebook and the re-targeting to Meta: I don’t think anyone was surprised that Facebook got into more trouble this year with their algorithms, their (un) ethical practices governing it, and the constant outflow of smart engineers. But that was all given a big left swipe and the company has a new identity, focus and brand – to help build the Metaverse. I’m not surprised at the move as it was the most logical ones for the world’s #1 social network. The only competition they have now is the physical world and what better way to fight it than bringing presence and total immersion to the virtual world.

– Industrialization of Machine Learning: Machine Learning Operations (MLOps) rose to the top of the most hot/hyped topic as organizations started grappling with getting out of experimentation to production level AI deployments. While the role of MLOps toolkits cannot be debated, this is most likely going to spurn an SRE-like movement involving people, processes and tools in ML engineering. This also highlights the need to cross-pollinate the disciplines of data science, data operations and software engineering for successful organizations.

– AI Governance & Ethics: There were a lot of expectations of increasing focus on Governance and Ethics in AI, post-senate hearings of big tech companies and social media induced riots. Although this space has seen a lot of innovation from companies launching to help Govern AI, the adoption of these practices is still lagging. The debate is red-hot right now, but Government regulations like the NY AI Hiring Bias Bill when passed in 2022, will create enough momentum for organizations to take a serious look at this space and get prepared.

I hope you enjoyed this short list, and I’m sure there are tons more that I have not captured here. I asked some friends and I got suggestions from the rise of the Data Mesh architecture to the Mars Perseverance Mission, I have tons of honorary mentions. Let me know what you felt was cool and a big moment in AI this year.

Happy Holidays and wish You and AI will do more in 2022.