AI Learns to Forget

AI Learns to Forget AI Learns to Forget AI Learns to ForgetWe sentient beings do leid spend our lives collecting, storing, and regurgitating every experience that has passed through our sensory organs. If we did, wed know every leaf our eyes had ever set on, every whisper and every roar that had ever reached our ears, every fact or face wed ever come across. Wed remember every detail equally and use up our finite memory banks.Thankfully, not everything sticks, and that which is not repeated disappears from our memory, or is stored in more remote vaults. Though often overlooked, the ability to forget, and to forget wisely, is crucial to our intelligence and our survival. This is one area where flesh and blood still towers over machines. Artificial intelligence systems do not know how to efficiently forget what is unimportant.A new computing technology mimics some aspects of human thought by learning how to forget unimportant memories while retaining more vital ones. Image Purdue Engin eeringFor now, that is. Researchers at Purdue University have learned how to use a quantum material, samarium nickelate, to create a device that will know what to forget. It emulates the learning and forgetting habits of animals. The researchers have used what theyve learned of this material to create algorithms that can be applied to efficient, practical lifelong learning, as they call it, for the world of artificial intelligence.Currently, the only forgetting neural networks do is the disastrous kind.For most of the algorithms that are available, if they have already learned a data set and you try to show them a new data set, they incur catastrophic forgetting, says Kaushik Roy, a Professor of Electrical and Computer Engineering and head of Purdues Nanoelectronics Research Laboratory. From a neural computing point of view, we may not have enough memory to remember everything. In an environment where we do incremental learning, where we keep on learning new things, it is very, very important to be able to forget.The key to forgettingand forgetting the right thingsis samarium nickelates ability tobreathe. Professor Shriram Ramanathans team at Purdue showed that when it is bombarded with hydrogen protons its conductivity increases, and when it is subsequently exposed to air its conductivity goes down. But the lattice structure of the materialand its overall conductivitychanges every time its exposed to hydrogen protons.This breathing sort of phenomenon gives rise to this conductivity change, says Priyadarshini Panda, a researcher at the universitys school of electrical and computer engineering, who worked on the project.That conductivity change is what we are able to utilize in our computing model to do this learning to forget. Its because of this adaptive, life-like respiratory behavior, or habituation, that Panda and her fellow researchers have decided to call the devices organismoids.After creating their first organismoid with the samarium nickelate, they em ulated the materials unique ability to create both algorithms and an electronic spin-based device that can be used with conventional hardware. The resulting device uses current, instead of hydrogen protons, to change conductivity. Though the algorithms could be used with any computer, greater forgetting efficiency can be achieved with the hardware device.Though Roy and Panda were inspired by our own excellent capacity for memory loss, they hope their algorithm will help artificial intelligence systems forget things better than humans can, making them smarter than we could ever imagine.When we learned to fly, we studied how a bird flaps its wing. But now we have a 747 which goes half way around the world and carries a payload of 400 passengers plus baggage, Roy says. We want to do better than the brain, go faster than the brainhowever, it may not do exactly what the brain does.Michael Abrams is an independent writer. For Further DiscussionIn an environment where we do incremental lea rning, where we keep on learning new things, it is very very important to be able to forget.Prof. Kaushik Roy, Purdue University