If you have a sibling and a game console, this should sound so familiar: "UP! OK, NOW GO LEFT! NO, LEFT! WATCH OUT FOR THAT GUY FOLLOWING YOU! I SAID LEFFFFFTTTTTTT!!!" Well, while the loud instructions are quite annoying (your brother's just trying to help you win this battle, you know, as a team), an AI was actually able to learn from this method.
Undergrad students from Stanford University taught an AI to play Montezuma's Revenge for the Atari 2600 by utilizing your sibling's way of educating, with the hope that this approach could help regular people like us to talk to AI in the future. "Everyone in the developed world is interacting with AI every day, whether they know it or not. Regular humans need to be able to talk to their machines," study
's co-author Russell Kaplan said in an interview.
By training the AI to understand natural human language and how it relates to actions in-game, they were able to guide it to follow simple instructions like "climb down the ladder" and "get the key." The result? A high score of 3500! But unlike google's DeepMind (which scored 6600 by the way), the researchers didn't use any pseudo-rewards and just did continuously put a human in the loop until the AI "understands" what the humans' actions meant instead.
"The way traditional AIs work is randomly mashing buttons until they get a reward from the environment, and then learning to mash buttons in the future. But in the real world, you'd have to get so lucky to do the correct random sequence of actions that it's virtually impossible to apply existing approaches productively," Kaplan said. But with natural human language instructions in the training process, the whole process of trial and error gets a little less random.
This study envisions better human-AI interactions with the hope that soon, we will be able to teach them the same way we teach other people. So take that, mom! Video games can be educational!