As a writer of both sarcastic and extremely aggressive robot characters, I’m naturally interested in any developments in the field of AI… that my laughably primitive, non-scientific brain can actually understand and process, that is. Mainstream media understandably has its own obsessions at the moment and consequently, amazing developments happen under the radar. Perhaps some of these ‘wins’ are the kind that, in a sci-fi novel, are key moments in the march towards the much-feared singularity, where Skynet becomes self-aware and… well, we know the rest.
One such potential milestone happened last year, when the Agent57 deep-reinforcement learning agent – created by DeepMind, the company that created AlphaGO – successfully outperformed a human benchmark on a suite of 57 Atari 2600 games dubbed the Arcade Learning environment (apparently, it refused to play E.T. The Extra-Terrestrial – joke). Games such as these may seem blocky and simplistic, but are a great testing ground for building adapative algorithms.
However, since 2012, four games in particular had proved particularly tricky for a range of developmental AIs, for different reasons. Two are Pitfall! and Montezuma’s Revenge, because they require an AI to balance exploration versus exploitation in its strategy. Solaris and Skiing, on the other hand, apparently represent ‘long-term credit-assignment problems’: an AI must match the consequences of its actions to the rewards it receives. The blog post outlines how the DeepMind developers overcame the challenges these four games pose, explaining how facets such as observation, short-term and episodic memory, off-policy learning and long-term novelty rewards combine to direct the AI’s successful gaming behaviour.
According to DeepMind’s report, intelligence is defined as “an agent that performs sufficiently well on a sufficiently wide range of tasks”, and being smarter than your average human on 57 Atari games, with the multitude of different challenges they represent, is some ask. The report is a fascinating read; and the rabbit hole goes very deep, very quick… new projects are wrestling with the Atari benchmark as we speak.
If you need me, I’ll be under my desk, afraid to refresh my browser, trying to imagine what the singularity is going to look like when it finally comes.
