Computer program that learns to play classic NES games

A bit out of context, but it reminded me of this funny moment. The only winning move is not to play. ▶ this video (Background: Someone training an algorithm to win NES games based on memory state)

Did anyone else get reminded of the Lexographic Ordering Solver that played NES games? This was featuring in Sigbovik 2013. link Video: ▶ this video

Not quite what you're describing, but no one has yet linked the classic Tom7 series where he applies deep learning to classic NES games: ▶ this video

Tom7 implemented some of this with his Learnfun and Playfun ( ▶ this video ) - It looks for lexicographical changes to bytes in memory (i.e., one byte seems to almost always increase, and then when it suddenly decrements, a neighboring byte is incremented) as a way of calculating memory location for increasing values (score, x-position), and similarly detects bytes that generally seem to decrease (lives). It then tries to play the game, choosing inputs that increase the values that should be increasing (move right, gain lives), or tries not to decrease the values that are decreasing (don't die).

A lot of fuzzers use Mario or other simple games as an internal testcase. I'm aware of a hypervisor fuzzer from 2016 that did it, and I'm positive there are others (both before and since). Hell, tom7 has a fuzzer for exploring program states that uses Super Mario Bros as the example from 2013 ( link , plus a youtube video ▶ this video ), and he's definitely not the first either.

tom7 is awesome. He made an awesome and hilarious video years ago about his SIGBOVIK paper/research on AI playing NES: ▶ this video

There was an AI created the absolutely brilliant Tom7 on on his YouTube channel 'SuckerPinch' on April 1, 2013 that paused a game of Tetris instead of losing. In my opinion, Tom doesn't get enough credit for his work. He released his AI a full 8+ months before DeepMind released their Atari playing AI. Watch the whole thing, it's delightful. The AI pauses Tetris right before it loses at around 16mins into the video. ▶ this video I also recommend his other videos. Brilliant guy. Edit: Here's the paper link

I'm not sure many people can pull this off. The author here was also great in his presentation about automated NES playing, ▶ this video . You don't see often academics & hacker work that is also combined with the right amount of humor. He even 3Dified the original Zelda!

This reminds me of AI research using NES Games. The AI eventually became proficient at completing Mario levels, and along the way it discovered novel strategies for survival, obtaining points, and finishing levels. > Check out this timestamp to watch the machine "cheat": ▶ this video > Researcher's site about the project: link > The Paper: The First Level of Super Mario Bros. is Easy with Lexicographic Orderings and Time Travel...after that it gets a little tricky. : link

Thanks for pointing that out. I wouldn't have noticed who that was otherwise. His video on learnfun/playfun is both hilarious and amazing. ▶ this video

Here is an actual implementation of trying to implement a general AI to play all NES games. It's pretty interesting, but his approach does involve the algorithm having access to the raw NES memory map: ▶ this video

To reference: ▶ this video It's fun how program learning to play teris pauses the game just before loosing.

Why not use game emulators? With popular NES emulators you can advance the game frame by frame. You can read the raw memory addresses that correspond to the score. You can dump the memory at any time and reload the game to a specific game state. You can even manipulate the games in many fun ways by messing around with the game memory. Or give an AI algorithm access to memory addresses as additional information, instead of relying on pure machine vision, if you want to do that.. Here's an example of a guy who made a general game playing algorithm that brute forces it's way through any NES game: ▶ this video This isn't necessarily interesting from an AI perspective - the playing algorithm is just brute force. But it shows what can be done with the platform, easily reloading to previous states and exploring counterfactual futures (which is exactly the sort of thing RL algorithms do.) He also has a cool algorithm for finding the objective function of an arbitrary game, by watching a human play, and seeing what memory addresses increment. Which is a lot more easy to use than writing OCR code to read the score and game over states from the screen.

You should look at some other things the author did. They fit in the same area of craziness, and are absolutely great. For example link , Portmantout: A portmanteau of every English word, or the first one, ▶ this video , Computer program that learns to play classic NES games. I also liked the humor in What, if anything, is epsilon?, link .

In the context of AI and gaming, I definitely recommend this series of three youtube videos: ▶ this video Some games are played better than human would.

If you're interested in a "dumber" approach to machines "learning" to play classic video games (including Super Mario Bros), tom7's video [1] and paper [2] on Playfun are both fascinating. [1] ▶ this video [2] link

Tom 7's work on machine learning and NES emulation is very impressive. This isn't the first time he's attacked that space; you can see previous experiences he had with an automatic game-playing AI he built called "LearnFun / PlayFun" on that YouTube channel also: ▶ this video link link

So as someone with only a lay understanding of what NP-Hard means - does anyone remember that neural network that someone trained to play the game[1]? So if you can train a neural network to solve an NP-Hard problem, does that mean by definition (since NP refers to a whole class of problems) that a neural network could eventually be trained to solve any given NPH problem given the time and compute resources to do so? [1]: ▶ this video

Haha, I love it. There was a SIGBOVIK submission a couple years ago that looked at byte patterns within video game RAM (specifically NES) to automate the playing of. Pretty entertaining stuff. ▶ this video