Musk says Digital Optimus can play halfway through Diablo just by watching the screen
The agent works from screen video alone, and the head of a lab trained on gamers' clips wants a five-game match.
Early on October 10, 2026, Elon Musk said Digital Optimus can now get about halfway through the Diablo campaign. That's the computer-use agent he calls a joint xAI-Tesla project, and he wrote on X that it plays by looking at the screen the way a person does.
The post went up at 3:34 am Eastern. Counter-Strike is going well too, he said, and League is in training. No video came with it. He didn't even say which Diablo.
We’re making interesting progress here. Digital Optimus can play about halfway through the Diablo campaign so far just by looking at the screen like a human. Skill at Counter-Strike and other fast games is good too. League is also in training. Goal is to generalize across all games.
What Digital Optimus is
Musk laid the project out on March 11, 2026, under two names, Macrohard and Digital Optimus. Grok plans. Digital Optimus acts: it watches the last five seconds of live screen video and answers with keystrokes and mouse moves.
He compared the pair to the two halves of a mind. Grok is the thinking part, Digital Optimus the instinctive one. It's meant to run on Tesla's AI4 computer (he put the price at $650), leaning only lightly on xAI's far more expensive Nvidia hardware.
- Hardware
- Tesla AI4 ($650)
- Agent
- Digital Optimus, a joint xAI-Tesla project
- Diablo
- about halfway through the campaign
- Also in training
- League
- Stated goal
- generalize across all games
The March post set a bigger target than games. Musk wrote that the system could, in principle, emulate the function of entire companies, which is where the Macrohard name (a joke at Microsoft's expense) comes from. So why games? My reading is that they're the cheap test. An agent that only sees pixels has to hold onto a plan for hours to get through a long campaign, which is roughly what finishing a long job at a desk asks of it too.
A challenge from a game-data lab
About four hours after Musk's post, Pim de Witte answered with a challenge. De Witte leads General Intuition, a lab that builds on Medal, the clip platform where players upload billions of gameplay highlights a year. Its models learn from that footage.
Can I challenge you? Digital Optimus vs General Intuition models. Same constraints / harnessing. Pixels in computer actions out only. We both pick 5 games and have them play each other!
His terms are the ones Musk already uses for Digital Optimus. Pixels in, actions out. Each side would pick five games, and when we checked on October 10 we couldn't find a reply from Musk.
General Intuition has shown some of its own work. Its site describes MIRA, a playable multiplayer world model made with Kyutai and Epic Games that learned four-player Rocket League from 10k hours of bot play and runs in real time at 20 fps.
What Musk didn't say
The October post doesn't give a difficulty setting or say how many hours the runs took. It doesn't say whether Grok was planning during them, either, or what hardware they ran on. Until Tesla or xAI posts footage, halfway is Musk's own account.
De Witte's challenge was still open on X when we checked on October 10.
More on xAI
- Musk says Grok Bot will route tasks to Claude Opus 5.5, Midjourney and SunoOctober 7, 2026
- Musk says SpaceXAI will be renamed SpaceXSI, following Trump's order renaming AIOctober 4, 2026
- New York City Council subpoenas Musk's SpaceXAI after it ignored an AI safety hearing inviteSeptember 28, 2026
- Musk is "cautiously optimistic" SpaceX will have a Fable/GPT-6 level model in 2 to 3 monthsSeptember 24, 2026
