Smealum, Derrek, and Plutoo had a Keynote at the Chaos Communication Congress (32C3), and the recording of the video is now online (embedded below)
In the talk, the 3 hackers explain how they broke the security of the Nintendo 3DS, which led to a lively 3DS Homebrew scene. They first describe an overview of the system (specifically details on the ARM11, and ARM9, the security CPU).
They then explain how they breach through the 4 levels of security (ARM11 Userland, ARM11 Kernel, ARM9 Userland, ARM9 Kernel), and how they involved the GPU to get access to the RAM. An interesting anecdote from Smealum is that in practice, the ARM9 Kernel has an unintentional syscall backdoor. One can feed it any operation pointer and it will run in Kernel Mode. ARM11 doesn’t have direct access to it, but anything in ARM9 can access it, meaning once a hacker gets Userland ARM9 access, it’s equivalent to getting Kernel access to that CPU. This makes the last layer of security pretty much moot.
The hackers added a few tongue-in-cheek pieces of advice for Nintendo and other console manufacturers, in particular “Secrets hidden in hardware are great, unless you leak them”, in reference to how they managed to extract encryption keys shared by the Wii U and the 3DS.
There’s alot being explained and I won’t summarize it all here. You can see the full presentation below. If you have interest in console security and hacking ( and if the words ROP, Webkit, NX don’t scare you), it’s a must see!
Note: the presentation actually starts 15 minutes into the video.
One important point from Smealum is that he believes the 3DS homebrew scene is lively and growing. He emphasized his disagreement with Fail0verflow’s statement a few years ago that console homebrew is dead. He showcased a cool screenshot if existing 3DS homebrew.