Serving tech enthusiasts for over 25 years.
TechSpot capability tech analysis and advice you can belief.
A hot potato: The initiating-provide project llama2.c is designed to bustle a gentle-weight model of the Llama 2 model completely in C code. This “baby” Llama 2 model is impressed by llama.cpp, a project created to enable LLM inference all over a large quantity of hardware, from native devices to cloud-primarily based completely platforms. These compact code experiments are now being leveraged to bustle AI skills on with regards to any tool with a chip, highlighting the rising accessibility and flexibility of AI instruments.
After seeing Exo Labs bustle a good language model on an outmoded Pentius II working Home windows 98, developer Andrei David decided to take on an even more unconventional space. Dusting off his Xbox 360 console, he space out to power the with regards to 2-decade-dilapidated machine to load an AI model from Meta AI’s Llama family of LLMs.
David shared on X that he successfully ported llama2.c to Microsoft’s 2005-generation gaming console. Alternatively, the technique wasn’t without vital hurdles. The Xbox 360’s PowerPC CPU is a huge-endian structure, which required wide endianness conversion for both the model’s configuration and weights. Moreover, he had to handle good adjustments and optimizations to the conventional code to form it work on the increasing older hardware.
LLM working on Xbox 360
With Xenon CPU (3.2GHz PowerPC w/ 3 cores) and 512MB unified RAM. Consistent with @karpathy‘s🩷llama2.c, ported to bustle on Microsoft’s grand console from 2005. Pure C implementation optimized for PowerPC structure and Xbox memory management.
Impressed by… pic.twitter.com/e9oMLaWIyi
– Andrei David (@AndreiDavid)”https://twitter.com/AndreiDavid/status/1877792408991121842?ref_src=twsrc%5Etfw”>January 10, 2025
Memory management posed but but any other vital space. The 60MB llama2 model needed to be carefully structured to fit within the Xbox 360’s unified memory structure, the put the CPU and GPU part the identical pool of RAM. In response to David, the Xbox 360’s memory structure used to be remarkably forward-thinking for its time, foreshadowing the memory management solutions now long-established in standard gaming consoles and APUs.
After wide coding and optimization, David successfully ran llama2 on his Xbox 360 the usage of a straightforward instructed: “Sleep Joe said.” Despite the llama2 model being upright 700 lines of C code without a external dependencies, David current that it could perchance well elevate “surprisingly” sturdy efficiency when tailored to a sufficiently slim domain.
David outlined that working within the constraints of a restricted platform relish the Xbox 360 forces you to prioritize efficient memory usage above all else. In response, but any other X user instructed that the 512MB of memory on Microsoft’s dilapidated console could also very effectively be ample to bustle other tiny LLM implementations, similar to smolLM, created by AI startup Hugging Face.
The developer gladly accredited the distance, so we’re going to rep a method to likely look additional LLM experiments on Xbox 360 within the now not-so-far-off future.
