Running Native GGUF Models on Jailbroken PS5 Hardware
Transforming gaming consoles into private, local model servers using llama.cpp and Vulkan.
1Console-Based LLM Infrastructure
The PS5LM project enables native execution of GGUF language models on jailbroken PlayStation 5 consoles. By leveraging the console's RDNA 2 GPU via the Vulkan backend, users can host private model servers without relying on cloud infrastructure or external PCs. It serves requests locally to any device on the network.
2Technical Stack and Integration
Built on Linux, the application utilizes llama.cpp and ggml-vulkan to manage model inference. It exposes standard OpenAI and Anthropic API endpoints, allowing integration with tools like Claude Code. The system supports various quantization levels, utilizing 11.4 GB of shared memory to handle complex model contexts and diverse architectural requirements.
3Decision Model Capabilities
Beyond standard text generation, the platform supports decision models powered by Ollaya. These models process typed decisions with calibrated probabilities using the /v1/systemone API. By performing a single forward pass, the system evaluates states and answers specific questions about text or images in a fraction of a second.
4Performance and Throughput Metrics
Performance varies by model size and quantization on a PS5 Slim. For instance, Qwen 3.5 0.8B achieves 207 tokens per second, while the 9B version reaches 55 tokens per second. Larger models like Qwen 3.8 27B, using UD-Q2_K_XL quantization, sustain approximately 20 tokens per second with a 64k context window.
5User Interface and Monitoring
The interface provides real-time monitoring of hardware health, including CPU and GPU load, temperature, and power consumption over time. Users can interact with the console via a TV-based dashboard or standard browser clients. Square and Triangle controller inputs allow for demo triggering and keyboard access directly from the couch.
6Model Management and Storage
The built-in planner automatically optimizes cache and context allocation based on available memory for each GGUF file. Users can download models directly from Hugging Face to internal storage, M.2 drives, or USB devices. The system maintains a suggested list of models that have been pre-measured for console compatibility.
7System Requirements and Compatibility
Installation requires a jailbroken PS5 running firmware versions between 7.00 and 13.60. Necessary payloads must be loaded to enable the custom environment. All user settings, downloaded models, and configuration data are stored in the /data/PS5LM directory, ensuring persistence across updates when the folder is replaced with new versions.
8Open Source Foundation
The project integrates contributions from various developers, including PS5_Vulkan, RADV drivers, and the ps5-homebrew-ui framework. It operates under the GPL-3.0 license, strictly excluding Sony SDK files or proprietary firmware. This architecture demonstrates the feasibility of repurposing high-performance gaming hardware for specialized local AI inference tasks and private server operations.
Original article: Hacker News — Running Native GGUF Models on Jailbroken PS5 Hardware