Perplexity and Nvidia Announce Fully Local, Zero-Cost AI Agent
Portable Computer shifts AI tasks from the cloud to users' own devices
1From Cloud to Local
Perplexity launched Portable Computer, a fully local version of its AI assistant Computer. Developed in partnership with Nvidia, the system runs on users' Nvidia DGX Spark supercomputers and Linux machines with RTX GPUs. The model, user files, and all processing remain on the device without cloud transmission. This eliminates token costs and enhances privacy.
2Nvidia's Strategic Move
After focusing on trillion-dollar AI data centers for two years, Nvidia signals that local AI has moved beyond hobbyist level. Nader, Director of Developer Technologies, stated, 'Open source models have made local AI practical.' Nvidia aims to boost hardware sales in this space. RTX 3090 or newer GPUs with 24GB VRAM are the minimum requirement.
3Fully Integrated Local Solution
Portable Computer delivers AI models, inference engine, tools, and app connectors in one package. Users avoid complex steps like downloading model weights or setting up inference servers. Nate, VP of Engineering at Perplexity, emphasized ease of use: 'Historically, local AI setup was very painful.' Currently available for Linux, Windows support coming in September 2024.
4Financial Analysis Demo
In one demonstration, the 27-billion-parameter Qwen model analyzed 1099 forms and investment documents at full GPU utilization on DGX Spark. The system identified instances where the hypothetical investor paid unnecessary fees. All processing occurred on-device, consuming zero cloud credits. Nate noted, 'The interface element that normally shows cloud credits remained at zero.'
5Hybrid Capability
While primarily local, the system can integrate with cloud when needed. Another demo showed locally analyzed user funnel data being pushed to Slack via Perplexity's connector ecosystem. The system also connects to Google Drive, Gmail, and GitHub, and can escalate to more powerful cloud models when local resources are insufficient.
6Supported Models
At launch, users can set up Qwen 3.8 27B or Perplexity's custom-trained PPLX 27B models. Nvidia's Nemotron 3.5 Lightning will be added soon. Currently available for Pro, Max, Enterprise Pro, and Enterprise Max subscribers on Linux, with Windows support arriving September 2024.
7Minimum Hardware Requirements
The system requires at least a 24GB VRAM RTX GPU - roughly equivalent to RTX 3090 or newer. Nate explained, 'We set this as the minimum threshold to ensure great experience while maintaining broad availability.' Most users already meet these requirements.
8Co-Design of Model and Harness
Perplexity argues that effective local agents require the model and its harness (prompts, tools, orchestration logic) to be designed together. General-purpose harnesses assume models with large context windows. However, models like Qwen 3.8 27B struggle beyond 100k tokens despite advertising 260k token capacity.
9Optimized System Architecture
The company developed a minimal harness: concise system prompts, a small core toolset, and on-demand capabilities. Popular connectors like Gmail and GitHub were converted from token-heavy MCP servers to compact CLI tools. The system includes self-verification hooks for task health monitoring.
10Security Measures
Portable Computer enforces always-on OS-level sandboxing. If sandboxing is unavailable, the harness disables itself - contrasting with open-source tools that run with full user permissions by default. Perplexity's internal benchmarks show impressive results across 53 tasks including deep research, financial analysis, and document creation.
11The Future of Local AI
This collaboration between Perplexity and Nvidia marks a significant step in reducing AI's cloud dependency. With growing privacy concerns, cost savings, and decreasing hardware requirements, local AI solutions will see wider adoption. Particularly crucial for sectors handling sensitive data like finance and healthcare, this technology will play a pivotal role in coming years.