Open models support local AI and distributed GPU networks

Summary

Open-weight models (Llama, Qwen, DeepSeek, Mistral, Gemma) enable local, cloud-free generative AI on consumer hardware. Meanwhile, ventures like Far Labs and Evolving Edge are attempting to monetize idle gaming GPUs by converting them into distributed inference networks. The article emphasizes that these peer-to-peer GPU markets have not yet been proven to be economically or securely scalable.

Why it matters

Edge AI execution eliminates per-token costs and keeps data on-device, offering genuine privacy and cost advantages for individuals and small businesses. If the distributed GPU model works, it could dramatically lower inference costs by utilizing millions of idle consumer cards. However, the article notes that electricity bills, platform commissions, and unverified latency claims still leave the business model uncertain.

Highlights

• Salad's 2021 history: It generated $3.6 million in revenue but paid GPU owners only about $500,000 (roughly 14 cents per dollar). This serves as a cautionary benchmark for Far Labs and Evolving Edge.
• In Spain (approx. €0.16/kWh), a single RTX 4090 running 24/7 can generate an electricity bill exceeding €40 per month. Therefore, hourly rental fees must exceed this floor just to avoid losses.
• The Pearl network incident—where 320,000 GPU-equivalents were spent on random matrix operations without producing useful AI output—serves as a concrete warning that 'distributed inference' might be a smokescreen.

Critical take

The article presents Far Labs' sub-100 ms latency promise and Evolving Edge's open-source node code as differentiating factors, yet acknowledges that neither has been verified at scale. It also highlights that open-source code does not guarantee that the installed binary matches the audited source. The text reads more like a trend summary than an independent assessment. Beyond Salad's 2021 data, there is no real user revenue data or concrete threat model analysis regarding sandboxing claims. While exciting, the article provides little evidence that the model is economically or technically robust.