So something that costs like we said, OpenAI spent $400 million training models for for GPT four. If they spent $400 million in the last couple of years, you could probably assume that doing the same training exercise could be done for 5 to $10 million. 18 months from now to generate the same model. That's a, you know, 100 x cost reduction.View on YouTube
Evidence as of late 2024–2025 indicates that at least one lab did in fact train a GPT‑4‑class model for roughly the $5–10M range, within the predicted 18‑month window, even though Friedberg’s baseline and implied reduction factor were overstated.
Baseline: GPT‑4 training cost
- OpenAI has not published exact figures, but multiple analyses and Sam Altman’s own comments put GPT‑4’s total training cost (including more than just raw GPU rental) in the tens to low hundreds of millions of dollars. Altman has said it cost "more than $100 million"; independent cost models (AI Index / Epoch AI) estimate the compute portion of the final training run at about $40–80M, with total development costs near $100M. (en.wikipedia.org)
Friedberg’s $400M starting point is likely an overestimate, so his implied “100x reduction” is numerically off, but the starting cost was still extremely large.
By late 2024: a GPT‑4‑class model for ≈$5–6M
- Chinese startup DeepSeek released its V3 model with a technical report dated 27 December 2024. An in‑depth account in Communications of the ACM notes that DeepSeek’s own report claimed 2.788M H800 GPU‑hours, at about $2/hour, for a training compute cost ≈$5.6M. (cacm.acm.org)
- That same CACM piece emphasizes that this was remarkable precisely because prior frontier LLMs like GPT‑4 were believed to cost on the order of $100M+ to train, with industry expectations in the $100M–$1B range for top models. (cacm.acm.org)
- Independent benchmark comparisons generally place DeepSeek‑V3 at or near GPT‑4 / GPT‑4o level: it matches or slightly exceeds GPT‑4 on many reasoning and coding benchmarks (e.g., MMLU and HumanEval) and is described as “rivaling” or “challenging” GPT‑4o/Claude 3.5 on aggregate performance, though GPT‑4o still wins some English‑centric tasks. (datastudios.org)
That is strong evidence that a GPT‑4‑class model was trained for roughly $5–6M of compute by late 2024.
Context: frontier costs overall moved up, not down
- Broad industry data show that the cost of training frontier models has been rising, not collapsing: a 2024–25 cost study finds amortized training costs for the most compute‑intensive models growing about 2.4× per year since 2016, with GPT‑4’s final‑run compute around $40M and Google’s Gemini Ultra near $30M, and projections of $1B+ runs by 2027. (ar5iv.org)
- AI Index–based summaries and related reporting put training costs for later frontier models like Gemini Ultra, Llama 3.1‑405B, Grok‑2, and Llama 4 in the $100M–$300M+ range, and the Wall Street Journal reports GPT‑5 (Orion) training runs costing up to $500M each. (visualcapitalist.com)
So the typical frontier training run did not fall to $5–10M; if anything, it became more expensive.
Why this still counts as the prediction being essentially right
- Friedberg’s core quantitative claim was that **“18 months from now” it would be possible to generate a GPT‑4‑level model for about $5–10M of training compute, instead of the roughly hundreds of millions previously spent. DeepSeek‑V3 provides a concrete, widely‑reported example of exactly that: a GPT‑4‑class model trained for about $5.6M by December 2024.
- However, important caveats are:
- His starting figure (~$400M) overshoots the best current estimates for GPT‑4’s training cost, so the actual realized reduction is more like 7–20×, not 40–80×, depending on which baseline you use. (ar5iv.org)
- The industry‑wide trend in frontier training costs is upward; DeepSeek appears as a notable but exceptional outlier, leveraging architectural choices (Mixture‑of‑Experts, 8‑bit training, etc.) and cheaper H800 chips to hit that cost point.
Given that a GPT‑4‑class model was in fact trained within roughly the predicted timeframe for about $5–6M of compute—squarely inside Friedberg’s $5–10M band—even though his baseline and generalization about a 40–80× industry‑wide cost collapse were inaccurate, the prediction is best classified as “right” with significant nuance.