1. Science and space

   AI: 7 million times more training compute in 12 years

   AlexNet (2012) vs DeepSeek-V3 (2024): 11.2 thousand times the parameters, 7.02 million times the training compute. Different tasks; this compares scale, not quality.

   Source: Epoch AI · model authors’ publicationsOpen on the chart →

   More

   Training compute increased much more than parameter count for this pair. AlexNet classifies images; DeepSeek-V3 is a language model.

   One logarithmic scale of multiples of the 2012 value. Equal vertical distances mean equal ratios. Only the published points are shown; values between them were not measured. This compares scale, not performance.

   • AlexNet: parameters parameters

     60,000,000 · 2012-09-30

     Epoch AI

   • AlexNet: training compute FLOP

     4.700e+17 · 2012-09-30

     Epoch AI

   • DeepSeek-V3: parameters parameters

     671,000,000,000 · 2024-12-24

     Epoch AI

   • DeepSeek-V3: training compute FLOP

     3.300e+24 · 2024-12-24

     Epoch AI

   Four central Epoch AI estimates marked Confident. DeepSeek-V3 total parameters and training FLOP divided by AlexNet values; bounds are not substituted.

   Confident is the source’s confidence label, rather than an exact physical measurement. DeepSeek-V3 uses MoE: 671 billion total parameters, not all active per token. Parameters, compute and answer quality are not interchangeable. Two models do not describe all AI progress.

   A question for this picture

   How do compute, cost and quality change within one task and architecture?

More findings

• Almost 1,700 Earths in volume. Just 25 in mass.
• 55 Cancri e: a year shorter than an Earth day
• 74% of known exoplanets were found by the dimming of their star
• TRAPPIST-1: seven planets, and the longest year there is 19 Earth days
• China spends as much on research as the US, yet has half as many patents filed in the US, Europe and Japan at once
• The Chelyabinsk fireball was 9 times as energetic as the next largest in NASA’s catalogue

All findings →