Training compute growth
Maximum disclosed training compute per year on a log10(FLOP) scale. Missing values remain absent; the current year is incomplete.
Training compute growth
Total 468 · Page 14/19 · Observed 9/25/2026, 3:30:45 AM
| Name | Group | Reported value | Date | Notes | Source |
|---|---|---|---|---|---|
| SciBERT | Allen Institute for AI | 89,268,480,000,000,000,000 FLOP | 2019-03-26 | Open weights (unrestricted) | Source ↗ |
| Adaptive Input Transformer + RD | Microsoft Research Asia,Soochow University | 85,800,000,000,000,000,000 FLOP | 2021-06-28 | Unreleased | Source ↗ |
| DeiT-B | Meta AI,Sorbonne University | 78,840,000,000,000,000,000 FLOP | 2021-01-15 | Open weights (unrestricted) | Source ↗ |
| Denoising Diffusion Probabilistic Models (LSUN Bedroom) | University of California (UC) Berkeley | 78,401,250,000,000,010,000 FLOP | 2021-06-11 | Open weights (unrestricted) | Source ↗ |
| S4 | Stanford University | 78,328,627,000,000,000,000 FLOP | 2021-10-31 | Open weights (unrestricted) | Source ↗ |
| DiffDock | Massachusetts Institute of Technology (MIT) | 72,000,000,000,000,000,000 FLOP | 2022-10-04 | Open weights (unrestricted) | Source ↗ |
| BEIT-3 | Microsoft | 70,000,000,000,000,000,000 FLOP | 2022-08-22 | Open weights (unrestricted) | Source ↗ |
| Mesh-TensorFlow Transformer 2.9B (translation) | Google Brain | 68,428,800,000,000,000,000 FLOP | 2018-11-05 | Unreleased | Source ↗ |
| ProteinBERT | Hebrew University of Jerusalem,Ben-Gurion University of the Negev,Deep Trading | 65,000,000,000,000,000,000 FLOP | 2022-02-10 | Open weights (unrestricted) | Source ↗ |
| PolyNet | Chinese University of Hong Kong (CUHK) | 64,000,000,000,000,000,000 FLOP | 2016-11-17 | — | Source ↗ |
| ConvS2S (ensemble of 8 models) | Meta AI | 56,400,000,000,000,000,000 FLOP | 2017-07-25 | — | Source ↗ |
| Seq2Seq LSTM | 56,000,000,000,000,000,000 FLOP | 2014-09-10 | — | Source ↗ | |
| eDiff-I | Nvidia | 54,600,000,000,000,000,000 FLOP | 2022-11-02 | Unreleased | Source ↗ |
| base LM+GNN+kNN | Shannon.AI,Nanjing University,Nanyang Technological University,Zhejiang University (ZJU) | 52,587,456,000,000,000,000 FLOP | 2021-10-17 | Open weights (unrestricted) | Source ↗ |
| MuZero | DeepMind | 48,000,000,000,000,000,000 FLOP | 2019-11-19 | Unreleased | Source ↗ |
| Transformer (Adaptive Input Embeddings) WT103 | Facebook AI Research | 44,700,000,000,000,000,000 FLOP | 2018-09-28 | Open weights (unrestricted) | Source ↗ |
| trRosetta | Nankai University,University of Washington,Tianjin University,Harvard University | 38,047,968,000,000,000,000 FLOP | 2019-08-22 | Open weights (unrestricted) | Source ↗ |
| FTW (For The Win) | DeepMind | 34,900,000,000,000,000,000 FLOP | 2018-07-03 | Unreleased | Source ↗ |
| FFN SwiGLU | 33,687,317,000,000,000,000 FLOP | 2020-02-14 | Unreleased | Source ↗ | |
| Base LM + kNN LM + Continuous Cache | Stanford University,Facebook AI Research | 30,529,200,000,000,000,000 FLOP | 2019-11-01 | Unreleased | Source ↗ |
| ERNIE-Doc (247M) | Baidu | 30,302,798,000,000,000,000 FLOP | 2020-12-31 | Open weights (unrestricted) | Source ↗ |
| ResNet-200 | Microsoft Research Asia | 29,741,645,000,000,000,000 FLOP | 2016-09-17 | Unreleased | Source ↗ |
| CaLM | University of Oxford | 29,000,000,000,000,000,000 FLOP | 2022-12-19 | Unreleased | Source ↗ |
| TaLK Convolution | Carleton University | 26,990,346,000,000,000,000 FLOP | 2020-02-08 | Unreleased | Source ↗ |
| Segatron-XL large, M=384 + HCP | Microsoft Research,University of Waterloo | 26,500,000,000,000,000,000 FLOP | 2022-03-21 | Unreleased | Source ↗ |
Data: Epoch AI · CC BY · View source · Automatically synced; not an exhaustive global sample