Training compute growth
Maximum disclosed training compute per year on a log10(FLOP) scale. Missing values remain absent; the current year is incomplete.
Training compute growth
Total 471 · Page 13/19 · Sample synced 11/09, 03:30 UTC
| Name | Group | Reported value | Date | Notes | Source |
|---|---|---|---|---|---|
| AlphaGo Fan | DeepMind | 380,000,000,000,000,000,000 FLOP | 2015-10-01 | Unreleased | Source ↗ |
| Transformer-XL Large + Phrase Induction | Massachusetts Institute of Technology (MIT),University of Illinois Urbana-Champaign (UIUC) | 378,486,510,000,000,000,000 FLOP | 2019-06-04 | Unreleased | Source ↗ |
| Transformer-XL (257M) | Carnegie Mellon University (CMU),Google Brain | 378,327,710,000,000,000,000 FLOP | 2019-01-09 | Open weights (unrestricted) | Source ↗ |
| DDPM-IP (CelebA) | Utrecht University | 350,000,000,000,000,000,000 FLOP | 2023-01-27 | Open weights (unrestricted) | Source ↗ |
| FourCastNet | Nvidia,NERSC, Lawrence Berkeley National Laboratory,University of Michigan,Rice University,California Institute of Technology,Purdue University | 345,047,040,000,000,000,000 FLOP | 2022-02-22 | Unreleased | Source ↗ |
| AlphaGo Master | DeepMind | 341,000,000,010,000,000,000 FLOP | 2017-10-19 | Unreleased | Source ↗ |
| SNM-skip | 297,600,000,001,000,000,000 FLOP | 2014-12-03 | — | Source ↗ | |
| BERT-Large | 285,000,000,000,000,000,000 FLOP | 2018-10-11 | Open weights (unrestricted) | Source ↗ | |
| CPM-Large | Tsinghua University,Beijing Academy of Artificial Intelligence / BAAI | 260,520,000,000,000,000,000 FLOP | 2020-12-01 | Open weights (unrestricted) | Source ↗ |
| MuseNet | OpenAI | 220,830,105,600,000,000,000 FLOP | 2019-04-25 | Unreleased | Source ↗ |
| DreamerV3 | DeepMind,University of Toronto | 220,320,000,000,000,000,000 FLOP | 2023-01-10 | Unreleased | Source ↗ |
| Discriminator Guidance | Korea Advanced Institute of Science and Technology (KAIST),NAVER | 215,700,000,010,000,000,000 FLOP | 2022-11-28 | Open weights (non-commercial) | Source ↗ |
| RoseTTAFold All-Atom (RFAA) | University of Washington,Seoul National University,University of Sheffield | 214,000,000,000,000,000,000 FLOP | 2023-10-09 | Open weights (unrestricted) | Source ↗ |
| ERNIE-GEN (large) | Baidu | 200,000,000,000,000,000,000 FLOP | 2020-08-06 | Open weights (non-commercial) | Source ↗ |
| InstructBLIP | Salesforce Research,Hong Kong University of Science and Technology (HKUST),Nanyang Technological University | 194,000,000,000,000,000,000 FLOP | 2023-05-11 | Open weights (non-commercial) | Source ↗ |
| ONE-PEACE | Alibaba,Huazhong University of Science and Technology | 180,000,000,000,000,000,000 FLOP | 2023-05-18 | Open weights (unrestricted) | Source ↗ |
| IMPALA | DeepMind | 168,000,000,000,000,000,000 FLOP | 2018-02-05 | Unreleased | Source ↗ |
| Mesh-TensorFlow Transformer 4.9B (language) | Google Brain | 161,740,800,000,000,000,000 FLOP | 2018-11-05 | Unreleased | Source ↗ |
| Contriever | Meta AI,University College London (UCL),PSL University,Université Grenoble Alpes | 157,000,000,000,000,000,000 FLOP | 2021-12-16 | Open weights (non-commercial) | Source ↗ |
| BERT-Large-CAS (PTB+WT2+WT103) | Amazon | 154,050,000,000,000,000,000 FLOP | 2019-04-20 | Unreleased | Source ↗ |
| BERT-RBP | Waseda University | 140,000,000,000,000,000,000 FLOP | 2022-04-07 | Open weights (non-commercial) | Source ↗ |
| Fusion in Encoder | Samsung | 130,000,000,000,000,000,000 FLOP | 2022-11-18 | Unreleased | Source ↗ |
| DNABERT | Northeastern University | 107,000,000,000,000,000,000 FLOP | 2021-08-15 | Open weights (unrestricted) | Source ↗ |
| AlphaZero | DeepMind | 106,000,000,010,000,000,000 FLOP | 2017-12-05 | Unreleased | Source ↗ |
| AlphaFold | DeepMind | 100,000,000,000,000,000,000 FLOP | 2020-01-15 | Unreleased | Source ↗ |
Data: Epoch AI · CC BY · View source · Automatically synced; not an exhaustive global sample