China AI DataTokenHeart News ↗
← Back to dashboardDATA DETAIL / TRAINING

Training compute growth

Maximum disclosed training compute per year on a log10(FLOP) scale. Missing values remain absent; the current year is incomplete.

Training compute growth

Training compute growth2012-01-01: 17.78log10(FLOP); 2013-01-01: 18.42log10(FLOP); 2014-01-01: 20.47log10(FLOP); 2015-01-01: 20.58log10(FLOP); 2016-01-01: 21.82log10(FLOP); 2017-01-01: 20.93log10(FLOP); 2018-01-01: 21.94log10(FLOP); 2019-01-01: 23.03log10(FLOP); 2020-01-01: 23.50log10(FLOP); 2021-01-01: 24.31log10(FLOP); 2022-01-01: 24.44log10(FLOP); 2023-01-01: 25.70log10(FLOP); 2024-01-01: 25.58log10(FLOP); 2025-01-01: 26.70log10(FLOP); 2026-01-01: 27.00log10(FLOP)013.5272012-01-01: 17.78 log10(FLOP)2013-01-01: 18.42 log10(FLOP)2014-01-01: 20.47 log10(FLOP)2015-01-01: 20.58 log10(FLOP)2016-01-01: 21.82 log10(FLOP)2017-01-01: 20.93 log10(FLOP)2018-01-01: 21.94 log10(FLOP)2019-01-01: 23.03 log10(FLOP)2020-01-01: 23.50 log10(FLOP)2021-01-01: 24.31 log10(FLOP)2022-01-01: 24.44 log10(FLOP)2023-01-01: 25.70 log10(FLOP)2024-01-01: 25.58 log10(FLOP)2025-01-01: 26.70 log10(FLOP)2026-01-01: 27.00 log10(FLOP)01/0101/01 UTC

Total 468 · Page 11/19 · Observed 9/25/2026, 3:30:45 AM

NameGroupReported valueDateNotesSource
GatoDeepMind4,020,000,000,000,000,000,000 FLOP2022-05-12UnreleasedSource ↗
wave2vec 2.0 LARGEFacebook3,870,720,000,000,000,000,000 FLOP2020-10-22Open weights (unrestricted)Source ↗
ViT-G (model soup)University of Washington,Columbia University,Google,Meta AI,Tel Aviv University3,400,000,000,000,000,000,000 FLOP2022-03-10Open weights (non-commercial)Source ↗
ELECTRAStanford University,Google,Google Brain3,100,000,000,000,000,000,000 FLOP2020-03-23Open weights (unrestricted)Source ↗
Incoder-6.7BFacebook AI Research,University of Washington,University of California (UC) Berkeley,Carnegie Mellon University (CMU),Toyota Technological Institute at Chicago3,000,010,000,000,000,000,000 FLOP2023-04-09Open weights (non-commercial)Source ↗
AlphaFold 2DeepMind2,990,000,000,000,000,000,000 FLOP2020-11-30Open weights (unrestricted)Source ↗
CodeT5-largeSalesforce2,720,000,000,000,000,000,000 FLOP2022-07-05Open weights (unrestricted)Source ↗
StyleGAN3-RNvidia,Aalto University2,427,840,000,000,000,000,000 FLOP2021-06-21Open weights (non-commercial)Source ↗
InternImageShanghai AI Lab,Tsinghua University,Nanjing University,SenseTime,Chinese University of Hong Kong (CUHK)2,408,000,000,000,000,000,000 FLOP2022-11-10Open weights (unrestricted)Source ↗
ALBERT-xxlargeToyota Technological Institute at Chicago,Google2,390,000,000,000,000,000,000 FLOP2020-02-09Open weights (unrestricted)Source ↗
VILA1.5-13BNvidia,Massachusetts Institute of Technology (MIT)2,300,313,600,000,000,000,000 FLOP2024-05-03Open weights (non-commercial)Source ↗
VILA-13BNvidia,Massachusetts Institute of Technology (MIT)2,300,313,600,000,000,000,000 FLOP2023-12-12Open weights (non-commercial)Source ↗
NASv3 (CIFAR-10)Google Brain2,200,000,000,000,000,000,000 FLOP2016-11-05—Source ↗
GPT-2 (1.5B)OpenAI1,920,000,000,001,000,000,000 FLOP2019-02-14Open weights (unrestricted)Source ↗
EMDRMila - Quebec AI (originally Montreal Institute for Learning Algorithms),McGill University,DeepMind1,910,000,000,000,000,000,000 FLOP2021-06-09Open weights (unrestricted)Source ↗
AlphaGo LeeDeepMind1,900,000,000,000,000,000,000 FLOP2016-01-27UnreleasedSource ↗
HyenaDNAStanford University,Harvard University,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms),University of Montreal / Université de Montréal1,811,000,000,000,000,000,000 FLOP2023-06-27Open weights (restricted use)Source ↗
StyleGAN3-TNvidia,Aalto University1,702,080,000,000,000,000,000 FLOP2021-06-21Open weights (non-commercial)Source ↗
KEPLERTsinghua University,Mila - Quebec AI (originally Montreal Institute for Learning Algorithms),HEC,CIFAR AI Research,Princeton University,University of Montreal / Université de Montréal1,660,000,000,000,000,000,000 FLOP2020-11-23UnreleasedSource ↗
CodeT5-baseSalesforce,Nanyang Technological University1,560,000,000,000,000,000,000 FLOP2021-11-01Open weights (unrestricted)Source ↗
German ELECTRA Largedeepset,Bayerische Staatsbibliothek Muenchen1,428,295,680,000,000,000,000 FLOP2020-10-21Open weights (unrestricted)Source ↗
GenSLMUniversity of Chicago,Nvidia,Harvard University,Cerebras Systems,Technical University of Munich,California Institute of Technology1,420,000,000,000,000,000,000 FLOP2022-10-11Open weights (unrestricted)Source ↗
BLIP-2 (Q-Former)Salesforce Research1,200,000,000,010,000,000,000 FLOP2023-01-30Open weights (unrestricted)Source ↗
Megatron-LM (1.2B)Nvidia1,130,400,000,000,000,000,000 FLOP2019-09-17UnreleasedSource ↗
PolyCoderCarnegie Mellon University (CMU)1,100,000,000,000,000,000,000 FLOP2022-02-26Open weights (unrestricted)Source ↗

Data: Epoch AI · CC BY · View source · Automatically synced; not an exhaustive global sample