OPT-175B

Meta · May 2022

The open GPT-3 replica whose training logbook — crashes, loss spikes, hardware failures, 2am restarts — is the closest thing we have to a model's childhood diary.

Sources

Official

• 2022-05 OPT paper — the model.
• 2022 metaseq logbook — the diary — chronicles of training.

Writing & commentary

tk — to collect

Tweets

tk — fleet pass pending; mirror to archive

Official record

tk — release date, specs, benchmarks, checkpoints

History

tk — context of the world at release; events and entanglements

Impressions

tk — day-of and longitudinal vibes to collect

← back to the Pantheon