- o3 can use "tens of millions" of tokens to solve a task (@fchollet via @simonw)
- this takes 13.8 minutes
20M / (13.8 * 60) = 24k tok/s
so, i figure the search has to be parallelized in some way, unless the "tens of millions" tasks are outliers that take significantly longer https://t.co/X3LTOPvHAn https://t.co/GAPpFWVHFg
- this takes 13.8 minutes
20M / (13.8 * 60) = 24k tok/s
so, i figure the search has to be parallelized in some way, unless the "tens of millions" tasks are outliers that take significantly longer https://t.co/X3LTOPvHAn https://t.co/GAPpFWVHFg