@KeyTryer I also think scaling is a good idea but I think it's hard to get right. In addition to pretraining being expensive, any kind of iteration on them i.e. during posttraining is more expensive/slow, and I think they tend to behave more weirdly & take much more skill to posttrain well