What's the lifespan of an LLM going to be in the next few years? Seems like at the current pace, cutting edge models will become obsolete pretty quickly. Since model training is very expensive, this means the LLM space has some parallels with the pharmaceutical industry (massive upfront capital costs, cheap marginal costs relative to value produced).
I find it quite fascinating how quickly machine learning has changed in this regard.
Deep Learning training was always very expensive but models werent getting such a massive bump in size every year (for state of the art) and now they are just getting 10x bigger every iteration but AI accelerators / GPUs are getting like 1.5x jump every 2 years so have fun for future AI academia / startups outside US.
I find it quite fascinating how quickly machine learning has changed in this regard.