“The memory wall in AI has mostly been solved with maths, not silicon.”
Memory per token of context has dropped 100x since 2017 while GPU memory only grew 18x. The math did the heavy lifting, not Nvidia. None of those gains showed up as lower prices. The labs spent every bit of it on longer contexts and agent swarms, and they will keep doing exactly that.