“In their new proof, the pair show that overparameterization is necessary for a network to be robust.”
Classical math said networks only need to be so big. Practice kept ignoring it and winning. Now the theory has caught up, and the answer to robust memorization is more parameters. That is great news for anyone who sells GPUs. Everyone else gets a bigger power bill.