“The (non-expert) users who would most benefit from an AI search summary will be the ones most likely misled by it.”
Google’s AI summary described an IBM PS/2 Model 280 that never existed. Every rerun invented different specs, including a 286 expandable to 128 MB. The 286 can address 16 MB. The right answer showed up maybe one time in ten, and Google puts this above the search results.