↓ Skip to main content

“Pandas alone consumes ~37M of memory: Just importing pandas, before even doing anything with it, consumes a significant amount of memory.”

Haki Benitahakibenita.com ↗

Data teams pull a million rows out of Postgres just to count them in Python. The database could do it in place, faster, with almost no memory. Pandas is the default because it’s what the bootcamp taught. SQL has been doing groupby since the 1970s.