“This enables models to operate on various modalities without any hand-designed, modality-specific processing.”
ByteFormer skips JPEG decoding and reads the raw file bytes. It lifts ImageNet top-1 from 72.2% to 77.33% over similar-size DeIT models and handles audio with the same architecture. Speech Commands accuracy lands at 95.42% against a 98.7% state of the art, and that gap sits lower in the abstract. Hand-built preprocessing pipelines are on borrowed time.