@karlovnv from Tarantool followed, sharing insights on how his team is pushing the limits of big data. @karlovnv showcased their work on real massive datasets, such as handling 3,000-column dataset, processing 70TB of data in RAM and doing these things really really fast (quicker than 10ms for a fraud detection use case!?). His talk demonstrating how DataFusion plays a key role in enabling these high-performance workloads.
Huge thanks to Sam from Synnada for this description: https://github.com/apache/datafusion/discussions/11431#discussioncomment-10832070