DoorDash: Building Scalable Real Time Event Processing with Kafka and Flink

Sep 20, 12:00 PM PDT
  • Virtual SF Big Analytics
  • 217 RSVP

Two years ago, Doordash started the journey of creating a real time event processing system to replace the legacy data pipelines and address our event processing needs to scale our business. We created a scalable system that could handle heterogeneous data sources and destinations, was easily accessible with different levels of abstractions and had end to end schema enforcement. We were able to accomplish this by shifting our strategy from heavily relying on AWS and third-party data services to leveraging open source frameworks that can be customized and better integrated with our infrastructure.

In this session, I will share what we learned and how we put together this system with Apache Flink, Kafka as well as Kubernetes.

Allen Wang (Doordash)

Allen Wang is the lead engineer at Doordash working on real-time data infrastructure. Before that, Allen worked at Netflix for more than 8 years working as Architect of Real-time data Infrastructure.
The event ended.
Watch Recording
*Recordings hosted on Youtube, click the link will open the Youtube page.
Contact Organizer