Time Series Data and Stream Processing Performance Tuning


Nov 10, 04:00PM PST(12:00AM GMT).
  • Virtual | San Francisco SF Big Analytics
  • Free 70 Attendees
Description
Speaker

This is hybrid event. You can join in two options:
- In-person: the venue is: LaunchDarkly, 1999 Harrison, Oakland, CA. (right between the 19th St BART station and Lake Merritt). with food/drink provided.
- Remotely: on zoom. RSVP below to receive your joining link

Agenda (PST time zone):
- 4:00 - 5:00pm: Intro
- 5:00 - 5:45pm: Tech talk 1 + QA
- 5:45 - 6:30pm: Tech talk 2 + QA
- 6:30pm: Close

Talk 1: Time series is all hype: don’t do it

Time series data is a hot topic. In this talk we’ll showcase a workload poorly suited to time series data (experimentation) that LaunchDarkly managed to implement as time series data. We’ll dive into the performance problems that caused and take you on the journey we took to discover a more performant schema for our workload. We (still) use postgresql for this workload, so this talk will discuss postgres indexing strategies and query execution.

Talk 2: Performance tuning applications within stream processing frameworks (Flink)

When processing vast quantities of data, special attention needs to be paid to both the organization of applications in a stream processing framework and the mechanics of the processing. Key elements toward this end include hot shard mitigation, data aggregation, and event time skew handling. Profiling tools can inform whether experimental changes are yielding the desired improvements. This talk will cover some learning on each of these topics in the context of experiences with Apache Flink.

Mike Zorn

Mike Zorn
Mike has been at LaunchDarkly since 2017, in that time, he’s helped LaunchDarkly’s infrastructure scale from a hundred million feature flag evaluations a day to the tens of trillions of evaluations that are served nowadays

Blake Wilson
Blake spent the past four years at LaunchDarkly working on the service’s many data pipelines. He has helped introduce Flink into LaunchDarkly where it has proliferated in use

The event ended.
Watch Recording
*Recordings hosted on Youtube, click the link will open the Youtube page.