NYC Tech Week - Search, Streams, and System Design Tradeoffs

Hello everyone! Join us for a joint meetup with Elastic on June 25th from 5:30pm! Agenda: 5:30 pm: Doors open; say hi and eat some food 6:00 pm: “12 Levels of Search — Easy to Complex”, by Jon Avezbaki, Developer Advocate at Elastic 6:30 pm: Q&A with Jon 6:40 pm: "One Does Not Simply Query a Stream" , by Viktor Gamov, Principal Developer Advocate at Confluent 7:10 pm: Q&A with Viktor 7:20 - 7:30 pm: Networking and event wrap-up "One Does Not Simply Query a Stream" Streaming data with Apache Kafka® has become the backbone of modern applications. While streams are ideal for continuous data flow, they lack built-in querying capabilities. Unlike databases with indexed lookups, Kafka’s append-only logs are designed for high-throughput processing—not for on-demand queries. This necessitates additional infrastructure to query streaming data effectively. Traditional approaches replicate stream data into external stores: relational databases like PostgreSQL for operational queries, object storage like S3 accessed via Flink, Spark, or Trino for analytics, and Elasticsearch for full-text…