Overview

KafkaESK, an open-source product currently in Beta, is an event-driven monitoring tool that can consume messages from Apache Kafka clusters and display the aggregated data on a dashboard for analysis and maintenance. Used in a pipeline that includes Apache Kafka Connect (for writing sources and sinks that either ingests the entire database and stream table updates to Kafka topics or continuously delivers data from the topics into external systems) and ksqlDB (for stream processing that enables executing continuous computations over an unbounded stream of events), KafkaESK can incrementally update in real-time as events arrive. This powerful tool can be used to digest live data from IoT/smart sensor technology, machine performance, and even website activity such as clickstreams. Project Roadmap: • User ability to dynamically customize data charts directly on the dashboard • Integrating a terminal into the dashboard GUI that will allow interaction with the ksqlDB CLI to: - craft materialized views over streams - receive real-time push updates - pull current state on demand - transform, filter, aggregate, and join collections - push and pull queries We are happy to have contributions, whether for trivial cleanups or big new features! (https://github.com/oslabs-beta/kafkaESK)