In-Stream Event Processing is a new approach for building near real time big data systems with rapidly growing user base and applications like clickstream analytics, preventive maintenance or fraud detection. Maturity of some open source projects enables building an enterprise grade In-Stream Processing service in-house. However the open source world comprises of many competing projects of different maturity, having different perspectives so the task to select effective and efficient projects is not straightforward. In the talk I’ll present a blueprint of an In-Stream Processing Service, enterprise grade reliable and scalable, cloud ready, build from 100% open source components.

What will you learn?

• Conceptual architecture for In-Stream Processing.
• A reference architecture based on Apache Kafka, Spark Streaming, Cassandra and Redis.
• Operational configuration aspects to ensure scalability and reliability for every component of the architecture.

Target audience: developers, architects and technical managers who are working on real time big data projects, considering them in near future or just curious about them.

Anton Ovchinnikov

Business Analyst, Grid Dynamics

Graduated from Saint-Petersburg State Technical University.

During 17 years in profession has gained an experience of software developer/architect as well as project manager and business consultant. Participated in project as a vendor as well as a customer.

Worked at local and international companies in a variety of industries: telecommunications, investment banking, consulting, electronic retail.