In this exercise, you will use the Quarkus Kafka extension to build a streaming application using MicroProfile Reative Streams Messaging and Apache Kafka, a distributed streaming platform.
What is Apache Kafka?
Apache Kafka is a distributed streaming platform. A streaming platform has three key capabilities:
- Publish and subscribe to streams of records, similar to a message queue or enterprise messaging system.
- Store streams of records in a fault-tolerant durable way.
- Process streams of records as they occur.
Kafka is generally used for two broad classes of applications:
- Building real-time streaming data pipelines that reliably get data between systems or applications
- Building real-time streaming applications that transform or react to the streams of data
Learn more at quarkus.io, or just drive on and get hands-on!
This guide has shown how you can interact with Kafka using Quarkus. It utilizes MicroProfile Reactive Messaging to build data streaming applications.
If you want to go further check the documentation of SmallRye Reactive Messaging, the implementation used in Quarkus.
If you’re interested in helping continue to improve Quarkus, developing third-party extensions, using Quarkus to develop applications, or if you’re just curious about it, please check out these links:
Reactive Streams with Quarkus and Kafka
Import the code
Let's refresh the code we'll be using. Run the following command to clone the sample project:
cd /root/projects && rm -rf rhoar-getting-started && git clone https://github.com/openshift-katacoda/rhoar-getting-started
Inspect Java runtime
An appropriate Java runtime has been installed for you. Ensure you can use it by running this command:
The command should report the version in use, for example (the versions and dates may be slightly different than the below example):
openjdk 11.0.10 2021-01-19 OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.10+9) OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.10+9, mixed mode)
If the command fails, wait a few moments and try again (it is installed in a background process and make take a few moments depending on system load).
You start with a basic Maven-based application with the usual
pom.xml entries for a Quarkus app.
We've also included a frontend HTML file at
src/main/resources/META-INF/resources/index.html that will render our stream.
The Application You Will Build
The app consists of 3 components that pass messages via Kafka and an in-memory stream, then uses SSE to push messages to the browser. It looks like:
Like other exercises, we’ll need another extension to integrate Quarkus with Kafka. Install it by clicking on the following command:
cd /root/projects/rhoar-getting-started/quarkus/kafka &&
mvn quarkus:add-extension -Dextensions="smallrye-reactive-messaging-kafka"
The first time you add the extension, new dependencies may be downloaded via maven. This should only happen once, after that things will go even faster.
This will add the necessary entries in your
pom.xml to bring in the Kafka extension. You'll see:
<dependency> <groupId>io.quarkus</groupId> <artifactId>quarkus-smallrye-reactive-messaging-kafka</artifactId> </dependency>