Coming from Apache Kafka Series -
Learn Apache Kafka for Beginners v3?
Keep Reading! This is important as we are removing Conduktor Playground and you need to change one thing in your playground.config to follow the course without interruption.
Conduktor Free Kafka Playground is shutting down.
We are sorry to announce that our Free Kafka Playground is *shutting down* and will be removed very soon. This was very helpful to have a free Kafka available in seconds and run some tests, without having to start Kafka on your laptop.
We are here to help you migrate to a free Kafka playground alternative: upstash.com. A single change will be necessary to do in your playground.config file (the configuration of your Kafka cluster).
If you are already quite comfortable with Kafka, you can use any other alternative to get a Kafka cluster such as: Confluent, Aiven, AWS MSK, Strimzi, Redpanda and skip this guide.
To summarize the situation:
Upstash will provide you a Free Kafka playground (instead of Conduktor)
You will keep using Conduktor UI to learn and view everything about Kafka
Let's explore what to do!
1) Install Conduktor with Docker
Follow the course to install Conduktor UI with Docker (lecture #57) or directly follow the instructions on our website here: Get Started with Conduktor.
Now that you have your nice UI to learn and visualize Kafka, let's connect Conduktor to a free Kafka cluster running on the Cloud, to avoid starting one on your laptop. We'll see how to do this in the next step.
2) Create your account on Upstash
Go to https://console.upstash.com/kafka and register your account. Verify your account by checking your emails. Upstash should tell you your account has been validated.
In the next step, we'll see how to create your free Kafka cluster.
3) Create your free Kafka cluster on Upstash
Now that you are registered, you can create your free Kafka cluster in seconds here: https://console.upstash.com/kafka.
When it's done, it will ask you to create a topic. You skip this step, you will create topics later in CLI and Conduktor.
In the next step, we'll see how to connect Conduktor and CLI tools to this free Kafka cluster.
4) Update playground.config
On Upstash, go to the Details of your fresh new Kafka cluster, and copy the content of the Properties tab. It should be a URL like: https://console.upstash.com/kafka/XXX
Paste them using your favorite text editor into a file called: playground.config.
5) Update your Kafka bootstrap server
You must not use cluster.playground.cdkt.io:9092 as Kafka bootstrap server anymore as it will be removed and inaccessible soon.
Use the one provided by Upstash: bootstrap.servers=xxx.upstash.io:9092.
Here is an example of how your commands will look like
kafka-topics.sh --command-config playground.config --bootstrap-server xxx.upstash.io:9092 --list
6) Connect Conduktor to Upstash
You have started Conduktor locally. You have a free Kafka cluster. Time to connect both! Add your cluster configuration to Conduktor: http://localhost:8080/admin/clusters.
Copy/paste these values into the appropriate fields as shown below:
Bootstrap Server: xxx-kafka.upstash.io:9092
Security Protocol: SASL with SCRAM-SHA-256
Username and Password
Test the connection and it should work! You're ready to use Conduktor with Upstash.
You're ready to continue using Conduktor!
To summarize, we have updated how to connect Conduktor and the CLI to another Kafka cluster because Conduktor Free Kafka Playground will be removed soon.
If some steps are unclear, please tell us: support@conduktor.io. Thanks!
Happy learning :)