Apache Kafka is a popular open-source messaging system that streams events in real-time. In this tutorial, we will walk through how to install Kafka from the official Apache Kafka website on an Ubuntu server.
Before installing Kafka on Ubuntu, there are a few prerequisites you should have installed:
First, go to the Apache Kafka website and download the latest stable release of Kafka. At the time of writing, the latest version is Kafka 2.7.1.
wget https://downloads.apache.org/kafka/2.7.1/kafka_2.13-2.7.1.tgz
When the download is complete, extract the contents of the file to the /opt directory:
sudo tar -xzf kafka_2.13-2.7.1.tgz -C /opt
This will create a directory called kafka_2.13-2.7.1 in the /opt directory.
Before starting Kafka, ensure that ZooKeeper is running. If you installed ZooKeeper on the same server, run:
sudo systemctl start zookeeper
To start Kafka, open a new terminal window and navigate to the Kafka directory:
cd /opt/kafka_2.13-2.7.1
Next, run the following command:
sudo ./bin/kafka-server-start.sh config/server.properties
This will start Kafka with the default configuration. If you want to customize Kafka's configuration, edit the config/server.properties file before starting Kafka.
Once Kafka is running, you can create a topic to start publishing messages. To create a new topic, run:
sudo ./bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic my-topic
This will create a topic called my-topic with one partition and one replication factor.
To publish messages to a topic, run:
sudo ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic my-topic
This will open a console where you can enter messages to publish to the topic.
To consume messages from a topic, open a new terminal window and run:
sudo ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic my-topic --from-beginning
This will open a console where you can see the messages published to the my-topic topic.
That's it! You now have Apache Kafka running on your Ubuntu server, and you can start building your own real-time data streaming applications.
If you want to self-host in an easy, hands free way, need an external IP address, or simply want your data in your own hands, give IPv6.rs a try!
Alternatively, for the best virtual desktop, try Shells!
