How to Install Apache Kafka on Ubuntu 20 04

Linux

Install Apache Kafka server on ubuntu 20.04 server

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics,data integration, and mission-critical applications.

In this tutorial,we will show you how to install Apache Kafka on Ubuntu 20.04

Step 1: Install java

first we need to update packages and install java 8 or 11 using apt.

sudo apt update
sudo apt install openjdk-11-jdk

once java is installed, you can check the version

java --version

sample output:

openjdk 11.0.10 2021-01-19
OpenJDK Runtime Environment (build 11.0.10+9-Ubuntu-0ubuntu1.20.04)
OpenJDK 64-Bit Server VM (build 11.0.10+9-Ubuntu-0ubuntu1.20.04, mixed mode, sharing)

Also Read -> How to Install Flask on Ubuntu 20 04

Step 2: download Apache kafka

you can download latest stable version from official webpage.

wget https://downloads.apache.org/kafka/2.8.0/kafka_2.13-2.8.0.tgz

untar it and move to any specifica location like below.

sudo tar -xvzf kafka_2.13-2.8.0.tgz
sudo mv kafka_2.13-2.8.0 /opt/kafka/

Also Read -> How to Install MySQL 8 on Ubuntu 20 04

Step 3: create kafka and zookeeper unitd files

Now create systemd unit files for the Zookeeper 

sudo nano /etc/systemd/system/zookeeper.service

Add below lines to above file

[Unit]
Description=Apache Zookeeper service
Documentation=http://zookeeper.apache.org
Requires=network.target remote-fs.target
After=network.target remote-fs.target
[Service]
Type=simple
ExecStart=/opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper.properties
ExecStop=/opt/kafka/bin/zookeeper-server-stop.sh
Restart=on-abnormal
[Install]
WantedBy=multi-user.target

next create systemd unit file for the kafka 

sudo nano /etc/systemd/system/kafka.service

[Unit]
Description=Apache Kafka Service
Documentation=http://kafka.apache.org/documentation.html
Requires=zookeeper.service
[Service]
Type=simple
Environment="JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64"
ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh
[Install]
WantedBy=multi-user.target

Also Read -> How to Install Apache Maven on Ubuntu 20 04

now reload deamons and enable with following commands.

sudo systemctl daemon-reload
sudo systemctl enable --now zookeeper
sudo systemctl enable --now kafka

now you check services status.

systemctl status zookeeper.service
systemctl status kafka.service

output:

● zookeeper.service - Apache Zookeeper service
     Loaded: loaded (/etc/systemd/system/zookeeper.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2021-04-26 10:19:09 UTC; 7min ago
       Docs: http://zookeeper.apache.org
   Main PID: 7931 (java)

● kafka.service - Apache Kafka Service
     Loaded: loaded (/etc/systemd/system/kafka.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2021-04-26 10:19:13 UTC; 7min ago
       Docs: http://kafka.apache.org/documentation.html
   Main PID: 8328 (java)

also Read -> How to Install ReactJS on Ubuntu 20 04

Step 4: create topic in kofka

Now we will create a topic named ubuntu with a single replication-factor and partition as below.

/opt/kafka
sudo bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic ubuntu

output:

Created topic ubuntu.

To check the list of topics created, use the following command.

cd /op/kafka/bin
./kafka-console-producer.sh --broker-list localhost:9092 --topic ubuntu

it prompt for messages to type

>hi
>how are you

 we can see the list of messages using below command.

cd /opt/kafka/bin
./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic ubuntu --from-beginning
hi
how are you

That's it.Now you have successfully installed and configured Apache Kafka service on an ubuntu 20.04 system.

Also Read -> How to Install Wine on Ubuntu 20 04