Install Apache Kafka on a Linux VPS by installing a supported Java runtime, extracting a pinned Kafka release, initializing KRaft storage, starting the broker, and validating it with a test topic. The commands below use Kafka 4.3.0’s quick-start archive as a reproducible example, while noting where configuration paths differ between releases. A single combined broker/controller is suitable for a lab or development server; a public or critical deployment needs private controller networking, authentication, encryption, authorization, and capacity planning before clients connect.
Need a server to try this on? Antihost offers Linux VPS plans with NVMe storage, full root access and a 14-day free trial.
What this installation creates
This guide builds a single-node Kafka cluster in KRaft mode. KRaft removes the ZooKeeper dependency and stores cluster metadata in Kafka’s own controller quorum. The example runs broker and controller roles in one process, which keeps a development VPS simple. It does not provide host-level high availability: if that VPS fails, both roles disappear.
As an Amazon Associate I earn from qualifying purchases.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Apache Kafka’s current Java documentation fully supports Java 17 and Java 21 for Kafka 4.1 and generally recommends the newest long-term-support release; the quick-start example for Kafka 4.3.0 requires Java 17 or newer. Java 21 is the sensible choice when your distribution supplies it, while Java 17 remains a supported option.
Requirements and decisions before you begin
| Item | Requirement or decision |
|---|---|
| Operating system | A supported 64-bit Linux distribution with systemd or an equivalent service manager. |
| Privileges | Root or sudo access for package installation, account creation, directories, and firewall rules. Kafka itself should run as a non-root kafka account. |
| Java | OpenJDK 17 or 21. Confirm the selected runtime with java -version. |
| Storage | Persistent disk mounted at a stable path. Kafka logs, controller metadata, and operating-system files must survive a reboot. |
| DNS | A hostname that resolves to the address clients will use. This matters because Kafka returns advertised addresses after the initial bootstrap connection. |
| Network policy | A host firewall that allows only required client, administration, and internal broker/controller traffic. |
| Kafka release | Pin one release for the installation. This example uses the archive name kafka_2.13-4.3.0.tgz; use the matching archive and bundled configuration for another release. |
Do not estimate VPS size from the Kafka package alone. Message rate, retention period, replication factor, partition count, disk I/O, and network throughput determine broker capacity. Apache’s KRaft operations guidance gives a typical baseline of 5 GB of memory and 5 GB of disk for a controller metadata log, not a universal broker recommendation.
#1 Best Overall
Prepare the Linux VPS
Update the host and install basic utilities
Use your distribution’s package manager to apply security updates and install a tar extractor, certificate tools, and a service manager if they are not already present. Package names vary by distribution; on Debian- or Ubuntu-based systems, the usual starting point is:
sudo apt update
sudo apt upgrade
sudo apt install -y ca-certificates curl tarOn a production server, schedule reboots required by kernel updates and record the maintenance window before Kafka is serving clients.
Create a dedicated service account
Kafka should not run as root. Create a system account without an interactive login, then create directories for the software, data, and logs:
sudo useradd --system --home-dir /opt/kafka --shell /usr/sbin/nologin kafka
sudo mkdir -p /opt/kafka /var/lib/kafka /var/log/kafka
sudo chown -R kafka:kafka /opt/kafka /var/lib/kafka /var/log/kafkaIf your distribution uses a different nologin path, substitute that path. Keep the Kafka data directory on persistent storage; placing it on an ephemeral VPS volume defeats the durability expected from a broker.
Install and verify Java
Install OpenJDK 21 when it is available from your distribution’s supported repositories. OpenJDK 17 is also valid for the Kafka releases covered here. After installation, verify both the major version and the executable selected by the shell:
java -version
command -v javaThe version output must show 17 or 21. If several Java versions are installed, select the intended one with your distribution’s alternatives mechanism or set JAVA_HOME explicitly in the service environment. Do not continue with an older runtime simply because the command exists; Kafka startup failures caused by an unsupported Java version are easier to prevent than diagnose.
Download and extract a pinned Kafka release
Obtain the binary archive from the official Apache Kafka release channel and verify it according to that release’s published checksum or signature instructions. Verification is especially important on an Internet-facing VPS because it confirms that the archive was not corrupted or replaced in transit.
The Kafka 4.3.0 quick start illustrates this extraction command:
cd /tmp
# Place the verified kafka_2.13-4.3.0.tgz file in this directory first.
tar -xzf kafka_2.13-4.3.0.tgz
sudo mv kafka_2.13-4.3.0 /opt/kafka/current
sudo chown -R kafka:kafka /opt/kafka/currentThe Scala suffix in the archive name is part of Kafka’s distribution naming; keep the exact name supplied by the release you downloaded. A stable path such as /opt/kafka/current makes service files and operational scripts independent of the version directory. Before upgrading, install the new release in a separate directory and change the symlink only after reading that release’s upgrade notes.
Initialize KRaft storage
KRaft storage must be formatted with a cluster identity before the broker can start. Run the commands as the kafka account so the generated metadata and data files have the correct owner:
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallsudo -u kafka -H bash
cd /opt/kafka/current
KAFKA_CLUSTER_ID="$(bin/kafka-storage.sh random-uuid)"
bin/kafka-storage.sh format --standalone -t "$KAFKA_CLUSTER_ID" -c config/server.properties
exitThe --standalone option creates a single-node controller configuration for a development or test cluster. The exact configuration path is release-dependent. Some packages provide a KRaft-specific file under config/kraft/; use the file shipped with your release and do not substitute an old file copied from another version.
Formatting is destructive to the target log directory. Never run the format command against an existing cluster unless you intentionally want to create a new cluster and have dealt with the data already stored there. A formatted directory carries the generated cluster identity; reusing it with a different identity prevents the node from joining the original cluster.
Configure listeners before accepting clients
A Kafka listener is both a bind address and a protocol declaration. Kafka supports PLAINTEXT, SSL, SASL_PLAINTEXT, and SASL_SSL. The listener used by clients, the listener used between brokers, and the private controller listener do not have to use the same protocol.
For a local development test, the bundled standalone configuration and localhost:9092 are sufficient. For a VPS, inspect and deliberately set the following properties in the release’s server configuration:
listenersdetermines the local interfaces and ports on which Kafka binds.advertised.listenerscontains the hostnames and ports returned to clients during metadata discovery. Every advertised address must resolve and be reachable from the client network, not merely from the VPS itself.listener.security.protocol.mapmaps each named listener toPLAINTEXT,SSL,SASL_PLAINTEXT, orSASL_SSL.controller.listener.namesidentifies the listener reserved for KRaft controllers.inter.broker.listener.nameselects the listener used for broker-to-broker traffic when the topology has multiple brokers.
A concrete single-node layout might bind a client listener on port 9092 and a controller listener on port 9093 bound only to loopback or a private interface. Treat those port numbers as an example, not a Kafka requirement. The important boundary is that the controller listener is private and is never advertised to client networks.
For Internet or cross-host access, replace the example’s unauthenticated client protocol with TLS (SSL) or SASL over TLS (SASL_SSL) as appropriate. Configure the keystore, truststore, certificate names, SASL mechanism, and credentials consistently on both broker and clients. Keep controller and inter-broker mappings internally consistent; a protocol mismatch can prevent a KRaft quorum or broker connection from forming.
Start Kafka and perform a foreground test
Start the broker in the foreground first. This exposes configuration and permission errors immediately:
cd /opt/kafka/current
sudo -u kafka -H bin/kafka-server-start.sh config/server.propertiesLeave this process running while you test from another shell. A successful start should create a broker process, bind the configured listeners, and report that the node has joined its standalone controller quorum. If the process exits, read the error before changing files; the first failure is usually the useful one.
Free tools Windows power users keep installed
One-click scans. No signup required.
Rank #3
When the foreground test works, stop it cleanly with Ctrl+C. Do not run a foreground process and a systemd-managed process against the same log directory at the same time.
Run Kafka under systemd on a persistent VPS
Systemd gives the broker a controlled working directory, automatic restart behavior, and a predictable boot order. Create /etc/systemd/system/kafka.service as root:
[Unit]
Description=Apache Kafka broker
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=kafka
Group=kafka
WorkingDirectory=/opt/kafka/current
Environment="JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64"
ExecStart=/opt/kafka/current/bin/kafka-server-start.sh /opt/kafka/current/config/server.properties
ExecStop=/bin/kill -TERM $MAINPID
Restart=on-failure
RestartSec=5
LimitNOFILE=100000
[Install]
WantedBy=multi-user.targetThe JAVA_HOME path is distribution-specific. Replace it with the directory that contains the Java installation selected by java -version, or omit the line if the service manager already provides the correct runtime.
Load and enable the unit:
sudo systemctl daemon-reload
sudo systemctl enable --now kafka
sudo systemctl status kafka --no-pager
sudo journalctl -u kafka -fThe foreground start command is the official quick-start workflow; this unit is operational packaging for a VPS. Keep logs in a controlled location and configure rotation appropriate to your retention policy so a verbose broker cannot fill the root filesystem.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Create and inspect a test topic
With the broker running and a client listener available, create a topic using the same listener address that clients will use:
cd /opt/kafka/current
bin/kafka-topics.sh --create
--topic quickstart-events
--bootstrap-server localhost:9092
bin/kafka-topics.sh --describe
--topic quickstart-events
--bootstrap-server localhost:9092The describe output should show the topic’s partition and leader metadata. On a one-node cluster, the single broker will necessarily hold the leadership and replica for this test topic. A production topic’s partition count and replication factor must reflect the workload and the number of brokers; do not treat the quick-start topic as a capacity design.
Produce and consume a record to verify the complete data path. Run the producer in one shell:
bin/kafka-console-producer.sh
--topic quickstart-events
--bootstrap-server localhost:9092Type a line and press Enter. In a second shell, run:
bin/kafka-console-consumer.sh
--topic quickstart-events
--from-beginning
--bootstrap-server localhost:9092For a TLS- or SASL-enabled listener, pass the client properties file required by your configuration to both utilities. A client that can connect to the bootstrap address but cannot consume usually has a security-property, authorization, advertised-address, or topic-permission problem rather than a basic process problem.
Secure a Kafka broker on a public VPS
Apache Kafka states that security is off by default: a fresh installation accepts unauthenticated PLAINTEXT connections on every listener and applies no authorization. That mode is appropriate only for a closed test environment. Do not expose the default listener to the Internet.
Bind only the interfaces you need
Bind the client listener to the interface or private address that should receive client traffic. Bind the controller listener to loopback or a private network shared only by the controller quorum. If the VPS has multiple interfaces, an accidental wildcard bind can expose a listener on a management or public network.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesAdvertise a reachable, correct address
Clients use advertised.listeners after metadata discovery. Set it to a DNS name or address that the actual clients can resolve and route to. A common failure pattern is a successful bootstrap followed by connection attempts to localhost, an RFC1918 address unavailable to remote clients, or an unresolvable hostname. Test resolution and connectivity from the client network, not only from the broker itself.
Use authentication and encrypted transport
Choose SSL when mutual or server-authenticated TLS is sufficient, or SASL_SSL when you also need a SASL authentication mechanism. Distribute trust material securely, use certificate names matching the advertised hostname, and set client properties that match the listener’s protocol and mechanism. Plain SASL without TLS can expose credentials and message traffic to a network observer.
Enable authorization and least privilege
Authentication identifies a principal; authorization controls what that principal can do. Grant producers only the topic permissions they need, consumers the corresponding read and group permissions, and administrators a separate, tightly controlled role. Test ACLs with a non-administrator account before opening the firewall so an accidental superuser configuration does not become the only working path.
Keep the controller listener off client networks
Apache explicitly advises never exposing the controller listener to clients. Firewall rules should allow controller traffic only between the controller nodes or the local process in a combined development node. Client-facing DNS records and load balancers must point to broker listeners, never to the KRaft quorum endpoint.
Recommended Free Tools
Protect the host and data separately
Kafka’s security model does not encrypt log data at rest. If stored messages or credentials require confidentiality, use encrypted filesystems or block devices, protect backups, restrict shell access, and consider message-level encryption where appropriate. Transport encryption does not protect a disk stolen from the VPS or a backup copied without access controls.
Firewall and port planning
Open only the broker listener ports required by known clients and administration sources. Keep the controller and inter-broker ports on a private network or host-only rule. The quick-start broker listener uses port 9092, but Kafka does not require that number; use the ports defined in your listener configuration.
- Allow client traffic only from application networks or approved source addresses.
- Allow SSH or administrative access from a management network, not from every Internet address.
- Allow controller and inter-broker traffic only among the nodes that participate in those roles.
- Do not add broad allow rules merely because a client failed; first inspect the advertised address, DNS, protocol, and authentication settings.
After applying the firewall, confirm listening sockets with your operating system’s socket tool and test from an approved client host. A port that is open locally but filtered upstream can look identical to a Kafka configuration error until both sides are checked.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.KRaft topology and VPS sizing
Combined mode for development
The --standalone format used above combines broker and controller roles. It is convenient for a development VPS, a local integration environment, or a disposable test cluster. It cannot tolerate loss of that one host and should not be presented as a highly available production design.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Separate roles for critical deployments
For a critical deployment, separate broker and controller roles and use a redundant controller quorum. KRaft guidance states that tolerating N concurrent controller failures requires 2N+1 controllers. Thus, a quorum that must survive one controller failure needs three controllers; surviving two concurrent failures needs five. Those controllers should be distributed across independent failure domains. Multiple VPS instances in one physical failure domain improve process isolation but do not guarantee host or zone availability.
Derive broker resources from workload
| Workload factor | Why it changes the VPS design |
|---|---|
| Ingress and egress rate | Determines network bandwidth, CPU spent on protocol and compression, and sustained disk write/read pressure. |
| Retention period and message size | Determine how much log storage is required before cleanup can reclaim segments. |
| Partition count | Increases metadata, open files, recovery work, and scheduling overhead even when traffic is low. |
| Replication factor | Multiplies storage and network traffic but provides resilience only when replicas are on separate brokers. |
| Controller metadata | Apache’s typical KRaft controller-log baseline is 5 GB of memory and 5 GB of disk; broker data needs are usually much larger. |
| Failure tolerance | Requires enough brokers and controllers to keep the intended quorum and replica assignments after a host loss. |
Measure your expected peak rate, not just the daily average. Leave headroom for consumer catch-up, rebalances, compaction or retention work, operating-system processes, and disk replacement. A single VPS can run Kafka, but it cannot provide host-level fault tolerance by itself.
Validation checklist after installation
- Runtime:
java -versionreports Java 17 or 21, and the systemd service uses that same runtime. - Process:
systemctl status kafkashows an active service without repeated restarts. - Storage: the Kafka data directory is persistent, owned by
kafka, and has free space for retention and recovery. - Listeners: socket inspection shows bindings only on intended interfaces and ports.
- Metadata:
kafka-topics.sh --describereturns the expected leader and partition information. - Data path: a producer record appears in a consumer using matching client security properties.
- Remote access: a client outside the VPS can resolve every advertised address and reach the required broker listener.
- Security: client authentication, TLS trust, ACLs, and firewall rules have been tested with a least-privilege account.
- Controller isolation: no public DNS record, load balancer, or broad firewall rule points clients at the controller listener.
Troubleshooting common failures
Kafka exits immediately with a Java error
Check java -version, JAVA_HOME in the service unit, and the Java binary visible to the kafka account. Kafka 4.1 supports Java 17 and 21; the 4.3.0 quick-start requires Java 17 or newer. Correct the service environment rather than installing multiple random runtimes until one happens to start.
The format command reports an existing or invalid log directory
Stop every Kafka process and inspect the configured log directories. Formatting is for a new cluster; it should not be used to repair an existing one. If this is a disposable test node, remove only the intended test data, recreate the directory with the kafka owner, generate a new cluster ID, and format again. For a real cluster, preserve the directory and investigate the original configuration and cluster identity.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →The broker starts but remote clients cannot connect
Check the client’s bootstrap address, then inspect the broker’s advertised.listeners. Verify DNS resolution and firewall rules from the client host. A successful TCP connection to the bootstrap listener does not prove that the addresses returned in metadata are usable.
TLS or SASL clients fail during handshake
Compare the client protocol with the listener’s mapped protocol, confirm that the certificate name matches the advertised hostname, and verify truststore or CA material. For SASL, check the mechanism and credentials. Keep controller and inter-broker listener mappings consistent with the KRaft quorum configuration; a client-facing change can otherwise cause internal connections to fail.
Topic creation works but produce or consume is denied
Authentication may be succeeding while authorization is not. Review ACLs for the exact principal, topic, and consumer group. Test with the intended producer and consumer identities rather than an administrator account, and pass the same client properties to the command-line tools that your application uses.
The service repeatedly restarts or the disk fills
Read the service journal and Kafka logs for the first failure, then check filesystem capacity and inode usage. Retention settings, high partition counts, large messages, and slow storage can all increase disk pressure. Do not solve a full disk by deleting active log segments; stop safely, preserve data, and follow the release’s recovery and retention procedures.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Clear out junk files and repair common Windows errors3Scan for outdated or missing drivers - takes under a minuteWhen to use ZooKeeper instead
ZooKeeper-based instructions describe an older Kafka workflow. New installations should prefer KRaft unless a compatibility requirement ties an existing environment to ZooKeeper. Migrating an established cluster is a separate project involving version-specific procedures, quorum planning, and rollback decisions; do not mix ZooKeeper and KRaft initialization commands on the same data directory.
Quick Recap
Final pre-exposure checklist
- Pin the Kafka release and verify its archive before extraction.
- Run Java 17 or 21 under a dedicated, non-root
kafkaaccount. - Format a new KRaft log directory once with a generated cluster ID.
- Test the broker in the foreground, then manage it with systemd.
- Create, describe, produce to, and consume from a test topic.
- Set listener bindings and advertised addresses for the actual client network.
- Move client traffic to TLS or SASL over TLS, configure authorization, and keep controller traffic private.
- Apply a source-restricted firewall and verify it from an approved remote client.
- Size storage, memory, network, partitions, and replication from measured workload and failure goals.
- Document the cluster ID, data path, listener map, credentials, certificates, and recovery owner before handing the VPS to applications.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
