In this tutorial, we will learn how to install Flume on Linux Mint Latest. Flume is a reliable distributed system for aggregating and moving large amounts of data from different sources into a centralized repository, such as Apache Hadoop.
Before we start with the installation process, make sure that your Linux Mint system is up to date with the latest packages and dependencies. Use the following command to update the system:
sudo apt update && sudo apt upgrade -y
Flume requires Java to run. First, we need to install Java by running the following command:
sudo apt install openjdk-11-jdk -y
Verify the Java installation using the following command:
java -version
You should see the installed Java version on your system.
Now, we will download and install Flume on our system. Use the following command to download Flume:
curl -O https://downloads.apache.org/flume/1.9.0/apache-flume-1.9.0-bin.tar.gz
Extract the downloaded file using the following command:
tar -xzvf apache-flume-1.9.0-bin.tar.gz
Move the extracted Flume directory to the /usr/local directory using the following command:
sudo mv apache-flume-1.9.0-bin /usr/local/flume
To set the environment variables for Flume, use the following command to edit the ~/.bashrc file:
nano ~/.bashrc
Add the following lines at the end of the file:
export FLUME_HOME=/usr/local/flume
export PATH=$PATH:$FLUME_HOME/bin
Save the changes and exit the file.
Run the following command to apply the changes:
source ~/.bashrc
To test the Flume installation, use the following command to start the Flume agent:
flume-ng agent --conf-file /usr/local/flume/conf/flume.conf --name agent -Dflume.root.logger=INFO,console
The above command will start the Flume agent in console mode with the default Flume configuration file.
You should see the following output:
[INFO ] [main] [org.apache.flume.node.Application] Flume starting
[INFO ] [main] [org.apache.flume.node.Application] Starting Channel MemoryChannel
[INFO ] [main] [org.apache.flume.node.Application] Starting Sink null
[INFO ] [main] [org.apache.flume.node.Application] Starting Source seq-source
[INFO ] [main] [org.apache.flume.node.Application] Starting Sink loggerSink
[INFO ] [event-1] [org.apache.flume.source.SequenceGeneratorSource] Sequence generator source starting
[INFO ] [lifecycleSupervisor-1-0] [org.apache.flume.lifecycle.LifecycleSupervisor] Starting lifecycle supervisor 1
[INFO ] [lifecycleSupervisor-1-0] [org.apache.flume.lifecycle.LifecycleSupervisor] Starting lifecycle for Sink loggerSink
[INFO ] [lifecycleSupervisor-1-0] [org.apache.flume.lifecycle.LifecycleSupervisor] Starting lifecycle for Source seq-source
[INFO ] [SpiTopologyOpenEventThread] [org.apache.flume.sink.LoggerSink] Opened
[INFO ] [event-1] [org.apache.flume.source.SequenceGeneratorSource] Generated sequence id 1
[INFO ] [event-1] [org.apache.flume.channel.MemoryChannel] Channel initialized: memoryChannel
[INFO ] [lifecycleSupervisor-1-0] [org.apache.flume.instrumentation.SinkCounter] Sink loggerSink started
[INFO ] [lifecycleSupervisor-1-0] [org.apache.flume.instrumentation.SourceCounter] Source seq-source started
[INFO ] [main] [org.apache.flume.node.Application] Application started
Hit CTRL+C to stop the Flume agent.
Congratulations! You have successfully installed and tested Flume on your Linux Mint system.
In this tutorial, we learned how to install Flume on Linux Mint Latest. We also learned how to test the Flume installation by starting a Flume agent in console mode. Flume is now ready to move and aggregate data from different sources into a centralized repository.
Happy fluming!
If you want to self-host in an easy, hands free way, need an external IP address, or simply want your data in your own hands, give IPv6.rs a try!
Alternatively, for the best virtual desktop, try Shells!
