A. Elasticsearch
sudo wget
https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.5.2.deb
sudo dpkg -i elasticsearch-1.5.2.deb
sudo update-rc.d elasticsearch defaults 95 10
sudo /etc/init.d/elasticsearch restart
Elasticsearch is now installed. Let's edit the configuration:
sudo vi /etc/elasticsearch/elasticsearch.yml
network.host: localhost
sudo service elasticsearch restart
sudo update-rc.d elasticsearch defaults 95 10
B. Kibana
wget
https://download.elasticsearch.org/kibana/kibana/kibana-4.0.1-linux-x64.tar.gz
tar xvf kibana-*.tar.gz
vi ~/kibana-4*/config/kibana.yml
host: "localhost"
sudo mkdir -p /opt/kibana
Now copy the Kibana files into your newly-created directory:
sudo cp -R ~/kibana-4
/ /opt/kibana/
Kibana can be started by running /opt/kibana/bin/kibana, but we want
it to run
as a service. Download a Kibana init script with this command:
cd /etc/init.d && sudo wget https://gist.githubusercontent.com/thisismitch/8b15ac909aed214ad04a/raw/bce61d85643c2dcdfbc2728c55a41dab444dca20/kibana4
Now enable the Kibana service, and start it:
sudo chmod +x /etc/init.d/kibana4
sudo update-rc.d kibana4 defaults 96 9
sudo service kibana4 start
C. Setup Logstash
sudo wget
http://download.elastic.co/logstash/logstash/packages/debian/logstash_1.5.0-1_all.deb
sudo dpkg -i logstash_1.5.0-1_all.deb
sudo update-rc.d logstash defaults 95 10
sudo /etc/init.d/logstash restart
D. Create ssl key for clients
sudo mkdir -p /etc/pki/tls/certs
sudo mkdir /etc/pki/tls/private
If you don't have a DNS setup—that would allow your servers, that you
will gather logs from, to resolve the IP address of your Logstash
Server—you will have to add your Logstash Server's private IP address to
the subjectAltName (SAN) field of the SSL certificate that we are about
to generate. To do so, open the OpenSSL configuration file:
sudo vi /etc/ssl/openssl.cnf
subjectAltName = IP: logstash_server_private_ip
cd /etc/pki/tls;
sudo openssl req -x509 -batch -nodes -days 365 -newkey rsa:2048
-keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt
E. Create logstash config files for getting data from client machine
- LogStash for LumberJack:
vi /etc/logstash/conf.d/01-lumberjack-input.conf
input {
lumberjack {
port => 5000
type => "logs"
ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
}
}
- grok syslogs into the correct format:
vi /etc/logstash/conf.d/10-syslog.conf
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp}
%{SYSLOGHOST:syslog_hostname}
%{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?:
%{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
- configure LogStash to store it’s logs in ElasticSearch
vi /etc/logstash/conf.d/30-lumberjack-output.conf
output {
elasticsearch { host => localhost }
stdout { codec => rubydebug }
}
E. install redis server
sudo apt-get install redis-server
Search for the following line in /etc/redis/reis.conf and replace
bind 127.0.0.0
with
bind 0.0.0.0
