WeFlex Log Monitor

This repository helps to collect logs from the WeFlex servers. We use an ELK stack to process the logs.

Dependencies

Pre-requisite: Docker must be installed on the host machine.

1. The WeflexLog Docker image is built from sebp/elk docker image. You can read the docs here.
2. Filebeat is required to be installed and running on the host machine to publish logs to server. A sample config is shown below.

While deploying the config to production server, you must copy the public and private ssl keys to the docker container to /etc/pki/tls/certs/logstash-beats.crt and /etc/pki/tls/private/logstash-beats.key OR any other path which must be consistent and communicated to filebeat and logstash config. To generate public and private keys, you can use letsencrypt.org service.

Install filebeat

yum install filebeat

Filebeat sample configuration

Copy the filebeat configuration to your host machine.

Filebeat configuration explanation

The below configuration does the following:

1. It assumes that you have a log at /Users/pbalan/Downloads/messages on your host machine which is required to be monitored and published to logstash. See line 28.

You must change it to point to your log you want to publish.

2. We set the tag to the logs published from the server on line 93.
3. We set the env to the logs published from the server on line 98.
4. Specify kibana host at line 123.
5. Enable logstash output on line 154.
6. Specify logstash host on line 157.
7. Specify the path to ssl certifcate authorities on line 161.

Start filebeat

sudo /etc/init.d/filebeat start

Roll up the container

• Use Dockerfile

  docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -it --name weflexlog_elk_1 weflextech/elk

• Use docker-compose.yml

  docker-compose up elk

Make sure the configuration files are up to date at /etc/logstash/conf.d/ by opening up a shell inside the docker container

docker exec -it <container-name> /bin/bash

Publish changes to Docker image

1. Pull the latest image.

   docker pull weflextech/elk:latest

2. Roll up a container if one is not running already.

   docker run -p 5601:5601 -p 9200:9200 -p 5044:5044 -it --name weflexlog_elk_1 weflextech/elk /bin/bash

3. ADD YOUR CHANGES.

4. Commit your changes.

   docker commit -m <message> -a <user> weflexlog_elk_1 weflextech/elk:latest

5. Login to docker hub.

   docker login

6. Push your changes.

   docker push weflextech/elk