Molamk

Monitor your laptop using the Elastic Stack

September 02, 2019

ELK Stack
ELK Stack

Elastricsearch

# Install Elasticsearch
brew tap elastic/tap
brew install elastic/tap/elasticsearch-full

# Run elasticsearch
elasticsearch

# Verify it's working correctly
curl localhost:9200
# Output should be
# {
#   "name" : "mos-MacBook-Pro.local",
#   "cluster_name" : "elasticsearch_mo",
#   "cluster_uuid" : "oJm3E09-R8KLOJszauIO0Q",
#   "version" : {
#     "number" : "7.3.1",
#     "build_flavor" : "default",
#     "build_type" : "tar",
#     "build_hash" : "4749ba6",
#     "build_date" : "2019-08-19T20:19:25.651794Z",
#     "build_snapshot" : false,
#     "lucene_version" : "8.1.0",
#     "minimum_wire_compatibility_version" : "6.8.0",
#     "minimum_index_compatibility_version" : "6.0.0-beta1"
#   },
#   "tagline" : "You Know, for Search"
# }

Kibana

# Install Kibana
brew install elastic/tap/kibana-full

# Run Kibana
kibana

To verify that Kibana is working fire up your browser and head over to localhost:5601. You should see something like

Beats

We’re going to install Metricbeat. Here’s a summarized description about what it does

“Collect metrics from your systems and services. From CPU to memory, Redis to NGINX, and much more, Metricbeat is a lightweight way to send system and service statistics.” - Elastic

# Install Metricbeat
brew install elastic/tap/metricbeat-full

# Enable it
metricbeat modules enable system

# Setup the initial environment
metricbeat setup -e

Now you can already view your metrics. Head over to http://localhost:5601/app/kibana#/dashboard/Metricbeat-system-overview-ecs. Here’s what it should look like:

Host Metrics Dashboard
Host Metrics Dashboard

Logstash

In our scenario, Metricbeat sends the data directly to Elasticsearch which is then displayed by Kibana. We can use a more complex (but flexible) approach by adding Logstash to our stack. Here’s a quick description by the Elastic Team:

“Logstash is an open source, server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite ‘stash.’” - Elastic

# Install Logstash
brew install elastic/tap/logstash-full

We then need to configure a pipeline so that Metricbeat send the data to Logstash. Let’s create a demo-metrics-pipeline.conf that will hold our config.

input {
  beats {
    port => 5044
  }
}

# The filter part of this file is commented out to indicate that it
# is optional.
# filter {
#
# }

output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}

Now let’s start Logstash by giving the config file to run.

logstash -f path/to/config/demo-metrics-pipeline.conf

Let’s edit /usr/local/etc/metricbeat to instruct Metricbeat to send its data to Logstash

#-------------------------- Elasticsearch output ------------------------------
#output.elasticsearch:
  # Array of hosts to connect to.
  #hosts: ["localhost:9200"]
.
.
.
#----------------------------- Logstash output --------------------------------
output.logstash:
  # The Logstash hosts
  hosts: ["localhost:5044"]

Add filter between input and output in the logstash config file (demo-metrics-pipeline.conf)

filter {
  if [system][process] {
    if [system][process][cmdline] {
      grok {
        match => {
          "[system][process][cmdline]" => "^%{PATH:[system][process][cmdline_path]}"
        }
        remove_field => "[system][process][cmdline]"
      }
    }
  }
}