Example: Scripts Pipeline

Pipeline Scripts

paStash can serve all sorts of script emitting JSON objects for distribution and post-processing. In this example, we'll use the input of ppsguard to power a gauge using the statsd protocol.

scola-pastash (colander)

input {
  udp {
    host => 127.0.0.1
    port => 1234
  }
}

filter {
  json_fields {}
}

output {
  studout {}
  statsd {
      host => localhost
      port => 8125
      metric_type => gauge
      metric_key => pps
    }
}

PPS Counter producer

npm install -g ppsguard
ppsguard --max_pps=100 --interface eth0 --udp 127.0.0.1:1234

Event

0|pastash  | [STDOUT] {
0|pastash  |   "cpu": 1.50927734375,
0|pastash  |   "pps": 101.996651754871111,
0|pastash  |   "mean": 141.236352751854155,
0|pastash  |   "message": "WARNING!"
0|pastash  |   "timestamp": "2018-01-02T17:33:44.737Z",
0|pastash  | }

---

PPS to Splunk using PM2

create new pastash config at /opt/pastash/ppsguard.conf

	input {
	  udp {
	    host => 127.0.0.1
	    port => 18911
	  }
	}

	filter {
	  json_fields {}
	}

	output {
	  splunk {
	    token => "XXX-YYY-ZZZ"
	    source => "ppsGuard"
	    index => "your-lovely-index"
	    sourcetype => "_json"
	    splunk_url => "https://your-own-id.splunkcloud.com:443/services/collector/event"
	    flat => true
	  }
	}

create file in /opt/pastash/pastash_pps.yml"

apps:
  - script   : pastash
    name: 'pastash_pps'
    watch: true
    args: ['--config_file=/opt/pastash/ppsguard.conf']

start the new pastash service:

pm2 start /opt/pastash/pastash_pps.yml

verify the service:

pm2 show pastash_pps

Client

create file in /opt/pastash/ppsguard.yml

apps:
 - script   : ppsguard
   name: 'ppsguard'
    args: ['--interface', 'eth0', '--max_pps=1', '--max_cpu=2', '--limit_every', 'second', '--udp', '127.0.0.1:18911']

start the new pastash service:

pm2 start /opt/pastash/ppsguard.yml

verify the service:

pm2 show ppsguard