EzDevInfo.com

Count riemann events in given time window

In riemann config for specific service I'm trying to assign to all its events metric=1, sum them within 5sec and send the result to influxdb.

I gave up with following:

(streams
  (where (service "offers")
    (fixed-time-window 5
      (smap folds/sum (with :metric 1 index))))
  influx)

it doesn't really work, events stored in influx do not match this rule. any hint?

riemann.io add jar to classpath

I have written custom clojure functions that I want to use in my riemann configuration. I am using leiningen to build jar file (with dependencies) containing my functions. What is the right way to include this jar file in the classpath when starting riemann ?

How to integrate riemann into the dropwizard to capture metrics?

I have a dropwizard application which emits yammer metrics and can be monitored via a URL like http://localhost:8081/admin/metrics which gives the result in form of jsons.

I want to send these monitor these metrics in riemann and I have no idea on how to start. I went through the riemann-java-client which has a RiemannReporter class for yammer metrics but I do not how to use this in my application.

How to integrate this client into my application or how to capture jsons from the url and send these as events to riemann server?

ERROR: Failed to buld gem native extension?

I am installing riemann monitoring tool on my PC. I run the following command from terminal:

Command:

sudo gem install riemann-tools

I get the following error:

ERROR:  Error installing riemann-tools:
        ERROR: Failed to build gem native extension.

            /usr/bin/ruby1.9.1 extconf.rb
    /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- mkmf (LoadError)
        from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
        from extconf.rb:1:in `<main>'

A similar error comes when installing riemann-dash but riemann-client get installed successfully.

I am using Ububtu 12.10. I read online about such errors and it had something with ruby version, but I am not able to figure out the reason for this.

Any help is appreciated.

configure nagios riemann integration for monitoring

Hi i want to know how can i configure Riemann to forward events to nagios Server This gives the API for Riemann nagios integration but there is not enough info of doing it.I am new to Riemann and i am unable to figure out a way.

How to send file to riemann server?

I have setup riemann server, now I want to send log file to riemann server. How can i do that? I did not find any helpful material from internet.

Custom email subject and body in riemann

I am using email alert in riemann. I have hardecoded the value for :subject and `:body'. I am facing some issue in this. I pasted my riemann code and the error I got below. I don't know how to resolve this.

(let [email (mailer {:host "smtp.gmail.com"
                            :port 25
                            :user "user"
                            :pass "password"
                            :auth "true"
                            :subject "Welcome All to Coding"
                            :body "Hello Team, \n Welcome to my coding \n Thank You!"
                            :from "mygmail@gmail.com"})])

I got the below error

WARN [2015-07-25 02:10:17,640] defaultEventExecutorGroup-2-2 - riemann.config -
riemann.email$mailer$make_stream__7892$stream__7893@62aa5 threw
java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IF
n
        at riemann.email$email_event.invoke(email.clj:15)
        at riemann.email$mailer$make_stream__7892$stream__7893.invoke(email.clj:
69)
        at riemann.config$eval96$stream__101$fn__106.invoke(riemann_v1.con
fig:39)
        at riemann.config$eval96$stream__101.invoke(riemann_v1.config:39)
        at riemann.streams$smap$stream__3695$fn__3706.invoke(streams.clj:163)
        at riemann.streams$smap$stream__3695.invoke(streams.clj:163)
        at riemann.streams$fixed_time_window_fn$stream__3946$fn__3979.invoke(str
eams.clj:381)
        at riemann.streams$fixed_time_window_fn$stream__3946.invoke(streams.clj:
381)
        at riemann.config$eval96$stream__145$fn__150.invoke(riemann_v1.con
fig:27)
        at riemann.config$eval96$stream__145.invoke(riemann_v1.config:27)
        at riemann.core$stream_BANG_$fn__5678.invoke(core.clj:19)
        at riemann.core$stream_BANG_.invoke(core.clj:18)
        at riemann.transport$handle.invoke(transport.clj:159)
        at riemann.transport.tcp$tcp_handler.invoke(tcp.clj:93)
        at riemann.transport.tcp$gen_tcp_handler$fn__5904.invoke(tcp.clj:65)
        at riemann.transport.tcp.proxy$io.netty.channel.ChannelInboundHandlerAda
pter$ff19274a.channelRead(Unknown Source)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(Abst
ractChannelHandlerContext.java:333)
        at io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractCha
nnelHandlerContext.java:32)
        at io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelH
andlerContext.java:324)
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecuto
r.java:36)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThread
EventExecutor.java:116)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorato
r.run(DefaultThreadFactory.java:137)
        at java.lang.Thread.run(Thread.java:745)

When I commented the :subject and :body. I am not getting the error.

Thanks in advance

Riemann - trigger resolve based on metric threshold

I am trying to set up an alert in riemann (through pagerduty) based on a threshold for a metric. If the threshold is breached the alert should be triggered, if the metric goes back within the threshold the alert should be resolved.

My steps are: 1) Create an event with state "warning" if threshold is breached 2) Create an event with state "ok" if threshold is not breached

My code for this looks like -

(let [index (default :ttl 120 (index))]
   (streams
      index
      (where (service #"test")
         (where (>= metric 100)
            (smap (fn [e]
                    (event {:service (:service e) :metric (:metric e) 
                            :state "warning" }) 
               index))))

(I have only shown the relevant bits of code)

I see that this code does not create a new event if threshold is breached.

I am not sure if I am making a mistake. Any help would be appreciated.

Regards,

Sathya

How can I use clj-http in riemann.config

I use riemann and now I write my riemann.config.

I want to use clj-http post all events from riemann stream to my web server. But I don't know how to import clj-http from riemann.jar.

I code (:use clj-http.client) or (:require [clj-http.client :as client]) in riemann.config but got error:

java.lang.ClassNotFoundException: clj-http.client

Could anyone help me ?

Clojure Dashboard query

I am trying to show a graph on riemann-dashboard using query "pingDelay > 0" .

I already have indexed my data using following code

(let [index (index)]
  (defn write-dht-metric [e]
    (let [dhtstate (re-find #"dht_status: health\.(\S+), msg count (\d+) \((\d+) bytes\).*peak \{ping = (\d+)" (:pgmsg e))]
      (if (not= dhtstate nil)
        (do
          (prn "RESULT>" dhtstate)
          (index {:host "dht-info"
                  :service (:service e)
                  :time (unix-time)
                  :dhtStatus (get dhtstate 1)
                  :msgCount (get dhtstate 2)
                  :pingDelay (get dhtstate 3)}
            )
          )
        )
      )
    )
  )

However, I am not getting anything on graph. Earlier, I thought that perhaps its because my "pingDelay" is in string "12345", so, i also tried ":pingDelay #(Long. (get dhtstate 3))" without any success.

Can anyone please help me about what I must do to make it work?

Regards

Riemann Dashboard not outputting internal Riemann metrics

I'm new to Riemann and also new to ruby and Clojure as well. I'm trying to output the internal riemann events via (streams prn) in my riemann.config file. I currently see messages being printed out in the terminal from where I launched riemann.

Ex:

#riemann.codec.Event{:host "localhost.localdomain", :service "riemann server ws 0.0.0.0:5556 in latency 0.999", :state "ok", :description nil, :metric nil, :tags nil, :time 283857867047/200, :ttl 20}

In my dashboard however I'm unable to get these to print to any sort of log or gauge.

I tried using the following as a service =~ "riemann %" from here

I get an orange message displaying 1 socket errors; check the server field above or a large question market above the title.

Not sure what else to try or do from here to identify what went wrong.

ALerting in Riemann?

I am using ELK (logstash, ES, Kibana) stack for log analysis and Riemann for alerting. I have logs in which users is one of the fields parsed by logstash and I send the events to riemann from riemann output plugin.

Logstash parses logs and user is one of the field. Eg: logs parsed

Timestamp              user     command-name
 2014-06-07...         root      sh ./scripts/abc.sh
 2014-06-08...         sid       sh ./scripts/xyz.sh
 2014-06-08...         abc       sh ./scripts/xyz.sh
 2014-06-09...         root      sh ./scripts/xyz.sh

Logstash:

riemann {
    riemann_event => {
        "service"     => "logins"
        "unique_user" => "%{user}"
    }
}

So users values will be like: root, sid, abc, root, sid, def, etc....

So I split stream by user i.e one stream for each unique user. Now, I want to alert when number of unique users count go more than 3. I wrote the following but it's not achieving my purpose.

Riemann:

(streams

 (where (service "logins")
  (by :unique_user
    (moving-time-window 3600 
     (smap (fn [events]
      (let
        [users (count events)]
         (if (> users 3)
          (email "abc@gmail.com")       
     ))))))))

I am new to Riemann and clojure. Any help is appreciated.

clojure.lang.LazySeq cannot be cast to clojure.lang.IFn

I'm new to Riemann and Clojure. All I want to do is to send email notifications to three email groups when some service's TTL is expired. I created some sort of config file where I store a list of emails:

{
  :email_group_1 (
                  "first@example.com"
                  "second@example.ru"
                 )
  :email_group_2 (
                  "third@example.com"
                 )
}

My riemann config looks like this:

(logging/init {:console true})
(import org.apache.log4j.Level)
(logging/set-level Level/DEBUG)

(require '[clojure.java.io :as io])
(import '[java.io PushbackReader])

(let [host "0.0.0.0"]
  (tcp-server {:host host :port 60001})
  (udp-server {:host host})
  (ws-server  {:host host :port 60003}))
(repl-server  {:host "127.0.0.1"})

(def cwd (System/getProperty "user.dir"))

(def emails
  (with-open [r (io/reader (str cwd "/etc/emails.clj"))]
             (read (PushbackReader. r))))

(periodically-expire 5)

(def email (mailer))

(defn notify [& egroups]
  (for [egroup egroups]
    (rollup 1 60 (apply email (emails egroup)))))

(let [index (index)]
  (streams
    (default :ttl 60
      index

      (expired
          (where (service "service_connect_active")
                    #(info "expired" %)
                    (notify :email_group_1 :email_group_2))))))

Code looks good (for me), but when this service is expired I get the following error:

09:45:39 riemann.1      | INFO [2015-05-08 10:45:39,313] Thread-5 - riemann.config - expired {:ttl 60, :time 357766884827/250, :state expired, :service service_connect_active, :host ava.local}
09:45:39 riemann.1      | WARN [2015-05-08 10:45:39,319] Thread-5 - riemann.config - clojure.lang.LazySeq@841649b8 threw
09:45:39 riemann.1      | java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to clojure.lang.IFn
09:45:39 riemann.1      |   at riemann.config$eval66$stream__70$fn__75.invoke(riemann.development.config:34)
09:45:39 riemann.1      |   at riemann.config$eval66$stream__70.invoke(riemann.development.config:45)
09:45:39 riemann.1      |   at riemann.streams$match$stream__3514$fn__3525.invoke(streams.clj:1209)
09:45:39 riemann.1      |   at riemann.streams$match$stream__3514.invoke(streams.clj:1209)
09:45:39 riemann.1      |   at riemann.streams$default$stream__3731$fn__3742.invoke(streams.clj:1328)
09:45:39 riemann.1      |   at riemann.streams$default$stream__3731.invoke(streams.clj:1328)
09:45:39 riemann.1      |   at riemann.core$stream_BANG_$fn__4415.invoke(core.clj:19)
09:45:39 riemann.1      |   at riemann.core$stream_BANG_.invoke(core.clj:18)
09:45:39 riemann.1      |   at riemann.core$reaper$worker__4529$fn__4539.invoke(core.clj:303)
09:45:39 riemann.1      |   at riemann.core$reaper$worker__4529.invoke(core.clj:297)
09:45:39 riemann.1      |   at riemann.service.ThreadService$thread_service_runner__1973$fn__1974.invoke(service.clj:71)
09:45:39 riemann.1      |   at riemann.service.ThreadService$thread_service_runner__1973.invoke(service.clj:70)
09:45:39 riemann.1      |   at clojure.lang.AFn.run(AFn.java:22)
09:45:39 riemann.1      |   at java.lang.Thread.run(Thread.java:745)

Could someone please help me? Thanks.

Riemann - Build a stream dynamically from a map

I have the following function which gets a map with service name and threshold. It checks if the service crossed a defined threshold and then calls multiple downstream children on the event.

(defn tc
  [s & children]
   (where
     (and (service (:service_name s)) (not (expired? event)))
       (by [:host :service]
         (where (> metric (:threshold s)
           (with :state "critical" 
             (apply sdo children)))))))

I would like to build a stream dynamically using a vector of maps:

(def services [{:service "cpu/usage" :threshold 90}
               {:service "memory/usage" :threshold 90}])

When trying to run it in a stream i'm getting the following warning:

(streams
  (doseq [s services] (tc s prn)))

WARN [2015-01-05 14:27:07,187] Thread-15 - riemann.core - instrumentation service caught
java.lang.NullPointerException
  at riemann.core$stream_BANG_$fn__11140.invoke(core.clj:19)
  at riemann.core$stream_BANG_.invoke(core.clj:18)
  at riemann.core$instrumentation_service$measure__11149.invoke(core.clj:57)
  at riemann.service.ThreadService$thread_service_runner__8782$fn__8783.invoke(service.clj:66)
  at riemann.service.ThreadService$thread_service_runner__8782.invoke(service.clj:65)
  at clojure.lang.AFn.run(AFn.java:22)
  at java.lang.Thread.run(Thread.java:701)

It works, if i run the streams function inside the doseq. This one works and gives the following output:

(doseq [s services]
  (streams (tc s prn)))

#riemann.codec.Event{:host "testhost", :service "memory/usage", :state "critical", :description nil, :metric 91.0, :tags nil, :time 1420460856, :ttl 60.0}

Riemann: triggering alerts with changed-state

I'm new to Riemann and clojure. I'm trying to configure alerts based on changed states. But the states never seem to be updated/indexed. So when I get to the changed-state block, state is nil. I can add the alerts within the splitp block, but that seems redundant. Maybe we will want different types of notifications between critical and warnings, but for now, I'd like to see this work (if possible).

(let [index (default :ttl 20 (index))]
  (streams
    index
    (where (not (state "expired"))
      (where (service "load/load/shortterm")
        (splitp < metric
          0.05 (with :state "critical" index)
          0.02 (with :state "warning" index)
               (with :state "ok" index))
      )
      #(info %)
      (changed-state {:init "ok"}
        (stable 60 :state
          #(info "CHANGED STATE" %)
          (email "user@host.com")))
    )
  )
)

Thanks for any help!

Riemann v0.2.9, collectd v5.5.0, OS CentOS 6.5