How To Push

 

This page discusses pushing data from your application into Zabbix through a Zapcat trapper. It shows how to schedule JMX mbean queries to be performed periodically and how to send data to the Zabbix server directly.


For Java developers, pushing data may seem easier than diving into Zabbix’s configuration, but it is actually quite a bit more work than using the pull model. For one, you won’t escape having to spend some time learning to use Zabbix. On the contrary, you’ll have to know a little more about it than if you just use agents.




One of the main drawbacks of using this trapper is that it decentralises the configuration of your Zabbix monitoring system. No longer are all items configured in the Zabbix server, they are also configured in the trappers. Worse, these configurations must match precisely, or the data will not be received by the Zabbix server.


This tutorial introduces a number of features on the trapper, one at a time. We will look at sending data directly, sending the result of JMX queries, scheduling periodic JMX queries and trapping JMX notifications.


Here is the setup that we will be using. It looks a lot like the setup with agents, but it is not the same.


Notice that we use a Zapcat trapper instead of an agent. The connection between the agent and the Zabbix server is also reversed. The server does not poll the agent, but the agent pushes the data into the server.


The Zapcat Trapper

The Zapcat trapper is also in the Zapcat library that you used for the Zapcat agent. The Zapcat trapper is a little more forgiving in the kinds of data sources it supports. While the Zapcat agent only really understands JMX queries, you can use the trapper to send any data you have. The only limitation if that scheduled queries are always JMX queries.


Here is the code that demonstrates the use of a trapper to send the application’s start-up time to the Zabbix server. This is used to create a log in Zabbix of the restart times of your application.

import org.kjkoster.zapcat.Trapper;

import org.kjkoster.zapcat.zabbix.ZabbixTrapper;


public class SampleTrapper {

    public static void main(String[] args) throws Exception {

        Trapper trapper = null;

        try {

            trapper = new ZabbixTrapper(“192.168.0.150”,

                                    "Zapcat test application");

           

            trapper.send("java.version",

                           System.getProperty("java.version"));


            // simulate lots of important work being done...

            Thread.sleep(Long.MAX_VALUE);

        } catch (Exception e) {

            e.printStackTrace();

        } finally {

            trapper.stop();

        }

    }

}


You will have to change the IP address of the Zabbix server into the IP address of your own Zabbix server. Once you are confident that you can send data from Java to Zabbix, you can start adding different data items and use different data types.


Receiving Data in Zabbix

If you have not done so already, install Zabbix. There are many installation instructions on the Internet, so I won’t repeat them here. We will configure a very simple trapper in Zabbix, on its own host name. If you are a long time Zabbix user, you will probably already have your own way of organising the data items. Please use your own organisation in favour of my suggestions here.


Log into Zabbix as administrator. Then select ‘Configuration’, ‘Hosts’ and press ‘Create Host’. If your Java application runs on a machine that already has a working Zabbix agent, you may choose to skip this step.


In the host configuration screen, enter the name of the host. Use the exact name that you chose to use in the Java code for the <host> element in the XML fragment. In my case that would be ‘mac.kjkoster.org’. You can safely ignore the rest of the configuration items for the purpose of this tutorial. Save the new host definition.


The next step is to add the data item to the host. To do so, choose ‘Configuration’ and ‘Items’. Here you can configure the individual data items on all hosts.


From the ‘Group’ selector, pick the group ‘all’ and from the ‘Host’ selector, pick the host you just added. Press ‘Create Item’ to begin adding the new data item to the host.


The data item configuration screen shows and hides fields as you change values, so if the data item configuration screen looks slightly different from what you see in the screen shot, don’t worry. Just fill in the fields from top to bottom and you will be all-right. Once you are done, press ‘Save’ to save the data item.


The ‘Type’ field tells Zabbix how the data for this item is obtained. By setting it to ‘ZABBIX trapper’, Zabbix knows that the application will push the data into the server somehow. It won’t try to go out and find the data.


The ‘key’ field must be the same as the <key> element that we used in the Java code. In my code, I have used ‘java.version’ as the key.


Finally, the ‘Type of information’ gives Zabbix a hint how to treat the data. In this case, we just want to have the raw character data when we display it. If you want to use numerical data, for example the amount of memory in use, you’d use a different type here.


That is it. Rerun you Java application and see if the server responds with ‘OK’ this time.


If it still responds with ‘NOT OK’, check that the host and the data item are configured precisely as shown in the screen shots. In the host overview, check that your host has the status ‘Monitored’.


In the item overview, check that you added the item to the correct host, check the key and check that the status of the item is ‘Active’.


If all goes well, you should now be able to view the data that you pushed into Zabbix. Select ‘Monitoring’ and ‘Latest data’. From the ‘Group’ selector, pick the group ‘all’ and from the ‘Host’ selector, pick your host. You can click on the link marked ‘History’ to get to the data points of this data item.


You should now see something similar to the screen shot below. Run the program a few times to see what happens.


Sending raw data like this is probably most useful in situations where you have data that does not change much, or when you need to precisely record the time of a certain event. For example, you could send the order count of the nightly batch of orders after the batch was processed.


Sending raw data is not really useful beyond some very specific use cases. Most interesting data is not only interesting when logged in Zabbix, but also when a system administrator hooks up jconsole to an application to peek inside. If we exposed all interesting data using JMX, we could query it using through Zabbix, through jconsole or through any one of many JMX enabled tools.


Sending JMX Data

Instead of manually specifying the data to be sent, we can give the trapper a JMX query to perform. The trapper will then send the result of the JMX query to the Zabbix server.


We might for example want to send the type of JIT compiler to the Zabbix server. Since the JIT cannot be dynamically changed, we only have to send this information once when the JVM is started.


The code to do so is shown below. Instead of the actual data, we specify the object name and the attribute name of the data that we want to send. The trapper looks up the mbean that listens to the object name and reads the specified attribute. The value of that attribute is sent to the Zabbix server.

import org.kjkoster.zapcat.Trapper;

import org.kjkoster.zapcat.zabbix.ZabbixTrapper;


public class SampleTrapper {

    public static void main(String[] args) throws Exception {

        Trapper trapper = null;

        try {

            trapper = new ZabbixTrapper(“192.168.0.150”,

                                    "Zapcat test application");

           

            trapper.send("compiler.name",

                         "java.lang:type=Compilation", "Name");


            // simulate lots of important work being done...

            Thread.sleep(Long.MAX_VALUE);

        } catch (Exception e) {

            e.printStackTrace();

        } finally {

            trapper.stop();

        }

    }

}

This code requires that you add another item to the Zabbix configuration, shown below.

Scheduling JMX Queries

Most monitoring data is not interesting when you only have a single value. We probably want to look at how a value developed and changed over time. Of course, you’d like to avoid having to write a lot of boilerplate code with threads and timers.


Zapcat supports scheduled JMX queries, as shown in the code below. Here, we instruct the trapper to send the value of the compiler’s execution time to the Zabbix server. The trapper performs this query every 30 seconds.

import org.kjkoster.zapcat.Trapper;

import org.kjkoster.zapcat.zabbix.ZabbixTrapper;


public class SampleTrapper {

    public static void main(String[] args) throws Exception {

        Trapper trapper = null;

        try {

            trapper = new ZabbixTrapper(“192.168.0.150”,

                                    "Zapcat test application");

           

            trapper.every(30, TimeUnit.SECONDS,

                          "compiler.time",

                          "java.lang:type=Compilation",

                          "TotalCompilationTime");


            // simulate lots of important work being done...

            Thread.sleep(Long.MAX_VALUE);

        } catch (Exception e) {

            e.printStackTrace();

        } finally {

            trapper.stop();

        }

    }

}

And for this compiler time too we need an item in Zabbix. We’re dealing with milliseconds again, so we employ a custom multiplier to convert the values to seconds in Zabbix.

Running the sample code for a few minutes gives us a nice graph in Zabbix.


Configuration

The address of the Zabbix server, its port number and the configured host name are hard coded in the Java sources. That’s fine for smaller installations. For larger systems systems, system administrators will want to configure these settings in a way that does not involve recompiling the whole binary.


The hard coded values may be overridden from the command line using the arguments shown below.

-Dorg.kjkoster.zapcat.zabbix.server=zabbix.kjkoster.org

-Dorg.kjkoster.zapcat.zabbix.serverport=12345

-Dorg.kjkoster.zapcat.zabbix.host=appserver8


These sample settings override the values in the code and make the trapper connect to the Zabbix server named ‘zabbix.kjkoster.org’ on port number 12345. Instead of the host key in the code it will use ‘appserver8’.


Troubleshooting

The first mistake I made when I started pushing data into the Zabbix server was to specify the wrong port number. I had accidently told my trapper to push data into the Zabbix agent that also runs on my Zabbix machine. Of course, the Zabbix agent did not know what to do with the data. Here is what the error message looks like.

20071209 220105,726 org.kjkoster.zapcat.zabbix.Sender WARN - ignoring exception

java.net.SocketTimeoutException: Read timed out

    at java.net.SocketInputStream.socketRead0(Native Method)

    at java.net.SocketInputStream.read(SocketInputStream.java:129)

    at java.net.SocketInputStream.read(SocketInputStream.java:90)

    at org.kjkoster.zapcat.zabbix.Sender.send(Sender.java:145)

    at org.kjkoster.zapcat.zabbix.Sender.run(Sender.java:99)


The fix is easy, just specify the port number that your Zabbix server is listening to. That is most likely port 10051, which is the default port number for Zabbix server. Port number 10050 is the default port of Zabbix agents and port 10052 is the default port for Zapcat agents.


Another problem is that I confused the name of the Zabbix server and the host name. The screen shot below shows the distinction.


The zabbix server is the machine or IP address of the computer that you have Zabbix installed on. You use the same values for all your Java applications that are monitored by your Zabbix machine.


The host key is the name of the Java application as it appears in the user interface of Zabbix. It is different for each of the Java applications. It is important that you use the exact same host key in Zabbix as in your application. The host key is case sensitive.


This concludes the data pushing tutorial.

 
photo: Carsten Müllerhttp://www.sxc.hu/profile/dermiller
if you find this interesting, terrible or just would like to know more, e-mail memailto:kjkoster@kjkoster.org?subject=
web statistics
download zapcathttp://sourceforge.net/project/showfiles.php?group_id=209024
http://java-monitor.com/