Build your own Mobicents Dashboard in 15 Minutes!

Monday, December 17, 2012 em 12:03

The Mobicents suite is growing everyday, from the early days of the initial JAIN SLEE container project, a lot of new fronts have been going on, such as SIP Servlets, Diameter Stack, Media Server, SS7 Stack, Restcomm, etc.

With such a vaste suite of projects, the need for common management and monitoring tools is a major concern we have been facing, as each project having it's own set of tools is not a optimal solution. At least a common framework/interface is desired.

JMX is a standard for monitoring and managing JVMs, but it's connector uses RMI, which is not the sexiest protocol these days. In that sense, we have searched for a JMX <=> HTTP bridge to use a REST-like protocol as a frontend to the JMX server. This brought us to Jolokia. With such bridge, we can easily extend our choices of management frameworks outside Java. Plus, it brings many other goodies, such as being firewall friendly (HTTP is allowed anywhere), security (filter what is accessible), support for bulk requests, etc.

Installing Jolokia

Jolokia is very simple to install in the JBoss AS container:
  1. Download Jolokia from here. Select the binary package.
  2. Extract the zip file.
  3. Copy the jolokia.war from the agents sub-directory to your JBoss AS deploy directory.
  4. If the JBoss AS is not running already, start it.

You should now be able to issue HTTP requests to Jolokia agent. Using curl (or pointing your browser to the URL), you can do:
$ curl http://localhost:8080/jolokia/read/java.lang:type=Memory/HeapMemoryUsage/used

{"timestamp":1355453720,"status":200,"request":{"mbean":"java.lang:type=Memory","path":"used",
"attribute":"HeapMemoryUsage","type":"read"},"value":310667560}

This is how we read a value, returned in a JSON format, using Jolokia REST API. The format used is the following: <base-url>/read/<mbean name>/<attribute name>/<inner path>. You can learn more reading the Jolokia Reference Manual.

So, now that we have this working, lets make some good use of it. Graphs are the best way to show this data over time. Looking for a good JavaScript graphs library, there are several options such as Google Chart Tools, HighCharts, jqPlot, YUI Charts, etc. For this demo we will be using HighCharts. It is not completely free, but it is a very good and feature rich chart library, plus there's this great article by Tomasz Nurkiewicz to help us getting started.

So, let's start to build our dashboard. Create an HTML file in your favorite editor and, here we go:

<html>
 <body>
  <div id="chart" />

  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.js"></script>
  <script src="https://ajax.googleapis.com/ajax/libs/jqueryui/1.9.2/jquery-ui.js"></script>
  <script src="http://jolokia.org/dist/1.0.6/js/jolokia.js"></script>
  <script src="http://jolokia.org/dist/1.0.6/js/jolokia-simple.js"></script>
  <script src="http://code.highcharts.com/highcharts.src.js"></script>

  <script id="source" language="javascript" type="text/javascript">
   $(document).ready(function() {
    var jolokia = new Jolokia("http://localhost:8080/jolokia");

    var chart = new Highcharts.Chart({
     chart: {
      renderTo: 'chart',
      defaultSeriesType: 'spline',
      events: {
       load: function() {
        var series = this.series[0];
        setInterval(function() {
         var x = (new Date()).getTime();
         var memoryUsed = jolokia.getAttribute("java.lang:type=Memory", "HeapMemoryUsage", "used");
         series.addPoint({
          x: new Date().getTime(),
          y: parseInt(memoryUsed)
         }, true, series.data.length >= 50);
        }, 1000);
       }
      }
     },
     title: {
      text: 'HeapMemoryUsage'
     },
     xAxis: {
      type: 'datetime'
     },
     yAxis: {
      title: { text: 'HeapMemoryUsage' }
     },
     series: [{
      data: [],
      name: 'Used Memory'
     }]
    });
   });
  </script>
 </body>
</html>

These 50 lines of HTML/JavaScript code already produce a very nice chart:

This is as easy as it gets! We may now add a lot of other extras, make it more generic to handle several charts, several data sources per chart, make use of jQuery UI Sortable to make a portlet-like dashboard, where it is possible to group and arrange those charts as desired, etc. Ready? Let's go!

<html>
 <head>
  <link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.9.2/themes/black-tie/jquery-ui.css"/>
  <style>
   .column { width: 400px; float: left; padding-bottom: 10px; }
   .portlet { margin: 0 1em 1em 0; }
   .portlet-header { margin: 0.3em; padding-bottom: 4px; padding-left: 0.2em; }
   .portlet-header .ui-icon { float: right; }
   .portlet-content { padding: 0.4em; }
   .ui-sortable-placeholder { border: 1px dotted black; visibility: visible !important; height: 238px !important; }
   .ui-sortable-placeholder * { visibility: hidden; }
  </style>
 </head>

 <body style="background-color: #EEE; font-family: Verdana; font-size: small;">
  <!-- The template to be used for new portlets -->
  <div style="display: none;">
   <div class="portlet ui-widget-content ui-helper-clearfix ui-corner-all" id="portlet-template" style="">
    <div class="portlet-header ui-widget-header ui-corner-all">
     <span class='ui-icon ui-icon-minusthick'></span>
     <span class="title"> </span>
    </div>
    <div class="portlet-content"></div>
   </div>
  </div>

  <div><h2 style="text-align: center;">.:[ MOBICENTS DASHBOARD ]:.</h2></div>
  <hr />
  <div id="charts" class="column" /></div>

  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.js"></script>
  <script src="https://ajax.googleapis.com/ajax/libs/jqueryui/1.9.2/jquery-ui.js"></script>
  <script src="http://jolokia.org/dist/1.0.6/js/jolokia.js"></script>
  <script src="http://jolokia.org/dist/1.0.6/js/jolokia-simple.js"></script>
  <script src="http://code.highcharts.com/highcharts.src.js"></script>

  <script id="source" language="javascript" type="text/javascript">
  $(document).ready(function() {
   jolokia = new Jolokia({url: "http://localhost:8080/jolokia", fetchInterval: 1000});

   var factory = new JmxChartsFactory();
    factory.create([
     {
      type: 'read',
      name: 'org.mobicents.slee:name=EventRouterStatistics',
      attribute: 'AverageEventRoutingTime'
     }
    ]);
    factory.create([
     {
      type: 'read',
      name: 'org.mobicents.slee:name=EventRouterStatistics',
      attribute: 'ActivitiesMapped'
     }
    ]);
    executors = []
    numExecutors = jolokia.getAttribute("org.mobicents.slee:name=EventRouterConfiguration","EventRouterThreads");
    for (var i = 0; i < numExecutors; i++) {
     executors[i] = {
      type: 'exec',
      name: 'org.mobicents.slee:name=EventRouterStatistics',
      operation: 'getAverageEventRoutingTime(int)',
      args: [i]
     }
    }
    factory.create(executors);
    factory.create([
     {
      type: 'read',
      name: 'java.lang:type=Memory',
      attribute: 'HeapMemoryUsage',
      path: 'committed'
     },
     {
      type: 'read',
      name: 'java.lang:type=Memory',
      attribute: 'HeapMemoryUsage',
      path: 'used'
     }
    ]);
    factory.create(
     {
      type: 'read',
      name: 'java.lang:type=OperatingSystem',
      attribute: 'SystemLoadAverage'
     }
    );
    factory.create(
     {
      type:  'read',
      name:  'java.lang:type=Threading',
      attribute: 'ThreadCount'
     }
    );
   });

   function JmxChartsFactory(keepHistorySec, pollInterval, columnsCount) {
    var series = [];
    var monitoredMbeans = [];
    var chartsCount = 0;

    // if not given a value for number of columns, use what fits.
    columnsCount = columnsCount || Math.floor($(window).width()/$(".column").width());
    // poll interval, defaults to 1000ms
    pollInterval = pollInterval || 1000;
    // how many data points to show in the graphs, defaults to 30
    var keepPoints = (keepHistorySec || 30) / (pollInterval / 1000);

    setupPortletsContainer(columnsCount);

    setInterval(function() {
     pollAndUpdateCharts();
    }, pollInterval);

    this.create = function(mbeans) {
     mbeans = $.makeArray(mbeans);
     series = series.concat(createChart(mbeans).series);
     monitoredMbeans = monitoredMbeans.concat(mbeans);
    };

    function pollAndUpdateCharts() {
     var requests = prepareBatchRequest();
     var responses = jolokia.request(requests);
     updateCharts(responses);
    }

    function createNewPortlet(name) {
     return $('#portlet-template')
       .clone(true)
       .appendTo($('.column')[chartsCount++ % columnsCount])
       .removeAttr('id')
       .find('.title').text((name.length > 50 ? '...' : '') + name.substring(name.length - 50, name.length)).end()
       .find('.portlet-content')[0];
    }

    function setupPortletsContainer() {
     var column = $('.column');
     for(var i = 1; i < columnsCount; ++i){
      column.clone().appendTo(column.parent());
     }
     $(".column").sortable({
      connectWith: ".column"
     });

     $(".portlet-header .ui-icon").click(function() {
      $(this).toggleClass("ui-icon-minusthick").toggleClass("ui-icon-plusthick");
      $(this).parents(".portlet:first").find(".portlet-content").toggle();
     });
     $(".column").disableSelection();
    }

    function prepareBatchRequest() {
     return $.map(monitoredMbeans, function(mbean) {
      switch(mbean.type) {
       case 'read':
        return {
         type: mbean.type,
         opts: mbean.args,
         mbean: mbean.name,
         attribute: mbean.attribute,
         path: mbean.path
        };
        break;
       case 'exec':
        return {
         type: mbean.type,
         arguments: mbean.args,
         mbean: mbean.name,
         operation: mbean.operation,
         path: mbean.path
        };
        break;
      }
     });
    }

    function updateCharts(responses) {
     var curChart = 0;
     $.each(responses, function() {
      var point = {
       x: this.timestamp * 1000,
       y: parseFloat(this.value)
      };
      var curSeries = series[curChart++];
      curSeries.addPoint(point, true, curSeries.data.length >= keepPoints);
     });
    }

    function createChart(mbeans) {
     return new Highcharts.Chart({
      chart: {
       renderTo: createNewPortlet(mbeans[0].name),
       height: 200,
       defaultSeriesType: 'spline',
      },
      title: { text: null },
      xAxis: { type: 'datetime' },
      yAxis: { title: { text: mbeans[0].attribute || mbeans[0].operation } },
      legend: {
       enabled: true,
       borderWidth: 0
      },
      credits: {enabled: false},
      series: $.map(mbeans, function(mbean) {
       return {
        data: [],
        name: mbean.path || mbean.attribute || mbean.args
       }
      })
     })
    }
   }
  </script>
 </body>
</html>

A bit more of code than previously, but with this we added 6 different graphs, some with more than one data series, we added portlet behavior... in my opinion, it is still quite simple for the output we get of it:

Is it cool or what? And all of this in a single static html file. Personally, I love it.

And this is the way we are heading with Mobicents Monitoring and Management... adding some extras such as thresholds with some kind of alarms/notifications, more customization like easily adding/removing charts at runtime through web interface, add some persistence to remember history and preferences, and many more to make this a solid tool to keep your Mobicents suite under control!

Mac OSX / Terminal Tip : Naming your terminal windows and tabs

Saturday, August 18, 2012 em 01:55
I'm a huge fan for those little tips that can save you some time and frustration.

Recently working with several terminal windows and tabs, I've been finding myself wandering around them to find the right terminal where I wanted to land on. All added up, it was a significant penalty in my "performance" and, even worse, made me mad!

I was already using different color schemes for the different terminal windows, by contexts, so I could quickly identify what was in each terminal once I see it. But since having 6+ terminal windows and colors is not practical, I group some in the same window, again, by context.


I've decided to look for a way to name the terminal window and/or tabs so I can quickly get to the one I needed. It's actually quite simple, just type (in the desired terminal tab):
echo -n -e "\033]0;Your Title Goes Here\007"
et voilà! Precious time saved!

Mobicents Diameter has a new home.. powered by git!

Wednesday, July 4, 2012 em 01:12
Due to the increasingly number and complexity of sub-projects, Mobicents has been splitting it's projects under independent project homes.

We are now at a transition stage, where each project lead will migrate his project at the appropriate time. For Diameter, after our major release (1.4.0.FINAL), it seemed the best time to do this migration.

The new home of the Mobicents Diameter project is located at http://code.google.com/p/jdiameter/. Feel free to visit us!

With this change, we have also taken the chance to move to a "better" and more powerful version control system, so we have changed from SVN to Git. We hope this will ease the contributions, which have been happening more often, thanks to out thriving community!

Despite being a allegedly better VCS, Git also has it's own shortcomings, such as not being able to checkout a single folder. The lack of this feature has impacted our structure, since it makes it impossible to independently release sub-components in the same Git repository.

While digging for a solution, I've came across the "sparse checkout" feature. While this may work for checking out only some folder(s), it is not supported by maven release plugin, thus not fixing the main problem. Another possible (and probably more correct) solution would be to use the "sub-repository" approach, where we'd split the components into different sub-repositories.

We actually did this on a first phase, but after realizing the big changes it would imply and being aware that google code messes with the automatic revision links (http://code.google.com/p/support/issues/detail?id=3245) we have decided to revert back to the single repository approach and abdicate from the release independency for each component. We have barely used it, anyway. The only exception is for the Jopr RHQ Plugin, so it got it's own sub-repository.

As a reference for someone going through the same process of moving from a [googlecode] SVN repository to a [googlecode] git repository, I'll leave a summary of the commands used for the move (please keep in mind that I'm totally new to git, so these may not be the optimal way, feel free to comment):

0. (optional) Since SVN only lists the username and git uses the email as well, the migration will cause weird authors such as "brainslog <brainslog@bf0df8d0-2c1f-0410-b170-bd30377b63dc>" it may be good to sanitize the authors. For that I've used the following script in the SVN repository:
svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors.txt

And then I've fixed the outcome in authors.txt to the correct: username = Full Name .

1. Clone the new (and probably empty) repository
git clone https://@code.google.com/p/jdiameter/ git-jdiameter

2. Clone from SVN (with authors fixed) to the cloned git repository, with full history, tags and branches
git svn clone -A authors.txt -Ttrunk/servers/diameter -ttags/servers/diameter -bbranches/servers/diameter https://mobicents.googlecode.com/svn git-jdiameter

Where -A points to the authors file, -T to the trunk of the SVN repository, -t to the tags and -b to the branches. Use only what you need, I didn't needed the branches part as we don't have any.

3. Enter the local git repository
cd git-jdiameter

4. Push!
git push --all

5. Create the git tags from SVN tags (I know this can be automatized but I needed to make some customizations to existing tags, as not all were correct)
git tag "1.0.0.BETA1" "refs/remotes/tags/jdiameter-1.0.0.BETA1"
...
git tag "1.0.0.FINAL" "refs/remotes/tags/1.0.0.FINAL"

6. Push.. again!
git push --tags

These are the 7 magical steps! In case that you don't care about history, simply do a "svn export --force http://mobicents.googlecode.com/svn/trunk/servers/diameter/ git-jdiameter" and push it, nothing else. Also, as mentioned, I did not had branches but I suppose they'd make it in the #4 push.

Mobicents Diameter users working with trunk master, must now switch to this new repository as the SVN will not be updated anymore! Do a "git clone https://code.google.com/p/jdiameter/" and hack away!

Mobicents Diameter 1.4.0.FINAL Released!

Monday, June 4, 2012 em 17:47
The Mobicents Diameter 1.4.0.FINAL has been released today (announcement), and it represents the last mile in a long enhancement, refactoring and improvement journey. A lot of features have been added along the way:
  • Improved Cluster Support with fine-grained data per Diameter Application;
  • Support for more 3GPP Applications (Gq', Rx, Gx, S6a) and improved/updated existing;
  • Support for Diameter RELAY / PROXY / REDIRECT Agents;
  • Performance improvement of over 9 times (now handling up to 9000 requests/second in Dev Machine, C2D @ 3.06GHz / 4GB RAM, no special setup);
  • Added several configuration parameters to optimize for different scenarios (regular usage vs high load vs small footprint, etc);
  • 100+ miscellaneous fixes to improve overall compliance and stability of the Mobicents Diameter Stack and it's JAIN SLEE Resource Adaptors;
  • Also important, Mobicents (including Diameter) license has changed from GPL to LGPL, which is generally good news for the users!
Still, there's a lot of work and improvements to be done for the upcoming 1.4.x series. To name a few:
  • Support for TLS security in Mobicents Diameter Stack (existing in 1.4.0.FINAL as experimental);
  • Support for  SCTP transport in Mobicents Diameter Stack (initial patch already contributed);
  • Support pluggable Load Balancing algorithms for Mobicents Diameter Stack; Integrate with Mobicents Load Balancer;
  • Create integration/real-world examples, eg, integrate with Mobicents JAIN SLEE B2BUA, continue work on Mobicents HSS and integrate it with Mobicents SIP Presence;
  • Improve documentation for developers using Mobicents Diameter Stack;
As you can see, there's a lot that has been improved but there's also a lot more to improve on, with our ambition aiming higher. Feel free to help us with your feedback and contributions to a better Mobicents Diameter!