SCTP Support in Mobicents Diameter 1.5.0.FINAL!

Tuesday, February 5, 2013 em 10:33
Mobicents Diameter 1.5.0.FINAL has been released last week (announcement) and as it's main highlight is the addition of Stream Control Transmission Protocol (SCTP, RFC 2960).

SCTP was designed to transport PSTN signaling messages over IP Networks, but become a general purpose protocol. It brings features from both UDP and TCP protocols with important additions. The two main improvements over traditional transport protocols are:
  • Multi-homing: instead of a single IP:port connection, an SCTP connection can make use of several IP addresses, if available at the endpoint, providing physical interface fault tolerance and higher availability.
  • Multi-streaming: While in TCP if there's a loss at the head of line, all subsequent packets must be delayed until a re-transmission (Head-of-Line Blocking), SCTP supports multiple streams (up to 64K) each one with it's own ordering, not causing a delay in other streams when there are losses.
It also provides many other interesting improvements (SYN-flood protection, preserved message boundaries, built-in heartbeat, etc.) that make SCTP a great transport protocol!

The integration of SCTP in Mobicents Diameter Stack was a contribution from Smilecoms with the Mobicents team support and by using the Mobicents SCTP Library. The requirements for using it are an operating system which supports SCTP (either natively or through third-party libraries) and JDK7.  

You can enable it in Mobicents Diameter by adding the following to your jdiameter-config.xml:
<?xml version="1.0"?>
<Configuration xmlns="http://www.jdiameter.org/jdiameter-server">
 ...
 <Network>
  <Peers>
   ...
   <-- For SCTP we have to add portRange, a port range to use for establishing connections -->
   <Peer name="aaa://127.0.0.1:1218" portRange="25000-35000" attempt_connect="false" rating="1" />
   ...
  </Peers>
  ...
 </Network>

 ...

 <Extensions>
  <-- Set SCTP classes as extension points for Connection and Network Guard -->
  <Connection value="org.jdiameter.client.impl.transport.sctp.SCTPClientConnection" />
  <NetworkGuard value="org.jdiameter.server.impl.io.sctp.NetworkGuard" />
 </Extensions>
</Configuration>

So, as you see in line 8, there's a new parameter that has to be added to the peer definition, which is the portRange. Such parameter is also available for TCP but it's usage is optional, while for SCTP it is mandatory as there's no wildcard value for picking any available port.

At lines 18 and 19, it's the extension point definition, so the stack uses SCTP connection client and Network Guard instead of the default TCP.

This is it.. as simple as it gets! Hope to get some feedback on this first version of Mobicents Diameter SCTP support. Feel free to leave comments on this post or at the feature issue.

Also, make sure to check all the other enhancements in Mobicents Diameter 1.5.0.FINAL!

Build your own Mobicents Dashboard in 15 Minutes!

Monday, December 17, 2012 em 12:03

The Mobicents suite is growing everyday, from the early days of the initial JAIN SLEE container project, a lot of new fronts have been going on, such as SIP Servlets, Diameter Stack, Media Server, SS7 Stack, Restcomm, etc.

With such a vaste suite of projects, the need for common management and monitoring tools is a major concern we have been facing, as each project having it's own set of tools is not a optimal solution. At least a common framework/interface is desired.

JMX is a standard for monitoring and managing JVMs, but it's connector uses RMI, which is not the sexiest protocol these days. In that sense, we have searched for a JMX <=> HTTP bridge to use a REST-like protocol as a frontend to the JMX server. This brought us to Jolokia. With such bridge, we can easily extend our choices of management frameworks outside Java. Plus, it brings many other goodies, such as being firewall friendly (HTTP is allowed anywhere), security (filter what is accessible), support for bulk requests, etc.

Installing Jolokia

Jolokia is very simple to install in the JBoss AS container:
  1. Download Jolokia from here. Select the binary package.
  2. Extract the zip file.
  3. Copy the jolokia.war from the agents sub-directory to your JBoss AS deploy directory.
  4. If the JBoss AS is not running already, start it.

You should now be able to issue HTTP requests to Jolokia agent. Using curl (or pointing your browser to the URL), you can do:
$ curl http://localhost:8080/jolokia/read/java.lang:type=Memory/HeapMemoryUsage/used

{"timestamp":1355453720,"status":200,"request":{"mbean":"java.lang:type=Memory","path":"used",
"attribute":"HeapMemoryUsage","type":"read"},"value":310667560}

This is how we read a value, returned in a JSON format, using Jolokia REST API. The format used is the following: <base-url>/read/<mbean name>/<attribute name>/<inner path>. You can learn more reading the Jolokia Reference Manual.

So, now that we have this working, lets make some good use of it. Graphs are the best way to show this data over time. Looking for a good JavaScript graphs library, there are several options such as Google Chart Tools, HighCharts, jqPlot, YUI Charts, etc. For this demo we will be using HighCharts. It is not completely free, but it is a very good and feature rich chart library, plus there's this great article by Tomasz Nurkiewicz to help us getting started.

So, let's start to build our dashboard. Create an HTML file in your favorite editor and, here we go:

<html>
 <body>
  <div id="chart" />

  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.js"></script>
  <script src="https://ajax.googleapis.com/ajax/libs/jqueryui/1.9.2/jquery-ui.js"></script>
  <script src="http://jolokia.org/dist/1.0.6/js/jolokia.js"></script>
  <script src="http://jolokia.org/dist/1.0.6/js/jolokia-simple.js"></script>
  <script src="http://code.highcharts.com/highcharts.src.js"></script>

  <script id="source" language="javascript" type="text/javascript">
   $(document).ready(function() {
    var jolokia = new Jolokia("http://localhost:8080/jolokia");

    var chart = new Highcharts.Chart({
     chart: {
      renderTo: 'chart',
      defaultSeriesType: 'spline',
      events: {
       load: function() {
        var series = this.series[0];
        setInterval(function() {
         var x = (new Date()).getTime();
         var memoryUsed = jolokia.getAttribute("java.lang:type=Memory", "HeapMemoryUsage", "used");
         series.addPoint({
          x: new Date().getTime(),
          y: parseInt(memoryUsed)
         }, true, series.data.length >= 50);
        }, 1000);
       }
      }
     },
     title: {
      text: 'HeapMemoryUsage'
     },
     xAxis: {
      type: 'datetime'
     },
     yAxis: {
      title: { text: 'HeapMemoryUsage' }
     },
     series: [{
      data: [],
      name: 'Used Memory'
     }]
    });
   });
  </script>
 </body>
</html>

These 50 lines of HTML/JavaScript code already produce a very nice chart:

This is as easy as it gets! We may now add a lot of other extras, make it more generic to handle several charts, several data sources per chart, make use of jQuery UI Sortable to make a portlet-like dashboard, where it is possible to group and arrange those charts as desired, etc. Ready? Let's go!

<html>
 <head>
  <link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.9.2/themes/black-tie/jquery-ui.css"/>
  <style>
   .column { width: 400px; float: left; padding-bottom: 10px; }
   .portlet { margin: 0 1em 1em 0; }
   .portlet-header { margin: 0.3em; padding-bottom: 4px; padding-left: 0.2em; }
   .portlet-header .ui-icon { float: right; }
   .portlet-content { padding: 0.4em; }
   .ui-sortable-placeholder { border: 1px dotted black; visibility: visible !important; height: 238px !important; }
   .ui-sortable-placeholder * { visibility: hidden; }
  </style>
 </head>

 <body style="background-color: #EEE; font-family: Verdana; font-size: small;">
  <!-- The template to be used for new portlets -->
  <div style="display: none;">
   <div class="portlet ui-widget-content ui-helper-clearfix ui-corner-all" id="portlet-template" style="">
    <div class="portlet-header ui-widget-header ui-corner-all">
     <span class='ui-icon ui-icon-minusthick'></span>
     <span class="title"> </span>
    </div>
    <div class="portlet-content"></div>
   </div>
  </div>

  <div><h2 style="text-align: center;">.:[ MOBICENTS DASHBOARD ]:.</h2></div>
  <hr />
  <div id="charts" class="column" /></div>

  <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.js"></script>
  <script src="https://ajax.googleapis.com/ajax/libs/jqueryui/1.9.2/jquery-ui.js"></script>
  <script src="http://jolokia.org/dist/1.0.6/js/jolokia.js"></script>
  <script src="http://jolokia.org/dist/1.0.6/js/jolokia-simple.js"></script>
  <script src="http://code.highcharts.com/highcharts.src.js"></script>

  <script id="source" language="javascript" type="text/javascript">
  $(document).ready(function() {
   jolokia = new Jolokia({url: "http://localhost:8080/jolokia", fetchInterval: 1000});

   var factory = new JmxChartsFactory();
    factory.create([
     {
      type: 'read',
      name: 'org.mobicents.slee:name=EventRouterStatistics',
      attribute: 'AverageEventRoutingTime'
     }
    ]);
    factory.create([
     {
      type: 'read',
      name: 'org.mobicents.slee:name=EventRouterStatistics',
      attribute: 'ActivitiesMapped'
     }
    ]);
    executors = []
    numExecutors = jolokia.getAttribute("org.mobicents.slee:name=EventRouterConfiguration","EventRouterThreads");
    for (var i = 0; i < numExecutors; i++) {
     executors[i] = {
      type: 'exec',
      name: 'org.mobicents.slee:name=EventRouterStatistics',
      operation: 'getAverageEventRoutingTime(int)',
      args: [i]
     }
    }
    factory.create(executors);
    factory.create([
     {
      type: 'read',
      name: 'java.lang:type=Memory',
      attribute: 'HeapMemoryUsage',
      path: 'committed'
     },
     {
      type: 'read',
      name: 'java.lang:type=Memory',
      attribute: 'HeapMemoryUsage',
      path: 'used'
     }
    ]);
    factory.create(
     {
      type: 'read',
      name: 'java.lang:type=OperatingSystem',
      attribute: 'SystemLoadAverage'
     }
    );
    factory.create(
     {
      type:  'read',
      name:  'java.lang:type=Threading',
      attribute: 'ThreadCount'
     }
    );
   });

   function JmxChartsFactory(keepHistorySec, pollInterval, columnsCount) {
    var series = [];
    var monitoredMbeans = [];
    var chartsCount = 0;

    // if not given a value for number of columns, use what fits.
    columnsCount = columnsCount || Math.floor($(window).width()/$(".column").width());
    // poll interval, defaults to 1000ms
    pollInterval = pollInterval || 1000;
    // how many data points to show in the graphs, defaults to 30
    var keepPoints = (keepHistorySec || 30) / (pollInterval / 1000);

    setupPortletsContainer(columnsCount);

    setInterval(function() {
     pollAndUpdateCharts();
    }, pollInterval);

    this.create = function(mbeans) {
     mbeans = $.makeArray(mbeans);
     series = series.concat(createChart(mbeans).series);
     monitoredMbeans = monitoredMbeans.concat(mbeans);
    };

    function pollAndUpdateCharts() {
     var requests = prepareBatchRequest();
     var responses = jolokia.request(requests);
     updateCharts(responses);
    }

    function createNewPortlet(name) {
     return $('#portlet-template')
       .clone(true)
       .appendTo($('.column')[chartsCount++ % columnsCount])
       .removeAttr('id')
       .find('.title').text((name.length > 50 ? '...' : '') + name.substring(name.length - 50, name.length)).end()
       .find('.portlet-content')[0];
    }

    function setupPortletsContainer() {
     var column = $('.column');
     for(var i = 1; i < columnsCount; ++i){
      column.clone().appendTo(column.parent());
     }
     $(".column").sortable({
      connectWith: ".column"
     });

     $(".portlet-header .ui-icon").click(function() {
      $(this).toggleClass("ui-icon-minusthick").toggleClass("ui-icon-plusthick");
      $(this).parents(".portlet:first").find(".portlet-content").toggle();
     });
     $(".column").disableSelection();
    }

    function prepareBatchRequest() {
     return $.map(monitoredMbeans, function(mbean) {
      switch(mbean.type) {
       case 'read':
        return {
         type: mbean.type,
         opts: mbean.args,
         mbean: mbean.name,
         attribute: mbean.attribute,
         path: mbean.path
        };
        break;
       case 'exec':
        return {
         type: mbean.type,
         arguments: mbean.args,
         mbean: mbean.name,
         operation: mbean.operation,
         path: mbean.path
        };
        break;
      }
     });
    }

    function updateCharts(responses) {
     var curChart = 0;
     $.each(responses, function() {
      var point = {
       x: this.timestamp * 1000,
       y: parseFloat(this.value)
      };
      var curSeries = series[curChart++];
      curSeries.addPoint(point, true, curSeries.data.length >= keepPoints);
     });
    }

    function createChart(mbeans) {
     return new Highcharts.Chart({
      chart: {
       renderTo: createNewPortlet(mbeans[0].name),
       height: 200,
       defaultSeriesType: 'spline',
      },
      title: { text: null },
      xAxis: { type: 'datetime' },
      yAxis: { title: { text: mbeans[0].attribute || mbeans[0].operation } },
      legend: {
       enabled: true,
       borderWidth: 0
      },
      credits: {enabled: false},
      series: $.map(mbeans, function(mbean) {
       return {
        data: [],
        name: mbean.path || mbean.attribute || mbean.args
       }
      })
     })
    }
   }
  </script>
 </body>
</html>

A bit more of code than previously, but with this we added 6 different graphs, some with more than one data series, we added portlet behavior... in my opinion, it is still quite simple for the output we get of it:

Is it cool or what? And all of this in a single static html file. Personally, I love it.

And this is the way we are heading with Mobicents Monitoring and Management... adding some extras such as thresholds with some kind of alarms/notifications, more customization like easily adding/removing charts at runtime through web interface, add some persistence to remember history and preferences, and many more to make this a solid tool to keep your Mobicents suite under control!

Mac OSX / Terminal Tip : Naming your terminal windows and tabs

Saturday, August 18, 2012 em 01:55
I'm a huge fan for those little tips that can save you some time and frustration.

Recently working with several terminal windows and tabs, I've been finding myself wandering around them to find the right terminal where I wanted to land on. All added up, it was a significant penalty in my "performance" and, even worse, made me mad!

I was already using different color schemes for the different terminal windows, by contexts, so I could quickly identify what was in each terminal once I see it. But since having 6+ terminal windows and colors is not practical, I group some in the same window, again, by context.


I've decided to look for a way to name the terminal window and/or tabs so I can quickly get to the one I needed. It's actually quite simple, just type (in the desired terminal tab):
echo -n -e "\033]0;Your Title Goes Here\007"
et voilà! Precious time saved!

Mobicents Diameter has a new home.. powered by git!

Wednesday, July 4, 2012 em 01:12
Due to the increasingly number and complexity of sub-projects, Mobicents has been splitting it's projects under independent project homes.

We are now at a transition stage, where each project lead will migrate his project at the appropriate time. For Diameter, after our major release (1.4.0.FINAL), it seemed the best time to do this migration.

The new home of the Mobicents Diameter project is located at http://code.google.com/p/jdiameter/. Feel free to visit us!

With this change, we have also taken the chance to move to a "better" and more powerful version control system, so we have changed from SVN to Git. We hope this will ease the contributions, which have been happening more often, thanks to out thriving community!

Despite being a allegedly better VCS, Git also has it's own shortcomings, such as not being able to checkout a single folder. The lack of this feature has impacted our structure, since it makes it impossible to independently release sub-components in the same Git repository.

While digging for a solution, I've came across the "sparse checkout" feature. While this may work for checking out only some folder(s), it is not supported by maven release plugin, thus not fixing the main problem. Another possible (and probably more correct) solution would be to use the "sub-repository" approach, where we'd split the components into different sub-repositories.

We actually did this on a first phase, but after realizing the big changes it would imply and being aware that google code messes with the automatic revision links (http://code.google.com/p/support/issues/detail?id=3245) we have decided to revert back to the single repository approach and abdicate from the release independency for each component. We have barely used it, anyway. The only exception is for the Jopr RHQ Plugin, so it got it's own sub-repository.

As a reference for someone going through the same process of moving from a [googlecode] SVN repository to a [googlecode] git repository, I'll leave a summary of the commands used for the move (please keep in mind that I'm totally new to git, so these may not be the optimal way, feel free to comment):

0. (optional) Since SVN only lists the username and git uses the email as well, the migration will cause weird authors such as "brainslog <brainslog@bf0df8d0-2c1f-0410-b170-bd30377b63dc>" it may be good to sanitize the authors. For that I've used the following script in the SVN repository:
svn log -q | awk -F '|' '/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}' | sort -u > authors.txt

And then I've fixed the outcome in authors.txt to the correct: username = Full Name .

1. Clone the new (and probably empty) repository
git clone https://@code.google.com/p/jdiameter/ git-jdiameter

2. Clone from SVN (with authors fixed) to the cloned git repository, with full history, tags and branches
git svn clone -A authors.txt -Ttrunk/servers/diameter -ttags/servers/diameter -bbranches/servers/diameter https://mobicents.googlecode.com/svn git-jdiameter

Where -A points to the authors file, -T to the trunk of the SVN repository, -t to the tags and -b to the branches. Use only what you need, I didn't needed the branches part as we don't have any.

3. Enter the local git repository
cd git-jdiameter

4. Push!
git push --all

5. Create the git tags from SVN tags (I know this can be automatized but I needed to make some customizations to existing tags, as not all were correct)
git tag "1.0.0.BETA1" "refs/remotes/tags/jdiameter-1.0.0.BETA1"
...
git tag "1.0.0.FINAL" "refs/remotes/tags/1.0.0.FINAL"

6. Push.. again!
git push --tags

These are the 7 magical steps! In case that you don't care about history, simply do a "svn export --force http://mobicents.googlecode.com/svn/trunk/servers/diameter/ git-jdiameter" and push it, nothing else. Also, as mentioned, I did not had branches but I suppose they'd make it in the #4 push.

Mobicents Diameter users working with trunk master, must now switch to this new repository as the SVN will not be updated anymore! Do a "git clone https://code.google.com/p/jdiameter/" and hack away!

Mobicents Diameter 1.4.0.FINAL Released!

Monday, June 4, 2012 em 17:47
The Mobicents Diameter 1.4.0.FINAL has been released today (announcement), and it represents the last mile in a long enhancement, refactoring and improvement journey. A lot of features have been added along the way:
  • Improved Cluster Support with fine-grained data per Diameter Application;
  • Support for more 3GPP Applications (Gq', Rx, Gx, S6a) and improved/updated existing;
  • Support for Diameter RELAY / PROXY / REDIRECT Agents;
  • Performance improvement of over 9 times (now handling up to 9000 requests/second in Dev Machine, C2D @ 3.06GHz / 4GB RAM, no special setup);
  • Added several configuration parameters to optimize for different scenarios (regular usage vs high load vs small footprint, etc);
  • 100+ miscellaneous fixes to improve overall compliance and stability of the Mobicents Diameter Stack and it's JAIN SLEE Resource Adaptors;
  • Also important, Mobicents (including Diameter) license has changed from GPL to LGPL, which is generally good news for the users!
Still, there's a lot of work and improvements to be done for the upcoming 1.4.x series. To name a few:
  • Support for TLS security in Mobicents Diameter Stack (existing in 1.4.0.FINAL as experimental);
  • Support for  SCTP transport in Mobicents Diameter Stack (initial patch already contributed);
  • Support pluggable Load Balancing algorithms for Mobicents Diameter Stack; Integrate with Mobicents Load Balancer;
  • Create integration/real-world examples, eg, integrate with Mobicents JAIN SLEE B2BUA, continue work on Mobicents HSS and integrate it with Mobicents SIP Presence;
  • Improve documentation for developers using Mobicents Diameter Stack;
As you can see, there's a lot that has been improved but there's also a lot more to improve on, with our ambition aiming higher. Feel free to help us with your feedback and contributions to a better Mobicents Diameter!

Mobicents Diameter 1.4.0.BETA2 is out!

Monday, July 11, 2011 em 13:16
It's been a while since the last Mobicents Diameter release, but it's finally here, the latest and greatest! This is a special release as it is the one with the most community contribution. Thanks to all the contributors for showing the value of open-source!

- Enhanced Stability: 30+ issues regarding stack functionality were  identified and fixed, providing a more stable and usable stack;
- Improved Compliance: Better and stricter compliance to specifications, resulting in a more compatible and strict stack;
- Gq' Application Support: Another valuable 3GPP application, Gq', is now supported both in Diameter Stack and in Mobicents JAIN SLEE, with it's Resource Adaptor. Thanks to Yulian Oifa for his contribution;
 - Better Cluster Support: Improved cluster support by updating to the latest Mobicents Cluster framework and fixed replication related issues; 
 - Experimental Agent Support: Initial support for experiments with supporting RELAY, PROXY and REDIRECT agents;
 - Extended Testsuite: Over 100 new JUnit tests were added to the testsuite in order to guarantee the best compliance and continuous regression testing.

Note: This is a BETA release as there are API and deep core changes happening, but it's been thoroughly tested, with the testsuite extended to cover more than ever, including message flows for all applications both in standalone and cluster mode!

Full release notes here.

Visit Mobicents Diameter website here.

Mobicents is now LGPL 2.1

Wednesday, March 2, 2011 em 12:16
After several discussions and shared thoughts between the Mobicents team and the community, it has been decided to align the Mobicents license with the remaining JBoss projects, changing to the LGPL 2.1 Open Source license!

Mobicents SIP Servlets was already using such license, but now it is applied to ALL Mobicents Projects (Diameter, JAIN SLEE, Media Server, SS7, etc), effective March 1, 2011.

We hope the community enjoys the change!