Category Archives: IMS / VoLTE

Diameter Routing Agents – Part 4 – Advanced FreeDiameter DRA Routing

Way back in part 2 we discussed the basic routing logic a DRA handles, but what if we want to do something a bit outside of the box in terms of how we route?

For me, one of the most useful use cases for a DRA is to route traffic based on IMSI / Username.
This means I can route all the traffic for MVNO X to MVNO X’s HSS, or for staging / test subs to the test HSS enviroment.

FreeDiameter has a bunch of built in logic that handles routing based on a weight, but we can override this, using the rt_default module.

In our last post we had this module commented out, but let’s uncomment it and start playing with it:

#Basic Diameter config for this box
Identity = "";
Realm = "";
Port = 3868;

LoadExtension = "dbg_msg_dumps.fdx" : "0x8888";
LoadExtension = "rt_redirect.fdx":"0x0080";
LoadExtension = "rt_default.fdx":"rt_default.conf";

TLS_Cred = "/etc/freeDiameter/cert.pem", "/etc/freeDiameter/privkey.pem";
TLS_CA = "/etc/freeDiameter/cert.pem";
TLS_DH_File = "/etc/freeDiameter/dh.pem";

ConnectPeer = "" { ConnectTo = ""; No_TLS; };
ConnectPeer = "hss01" { ConnectTo = ""; No_TLS; Port = 3868; Realm = "";};
ConnectPeer = "hss02" { ConnectTo = ""; No_TLS; Port = 3868; Realm = "";};
ConnectPeer = "hss-mvno-x" { ConnectTo = ""; No_TLS; Port = 3868; Realm = "";};
ConnectPeer = "hss-lab" { ConnectTo = ""; No_TLS; Port = 3868; Realm = "";};

In the above code we’ve uncommented rt_default and rt_redirect.

You’ll notice that rt_default references a config file, so we’ll create a new file in our /etc/freeDiameter directory called rt_default.conf, and this is where the magic will happen.

A few points before we get started:

  • This overrides the default routing priorities, but in order for a peer to be selected, it has to be in an Open (active) state
  • The peer still has to have advertised support for the requested application in the CER/CEA dialog
  • The peers will still need to have all been defined in the freeDiameter.conf file in order to be selected

So with that in mind, and the 5 peers we have defined in our config above (assuming all are connected), let’s look at some rules we can setup using rt_default.

Intro to rt_default Rules

The rt_default.conf file contains a list of rules, each rule has a criteria that if matched, will result in the specified action being taken. The actions all revolve around how to route the traffic.

So what can these criteria match on?
Here’s the options:

Item to MatchCode
rt_default Matching Criteria

We can either match based on a string or a regex, for example, if we want to match anything where the Destination-Realm is “” we’d use something like:

#Low score to HSS02
dr="" : dh="hss02" += -70 ;

Now you’ll notice there is some stuff after this, let’s look at that.

We’re matching anything where the destination-host is set to hss02 (that’s the bit before the colon), but what’s the bit after that?

Well if we imagine that all our Diameter peers are up, when a message comes in with Destination-Realm “”, looking for an HSS, then in our example setup, we have 4 HHS instances to choose from (assuming they’re all online).

In default Diameter routing, all of these peers are in the same realm, and as they’re all HSS instances, they all support the same applications – Our request could go to any of them.

But what we set in the above example is simply the following:

If the Destination-Realm is set to, then set the priority for routing to hss02 to the lowest possible value.

So that leaves the 3 other Diameter peers with a higher score than HSS02, so HSS02 won’t be used.

Let’s steer this a little more,

Let’s specify that we want to use HSS01 to handle all the requests (if it’s available), we can do that by adding a rule like this:

#Low score to HSS02
dr="" : dh="hss02" += -70 ;
#High score to HSS01
dr="" : dh="hss01" += 100 ;

But what if we want to route to hss-lab if the IMSI matches a specific value, well we can do that too.

#Low score to HSS02
dr="" : dh="hss02" += -70 ;
#High score to HSS01
dr="" : dh="hss01" += 100 ;
#Route traffic for IMSI to Lab HSS
un="001019999999999999" : dh="hss-lab" += 200 ;

Now that we’ve set an entry with a higher score than hss01 that will be matched if the username (IMSI) equals 001019999999999999, the traffic will get routed to hss-lab.

But that’s a whole IMSI, what if we want to match only part of a field?

Well, we can use regex in the Criteria as well, so let’s look at using some Regex, let’s say for example all our MVNO SIMs start with 001012xxxxxxx, let’s setup a rule to match that, and route to the MVNO HSS with a higher priority than our normal HSS:

#Low score to HSS02
dr="" : dh="hss02" += -70 ;
#High score to HSS01
dr="" : dh="hss01" += 100 ;
#Route traffic for IMSI to Lab HSS
un="001019999999999999" : dh="hss-lab" += 200 ;
#Route traffic where IMSI starts with 001012 to MVNO HSS
un=["^001012.*"] : dh="hss-mvno-x" += 200 ;

Let’s imagine that down the line we introduce HSS03 and HSS04, and we only want to use HSS01 if HSS03 and HSS04 are unavailable, and only to use HSS02 no other HSSes are available, and we want to split the traffic 50/50 across HSS03 and HSS04.

Firstly we’d need to add HSS03 and HSS04 to our FreeDiameter.conf file:

ConnectPeer = "hss02" { ConnectTo = ""; No_TLS; Port = 3868; Realm = "";};
ConnectPeer = "hss03" { ConnectTo = ""; No_TLS; Port = 3868; Realm = "";};
ConnectPeer = "hss04" { ConnectTo = ""; No_TLS; Port = 3868; Realm = "";};

Then in our rt_default.conf we’d need to tweak our scores again:

#Low score to HSS02
dr="" : dh="hss02" += 10 ;
#Medium score to HSS01
dr="" : dh="hss01" += 20 ;
#Route traffic for IMSI to Lab HSS
un="001019999999999999" : dh="hss-lab" += 200 ;
#Route traffic where IMSI starts with 001012 to MVNO HSS
un=["^001012.*"] : dh="hss-mvno-x" += 200 ;
#High Score for HSS03 and HSS04
dr="" : dh="hss02" += 100 ;
dr="" : dh="hss04" += 100 ;

One quick tip to keep your logic a bit simpler, is that we can set a variety of different values based on keywords (listed below) rather than on a weight/score:

Do not deliver to peer (set lowest priority)NO_DELIVERY-70
The peer is a default route for all messagesDEFAULT5
The peer is a default route for this realmDEFAULT_REALM10
Route to the specified Host with highest priorityFINALDEST100
Rather than manually specifying the store you can use keywords like above to set the value

In our next post we’ll look at using FreeDiameter based DRA in roaming scenarios where we route messages across Diameter Realms.

Diameter Routing Agents – Part 3 – Building a DRA with FreeDiameter

I’ve covered the basics of Diameter Routing Agents (DRAs) in the past, and even shared an unstable DRA built using Kamailio, but today I thought I’d cover building something a little more “production ready”.

FreeDiameter has been around for a while, and we’ve covered configuring the FreeDiameter components in Open5GS when it comes to the S6a interface, so you may have already come across FreeDiameter in the past, but been left a bit baffled as to how to get it to actually do something.

FreeDiameter is a FOSS implimentation of the Diameter protocol stack, and is predominantly used as a building point for developers to build Diameter applications on top of.

But for our scenario, we’ll just be using plain FreeDiameter.

So let’s get into it,

You’ll need FreeDiameter installed, and you’ll need a certificate for your FreeDiameter instance, more on that in this post.

Once that’s setup we’ll need to define some basics,

Inside freeDiameter.conf we’ll need to include the identity of our DRA, load the extensions and reference the certificate files:

#Basic Diameter config for this box
Identity = "";
Realm = "";
Port = 3868;

LoadExtension = "dbg_msg_dumps.fdx" : "0x8888";
#LoadExtension = "rt_redirect.fdx":"0x0080";
#LoadExtension = "rt_default.fdx":"rt_default.conf";

TLS_Cred = "/etc/freeDiameter/cert.pem", "/etc/freeDiameter/privkey.pem";
TLS_CA = "/etc/freeDiameter/cert.pem";
TLS_DH_File = "/etc/freeDiameter/dh.pem";

Next up we’ll need to define the Diameter peers we’ll be routing between.

For each connection / peer / host we’ll need to define here:

ConnectPeer = "" { ConnectTo = ""; No_TLS; };
ConnectPeer = "hss01" { ConnectTo = ""; No_TLS; Port = 3868; Realm = "";};

And we’ll configure our HSS and MME defined in the ConnectPeers to connect/accept connections from,

Now if we start freeDiameter, we can start routing between the hosts. No config needed.

If we define another HSS in the ConnectPeers, any S6a requests from the MME may get routed to that as well (50/50 split).

In our next post, we’ll look at using the rt_default extension to control how we route and look at some more advanced use cases.

Diameter Routing Agents (Why you need them, and how to build them) – Part 2 – Routing

What I typically refer to as Diameter interfaces / reference points, such as S6a, Sh, Sx, Sy, Gx, Gy, Zh, etc, etc, are also known as Applications.

Diameter Application Support

If you look inside the Capabilities Exchange Request / Answer dialog, what you’ll see is each side advertising the Applications (interfaces) that they support, each one being identified by an Application ID.

CER showing support for the 3GPP Zh Application-ID (Interface)

If two peers share a common Application-Id, then they can communicate using that Application / Interface.

For example, the above screenshot shows a peer with support for the Zh Interface (Spoiler alert, XCAP Gateway / BSF coming soon!). If two Diameter peers both have support for the Zh interface, then they can use that to send requests / responses to each other.

This is the basis of Diameter Routing.

Diameter Routing Tables

Like any router, our DRA needs to have logic to select which peer to route each message to.

For each Diameter connection to our DRA, it will build up a Diameter Routing table, with information on each peer, including the realm and applications it advertises support for.

Then, based on the logic defined in the DRA to select which Diameter peer to route each request to.

In its simplest form, Diameter routing is based on a few things:

  1. Look at the DestinationRealm, and see if we have any peers at that realm
  2. If we do then look at the DestinationHost, if that’s set, and the host is connected, and if it supports the specified Application-Id, then route it to that host
  3. If no DestinationHost is specified, look at the peers we have available and find the one that supports the specified Application-Id, then route it to that host
Simplified Diameter Routing Table used by DRAs

With this in mind, we can go back to looking at how our DRA may route a request from a connected MME towards an HSS.

Let’s look at some examples of this at play.

The request from MME02 is for DestinationRealm, which our DRA knows it has 4 connected peers in (3 if we exclude the source of the request, as we don’t want to route it back to itself of course).

So we have 3 contenders still for who could get the request, but wait! We have a DestinationHost specified, so the DRA confirms the host is available, and that it supports the requested ApplicationId and routes it to HSS02.

So just because we are going through a DRA does not mean we can’t specific which destination host we need, just like we would if we had a direct link between each Diameter peer.

Conversely, if we sent another S6a request from MME01 but with no DestinationHost set, let’s see how that would look.

Again, the request is from MME02 is for DestinationRealm, which our DRA knows it has 3 other peers it could route this to. But only two of those peers support the S6a Application, so the request would be split between the two peers evenly.

Clever Routing with DRAs

So with our DRA in place we can simplify the network, we don’t need to build peer links between every Diameter device to every other, but let’s look at some other ways DRAs can help us.

Load Control

We may want to always send requests to HSS01 and only use HSS02 if HSS01 is not available, we can do this with a DRA.

Or we may want to split load 75% on one HSS and 25% on the other.

Both are great use cases for a DRA.

Routing based on Username

We may want to route requests in the DRA based on other factors, such as the IMSI.

Our IMSIs may start with 001010001xxx, but if we introduced an MVNO with IMSIs starting with 001010002xxx, we’d need to know to route all traffic where the IMSI belongs to the home network to the home network HSS, and all the MVNO IMSI traffic to the MVNO’s HSS, and DRAs handle this.

Inter-Realm Routing

One of the main use cases you’ll see for DRAs is in Roaming scenarios.

For example, if we have a roaming agreement with a subscriber who’s IMSIs start with 90170, we can route all the traffic for their subs towards their HSS.

But wait, their Realm will be, so in that scenario we’ll need to add a rule to route the request to a different realm.

DRAs handle this also.

In our next post we’ll start actually setting up a DRA with a default route table, and then look at some more advanced options for Diameter routing like we’ve just discussed.

One slight caveat, is that mutual support does not always mean what you may expect.
For example an MME and an HSS both support S6a, which is identified by Auth-Application-Id 16777251 (Vendor ID 10415), but one is a client and one is a server.
Keep this in mind!

Diameter Routing Agents (Why you need them, and how to build them) – Part 1

Answer Question 1: Because they make things simpler and more flexible for your Diameter traffic.
Answer Question 2: With free software of course!

All about DRAs

But let’s dive a little deeper. Let’s look at the connection between an MME and an HSS (the S6a interface).

Direct Diameter link between two Diameter Peers

We configure the Diameter peers on MME1 and HSS01 so they know about each other and how to communicate, the link comes up and presto, away we go.

But we’re building networks here! N+1 redundancy and all that, so now we have two HSSes and two MMEs.

Direct Diameter link between 4 Diameter peers

Okay, bit messy, but that’s okay…

But then our network grows to 10 MMEs, and 3 HSSes and you can probably see where this is going, but let’s drive the point home.

Direct Diameter connections for a network with 10x MME and 3x HSS

Now imagine once you’ve set all this up you need to do some maintenance work on HSS03, so need to shut down the Diameter peer on 10 different MMEs in order to isolate it and deisolate it.

The problem here is pretty evident, all those links are messy, cumbersome and they just don’t scale.

If you’re someone with a bit of networking experience (and let’s face it, you’re here after all), then you’re probably thinking “What if we just had a central system to route all the Diameter messages?”

An Agent that could Route Diameter, a Diameter Routing Agent perhaps…

By introducing a DRA we build Diameter peer links between each of our Diameter devices (MME / HSS, etc) and the DRA, rather than directly between each peer.

Then from the DRA we can route Diameter requests and responses between them.

Let’s go back to our 10x MME and 3x HSS network and see how it looks with a DRA instead.

So much cleaner!

Not only does this look better, but it makes our life operating the network a whole lot easier.

Each MME sends their S6a traffic to the DRA, which finds a healthy HSS from the 3 and sends the requests to it, and relays the responses as well.

We can do clever load balancing now as well.

Plus if a peer goes down, the DRA detects the failure and just routes to one of the others.

If we were to introduce a new HSS, we wouldn’t need to configure anything on the MMEs, just add HSS04 to the DRA and it’ll start getting traffic.

Plus from an operations standpoint, now if we want to to take an HSS offline for maintenance, we just shut down the link on the HSS and all HSS traffic will get routed to the other two HSS instances.

In our next post we’ll talk about the Routing part of the DRA, how the decisions are made and all the nuances, and then in the following post we’ll actually build a DRA and start routing some traffic around!

Filtering for 3GPP DNS in Wireshark

If you work with IMS or Packet Core, there’s a good chance you need DNS to work, and it doesn’t always.

When I run traces, I’ve always found I get swamped with DNS traffic, UE traffic, OS monitoring, updates, etc, all combine into a big firehose – while my Wireshark filters for finding EPC and IMS traffic is pretty good, my achilles heel has always been filtering the DNS traffic to just get the queries and responses I want out of it.

Well, today I made that a bit better.

By adding this to your Wireshark filter:

dns contains 33:67:70:70:6e:65:74:77:6f:72:6b:03:6f:72:67:00

You’ll only see DNS Queries and Responses for domains at the domain.

This makes my traces much easier to read, and hopefully will do the same for you!

Bonus, here’s my current Wireshark filter for working EPC/IMS:

(diameter and diameter.cmd.code != 280) or  (sip and !(sip.Method == "OPTIONS") and !(sip.CSeq.method == "OPTIONS")) or (smpp and (smpp.command_id != 0x00000015 and smpp.command_id != 0x80000015)) or (mgcp and !(mgcp.req.verb == "AUEP") and !(mgcp.rsp.rspcode == 500)) or isup or sccp or rtpevent or s1ap or gtpv2 or pfcp or (dns contains 33:67:70:70:6e:65:74:77:6f:72:6b:03:6f:72:67:00)

FreeDiameter – Generating Certificates

Even if you’re not using TLS in your FreeDiameter instance, you’ll still need a certificate in order to start the stack.

Luckily, creating a self-signed certificate is pretty simple,

Firstly we generate your a private key and public certificate for our required domain – in the below example I’m using, but you’ll need to replace that with the domain name of your freeDiameter instance.

openssl req -new -batch -x509 -days 3650 -nodes     \
   -newkey rsa:1024 -out /etc/freeDiameter/cert.pem -keyout /etc/freeDiameter/privkey.pem \
   -subj /

Next we generate a new set of Diffie-Hellman parameter set using OpenSSL.

openssl dhparam -out /etc/freeDiameter/dh.pem 1024 

Lastly we’ll put all this config into the freeDiameter config file:

TLS_Cred = "/etc/freeDiameter/cert.pem", "/etc/freeDiameter/privkey.pem";
TLS_CA = "/etc/freeDiameter/cert.pem";
TLS_DH_File = "/etc/freeDiameter/dh.pem";

If you’re using freeDiameter as part of another software stack (Such as Open5Gs) the below filenames will contain the config for that particular freeDiameter components of the stack:

  • freeDiameter.conf – Vanilla freeDiameter
  • mme.conf – Open5Gs MME
  • pcrf.conf – Open5Gs PCRF
  • smf.conf – Open5Gs SMF / P-GW-C
  • hss.conf – Open5Gs HSS

Sangoma Transcoding Cards Setup

The Wiki on the Sangoma documentation page is really out of date and can’t be easily edited by the public, so here’s the skinny on how to setup a Sangoma transcoding card on a modern Debian system:

apt-get install libxml2* wget make gcc
tar xzf sng-tc-linux-1.3.11.x86_64.tgz
cd sng-tc-linux-1.3.11.x86_64/
make install
cp lib/* /usr/local/lib/

At this point you should be able to check for the presence of the card with:

sngtc_tool -dev ens33 -list_modules

Where ens33 is the name of the NIC that the server that shares a broadcast domain with the transcoder.

Successfully discovering the Sangoma D150 transcoder

If instead you see something like this:

root@fs-131:/etc/sngtc#  sngtc_tool -dev ens33 -list_modules
Failed to detect and initialize modules with size 1

That means the server can’t find the transcoding device. If you’re using a D150 (The Ethernet enabled versions) then you’ve got to make sure that the NIC you specified is on the same VLAN / broadcast domain as the server, for testing you can try directly connecting it to the NIC.

I also found I had to restart the device a few times to get it to a “happy” state.

It’s worth pointing out that there are no LEDs lit when the system is powered on, only when you connect a NIC.

Next we’ll need to setup the sngtc_server so these resources can be accessed via FreeSWITCH or Asterisk.

Config is pretty simple if you’re using an all-in-one deployment, all you’ll need to change is the NIC in a file you create in /etc/sngtc/sngtc_server.conf.xml:

<configuration name="sngtc_server.conf" description="Sangoma Transcoding Manager Configuration">

                By default the SOAP server uses a private local IP and port that will work for out of the box installations
                where the SOAP client (Asterisk/FreeSWITCH) and server (sngtc_server) run in the same box.
                However, if you want to distribute multiple clients across the network, you need to edit this values to
                listen on an IP and port that is reachable for those clients.
                <param name="bindaddr" value="" />
                <param name="bindport" value="9000" />


                <!-- The name of the vocallo is the ethernet device name as displayed by ifconfig -->
                <vocallo name="ens33">
                        <!-- Starting UDP port for the vocallo -->
                        <param name="base_udp" value="5000"/>
                        <!-- Starting IP address octet to use for the vocallo modules -->
                        <param name="base_ip_octet" value="182"/>



With that set we can actually try starting the server,

Again, all going well you should see something like this in the log:

And then at the end you should see:

[SNGTC_INFO ] * 16:43:58: [00-0c-xx-yy-zz] RoundTripMs = 6 ulExtractTimeMs=0 ulCmdTimeoutMs 1000
[SNGTC_INFO ] * 16:43:58: 00-0c-xx-yy-zz: Reset Finished

[SNGTC_INFO ] * 16:43:58: 00-0c-xx-yy-zz: Setting cpu threshold Hi=90/Lo=80
[SNGTC_INFO ] * 16:43:58: Sangoma Transcoding Server Ready
[SNGTC_INFO ] * 16:43:58: Monitoring Sangoma Transcoding Modules

Once we know it’s starting up manually we can try and start the daemon.

sngtc_server_ctrl start

Should result in:

sngtc_server: Starting sngtc_server in safe mode ...
sngtc_server: Starting processes...
Starting sngtc_server...OK

And with that, we’re off and running and ready to configure this for use in FreeSWITCH or Asterisk.

Testing Mobile Networks with Remote Test Phones

I build phone networks, and unfortunately, I’m not able to be everywhere at once.

This means sometimes I have to test things in networks I may not be within the coverage of.

To get around this, I’ve setup something pretty simple, but also pretty powerful – Remote test phones.

Using a Raspberry Pi, Intel NUC, or any old computer, I’m able to remotely control Android handsets out in the field, in the coverage footprint of whatever network I need.

This means I can make test calls, run speed testing, signal strength measurements, on real phones out in the network, without leaving my office.

Base OS

Because of some particularities with Wayland and X11, for this I’d steer clear of Ubuntu distributions, and suggest using Debian if you’re using x86 hardware, and Raspbian if you’re using a Pi.

Setup Android Debug Bridge (adb)

The base of this whole system is ADB, the Android Debug Bridge, which exposes the ability to remotely control an Android phone over USB.

You can also do this over WiFi, but I find for device testing, wired allows me to airplane mode a device or disable data, which I can’t do if the device is connected to ADB via WiFi.

There’s lot of info online about setting Android Debug Bridge up on your device, unlocking the Developer Mode settings, etc, if you’ve not done this before I’ll just refer you to the official docs.

Before we plug in the phones we’ll need to setup the software on our remote testing machine, which is simple enough:

[email protected]:~$ sudo apt install android-tools-adb
sudo apt install android-tools-fastboot

Now we can plug in each of the remote phones we want to use for testing and run the command “adb devices” which should list the phones with connected to the machine with ADB enabled:

[email protected]:~$ adb devices
List of devices attached
ABCDEFGHIJK	unauthenticated
LMNOPQRSTUV	unauthenticated

You’ll get a popup on each device asking if you want to allow USB debugging – If this is going to be a set-and-forget deployment, make sure you tick “Always allow from this Computer” so you don’t have to drive out and repeat this step, and away you go.

How to Access Developer Options and Enable USB Debugging on Android

Lastly we can run adb devices again to confirm everything is in the connected state


scrcpy an open-source remote screen mirror / controller that allows us to control Android devices from a computer.

In our case we’re going to install with Snap (if you hate snaps as many folks do, you can also compile from source):

[email protected]:~$ snap install scrcpy

Remote Access

If you’re a regular Linux user, the last bit is the easiest.

We’re just going to use SSH to access the Linux machine, but with X11 forwarding.

If you’ve not come across X11 fowarding before, from a Linux machine just add the -X option to your SSH command, for example from my laptop I run:

nick@oldfaithful:~$ ssh [email protected] -X

Where is the remote tester device.

After SSHing into the box, we can just run scrcpy and boom, there’s the window we can interact with.

If you’ve got multiple devices connected to the same device, you’ll need to specify the ADB device ID, and of course, you can have multiple sessions open at the same time.

scrcpy -s 61771fe5

That’s it, as simple as that.


A few settings you may need to set:

I like to enable the “Show taps” option so I can see where my mouse is on the touchscreen and see what I’ve done, it makes it a lot easier when recording from the screen as well for the person watching to follow along.

You’ll probably also want to disable the lock screen and keep the screen awake

Some OEMs have an additonal tick box if you want to be able to interact with the device (rather than just view the screen), which often requires signing into an account, if you see this toggle, you’ll need to turn it on:

Ansible Playbook

I’ve had to build a few of these, so I’ve put an Ansible Playbook on Github so you can create your own.

You can grab it from here.

Kamailio I-CSCF – SRV Lookup Behaviour

Recently I had a strange issue I thought I’d share.

Using Kamailio as an Interrogating-CSCF, Kamailio was getting the S-CSCF details from the User-Authorization-Answer’s “Server-Name” (602) AVP.

The value was set to:

But the I-CSCF was only looking up A-Records for, not using DNS-SRV.

The problem? The Server-Name I had configured as a full SIP URI in PyHSS including the port, meant that Kamailio only looks up the A-Record, and did not do a DNS-SRV lookup for the domain.

Dropping the port number saw all those delicious SRV records being queried.

Something to keep in mind if you use S-CSCF pooling with a Kamailio based I-CSCF, if you want to use SRV records for load balancing / traffic sharing, don’t include the port, and if instead you want it to go to the specified host found by an A-record, include the port.

SMS with Alphanumeric Source

Sending SMS with an alphanumeric String as the Source

If you’ve ever received an SMS from your operator, and the sender was the Operator name for example, you may be left wondering how it’s done.

In IMS you’d think this could be quite simple – You’d set the From header to be the name rather than the MSISDN, but for most SMSoIP deployments, the From header is ignored and instead the c header inside the SMS body is used.

So how do we get it to show text?

Well the TP-Originating address has the “Type of Number” (ToN) field which is typically set to International/National, but value 5 allows for the Digits to instead be alphanumeric characters.

GSM 7 bit encoding on the text in the TP-Originating Address digits and presto, you can send SMS to subscribers where the message shows as From an alphanumeric source.

On Android SMSs received from alphanumeric sources cannot be responded to (“no more “DO NOT REPLY TO THIS MESSAGE” at the end of each text), but on iOS devices you can respond, but if I send an SMS from “Nick” the reply from the subscriber using the iPhone will be sent to MSISDN 6425 (Nick on the telephone keypad).

FreeSWITCH – Incompatible Destination

A recent little issue I ran into the other day, that I figured may be of use to someone in the future.

When making a call to FreeSWITCH I would get an “INCOMPATIBLE DESTINATION” response to the SIP INVITE.

Here’s what I saw in the log:

2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5650 Audio Codec Compare [GSM:3:8000:20:13200:1]/[opus:116:48000:20:0:1]
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5650 Audio Codec Compare [GSM:3:8000:20:13200:1]/[G722:9:8000:20:64000:1]
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5650 Audio Codec Compare [GSM:3:8000:20:13200:1]/[PCMU:0:8000:20:64000:1]
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5650 Audio Codec Compare [GSM:3:8000:20:13200:1]/[PCMA:8:8000:20:64000:1]
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5944 No 2833 in SDP. Liberal DTMF mode adding 101 as telephone-event.
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5973 sofia/internal/[email protected]:5060 Set 2833 dtmf send payload to 101 recv payload to 101
2022-02-19 13:04:04.027963 99.47% [NOTICE] switch_channel.c:3993 Hangup sofia/internal/49042@ [CS_EXECUTE] [INCOMPATIBLE_DESTINATION]

The hint to the cause of the error is above it – Codec comparison. If we look at the Audio Codec Compare lines, we can see the GSM codec we are trying to use, does not match the codecs configured in FreeSWITCH, hence getting the INCOMPATIBLE_DESTINATION error – None of the codecs offered match the codecs supported in FreeSWITCH.

So where do we go to fix this?

Well the SIP profile itself defines the codecs that are supported on this SIP profile,

FreeSWITCH SIP Profile (Sofia) codec settings

If you’re using a mostly default config, you’ll see this is set to a global variable, called $${global_codec_prefs}, so let’s take a look at vars.xml where this is defined:

FreeSWITCH default codec selection global variable

And there’s our problem, we need to add the GSM codec into that list to allow the calls,

So we change it to add the codecs we want to support, and reload the changes,

The Codec preferences I need for this IMS Application Server

Now when we want to make a call, success!

Successful call
IMS DNS Failing

Kamailio, IMS & DNS Headches

I’m sure I’ve ranted about the importance of DNS in IMS networks in the past on here already.

Recently I was rebuilding a P-CSCF and kept getting an error saying that the DNS was failing to resolve:

 4(5993) CRITICAL: <core> [core/dns_cache.c:3136]: dns_srv_sip_resolve(): unknown proto 0
 4(5993) ERROR: tm [ut.h:284]: uri2dst2(): failed to resolve "" :bug - critical error (-13)
 4(5993) ERROR: tm [t_fwd.c:1759]: t_forward_nonack(): failure to add branches
 4(5993) ERROR: sl [sl_funcs.c:414]: sl_reply_error(): stateless error reply used: Unresolvable destination (478/SL)

This was a rebuild, another P-CSCF was running fine and handling traffic with the same DNS server set.

I checked the netplan config and confirmed the DNS server was set correctly.

If I did an nslookup on the address that was failing to resolve – pointing it at the correct DNS server, the A & SRV records came back OK, and everything was fine.

Stranger still, after clearing the DNS Cache, and running a packet capture, I couldn’t see any DNS queries at all….

The problem? Kamailio uses resolv.conf by default on Ubuntu Server, and that was pointing to localhost.

After updating resolv.conf to point to the DNS server handling the IMS domains, I was good to go again.

A super valuable resource for all things DNS & Kamailio is this doc.

FreeSWITCH, Kamailio & IMS Extensions

Recently I’ve been doing some work with FreeSWITCH as an IMS Conference Factory, I’ve written a bit about it before in this post on using FreeSWITCH with the AMR codec.

Pretty early on in my testing I faced a problem with subsequent in-dialog responses, like re-INVITEs used for holding the calls.

Every subsequent message, was getting a “420 Bad Extension” response from FreeSWITCH.

So what didn’t it like and why was FreeSWITCH generating 420 Bad Extension Responses to these subsequent messages?

Well, the “Extensions” FreeSWITCH is referring to are not extensions in the Telephony sense – as in related to the Dialplan, like an Extension Number to identify a user, but rather the Extensions (as in expansions) to the SIP Protocol introduced for IMS.

The re-INVITE contains a Require header with sec-agree which is a SIP Extension introduced for IMS, which FreeSWITCH does not have support for, and the re-INVITE says is required to support the call (Not true in this case).

Using a Kamailio based S-CSCF means it is easy to strip these Headers before forwarding the requests onto the Application Server, which is what I’ve done, and bingo, no more errors!

CGrates in Baby Steps – Part 1

So you have a VoIP service and you want to rate the calls to charge your customers?

You’re running a mobile network and you need to meter data used by subscribers?

Need to do least-cost routing?

You want to offer prepaid mobile services?

Want to integrate with Asterisk, Kamailio, FreeSWITCH, Radius, Diameter, Packet Core, IMS, you name it!

Well friends, step right up, because today, we’re talking CGrates!

So before we get started, this isn’t going to be a 5 minute tutorial, I’ve a feeling this may end up a big multipart series like some of the others I’ve done.
There is a learning curve here, and we’ll climb it together – but it is a climb.


Let’s start with a Debian based OS, installation is a doddle:

sudo wget -O - | sudo apt-key add -
echo "deb nightly main" | sudo tee /etc/apt/sources.list.d/cgrates.list
sudo apt-get update
sudo apt-get install cgrates -y
apt-get install mysql-server redis-server git -y

We’re going to use Redis for the DataDB and MariaDB as the StorDB (More on these concepts later), you should know that other backend options are available, but for keeping things simple we’ll just use these two.

Next we’ll get the database and config setup,

cd /usr/share/cgrates/storage/mysql/
./ root localhost
cgr-migrator -exec=*set_versions

Lastly we’ll clone the config files from the GitHub repo:

Rating Concepts

So let’s talk rating.

In its simplest form, rating is taking a service being provided and calculating the cost for it.

The start of this series will focus on voice calls (With SMS, MMS, Data to come), where the calling party (The person making the call) pays, so let’s imagine calling a Mobile number (Starting with 614) costs $0.22 per minute.

To perform rating we need to determine the Destination, the Rate to be applied, and the time to charge for.

For our example earlier, a call to a mobile (Any number starting with 614) should be charged at $0.22 per minute. So a 1 minute call will cost $0.22 and a 2 minute long call will cost $0.44, and so on.

We’ll also charge calls to fixed numbers (Prefix 612, 613, 617 and 617) at a flat $0.20 regardless of how long the call goes for.

So let’s start putting this whole thing together.

Introduction to RALs

RALs is the component in CGrates that takes care of Rating and Accounting Logic, and in this post, we’ll be looking at Rating.

The rates have hierarchical structure, which we’ll go into throughout this post. I took my notepad doodle of how everything fits together and digitized it below:


Destinations are fairly simple, we’ll set them up in our Destinations.csv file, and it will look something like this:


Each entry has an ID (referred to higher up as the Destination ID), and a prefix.

Also notice that some Prefixes share an ID, for example 612, 613, 617 & 618 are under the Destination ID named “DST_AUS_Fixed”, so a call to any of those prefixes would match DST_AUS_Fixed.


Rates define the price we charge for a service and are defined by our Rates.csv file.


Let’s look at the fields we have:

  • ID (Rate ID)
  • ConnectFee – This is the amount charged when the call is answered / connected
  • The Rate is how much we will charge, it’s loosely cents, but could be any currency. By default CGrates looks down to 4 decimal places.
  • RateUnit is how often this rate is applied in seconds
  • RateIncriment is how often this is evaluated in seconds
  • GroupIntervalStart – Activates an event when triggered

So let’s look at how this could be done, and the gotchas that exist.

So let’s look at some different use cases and how we’d handle them.

Per Minute Billing

This would charge a rate per minute, at the start of the call, the first 60 seconds will cost the caller $0.25.

At the 61 second mark, they will be charged another $0.25.

60 seconds after that they will be charged another $0.25 and so on.


This is nice and clean, a 1 second call costs $0.25, a 60 second call costs $0.25, and a 61 second call costs $0.50, and so on.

This is the standard billing mechanism for residential services, but it does not pro-rata the call – For example a 1 second call is the same cost as a 59 second call ($0.25), and only if you tick over to 61 seconds does it get charged again (Total of $0.50).

Per Second Billing

If you’re doing a high volume of calls, paying for a 3 second long call where someone’s voicemail answers the call and was hung up, may seem a bit steep to pay the same for that as you would pay for 59 seconds of talk time.

Instead Per Second Billing is more common for high volume customers or carrier-interconnects.

This means the rate still be set at $0.25 per minute, but calculated per second.

So the cost of 60 seconds of call is $0.25, but the cost of 30 second call (half a minute) should cost half of that, so a 30 second call would cost $0.125.


How often we asses the charging is defined by the RateIncrement parameter in the Rate Table.

We could achieve the same outcome another way, by setting the RateIncriment to 1 second, and the dividing the rate per minute by 60, we would get the same outcome, but would be more messy and harder to maintain, but you could think of this as $0.25 per minute, or $0.004166667 per second ($0.25/60 seconds).

Flat Rate Billing

Another option that’s commonly used is to charge a flat rate for the call, so when the call is answered, you’re charged that rate, regardless of the length of the call.

Regardless if the call is for 1 second or 10 hours, the charge is the same.


For this we just set the ConnectFee, leaving the Rate at 0, so the cost will be applied on connection, with no costs applied per time period.

This means a 1 second call will cost $0.25, while a 3600 second call will still cost $0.25.

We charge a connect fee, but no rate.

Linking Destinations to the Rates to Charge

Now we’ve defined our Destinations and our Rates, we can link the two, defining what Destinations get charged what Rates.

This is defined in DestinationRates.csv


Let’s look at the Fields,

  • ID (Destination Rate ID)
  • DestinationID – Refers to the DestinationID defined in the Destinations.csv file
  • RatesTag – Referes to the Rate ID we defined in Rates.csv
  • RoundingMethod – Defines if we round up or down
  • RoundingDecimals – Defines how many decimal places to consider before rounding
  • MaxCost – The maximum cost this can go up to
  • MaxCostStrategy – What to do if the Maximum Cost is reached – Either make the rest of the call Free or Disconnect the call

So for each entry we’ll define an ID, reference the Destination and the Rate to be applied, the other parts we’ll leave as boilerplate for now, and presto. We have linked our Destinations to Rates.

Rating Plans

We may want to offer different plans for different customers, with different rates.

That’s what we define in our Rating Plans.

  • ID (RatingPlanID)
  • DestinationRatesId (As defined in DestinationRates.csv)
  • TimingTag – References a time profile if used
  • Weight – Used to determine what precedence to use if multiple matches

So as you may imagine we need to link the DestinationRateIDs we just defined together into a Rating Plan, so that’s what I’ve done in the example above.

Rating Profiles

The last step in our chain is to link Customers / Subscribers to the profiles we’ve just defined.

How you allocate a customer to a particular Rating Plan is up to you, there’s numerous ways to approach it, but for this example we’re going to use one Rating Profile for all callers coming from the “” tenant:


Let’s go through the fields here,

  • Tenant is a grouping of Customers
  • Category is used to define the type of service we’re charging for, in this case it’s a call, but could also be an SMS, Data usage, or a custom definition.
  • Subject is typically the calling party, we could set this to be the Caller ID, but in this case I’ve used a wildcard “*any”
  • ActivationTime allows us to define a start time for the Rating Profile, for example if all our rates go up on the 1st of each month, we can update the Plans and add a new entry in the Rating Profile with the new Plans with the start time set
  • RatingPlanID sets the Rating Plan that is used as we defined in RatingPlans.csv

Loading the Rates into CGrates

At the start we’ll be dealing with CGrates through CSV files we import, this is just one way to interface with CGrates, there’s others we’ll cover in due time.

CGRates has a clever realtime architecture that we won’t go into in any great depth, but in order to load data in from a CSV file there’s a simple handy tool to run the process,

root@cgrateswitch:/home/nick# cgr-loader -verbose -path=/home/nick/tutorial/ -flush_stordb

Obviously you’ll need to replace with the folder you cloned from GitHub.

Trying it Out

In order for CGrates to work with Kamailio, FreeSWITCH, Asterisk, Diameter, Radius, and a stack of custom options, for rating calls, it has to have common mechanisms for retrieving this data.

CGrates provides an API for rating calls, that’s used by these platforms, and there’s a tool we can use to emulate the signaling for call being charged, without needing to pickup the phone or integrate a platform into it.

root@cgrateswitch:/home/nick# cgr-console 'cost Category="call" Tenant="" Subject="3005" Destination="614" AnswerTime="2014-08-04T13:00:00Z" Usage="60s"'

The tenant will need to match those defined in the RatingProfiles.csv, the Subject is the Calling Party identity, in our case we’re using a wildcard match so it doesn’t matter really what it’s set to, the Destination is the destination of the call, AnswerTime is time of the call being answered (pretty self explanatory) and the usage defines how many seconds the call has progressed for.

The output is a JSON string, containing a stack of useful information for us, including the Cost of the call, but also the rates that go into the decision making process so we can see the logic that went into the price.

 "AccountSummary": null,
 "Accounting": {},
 "CGRID": "",
 "Charges": [
   "CompressFactor": 1,
   "Increments": [
     "AccountingID": "",
     "CompressFactor": 1,
     "Cost": 0,
     "Usage": "0s"
     "AccountingID": "",
     "CompressFactor": 1,
     "Cost": 25,
     "Usage": "1m0s"
   "RatingID": "febb614"
 "Cost": 25,
 "Rates": {
  "7d4a755": [
    "GroupIntervalStart": "0s",
    "RateIncrement": "1m0s",
    "RateUnit": "1m0s",
    "Value": 25
 "Rating": {
  "febb614": {
   "ConnectFee": 0,
   "MaxCost": 0.12,
   "MaxCostStrategy": "*disconnect",
   "RatesID": "7d4a755",
   "RatingFiltersID": "7e42edc",
   "RoundingDecimals": 4,
   "RoundingMethod": "*up",
   "TimingID": "c15a254"
 "RatingFilters": {
  "7e42edc": {
   "DestinationID": "DST_AUS_Mobile",
   "DestinationPrefix": "614",
   "RatingPlanID": "RP_AUS",
   "Subject": "*"
 "RunID": "",
 "StartTime": "2014-08-04T13:00:00Z",
 "Timings": {
  "c15a254": {
   "MonthDays": [],
   "Months": [],
   "StartTime": "00:00:00",
   "WeekDays": [],
   "Years": []
 "Usage": "1m0s"

So have a play with setting up more Destinations, Rates, DestinationRates and RatingPlans, in these CSV files, and in our next post we’ll dig a little deeper… And throw away the CSVs all together!

The Surprisingly Complicated World of SMS: Apple iPhone MT SMS

In iOS 15, Apple added support for iPhones to support SMS over IMS networks – SMSoIP. Previously iPhone users have been relying on CSFB / SMSoNAS (Using the SGs interface) to send SMS on 4G networks.

Getting this working recently led me to some issues that took me longer than I’d like to admit to work out the root cause of…

I was finding that when sending a Mobile Termianted SMS to an iPhone as a SIP MESSAGE, the iPhone would send back the 200 OK to confirm delivery, but it never showed up on the screen to the user.

The GSM A-I/F headers in an SMS PDU are used primarily for indicating the sender of an SMS (Some carriers are configured to get this from the SIP From header, but the SMS PDU is most common).

The RP-Destination Address is used to indicate the destination for the SMS, and on all the models of handset I’ve been testing with, this is set to the MSISDN of the Subscriber.

But some devices are really finicky about it’s contents. Case in point, Apple iPhones.

If you send a Mobile Terminated SMS to an iPhone, like the one below, the iPhone will accept and send back a 200 OK to this request.

The problem is it will never be displayed to the user… The message is marked as delivered, the phone has accepted it it just hasn’t shown it…

SMS reports as delivered by the iPhone (200 OK back) but never gets displayed to the user of the phone as the RP-Destination Address header is populated

The fix is simple enough, if you set the RP-Destination Address header to 0, the message will be displayed to the user, but still took me a shamefully long time to work out the problem.

RP-Destination Address set to 0 sent to the iPhone, this time it’ll get displayed to the user.

FreeSWITCH as an IMS Application Server

After getting AMR support in FreeSWITCH I set about creating an IMS Application Server for VoLTE / IMS networks using FreeSWITCH.

So in IMS what is an Application Server? Well, the answer is almost anything that’s not a CSCF.

An Application Server could handle your Voicemail, recorded announcements, a Conference Factory, or help interconnect with other systems (without using a BGCF).

I’ll be using mine as a simple bridge between my SIP network and the IMS core I’ve got for VoLTE, with FreeSWITCH transcoding between AMR to PCMA.

Setting up FreeSWITCH

You’ll need to setup FreeSWITCH as per your needs, so that’s however you want to use it.

This post won’t cover setting up FreeSWITCH, there’s plenty of good resources out there for that.

The only difference is when you install FreeSWITCH, you will want to compile with AMR Support, so that you can interact with mobile phones using the AMR codec, which I’ve documented how to do here.

Setting up your IMS

In order to get calls from the IMS to the Application Server, we need a way of routing the calls to the Application Server.

There are two standards-compliant ways to achieve this,

The first is to use ENUM to route the calls you want to send to the Application Server, to the application server.

If you want to go down that path using Kamailio as your IMS I’ve got a post on that topic here.

But this is a blunt instrument, after all, it’ll only ever be used at the start of the call, what if we want to send it to an AS because a destination can’t be reached and we want to play back a recorded announcement?

Well that’s where iFCs come into the picture. Through the use of Initial Filter Criterias, we’re able to route different types of SIP traffic, requests and responses, based on our needs. Again we can do this in Kamailio, with a little help from an HSS like PyHSS.

Lifecycle of a Dedicated Bearer – From Flow-Description AVP to Traffic Flow Templates

To support Dedicated Bearers we first have to have a way of profiling the traffic, to classify the traffic as being the type we want to provide the Dedicated Bearer for.

The first step involves a request from an Application Function (AF) to the PCRF via the Rx interface.

The most common type of AF would be a P-CSCF. When a VoLTE call gets setup the P-CSCF requests that a dedicated bearer be setup for the IP Address and Ports involved in the VoLTE call, to ensure users get the best possible call quality.

But Application Functions aren’t limited to just VoLTE – You could also embed an Application Function into the server for an online game to enable a dedicated bearer for users playing that game, or a sports streaming app that detects when a user starts streaming sports and creates a dedicated bearer for that user to send the traffic down.

The request to setup a dedicated bearer comes in the form of a Diameter request message from the AF, using the Rx reference point, typically from the P-CSCF to the PCRF in the network in an “AA-Request”.

Of main interest in the AA-Request is the Media Component AVP, that contains all the details needed to identify the traffic flow.

Now our PCRF is in charge of policy, and know which P-GW is serving the required subscriber. So the PCRF takes this information and sends a Gx Re-Auth Request to the PCEF in the P-GW serving the subscriber, with a Charging Rule the PCEF in the P-GW needs to install, to profile and apply QoS to the bearer.

So within the Gx Re-Auth Request is the Charging-Rule Definition, made up of Flow-Description AVP which I’ve written about here, that is used to identify and profile traffic flows and QoS parameters to apply to matching traffic.

Charging Rule Definition’s Flow-Information AVPs showing the information needed to profile the traffic

The QoS Description AVP defines which QoS parameters (QCI / ARP / Guaranteed & Maximum Bandwidth) should be applied to the traffic that matches the rules we just defined.

QoS information AVP
QoS Information AVP showing requested QoS Parameters

The P-GW sends back a Gx Re-Auth Answer, and gets to work actually setting up these bearers.

With the rule installed on the PCEF, it’s time to get this new bearer set up on the UE / eNodeB.

The P-GW sends a GTPv2 “Create Bearer Request” to the S-GW which forwards it onto the MME, to setup / define the Dedicated Bearer to be setup on the eNodeB.

GTPv2 “Create Bearer Request” sent by the P-Gw to the S-GW forwarded from the S-GW to the MME

The MME translates this into an S1 “E-RAB Setup Request” which it sends to the eNodeB to setup,

S1 E-RAB Setup request showing the E-RAB to be setup

Assuming the eNodeB has the resources to setup this bearer, it provides the details to the UE and sets up the bearer, sending confirmation back to the MME in the S1 “E-RAB Setup Response” message, which the MME translates back into GTPv2 for a “Create Bearer Response”

All this effort to keep your VoLTE calls sounding great!

The Surprisingly Complicated World of SMS: Special Characters

SMS by default uses the GSM-7 bit alphabet, thanks to the fact each letter is only 7 bits long, this means you can cram 160 characters into a 140 byte message body.

However, this 7-bit alphabet is, well, limited, because it’s 7 bits long it means we can only have 128 different combinations of these bits, or to put it another way, with only 128 different unique combinations of these bits, we can only define 128 characters.

You have the standard 26 latin alphabet characters that Sesame Street drilled into you, some characters with accents, digits, and a limited set of symbols.

The GSM 7 bit alphabet does not include is character sets and symbols common for non-English written languages.

Shift Tables

To deal with this 3GPP introduced “National Language Shift Tables”, which are enable a sort of find-and-replace approach to the 7-bit alphabet, where certain characters that are unused in one alphabet, take the value of characters from the local alphabet.

So if you want to send the character Ğ (Found in the Turkish and Azerbaijani alphabets) you’d select the Turkish language Shift table, that replaces the capital G (71) with Ğ.

Of course you need to have two things to do this, you need the Language Shift Table to tell you what local-language letters replace what default letters, and a mechanism to state that you’re using a language shift table.

3GPP define the National Language Shift tables in TS 23.038, where you can lookup the character you want to encode, so you know what 7 bit value it uses, for example our character Ğ is 1000111 in the 7-bit alphabet.

Next we need to indicate that we don’t want 1000111 in the 7-bit alphabet to be rendered as “G”, we want to use the “Turkish National Language Single Shift Table” which will render it as “Ğ”. We do this in the User Data Header of the SMS Body, the same way we’d indicate that an SMS is a concatenated SMS.

But by adding a header in the User Data Header of the SMS Body, we eat into the space we can use to send the message body, with a single User Data Header indicating that the Turkish National Language Single Shift Table is being used, we go from a maximum of 160 characters without the User Data Header, to 134 characters.

I’ve shared a lot more information on the User Data Header in this post on Concatenated SMS, should you be interested.

UCS2 Encoding

So that’s all well and good for other languages that have some overlap in letters, where we can substitute “G” for “Ğ”, but Unicode have 3304 emojis defined at the time of writing.

No matter how many shift tables you define, you’re not going to cover all of these in a 7-bit alphabet.

So all this encoding falls to 💩 when someone adds an Emoji.

The “😀” Emoji, represented as U+1F600 in Unicode, can be encoded as 0xF09F9880 in UTF-8 or 0xD83DDE00 in UTF-16.

So in 3GPP Networks, when you need more than 128 characters to work with, and when shift tables won’t cut the mustard, you can change the encoding used to use the International Standards Organisations’ “Universal coded character set 2” (UCS-2).

Unfortunately UCS-2 never really took off, but luckily it overlaps with UTF-16 character set, which is a lot more common.

So if you’ve got a “😀” Emoji in your SMS body the encoding of the message will be changed from GSM-7 to use a different encoding -UTF-16 / UCS2.

SMS Body showing TP-DCS character set is UCS2 / UTF-16 as Emojis are present

There’s a catch here, if you’re moving from a 7-bit alphabet to a 16 bit alphabet, you’re going to have a lot less space to work with.

A single SMS contains 1120 bits for the user data (The actual message).

With GSM-7 bit encoding, each letter takes up 7 bits, so 1120÷7 gives us 160 characters.

With UTF-16/UCS2 encoding, each letter takes up 16 bits so 1120÷16 only give us 70 characters.

So what happens next?

Often when Emojis are used, as our message is now limited to 70 characters concatenated messages are used, which takes a further 8 bytes of our message body if concatenated messages are used, further limiting the message length.

The Surprisingly Complicated World of SMS: Concatenated / Multipart SMS

Most people think of 160 characters as the length of an SMS. But the payload is actually 140 bytes, but with better encoding 1 character doesn’t require 1 byte.

The above paragraph is exactly 160 characters. It would fit into a standard SMS.

By using the GSM 7 bit alphabet, you can cram 16 characters into 140 bytes (octets) of space, which is kind of cool.

140 bytes of data containing 160 characters of text

But people often need to convey more text than just 160 characters, or if you’re using characters that don’t exist in the GSM 7-bit alphabet, that limit becomes even less than 160 characters (different encodings other than GSM-7 need more data to transfer the same number of characters) so we get into multipart SMS, another feature in the surprisingly complicated world of SMS.

You’d think if you took a 160 character SMS, and concatenated it onto another 160 character SMS, you’d get a total of 320 characters, right (160+160=320)?
Alas it’s not that simple.

In order to achieve the concatenation of messages in a way that’s transparent to the users (rather than a series of SMSes coming through one-after-the-ther) a User-Data Header (TP-User-Data-Header-Indicator aka TP-UHDI) is added to the TP-User Data of the TPDU (the part that actually contains the user message).

This User-Data Header takes up 7 bytes, which with GSM encoding robs us of 6 characters from the message length. (Not a typo, GSM7 encoding does not mean 1 character = 1 byte, hence we can get 160 characters into 140 bytes of space)
So a two SMS concatenated message would only allow 268 characters to be sent (134 characters + 134 characters).

Let’s take a look at this header that’s robbing us of message length, but enabling us to concatenate messages.

For starters, the information about how many parts in the concatenated message, and what part number this one is, is located in the message body, hence robbing us of characters.

But we only know about the presence of this header being in the message body because the SMS-SUBMIT TPDU has the TP-UDHI flag (TP-User-Data-Header-Indicator) set, so we know the User Data is prefixed with the User-Data-Header.

Now if we have a look in the TP-User-Data we can see the User-Data Header, this can actually carry a few different payloads, but in our case, it’s carrying the Concatenated Short Messages IE, which tells us the message identifier (unique per single-but-multi-part message, the number of parts in the message (in this case 2) and the part number this is (part 1 of 2).

First part of a two part SMS

Now the phone has indicated this is a multipart message, the length of the data is still 160, but the length of the actual message is now limited to 134 characters with GSM7 encoding.

The encoding isn’t as bad as you might expect:
1st byte indicates the total length of the User Data Headers (After this the actual user data begins),
2nd byte is the IE identifier, for Concatenated Short Messages, this is 00,
3rd byte is the length of the Concatenated Short Messages IE,
4th byte is the message identifier in hex,
5th byte is the number of message parts in hex (So up to 255 message parts)
6th byte is the message part number, to aid in putting it back together in order.

3GPP TS 23.040 – TP-User Data (TP-UD) – Encoding of User Data Header and generic IE
Concatenated short message IE encoding

So what we end up with is a header inside our user payload, advising that this is a concatenated SMS, the message identifier, the number of parts in the message, and the part number of this particular message.

Last part of two part SMS

The SMSc on receipt of these has to spool them back out to the destination with the same message part number, and same headers in place.

The phone receiving the SMS has to wait for all the parts to come through and then reassemble before rendering to the user.

So that’s how concatenated SMS works. While this may seem convoluted and silly in a world where transfering more than 140 bytes of data is trivial, SMS was introduced in the early 1990s, and in theory at least, a user with a phone that supported SMS purchased when SMS was introduced, should still be able to interwork with phones today.

ENUM – DNS based Call Routing

DNS is commonly used for resolving domain names to IP Addresses, and is often described as being like “the phone book of the Internet”.

So what’s the phone book of phone books?

The answer, is (kind of) DNS. With the aid of E.164 number to URI mapping (ENUM), DNS can be used to resolve phone numbers into SIP URIs to route the traffic to.

So what is ENUM?

ENUM allows us to bypass the need for a central switch for routing calls to numbers, and instead, through a DNS lookup, resolve a phone number into a reachable SIP URI that is the ultimate destination for the traffic.

Imagine you want to call a company, you dial the phone number for that company, your phone does a DNS query against the phone number, which returns the SIP URI of the company’s PBX, and your phone sends the SIP INVITE directly to the company’s PBX, with no intermediary party carrying the call.

3GPP have specified ENUM as the prefered mechanism for resolving phone numbers into SIP addresses, and while it’s widespread adoption on the public Internet is still in its early days (See my post on The Sad story of ENUM in Australia) it is increasingly common in IMS networks and inside operator networks.

IETF defined RFC 6116 for “The E.164 to Uniform Resource Identifiers (URI) Dynamic Delegation Discovery System (DDDS) Application (ENUM)”, which defines how the system works.

So how does ENUM actually work?

ENUM allow us to lookup a phone number on a DNS server and find the SIP URI a server that will handle traffic for the phone number, but it’s a bit more complicated than the A or AAAA records you’d use to resolve a website, ENUM relies on NAPTR records.

Let’s look at the steps involved in taking an E.164 number and knowing where to send it.

Step 1 – Reverse the Numbers

We read phone numbers from left to right.

This is because historically the switch needs to get all the long-distance routing sorted first. The switch has to route your call to the exchange that serves that subscriber, which is what all the area codes and prefixes assigned to areas are all about (Throwback to SZU for any old Telco buffs).

For an E.164 number you’ve got a Country Code, Area Code and then the Subscriber Number. The number gets more specific as it goes along.

But getting more specific as you go along is the opposite how how DNS works, millions of domains share the .com suffix, and the unique / specific part is the bits before that.

So the first step in the ENUM process is to reverse the phone number, so let’s take phone number (03) 5550 0912, which in E.164 is +61 3 5550 0912.

As the spaces in the phone numbers are there for the humans, we’ll drop all of them and reverse the number, as DNS is more specific right-to-left, so we end up with

Step 2 – Add the Suffix

The ITU ENUM specifies the suffix be assigned for public ENUM entries. Private ENUM deployments may use their own suffix, but to make life simple I’m going to use as if it were public.

So we’ll append the domain onto our reversed and formatted E.164 phone number:

Step 3 – Query it

Next we’ll run a Naming Authority Pointer (NAPTR) query against the domain, to get back a list of records for that number.

DNS is a big topic, and NAPTR and SRV takes up a good chunk of it, but what you need to know is that by using NAPTR we’re not limited to just a single response, we could have a weighted pool of servers handling traffic for this phone number, and be able to control load through the use of NAPTR, amongst other things.

Of course, if our phone can query the public NAPTR records, then so can anyone else, so we can just use a tool like Dig to query the record ourselves,

dig @ -t naptr

In the answers section I’ve setup this DNS server to only return a single response, with the regex SIP URI to use, in my case that’s sip:[email protected]

You’ll obviously need to replace the DNS server with your DNS server, and the query with the reversed and formatted version of the E.164 number you wish to query.

Step 4 – Send SIP traffic

After looking at the NAPTR records returned and using the weight and priority to determine which server/s to send to first, our phone forwards an INVITE to the URI returned in the NAPTR record.

How to interpret the returned results?

The first thing to keep in mind when working with ENUM is multiple records being returned is supported, and even encouraged.

NAPTR results return 7 fields, which define how it should be handled.

The host part is fairly obvious, and defines the host / DNS entry we’re talking about.

The Service defines what type of service this is. ENUM can be expanded beyond just voice, for example you may want to also return an email address or IM address as well as a SIP Address on an ENUM query, which you can do. By default voice uses the “E2U+sip” service to identify SIP URIs to route calls to, so in this context that’s what we’re interested in, but keep in mind there are other types out there,

Example ENUM query against a phone number showing other types of services (Email & Web)

The Order simply defines the order in which the rules are to be parsed. Lower numbers are processed first, if no matches then the next lowest, and so on until the highest number is reached.

The Pref is the processing preference. For load balancing 50/50 between two sites say a Melbourne and Sydney site, we’d return two results, with the same Order, and the same Pref, would see traffic split 50/50 between the two sites.
We could split this further, a Pref value of 10 for Melbourne, 10 for Sydney, 5 for Brisbane and 5 for Perth would see 33% of calls route to Melbourne, 33% of calls route to Sydney, 16.5% of calls route to Brisbane and 16.5% of calls route to Perth.
This is because we’d have a total preference value of 30, and the individual preference for each entry would work out as the fraction of the total (ie Pref 10 out of 30 = 10/30 or 33.3%).

The Flags denote the type of record we’re going to get, for most ENUM traffic this is going to be set to U, to denote a SIP URI with Regex.

The regexp field contains our SIP URI in the form of a Regular expression, which can include pattern matching and replacement. This is most commonly used to fill in the phone number into the SIP URI, for example instead of hardcoding the phone number into the response, we could use a Regular expression to fill in the requested number into the SIP URI.

!(^.*$)!sip:+1\[email protected]!

ENUM sounds great, how do I get it?

Here’s the tricky part.

If you’re looking to implement ENUM for an internal network, great, I’ll have some more posts here over the next few weeks covering off configuration of a DNS server to support ENUM lookups, and using Kamailio to lookup ENUM routes.

In terms of public ENUM, while many carriers are using ENUM inside their networks, public adoption of ENUM in most markets has been slow, for a number of reasons.

Many incumbent operators have been reluctant to embrace public ENUM as their role as an operator would be relegated to that of a Domain registrar.
Additionally, there’s real security risks involved in moving to ENUM – opening your phone system up to the world to accept inbound calls from anywhere. This could lead to DOS-style attacks of flooding phone numbers with automatically generated traffic, privacy risks and even less validation in terms of caller ID trust.

RIPE maintains the website listing the status of ENUM for each country / region.