People think SIP they think VoIP & phone calls, but SIP it’s the Phone Call Initiation Protocol it’s the Session Initiation Protocol – Sure VoIP guys like me love SIP, but it’s not just about VoIP.
Have you sent an SMS on a modern mobile phone recently? Chances are you sent a SMS over SIP using SIP MESSAGE method.
So let’s look a bit at SIP SIMPLE, the catchily titled acronym translates to Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (Admittedly less catchy in it’s full form).
There’s two way SIP SIMPLE can be used to implement Instant Messaging, Paging Mode with each message sent as a single transaction, and Session Mode where a session is setup between users and IMs exchanged with the same Call ID / transaction.
I’m going to cover the Paging Mode implementation because it’s simpler easier to understand.
Before we get too far this is another example of confusing terminology, let’s just clear this up; According to the RFC any SIP request is a SIP Message, like a SIP OPTIONS message, a SIP INVITE message. But the method of a SIP INVITE message is INVITE, the method of a SIP OPTIONS message is OPTIONS. There’s a SIP MESSAGE method, meaning you can send a SIP MESSAGE message using the MESSAGE method. Clear as mud? I’ll always refer to the SIP Method in Capitals, like MESSAGE, INVITE, UPDATE, etc.
The SIP MESSAGE method looks / acts very similar to a SIP INVITE, in that it’s got all the standard SIP headers, but also a Message Body, in which our message body lives (funny about that), typically we’ll send messages using the Content-Type: text/plain to denote we’re sending a plaintext message.
Example MESSAGE Message Flow
Like a SIP OPTIONS Method, the MESSAGE method is simply answered with a 200 OK (No Ack).
Let’s have a look at how the MESSAGE message looks:
In my last post I talked about using KEMI in Kamailio and how you can integrate in a different programming language to handle your SIP request handling in a language you already know – Like Python!
Before we get too excited there’s some boilerplate we’ve got to add to our Python script, we need to create a class called kamailio and populate the class by defining some functions, we’ll define an __init__ to handle loading of the class, define a child_init for handling child processes, define ksr_request_route to handle the initial requests. We’ll also need to define a mod_init – outside of the Kamailio class to initialize the class.
import sys
import Router.Logger as Logger
import KSR as KSR
import requests
# global function to instantiate a kamailio class object
# -- executed when kamailio app_python module is initialized
def mod_init():
KSR.info("===== from Python mod init\n");
return kamailio();
# -- {start defining kamailio class}
class kamailio:
def __init__(self):
KSR.info('===== kamailio.__init__\n');
# executed when kamailio child processes are initialized
def child_init(self, rank):
KSR.info('===== kamailio.child_init(%d)\n' % rank);
return 0;
# SIP request routing
# -- equivalent of request_route{}
def ksr_request_route(self, msg):
KSR.info("===== request - from kamailio python script\n");
KSR.dbg("method " + KSR.pv.get("$rm") + " r-uri " + KSR.pv.get("$ru"))
Most of these should be pretty self explanatory for anyone who’s done a bit more in-depth Python programming, but it’s no big deal if you don’t understand all this, the only part you need to understand is the ksr_request_route function.
ksr_request_route: translates to our request_route{} in the Kamailio native scripting language, all requests that come in will start off in this part.
Python Kamailio Routing
So let’s start to build upon this, so we’ll blindly accept all SIP registrations;
Here you’ll see we’ve added an if statement, as if we were doing any other If statement in Python, in this case we’re asking if the KSR.is_method(“REGISTER”), and if it is, we’ll send back a 200 OK response.
Let’s pause and talk about KSR
All the Kamailio bits we’ll use in Python will have the KSR. prefix, so let’s take a quick break here to talk about KSR. The KSR. functions are the KEMI functions we’ve exposed to Python.
Without them, we’re just writing Python, and we’d have to do all the functions provided by Kamailio nativeley in Python, which would be crazy.
So we leverage the Kamailio modules you know and love from Python using Python’s logic / programming syntax, as well as opening up the ability to pull in other libraries from Python.
Let’s look at how we might send a stateless reply,
There’s a module function to send a stateless reply;
KSR.sl.send_reply(200, "OK")
The vast majority of functions are abstracted as module functions, like the example above, but not all of them.
So every function doesn’t need to be wrapped up as a module, there’s also a way to call any function that you’d call from the native scripting language, wrapped up, kind of like an Exec command:
KSR.x.modf("sl_send_reply", "200", "OK");
So thanks to this we can call any Kamailio function from Python, even if it’s not explicitly in the KEMI abstraction.
Python Kamailio Routing (Continued)
So earlier we managed REGISTER requests and sent back a 200 OK response.
What about forwarding a SIP Request to another proxy? Let’s follow on with an elif statement to test if the method is an INVITE and statelessly forward it.
elif KSR.is_method("INVITE"):
#Lookup our public IP address
try:
ip = requests.get('https://api.ipify.org').text
except:
ip = "Failed to resolve"
#Add that as a header
KSR.hdr.append("X-KEMI: I came from KEMI at " + str(ip) + "\r\n");
#Set host IP to 10.1.1.1
KSR.sethost("10.1.1.1");
#Forward the request on
KSR.forward()
Now an incoming SIP invite will be proxied / forwarded to 10.1.1.1, all from Python.
But so far we’ve only done things in KEMI / Python that we could do in our native Kamailio scripting language, so let’s use some Python in our Python!
I utterly love the Python Requests library, so let’s use that to look up our public IP address and add it as a header to our forwarded SIP INVITE;
elif KSR.is_method("INVITE"):
#Lookup our public IP address
try:
ip = requests.get('https://api.ipify.org').text
except:
ip = "Failed to resolve"
#Add that as a header
KSR.hdr.append("X-KEMI: I came from KEMI at " + str(ip) + "\r\n");
#Set host IP to 10.1.1.1
KSR.sethost("10.1.1.1");
#Forward the request on
KSR.forward()
Bottom header is the X-KEMI custom header we included with our public IP
So let’s wrap this up a bit and handle any other request that’s not an INVITE or a REGISTER, with a 500 error code.
# SIP request routing
# -- equivalent of request_route{}
def ksr_request_route(self, msg):
KSR.dbg("method " + KSR.pv.get("$rm") + " r-uri " + KSR.pv.get("$ru"))
if KSR.is_method("REGISTER"):
KSR.sl.send_reply(200, "OK")
elif KSR.is_method("INVITE"):
#Lookup our public IP address
try:
ip = requests.get('https://api.ipify.org').text
except:
ip = "Failed to resolve"
#Add that as a header
KSR.hdr.append("X-KEMI: I came from KEMI at " + str(ip) + "\r\n");
#Set host IP to 10.1.1.1
KSR.sethost("10.1.1.1");
#Forward the request on
KSR.forward()
else:
KSR.sl.send_reply(500, "Got no idea...")
I’ve talked about using the UAC module, but as promised, here’s how we can use the UAC module to send SIP REGISTER requests to another SIP server so we can register to another SIP proxy.
Let’s say we’re using Kamailio to talk to a SIP Trunk that requires us to register with them so they know where to send the calls. We’d need to use Kamailio UAC module to manage SIP Registration with our remote SIP Trunk.
But Kamailio’s a proxy, why are we sending requests from it? A proxy just handles messages, right? Proxies don’t originate messages, it’s true, and Kamailio can be a proxy, but with the UAC module we can use Kamailio as a Client instead of a server. Keep in mind Kamailio is what we tell it to be.
Getting Started
Before we can go spewing registrations out all over the internet we need to start by getting a few things in place;
First of which is configuring UAC module, which is something I covered off in my last post,
Once we’ve got that done we’ll need to tell the UAC module our IP Address for the from address for our Contact field, and the database URL of what we’ve setup.
Once we’ve got a database connection in place and UAC module loaded, then we can configure an entry in the uacreg table in the database, in my example I’m going to be registering to an Asterisk box on 192.168.1.205, so I’ll insert that into my database:
mysql> INSERT INTO `uacreg` VALUES (NULL,'myusername','myusername','192.168.1.205','myusername','192.168.1.205','asterisk','myusername','mypassword','','sip:192.168.1.205:5060',60,0,0);
Note: If you’re using a later version of Kamailio (5.4+) then the DB schema changes and you may want something like this:
Having a look at the fields in our table makes it a bit clearer as to what we’ve got in place, setting flags to 0 will see Kamailio attempt registration. Make sure the auth_proxy is a SIP URI (Starts with sip:) and leave the auth_ha1 password empty as we haven’t calculated it.
After we’ve got our database connection in place, UAC module configured and database entries added, it’s time to put it into play, we’ll use Kamcmd to check it’s status:
kamcmd> uac.reg_reload
kamcmd> uac.reg_dump
Unfortunately from Kamcmd we’re not able to see registration status, but Sngrep will show us what’s going on:
From Sngrep we can see the REGISTRATION going out, the authentication challenge and the 200 OK at the end.
Make sure you’ve got your Realm correct, otherwise you may see an error like this:
RROR: {2 10 REGISTER [email protected]} uac [uac_reg.c:946]: uac_reg_tm_callback(): realms do not match. requested realm: [localhost]
Kamailio is a great SIP proxy, but sometimes you might want to see requests originate from Kamailio.
While this isn’t typical proxy behaviour, RFC definitions of a proxy and technical requirements are often two different things. The UAC module allows us to use Kamailio to act as a User Agent Client instead of just a UAS.
There’s one feature I won’t cover in this post, and that’s initiating and outbound SIP Registration using the UAC module, that will get a post of it’s own in the not to distant future.
You may already be sort of using Kamailio is a UAC, if you’re using Dispatcher and sending SIP Pings, then Kamailio is sending SIP OPTIONS messages to the dispatcher destinations. If you’re using the NAT module and sending Keepalives, then you’re also using Kamailio as a UAC. The only difference is the Dispatcher and NAT Helper modules do this for us, and we’re going to originate our own traffic.
There’s a bit of a catch here, when Kamailio receives a request it follows a set of logic and does something with that request. We’re going to remain constrained by this for our example, just to keep things simple.
So let’s work on an example, if a user on our network dials a call to an emergency services number, we’ll send a text message to my IP phone to let me know who’s dialed the emergency services number.
So to start with we’ll need to load the Kamailio UAC module, using LoadModule as we would with any other module:
loadmodule "uac.so"
If you’re working on the default config file that ships with Kamailio you’ll probably have to change how record routing is handled to support UAC,
modparam("rr", "append_fromtag", 1)
Now we should have UAC support added in Kamailio, I’m going to do a bare bones example of the routing logic below, but obviously if you wanted to put this into practice in real life you’d want to actually route the SIP INVITE to an emergency services destination.
First we’ll need to find if the request is an INVITE with the Request URI to an emergency services number, I’ve programmed this in with the Australian emergency services numbers:
if(is_method("INVITE") && ($rU == "000" or $tU == "112" or $tU == "116")){
#Matches any INVITEs with the Request URI to Address as 000, 112 or 116
xlog("Emergency call from $fU to $rU (Emergency number) CSeq is $cs ");
}
Now calls to 000, 112 or 116 will see the alert apear in Xlog:
07:22:41 voice-dev3 /usr/sbin/kamailio[10765]: ERROR: : Emergency call from Test to 112 (Emergency number)
So next up we need to handle the sending a SIP MESSAGE request to my IP phone on the IP 10.0.1.5 – You’re probably thinking we could use the Registrar module to lookup my registered IP address, and you’re right, but to keep things simple I’m just hardcoding it in.
So to keep our routing neat we’ll send calls to the route route(“EmergencyNotify”); and so the demo works I’ll send back a 200 OK and exit – In real life you’d want to handle this request and forward it onto emergency services.
if(is_method("INVITE") && ($rU == "000" or $tU == "112" or $tU == "116")){
#Matches any INVITEs with the Request URI to Address as 000, 112 or 116
xlog("Emergency call from $fU to $rU (Emergency number) CSeq is $cs ");
route("EmergencyNotify");
#You obviously would want this to route to an emergency services destination...
sl_reply("200", "ok");
exit;
}
if(is_method("INVITE")){
#Matches everything else
xlog("Just a regular call from $fU to $rU");
}
Obviously we need to now create a route called route[“EmergencyNotify”]{ } where we’ll put our UAC logic.
For the UAC module we need to craft the SIP Request we’re going to send; we’re going to be sending a SIP MESSAGE request,
So now we’ve sort of put it all together, when a call comes into an emergency destination, like 000, the route EmergencyNotify is called which sends a SIP MESSAGE request to my IP Phone to alert me.
When a caller dials 000 I can see Kamailio sends a SIP MESSAGE to my IP Phone:
Let’s have a look at how this looks on my IP Phone:
For most Voice / Telco engineers IPsec is a VPN technology, maybe something used when backhauling over an untrusted link, etc, but voice over IP traffic is typically secured with TLS and SRTP.
IMS / Voice over LTE handles things a bit differently, it encapsulates the SIP & RTP traffic between the UE and the P-CSCF in IPsec Encapsulating Security Payload (ESP) payloads.
In this post we’ll take a look at how it works and what it looks like.
It’s worth noting that Kamailio recently added support for IPsec encapsulation on a P-CSCF, in the IMS IPSec-Register module. I’ll cover usage of this at a later date.
The Message Exchange
The exchange starts off looking like any other SIP Registration session, in this case using TCP for transport. The UE sends a REGISTER to the Proxy-CSCF which eventually forwards the request through to a Serving-CSCF.
This is where we diverge from the standard SIP REGISTER message exchange. The Serving-CSCF generates a 401 Unauthorized response, containing an authentication challenge in the WWW-Authenticate header, and also a Ciphering Key & Integrity Key (ck= and ik=) also in the WWW-Authenticate header.
The Serving-CSCF sends the Proxy-CSCF the 401 response it created. The Proxy-CSCF assigns a SPI for the IPsec ESP to use, a server port and client port and indicates the used encryption algorithm (ealg) and algorithm to use (In this case HMAC-SHA-1-96.) and adds a new header to the 401 Unauthorized called Security–Server header to share this information with the UE.
The Proxy-CSCF also strips the Ciphering Key (ck=) and Integrity Key (ik=) headers from the SIP authentication challenge (WWW-Auth) and uses them as the ciphering and integrity keys for the IPsec connection.
Finally after setting up the IPsec server side of things, it forwards the 401 Unauthorized response onto the UE.
Upon receipt of the 401 response, the UE looks at the authentication challenge.
If the network is considered authenticated by the UE it generates a response to the Authentication Challenge, but it doesn’t deliver it over TCP. Using the information generated in the authentication challenge the UE encapsulates everything from the network layer (IPv4) up and sends it to the P-CSCF in an IPsec ESP.
Communication between the UE and the P-CSCF is now encapsulated in IPsec.
Wireshark trace of IPsec IMS Traffic between UE and P-CSCF
When learning to use Kamailio you might find yourself thinking about if you really want to learn to write a Kamailio configuration file, which is another weird scripting language to learn to achieve a task.
Enter KEMI – Kamailio Embedded Interface. KEMI allows you to abstract the routing logic to another programing language. In layman’s terms this means you can write your routing blocks, like request_route{}, reply_route{}, etc, in languages you already know – like Lua, JavaScript, Ruby – and my favorite – Python!
Why would you use KEMI?
Write in a language you already know;
You don’t need to learn how to do write complex routing logic in Kamailio’s native scripting language, you can instead do it in a language you’re already familiar with, writing your Routing Blocks in another programming language.
Change Routing on the Fly;
By writing the routing logic in KEMI allows you to change your routing blocks without having to restart Kamailio, something you can’t do with the “native” scripting language – This means you can change your routing live.
Note: This isn’t yet in place for all languages – Some still require a restart.
Leverage your prefered language’s libraries;
While Kamailio’s got a huge list of modules to interface with a vast number of different things, the ~200 Kamailio modules don’t compare with the thousands of premade libraries that exist for languages like Python, Ruby, JavaScript, etc.
Prerequisites
We’ll obviously need Kamailio installed, but we’ll also need the programming language we want to leverage setup (fairly obvious).
Configuring Kamailio to talk to KEMI
KEMI only takes care of the routing of SIP messages inside our routing blocks – So we’ve still got the Kamailio cfg file (kamailio.cfg) that we use to bind and setup the service as required, load the modules we want and configure them.
Essentially we need to load the app for the language we use, in this example we’ll use app_python3.so and use that as our Config Engine.
IPsec ESP can be used in 3 different ways on the Gm interface between the Ue and the P-CSCF:
Integrity Protection – To prevent tampering
Ciphering – To prevent inception / eavesdropping
Integrity Protection & Ciphering
On Wireshark, you’ll see the ESP, but you won’t see the payload contents, just the fact it’s an Encapsulated Security Payload, it’s SPI and Sequence number.
By default, Kamailio’s P-CSCF only acts in Integrity Protection mode, meaning the ESP payloads aren’t actually encrypted, with a few clicks we can get Wireshark to decode this data;
Just open up Wireshark Preferences, expand Protocols and jump to ESP
Now we can set the decoding preferences for our ESP payloads,
In our case we’ll tick the “Attempt to detect/decode NULL encrypted ESP payloads” box and close the box by clicking OK button.
Now Wireshark will scan through all the frames again, anything that’s an ESP payload it will attempt to parse.
Now if we go back to the ESP payload with SQN 1 I showed a screenshot of earlier, we can see the contents are a TCP SYN.
Now we can see what’s going on inside this ESP data between the P-CSCF and the UE!
As a matter of interest if you can see the IK and CK values in the 401 response before they’re stripped you can decode encrypted ESP payloads from Wireshark, from the same Protocol -> ESP section you can load the Ciphering and Integrity keys used in that session to decrypt them.
On top of plain vanilla RFC3261, there’s a series of “Extension” methods added to SIP to expand it’s functionality, common extension methods are INFO, MESSAGE, NOTIFY, PRACK and UPDATE. Although now commonplace, of these is not defined in RFC3261 so is considered an “extension” to SIP.
It’s worth just pausing here to reiterate we’re not talking extensions like in a PBX context, like extra phones, we’re talking extensions like you’d add to a house, like extra functionality.
A SIP client can request functionality from a server (UAC to a UAS), if the server does not have support for that functionality, it can reject the session on those grounds and send back a response indicating it doesn’t know how to handle that extension, like a 420 Bad Extension – Bad SIP Protocol Extension used, not understood by the server. Response.
So clients can determine what functionality a server doesn’t support if it rejects the request, but there was no way to see what functionality the server does support, and what functionality the client requires.
If a UAC or UAS requires support for an extension – For example a Media Gateway has to understand PRACK, it can use the Require header to specify the request should be rejected if support for the listed extensions is not provided.
These headers are most commonly seen in SIP OPTIONS requests.
Kamailio is generally thought of as a SIP router, but it can in fact handle Diameter signaling as well.
Everything to do with Diameter in Kamailio relies on the C Diameter Peer and CDP_AVP modules which abstract the handling of Diameter messages, and allow us to handle them sort of like SIP messages.
CDP on it’s own doesn’t actually allow us to send Diameter messages, but it’s relied upon by other modules, like CDP_AVP and many of the Kamailio IMS modules, to handle Diameter signaling.
Before we can start shooting Diameter messages all over the place we’ve first got to configure our Kamailio instance, to bring up other Diameter peers, and learn about their capabilities.
C Diameter Peer (Aka CDP) manages the Diameter connections, the Device Watchdog Request/Answers etc, all in the background.
We’ll need to define our Diameter peers for CDP to use so Kamailio can talk to them. This is done in an XML file which lays out our Diameter peers and all the connection information.
In our Kamailio config we’ll add the following lines:
This will load the CDP modules and instruct Kamailio to pull it’s CDP info from an XML config file at /etc/kamailio/diametercfg.xml
Let’s look at the basic example given when installed:
<?xml version="1.0" encoding="UTF-8"?>
<!--
DiameterPeer Parameters
- FQDN - FQDN of this peer, as it should apper in the Origin-Host AVP
- Realm - Realm of this peer, as it should apper in the Origin-Realm AVP
- Vendor_Id - Default Vendor-Id to appear in the Capabilities Exchange
- Product_Name - Product Name to appear in the Capabilities Exchange
- AcceptUnknownPeers - Whether to accept (1) or deny (0) connections from peers with FQDN
not configured below
- DropUnknownOnDisconnect - Whether to drop (1) or keep (0) and retry connections (until restart)
unknown peers in the list of peers after a disconnection.
- Tc - Value for the RFC3588 Tc timer - default 30 seconds
- Workers - Number of incoming messages processing workers forked processes.
- Queue - Length of queue of tasks for the workers:
- too small and the incoming messages will be blocked too often;
- too large and the senders of incoming messages will have a longer feedback loop to notice that
this Diameter peer is overloaded in processing incoming requests;
- a good choice is to have it about 2 times the number of workers. This will mean that each worker
will have about 2 tasks in the queue to process before new incoming messages will start to block.
- ConnectTimeout - time in seconds to wait for an outbound TCP connection to be established.
- TransactionTimeout - time in seconds after which the transaction timeout callback will be fired,
when using transactional processing.
- SessionsHashSize - size of the hash-table to use for the Diameter sessions. When searching for a
session, the time required for this operation will be that of sequential searching in a list of
NumberOfActiveSessions/SessionsHashSize. So higher the better, yet each hashslot will consume an
extra 2xsizeof(void*) bytes (typically 8 or 16 bytes extra).
- DefaultAuthSessionTimeout - default value to use when there is no Authorization Session Timeout
AVP present.
- MaxAuthSessionTimeout - maximum Authorization Session Timeout as a cut-out measure meant to
enforce session refreshes.
-->
<DiameterPeer
FQDN="pcscf.ims.smilecoms.com"
Realm="ims.smilecoms.com"
Vendor_Id="10415"
Product_Name="CDiameterPeer"
AcceptUnknownPeers="0"
DropUnknownOnDisconnect="1"
Tc="30"
Workers="4"
QueueLength="32"
ConnectTimeout="5"
TransactionTimeout="5"
SessionsHashSize="128"
DefaultAuthSessionTimeout="60"
MaxAuthSessionTimeout="300"
>
<!--
Definition of peers to connect to and accept connections from. For each peer found in here
a dedicated receiver process will be forked. All other unkwnown peers will share a single
receiver. NB: You must have a peer definition for each peer listed in the realm routing section
-->
<Peer FQDN="pcrf1.ims.smilecoms.com" Realm="ims.smilecoms.com" port="3868"/>
<Peer FQDN="pcrf2.ims.smilecoms.com" Realm="ims.smilecoms.com" port="3868"/>
<Peer FQDN="pcrf3.ims.smilecoms.com" Realm="ims.smilecoms.com" port="3868"/>
<Peer FQDN="pcrf4.ims.smilecoms.com" Realm="ims.smilecoms.com" port="3868"/>
<Peer FQDN="pcrf5.ims.smilecoms.com" Realm="ims.smilecoms.com" port="3868"/>
<Peer FQDN="pcrf6.ims.smilecoms.com" Realm="ims.smilecoms.com" port="3868"/>
<!--
Definition of incoming connection acceptors. If no bind is specified, the acceptor will bind
on all available interfaces.
-->
<Acceptor port="3868" />
<Acceptor port="3869" bind="127.0.0.1" />
<Acceptor port="3870" bind="192.168.1.1" />
<!--
Definition of Auth (authorization) and Acct (accounting) supported applications. This
information is sent as part of the Capabilities Exchange procedures on connecting to
peers. If no common application is found, the peers will disconnect. Messages will only
be sent to a peer if that peer actually has declared support for the application id of
the message.
-->
<Acct id="16777216" vendor="10415" />
<Acct id="16777216" vendor="0" />
<Auth id="16777216" vendor="10415"/>
<Auth id="16777216" vendor="0" />
<!--
Supported Vendor IDs - list of values which will be sent in the CER/CEA in the
Supported-Vendor-ID AVPs
-->
<SupportedVendor vendor="10415" />
<!--
Realm routing definition.
Each Realm can have a different table of peers to route towards. In case the Destination
Realm AVP contains a Realm not defined here, the DefaultRoute entries will be used.
Note: In case a message already contains a Destination-Host AVP, Realm Routeing will not be
applied.
Note: Routing will only happen towards connected and application id supporting peers.
The metric is used to order the list of prefered peers, while looking for a connected and
application id supporting peer. In the end, of course, just one peer will be selected.
-->
<Realm name="ims.smilecoms.com">
<Route FQDN="pcrf1.ims.smilecoms.com" metric="3"/>
<Route FQDN="pcrf2.ims.smilecoms.com" metric="5"/>
</Realm>
<Realm name="temp.ims.smilecoms.com">
<Route FQDN="pcrf3.ims.smilecoms.com" metric="7"/>
<Route FQDN="pcrf4.ims.smilecoms.com" metric="11"/>
</Realm>
<DefaultRoute FQDN="pcrf5.ims.smilecoms.com" metric="15"/>
<DefaultRoute FQDN="pcrf6.ims.smilecoms.com" metric="13"/>
</DiameterPeer>
First we need to start by telling CDP about the Diameter peer it’s going to be – we do this in the <DiameterPeer section where we define the FQDN and Diameter Realm we’re going to use, as well as some general configuration parameters.
<Peers are of course, Diameter peers. Defining them here will mean a connection is established to each one, Capabilities exchanged and Watchdog request/responses managed. We define the usage of each Peer further on in the config.
The Acceptor section – fairly obviously – sets the bindings for the addresses and ports we’ll listen on.
Next up we need to define the Diameter applications we support in the <Acct id=” /> and <SupportedVendor> parameters, this can be a little unintuitive as we could list support for every Diameter application here, but unless you’ve got a module that can handle those applications, it’s of no use.
Instead of using Dispatcher to manage sending Diameter requests, CDP handles this for us. CDP keeps track of the Peers status and it’s capabilities, but we can group like Peers together, for example we may have a pool of PCRF NEs, so we can group them together into a <Realm >. Instead of calling a peer directly we can call the realm and CDP will dispatch the request to an up peer inside the realm, similar to Dispatcher Groups.
Finally we can configure a <DefaultRoute> which will be used if we don’t specify the peer or realm the request needs to be sent to. Multiple default routes can exist, differentiated based on preference.
We can check the status of peers using Kamcmd’s cdp.list_peers command which lists the peers, their states and capabilities.
One question that’s not as obvious as it perhaps should be is the different states shown with kamcmd dispatcher.list command;
So what do the flags for state mean?
The first letter in the flag means is the current state, Active (A), Inactive (I) or Disabled (D).
The second letter in the flag means monitor status, Probing (P) meaning actively checked with SIP Options pings, or Not Set (X) denoting the device isn’t actively checked with SIP Options pings.
AP – Actively Probing – SIP OPTIONS are getting a response, routing to this destination is possible, and it’s “Up” for all intents and purposes.
IP – Inactively Probing – Destination is not meeting the threshold of SIP OPTIONS request responses it needs to be considered active. The destination is either down or not responding to all SIP OPTIONS pings. Often this is due to needing X number of positive responses before considering the destination as “Up”.
DX – Disabled & Not Probing – This device is disabled, no SIP OPTIONS are sent.
AX – Active & Not Probing– No SIP OPTIONS are sent to check state, but is is effectively “Up” even though the remote end may not be reachable.
In the third part of the Kamailio 101 series I briefly touched upon pseudovariables, but let’s look into what exactly they are and how we can manipulate them to change headers.
The term “pseudo-variable” is used for special tokens that can be given as parameters to different script functions and they will be replaced with a value before the execution of the function.
You’ve probably seen in any number of the previous Kamailio Bytes posts me use pseudovariables, often in xlog or in if statements, they’re generally short strings prefixed with a $ sign like $fU, $tU, $ua, etc.
When Kamailio gets a SIP message it explodes it into a pile of variables, getting the To URI and putting it into a psudovariable called $tU, etc.
We can update the value of say $tU and then forward the SIP message on, but the To URI will now use our updated value.
When it comes to rewriting caller ID, changing domains, manipulating specific headers etc, pseudovariables is where it mostly happens.
Kamailio allows us to read these variables and for most of them rewrite them – But there’s a catch. We can mess with the headers which could result in our traffic being considered invalid by the next SIP proxy / device in the chain, or we could mess with the routing headers like Route, Via, etc, and find that our responses never get where they need to go.
So be careful! Headers exist for a reason, some are informational for end users, others are functional so other SIP proxies and UACs can know what’s going on.
Rewriting SIP From Username Header (Caller ID)
When Kamailio’s SIP parser receives a SIP request/response it decodes the vast majority of the SIP headers into a variety of pseudovariables, we can then reference these variables we can then reference from our routing logic.
Let’s pause here and go back to the Stateless SIP Proxy Example, as we’ll build directly on that.
Follow the instructions in that post to get your stateless SIP proxy up and running, and we’ll make this simple change:
####### Routing Logic ########
/* Main SIP request routing logic
* - processing of any incoming SIP request starts with this route
* - note: this is the same as route { ... } */
request_route {
xlog("Received $rm to $ru - Forwarding");
$fU = "Nick Blog Example"; #Set From Username to this value
#Forward to new IP
forward("192.168.1.110");
}
Now when our traffic is proxied the From Username will show “Nick Blog Example” instead of what it previously showed.
The blurry photo didn’t make anything that much clearer, but they looked like two motion switches, and being a big fan of really old telco hardware, I found myself driving to an auction selling things very much unrelated to telephone exchanges to bid on what I thought might have been a step-by-step exchange.
Photo from listing
$50 later I am now the proud owner of an Automatic Telephone & Electric Co (ATE) Liverpool works 50 line PAX (Private Automatic Exchange).
The switch
My office now has less room, a big burly battery eliminator and ring machine take up the space on my desk, but I couldn’t be happier with it.
Uniselectors
Of the 5 final selectors I’ve got two somehow worked “out of the box”, while the other 3 all need some serious adjustment, but she clicks and she’s mostly complete, so should be a good summer holiday project!
I’ll post some video up when she’s fully functional.
While poking around the development and debugging features on Samsung handsets I found the ability to run IMS Debugging directly from the handset.
Alas, the option is only available in the commercial version, it’s just there for carriers, and requires a One Time Password to unlock.
When tapping on the option a challenge is generated with a key.
Interestingly I noticed that the key changes each time and can reject you even in aeroplane mode, suggesting the authentication happens client side.
This left me thinking – If the authentication happens client side, then the App has to know what the valid password for the key shown is…
Some research revealed you can pull APKs off an Android phone, so I downloaded a utility called “APK Extractor” from the Play store, and used it to extract the Samsung Sysdump utility.
So now I was armed with the APK on my local machine, the next step was to see if I could decompile the APK back into source code.
Some Googling found me an online APK decompiler, which I fed the compiled APK file and got back the source code.
I did some poking around inside the source code, and then I found an interesting directory:
Here’s a screenshot of the vanilla code that came out of the app.
I’m not a Java expert, but even I could see the “CheckOTP” function and understand that that’s what validates the One Time Passwords.
The while loop threw me a little – until I read through the rest of the code; the “key” in the popup box is actually a text string representing the current UNIX timestamp down to the minute level. The correct password is an operation done on the “key”, however the CheckOTP function doesn’t know the challenge key, but has the current time, so generates a challenge key for each timestamp back a few minutes and a few minutes into the future.
I modified the code slightly to allow me to enter the presented “key” and get the correct password back. It’s worth noting you need to act quickly, enter the “key” and enter the response within a minute or so.
In the end I’ve posted the code on an online Java compiler,
Samsung handsets have a feature built in to allow debugging from the handset, called Sysdump.
Entering *#9900# from the Dialing Screen will bring up the Sysdump App, from here you can dump logs from the device, and run a variety of debugging procedures.
But for private LTE operators, the two most interesting options are by far the TCPDUMP START option and IMS Logger, but both are grayed out.
Tapping on them asks for a one-time password and has a challenge key.
These options are not available in the commercial version of the OS and need to be unlocked with a one time key generated by a tool Samsung for unlocking engineering firmware on handsets.
Luckily this authentication happens client side, which means we can work out the password it’s expecting.
Once you’ve entered the code and successfully unlocked the IMS Debugging tool there’s a few really cool features in the hamburger menu in the top right.
DM View
This shows the SIP / IMS Messaging and the current signal strength parameters (used to determine which RAN type to use (Ie falling back from VoLTE to UMTS / Circuit Switched when the LTE signal strength drops).
Tapping on the SIP messages expands them and allows you to see the contents of the SIP messages.
Viewing SIP Messaging directly from the handset
Interesting the actual nitty-gritty parameters in the SIP headers are missing, replaced with X for anything “private” or identifiable.
Luckily all this info can be found in the Pcap.
The DM View is great for getting a quick look at what’s going on, on the mobile device itself, without needing a PC.
Logging
The real power comes in the logging functions,
There’s a lot of logging options, including screen recording, TCPdump (as in Packet Captures) and Syslog logging.
From the hamburger menu we can select the logging parameters we want to change.
From the Filter Options menu we can set what info we’re going to log,
There’s always a lot of talk and opinion about the technologies the NBN employs, it’s effectiveness, etc.
I’ve made a conscious decision to steer clear of opinion in this blog, but there’s often talk and blame shifting between NBNco and RSPs, so I thought I’d cover how the business model works.
Because of this I thought it’d be interesting to write about how the network actually works between carriers (RSPs) and NBNco.
Physical Structure
Last Mile
The last mile in US terms, CAN in Australian Telecom lingo, is connecting the subscriber edge to the network.
NBNCo employs a few different technologies for this, depending on a number of factors;
Fiber to the Premises (Original standard – End to end fibre)
All these last mile services get consolidated and eventually end up at a local PoI – Point of Interconnect, (typically called a POP if you’re any other telco).
These are typically hosted inside exchanges, but not every exchange is an NBNco PoI, if it’s not it uses NBN Backhaul to get to the nearest PoI.
NBNco currently operates 121 PoI sites.
NBNco don’t exclusively use TEBA sites, some are hosted in NBNco “Depots”, there’s currently 10 sites not in TEBA footprints.
At the PoI
Retail Service Providers (RSPs) have to have racks inside the PoI locations, and essentially setup layer 2 cross-connects to the NBNco racks.
Once the traffic is on the RSP network, it’s the RSP’s responsibility to carry it where it needs to go, via their own network / backhaul.
Billing and Metering
Of course, if NBNco is handing off the pipes of customer traffic off to each RSP they need a way to charge the RSPs for this, this is handled by two elements – CVCs equating to the bandwidth at the PoI and AVCs equating to a fixed standing charge per connection monthly.
CVC – Connectivity Virtual Circuit
At the PoI the connection between the NBNco rack and the RSP rack is metered over a CVC – Connectivity Virtual Circuit.
This is shared across all users of that RSP at that PoI.
Let’s say I’m an RSP and I’ve purchased a 1Mbps CVC shared across my 1,000 customers at that PoI, the customers aren’t going to have a good experience.
Of course, CVC bandwidth isn’t free, previously NBNco charged on average $15.25/Mbps.
This had the effect of ensuring each RSP had just enough CVC bandwidth for their customers, but this led to some customers having a poor experience on switching to NBNco as they found their speeds dropped due to not enough CVC bandwidth at the PoI for that RSP.
In June 2017 NBNco announced a change to the pricing structure to try and encourage RSPs to buy more CVC bandwidth to ensure customers speeds weren’t bottlenecked at the CVC.
The new pricing structure makes it more financially attractive to buy more CVC bandwidth based on how many active connections (AVCs) an RSP has in place.
NBNco now charges $17.50 per symmetrical Mbps for each traffic class. (More on traffic classes later)
This means at each PoI the RSP must have a pool of CVC bandwidth large enough to meet the needs of all the customers connections (AVCs) bandwidth needs at that PoI.
AVC – Access Virtual Circuit
NBNco charges AVC fees based on the speed tier the end user will have and the traffic class (QoS) the service has applied.
(This speed tier is regardless of if the RSP has the CVC bandwidth to support this)
Pricing of TC4 (Best Effort) AVCs
Introducing NNIs
NBNco acknolged in Jul 2018 that for some carriers (RSPs) having presence in 121 sites puts up a large barrier to entry.
To counteract this they introduced Network-Network Interface (NNI).
Imagine you’re operating an RSP with a footprint in capital cities and PoI / CVCs in populated areas, you can’t serve customers in remote areas without having a presence at their local NBNco PoI location and buying CVC bandwidth for that location – It just wouldn’t stack up financially.
NBNco introduced the NNI product to essentially backhaul the traffic from these customers to the nearest PoI their RSP is at and share the CVC bandwidth at that PoI.
The PLMN Identifier is used to identify the radio networks in use, it’s made up of the MCC – Mobile Country Code and MNC – Mobile Network Code.
But sadly it’s not as simple as just concatenating MCC and MNC like in the IMSI, there’s a bit more to it.
In the example above the Tracking Area Identity includes the PLMN Identity, and Wireshark has been kind enough to split it out into MCC and MNC, but how does it get that from the value 12f410?
This one took me longer to work out than I’d like to admit, and saw me looking through the GSM spec, but here goes:
PLMN Contents: Mobile Country Code (MCC) followed by the Mobile Network Code (MNC). Coding: according to TS GSM 04.08 [14].
If storage for fewer than the maximum possible number n is required, the excess bytes shall be set to ‘FF’. For instance, using 246 for the MCC and 81 for the MNC and if this is the first and only PLMN, the contents reads as follows: Bytes 1-3: ’42’ ‘F6′ ’18’ Bytes 4-6: ‘FF’ ‘FF’ ‘FF’ etc.
TS GSM 04.08 [14].
Making sense to you now? Me neither.
Here’s the Python code I wrote to encode MCC and MNCs to PLMN Identifiers and to decode PLMN into MCC and MNC, and then we’ll talk about what’s happening:
In the above example I take MCC 505 (Australia) and MCC 93 and generate the PLMN ID 05f539.
The first step in decoding is to take the first two bits (in our case 05 and reverse them – 50, then we take the third and fourth bits (f5) and reverse them too, and strip the letter f, now we have just 5. We join that with what we had earlier and there’s our MCC – 505.
Next we get our MNC, for this we take bytes 5 & 6 (39) and reverse them, and there’s our MNC – 93.
Together we’ve got MCC 505 and MNC 93.
The one answer I’m still looking for; why not just encode 50593? What is gained by encoding it as 05f539?
After a few quiet months I’m excited to say I’ve pushed through some improvements recently to PyHSS and it’s growing into a more usable HSS platform.
MongoDB Backend
This has a few obvious advantages – More salable, etc, but also opens up the ability to customize more of the subscriber parameters, like GBR bearers, etc, that simple flat text files just wouldn’t support, as well as the obvious issues with threading and writing to and from text files at scale.
Knock knock.
Race condition.
Who’s there?
— Threading Joke.
For now I’m using the Open5GS MongoDB schema, so the Open5Gs web UI can be used for administering the system and adding subscribers.
The CSV / text file backend is still there and still works, the MongoDB backend is only used if you enable it in the YAML file.
The documentation for setting this up is in the readme.
SQN Resync
If you’re working across multiple different HSS’ or perhaps messing with some crypto stuff on your USIM, there’s a chance you’ll get the SQN (The Sequence Number) on the USIM out of sync with what’s on the HSS.
This manifests itself as an Update Location Request being sent from the UE in response to an Authentication Information Answer and coming back with a Re-Syncronization-Info AVP in the Authentication Info AVP. I’ll talk more about how this works in another post, but in short PyHSS now looks at this value and uses it combined with the original RAND value sent in the Authentication Information Answer, to find the correct SQN value and update whichever database backend you’re using accordingly, and then send another Authentication Information Answer with authentication vectors with the correct SQN.
SQN Resync is something that’s really cryptographically difficult to implement / confusing, hence this taking so long.
What’s next? – IMS / Multimedia Auth
The next feature that’s coming soon is the Multimedia Authentication Request / Answer to allow CSCFs to query for IMS Registration and manage the Cx and Dx interfaces.
Code for this is already in place but failing some tests, not sure if that’s to do with the MAA response or something on my CSCFs,
LTE has great concepts like NAS that abstract the actual transport layers, so the NAS packet is generated by the UE and then read by the MME.
One thing that’s a real headache about private LTE is the authentication side of things. You’ll probably bash your head against a SIM programmer for some time.
As your probably know when connecting to a network, the UE shares it’s IMSI / TIMSI with the network, and the MME requests authentication information from the HSS using the Authentication Information Request over Diameter.
The HSS then returns a random value (RAND), expected result (XRES), authentication token (AUTN) and a KASME for generating further keys,
The RAND and AUTN values are sent to the UE, the USIM in the UE calculates the RES (result) and sends it back to the MME. If the RES value received by the MME is equal to the expected RES (XRES) then the subscriber is mutually authenticated.
Using this tool I was able to plug a USIM into my USIM reader, using the Diameter client built into PyHSS I was able to ask for Authentication vectors for a UE using the Authentication Information Request to the HSS and was sent back the Authentication Information Answer containing the RAND and AUTN values, as well as the XRES value.
Diameter – Authentication Information Response showing E-UTRAN Vectors
Then I used the osmo-sim-auth app to query the RES and RAND values against the USIM.
The RES I got back matched the XRES, meaning the HSS and the USIM are in sync (SQNs match) and they mutually authenticated.
I thought I’d expand a little on how the Crypto side of things works in LTE & NR (also known as 4G & 5G).
Authentication primarily happens in two places, one at each end of the network, the Home Subscriber Server and in the USIM card. Let’s take a look at each of them.
On the USIM
On the USIM we’ve got two values that are entered in when the USIM is provisioned, the K key – Our secret key, and an OPc key (operator key).
These two keys are the basis of all the cryptography that goes on, so should never be divulged.
The only other place to have these two keys in the HSS, which associates each K key and OPc key combination with an IMSI.
The USIM also stores the SQN a sequence number, this is used to prevent replay attacks and is incremented after each authentication challenge, starting at 1 for the first authentication challenge and counting up from there.
On the HSS
On the HSS we have the K key (Secret key), OPc key (Operator key) and SQN (Sequence Number) for each IMSI on our network.
Each time a IMSI authenticates itself we increment the SQN, so the value of the SQN on the HSS and on the USIM should (almost) always match.
Authentication Options
Let’s imagine we’re designing the authentication between the USIM and the Network; let’s look at some options for how we can authenticate everyone and why we use the process we use.
Failed Option 1 – Passwords in the Clear
The HSS could ask the USIM to send it’s K and OPc values, compare them to what the HSS has in place and then either accept or reject the USIM depending on if they match.
The obvious problem with this that to send this information we broadcast our supposedly secret K and OPc keys over the air, so anyone listening would get our secret values, and they’re not so secret anymore.
This is why we don’t use this method.
Failed Option 2 – Basic Crypto
So we’ve seen that sending our keys publicly, is out of the question.
The HSS could ask the USIM to mix it’s K key and OPc key in such a way that only someone with both keys could unmix them.
This is done with some cryptographic black magic, all you need to know is it’s a one way function you enter in values and you get the same result every time with the same input, but you can’t work out the input from the result.
The HSS could then get the USIM to send back the result of mixing up both keys, mix the two keys it knows and compare them.
The HSS mixes the two keys itself, and get’s it’s own result called XRES (Expected Result). If the RES (result) of mixing up the keys by the USIM is matches the result when the HSS mixes the keys in the same way (XRES (Expected Result)), the user is authenticated.
The result of mixing the keys by the USIM is called RES (Result), while the result of the HSS mixing the keys is called XRES (Expected Result).
This is abetter solution but has some limitations, because our special mixing of keys gets the same RES each time we put in our OPc and K keys each time a subscriber authenticates to the network the RES (result) of mixing the keys is going to be the same.
This is vulnerable to replay attacks. An attacker don’t need to know the two secret keys (K & OPc) that went into creating the RES (resulting output) , the attacker would just need to know the result of RES, which is sent over the air for anyone to hear. If the attacker sends the same RES they could still authenticate.
This is why we don’t use this method.
Failed Option 3 – Mix keys & add Random
To prevent these replay attacks we add an element of randomness, so the HSS generates a random string of garbage called RAND, and sends it to the USIM.
The USIM then mixes RAND (the random string) the K key and OPc key and sends back the RES (Result).
Because we introduced a RAND value, every time the RAND is different the RES is different. This prevents against the replay attacks we were vulnerable to in our last example.
If the result the USIM calculated with the K key, OPc key and random data is the same as the USIM calculated with the same K key, OPc key and same random data, the user is authenticated.
While an attacker could reply with the same RES, the random data (RAND) will change each time the user authenticates, meaning that response will be invalid.
While an attacker could reply with the same RES, the random data (RAND) will change each time the user authenticates, meaning that response will be invalid.
The problem here is now the network has authenticated the USIM, the USIM hasn’t actually verified it’s talking to the real network.
This is why we don’t use this method.
GSM authentication worked like this, but in a GSM network you could setup your HLR (The GSM version of a HSS) to allow in every subscriber regardless of what the value of RES they sent back was, meaning it didn’t look at the keys at all, this meant attackers could setup fake base stations to capture users.
Option 4 – Mutual Authentication (Real World*)
So from the previous options we’ve learned:
Our network needs to authenticate our subscribers, in a way that can’t be spoofed / replayed so we know who to bill & where to route traffic.
Our subscribers need to authenticate the network so they know they can trust it to carry their traffic.
So our USIM needs to authenticate the network, in the same way the network authenticates the USIM.
To do this we introduce a new key for network authentication, called AUTN.
The AUTN key is generated by the HSS by mixing the secret keys and RAND values together, but in a different way to how we mix the keys to get RES. (Otherwise we’d get the same key).
This AUTN key is sent to the USIM along with the RAND value. The USIM runs the same mixing on it’s private keys and RAND the HSS did to generate the AUTN , except this is the USIM generated – An Expected AUTN key (XAUTN). The USIM compares XAUTN and AUTN to make sure they match. If they do, the USIM then knows the network knows their secret keys.
The USIM then does the same mixing it did in the previous option to generate the RES key and send it back.
The network has now authenticated the subscriber (HSS has authenticated the USIM via RES key) and the subscriber has authenticated the USIM (USIM authenticates HSS via AUTN key).
*This is a slightly simplified version of how EUTRAN / LTE authentication works between the HSS and the USIM – In reality there are a few extra values, such as SQN to take into consideration and the USIM talks to to the MME not the HSS directly.
I’ll do a follow up post covering the more nitty-gritty elements, AMF and SQN fields, OP vs OPc keys, SQN Resync, how this information is transfered in the Authentication Information Answer and how KASME keys are used / distributed.
Want more telecom goodness?
I have a good old fashioned RSS feed you can subscribe to.
