Tag Archives: SIP

Playing back AMR streams from Packet Captures

The other day I found myself banging my head on the table to diagnose an issue with Ringback tone on an SS7 link and the IMS.

On the IMS side, no RBT was heard, but I could see the Media Gateway was sending RTP packets to the TAS, and the TAS was sending it to the UE, but was there actual content in the RTP packets or was it just silence?

If this was PCM / G711 we’d be able to just playback in Wireshark, but alas we can’t do this for the AMR codec.

Filter the RTP stream out in Wireshark

Inside Wireshark I filtered each of the audio streams in one direction (one for the A-Party audio and one for the B-Party audio)

Then I needed to save each of the streams as a separate PCAP file (Not PCAPng).

Turn into AMR File

With the audio stream for one direction saved, we can turn it into an AMR file, using Juan Noguera (Spinlogic)’s AMR Extractor tool.

Clone the Repo from git, and then in the same directory run:

python3 pcap_parser.py -i AMR_B_Leg.pcap -o AMR_B_Leg.3ga

Playback with VLC / Audacity

I was able to play the file with VLC, and load it into Audacity to easily see that yes, the Ringback Tone was present in the AMR stream!

CGrateS MySQL Rounding Error

I put my rates in with a stack of decimal points, because accuracy matters!

But when I manually calculated the outputted costs associated with each transaction, I seemed to have some rounding errors.

So what was the issue?

The schema in MySQL was set to DECIMAL(10,4) which gives us 10 digits after the decimal point and 4 digits after.

A quick alter table and a reimport of the rates and I was on my way!

"alter table tp_rates modify column  rate DECIMAL(10,10);"

Lesson learned and hopefully of use to any other CGrateS users who may be using MySQL as a StoreDB.

FreeSWITCH Bridge Timers 101

A cheat sheet for anyone trying to control FreeSWITCH bridge behaviour if you’re trying to move calls around if not answered / responded to:

originate_timeout

How long to wait for any response to from the remote peer (100 TRYING, 180 RINGING, etc).

This is useful for knowing when to give up and try a different peer as this peer is dead.

originate_retries

How many times to retransmit the INVITE if no 100 TRYING / 180 RINGING is received.

Like originate_timeout, this is handy for giving up sooner when a peer is dead and moving onto others.

progress_timeout

How long we wait between sending the SIP INVITE before we get a 180/183 before we give up.

This is handy to find out if the remote end isn’t able to reach the endpoint you’re after (page the UE in a cellular context).

bridge_answer_timeout

How long do we wait between the INVITE and a 200 OK (Including RINGING) – This is useful for “no answer” timeouts.

If you want to know why a bridge failed, ie no answer timeout reached, error on the remote end, etc, we can see why with the following variable:

variable_last_bridge_hangup_cause: [PROGRESS_TIMEOUT]

Which will allow you to tell if it’s no answer or progress timeout to blame.

FreeSWITCH – Keep Call-ID the same on both legs of a Bridged Call

I needed to have both legs of the B2BUA bridge call through FreeSWITCH using the same Call-ID (long story), and went down the rabbit hole of looking for how to do this.

A post from 15 years ago on the mailing list from Anthony Minessale said he added “sip_outgoing_call_id” variable for this, and I found the commit, but it doesn’t work – More digging shows this variable disappears somewhere in history.

But by looking at what it changed I found sip_invite_call_id does the same thing now, so if you want to make both legs use the same Call-ID here ya go:

<action application="set" data="sip_invite_call_id=mycustomcallid"/>

Yeah, this post probably could’ve been a Tweet….

RTPengine – Installation & Configuration (Ubuntu 20.04 / 22.04)

I wrote a post a few years back covering installing RTPengine on Ubuntu (14.04 / 18.04) but it doesn’t apply in later Ubuntu releases such as 20.04 and 22.04.

To make everyone’s lives easier; David Lublink publishes premade repos for Ubuntu Jammy (22.04) & Focal (20.04).

Note: It looks like Ubuntu 23.04 includes RTPengine in the standard repos, so this won’t be needed in the future.

sudo add-apt-repository ppa:davidlublink/rtpengine
sudo apt update
sudo apt-get install ngcp-rtpengine

The Ambient Capabilities in the systemctl file bit me,

Commenting out :

#AmbientCapabilities=CAP_NET_ADMIN CAP_SYS_NICE

In /lib/systemd/system/ngcp-rtpengine-daemon.service and then reloading the service and restarting and I was off and running:

systemctl daemon-reload
systemctl restart rtpengine

Getting it Running

Now we’ve got RTPengine installed let’s setup the basics,

There’s an example config file we’ll copy and edit:

vi /etc/rtpengine/rtpengine.conf

We’ll uncomment the interface line and set the IP to the IP we’ll be listening on:

Once we’ve set this to our IP we can start the service:

systemctl restart rtpengine

All going well it’ll start and rtpengine will be running.

You can learn about all the startup parameters and what everything in the config means in the readme.

Want more RTP info?

If you want to integrate RTPengine with Kamailio take a look at my post on how to set up RTPengine with Kamailio.

For more in-depth info on the workings of RTP check out my post RTP – More than you wanted to Know

Authenticating Fixed Line Subscribers into IMS

We recently added support in PyHSS for fixed line SIP subscribers to attach to the IMS.

Traditional telecom operators are finding their fixed line network to be a bit of a money pit, something they’re required to keep operating to meet regulatory obligations, but the switches are sitting idle 99% of the time. As such we’re seeing more and more operators move fixed line subs onto their IMS.

This new feature means we can use PyHSS to serve as the brains for a fixed network, as well as for mobile, but there’s one catch – How we authenticate subscribers changes.

Most banks of line cards in a legacy telecom switches, or IP Phones, don’t have SIM slots to allow us to authenticate, so instead we’re forced to fallback to what they do support.

Unfortunately for the most part, what is supported by these IP phones or telecom switches is SIP MD5 Digest Authentication.

The Nonce is generated by the HSS and put into the Multimedia-Authentication-Answer, along with the subscriber’s password and sent in the clear to the S-CSCF.

Subscriber with Password made up of all 1's MAA response from HSS for Digest-MD5 Auth

The HSS then generates the the Multimedia-Auth Answer, it generates a nonce (in the 3GPP-SIP-Authenticate / 609 AVP) and sends the Subscriber’s password in the 3GPP-SIP-Authorization (610) AVP in response back to the S-CSCF.

I would have thought a better option would be for the HSS to generate the Nonce and Digest, and then the S-CSCF to just send the Nonce to the Sub and compare the returned Digest from the Sub against the expected Digest from the HSS, but it would limit flexibility (realm adaptation, etc) I guess.

The UE/UA (I guess it’s a UA in this context as it’s not a mobile) then generates its own Digest from the Nonce and sends it back to the S-CSCF via the P-CSCF.

The S-CSCF compares the received Digest response against the one it generated, and if the two match, the sub is authenticated and allowed to attach onto the network.

IMS iFC – SPT Session Cases

Mostly just reference material for me:

Possible values:

  • 0 (ORIGINATING_SESSION)
  • 1 TERMINATING_REGISTERED
  • 2 (TERMINATING_UNREGISTERED)
  • 3 (ORIGINATING_UNREGISTERED

In the past I had my iFCs setup to look for the P-Access-Network-Info header to know if the call was coming from the IMS, but it wasn’t foolproof – Fixed line IMS subs didn’t have this header.

            <TriggerPoint>
                <ConditionTypeCNF>1</ConditionTypeCNF>
                <SPT>
                    <ConditionNegated>0</ConditionNegated>
                    <Group>0</Group>
                    <Method>INVITE</Method>
                    <Extension></Extension>
                </SPT>
                <SPT>
                    <ConditionNegated>0</ConditionNegated>
                    <Group>1</Group>
                    <SIPHeader>
                      <Header>P-Access-Network-Info</Header>
                    </SIPHeader>
                </SPT>                
            </TriggerPoint>

But now I’m using the Session Cases to know if the call is coming from a registered IMS user:

        <!-- SIP INVITE Traffic from Registered Sub-->
        <InitialFilterCriteria>
            <Priority>30</Priority>
            <TriggerPoint>
                <ConditionTypeCNF>1</ConditionTypeCNF>
                <SPT>
                    <ConditionNegated>0</ConditionNegated>
                    <Group>0</Group>
                    <Method>INVITE</Method>
                    <Extension></Extension>
                </SPT>
                <SPT>
                    <Group>0</Group>
                    <SessionCase>0</SessionCase>
                </SPT>             
            </TriggerPoint>

HOMER API in Python

We’re doing more and more network automation, and something that came up as valuable to us would be to have all the IPs in HOMER SIP Capture come up as the hostnames of the VM running the service.

Luckily for us HOMER has an API for this ready to roll, and best of all, it’s Swagger based and easily documented (awesome!).

(Probably through my own failure to properly RTFM) I was struggling to work out the correct (current) way to Authenticate against the API service using a username and password.

Because the HOMER team are awesome however, the web UI for HOMER, is just an API client.

This means to look at how to log into the API, I just needed to fire up Wireshark, log into the Web UI via my browser and then flick through the packets for a real world example of how to do this.

Homer Login JSON body as seen by Wireshark

In the Login action I could see the browser posts a JSON body with the username and password to /api/v3/auth

{"username":"admin","password":"sipcapture","type":"internal"}

And in return the Homer API Server responds with a 201 Created an a auth token back:

Now in order to use the API we just need to include that token in our Authorization: header then we can hit all the API endpoints we want!

For me, the goal we were setting out to achieve was to setup the aliases from our automatically populated list of hosts. So using the info above I setup a simple Python script with Requests to achieve this:

import requests
s = requests.Session()

#Login and get Token
url = 'http://homer:9080/api/v3/auth'
json_data = {"username":"admin","password":"sipcapture"}
x = s.post(url, json = json_data)
print(x.content)
token = x.json()['token']
print("Token is: " + str(token))


#Add new Alias
alias_json = {
          "alias": "Blog Example",
          "captureID": "0",
          "id": 0,
          "ip": "1.2.3.4",
          "mask": 32,
          "port": 5060,
          "status": True
        }

x = s.post('http://homer:9080/api/v3/alias', json = alias_json, headers={'Authorization': 'Bearer ' + token})
print(x.status_code)
print(x.content)


#Print all Aliases
x = s.get('http://homer:9080/api/v3/alias', headers={'Authorization': 'Bearer ' + token})
print(x.json())

And bingo we’re done, a new alias defined.

We wrapped this up in a for loop for each of the hosts / subnets we use and hooked it into our build system and away we go!

With the Homer API the world is your oyster in terms of functionality, all the features of the Web UI are exposed on the API as the Web UI just uses the API (something I wish was more common!).

Using the Swagger based API docs you can see examples of how to achieve everything you need to, and if you ever get stuck, just fire up Wireshark and do it in the Homer WebUI for an example of how the bodies should look.

Thanks to the Homer team at QXIP for making such a great product!

Failures in cobbling together a USSD Gateway

One day recently I was messing with the XCAP server, trying to set the Call Forward timeout. In the process I triggered the UE to send a USSD request to the IMS.

Huh, I thought, “I wonder how hard it would be to build a USSD Gateway for our IMS?”, and this my friends, is the story of how I wasted a good chunk of my weekend trying (and failing) to add support for USSD.

You might be asking “Who still uses USSD?” – The use cases for USSD are pretty thin on the ground in this day and age, but I guess balance query, and uh…

But this is the story of what I tried before giving up and going outside…

Routing

First I’d need to get the USSD traffic towards the USSD Gateway, this means modifying iFCs. Skimming over the spec I can see the Recv-Info: header for USSD traffic should be set to “g.3gpp.ussd” so I knocked up an iFC to match that, and route the traffic to my dev USSD Gateway, and added it to the subscriber profile in PyHSS:

  <!-- SIP USSD Traffic to USSD-GW-->
        <InitialFilterCriteria>
            <Priority>25</Priority>
            <TriggerPoint>
                <ConditionTypeCNF>1</ConditionTypeCNF>
                <SPT>
                    <ConditionNegated>0</ConditionNegated>
                    <Group>1</Group>
                    <SIPHeader>
                      <Header>Recv-Info</Header>
                      <Content>"g.3gpp.ussd"</Content>
                    </SIPHeader>
                </SPT>                
            </TriggerPoint>
            <ApplicationServer>
                <ServerName>sip:ussdgw:5060</ServerName>
                <DefaultHandling>0</DefaultHandling>
            </ApplicationServer>
        </InitialFilterCriteria>

Easy peasy, now we have the USSD requests hitting our USSD Gateway.

The Response

I’ll admit that I didn’t jump straight to the TS doc from the start.

The first place I headed was Google to see if I could find any PCAPs of USSD over IMS/SIP.

And I did – Restcomm seems to have had a USSD product a few years back, and trawling around their stuff provided some reference PCAPs of USSD over SIP.

So the flow seemed pretty simple, SIP INVITE to set up the session, SIP INFO for in-dialog responses and a BYE at the end.

With all the USSD guts transferred as XML bodies, in a way that’s pretty easy to understand.

Being a Kamailio fan, that’s the first place I started, but quickly realised that SIP proxies, aren’t great at acting as the UAS.

So I needed to generate in-dialog SIP INFO messages, so I turned to the UAC module to generate the SIP INFO response.

My Kamailio code is super simple, but let’s have a look:

request_route {

        xlog("Request $rm from $fU");

        if(is_method("INVITE")){
                xlog("USSD from $fU to $rU (Emergency number) CSeq is $cs ");
                sl_reply("200", "OK Trying USSD Phase 1");      #Generate 200 OK
                route("USSD_Response"); #Call USSD_Response route block
                exit;
        }
}

route["USSD_Response"]{
        xlog("USSD_Response Route");
        #Generate a new UAC Request
        $uac_req(method)="INFO";
        $uac_req(ruri)=$fu;     #Copy From URI to Request URI
        $uac_req(furi)=$tu;     #Copy To URI to From URI
        $uac_req(turi)=$fu;     #Copy From URI to To URI
        $uac_req(callid)=$ci;   #Copy Call-ID
                                #Set Content Type to 3GPP USSD
        $uac_req(hdrs)=$uac_req(hdrs) + "Content-Type: application/vnd.3gpp.ussd+xml\r\n";
                                #Set the USSD XML Response body
        $uac_req(body)="<?xml version='1.0' encoding='UTF-8'?>
        <ussd-data>
                <language value=\"en\"/>
                <ussd-string value=\"Bienvenido. Seleccione una opcion: 1 o 2.\"/>
        </ussd-data>";
        $uac_req(evroute)=1;    #Set the event route to use on return replies
        uac_req_send();         #Send it!
}

So the UAC module generates the 200 OK and sends it back.

“That was quick” I told myself, patting myself on the back before trying it out for the first time.

Huston, we have a problem – Although the Call-ID is the same, it’s not an in-dialog response as the tags aren’t present, this means our UE send back a 405 to the SIP INFO.

Right. Perhaps this is the time to read the Spec…

Okay, so the SIP INFO needs to be in dialog. Can we do that with the UAC module? Perhaps not…

But the Transaction Module ™ in Kamailio exposes and option on the ctl API to generate an in-dialog UAC – this could be perfect…

But alas real life came back to rear its ugly head, and this adventure will have to continue another day…

Update: Thanks to a kindly provided PCAP I now know what I was doing wrong, and so we’ll soon have a follow up to this post named “Successes in cobbling together a USSD Gateway” just as soon as I have a weekend free.

Kamailio Bytes: Adding Prometheus + Grafana to Kamailio

I recently fell in love with the Prometheus + Grafana combo, and I’m including it in as much of my workflow as possible, so today we’ll be integrating this with another favorite – Kamailio.

Why would we want to integrate Kamailio into Prometheus + Grafana? Observability, monitoring, alerting, cool dashboards to make it look like you’re doing complicated stuff, this duo have it all!

I’m going to assume some level of familiarity with Prometheus here, and at least a basic level of understanding of Kamailio (if you’ve never worked with Kamailio before, check out my Kamailio 101 Series, then jump back here).

So what will we achieve today?

We’ll start with the simple SIP Registrar in Kamailio from this post, and we’ll add on the xhttp_prom module, and use it to expose some stats on the rate of requests, and responses sent to those requests.

So to get started we’ll need to load some extra modules, xhttp_prom module requires xhttp (If you’d like to learn the basics of xhttp there’s also a Kamailio Bytes – xHTTP Module post covering the basics) so we’ll load both.

xHTTP also has some extra requirements to load, so in the top of our config we’ll explicitly specify what ports we want to bind to, and set two parameters that control how Kamailio handles HTTP requests (otherwise you’ll not get responses for HTTP GET requests).

listen=tcp:0.0.0.0:9090
listen=tcp:0.0.0.0:5060
listen=udp:0.0.0.0:5060

http_reply_parse=yes
tcp_accept_no_cl=yes

Then where you load all your modules we’ll load xhttp and xhttp_prom, and set the basic parameters:

loadmodule "xhttp.so"
loadmodule "xhttp_prom.so"

# Define two counters and a gauge
modparam("xhttp_prom", "xhttp_prom_stats", "all")

By setting xhttp_prom module to expose all stats, this exposes all of Kamailio’s internal stats as counters to Prometheus – This means we don’t need to define all our own counters / histograms / gauges, instead we can use the built in ones from Kamailio. Of course we can define our own custom ones, but we’ll do that in our next post.

Lastly we’ll need to add an event route to handle HTTP requests to the /metrics URL:

event_route[xhttp:request] {
	xlog("Got a request!");
	xlog("$ru");
	$var(xhttp_prom_root) = $(hu{s.substr,0,8});
	if ($var(xhttp_prom_root) == "/metrics") {
			xlog("Called metrics");
			prom_dispatch();
			xlog("prom_dispatch() called");
			return;
	} else
		xhttp_reply("200", "OK", "text/html",
        		"<html><body>Wrong URL $hu</body></html>");
}

Restart, and browse to the IP of your Kamailio instance (mine is 10.01.23) port 9090 /metrics and you’ll get something like this:

Kamailio metrics endpoing used by Prometheus

That my friends, is the sort of data that Prometheus gobbles up, so let’s point Prometheus at it and see what data we get back.

Over on my Prometheus server I’ve edited /etc/prometheus/prometheus.yml to target our new Prometheus endpoint.

  - job_name: "kamailo"
    static_configs:
      - targets: ["10.0.1.23:9090"]  
    honor_timestamps: false

So how can we see this data? Well first off if we log into Prometheus we can see the data flowing in:

If we throw some SIP REGISTER traffic at our Kamailio instance and check on the kamailio_registrar_accepted_regs stat we can see our registrations.

After a few clicks in Grafana we can run some graphs for this data too:

So that’s it, Kamailio’s core stats are now exposed to Prometheus, and we can render this information in Grafana.

There’s a copy of the full code used here available in the Github, and in our next post we’ll look at defining our own metrics in Kamailio and then interacting with them.

SMS-over-IP Message Efficiency – K

Recently I read a post from someone talking about efficiency of USSD over IMS, and how crazy it was that such a small amount of data used so much overhead to get transferred across the network.

Having built an SMSc a while ago, my mind immediately jumped to SMS over IMS as being a great example of having so much overhead.

If we’re to consider sending the response “K” to a text message, how much overhead is there?

SMS PDU containing the message “K”

I’m using a common Qualcomm based smartphone, and here’s the numbers I’ve got from Wireshark when I send the message:

Transport Ethernet Header – 14 Bytes
Transport IP Header – 20 Bytes
Transport UDP Header – 8 Bytes
Transport GTP Header – 12 Bytes
User IP Header – 20 Bytes
IPsec ESP Header (For Um interface protection) – 22 Bytes
Encapsulated UDP Header – 8 Bytes
SIP Headers – 707 Bytes
SMS Header – 16 Bytes
SMS Message Body “K” – 1 Byte

Overall SIP, ESP, GTP and Transport PCAP for SIP MESSAGE

That seems pretty bad in terms of efficiency, but let’s look at how that actually works out:

This means our actual message body makes up just 1 byte of 828 bytes, or 0.12% of the size of the overall payload.

Even combined with the SMS header (which contains all the addressing information needed to route an SMS) it’s still just on 2% of the overall message.

So USSD efficiency isn’t great, but it’s not alone!

Get all the FreeSWITCH Folder Paths

Thanks to it’s reliability, I find I go long periods of time without needing to do anything on FreeSWITCH.

But every now and then I log into a system and I can’t find the path I’m looking for, where do the recordings get stored?

The CDR storage location?

Here’s a simple trick to show the directory paths for a FreeSWITCH instance:

fs_cli -x 'global_getvar'| grep _dir

This will output all the paths you could possibly want:

nick@fs-131:~$ fs_cli -x 'global_getvar'| grep _dir
base_dir=/usr
recordings_dir=/var/lib/freeswitch/recordings
sounds_dir=/usr/share/freeswitch/sounds
conf_dir=/etc/freeswitch
log_dir=/var/log/freeswitch
run_dir=/var/run/freeswitch
db_dir=/var/lib/freeswitch/db
mod_dir=/usr/lib/freeswitch/mod
htdocs_dir=/usr/share/freeswitch/htdocs
script_dir=/usr/share/freeswitch/scripts
temp_dir=/tmp
grammar_dir=/usr/share/freeswitch/grammar
fonts_dir=/usr/share/freeswitch/fonts
images_dir=/var/lib/freeswitch/images
certs_dir=/etc/freeswitch/tls
storage_dir=/var/lib/freeswitch/storage
cache_dir=/var/cache/freeswitch
data_dir=/usr/share/freeswitch
localstate_dir=/var/lib/freeswitch

Saved me a lot of poking around, hopefully it’ll make others lives easier too.

CGrateS in Baby Steps – Part 4 – Rating Calls

In our last few posts we got CGrateS setup in order to have rates and tariffs in the system, so we can price a call.

Where we ended we were able to use the APIerSv1.GetCost method to get the cost of a call, and today, we’re going to actually create some rated CDRs.

So again this will be done through the API, using the CDRsV1.ProcessExternalCDR method.

So let’s give it a whirl:

#Add a CDR
print("Testing call..")
cdr = CGRateS_Obj.SendData({"method": "CDRsV1.ProcessExternalCDR", "params": [ { \
"Direction": "*out",
    "Category": "call",
    "RequestType": "*raw",
    "ToR": "*monetary",
    "Tenant": "cgrates.org",
    "Account": "1002",
    "Subject": "1002",
    "Destination": "61411111",
    "AnswerTime": "2022-02-15 13:07:39",
    "SetupTime": "2022-02-15 13:07:30",
    "Usage": "181s",
    "OriginID": "API Function Example"
    }], "id": 0})
pprint.pprint(cdr)

So the output of this, you may notice returns “Partially Executed” in the output, that’s no good.

{'method': 'CDRsV1.ProcessExternalCDR', 'params': [{'Direction': '*out', 'Category': 'call', 'RequestType': '*raw', 'ToR': '*monetary', 'Tenant': 'cgrates.org', 'Account': '1002', 'Subject': '1002', 'Destination': '61411111', 'AnswerTime': '2022-02-15 13:07:39', 'SetupTime': '2022-02-15 13:07:30', 'Usage': '181s', 'OriginID': 'API Function Example'}], 'id': 0}
OrderedDict([('id', 0), ('result', None), ('error', 'PARTIALLY_EXECUTED')])

So what’s going on here?

Well, there’s another concept I haven’t introduced yet, and that’s ChargerS, this is a concept / component we’ll dig into deeper for derived charging, but for now just know we need to add a ChargerS rule in order to get CDRs rated:

#Define Charger
print(CGRateS_Obj.SendData({
    "method": "APIerSv1.SetChargerProfile",
    "params": [
        {
            "Tenant": "cgrates.org",
            "ID": "DEFAULT",
            'FilterIDs': [],
            'AttributeIDs' : ['*none'],
            'Weight': 0,
        }
    ]   }   ))   
#Set Charger
print("GetChargerProfile: ")
GetChargerProfile = CGRateS_Obj.SendData({"jsonrpc": "2.0", "method": "ApierV1.GetChargerProfile", "params": [{"TPid": "cgrates.org", "ID" : "DEFAULT"}]})
print("GetChargerProfile: ")
pprint.pprint(GetChargerProfile)

Now if we try rating the CDR again we should get a successful output:

{'method': 'CDRsV1.ProcessExternalCDR', 'params': [{'Direction': '*out', 'Category': 'call', 'RequestType': '*raw', 'ToR': '*monetary', 'Tenant': 'cgrates.org', 'Account': '1002', 'Subject': '1002', 'Destination': '6141111124211', 'AnswerTime': '2022-02-15 13:07:39', 'SetupTime': '2022-02-15 13:07:30', 'Usage': '181s', 'OriginID': 'API Function Example'}], 'id': 0}
OrderedDict([('id', 0), ('result', 'OK'), ('error', None)])

Great, so where did the CDR go?

Well, if you’ve got CDR storage in StoreDB enabled (And you probably do if you’ve been following up until this point), then the answer is a MySQL table, and we can retrive the data with:

sudo mysql cgrates -e "select * from cdrs \G"

For those of you with a bit of MySQL experience under your belt, you’d be able to envisage using the SUM function to total a monthly bill for a customer from this.

Of course we can add CDRs via the API, and you probably already guessed this, but we can retrive CDRs via the API as well, filtering on the key criteria:

#Get CDRs
cdrs = CGRateS_Obj.SendData({"method": "ApierV1.GetCDRs", "params": [ { \
"Direction": "*out",
   "Tenants": ["cgrates.org"],
   "Accounts": ["1002"],
    "TimeStart": "2022-02-14 13:07:39",
    "TimeEnd": "2022-02-16 13:07:39",
    "Limit": 100
    }], "id": 0})
pprint.pprint(cdrs)

This would be useful for generating an invoice or populating recent calls for a customer portal.

Maybe creating rated CDRs and sticking them into a database is exactly what you’re looking to achieve in CGrateS – And if so, great, this is where you can stop – but for many use cases, there’s a want for an automated solution – For your platform to automatically integrate with CGrateS.

If you’ve got an Asterisk/FreeSWITCH/Kamailio or OpenSIPs based platform, then you can integrate CGrateS directly into your platform to add the CDRs automatically, as well as access features like prepaid credit control, concurrent call limits, etc, etc.
The process is a little different on each of these platforms, but ultimately under the hood, all of these platforms have some middleware that generates the same API calls we just ran to create the CDR.

So far this tutorial has been heavy on teaching the API, because that’s what CGrateS ultimately is – An API service.

Our platforms like Asterisk and Kamailio with the CGrateS plugins are just CGrateS API clients, and so once we understand how to use and interact with the API it’s a breeze to plug in the module for your platform to generate the API calls to CGrateS required to integrate.

You can find all the code used in today’s lesson in the GitHub repo for this tutorial series.

Kamailio Bytes – Extracting SDP Parameters with Kamailio

So the other day I needed to extract the IP and Port parameters from an SDP body – Not the whole line mind, but the values themselves.

As with so many things in Kamailio, there’s a lot of ways to achieve an outcome, but here’s how I approached this problem.

Using the SDPops module we can get a particular line in the SDP, for example, we can get the media line with:

#Get SDP line starting with m= and put it into AVP $avp(mline)
sdp_get_line_startswith("$avp(mline)", "m=")
#Print value of $avp(mline)
xlog("m-line: $avp(mline)\n");

This gets us the line, but now we need to extract the data, in the example from the screenshot the M line has the value:

m=audio 4002 RTP/AVP 8 101

But we only want the port from the M line.

This is where I’ve used the Kamailio Dialplan module and regex to extract the port from this line.

With a fairly simple regex pattern, we can get a group match for the Port from the m= line.

So I took this regular expression, and put it into the Kamailio Dialplan database with dialplan ID 400 for this example:

INSERT INTO `dialplan` VALUES (4,400,10,1,'m=audio (\\d*)',0,'m=audio (\\d*)','\\1','SDP M Port Stripper');

Now using Dialplan ID 400 we can translate an inputted m= SDP line, and get back the port used, so let’s put that into practice:

        if(sdp_get_line_startswith("$avp(mline)", "m=")) {
            xlog("m-line: $avp(mline)\n");
            xlog("raw: $avp(mline)");
            xlog("Extracting Port from Media Line");
            dp_translate("400", "$avp(mline)/$avp(m_port_b_leg)");
            xlog("Translated m_port_b_leg is: $avp(m_port_b_leg)");
        }

Now we have an AVP called $avp(m_port_b_leg) which contains the RTP Port from the SDP.

Now we’ve got a few other values we might want to get, such as the IP the RTP is to go to, etc, we can extract this in the same way, with Dialplans and store them as AVPs:

        #Print current SDP Values and store as Vars
        if(sdp_get_line_startswith("$avp(mline)", "m=")) {
            xlog("m-line: $avp(mline)\n");
            xlog("raw: $avp(mline)");
            xlog("Extracting Port from Media Line");
            dp_translate("400", "$avp(mline)/$avp(m_port_b_leg)");
            xlog("Translated m_port_b_leg is: $avp(m_port_b_leg)");
        }

        if(sdp_get_line_startswith("$avp(oline)", "o=")) {
            xlog("o-line: $avp(oline)\n");
            dp_translate("401", "$avp(oline)/$avp(o_line_port_1)");
            xlog("O Line Port 1: $avp(o_line_port_1)");
            dp_translate("402", "$avp(oline)/$avp(o_line_port_2)");
            xlog("O Line Port 2: $avp(o_line_port_2)");
            dp_translate("403", "$avp(oline)/$avp(o_ip_b_leg)");
            xlog("O IP: $avp(o_ip_b_leg)");
        }

And all the Regex you’ll need:

INSERT INTO `dialplan` VALUES 
(4,400,10,1,'m=audio (\\d*)',0,'m=audio (\\d*)','\\1','SDP M Port Stripper'),
(5,401,10,1,'o=[^ ]* (\\d*) (\\d*) IN IP4 (\\d*.d*.\\d*.\\d*)',0,'o=[^ ]* (\\d*) (\\d*) IN IP4 (\\d*.d*.\\d*.\\d*)','\\1','O Port 1'),
(6,402,10,1,'o=[^ ]* (\\d*) (\\d*) IN IP4 (\\d*.d*.\\d*.\\d*)',0,'o=[^ ]* (\\d*) (\\d*) IN IP4 (\\d*.d*.\\d*.\\d*)','\\2','O Port 2'),
(7,403,10,1,'o=[^ ]* (\\d*) (\\d*) IN IP4 (\\d*.d*.\\d*.\\d*)',0,'o=[^ ]* (\\d*) (\\d*) IN IP4 (\\d*[.]\\d*[.]\\d*[.]\\d*)','\\3','O IP');


FreeSWITCH – Incompatible Destination

A recent little issue I ran into the other day, that I figured may be of use to someone in the future.

When making a call to FreeSWITCH I would get an “INCOMPATIBLE DESTINATION” response to the SIP INVITE.

Here’s what I saw in the log:

FreeSWITCH showing an “INCOMPATIBLE DESTINATION” error
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5650 Audio Codec Compare [GSM:3:8000:20:13200:1]/[opus:116:48000:20:0:1]
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5650 Audio Codec Compare [GSM:3:8000:20:13200:1]/[G722:9:8000:20:64000:1]
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5650 Audio Codec Compare [GSM:3:8000:20:13200:1]/[PCMU:0:8000:20:64000:1]
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5650 Audio Codec Compare [GSM:3:8000:20:13200:1]/[PCMA:8:8000:20:64000:1]
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5944 No 2833 in SDP. Liberal DTMF mode adding 101 as telephone-event.
2022-02-19 13:04:04.027963 99.47% [DEBUG] switch_core_media.c:5973 sofia/internal/[email protected]:5060 Set 2833 dtmf send payload to 101 recv payload to 101
2022-02-19 13:04:04.027963 99.47% [NOTICE] switch_channel.c:3993 Hangup sofia/internal/49042@127.0.0.3:5060 [CS_EXECUTE] [INCOMPATIBLE_DESTINATION]

The hint to the cause of the error is above it – Codec comparison. If we look at the Audio Codec Compare lines, we can see the GSM codec we are trying to use, does not match the codecs configured in FreeSWITCH, hence getting the INCOMPATIBLE_DESTINATION error – None of the codecs offered match the codecs supported in FreeSWITCH.

So where do we go to fix this?

Well the SIP profile itself defines the codecs that are supported on this SIP profile,

FreeSWITCH SIP Profile (Sofia) codec settings

If you’re using a mostly default config, you’ll see this is set to a global variable, called $${global_codec_prefs}, so let’s take a look at vars.xml where this is defined:

FreeSWITCH default codec selection global variable

And there’s our problem, we need to add the GSM codec into that list to allow the calls,

So we change it to add the codecs we want to support, and reload the changes,

The Codec preferences I need for this IMS Application Server

Now when we want to make a call, success!

Successful call

FreeSWITCH, Kamailio & IMS Extensions

Recently I’ve been doing some work with FreeSWITCH as an IMS Conference Factory, I’ve written a bit about it before in this post on using FreeSWITCH with the AMR codec.

Pretty early on in my testing I faced a problem with subsequent in-dialog responses, like re-INVITEs used for holding the calls.

Every subsequent message, was getting a “420 Bad Extension” response from FreeSWITCH.

So what didn’t it like and why was FreeSWITCH generating 420 Bad Extension Responses to these subsequent messages?

Well, the “Extensions” FreeSWITCH is referring to are not extensions in the Telephony sense – as in related to the Dialplan, like an Extension Number to identify a user, but rather the Extensions (as in expansions) to the SIP Protocol introduced for IMS.

The re-INVITE contains a Require header with sec-agree which is a SIP Extension introduced for IMS, which FreeSWITCH does not have support for, and the re-INVITE says is required to support the call (Not true in this case).

Using a Kamailio based S-CSCF means it is easy to strip these Headers before forwarding the requests onto the Application Server, which is what I’ve done, and bingo, no more errors!

The Surprisingly Complicated World of SMS: Apple iPhone MT SMS

In iOS 15, Apple added support for iPhones to support SMS over IMS networks – SMSoIP. Previously iPhone users have been relying on CSFB / SMSoNAS (Using the SGs interface) to send SMS on 4G networks.

Getting this working recently led me to some issues that took me longer than I’d like to admit to work out the root cause of…

I was finding that when sending a Mobile Termianted SMS to an iPhone as a SIP MESSAGE, the iPhone would send back the 200 OK to confirm delivery, but it never showed up on the screen to the user.

The GSM A-I/F headers in an SMS PDU are used primarily for indicating the sender of an SMS (Some carriers are configured to get this from the SIP From header, but the SMS PDU is most common).

The RP-Destination Address is used to indicate the destination for the SMS, and on all the models of handset I’ve been testing with, this is set to the MSISDN of the Subscriber.

But some devices are really finicky about it’s contents. Case in point, Apple iPhones.

If you send a Mobile Terminated SMS to an iPhone, like the one below, the iPhone will accept and send back a 200 OK to this request.

The problem is it will never be displayed to the user… The message is marked as delivered, the phone has accepted it it just hasn’t shown it…

SMS reports as delivered by the iPhone (200 OK back) but never gets displayed to the user of the phone as the RP-Destination Address header is populated

The fix is simple enough, if you set the RP-Destination Address header to 0, the message will be displayed to the user, but still took me a shamefully long time to work out the problem.

RP-Destination Address set to 0 sent to the iPhone, this time it’ll get displayed to the user.

Handling multiple SIP headers with the same name in Kamailio

The SIP RFC allows for multiple SIP headers to have the same name,

For example, it’s very common to have lots of Via headers present in a request.

In Kamailio, we often may wish to add headers, view the contents of headers and perform an action or re-write headers (Disclaimer about not rewriting Vias as that goes beyond the purview of a SIP Proxy but whatever).

Let’s look at a use case where we have multiple instances of the X-NickTest: header, looking something like this:

INVITE sip:[email protected]:5061 SIP/2.0
X-NickTest: ENTRY ONE
X-NickTest: ENTRY TWO
X-NickTest: ENTRY THREE
...

Let’s look at how we’d access this inside Kamailio.

First, we could just use the psedovariable for header – $hdr()

xlog("Value of X-NickTest is: $hdr(X-NickTest)");

But this would just result in the first entry being printed out:

Value of X-NickTest is: ENTRY ONE

If we know how many instances there are of the header, we can access it by it’s id in the array, for example:

xlog("Value of first X-NickTest is: $hdr(X-NickTest)[0]");
xlog("Value of second X-NickTest is: $hdr(X-NickTest)[1]");
xlog("Value of third  X-NickTest is: $hdr(X-NickTest)[2]");

But we may not know how many to expect either, but we can find out using $hdrc(name) to get the number of headers returned.

xlog("X-NickTest has $hdrc(X-NickTest) entries");

You’re probably seeing where I’m going with this, the next logical step is to loop through them, which we can also do something like this:

$var(i) = 0;
while($var(i) < $hdrc(X-NickTest)) {
         xlog(X-NickTest entry [$var(i)] has value $hdrc(X-NickTest)[$var(i)]);
         $var(i) = $var(i) + 1;
}

Originating calls in FreeSWITCH

Through fs_cli you can orignate calls from FreeSWITCH.

At the CLI you can use the originate command to start a call, this can be used for everything from scheduled wake up calls, outbound call centers, to war dialing.

For example, what I’m using:

originate sofia/external/[email protected]:5061 61399999995 XML default
  • originate is the command on the FS_CLI
  • sofia/external/[email protected]:5061 is the call URL, with the application (I’m using mod_sofia, so sofia), the Sofia Profile (in my case external) and the SIP URI, or, if you have gateways configured, the to URI and the gateway to use.
  • 6139999995 is the Application
  • XML is the Dialplan to reference
  • default is the Context to use

But running this on the CLI is only so useful, we can use an ESL socket to use software to connect to FreeSWITCH’s API (Through the same mechanism fs_cli uses) in order to programmatically start calls.

But to do that first we need to expose the ESL API for inbound connections (Clients connecting to FreeSWITCH’s ESL API, which is different to FreeSWITCH connecting to an external ESL Server where FreeSWITCH is the client).

We’ll need to edit the event_socket.conf.xml file to define how this can be accessed:

<configuration name="event_socket.conf" description="Socket Client">
  <settings>
    <param name="nat-map" value="false"/>
    <param name="listen-ip" value="0.0.0.0"/>
    <param name="listen-port" value="8021"/>
    <param name="password" value="yoursecretpassword"/>
    <param name="apply-inbound-acl" value="lan"/>
    <param name="stop-on-bind-error" value="true"/>
  </settings>
</configuration>

Obviously you’ll need to secure this appropriately, good long password, and tight ACLs.

You may notice after applying these changes in the config, you’re no longer able to run fs_cli and access FreeSWITCH, this is because FreeSWITCH’s fs_cli tool connects to FreeSWITCH over ESL, and we’ve just changed tha parameters. You should still be able to connect by specifying the IP Address, port and the secret password we set:

fs_cli --host=10.0.1.16 --password=yoursecretpassword --port=8021

This also means we can run fs_cli from other hosts if permitted through the ACLs (kinda handy for managing larger clusters of FreeSWITCH instances).

But now we can also connect a remote ESL client to it to run commands like our Originate command to setup calls, I’m using GreenSwitch with ESL in Python:

import gevent
import greenswitch
import sys
#import Fonedex_TelephonyAPI
#sys.path.append('../WebUI/Flask/')
import uuid

import logging
logging.basicConfig(level=logging.DEBUG)


esl_server_host = "10.0.1.16"
logging.debug("Originating call to " + str(destination) + " from " + str(source))
logging.debug("Routing the call to " + str(dialplan_entry))
fs = greenswitch.InboundESL(host=str(esl_server_host), port=8021, password='yoursecretpassword')
  try:
      fs.connect()
      logging.debug("Connected to ESL server at " + str(esl_server_host))
  except:
      raise SystemError("Failed to connect to ESL Server at " + str(esl_server_host))

r = fs.send('bgapi originate {origination_caller_id_number=' + str(source) + '}sofia/external/' + str(destination) + '@10.0.1.252:5061 default XML')

And presto, a call is originated!

VoIP is an only child – ‘Gotchas’ on running VoIP applications inside Containers

It’s 2021, and everyone loves Containers; Docker & Kubernetes are changing how software is developed, deployed and scaled.

And yet so much of the Telco world still uses bare metal servers and dedicated hardware for processing.

So why not use Containers or VMs more for VoIP applications?

Disclaimer – When I’m talking VoIP about VoIP I mean the actual Voice over IP, that’s the Media Stream, RTP, the Audio, etc, not the Signaling (SIP). SIP is fine with Containers, it’s the media that has a bad time and that this post focuses on,

Virtualization Fundamentals

Once upon a time in Development land every application ran on it’s own server running in a DC / Central Office.

This was expensive to deploy (buying servers), operate (lots of power used) and maintain (lots of hardware to keep online).

Each server was actually sitting idle for a large part of the time, with the application running on it only using a some of the available resources some of the time.

One day Virtualization came and suddenly 10 physical servers could be virtualized into 10 VMs.

These VMs still need to run on servers but as each VM isn’t using 100% of it’s allocated resources all the time, instead of needing 10 servers to run it on you could run it on say 3 servers, and even do clever things like migrate VMs between servers if one were to fail.

VMs share the resources of the server it’s running on.

A server running VMs (Hypervisor) is able to run multiple VMs by splitting the resources between VMs.

If a VM A wants to run an operation at the same time a VM B & VM C, the operations can’t be run on each VM at the same time* so the hypervisor will queue up the requests and schedule them in, typically based on first-in-first out or based on a resource priority policy on the Hypervisor.

This is fine for a if VM A, B & C were all Web Servers.
A request coming into each of them at the same time would see the VM the Hypervisor schedules the resources to respond to the request slightly faster, with the other VMs responding to the request when the hypervisor has scheduled the resources to the respective VM.

VoIP is an only child

VoIP has grown up on dedicated hardware. It’s an only child that does not know how to share, because it’s never had to.

Having to wait for resources to be scheduled by the Hypervisor to to VM in order for it to execute an operation is fine and almost unnoticeable for web servers, it can have some pretty big impacts on call quality.

If we’re running RTPproxy or RTPengine in order to relay media, scheduling delays can mean that the media stream ends up “bursty”.

RTP packets needing relaying are queued in the buffer on the VM and only relayed when the hypervisor is able to schedule resources, this means there can be a lot of packet-delay-variation (PDV) and increased latency for services running on VMs.

VMs and Containers both have this same fate, DPDK and SR-IOV assist in throughput, but they don’t stop interrupt headaches.

VMs that deprive other VMs on the same host of resources are known as “Noisy neighbors”.

The simple fix for all these problems? There isn’t one.

Each of these issues can be overcome, dedicating resources, to a specific VM or container, cleverly distributing load, but it is costly in terms of resources and time to tweak and implement, and some of these options undermine the value of virtualization or containerization.

As technology marches forward we have scenarios where Kubernetes can expose FPGA resources to pass them through to Pods, but right now, if you need to transcode more than ~100 calls efficiently, you’re going to need a hardware device.

And while it can be done by throwing more x86 / ARM compute resources at the problem, hardware still wins out as cheaper in most instances.

Sorry, no easy answers here…