Category Archives: VoIP

Background to the “VoLTE Mess”

5G SA, IMS / VoLTE, LTE, Mobile Networks, RFCs & Standards, VoIP, , , ,

I’ve been writing a fair bit recently about the “VoLTE Mess” – It’s something that’s been around for a long time, mostly impacting greenfield players rolling out LTE only, but now the big carriers are starting to feel it as they shut off their 2G and 3G networks, so I figured a brief history was in order to understand how we got here.

Note: I use the terms 4G or LTE interchangeably

The Introduction of LTE

LTE (4G) is more “spectrally efficient” than the technologies that came before it. In simple terms, 1 “chunk” of spectrum will get you more speed (capacity) on LTE than the same size chunk of spectrum would on 2G or 3G.

From my post on 5G being a bit overhyped

So imagine it’s 2008 and you’re the CTO of a mobile network operator.
Your network is congested thanks to carrying more data traffic than it was ever designed for (the first iPhone had launched the year before) and the network is struggling under the weight of all this new data traffic.
You have two options here, to build more cell sites for more density (very expensive) or buy more spectrum (extremely expensive) – Both options see you going cap in hand to the finance team and asking for eye-wateringly large amounts of capital for either option.

But then the answer to your prayers arrives in the form of 3GPP’s Release 8 specification with the introduction of LTE. Now by taking some 2G or 3G spectrum, and by using it on 4G, you can get ~5x more capacity from the same spectrum. So just by changing spectrum you own from 2G or 3G to 4G, you’ve got 5x more capacity. Hallelujah!

So you go to Nortel and buy a packet core, and Alcatel and Siemens provide 4G RAN (eNodeBs) which you selectively deploy on the cell sites that are the most congested.
The finance team and the board are happy and your marketing team runs amok with claims of 4G data speeds.
You’ve dodged the crisis, phew.

This is the path that all established mobile operators took; throw LTE at the congested cell sites, to cheaply and easily free up capacity, and as the natural hardware replacement cycle kicked in, or cell sites reached capacity, swap out the hardware to kit that supports LTE in addition to the 2G and 3G tech.

Circuit Switched Fallback

But it’s hard to talk about the machinations of late 2000s telecom executives, without at least mentioning Hitler.

This video below from 15 years ago is pretty obscure and fairly technical, but the crux of it it is that Hitler is livid because LTE does not have a “CS Domain” aka circuit switched voice (the way 2G and 3G had handled voice calls).

It was optional to include support for voice calls in the LTE network (Voice over LTE) when you launched LTE services. So if you already had a 2G or 3G network (CS Network) you could just keep using 2G and 3G for your voice calls, while getting that sweet capacity relief.

So our hypothetical CTO, strapped for cash and data capacity, just didn’t bother to support VoLTE when they launched LTE – Doing so would have taken more time to launch, during which time the capacity problem would become worse, so “don’t worry about VoLTE for now” was the mantra.

All the operators who still had 2G and 3G networks, opted to just “Fallback” to using the 2G / 3G network for calling. This is called “Circuit Switched Fallback” aka CSFB.

Operators loved this as they got the capacity relief provided by shifting to 4G/LTE (more capacity in the network is always good) and could all rant about how their network was the fastest and had 4G first, this however was what could be described as a “Foot gun” – Something you can shoot yourself in the foot with in the future.

Operators eventually introduce VoLTE

Time ticked on an operators built out their 4G networks, and many in the past 10 years or so have launched VoLTE in their own networks.

For phones that support it, in areas with blanket 4G coverage, they can use VoLTE for all their calls.

But that’s the sticking point right there – If the phones support it.

But if the phones don’t support it, they’re roaming or making emergency calls, there is always been the safety blanket of 2G or 3G and Circuit Switched fallback to well, fall back to.

There’s no driver for operators who plan to (or are required to) operate a 2G or 3G network for the foreseeable future, to ensure a high level of VoLTE support in their devices.

For an operator today with 2G or 3G, Voice over LTE is still optional.
Many operators still rely exclusively on Circuit Switched Fallback, and there are only a handful of countries that have turned off 2G and 3G and rely solely on VoLTE.

VoLTE Handset Support

For the past 16 years phone manufacturers have been making LTE capable phones.

But that does not mean they’ve been making phones that support Voice over LTE.

But it’s never been an issue up until this point, as there’s always been a circuit switched (2G/3G) network to fall back to, so the fact that these chips may not support VoLTE was not a big problem.

Many of the cheaper chipsets that power phones simply don’t support VoLTE – These chips do support LTE for data connections but rely on Circuit Switched Fallback for voice calls. This is in part due to the increased complexity, but also because some of the technologies for VoLTE (like AMR) required intellectual property deals to licence to use, so would add to the component cost to manufacture, and in the chips game, keeping down component cost is critical.

Even for chips that do support Voice over LTE, it’s “special”. Unlike calling in 2G or 3G that worked the same for every operator, phone manufacturers require a “Carrier Bundle” for each operator, containing that specific operators’ special flavor of VoLTE, that operator uses in their network.

This is because while VoLTE is standardized (Despite some claims to the contrary) a lot of “optional” bits have existed, and different operators built networks with subtle differences in the “flavor” of their Voice over LTE (IMS) stack they used. The OEMs (Phone / Chip manufacturers) had to handle these changes in the devices they made, for in order to sell their phones through that operator.

This means I can have a phone from vendor X that works with VoLTE on Network Y, but does not support VoLTE on Network Z.

Worse still, knowing which phones are supported is a bit of a guessing game.

Most operators sell phones directly to their customer base, so buying an Network Y branded phone from Vendor X, you know it’s going to support Network Y’s VoLTE settings, but if you change carriers, who knows if it’ll still support it?

When you’ve still got a Circuit Switched network it’s not the end of the world, you’ll just use CSFB and probably not realize it, until operators go to shut down 2G / 3G networks…

IMS Profile selection on an engineering mode MTK based Android handset

Navigating the Maze of VoLTE Compatibility

Here are some simple checklist you can ask your elderly family members if they ask if their phone is VoLTE compatible:

  • Does the underlying chipset the phone is based on support VoLTE? (you can find this out by disassembling the phone and checking the datasheets for the components from the OEMs after signing NDAs for each)
  • Does the underlying chipset require a “carrier bundle” of settings to have been loaded for this operator in order to support VoLTE (See Qualcomm MBM as an example)?
  • What version of this list am I currently on (generally set in the factory) and does it support this operator? (You can check by decapping the ICs and dumping their NVRAM and then running it through a decompiler)
  • Does my phones OS (Android / iOS) require a “carrier bundle” of it’s own to enable VoLTE? Is my operator in the version of the database on the phone? (See Android’s Carrier Database for example) (You can find the answer by rooting the phone and running some privileged commands to poke around the internal file system)
  • Does my operator / MNO support VoLTE – Does my plan / package support VoLTE? (You can easily find the answer by visiting the store and asking questions that don’t appear on the script)

If you managed to answer yes to all of the above, congratulations! You have conditional VoLTE support on your phone, although you probably don’t have a working phone anymore.

Wait, conditional VoLTE support?

That’s right folks, VoLTE will work in some scenarios with your operator!

If you plan on traveling, well your phone may support VoLTE at home, but does the phone have VoLTE roaming enabled?
Many phones support VoLTE in the home network, but resort to CSFB when roaming.

If it does support VoLTE roaming, does the network you’re visiting support VoLTE roaming? Has the roaming agreement (IRA) between the operator you’re using while traveling and your home operator been updated to include VoLTE Roaming? These IRAs (AA.12 / AA.13 docs) also indicate if the network must turn off IPsec encryption for the VoLTE traffic when roaming, which is controlled by the phone anyway.

Phew, all this talk of VoLTE roaming while traveling scares me, I think I’ll stay home in the safety of the Australian bush with all these great friendly animals around a phone that supports VoLTE on my home network.

Ah – After spending some time in the Australian bush one of our many deadly animals bit me. Time to call for help! Wait, what about emergency calls over VoLTE? Again, many phones support VoLTE for normal calls, fall back to 2G or 3G for the emergency call, so if you have one of those phones (You’ll only find out if you try to make an emergency call and it fails) and try to make an emergency call in a country without 2G or 3G, you’d better find a payphone.

There’s many real world examples of this, our friends at OptimERA have been lobbying the FCC since 2019 on this.

Sarcasm aside, there’s no dataset or compatibility matrix here – No simple way to see if your phone will work for VoLTE on a given operator, even if the underlying chip does support VoLTE.

Operators in Australia which recently shut down their 3G network, were mandated to block devices that didn’t support VoLTE for emergency calling. They did this using an Equipment Identity Register, and blocking devices based on the Type Allocation Code, but this scattergun approach just blocked non-carrier issued devices, regardless of it they supported VoLTE or VoLTE emergency calling.

Blame Game

So who’s to blame here?

There’s no one group to blame here, the industry has created a shitty cycle here:

  • Standards orgs for having too many “flavors” available
  • Operators deploying their own “Flavors” of VoLTE then mandating OEMs / Chip manufacturers comply with their “flavor”.
  • OEMs / Chip manufactures respond by adding “Carrier Bundles” to account for this per-operator customization

I’ve got some ideas on a way to unscramble this egg, and it’s going to take a push from the industry.

If you’re in the industry and keen to push for a fix, get in touch!

It’s time to get a long term solution to this problem, and we as an industry need to lead the change.

Tales from the Trenches – IMS TCP Socket Handling

IMS / VoLTE, Mobile Networks, RFCs & Standards, VoIP, , , ,

Oh boy this has been a pain in the backside with IMS / VoLTE devices using TCP and how they handle the underlying TCP sockets.

A mobile phone from manufacturer A, wants every SIP dialog to be in it’s own TCP session, while a phone from manufacturer B wants a unique TCP session per transaction, while manufacturer C thinks that every SIP message should reuse the same transaction.

So an MT call to manufacturer A, who wants every SIP dialog in it’s own transaction would look something like this:

PCSCF:44738 -> UE:5060; TCP SYN
UE:5060 -> PCSCF:44738; TCP SYN/ACK
PCSCF:44738 -> UE:5060; TCP ACK
--- TCP connection is now open to UE from P-CSCF---
--- Start of new SIP Transaction 1 & Dialog ---
PCSCF:44738 -> UE:5060; TCP PSH - SIP INVITE....
UE:5060 -> PCSCF:44738; TCP ACK


UE:5060 -> PCSCF:44738; TCP PSH - SIP 200....
PCSCF:44738 -> UE:5060; TCP ACK, PSH - SIP ACK....
UE:5060 -> PCSCF:44738; TCP ACK
--- End of SIP Transaction 1 ---

--- Start of SIP Transaction 2 ---
PCSCF:44738 -> UE:5060; TCP PSH - SIP BYE....
UE:5060 -> PCSCF:44738; TCP ACK, PSH - SIP 200....
--- End of SIP Transaction 2 & SIP Dialog ---
PCSCF:44738 -> UE:5060; TCP FIN
UE:5060 -> PCSCF:44738; TCP ACK
--- End of TCP Connection ---

Where UE:5060 – is the IP & port of the UE, as advertised in the Contact: header, while PCSCF:44738 is the PCSCF IP and a random TCP port used for this connection.

But for manufacturer B, who wants a unique TCP session per transaction, they want it to look like this:

PCSCF:44738 -> UE:5060; TCP SYN
UE:5060 -> PCSCF:44738; TCP SYN/ACK
PCSCF:44738 -> UE:5060; TCP ACK
--- TCP connection is now open to UE from P-CSCF---
--- Start of new SIP Transaction 1 & Dialog ---
PCSCF:44738 -> UE:5060; TCP PSH - SIP INVITE....
UE:5060 -> PCSCF:44738; TCP ACK


UE:5060 -> PCSCF:44738; TCP PSH - SIP 200....
PCSCF:44738 -> UE:5060; TCP ACK, PSH - SIP ACK....
UE:5060 -> PCSCF:44738; TCP ACK
PCSCF:44738 -> UE:5060; TCP FIN
UE:5060 -> PCSCF:44738; TCP ACK
--- End of SIP Transaction 1 & TCP Session 1 ---

--- Start of TCP Session 2 ----
PCSCF:32627 -> UE:5060; TCP SYN
UE:5060 -> PCSCF:32627; TCP SYN/ACK
PCSCF:32627 -> UE:5060; TCP ACK
--- Start of SIP Transaction 2 ---
PCSCF:32627 -> UE:5060; TCP PSH - SIP BYE....
UE:5060 -> PCSCF:32627; TCP ACK, PSH - SIP 200....
--- End of SIP Transaction 2 & SIP Dialog ---
PCSCF:32627 -> UE:5060; TCP FIN
UE:5060 -> PCSCF:32627; TCP ACK
--- End of TCP Connection 2 ---

And then manufacturer C wants just the one TCP session to be used for everything, so they open the TCP connection when they register, and that’s all we use for everything.

Is there any logic to this? Nope, seems to be tied to the underlying chipset (Qualcomm vs Mediatek vs Unisoc) and the SIP stack used (Qualcomm, MTK, Unisoc, Samsung, Apple).

We’ve profiled devices into one of 3 behaviors, and then we tag them based on user agent as to what “persona” they demand from the network.

I can’t believe I’m still talking about VoLTE / IMS handset support and it’s almost 2025…. For context IMS was “standardized” 17 years ago.

Module debug in Kamailio

Kamailio, VoIP, , ,

Sometimes Kamailio modules don’t behave how you expect them to, and you want to dive a little deeper into what’s going on.

If we load the debugger.so module in our kamailio.cfg file, we can get a bit more of an insight.

For example, if we wanted to deep dive into what RTPengine was thinking we’d include:

loadmodule "debugger"
modparam("debugger", "mod_level_mode", 1)
modparam("debugger", "mod_level", "rtpengine=3")

And bam, now when we run Kamailio we’ll get all the logic that rtpengine exposes when being called.

By setting cfgtrace we can enable a config trace as well, and as we step through our Kamailio config file for a SIP message, we can see what’s going on at every step.

So if you’re like me, you can use the debugger module, combined with reading through the source code, to prove every single time that the issue is with my not understanding the function properly, and never the logic of the Kamailio module!

Kamailio – NDB_Redis – undefined symbol: redisvCommand

Kamailio, Notes, VoIP, ,

I ran into this issue the other day while compiling Kamailio from source:

Jul 03 23:49:35 kamailio[305199]: ERROR: <core> [core/sr_module.c:608]: ksr_load_module(): could not open module </usr/local/lib64/kamailio/modules/ndb_redis.so>: /usr/local/lib64/kamailio/modules/ndb_redis.so: undefined symbol: redisvCommand
Jul 03 23:49:35 kamailio[305199]: CRITICAL: <core> [core/cfg.y:4024]: yyerror_at(): parse error in config file /etc/kamailio/kamailio.cfg, line 487, column 12-22: failed to load module

So what was going on?

Kamailio’s NDB Redis module relies on libhiredis which was installed, but the path it installs to wasn’t in the LD_LIBRARY_PATH environment variable.

So I added /usr/lib/x86_64-linux-gnu to LD_LIBRARY_PATH, recompiled and presto, the error is gone!

export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH

Creating a Fixed Line IMS subscriber in PyHSS

IMS / VoLTE, Mobile Networks, Software, VoIP, , , , ,

I generally do this with Python or via the Swagger UI for the Web UI, but here’s how we can create a fixed-line IMS subscriber in PyHSS, so we can register it with a softphone, without using EAP-AKA.


Firstly we create the AuC object for this password combo.

curl -X 'PUT' \
  'http://10.97.0.36:8080/auc/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "ki": "yoursippassword123",
  "opc": "",
  "amf": "8000",
  "sqn": "1"
}'

Get back the AuC ID from the JSON body, we’ll use this to provision the Sub:

 "auc_id": 110,

Next we create the subscriber, the speeds will be 0 as there is no data on the service, but we will add an default APN so the validation passes:

curl -X 'PUT' \
  'http://10.97.0.36:8080/subscriber/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "default_apn": 1,
    "roaming_rule_list": null,
    "apn_list": "0",
    "subscribed_rau_tau_timer": 600,
    "msisdn": "123451000001",
    "ue_ambr_dl": 0,
    "ue_ambr_ul": 0,
    "imsi": "001001000090001",
    "nam": 2,
    "enabled": true,
    "roaming_enabled": null,
    "auc_id": 110
  }'

Alright, that’s the basics done, now we’ll create the IMS subscriber.

Provision it:

curl -X 'PUT' \
  'http://10.97.0.36:8080/ims_subscriber/' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "pcscf_realm": null,
  "scscf_realm": null,
  "pcscf_active_session": null,
  "scscf_peer": null,
  "msisdn": "123451000001",
  "pcscf_timestamp": null,
  "sh_template_path": "default_sh_user_data.xml",
  "msisdn_list": "123451000001",
  "pcscf_peer": null,
  "last_modified": "2024-04-25T00:33:37Z",
  "imsi": "001001000090001",
  "xcap_profile": null,
  "ifc_path": "default_ifc.xml",
  "sh_profile": "\n<!-- This container for the XCAP Data for the Subscriber -->\n<RepositoryData>\n    <ServiceIndication>ApplicationServer</ServiceIndication>\n    <SequenceNumber>0</SequenceNumber>\n    <ServiceData>\n        <!-- This is the actual XCAP Data for the Subscriber -->\n        \n        <!-- XCAP Default Template (no XCAP Data stored in Database) -->\n        <simservs xmlns=\"http://uri.etsi.org/ngn/params/xml/simservs/xcap\" xmlns:cp=\"urn:ietf:params:xml:ns:common-policy\">\n            <originating-identity-presentation active=\"true\" />\n            <originating-identity-presentation-restriction active=\"true\">\n                <default-behaviour>presentation-not-restricted</default-behaviour>\n            </originating-identity-presentation-restriction>\n            <communication-diversion active=\"true\">\n                <!-- No Answer Time -->\n                <NoReplyTimer>20</NoReplyTimer>\n                <cp:ruleset>\n                    <!-- Call Forward All Rule -->\n                    <cp:rule id=\"rule0\">\n                        <cp:conditions>\n                            <communication-diverted />\n                        </cp:conditions>\n                        <cp:actions>\n                            <forward-to>\n                                <target>sip:[email protected]</target>\n                            </forward-to>\n                        </cp:actions>\n                    </cp:rule>\n                    <!-- Call Forward Not Registered Rule -->\n                    <cp:rule id=\"rule1\">\n                        <cp:conditions>\n                            <not-registered />\n                        </cp:conditions>\n                        <cp:actions>\n                            <forward-to>\n                                <target>sip:[email protected]</target>\n                            </forward-to>\n                        </cp:actions>\n                    </cp:rule>\n                    <!-- Call Forward No Answer Rule -->\n                    <cp:rule id=\"rule2\">\n                        <cp:conditions>\n                            <no-answer />\n                        </cp:conditions>\n                        <cp:actions>\n                            <forward-to>\n                                <target>sip:[email protected]</target>\n                            </forward-to>\n                        </cp:actions>\n                    </cp:rule>\n                    <!-- Call Forward Busy Rule -->\n                    <cp:rule id=\"rule3\">\n                        <cp:conditions>\n                            <busy />\n                        </cp:conditions>\n                        <cp:actions>\n                            <forward-to>\n                                <target>sip:[email protected]</target>\n                            </forward-to>\n                        </cp:actions>\n                    </cp:rule>\n                    <!-- Call Forward Unreachable Rule -->\n                    <cp:rule id=\"rule4\">\n                        <cp:conditions>\n                            <not-reachable />\n                        </cp:conditions>\n                        <cp:actions>\n                            <forward-to>\n                                <target>sip:[email protected]</target>\n                            </forward-to>\n                        </cp:actions>\n                    </cp:rule>\n                </cp:ruleset>\n            </communication-diversion>\n        \n            <incoming-communication-barring active=\"true\">\n                <cp:ruleset>\n                    <cp:rule id=\"rule0\">\n                        <cp:conditions />\n                        <cp:actions>\n                            <allow>true</allow>\n                        </cp:actions>\n                    </cp:rule>\n                </cp:ruleset>\n            </incoming-communication-barring>\n\n            <outgoing-communication-barring active=\"false\">\n            </outgoing-communication-barring>\n        </simservs>\n\n    </ServiceData>\n    \n</RepositoryData>\n",
  "pcscf": null,
  "scscf": null,
  "scscf_timestamp": null
}'

And with that we’re done,

We can now register 001001000090001 at our IMS, with password yoursippassword123 which has the MSISDN / phone number 123451000001.

Easy!

CGrateS – SERVER_ERROR: unexpected end of JSON input

CGrateS, Notes, Software, VoIP

I started seeing this error the other day when running CDRsv1.GetCDRs on the CGrateS API:

SERVER_ERROR: unexpected end of JSON input

It seemed related to certain CDRs in the cdrs table of StoreDB.

After some digging, I found the stupid simple problem:

I’d written too much data to extra_fields, leading MySQL to cut off the data mid way through, meaning it couldn’t be reconstructed as JSON by CGrateS again.

Like the rounding issue I had, this wasn’t an issue with CGrateS but with MySQL.

Quick fix:

sudo mysql cgrates -e "ALTER TABLE cdrs MODIFY extra_fields LONGTEXT;"

And new fields can exceed this length without being cut off.

Grafana and CGrateS

CGrateS, Software, VoIP,

I’m a really big fan of CGrateS, and I’m a fan of Grafana,

So what if you combined the two?

CGrateS uses a StoreDB – In my case MySQL, but could be Postgres or MongoDB, etc, and Grafana can get data from these sources too.

So let’s join them together!

For starters, I’ve got a bunch of CDRs in my cgrates.cdrs table inside MySQL.

Setting it up is a doddle, firstly inside Grafana we link it into MySQL:

Next up we create a dashboard and add a panel.

For this instance I’m metering data usage, so I’ve set the units to Bytes/SI (but if you’re using Voice you’ll need to adjust this to time).

Here’s my Query to find the Usage:

SELECT
  DATE(setup_time) AS time,
  SUM(`usage`) AS total_usage
FROM
  cgrates.cdrs
GROUP BY
  DATE(setup_time)
ORDER BY
  time ASC

And I’ve created another one for Cost:

Keep in mind for the units it’s up to you what the units are, dollars, cents, 1/10th of a cent, etc, etc – In my case 1 in CGrateS equates to 1 cent:

SELECT

  DATE(setup_time) AS time,
  SUM(`cost`)/100 AS cost
FROM
  cgrates.cdrs
GROUP BY
  DATE(setup_time)
ORDER BY
  time ASC

Lastly I’ve added a board to show the usage per Account, which I get with the below query:

SELECT
  account AS label,
  SUM(`usage`) AS value
FROM
  cgrates.cdrs
GROUP BY
  account
ORDER BY
  value DESC

Not complete by any means, but shows what you can do with CGrateS and Grafana.

Importing CDRs into CGrateS with Event Reader Service

CGrateS, Software, VoIP, , , , ,

After we setup CgrateS the next thing we’d generally want to do would be to rate some traffic.

Of course, that could be realtime traffic, from Diameter, Radius, Kamailio, FreeSWITCH, Asterisk or whatever your case may be, but it could just as easily be CSV files, records from a database or a text file.

We’re going to be rating CDRs from simple CSV files with the date of the event, calling party, called party, and talk time, but of course your CDR exports will have a different format, and that’s to be expected – we tailor the Event Reader Service to match the format of the files we need.

The Event Reader Service, like everything inside CgrateS, is modular.
ERS is a module we load that parses files using the rules we define, and creates Events that CgrateS can process and charge for, based on the rules we define.

But before I can tell you that story, I have to tell you this story…

Nick’s imaginary CSV factory

In the repo I’ve added a DummyCSV.csv, it’s (as you might have guessed) a CSV file.

This CSV file is like a million other CSV formats out there – We’ve got a CSV file with Start Time, End Time, Customer, Talk Time, Calling Party, Called Party, Animal (for reasons) and CallID to uniquely identify this CDR.

Protip: The Rainbow CSV VScode extension makes viewing/editing/querying CSV files in VScode much easier.

Call Start TimeRow 0
Call End TimeRow 1
CustomerRow 2
Talk TimeRow 3
Calling PartyRow 4
Called PartyRow 5
AnimalRow 6
CallIDRow 7
File Format

Next we need to feed this into CGrateS, and for that we’ll be using the Event Reporter Service.

JSON config files don’t make for riveting blog posts, but you’ve made it this far, so let’s power through.

ERS is setup in CGrateS’ JSON config file, where we’ll need to define one or more readers which are the the logic we define inside CGrateS to tell it what fields are what, where to find the files we need to import, and set all the parameters for the imports.

This means if we have a CSV file type we get from one of our suppliers with CDRs in it, we’d define a reader to parse that type of file.
Likewise, if we’ve got a CSV of SMS traffic out of our SMSc, we’d need to define another reader to parse the CDRs in that format – Generally we’ll do a Reader for each file type we want to parse.

So let’s define a reader for this CSV spec we’ve just defined:


"ers": {
	"enabled": true,
	"readers": [
		{
			"id": "blog_example_csv_parser",
			"enabled": true,
			"run_delay":  "-1",
			"type": "*file_csv",
			"opts": {
				"csvFieldSeparator":",",
				"csvLazyQuotes": true,
				//csvLazyQuotes Counts the row length and if does not match this value declares an error
				//-1 means to look at the first row and use that as the row length
				"csvRowLength": -1
			},
			"source_path": "/var/spool/cgrates/blog_example_csv_parser/in",
			"processed_path": "/var/spool/cgrates/blog_example_csv_parser/out",
			"concurrent_requests": 1024,	//How many files to process at the same time
			"flags": [
				"*cdrs",
				"*log"
			],
			"tenant": "cgrates.org",
			"filters": [
				"*string:~*req.2:Nick", //Only process CDRs where Customer column == "Nick"
			],
			"fields":[]
}]}

This should hopefully be relatively simple (I’ve commented it as best I can).

The ID of the ERS object is just the name of this reader – you can name it anything you like, keeping in mind we can have multiple readers defined for different file formats we may want to read, and setting the ID just helps to differentiate them.

The run_delay of -1 means ERS will run as soon as a file is moved into the source_path directory, and the type is a CSV file – Note that’s moved not copied. We’ve got to move the file, not just copy it, as CGrateS waits for the inode notify.

In the opts section we set the specifics for the CSV we’re reading, field separator if how we’re separating the values in our CSV, and in our case, we’re using commas to delineate the fields, but if you were using a file using semicolons or another delineator, you’d adjust this.

Lastly we’ve got the paths, the source path is where we’ll need to move the files to get processed into, and the processed_path is where the processed files will end up.

For now I’ve set the flags to *log and *cdrs – By calling log we’ll make our lives a bit easier for debugging, and CDRs will send the event to the CDRs module to generate a rated CDR in CGrateS, which we could then use to bill a customer, a supplier, etc, and access via the API or exporting using Event Exporter Service.

Lastly under FilterS we’re able to define the filters that should define if we should process a row or not.
You don’t know how much you need this feature until you need this feature.
The filter rule I’ve included will only process lines where the Customer field in the CSV (row #2) is equal to “Nick”. You could use this to also filter only calls that have been answered, only calls to off-net, etc, etc – FilterS needs a blog post all on it’s own (and if you’re reading this in the future I may have already written one).

Alright, so far so good, we’ve just defined the metadata we need to do to read the file, but now how do we actually get down to parsing the lines in the file?

Well, that’s where the data in Fields: [] comes in.

If you’ve been following along the CgrateS in baby steps series, you’ll have rated a CDR using the API, that looked something like this:

{"method": "CDRsV1.ProcessExternalCDR", "params": [ { \
"Category": "call",
"RequestType": "*raw",
"ToR": "*monetary",
"Tenant": "cgrates.org",
"Account": "1002",
"Subject": "1002",
"Destination": "6141111124211",
"AnswerTime": "2022-02-15 13:07:39",
"SetupTime": "2022-02-15 13:07:30",
"Usage": "181s",
"OriginID": "API Function Example"
}], "id": 0}

ERS is going to use the same API to rate a CDR, calling more-or-less the same API, so we’re going to set the parameters that go into this from the CSV contents inside the fields:

			"fields":[

				//Type of Record (Voice)
				{"tag": "ToR", "path": "*cgreq.ToR", "type": "*constant", "value": "*voice"},

				//Category set to "call" to match RatingProfile_VoiceCalls from our RatingProfile
				{"tag": "Category", "path": "*cgreq.Category", "type": "*constant", "value": "call"},

				//RequestType is *rated as we won't be deducting from an account balance
				{"tag": "RequestType", "path": "*cgreq.RequestType", "type": "*constant", "value": "*rated"},
				
]

That’s the static values out of the way, next up we’ll define our values we pluck from the CSV. We can get the value of each row from “~*req.ColumnNumber” where ColumnNumber is the column number starting from 0.


				//Unique ID for this call - We get this from the CallID field in the CSV
				{"tag": "OriginID", "path": "*cgreq.OriginID", "type": "*variable","value":"~*req.7"},
				
				//Account is the Source of the call 
				{"tag": "Account", "path": "*cgreq.Account", "type": "*variable", "value": "~*req.4"},
				
				//Destination is B Party Number - We use 'Called Party Number'
				{"tag": "Destination", "path": "*cgreq.Destination", "type": "*variable", "value": "~*req.5"},
				{"tag": "Subject", "path": "*cgreq.Subject", "type": "*variable", "value": "~*req.5"},

				//Call Setup Time (In this case, CGrateS can already process this as a datetime object)
				{"tag": "SetupTime", "path": "*cgreq.SetupTime", "type": "*variable", "value": "~*req.0"},

				//Usage in seconds - We use 'Call duration'
				{"tag": "Usage", "path": "*cgreq.Usage", "type": "*variable", "value": "~*req.3"},

				//We can include extra columns with extra data - Like this one:
				{"tag": "Animal", "path": "*cgreq.Animal", "type": "*variable", "value": "~*req.6"},
]

Perfect,

If you’re struggling to get your JSON file right, that’s OK, I’ve included the JSON CGrateS.config file here.

You’ll need to restart CGrateS after putting the config changes in, but your instance will probably fail to start as we’ll need to create the directories we specified CGrateS should monitor for incoming CSV files:

mkdir /var/spool/cgrates/blog_example_csv_parser/
mkdir /var/spool/cgrates/blog_example_csv_parser/in
mkdir /var/spool/cgrates/blog_example_csv_parser/out

Right, now if we start CGrateS it should run.

But before we can put this all into play, we’ll need to setup some rates. My previous posts have covered how to do this, so for that I’ve included a Python script to setup all the rates, which you can run once you’ve restarted CGrateS.

Alright, with that out of the way, we can test it out, move our Dummy.csv file to /var/spool/cgrates/blog_example_csv_parser/in and see what happens.

mv Dummy.csv /var/spool/cgrates/blog_example_csv_parser/in/

All going well in your CGrateS log you’ll see all the events flying past for each row.

Then either via the CGrateS API, or just looking into the MySQL “cdrs” table you should see the records we just created.

And with that, you’ve rated CDRs from a CSV file and put them into CGrateS.

Libpython3.11 problems with Kamailio on Ubuntu 22.04

Kamailio, Python, Software, VoIP, , , , , ,

I run Ubuntu on my desktop and I mess with Kamailio a lot.

Recently I was doing some work with KEMI using Python, but installing the latest versions of Kamailio from the Debian Repos for Kamailio wasn’t working.

The following packages have unmet dependencies:
 kamailio-python3-modules : Depends: libpython3.11 (>= 3.11.0) but 3.11.0~rc1-1~22.04 is to be installed

Kamailio’s Python modules expect libpython3.11 or higher, but Ubuntu 22.04 repos only contain the release candidate – not the final version:

root@amanaki:/home/nick# apt-cache policy libpython3.11
libpython3.11:
  Installed: 3.11.0~rc1-1~22.04
  Candidate: 3.11.0~rc1-1~22.04
  Version table:
 *** 3.11.0~rc1-1~22.04 500
        500 http://au.archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages
        100 /var/lib/dpkg/status

Luckily the deadsnakes PPA to the rescue!

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get purge kamailio
sudo apt --fix-broken install
sudo apt-get upgrade
apt-get install kamailio kamailio-python3-modules

And done!

Kamailio Websocket – Allowing binding to Port 443 & Port 80

Kamailio, VoIP,

If you’re setting up Kamailio for support for WebSocket and need to bind to TCP port 80 or TCP port 443, you may run into the issue that permission is denied to bind to these ports when you try and start the service.

On Ubuntu, you can fix this with:

sudo setcap 'cap_net_bind_service=+ep' /usr/sbin/kamailio

Where /usr/sbin/kamailio is the install path for kamailio, you can check this using:

root@amanaki:/etc/kamailio# type kamailio
kamailio is hashed (/usr/sbin/kamailio)

CGrateS – Exporting CDRs

CGrateS, IMS / VoLTE, Mobile Networks, SDM, Software, VoIP,

Having rated CDRs in CGrateS is great, but in reality, you probably want to get them into a billing system, CSV file, S3 bucket, CRM, invoice, Grafana, SQL table, etc, etc.

The Event Exporter Service (EES (previously called CDRe)) handles exporting CDRs from CGrateS.

Like everything in CGrateS, it’s highly configurable, and, again, like everything in CGrateS, supports every combination of services you can think of, plus a stack you haven’t thought of.

CDRs can be exported one of two ways, in real time, as the CDR is generated (online), or after the fact, exporting from the database containing the CDRs (offline).

Exporting in realtime (online) is a great option if you don’t want (or need) to store the CDRs in CGrateS; if you’re just using CGrateS to rate calls and spit them into a seperate system, this is a fantastic option, as it allows your CGrateS instances to remain light and not get clogged up with lots of old CDRs – That said, of course you can export the CDRs in realtime and still store them in CGrateS, that’s also a totally valid approach as well.

The more traditional approach is offline CDR export, where periodically or when an event is triggered, you scrape up a pile of CDRs and send them to your external systems.

For both options, we’ll need to define at least one exporter in our cgrates.json config file. For this example we’ll define a HTTP POST that we will trigger for realtime (online) CDR exporting, and a CSV file we dump to periodically when called from the API.

So first things first, we enable the EES module in the config:

"ees": {
		"enabled": true,
		"exporters": [
		]
	}

We’ll start with defining one exporter, named CSVExporter, that will output files to a folder named “testCSV” in the /tmp/ directory, but you can plonk these files wherever you like:

"ees": {
		"enabled": true,
		"synchronous": true,
		"exporters": [
			{
				"id": "CSVExporter",
				"type": "*file_csv",
				"export_path": "/tmp/testCSV",
				"flags": ["*log"],
				"attempts": 1,
				"synchronous": true,
				"field_separator": ",",
			},
		]
	}

We’ve got a lot of different types of export available to us, but type *file_csv is the easiest, so that’s where we’ll start.

Setting synchronous to true will mean we’ll only run one export job at a time, but it also means we’ll get back the result via the API, which will allow us to keep track of the ID of the last record we updated, so we don’t export the same record multiple times, more on this later.

Flags allows us to, if we wanted, bounce the event through AttributeS, for example, by adding *attributes to the flags, but in this case, it’s just logging to syslog.

Of course, just enabling ees won’t actually send calls to it, we’ll need to add “ees_conns“: [“*localhost”], to “apiers”: and “cdrs” so they know to bounce the events through it:

	"apiers": {
		"enabled": true,
                ...
		"ees_conns": ["*localhost"],
	},

	"cdrs": {
		"enabled": true,
		...
		"ees_conns": ["*localhost"],
	},

Okay, enough talk, let’s get exporting some CDRs!

If you’ve already got CDRs on your system from our previous tutorial, fantastic, but if not, let’s get up and running with a quick and dirty script to define some destinations, a charger, an account balance and then use some of the balance to generate a CDR:

import cgrateshttpapi
import pprint
import uuid
import datetime
now = datetime.datetime.now()
CGRateS_Obj = cgrateshttpapi.CGRateS('localhost', 2080)

#Define Destinations
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":'cgrates.org',"ID":"Dest_AU_Mobile","Prefixes":["614"]}]})

#Load TariffPlan we just defined from StorDB to DataDB
CGRateS_Obj.SendData({"method":"APIerSv1.LoadTariffPlanFromStorDb","params":[{"TPid":'cgrates.org',"DryRun":False,"Validate":True,"APIOpts":None,"Caching":None}],"id":0})

#Define default Charger
print(CGRateS_Obj.SendData({"method": "APIerSv1.SetChargerProfile","params": [{"Tenant": "cgrates.org","ID": "DEFAULT",'FilterIDs': [],'AttributeIDs' : ['*none'],'Weight': 0,}]}))

account = "Nick_Test_123"

#Add a balance to the account with type *sms with 100 sms events
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV1.SetBalance","params": [{"Tenant": "cgrates.org","Account": account,"BalanceType": "*sms","DestinationIDs": 'Dest_NZ_Mobile;Dest_AU_Mobile',"Categories": "*any","Balance": {"ID": "100_SMS_Bundle_AU_NZ_Mobile","Value": 100,"Weight": 25}}]}))

#Process CDR Event for a single SMS
pprint.pprint(CGRateS_Obj.SendData({"method": "CDRsV2.ProcessExternalCDR","params": [{"OriginID": str(uuid.uuid1()),"ToR": "*sms","RequestType": "*pseudoprepaid","AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),"SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),"Tenant": "cgrates.org","Account": account,"Destination" : "61412345678","Usage": "1",}]}))

Right, with that out of the way, we should now have something in our CDRs table, a quick SQL query confirms this is the case:

Bingo, there we go.

So let’s try an offline export via the API:

result = CGRateS_Obj.SendData({
	"method": "APIerSv1.ExportCDRs",
	"params": [
		{
			"ExporterIDs": [
				"CSVExporter"
			],
			"Verbose": True,
			"Accounts": [account
			]
		}
	]
})
pprint.pprint(result)

So, as you may have guessed, we’ve called the ExportCDRs API endpoint, we’ve specified which ExporterIDs we want to reference (these link back to the objects in the config, and the one we have defined currently is named CSVExporter).

Setting Verbose: True means that CGrateS gives us back a lot of info from the API call, here’s what we get back:

{"error": None,
 "id": None,
 "result": {"CSVExporter": {"ExportPath": "/tmp/testCSV/CSVExporter_21e9bc2.csv",
                            "FirstEventATime": "2024-01-02T18: 09: 29+11: 00",
                            "FirstExpOrderID": 14,
                            "LastEventATime": "2024-01-02T18: 40: 53+11: 00",
                            "LastExpOrderID": 25,
                            "NegativeExports": [],
                            "NumberOfEvents": 12,
                            "PositiveExports": ["f45dd29",
                                                ...
                                                "6163255"
            ],
                            "TimeNow": "2024-01-02T18: 40: 53.791517662+11: 00",
                            "TotalCost": 0,
                            "TotalSMSUsage": 12
        }
    }
}

Now that looks pretty positive, we got 12 events of SMS usage exported, which we can see in the file /tmp/testCSV/CSVExporter_21e9bc2.csv – and if we cat out the file, yeap, there’s all the CDRs.

But it’s a bit of a mess, there’s a lot of fields in there, so let’s adjust what goes into the CSV.

Let’s start by filtering what goes into the exporter, to only give us SMS events, of course you could adjust the filters here to target exporting only the records you want, based on anything you can define with Filters (and there’s a lot you can define with filters).

	"ees": {
		"enabled": true,
		"exporters": [
			{
				"id": "CSVExporter",
				"type": "*file_csv",
				"export_path": "/tmp/testCSV",
				"flags": ["*log"],
				"attempts": 1,
				"filters": ["*string:~*req.ToR:*sms"],
				"synchronous": true,
				"field_separator": ",",
				...

Now we’re only exporting SMS records, so let’s clean up the output of the CSV to just give us the data we want, which is the CDR ID, time, account, destination and usage.

	"ees": {
		"enabled": true,
		"exporters": [
			{
				"id": "CSVExporter",
				"type": "*file_csv",
				"export_path": "/tmp/testCSV",
				"flags": ["*log"],
				"attempts": 1,
				"filters": ["*string:~*req.ToR:*sms"],
				"synchronous": true,
				"field_separator": ",",
				"fields":[
					//Headers
					{"tag": "CGRID", "path": "*hdr.CGRID", "type": "*constant", "value": "CGRID"},
					{"tag": "AnswerTime", "path": "*hdr.AnswerTime", "type": "*constant", "value": "AnswerTime"},
					{"tag": "Account", "path": "*hdr.Account", "type": "*constant", "value": "Account"},
					{"tag": "Destination", "path": "*hdr.Destination", "type": "*constant", "value": "Destination"},
					{"tag": "Usage", "path": "*hdr.Usage", "type": "*constant", "value": "Usage"},
					//Values
					{"tag": "CGRID", "path": "*exp.CGRID", "type": "*variable", "value": "~*req.CGRID"},
					{"tag": "AnswerTime", "path": "*exp.AnswerTime", "type": "*variable", "value": "~*req.AnswerTime{*time_string:2006-01-02T15:04:05Z}"},
					{"tag": "Account", "path": "*exp.Account", "type": "*variable", "value": "~*req.Account"},
					{"tag": "Destination", "path": "*exp.Destination", "type": "*variable", "value": "~*req.Destination"},
					{"tag": "Usage", "path": "*exp.Usage", "type": "*variable", "value": "~*req.Usage"},

				],
			},
...

Now after a restart of CGrateS, our exports look like this:

Stunning, truly beautiful, look at that output!

Right, well you may at this point have noticed a problem if you’ve run this more than once. The problem is that is every time we run this, we get all the CDRs since the beginning of time.

Now there’s a few ways we can handle this, if we only want CDRs generated in the past day for example, we can filter that as an input on the ExportCDRs API call, using the multitude of filters available to us as documented in the API docs.

But where filtering by date/time falls down, is that if an offline CDR of a call on Monday, only got ingested on Tuesday, it would be missed by the export.

But, setting Verbose: True on the ExportCDRs API call gives us a handy trick, we’ve been told what the highest ID in the CDRs table we just exported in the response from the API in LastExpOrderID field.

If we jump over to the SQL database we use for StorDB, we can see that 33 is the ID of the highest CDR in the system.

So let’s try something, let’s run the exporter again, but this time let’s get all the CDRs where the ID is higher than 33:

#Process CDR Event for a single SMS
pprint.pprint(CGRateS_Obj.SendData({"method": "CDRsV2.ProcessExternalCDR","params": [{"OriginID": str(uuid.uuid1()),"ToR": "*sms","RequestType": "*pseudoprepaid","AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),"SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),"Tenant": "cgrates.org","Account": account,"Destination" : "61412345678","Usage": "1",}]}))

#Trigger export where the OrderID is above 33
result = CGRateS_Obj.SendData({"method":"APIerSv1.ExportCDRs","params":[
    {"ExporterIDs": ["CSVExporter"],
     "Verbose" : True,
     "ExtraArgs" : {
        "OrderIDStart" : int(33),
     },
     "Accounts" : [account]}
]})
pprint.pprint(result)

Boom, now if we have a look at the output we can see the export covered two records, and the last ID was 35.

{'method': 'APIerSv1.ExportCDRs', 'params': [{'ExporterIDs': ['CSVExporter'], 'Verbose': True, 'ExtraArgs': {'OrderIDStart': 33}, 'run_id': 'carrier_interconnect', 'Accounts': ['Nick_Test_123']}]}
{'error': None,
 'id': None,
 'result': {'CSVExporter': {'ExportPath': '/tmp/testCSV/CSVExporter_c444cd9.csv',
                            'FirstEventATime': '2024-01-02T19:19:59+11:00',
                            'FirstExpOrderID': 34,
                            'LastEventATime': '2024-01-02T19:20:08+11:00',
                            'LastExpOrderID': 35,
                            'NegativeExports': [],
                            'NumberOfEvents': 2,
                            'PositiveExports': ['034aba2', '22e4fa7'],
                            'TimeNow': '2024-01-02T19:20:08.355664133+11:00',
                            'TotalCost': 0,
                            'TotalSMSUsage': 2}}}

So as long as we keep track of the LastExpOrderID value, and feed that as in input every time we run ExportCDRs, we can ensure we never miss a CDR, and never get the same CDR twice.

How do you know if they’re roaming? Charging challenges in IMS for Roamers

CGrateS, IMS / VoLTE, Mobile Networks, SDM, VoIP, , , ,

I got an email the other day asking a simple question:

How do I know if a subscriber is VoLTE roaming or not when they send an SMS to charge for it?

My immediate reaction was to look at the SIP headers, P-Access-Network-Info will tell you where the subscriber is located, end of.

Right?

Well not quite, this will tell the SMSc the location of the subscriber sending the SMS. If the PLMN in the P-Access-Network-Info != the home PLMN, the sub is roaming.

But does this information get passed to the OCS / OFCS?

The SMSc uses “Event based charging” to perform credit control, so let’s have a look at what AVPs are present in the Credit Control Request from the SMSc:

Hmm, the SMS-Information AVP (2000) contains a bunch of information about the SMS being sent, but I don’t see anything about the location of the sender in there.

Originator-Interface is just set to “SIP”, of course in a 2G/3G roaming scenario the Originator-SCCP-Address would be that of the Visited PLMN, but for us it is our SCCP address.

Maybe the standard allows for an additional optional AVP in the SMS-Information-AVP we’re missing? Let’s check TS 32.299:

Nope.

So how to deal with this?

While the standards aren’t totally clear on this, we added an IMS-Info AVP and inside that populated the Access-Network-Information directly from the SIP header, and then picked that off inside our OCS in order to apply the correct rules.

Kamailio Bytes: Stripping SIP Multipart Bodies

FreeSWITCH, IMS / VoLTE, Kamailio, Mobile Networks, VoIP, , ,

For some calls in (such as some IMS emergency calls) you’ll get MIME Multipart Media Encapsulation as the SIP body, as the content-type set to:

Content-Type: multipart/mixed;boundary=968f194ab800ab27

If you’re used to dealing with SIP, you’d expect to see:

Content-Type: application/sdp

This Content-Type multipart/mixed;boundary is totally valid in SIP, in fact RFC 5261 (Message Body Handling in the Session Initiation Protocol (SIP)) details the use of MIME in SIP, and the Geolocation extension uses this, as we see below from a 911 call example.

But while this extension is standardised, and having your SIP Body containing multipart MIME is legal, not everything supports this, including the FreeSWITCH bridge module, which just appends a new SDP body into the Mime Multipart

Site note: I noticed FreeSWITCH Bridge function just appends the new SIP body in the multipart MIME, leaving the original, SDP:

Okay, so how do we replace the MIME Multipart SIP body with a standard SDP?

Well, with Kamalio’s SDP Ops Module, it’s fairly easy:

#If the body is multipart then strip it and replace with a single part
if (has_body("multipart/mixed")) {
	xlog("This has a multipart body");
	if (filter_body("application/sdp")) {
		remove_hf("Content-Type");
		append_hf("Content-Type: application/sdp\r\n");
	} else {
		xlog("Body part application/sdp not found\n");
		}
}

I’ve written about using SDPops to modify SDP before.

And with that we’ll take an SIP message like the one shown on the left, and when relayed, end up with the message on the right:

Simple fix, but saved me having to fix the fault in FreeSWITCH.

CGrateS – AttributeS

CGrateS, Software, VoIP, , ,

The docs describe AttributeS as a Key-Value-Store, but that’s probably selling it short – You can do some really cool stuff with AttributeS, and in this post, we’re going to learn about using AttributeS to transform stuff.

Note: Before we get started, I’d suggest copying this config file to use for testing.

Let’s look at a really basic example, where we add some data into AttributeS, match based on Account in CGrateS, and get back that data.

Let’s look at how this would look on the API:

{
    "method": "APIerSv2.SetAttributeProfile",
    "params": [{
        "Tenant": "cgrates.org",
        "ID": "ATTR_Nick_Key_Value_Example",
        "Contexts": ["*any"],
        "FilterIDs": [
            "*string:~*req.Account:1234"
        ],
        "Attributes": [
            {
            "FilterIDs": [],
            "Path": "*req.ExampleKey",
            "Type": "*constant",
            "Value": "ExampleValue"
            }
        ],
        "Blocker": False,
        "Weight": 10

    }],
}

So what are we doing in this API call?

Well, for starters we’re calling the SetAttributeProfile endpoint, this is where we go to create / update Attribute Profiles, but in this case, because we’re hitting it for the first time with this ID, we’re creating a new entry called “ATTR_Nick_Key_Value_Example“, this will match any Contexts (more on them later) where the FilterIDs is a string, where the request Account, is equal to 1234.

Let’s run this against the CGrateS API and take a look at the result:

import cgrateshttpapi
import pprint

CGRateS_Obj = cgrateshttpapi.CGRateS('localhost', 2080)

SetAttributeProfile = {
    "method": "APIerSv2.SetAttributeProfile",
    "params": [{
        "Tenant": "cgrates.org",
        "ID": "ATTR_Nick_Key_Value_Example",
        "Contexts": ["*any"],
        "FilterIDs": [
            "*string:~*req.Account:1234"
        ],
        "Attributes": [
            {
            "FilterIDs": [],
            "Path": "*req.ExampleKey",
            "Type": "*constant",
            "Value": "ExampleValue"
            }
        ],
        "Blocker": False,
        "Weight": 10
    }],
}
result = CGRateS_Obj.SendData(SetAttributeProfile)
pprint.pprint(result)

result = CGRateS_Obj.SendData({"method":"AttributeSv1.ProcessEvent",
                               "params":[
                                   {"Tenant":"cgrates.org",
                                    "Event":{"Account":"1234"},"APIOpts":{}}]})
pprint.pprint(result)

All going well you should have got the following back:

{'method': 'AttributeSv1.ProcessEvent', 'params': [{'Tenant': 'cgrates.org', 'Event': {'Account': '1234'}, 'APIOpts': {}}]}
{'error': None,
 'id': None,
 'result': {'APIOpts': {},
            'AlteredFields': ['*req.ExampleKey'],
            'Event': {'Account': '1234', 'ExampleKey': 'ExampleValue'},
            'ID': '',
            'MatchedProfiles': ['cgrates.org:ATTR_Nick_Key_Value_Example'],
            'Tenant': 'cgrates.org',
            'Time': None}}

This tells us we matched the Attribute with the ID ATTR_Nick_Key_Value_Example, and inside Event we can see that ExampleKey was added with value ExampleValue.

Okay, you’re saying, well what was the point of that?

Well, what if as a key in the attributes, we had the password for the SIP account, which we passed to our SIP switch (Kamailio, FreeSWITCH or Asterisk for example), and used that to authenticate?

Let’s see how that would look:

{
    "method": "APIerSv2.SetAttributeProfile",
    "params": [{
        "Tenant": "cgrates.org",
        "ID": "ATTR_Nick_Password_Example",
        "Contexts": ["*any"],
        "FilterIDs": [
            "*string:~*req.Account:1234"
        ],
        "Attributes": [
            {
            "FilterIDs": [],
            "Path": "*req.SIP_password",
            "Type": "*constant",
            "Value": "sosecretiputitonthewebsite"
            }
        ],
        "Blocker": False,
        "Weight": 10
    }],
}

Now if the CGrateS Agent for your SIP Switch, includes the *attributes flag, and the call is coming from 1234, we’ll get back a key called “SIP_password” with the value “sosecretiputitonthewebsite”, which you can use to auth the SIP account.

We can also return multiple AttributeS, for example, we created two Attributes (ATTR_Nick_Password_Example and ATTR_Nick_Key_Value_Example) which match on the account 1234. This means we’ll get back the SIP Password from ATTR_Nick_Password_Example and the key:value we set in ATTR_Nick_Key_Value_Example:

{'method': 'AttributeSv1.ProcessEvent', 'params': [{'Tenant': 'cgrates.org', 'Event': {'Account': '1234'}}]}
{'error': None,
 'id': None,
 'result': {'APIOpts': {},
            'AlteredFields': ['*req.SIP_password', '*req.ExampleKey'],
            'Event': {'Account': '1234',
                      'ExampleKey': 'ExampleValue',
                      'SIP_password': 'sosecretiputitonthewebsite'},
            'ID': '',
            'MatchedProfiles': ['cgrates.org:ATTR_Nick_Password_Example',
                                'cgrates.org:ATTR_Nick_Key_Value_Example'],
            'Tenant': 'cgrates.org',
            'Time': None}}

The order can be controlled by the Weight flag in the attribute, and if you want to stop matching any other AttributeS rules after the current Attribute, you can set the Blocker=True flag when you create/update the Attribute.

Okay, I hear you saying, that’s all well and good, I can add arbitrary key/values to stuff. Here endeth the lesson right?

Well not quite, because we can add key/values, but we can also rewrite variables using AttributeS.

Let’s imagine we’ve got 3 phone numbers (DIDs) associated with an account inside CGrateS, for example’s sake let’s say we have 12340001, 12340002 and 12340003, and we want any calls from these numbers to be billed to a CGrateS account called “NickTest1234”.

Our SIP switch doesn’t need to know anything about “NickTest1234”, just the 3 DIDs it can use to call out from your SIP stack. But to do this, we’d need CGrateS to transform any events from these DIDs to replace the Account value inside CGrateS, with NickTest1234.

Let’s see how that would look:

{'method': 'APIerSv2.SetAttributeProfile', 'params': [{'Tenant': 'cgrates.org', 'ID': 'ATTR_Calling_NickTest1234_12340001', 'Contexts': ['*any'], 'FilterIDs': ['*string:~*req.Account:12340001'], 'Attributes': [{'Path': '*req.Account', 'Type': '*constant', 'Value': 'NickTest1234'}], 'Weight': 0}], 'id': 1}

{'method': 'APIerSv2.SetAttributeProfile', 'params': [{'Tenant': 'cgrates.org', 'ID': 'ATTR_Calling_NickTest1234_12340002', 'Contexts': ['*any'], 'FilterIDs': ['*string:~*req.Account:12340002'], 'Attributes': [{'Path': '*req.Account', 'Type': '*constant', 'Value': 'NickTest1234'}], 'Weight': 0}], 'id': 2}

{'method': 'APIerSv2.SetAttributeProfile', 'params': [{'Tenant': 'cgrates.org', 'ID': 'ATTR_Calling_NickTest1234_12340003', 'Contexts': ['*any'], 'FilterIDs': ['*string:~*req.Account:12340003'], 'Attributes': [{'Path': '*req.Account', 'Type': '*constant', 'Value': 'NickTest1234'}], 'Weight': 0}], 'id': 3}

In the example code to go with this I’ve put together a simple for loop to add these – You can find the code on Github (link at the bottom).

So with these defined, let’s try and rate something, we’ll add a default Charger, and add an SMS balance, before simulating an SMS where the account is set to 12340003:

#Define default Charger
print(CGRateS_Obj.SendData({"method":"APIerSv1.SetChargerProfile","params":[{"Tenant":"cgrates.org","ID":"DEFAULT","FilterIDs":[],"AttributeIDs":["*none"],"Weight":0}]}))

#Add an SMS Balance
print(CGRateS_Obj.SendData({"method":"ApierV1.SetBalance","params":[{"Tenant":"cgrates.org","Account":"Nick_Test_123","BalanceType":"*sms","Categories":"*any","Balance":{"ID":"SMS_Balance_1","Value":"100","Weight":25}}],"id":13}))

import uuid
import datetime
now = datetime.datetime.now()
result = CGRateS_Obj.SendData({
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*sms",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Tenant": "cgrates.org",
            #This is going to be transformed to Nick_Test_123 by Attributes
            "Account": "12340003",
            "Usage": "1",
        }
    ]
})
pprint.pprint(result)

Right, so all going well, here’s what you should see in the CDRs table:

Bingo! Despite the fact the Account in the ProcessExternalCDR was set to 12340003, and had no mention of “NickTest1234”, CGrateS transformed it to NickTest1234.

How did that happen? Well, inside our cgrates.json file we have set the cdrs and chargers modules to have a link to Attributes, which means that when we call CDRs or Chargers modules via the API, these will in turn bounce the data through AttributesS for any transformations.

This means we don’t need to run AttributeSv1.ProcessEvent ourselves, when we call CDRsV2.ProcessExternalCDR, the CDRs module will call AttributeSv1.ProcessEvent for us.

We can actually see this happening, using ngrep, which as you work more with CGrateS, is a tool you’ll get very familiar with, let’s take a peek:

sudo ngrep -t -W byline port 2012 -d lo

Now if we run the CDRsV2.ProcessExternalCDR again, we’ll see the CDRs module has called Attributes for us:

Boom, there it is, same as we ran, but it’s being handled by CGrateS for us.

If you look carefully you’ll see the context in the API request is set to “*cdrs”, this means the CDRs module is calling Attributes.

When we define each of our Attributes, as we did earlier in the post, we can set what contexts they are valid in, for example we may want to apply the transformation when called by CDRs, but not other modules, you can restrict that when you define the Attribute by setting “Contexts”: [“*cdrs”].

Okay, so we’ve done some account replacement, what else can we do?

Well, let’s look at some other use cases,

Here in Australia we’ve got a few valid dialing formats, you could dial E.164 format (Numbers look like: +61212341234), 0NSN format (Numbers look like: 02 1234 1234) or NSN format (Numbers look like: 1234 1234 assuming you’re in the 03 area code yourself).
If we want to define all our Destinations in E.164 format, we’ll need to to normalise the format using AttributeS, so the numbers always come as E.164.

Let’s give it a whirl with a static translation:

{
    "method": "APIerSv2.SetAttributeProfile",
    "params": [{
        "Tenant": "cgrates.org",
        "ID": "ATTR_0NSN_to_E164_Single",
        "Contexts": ["*any"],
        "FilterIDs": [
            "*string:~*req.Subject:0212341234"
        ],
        "Attributes": [
            {
            "FilterIDs": [],
            "Path": "*req.Subject",
            "Type": "*constant",
            "Value": "61212341234"
            }
        ],
        "Blocker": False,
        "Weight": 10
    }],
}

Now this will work, if we simulate an Event to AttributeS with the Subject 0212341234, it’ll get transformed by AttributeS to 61212341234.

The issue here is probably pretty obvious, the only matches one number, if we dial 0212341235 this all falls apart.

Enter our old friend Regex.

For starters, we’ll change the FilterIDs to match where the Account is NickTest7, this way we can set the rules on a per CGrateS account basis.

{
    "method": "APIerSv2.SetAttributeProfile",
    "params": [{
        "Tenant": "cgrates.org",
        "ID": "ATTR_0NSN_to_E164_02_Area_Code",
        "Contexts": ["*any"],
        "FilterIDs": [
            "*string:~*req.Account:NickTest7"
        ],
        "Attributes": [
            {
            "FilterIDs": [],
            "Path": "*req.Subject",
            "Type": "*variable",
            "Value": "~*req.Subject:s/^0(\d{1})(\d{8})$/61${1}${2}/"
            },
            {
            "FilterIDs": [],
            "Path": "*req.Subject",
            "Type": "*variable",
            "Value": "~*req.Subject:s/^(\d{8})$/612${1}/"
            },
        ],
        "Blocker": False,
        "Weight": 10
    }],
}

And then under AttributeS we’ve defined a rule to replace anything matching the 0NSN regex, to strip the first digit and append a 61, to put it in E.164 format, and in SN format as the second entry.

We can now test this out:

{'method': 'AttributeSv1.ProcessEvent', 'params': [{'Tenant': 'cgrates.org', 'Event': {'Account': 'NickTest7', 'Subject': '0312341234'}, 'APIOpts': {'*processRuns': 5, '*profileRuns': 5, '*subsys': '*sessions'}}]}
{'error': None,
 'id': None,
 'result': {'APIOpts': {'*processRuns': 5,
                        '*profileRuns': 5,
                        '*subsys': '*sessions'},
            'AlteredFields': ['*req.Subject'],
            'Event': {'Account': 'NickTest7', 'Subject': '61312341234'},
            'ID': '',
            'MatchedProfiles': ['cgrates.org:ATTR_0NSN_to_E164_02_Area_Code'],
            'Tenant': 'cgrates.org',
            'Time': None}}



{'method': 'AttributeSv1.ProcessEvent', 'params': [{'Tenant': 'cgrates.org', 'Event': {'Account': 'NickTest7', 'Subject': '12341234'}, 'APIOpts': {'*processRuns': 5, '*profileRuns': 5, '*subsys': '*sessions'}}]}
{'error': None,
 'id': None,
 'result': {'APIOpts': {'*processRuns': 5,
                        '*profileRuns': 5,
                        '*subsys': '*sessions'},
            'AlteredFields': ['*req.Subject'],
            'Event': {'Account': 'NickTest7', 'Subject': '61212341234'},
            'ID': '',
            'MatchedProfiles': ['cgrates.org:ATTR_0NSN_to_E164_02_Area_Code'],
            'Tenant': 'cgrates.org',
            'Time': None}}

And there you have it folks; our number format standardized.

We can combo / cascade AttributeS rules together, with the aid of the Weight and Blocker flags in the API.

Let’s imagine the 61212341234 number has been ported from Operator1 to Operator2, and the Destinations we’ve defined in CGrateS for this prefix are currently set to DST_Operator1.
But because this number has been ported we should use DST_Operator2, so we charge the Operator2, as this number has been ported.

This means we don’t need to duplicate destination definitions to show this number has been ported, as this will be updated as the call gets rated, so we just assign the Attribute to each ported number.

So let’s match where the Subject of the call is 61212341234 (even though we’re going to input the Subject as 12341234), and rewrite the Destination attribute to DST_Operator2:

{
    "method": "APIerSv2.SetAttributeProfile",
    "params": [{
        "Tenant": "cgrates.org",
        "ID": "ATTR_Ported_61212341234",
        "Contexts": ["*any"],
        "FilterIDs": [
            "*string:~*req.Subject:61212341234",
        ],
        "Attributes": [
            {
            "FilterIDs": [],
            "Path": "*req.Destination",
            "Type": "*constant",
            "Value": "DST_Operator2"
            },
        ],
        "Blocker": False,
        "Weight": 5
    }],
}

From the results we can see we matched two AttributeS rules, the first, ATTR_0NSN_to_E164_02_Area_Code reformatted the Subject of the call from 12341234 to 61212341234, then the updated Subject was passed through to ATTR_Ported_61212341234, which updated the Destination attribute to DST_Operator2.

{'method': 'AttributeSv1.ProcessEvent', 'params': [{'Tenant': 'cgrates.org', 'Event': {'Account': 'NickTest7', 'Subject': '12341234'}, 'APIOpts': {'*processRuns': 5, '*profileRuns': 5, '*subsys': '*sessions'}}]}
{'error': None,
 'id': None,
 'result': {'APIOpts': {'*processRuns': 5,
                        '*profileRuns': 5,
                        '*subsys': '*sessions'},
            'AlteredFields': ['*req.Subject', '*req.Destination'],
            'Event': {'Account': 'NickTest7',
                      'Destination': 'DST_Operator2',
                      'Subject': '61212341234'},
            'ID': '',
            'MatchedProfiles': ['cgrates.org:ATTR_0NSN_to_E164_02_Area_Code',
                                'cgrates.org:ATTR_Ported_61212341234'],
            'Tenant': 'cgrates.org',
            'Time': None}}

Hopefully this has helped you to dip a toe into the CGrateS AttributeS pool, and give you some ideas of what we can achieve inside AttributeS.

A complete working code & config is available on my Github here.

If you’re having issues, make sure you have loaded the config file, are running the latest version, and if in doubt (and not on a production system), this script will clear all the data for you so you can rule out anything interfering.

Android and Emergency Calling

GSM, IMS / VoLTE, LTE, Mobile Networks, RFCs & Standards, SDM, SIM Cards, VoIP, , , , , , ,

In the last post we looked at emergency calling when roaming, and I mentioned that there are databases on the handsets for emergency numbers, to allow for example, calling 999 from a US phone, with a US SIM, roaming into the UK.

Android, being open source, allows us to see how this logic works, and it’s important for operators to understand this logic, as it’s what dictates the behavior in many scenarios.

It’s important to note that I’m not covering Apple here, this information is not publicly available to share for iOS devices, so I won’t be sharing anything on this – Apple has their own ecosystem to handle emergency calling, if you’re from an operator and reading this, I’d suggest getting in touch with your Apple account manager to discuss it, they’re always great to work with.

The Android Open Source Project has an “emergency number database”. This database has each of the emergency phone numbers and the corresponding service, for each country.

This file can be read at packages/services/Telephony/ecc/input/eccdata.txt on a phone with engineering mode.

Let’s take a look what’s in mainline Android for Australia:

You can check ECC for countries from the database on the AOSP repo.

This is one of the ways handsets know what codes represent emergency calling codes in different countries, alongside the values set in the SIM and provided by the visited network.

CGrateS – ActionTriggers

CGrateS, EPC, IMS / VoLTE, Mobile Networks, Notes, SDM, Software, Uncategorized, VoIP, , , ,

In our last post we looked at Actions and ActionPlans, and one of the really funky things we can do is setting ActionPlans to trigger on a time schedule or setting ActionTriggers to trigger on an event.

We’re going to build on the examples we had on the last post, so we’ll assume your code is up to the point where we’ve added a Signup Bonus to an account, using an ActionPlan we assigned when creating the account.

In this post, we’re going to create an action that charges $6, called “Action_Monthly_Charge“, and tie it to an ActionPlan called “ActionPlan_Monthly_Charge“, but to demo how this works rather than charging this Monthly, we’re going to charge it every minute.

Then with our balances ticking down, we’ll set up an ActionTrigger to trigger when the balance drops below $95, and alert us.

Defining the Monthly Charge Action

The Action for the Monthly charge will look much like the other actions we’ve defined, except the Identifier is *debit so we know we’re deducting from the balance, and we’ll log to the CDRs table too:

# Action to add a Monthly charge of $6
Action_Monthly_Charge = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Monthly_Charge",
          "Actions": [
              {
                'Identifier': '*debit',
                'BalanceType': '*monetary',
               'Units': 6,
               'Id': 'Action_Monthly_Charge_Debit',
               'Weight': 70},
              {
                  "Identifier": "*log",
                  "Weight": 60,
                  'Id' : "Action_Monthly_Charge_Log"
              },
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Recurring Charge\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Monthly_Charge))

Next we’ll need to wrap this up into an ActionPlan, this is where some of the magic happens. Inside the action plan we can set a once off time, or a recurring time, kinda like Cron.

We’re setting the time to *every_minute so things will happen quickly while we watch, this action will get triggered every 60 seconds. In real life of course, for a Monthly charge, we’d want to trigger this Action monthly, so we’d set this value to *monthly. If we wanted this to charge on the 2nd of the month we’d set the MonthDays to “2”, etc, etc.

# # Create ActionPlan using SetActionPlan to trigger the Action_Monthly_Charge
SetActionPlan_Daily_Action_Monthly_Charge_JSON = {
    "method": "ApierV1.SetActionPlan",
    "params": [{
        "Id": "ActionPlan_Monthly_Charge",
        "ActionPlan": [{
            "ActionsId": "Action_Monthly_Charge",
            "Years": "*any",
            "Months": "*any",
            "MonthDays": "*any",
            "WeekDays": "*any",
            "Time": "*every_minute",
            "Weight": 10
        }],
        "Overwrite": True,
        "ReloadScheduler": True
    }]
}
pprint.pprint(CGRateS_Obj.SendData(
    SetActionPlan_Daily_Action_Monthly_Charge_JSON))

Alright, but now what’s going to happen?

If you think the accounts will start getting debited every 60 seconds after applying this, you’d be wrong, we need to associate this ActionPlan with an Account first, this is how we control which accounts get which ActionPlans tied to them, to do this we’ll use the SetAccout API again we’ve been using to create accounts:

# Create the Account object inside CGrateS & assign ActionPlan_Signup_Bonus and ActionPlan_Monthly_Charge
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionPlanIds": ["ActionPlan_Signup_Bonus", "ActionPlan_Monthly_Charge"],
            "ActionPlansOverwrite": True,
            "ReloadScheduler":True
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

So what’s going to happen if we run this?

Well, for starters the ActionPlan named “ActionPlan_Signup_Bonus” is going to be triggered, as in the ActionPlan it’s Timing is set to *asap, so CGrateS will apply the corresponding Action (“Action_Add_Signup_Bonus“) right away, which will credit the account $99.

But a minute after that, we’ll trigger the ActionPlan named “ActionPlan_Monthly_Charge”, as the timing for this is set to *every_minute, when the Action “Action_Monthly_Charge” is triggered, it’s going to be deducting $6 from the balance.

We can check this by using the GetAccount API:

# Get Account Info
pprint.pprint(CGRateS_Obj.SendData({'method': 'ApierV2.GetAccount', 'params': [
              {"Tenant": "cgrates.org", "Account": str(Account)}]}))

You should see a balance of $99 to start with, and then after 60 seconds, it should be down to $93, and so on.

{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-17T14:57:20.71493633+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '3a896369-8107-4e32-bcef-2d078c981b8a',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-17T14:57:21.802521707+11:00'}}

Triggering Actions based on Balances with ActionTriggers

Okay, so we’ve set up recurring charges, now let’s get notified if the balance drops below $95, we’ll start, like we have before, with defining an Action, this will log to the CDRs table, HTTP post and write to syslog:


#Define a new Action to send an HTTP POST
Action_HTTP_Notify_95 = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_HTTP_Notify_95",
          "Actions": [
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Balance dipped below $95\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
              {
                  "Identifier": "*http_post_async",
                  "ExtraParameters": "http://10.177.2.135/95_remaining",
                  "ExpiryTime": "*unlimited",
                  "Weight": 700
              },
              {
                  "Identifier": "*log",
                  "Weight": 1200
              }
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_HTTP_Notify_95))

Now we’ll define an ActionTrigger to check if the balance is below $95 and trigger our newly created Action (“Action_HTTP_Notify_95“) when that condition is met:


#Define ActionTrigger
ActionTrigger_95_Remaining_JSON = {
    "method": "APIerSv1.SetActionTrigger",
    "params": [
        {
            "GroupID" : "ActionTrigger_95_Remaining",
            "ActionTrigger": 
                {
                    "BalanceType": "*monetary",
                    "Balance" : {
                        'BalanceType': '*monetary',
                        'ID' : "*default",
                        'BalanceID' : "*default",
                        'Value' : 95,
                        },
                    "ThresholdType": "*min_balance",
                    "ThresholdValue": 95,
                    "Weight": 10,
                    "ActionsID" : "Action_HTTP_Notify_95",
                },
            "Overwrite": True
        }
    ]
}
pprint.pprint(CGRateS_Obj.SendData(ActionTrigger_95_Remaining_JSON))

We’ve defined the ThresholdType of *min_balance, but we could equally set this to ThresholdType to *max_balance, *balance_expired or trigger when a certain Counter has been triggered enough times.

Adding an ActionTrigger to an Account

Again, like the ActionPlan we created before, before the ActionTrigger we just created will be used, we need to associate it with an Account, for this we’ll use the AddAccountActionTriggers API, specify the Account and the ActionTriggerID for the ActionTrigger we just created.


#Add ActionTrigger to Account 
Add_ActionTrigger_to_Account_JSON = {
    "method": "APIerSv1.AddAccountActionTriggers",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionTriggerIDs": ["ActionTrigger_95_Remaining"],
            "ActionTriggersOverwrite": True
        }
    ]
}
pprint.pprint(CGRateS_Obj.SendData(Add_ActionTrigger_to_Account_JSON))

If we run this all together, creating the account with the “ActionPlan_Signup_Bonus” will give the account a $99 Balance. But after 60 seconds, “ActionPlan_Monthly_Charge” will kick in, and every 60 seconds after that, at which point the balance will get to below $95 when CGrateS will trigger the ActionTrigger “ActionTrigger_95_Remaining” and get the HTTP POST to the HTTP endpoint and log entry:

We can check on this using the ApierV2.GetAccount method, where we’ll see the ActionTrigger we just defined.

Checking out the LastExecutionTime we can see if the ActionTrigger been triggered or not.

So using this technique, we can notify a customer when they’ve used a certain amount of their balance, but we can lock out Accounts who have spent more than their allocated spend limit by setting an Action that suspends the Account once it reaches a certain level. We notify customers when balance expires, or if a certain number of counters has been triggered.

As always I’ve put all the code used in this example, from start to finish, up on GitHub.

CGrateS – Actions & Action Plans

CGrateS, Mobile Networks, Notes, Python, SDM, Software, VoIP, ,

In our last post we added a series of different balances to an account, these were actions we took via the API specifically to add a balance.

But there’s a lot more actions we may want to do beyond just adding balance.

CGrateS has the concept of “Actions” which are, as the name suggests, things we want to do to the system.

Some example Actions would be:

  • Adding / Deducting / Resetting a balance
  • Adding a CDR log
  • Enable/Disable an account
  • Sending HTTP POST request or email notification
  • Deleting / suspending account
  • Transferring balances

We can run these actions on a timed basis, or when an event is triggered, and group Actions together to run multiple actions via an ActionTrigger, this means we can trigger these Actions, not just by sending an API request, but based on the state of the subscriber / account.

Let’s look at some examples,

We can define an Action named “Action_Monthly_Fee” to debit $12 from the monetary balance of an account, and add a CDR with the name “Monthly Account Fee” when it does so.
We can use ActionTriggers to run this every month on the account automatically.

We can define an Action named “Usage_Warning_10GB” to send an email to the Account owner to inform them they’ve used 10GB of usage, and use ActionTriggers to send this when the customer has used 10GB of their *data balance.

Using Actions

Note: The Python script I’ve used with all the examples in this post is available on GitHub here.

Let’s start by defining an Account, just as we have before:

# Create the Account object inside CGrateS
Account = "Nick_Test_123"
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account)
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Let’s start basic; to sweeten the deal for new Accounts, we’ll give them $99 of balance to use in the first month they have the service. Rather than hitting the AddBalance API, we’ll define an Action named “Action_Add_Signup_Bonus” to credit $99 of monetary balance to an account.

If you go back to our last post, you should know what we’d need to do to add this balance manually with the AddBalance API, but let’s look at how we can create the same balance add functionality using Actions:

#Add a Signup Bonus of $99 to the account with type *monetary expiring a month after it's added
Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 99,
                  "ExpiryTime": "*month",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 1200,
                  "ExtraParameters": "",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 10
              }
]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Alright, this should look pretty familiar if you’ve just come from Account Balances.
You’ll notice we’re no longer calling, SetBalance, we’re now calling SetActions, to create the ActionsId with the name “Action_Add_Signup_Bonus“.
In “Action_Add_Signup_Bonus” we’ve got an actions we’ll do when “Action_Add_Signup_Bonus” is called.
We can define multiple actions, but for now we’ve only got one action defined, which has the Identifier (which defines what the action does) set to *topup to add balance.
As you probably guessed, we’re triggering a top up, and setting the BalanceId, BalanceType, Units, ExpiryTime and BalanceWeight just as we would using SetBalance to add a balance.

So how do we use the Action we just created? Well, there’s a lot of options, but let’s start with the most basic – Via the API:

# Trigger ExecuteAction
Account_Action_trigger_JSON = {"method": "APIerSv1.ExecuteAction", "params": [
    {"Tenant": "cgrates.org", "Account": str(Account), "ActionsId": "Action_Add_Signup_Bonus"}]}
pprint.pprint(CGRateS_Obj.SendData(Account_Action_trigger_JSON))

Boom, we’ve called the ExecuteAction API call, to execute the Action named “Action_Add_Signup_Bonus“.

We can check on this with GetAccount again and check the results:

# Get Account Info
pprint.pprint(CGRateS_Obj.SendData({'method': 'ApierV2.GetAccount', 'params': [
              {"Tenant": "cgrates.org", "Account": str(Account)}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-15T10:27:52.865119544+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '01cfb471-ba38-453a-b0e2-8ddb397dfe9c',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-15T10:27:52.865144268+11:00'}}

Great start!

Making Actions Useful

Well congratulations, we took something we previously did with one API call (SetBalance), and we did it with two (SetAction and ExcecuteAction)!

But let’s start paying efficiency dividends,

When we add a balance, let’s also add a CDR log event so we’ll know the account was credited with the balance when we call the GetCDRs API call.

We’d just modify our SetActions to include an extra step:

Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
...
              }, 
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Your sign up Bonus\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 10
              }
]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Boom, now we’ll get a CDR created when the Action is triggered.

But let’s push this a bit more and add some more steps in the Action:

As well as adding balance and putting in a CDR to record what we did, let’s also send a notification to our customer via an HTTP API (BYO customer push notification system) and log to Syslog what’s going on.

# Add a Signup Bonus of $99 to the account with type *monetary expiring a month after it's added
Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 99,
                  "ExpiryTime": "*month",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 1200,
                  "ExtraParameters": "",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 90
              },
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Your sign up Bonus\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
              {
                  "Identifier": "*http_post_async",
                  "ExtraParameters": "http://10.177.2.135/example_endpoint",
                  "ExpiryTime": "*unlimited",
                  "Weight": 70
              },
              {
                  "Identifier": "*log",
                  "Weight": 60
              }
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Phew! That’s a big action, but if we execute the action again using ExecuteAction, we’ll get all these things happening at once:

Okay, now we’re getting somewhere!

ActionPlans

Having an Action we can trigger manually via the API is one thing, but being able to trigger it automatically is where it really comes into its own.

Let’s define an ActionPlan, that is going to call our Action named “Action_Add_Signup_Bonus” as soon as the ActionPlan is assigned to an Account.

# Create ActionPlan using SetActionPlan to trigger the Action_Signup_Bonus ASAP
SetActionPlan_Signup_Bonus_JSON = {
    "method": "ApierV1.SetActionPlan",
    "params": [{
        "Id": "ActionPlan_Signup_Bonus",
        "ActionPlan": [{
            "ActionsId": "Action_Add_Signup_Bonus",
            "Years": "*any",
            "Months": "*any",
            "MonthDays": "*any",
            "WeekDays": "*any",
            "Time": "*asap",
            "Weight": 10
        }],
        "Overwrite": True,
        "ReloadScheduler": True
    }]
}
pprint.pprint(CGRateS_Obj.SendData(SetActionPlan_Signup_Bonus_JSON))

So what have we done here? We’ve made an ActionPlan named “Action_Add_Signup_Bonus”, which, when associated with an account, will run the Action “Action_Add_Signup_Bonus” as soon as it’s tied to the account, thanks to the Time “*asap“.

Now if we create or update an Account using the SetAccount method, we can set the ActionPlanIds to reference our “ActionPlan_Signup_Bonus” and it’ll be triggered straight away.

# Create the Account object inside CGrateS
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionPlanIds": ["ActionPlan_Signup_Bonus"],
            "ActionPlansOverwrite": True,
            "ReloadScheduler":True
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Now if we were to run a GetAccount API call, we’ll see the Account balance assigned that was created by the action Action_Add_Signup_Bonus which was triggered by ActionPlan assigned to the account:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-16T12:41:02.530985381+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '7bdbee5c-0888-4da2-b42f-5d6b8966ee2d',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-16T12:41:12.7236096+11:00'}}

But here’s where it gets interesting, in the ActionPlan we just defined the Time was set to “*asap“, which means the Action is triggered as soon as it was assigned to the account, but if we set the Time value to “*monthly“, the Action would get triggered every month, or *every_minute to trigger every minute, or *month_end to trigger at the end of every month.

Code for these examples is available here.

I’m trying to keep these posts shorter as there’s a lot to cover. Stick around for our next post, we’ll look at some more ActionTriggers to keep decreasing the balance of the account, and setting up ActionTriggers to send a notification to the customer to tell them when their balance is getting low, or any other event based Action you can think of!

CGrateS – Accounts & Balances

CGrateS, IMS / VoLTE, Mobile Networks, Notes, Software, VoIP, , ,

So far we’ve used CGrateS to rate a basic CDR and get a cost for it, but in the real world, we’d usually associate the cost with an account, which would represent a business or a person, who will ultimately be charged for using the service.

Note: I’ve put the code for all this in Github, if you’ve got issues following along, or don’t want to copy and paste the code from the website, you can grab the code here.

Creating an Account

Let’s start off by creating an account inside CGrateS – This is kinda pointless, but we’ll talk more about that later:

#Create the Account object inside CGrateS 
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123"
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Running this onto the API should create an account named “Nick_Test_123”, but let’s confirm that’s the case:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))

Running this will give us the information about the account we just created:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
OrderedDict([('id', None),
             ('result',
              OrderedDict([('ID', 'cgrates.org:Nick_Test_123'),
                           ('BalanceMap', None),
                           ('UnitCounters', None),
                           ('ActionTriggers', None),
                           ('AllowNegative', False),
                           ('Disabled', False),
                           ('UpdateTime',
                            '2023-10-09T16:53:37.524466041+11:00')])),
             ('error', None)])

That was easy!

There’s not really much to see on our account at this stage, other than the UpdateTime, there’s nothing really going on, we don’t have any Balances.

Adding Balance for Voice

Accounts exist for spending, so let’s add a balance to this account to send from.

We’ll use the SetBalance API to create a new balance with 5 minutes of talk time, that we can use for making a call, and talking, for (you guessed it) – 5 minutes, so and we’ll use the balance “5_minute_voice_balance” that we’ll create:

#Add a balance to the account with type *voice with 5 minutes of Talk Time
Create_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "5_minute_voice_balance",
                "Value": "5m",
                "Weight": 25
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Voice_Balance_JSON))

Now if we run the GetAccount API command again, we should see the new balance we just created:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': '37423d07-d99a-40b1-851a-981c3df02cb3',
                                       'Value': 300000000000,
                                       'Weight': 25}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T17:58:23.801531205+11:00'}}

So now we’ve got a new balance named ‘5_minute_voice_balance‘:

  • The type is *voice, because this balance is storing talk time
  • The weight of this balance is 25, this means this balance should take priority over any balances with a lower value than 25 (that’s right, we can (and will) do tiered balances)
  • The value is 300000000000 nanoseconds, which equates to 5 minutes (yes, that’s the correct number of zeros)

Okay, but Nick_Test_123 probably wants to make some calls, so let’s generate a 2.5 minute call event and check out what happens.


#Generate a new call event for a 2.5 minute (150 second) call
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "150s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

Alright, now we’ve got a call event, let’s call the GetAccount API again to check the balance:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': '37423d07-d99a-40b1-851a-981c3df02cb3',
                                       'Value': 150000000000,
                                       'Weight': 25}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T17:58:23.80925546+11:00'}}

And there you have it, we’ve used 150 seconds of our 300 second (5 minutes) of talk time in this balance, leaving with us 150000000000 nanoseconds (150 seconds) remaining!

And with that progress, now is a great time to pause and talk about some theory that’s really important to grasp!

Balance Types

But *voice is just one balance type – We can support multiple balance types; we’ve just given a balance of *voice for talk time, but we could also give a credit to the balance for *sms or *data, you name it (*generic) and cash (*monetary) and we can have multiple separate balances for each.

This means we can have one account with something like:

  • 100 minutes of Local / National Calls (Expires at the end of the month)
  • 40 minutes of Mobile Calls (Expires 24 hours after it’s been created)
  • 80 minutes of Mobile Calls (During “Happy Hour” from 6 to 7pm)
  • 50 minutes of International Calls (Expires in 30 days)

And not just voice balance, the same account could also have:

  • 1GB of Data usage
  • 50 SMS to on-net destinations
  • $200 of Cash (expiring never)

Phew! That’s a lot of balances, but we can do it all through CGrateS!

What Balance to Use

So if we’ve got a stack of balances defined, how does CGrateS know what balance to use?

Firstly CGrateS is going to evaluate the BalanceType, this is set on events, so if we get an event for *data CGrateS will check out the balances available for *data, and evaluate the balances by Weight, with the highest weight evaluated first.
If we get to the end of all the available balances for that BalanceType, CGrateS then evaluates *generic and then *monetary balances, again, ordered by Weight.

We can set what balance gets used based on the Destination; using DestinationIDs we can filter the Balance to only apply for calls to Local/National numbers, so a call to an International destination won’t use that balance.

We can also set an Expiry on the Balance, for example we can give a customer 30 days to use the balance, after which it expires and can’t be used, likewise we can set Timings so enable scenarios like a “Happy hour” with extra calls between 6pm and 7pm.

When we define a balance we can also set the Blocker flag to True, if this is set, it means CGrateS will not look evaluate any balances after reaching that balance.

Adding a Balance for Local / National & Mobile Calls

Let’s jump back into the practice, and define two new Balances; one for Local/National calls, and another for Mobile calls.
But first we’ll need to know what destinations are mobiles and what are local/national (fixed). We’ve covered setting Destinations previously, so let’s set up the Destinations:


CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_Fixed","Prefixes":["612", "613", "617", "618"]}]})
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_Mobile","Prefixes":["614"]}]})
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_TollFree","Prefixes":["6113", "6118"]}]})
#Load TariffPlan we just defiend from StorDB to DataDB
print(CGRateS_Obj.SendData({"method":"APIerSv1.LoadTariffPlanFromStorDb","params":[{"TPid":"cgrates.org","DryRun":False,"Validate":True,"APIOpts":None,"Caching":None}],"id":0}))

Alright, now let’s add a balance for our local/national (fixed) calls.

To do this, we’ll add two new balances, but we’ll need to differentiate this from the 5_minute_voice_balance we created earlier, and to achieve this we weill:

  • Set a higher Weight value than we have set on 5_minute_voice_balance (25) so this balance will get consumed before 5_minute_voice_balance does
  • Set the DestinationIDs to match the destinations (Dest_AU_Mobile for Mobile and Dest_AU_Fixed for Local/National) we want the balance to apply to

ProTip: When you we create our Balance we can set what Destinations we want to use this balance for, if you want to specify multiple balances, we can do it by setting the Balance names as a string delimited by semicolons, like “DestinationIDs”: “Dest_AU_Fixed;Dest_AU_Mobile;Dest_AU_TollFree”

We’ll also set a balance expiry, which we’ll cover shortly, but now let’s define out 100 minutes for Local/National expiring at the end of the month:

#Add a balance to the account with type *voice with 100 minutes of talk time to Local / National Destinations expiring at the end of the month
Create_Local_National_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "Local_National_100_minutes_voice_balance",
                "Value": "100m",
                "ExpiryTime": "*month_end",
                "Weight": 60,
                "DestinationIDs": "Dest_AU_Fixed",
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Local_National_Voice_Balance_JSON))

We’ll also add our 24 hours to use to use 40 minutes of talk to mobiles, and a GetAccount to check the result:

#Add a balance to the account with type *voice with 40 minutes of talk time to Mobile Destinations expiring in 24 hours
Create_Mobile_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "Mobile_40_minutes_voice_balance",
                "Value": "40m",
                "ExpiryTime": "*daily",
                "Weight": 60,
                "DestinationIDs": "Dest_AU_Mobile",
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Mobile_Voice_Balance_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

Alright, let’s try running that:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'ad9d8bdd-64df-430f-af9d-3fc0410fd16b',
                                       'Value': 150000000000,
                                       'Weight': 25},
                                      {'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': {'Dest_AU_Fixed': True},
                                       'Disabled': False,
                                       'ExpirationDate': '2023-10-31T23:59:59+11:00',
                                       'Factor': None,
                                       'ID': 'Local_National_100_minutes_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'e4a2c211-8112-4e40-b3e6-250863404cc9',
                                       'Value': 6000000000000,
                                       'Weight': 60},
                                      {'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': {'Dest_AU_Mobile': True},
                                       'Disabled': False,
                                       'ExpirationDate': '2023-10-15T18:15:11.521636734+11:00',
                                       'Factor': None,
                                       'ID': 'Mobile_40_minutes_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'd4cbf6d8-50a5-4c97-82c2-dfe9936ae8d1',
                                       'Value': 2400000000000,
                                       'Weight': 60}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T18:15:11.524242437+11:00'}}

Alright! We now have 3 balances defined!

Notice in the API in the expiry I put *daily and *month_end, but in the output it’s got a real date and time (I wrote this 14/10/23 around 07:00 UTC, hence why those dates are what they are).
I could have specified the date and time in the API of a specific time I wanted the balance to expire (You can too, just replace “*daily” with “2024-01-01T00:00:00Z” for example), but that’s a pain in the butt, especially considering most of the time these values will be something common.
The *month_end is a special “meta” value, there’s a heap of these that allow us to do things like “current time + 20 minutes” (+20m), this time next month (*monthly), “this time tomorrow” (*daily), or “this time next week” (+168h) – You can find the full list of special dates here.

From a product perspective, setting an expiry on balances means we can set credit to expire 2 years after the subscriber tops up, but the same logic can be used so a subscriber could purchase a 7 day addon pack, that expires in 7 days, or a monthly plan can automatically expire in 30 days.

Now if we call the ProcessExternalCDR API again with a call to a Mobile and a Fixed number, we’ll see the respective balances get deducted.


#Generate a new call event for a 2.5 minute (150 second) call to a mobile number
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61412341234",
            "Destination": "61412341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "30s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

#Generate a new call event for a 2.5 minute (150 second) call to a fixed line local/national number
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61212341234",
            "Destination": "61212341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "30s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

We should see the minutes reduced by 30 seconds for our Local_National_100_minutes_voice_balance and Mobile_40_minutes_voice_balance balances, while our 5_minute_voice_balance hasn’t been touched.

{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":"None",
   "Disabled":false,
   "ExpirationDate":"0001-01-01T00:00:00Z",
   "Factor":"None",
   "ID":"5_minute_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"29f21735-1d62-49b1-9c53-80eab6f7b005",
   "Value":150000000000,
   "Weight":25
},
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Fixed":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-31T23:59:59+11:00",
   "Factor":"None",
   "ID":"Local_National_100_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"54db4f60-342e-4738-aaf1-a1304badc41d",
   "Value":5970000000000,
   "Weight":60
},
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Mobile":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-15T18:32:34.888821482+11:00",
   "Factor":"None",
   "ID":"Mobile_40_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"501eb00e-e947-4675-926f-080911e66897",
   "Value":2370000000000,
   "Weight":60
}

One last thing we’ll try before we end, our Mobile_40_minutes_voice_balance has still got 39.5 minutes left, and our 5_minute_voice_balance has still got minutes remaining, so if we try and make a call that’s 2450 seconds (~41 minutes), we should consume all the remaining minutes in Mobile_40_minutes_voice_balance and the go onto consume the remaining 1 minute out of 5_minute_voice_balance.

Let’s test this theory!

#Generate a new call event for a 42 minute call to a mobile to use all of our Mobile_40_minutes_voice_balance and start consuming 5_minute_voice_balance
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61412341234",
            "Destination": "61412341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "2450s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

Let’s check the output:

{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":"None",
   "Disabled":false,
   "ExpirationDate":"0001-01-01T00:00:00Z",
   "Factor":"None",
   "ID":"5_minute_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"29f21735-1d62-49b1-9c53-80eab6f7b005",
   "Value":70000000000,
   "Weight":25
},
...
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Mobile":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-15T18:44:18.161474861+11:00",
   "Factor":"None",
   "ID":"Mobile_40_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"501eb00e-e947-4675-926f-080911e66897",
   "Value":0.0161939859,
   "Weight":60
}

Boom, there we have it! Used all of the minutes in Mobile_40_minutes_voice_balance and started eating into the 5_minute_voice_balance.

Note: I’ve put the code for all this in Github, if you’ve got issues following along, or don’t want to copy and paste the code from the website, you can grab the code here.

Alright, that was a long post! Sorry about that, and props for making it to the end, still so much to learn about CGrateS.

Tales from the Trenches: mode-set in AMR

IMS / VoLTE, Mobile Networks, VoIP, , , , ,

This one was a bit of a head scratcher for me, but I’m always glad to learn something new.

The handset made a VoLTE call, and it’s SDP offer shows it can support AMR and AMR-WB:

        Media Attribute (a): rtpmap:116 AMR-WB/16000/1
        Media Attribute (a): fmtp:116 mode-set=0,1,2,3,4,5,6,7,8;mode-change-capability=2;max-red=220
        Media Attribute (a): rtpmap:118 AMR/8000/1
        Media Attribute (a): fmtp:118 mode-set=0,1,2,3,4,5,6,7;mode-change-capability=2;max-red=220
        Media Attribute (a): rtpmap:111 telephone-event/16000
        Media Attribute (a): fmtp:111 0-15

Okay, that’s pretty normal, I can see we have the mode-set parameter defined, which indicates what modes the handset supports for each codec.

In our problem scenario, the Media Gateway that the call was sent to responded with this SDP answer:

        Media Description, name and address (m): audio 24504 RTP/AVP 118 110
        Media Attribute (a): rtpmap:118 AMR/8000
        Media Attribute (a): fmtp:118 mode-set=7
        Media Attribute (a): rtpmap:110 telephone-event/8000
        Media Attribute (a): fmtp:110 0-15
        Media Attribute (a): ptime:20
        Media Attribute (a): sendrecv
        [Generated Call-ID: FA163E564B37-f4d-98f56700-735d25-65357ee0-9c488]

But we got an error about not available codecs and the call drops, what gives?

Both sides support AMR (Only the phone supports AMR-WB), and the Media Gateway, as the answerer, supports mode-set 7, which is supported by the UE, so we should be good?

Well, not quite:

If mode-set is specified, it MUST be abided, and frames encoded with modes outside of the subset MUST NOT be sent in any RTP payload or used in codec mode requests. If not present, all codec modes are allowed for the payload type.

RFC 4867 – RTP Payload Format for AMR and AMR-WB

Okay, I get it, the answerer (media gateway) only supports mode 7, but the UE supports all the modes, so we should be fine right?

Well, no.

Section 8.3.1 in the RFC goes on to say in the Offer-Answer Model Considerations:

The parameter [mode-set] is bi-directional, i.e., the restricted set applies to media both to be received and sent by the declaring entity. If a mode set was supplied in the offer, the answerer SHALL return the mode-set unmodified or reject the payload type. However, the answerer is free to choose a mode-set in the answer only if no mode-set was supplied in the offer for a unicast two-peer session.

And there is our problem, and why the call is getting rejected.

The Media Gateway (the answerer in this scenario) is sending back the mode-set it supports (7) but as the UE / handset (offerer) included the mode-set, the Media Gateway should either respond with the same mode set (if it supported all the requested modes) or reject it.

Instead we’re seeing the Media Gateway repond with the mode set, which it supports, which it should not do: The Media Gateway should either return the same mode-set (unmodified / unchanged) or reject it.

And boom, another ticket to another vendor…

Tales from the Trenches: The issue with Emergency Calling URNs in IMS Networks

IMS / VoLTE, Mobile Networks, RFCs & Standards, VoIP, , , , , ,

A lot of countries have a single point of contact for emergency services; in Europe you’d call 112 in an emergency, 000 in Australia or 911 in the US. Calling this number in the country will get you the emergency services.

This means a caller can order an ambulance for smoke inhalation, and the fire brigade, in one call.

But that’s not the case in every country; many countries don’t have one number for the emergency services, they’ve got multiple; a phone number for police, a different number for fire brigade and a different number for an ambulance.

For example, in Brazil if you need the police, you call 190, while a for example, uses 193 as the emergency number for the fire department, the police can be reached at 190 or 191 depending on if it’s road policing or general, and medical emergencies are covered by 192. Other countries have similar setups.

This is all well and good if you’re in Brazil, and you call 192 for an ambulance, the phone sends a SIP INVITE with a Request URI of sip:[email protected], because we can put a rule into our E-CSCF to say if the number is 192 to route it to the answer point for ambulances – But that’s not often the case on emergency calls.

In IMS, handsets generally detect the number dialed is on the Emergency Calling Code (ECC) list from the USIM Card.

The use of the ECC list means the phone knows this is an emergency call, and this is really important. For countries that use AML this can trigger sending of the AML SMS that process, and Emergency Calls should always be allowed to be made, even without credit, a valid SIM card, or even a SIM in the phone at all.

But this comes with a cost; when a user dials 911, the phones doesn’t (generally) send a call to sip:[email protected] like it would with any other dialled number, but rather the SIP INVITE is sent to urn:service:sos which will be routed to the PSAP by the E-CSCF. When a call comes through to these URNs they’re given top priority in the network

This is all well and good in a country where it doesn’t matter which emergency service you called, because all emergency calls route to a single PSAP, but in a country with multiple numbers, it’s really important when you call and ambulance, your call doesn’t get routed to animal control.

That means the phone has to look at what emergency number you’ve dialed, and map the URN it sends the call to to match what you’ve actually requested.

Recently we’ve been helping an operator in a country with a numbering plan like this, and we’ve been finding the limits of the standards here.
So let’s start by looking at what the standards state:

IMS Emergency Calling is governed by TS 103.479 which in turn delegates to IETF RFC 5031, but for the calling number to URN translation, it’s pretty quiet.

Let’s look at what RFC 5031 allows for URNs:

  • urn:service:sos.ambulance
  • urn:service:sos.animal-control
  • urn:service:sos.fire
  • urn:service:sos.gas
  • urn:service:sos.marine
  • urn:service:sos.mountain
  • urn:service:sos.physician
  • urn:service:sos.poison
  • urn:service:sos.police

The USIM’s Emergency Calling Codes EF would be the perfect source of this data; for each emergency calling code defined, you’ve got a flag to indicate what it’s for, here’s what we’ve got available on the SIM Card:

  • Bit 1 Police
  • Bit 2 Ambulance
  • Bit 3 Fire Brigade
  • Bit 4 Marine Guard
  • Bit 5 Mountain Rescue
  • Bit 6 manually initiated eCall
  • Bit 7 automatically initiated eCall
  • Bit 8 is spare and set to “0”

So these could be mapped pretty easily you’d think, so if the call is made to an Emergency Calling Code flagged with Bit 4, the URN would go to urn:service:sos.mountain.

Alas from our research, we’ve found most OEMs send calls to the generic urn:service:sos, regardless of the dialled number and the ECC flags that are set on the SIM for that number.

One of the big chip vendors sends calls to an ECC flagged as Ambulance to urn:service:sos.fire, which is totally infuriating, and we’ve had to put a rule in our E-CSCF to handle this if the User Agent is set to one of their phones.

Is there room for improvement here? For sure! Emergency calling is super important, and time is of the essence, while animal control can probably transfer you to an ambulance, an emergency is by very nature time sensitive, and any time wasted can lead to worse outcomes.

While carrier bundles from the OEMs can handle this, the global ability to take any phone, from any country and call an emergency number is so important, that relying on a country-by-country approach here won’t suffice.

What could we do as an industry to address this?

Acknowledging that not all countries have a single point of contact for emergency service, introducing a simple mechanism in the UE SIP message to indicate what number (Emergency Calling Code) the user actually dialled would be invaluable here.

URNs are important, but knowing the dialed number when it comes to PSAP routing, is so important – This wouldn’t even need to be its own SIP header, it could just be thrown into the Contact header as another parameter.

Highly developed markets are often the first to embrace new tech (for us this means VoLTE and VoNR), but this means that these issues seen by less developed markets won’t appear until long after the standard has been set in stone, and often countries like this aren’t at the table of the standards bodies to discuss such requirements.

This easy, reasonable update to the standard, has the potential to save lives, and next time this comes up in a working group I’ll be advocating for a change.