There’s a bunch of reasons you might want to re-rate CDRs in CGrateS.
For me recently I wanted to introduce StatS to process historical CDR data, and I’d messed up some rates and wanting to correct it without deleting the existing data.
We can re-rate CDRs with the *rerate flag like so:
Something to be aware of that’s tripped me up, is that if while re-rating any of the CDRs fails, CGrateS will stop rating the CDRs after it. For example if you get something like this:
Then the full list of CDRs you’ve requested to be re-rated won’t have been re-rated, only the CDRs up to the error, then CGrateS will stop processing the records after it.
So keep an eye on ngrep and make sure you’ve got all your rates and destinations defined correctly, I found putting:
To filter out any CDRs with a -1 cost in the CDR filters means I filter out any CDRs that had failed to rate last time (Assuming you don’t want to fix CDRs that have failed to get rated).
Had an interesting fault come across my desk the other day; calls were failing when the called party (an SSP we talk to via SS7/ISUP) had an exchange based call forward in place.
We’re a SIP based network, but we do talk some SS7/ISUP on the edges, and it was important that we handled this correctly.
I could see in the Address Complete Message (ACM) sent back to our network that there was redirection information here:
We would see the B party SSP release the call as soon as it sent this.
This made me wonder if we, as the originating network, were supposed to redirect to the new B party and send a new Initial Address Message?
After a lot of digging in the ITU Q.7xx docs (I’m not where near as fast at finding information in specs written prior to my birth, than I am with the 3GPP specs) I found my answer – These headers are informational only, the B party SSP is meant to re-target the message, and send us an Alerting or Answer message when it’s done so.
The StatS subsystem allows us to calculate statistics based on CGrateS events.
Each StatS object contains one or more “metrics” which are things like Average call duration, Total call duration, Average call cost or totals and average of other fields.
The first thing we’ll need to do is enable stats in our JSON config file:
Well we’ve created a StatQueueProfile named StatQueueProfile_VoiceStats, in which we’ll store a maximum of 10000000 datapoints (this is important because to calculate an average we need to know all the previous datapoints), for a maximum of forever (Because TTL is -1, if we wanted to store for 1 hour we’d set TTL to 1h.
We’re not matching any FilterIDs, but based on what we covered on the post in FilterS, you can imagine using this to match calls from a given Account / customer, or to a specific group of destinations, or maybe from a given supplier, etc, etc.
What we do have that’s interesting is we have defined a series of metrics.
So what happens if we now generate a bunch of calls? Well, for starters as we’ve got no FilterS defined here, every call will match this StatQueueProfile, and so we’ll collect data for each.
The example code I’ve provided in the repo for this post generates a bunch of calls, and we can check the values for all our Metrics with GetQueueStringMetrics for our :
If we’ve got a TTL set, old values that have existed in the QueueProfile longer than the TTL are removed, but we can also manually clear the values by using the ResetStatQueue endpoint:
One thing to keep in mind is you can’t modify a StatQueue object via the API without resetting the values.
string_indexed_fields in the config file
Sidebar on this – By specifying the string_indexed_fields means that CGrateS will not evaluate every field against Filter rules, but instead only those defined here. This means if you’ve got an event with say 20 fields (AnswerTime, Account, Subject, Destination, RunID, SetupTime, Extra Fields, etc, etc) each of these gets evaluated against a filter, which is pretty processor intensive if your FilterS only ever look at Account and Destination, so by specifying which fields are indexed here to only the fields you use in your filters, you can boost your performance. On the flip side, you can leave this blank to evaluate all fields, but you’ll take a performance hit by doing so.
FilterS do what it says in the name, they are a generic way to define filter rules that an event may or may not match.
Think of them as like a WHERE statement in SQL, they allow us to condition match.
So what would we use FilterS for? Well, let’s first checkout some example use cases:
We might want to provide 100 free minutes on Tuesdays, we know from this post on creating Balances in CGrateS how to create the balance, but we’d use FilterS to make sure the balance is only used on the Tuesday, by adding a filter to check for the day of the week to only match on a Tuesday.
We might define an Attribute to rewrite the Destination number into E.164, but we only want to apply that transformation if the number is in 0NSN format, we apply the translation with AttributeS but we would create a filter to match Destinations that match the given prefixes.
We might want to trigger a counter for calls where the duration of the call (Usage) is greater than 1 hour, we can do this with Thresholds to handle the counting and FilterS to only match if the call duration is greater than 1 hour.
A customer may have multiple DIDs / phone numbers they present as the From header, and we need a way to map phone number “99990001” through to “99990099” as the Account and change the Account to “Customer X”, we can do that with AttributeS to update the Account value in the request, and FilterS to control if that AttributeS rule is matched or not.
FilterS are used all over in CGrateS, if you’ve been following along, you’ve already come across FilterS in the FilterIDs fields in the API, which we’re going to look at using today.
There’s two ways to handle Filters inside CGrateS, they both act the same way but each have some pros and cons.
Inline Filters
The first option is an “inline” filter. Take for example this AttributeS rule using an inline Filter.
A filter is made up of 3 components, the match “type”, the element to compare using the match and the values.
Match Types: The above example is matching based on it being a string (match type *string), but we can also match on prefixes, suffixes, destinations, empty, not equal to something, greater than, less than, timings and more.
Match Elements: Next up we’ve got the element, this is what part of a CGrateS event we’re matching with the Match Type we’ve selected. In the above example we’re matching for if the value is type *string and the Element is ~*req.Account. If you look at the requests (~*req.) in CGrateS, you can see the events, there’s all the standard fields like Account, Subject, Category, Tenant, Destination, etc, plus any custom ones you’re using, all of which we can use as an element to compare with our match type.
Match Values: Lastly we’ve got the conditions we’ll match on, in the example above it’s the string “Nick” – So what we’re checking is the match is *string and the element we’re getting the string from is ~*req.Account and if that matches the value “Nick” then ding-ding-ding- we’ve matched.
Obviously the values change based on what we’re doing, if we were prefix matching, we’d put the prefix to match in the value.
Value can also be a list, separated by the pipe (|) symbol for inline filters, so for example we could match “Nick” and also “Nicholas” (if I’m in trouble) with this inline filter:
Let’s look at one more example, if the Usage is greater than 1 hour:
*gt:~*req.Usage:1h
Inline filter for any Australian E164 prefixes
FilterProfiles
Now we’ve covered the basics of creating Filters with the “Inline” method, let’s consider the limits of this.
If I had defined objects in AttributeS, ThresholdS, ResourceS, Balances and StatS to match when ~*req.Account is “Nick” using an inline filter, and then I change my name, I’d have to go to each of those elements and update them, and that’d be a pain (especially because I’d need to also change my domain name.)
Instead I can create a “Filter Profile” – A reference to a filter that I can reference from AttributeS, ThresholdS, ResourceS, Balances and StatS, and then I only need to update the Filter.
Let’s look at how that would look, first we’d create a new Filter Profile object using the API with:
This is the same as the below inline Filter, like the inline filter it’ll match any time the ~*req.Account is a string that matches “Nick” or “Nicholas”
"*string:~*req.Account:Nick|Nicholas"
And then to update our AttributeS example from earlier, rather than defining the inline filter in the FilterIDs section, we just put the ID of the filter we created above:
We saw in the example above that we could do Logical OR operations, if the Account is equal to “Nick” or “Nicholas”. But what one neat thing we can do with FilterProfiles is to do local AND Operations.
Let’s create a new FilterProfile called Filter_Sunday to match when the AnswerTime matches Timing named “Timing_Sunday”:
Now we can define an Attribute that will only match if the Account is equal to “Nick” or “Nicholas” AND the AnswerTime matches our “Timing_Sunday” timing profile:
So we can evaluate as AND by just putting both FilterProfiles in the FilterIDs field:
"FilterIDs": ["FLTR_X", "FLTR_Y"],
It’s up to you where you use Inline Filters vs Filter Profiles. As a general rule, if you don’t mind setting it on every object you’re touching, or you don’t reuse the Filter much, inline Filters is probably the way to go. But if you use multiple subsystems and want to keep your logic more readable, perhaps use Filter Profiles – but again, there’s no hard rules.
Filter Profiles is something we’ve got fairly good coverage of in the CGrateS UI, but as far as I’m aware there’s not a simple “Test Filter” API endpoint, so generally I test out with AttributeS.
Up until this point in the series, I’ve tried to hide all the complexity of CGrateS, so people following along can see some progress and feel like they’re making it somewhere with CGrateS, but it’s time to tear off the plaster and talk about the actual concepts, about what’s under the hood, and how all the components interact, as it’ll make it much easier then for us to learn more about how to use CGrateS.
This will be the last post in the “CGrateS in Baby Steps” series (Which I started in 2022), if you’ve made it this far congratulations, all the future posts will be on specific topics and build upon the concepts we’ve covered here.
This took me a while to grasp – CGrateS is both crazy complex and beautifully simple, but getting to the stage where you can “see through the matrix” on CGrateS and see the beautiful simplicity involves a bit of understanding how everything fits together.
Once you realize once you can see the pattern, and understand the building blocks, everything else CGrateS related becomes super simple.
Agents
in CGrateS Agents are consumers of the services. That’s a super generic answer, but let’s take a closer look at what that actually means with some examples:
Diameter is a protocol that can be used for Online Charging. CGrateS has a common interface for API calls that can perform Online Charging. The CGrateS Diameter Agent translates between Diameter on one side, and CGrateS API calls on the other.
Likewise, if we want to speak Radius, we can use the CGrateS Radius Agent, this translates between RADIUS and the CGrateS API calls.
FreeSWITCH, Asterisk and Kamailio don’t use specific protocols like Diameter or Radius, but rather modules or plugins to connect that application to a CGrateS Agent, and they all just end up talking the same CGrateS API calls.
Lastly, there’s even an HTTP agent so you could define your own agent to talk another protocol if you wanted to use CGrateS for anything else (We’ve been playing with CAMEL based charging with CGrateS and 5GC charging).
The config for each of the CGrateS Agents happens in the cgrates.json config file (Typically in /etc/cgrates).
Because the Agents just translate everything into API calls, logic for billing a call from FreeSWITCH is the same as for Diameter, the same as for RADIUS, the same as for SIP, the same as for Asterisk.
The Agents just translate all the domain-specific stuff into the common CGrateS RPC API, which we’ve been working with up to this point.
This is key part to understand; because once you understand how to do the CGrateS part, moving from Asterisk to FreeSWITCH, to DNS, to RADIUS, to any other Agent, it’s all the same to you.
The Agents just translate domain-specific stuff (Diameter requests, CSV files, Asterisk Calls, FreeSWITCH calls, etc, etc) and act as a translator to translate these requests into CGrateS RPC API calls.
On the left side of the image below are the Agents, and on the right side, the Subsystems that do stuff with things.
Subsystems
So with these API calls, where do they go, what do they do?
Well, it’s the Subsystems that do the things.
What things?
Well, everything of use.
Each subsystem has a purpose, AttributeS transforms stuff, EeS exports CDRs, RALs applies our charging logic, CDRs writes CDRs to StorDB, etc, etc.
In each event we can set flags to denote which subsystems it should be routed to, and we can set the links between components in our cgrates.json file.
Based on the flags, we pass events between these subsystems.
Events
So our Agents create the API calls, which contain Events, which are JSON RPC calls.
They look like all the API examples we’ve played with, because that’s exactly what they are.
We can access them via the JSON RPC API, but when you start a call on Kamailio, the Kamailio Agent generates a JSON RPC API call containing an Event into CGrateS for that call on Kamailio.
When you send a DNS request, the DNS Agent translate this DNS request into a CGrateS JSON RPC API containing the event for the DNS request.
Let’s take an example, we’re going to use the ErS as it’s the simplest to demonstrate with.
So if you setup your enviroment per the tutorial above (but don’t load the CSV yet), we’ll start running some experiments…
Anatomy of an Event
We can “sniff” the events bouncing around between the Agent and the various Subsystems in real time, by using ngrep:
sudo ngrep -t -W byline port 2012 or port 2080 or port 8021 or port 2014 or port 2053 -d any
So let’ we’ve got ngrep running, we can move our CSV file in to be processed in another tab.
Plonking the CSV file into the path ERS is monitoring will mean the ErS Agent will generate a CGrateS JSON RPC “event” for each row in the file, it’ll look something like this:
Sidebar – you’re going to spend a lot of time with `ngrep`.
Alright, that event probably looks familiar, after all, it’s the same structure as the API requests we’ve made to CGrateS so far, to set rates and handle accounts.
But what we’re witnessing here isn’t us making an API request to the JSON RPC interface from a Python script, it’s the ERS Agent inside CGrateS, calling CGrateS.
The ERS Agent inside CGrateS reads the CSV file we dropped in, and based on what we had set in the ERS section of the CGrateS config file (cgrates.json), the ERS Agent create JSON RPC events and sent it to CGrateS for processing.
You may be thinking “Wow, the ERS Agent is really dumb, it just sends an API request (events)”, and you’d be right.
We could replace the ERS Agent with a Python script to read the CSV and send the same request, and we’d get the exact same outcome, but CGrateS is mostly “batteries included” so we don’t have to.
Ok, so you’ve heard me drum in the fact that Agents are pretty simple, and all they do is make JSON RPC requests for the event which are sent to CGrateS. So now what happens?
Well, the event is calling CDRsV1.ProcessEvent, so that means the Event is passed by CGRengine to the CDRs subsystem.
What does CDRs subsystem do with it? Well, that’s going to depend on what’s in our cgrates.json config file,
In the above example, CDRs is setup with connections to the different subsystems, AttributeS, Chargers and RALs are all the subsystems linked from here.
Having these links here does not force the Event to always route to these Subsystems, but unless we’ve got the links there, the Event won’t be able to get routed from CDRs to that subsystem if we want it to.
But we can see what’s going to happen with this request based on our CDRsV1.ProcessEvent event, it’s got Flags set to rals, so we know it wants RALs to be called.
So looking in ngrep we see our CDRsV1.ProcessExternalCDR event makes it to the CDRs module with ID 1.
The API call has flags set to *rals so the CDRs will call RALs , and inside our config the CDRs section has a link in the config (shown in the image below) to RALs (rals_conns) – if we didn’t have that link, CGrateS wouldn’t know how to connect to RALs, and the event would fail.
Note at the bottom the APIOpts section tells us this API call was made by the *cdrs subsystem and the ID is 2 (This is a different request to the original CDRsV1.ProcessExternalCDR request which had ID 1 – we can use this to match responses to requests).
Again, because our config also includes links ChargerS and RALS subsystems, we’ll see requests to (you guessed it) ChargerS (The ChargerSv1.ProcessEvent) and RALS (Responder.GetCost).
# T 2024/12/22 09:09:47.465711 127.0.0.1:2012 -> 127.0.0.1:50456 [AP] #414 {"id":4,"result":{"Category":"call","Tenant":"cgrates.org","Subject":"61812341234","Account":"61412341234","Destination":"61812341234","ToR":"*voice","Cost":14,"Timespans":[{"TimeStart":"2024-01-01T01:00:00+11:00","TimeEnd":"2024-01-01T01:01:00+11:00","Cost":14,"RateInterval":{"Timing":{"ID":"*any","Years":[],"Months":[],"MonthDays":[],"WeekDays":[],"StartTime":"00:00:00","EndTime":""},"Rating":{"ConnectFee":0,"RoundingMethod":"*up","RoundingDecimals":4,"MaxCost":0,"MaxCostStrategy":"","Rates":[{"GroupIntervalStart":0,"Value":14,"RateIncrement":60000000000,"RateUnit":60000000000}]},"Weight":10},"DurationIndex":60000000000,"Increments":[{"Duration":0,"Cost":0,"BalanceInfo":{"Unit":null,"Monetary":null,"AccountID":""},"CompressFactor":1},{"Duration":60000000000,"Cost":14,"BalanceInfo":{"Unit":null,"Monetary":null,"AccountID":""},"CompressFactor":1}],"RoundIncrement":null,"MatchedSubject":"*out:cgrates.org:call:*any","MatchedPrefix":"618","MatchedDestId":"Dest_AU_Fixed","RatingPlanId":"RatingPlan_VoiceCalls","CompressFactor":1}],"RatedUsage":60000000000,"AccountSummary":null},"error":null}
What we’re seeing is the CDRs module, calling RALs, to get the cost information for this event.
Finally the CDRsV1.ProcessEvent that was initially sent by ErS gets a result (we can find the result to the request as it’ll have the same id parameter)
So that’s it, that’s the secret sauce – CGrateS is just a bunch of little APIs we combo together to create something great.
Recap
Agents translate data sources into API calls.
Each little API belongs to a Subsystem, like ChargerS, AttributeS or RALs, and we can chain them together in our config file or through the flags in the API request.
Once you’ve got your head wrapped around this, everything in CGrateS becomes way easier.
From now on I’ll pivot to talking about specific modules, and how we use them, starting with AttributeS (which I wrote last year while still drafting this), and diving into how to use each module in more detail.
I’ve been writing a fair bit recently about the “VoLTE Mess” – It’s something that’s been around for a long time, mostly impacting greenfield players rolling out LTE only, but now the big carriers are starting to feel it as they shut off their 2G and 3G networks, so I figured a brief history was in order to understand how we got here.
Note: I use the terms 4G or LTE interchangeably
The Introduction of LTE
LTE (4G) is more “spectrally efficient” than the technologies that came before it. In simple terms, 1 “chunk” of spectrum will get you more speed (capacity) on LTE than the same size chunk of spectrum would on 2G or 3G.
So imagine it’s 2008 and you’re the CTO of a mobile network operator. Your network is congested thanks to carrying more data traffic than it was ever designed for (the first iPhone had launched the year before) and the network is struggling under the weight of all this new data traffic. You have two options here, to build more cell sites for more density (very expensive) or buy more spectrum (extremely expensive) – Both options see you going cap in hand to the finance team and asking for eye-wateringly large amounts of capital for either option.
But then the answer to your prayers arrives in the form of 3GPP’s Release 8 specification with the introduction of LTE. Now by taking some 2G or 3G spectrum, and by using it on 4G, you can get ~5x more capacity from the same spectrum. So just by changing spectrum you own from 2G or 3G to 4G, you’ve got 5x more capacity. Hallelujah!
So you go to Nortel and buy a packet core, and Alcatel and Siemens provide 4G RAN (eNodeBs) which you selectively deploy on the cell sites that are the most congested. The finance team and the board are happy and your marketing team runs amok with claims of 4G data speeds. You’ve dodged the crisis, phew.
This is the path that all established mobile operators took; throw LTE at the congested cell sites, to cheaply and easily free up capacity, and as the natural hardware replacement cycle kicked in, or cell sites reached capacity, swap out the hardware to kit that supports LTE in addition to the 2G and 3G tech.
Circuit Switched Fallback
But it’s hard to talk about the machinations of late 2000s telecom executives, without at least mentioning Hitler.
This video below from 15 years ago is pretty obscure and fairly technical, but the crux of it it is that Hitler is livid because LTE does not have a “CS Domain” aka circuit switched voice (the way 2G and 3G had handled voice calls).
It was optional to include support for voice calls in the LTE network (Voice over LTE) when you launched LTE services. So if you already had a 2G or 3G network (CS Network) you could just keep using 2G and 3G for your voice calls, while getting that sweet capacity relief.
So our hypothetical CTO, strapped for cash and data capacity, just didn’t bother to support VoLTE when they launched LTE – Doing so would have taken more time to launch, during which time the capacity problem would become worse, so “don’t worry about VoLTE for now” was the mantra.
All the operators who still had 2G and 3G networks, opted to just “Fallback” to using the 2G / 3G network for calling. This is called “Circuit Switched Fallback” aka CSFB.
Operators loved this as they got the capacity relief provided by shifting to 4G/LTE (more capacity in the network is always good) and could all rant about how their network was the fastest and had 4G first, this however was what could be described as a “Foot gun” – Something you can shoot yourself in the foot with in the future.
Operators eventually introduce VoLTE
Time ticked on an operators built out their 4G networks, and many in the past 10 years or so have launched VoLTE in their own networks.
For phones that support it, in areas with blanket 4G coverage, they can use VoLTE for all their calls.
But that’s the sticking point right there – If the phones support it.
But if the phones don’t support it, they’re roaming or making emergency calls, there is always been the safety blanket of 2G or 3G and Circuit Switched fallback to well, fall back to.
There’s no driver for operators who plan to (or are required to) operate a 2G or 3G network for the foreseeable future, to ensure a high level of VoLTE support in their devices.
For an operator today with 2G or 3G, Voice over LTE is still optional. Many operators still rely exclusively on Circuit Switched Fallback, and there are only a handful of countries that have turned off 2G and 3G and rely solely on VoLTE.
VoLTE Handset Support
For the past 16 years phone manufacturers have been making LTE capable phones.
But that does not mean they’ve been making phones that support Voice over LTE.
But it’s never been an issue up until this point, as there’s always been a circuit switched (2G/3G) network to fall back to, so the fact that these chips may not support VoLTE was not a big problem.
Many of the cheaper chipsets that power phones simply don’t support VoLTE – These chips do support LTE for data connections but rely on Circuit Switched Fallback for voice calls. This is in part due to the increased complexity, but also because some of the technologies for VoLTE (like AMR) required intellectual property deals to licence to use, so would add to the component cost to manufacture, and in the chips game, keeping down component cost is critical.
Even for chips that do support Voice over LTE, it’s “special”. Unlike calling in 2G or 3G that worked the same for every operator, phone manufacturers require a “Carrier Bundle” for each operator, containing that specific operators’ special flavor of VoLTE, that operator uses in their network.
This is because while VoLTE is standardized (Despite some claims to the contrary) a lot of “optional” bits have existed, and different operators built networks with subtle differences in the “flavor” of their Voice over LTE (IMS) stack they used. The OEMs (Phone / Chip manufacturers) had to handle these changes in the devices they made, for in order to sell their phones through that operator.
This means I can have a phone from vendor X that works with VoLTE on Network Y, but does not support VoLTE on Network Z.
Worse still, knowing which phones are supported is a bit of a guessing game.
Most operators sell phones directly to their customer base, so buying an Network Y branded phone from Vendor X, you know it’s going to support Network Y’s VoLTE settings, but if you change carriers, who knows if it’ll still support it?
When you’ve still got a Circuit Switched network it’s not the end of the world, you’ll just use CSFB and probably not realize it, until operators go to shut down 2G / 3G networks…
IMS Profile selection on an engineering mode MTK based Android handset
Navigating the Maze of VoLTE Compatibility
Here are some simple checklist you can ask your elderly family members if they ask if their phone is VoLTE compatible:
Does the underlying chipset the phone is based on support VoLTE? (you can find this out by disassembling the phone and checking the datasheets for the components from the OEMs after signing NDAs for each)
Does the underlying chipset require a “carrier bundle” of settings to have been loaded for this operator in order to support VoLTE (See Qualcomm MBM as an example)?
What version of this list am I currently on (generally set in the factory) and does it support this operator? (You can check by decapping the ICs and dumping their NVRAM and then running it through a decompiler)
Does my phones OS (Android / iOS) require a “carrier bundle” of it’s own to enable VoLTE? Is my operator in the version of the database on the phone? (See Android’s Carrier Database for example) (You can find the answer by rooting the phone and running some privileged commands to poke around the internal file system)
Does my operator / MNO support VoLTE – Does my plan / package support VoLTE? (You can easily find the answer by visiting the store and asking questions that don’t appear on the script)
If you managed to answer yes to all of the above, congratulations! You have conditional VoLTE support on your phone, although you probably don’t have a working phone anymore.
Wait, conditional VoLTE support?
That’s right folks, VoLTE will work in some scenarios with your operator!
If you plan on traveling, well your phone may support VoLTE at home, but does the phone have VoLTE roaming enabled? Many phones support VoLTE in the home network, but resort to CSFB when roaming.
If it does support VoLTE roaming, does the network you’re visiting support VoLTE roaming? Has the roaming agreement (IRA) between the operator you’re using while traveling and your home operator been updated to include VoLTE Roaming? These IRAs (AA.12 / AA.13 docs) also indicate if the network must turn off IPsec encryption for the VoLTE traffic when roaming, which is controlled by the phone anyway.
Phew, all this talk of VoLTE roaming while traveling scares me, I think I’ll stay home in the safety of the Australian bush with all these great friendly animals around a phone that supports VoLTE on my home network.
Ah – After spending some time in the Australian bush one of our many deadly animals bit me. Time to call for help! Wait, what about emergency calls over VoLTE? Again, many phones support VoLTE for normal calls, fall back to 2G or 3G for the emergency call, so if you have one of those phones (You’ll only find out if you try to make an emergency call and it fails) and try to make an emergency call in a country without 2G or 3G, you’d better find a payphone.
Sarcasm aside, there’s no dataset or compatibility matrix here – No simple way to see if your phone will work for VoLTE on a given operator, even if the underlying chip does support VoLTE.
Operators in Australia which recently shut down their 3G network, were mandated to block devices that didn’t support VoLTE for emergency calling. They did this using an Equipment Identity Register, and blocking devices based on the Type Allocation Code, but this scattergun approach just blocked non-carrier issued devices, regardless of it they supported VoLTE or VoLTE emergency calling.
Blame Game
So who’s to blame here?
There’s no one group to blame here, the industry has created a shitty cycle here:
Standards orgs for having too many “flavors” available
Operators deploying their own “Flavors” of VoLTE then mandating OEMs / Chip manufacturers comply with their “flavor”.
OEMs / Chip manufactures respond by adding “Carrier Bundles” to account for this per-operator customization
I’ve got some ideas on a way to unscramble this egg, and it’s going to take a push from the industry.
If you’re in the industry and keen to push for a fix, get in touch!
It’s time to get a long term solution to this problem, and we as an industry need to lead the change.
Oh boy this has been a pain in the backside with IMS / VoLTE devices using TCP and how they handle the underlying TCP sockets.
A mobile phone from manufacturer A, wants every SIP dialog to be in it’s own TCP session, while a phone from manufacturer B wants a unique TCP session per transaction, while manufacturer C thinks that every SIP message should reuse the same transaction.
So an MT call to manufacturer A, who wants every SIP dialog in it’s own transaction would look something like this:
PCSCF:44738 -> UE:5060; TCP SYN UE:5060 -> PCSCF:44738; TCP SYN/ACK PCSCF:44738 -> UE:5060; TCP ACK --- TCP connection is now open to UE from P-CSCF--- --- Start of new SIP Transaction 1 & Dialog --- PCSCF:44738 -> UE:5060; TCP PSH - SIP INVITE.... UE:5060 -> PCSCF:44738; TCP ACK
--- Start of SIP Transaction 2 --- PCSCF:44738 -> UE:5060; TCP PSH - SIP BYE.... UE:5060 -> PCSCF:44738; TCP ACK, PSH - SIP 200.... --- End of SIP Transaction 2 & SIP Dialog --- PCSCF:44738 -> UE:5060; TCP FIN UE:5060 -> PCSCF:44738; TCP ACK --- End of TCP Connection ---
Where UE:5060 – is the IP & port of the UE, as advertised in the Contact: header, while PCSCF:44738 is the PCSCF IP and a random TCP port used for this connection.
But for manufacturer B, who wants a unique TCP session per transaction, they want it to look like this:
PCSCF:44738 -> UE:5060; TCP SYN UE:5060 -> PCSCF:44738; TCP SYN/ACK PCSCF:44738 -> UE:5060; TCP ACK --- TCP connection is now open to UE from P-CSCF--- --- Start of new SIP Transaction 1 & Dialog --- PCSCF:44738 -> UE:5060; TCP PSH - SIP INVITE.... UE:5060 -> PCSCF:44738; TCP ACK
--- Start of TCP Session 2 ---- PCSCF:32627 -> UE:5060; TCP SYN UE:5060 -> PCSCF:32627; TCP SYN/ACK PCSCF:32627 -> UE:5060; TCP ACK --- Start of SIP Transaction 2 --- PCSCF:32627 -> UE:5060; TCP PSH - SIP BYE.... UE:5060 -> PCSCF:32627; TCP ACK, PSH - SIP 200.... --- End of SIP Transaction 2 & SIP Dialog --- PCSCF:32627 -> UE:5060; TCP FIN UE:5060 -> PCSCF:32627; TCP ACK --- End of TCP Connection 2 ---
And then manufacturer C wants just the one TCP session to be used for everything, so they open the TCP connection when they register, and that’s all we use for everything.
Is there any logic to this? Nope, seems to be tied to the underlying chipset (Qualcomm vs Mediatek vs Unisoc) and the SIP stack used (Qualcomm, MTK, Unisoc, Samsung, Apple).
We’ve profiled devices into one of 3 behaviors, and then we tag them based on user agent as to what “persona” they demand from the network.
I can’t believe I’m still talking about VoLTE / IMS handset support and it’s almost 2025…. For context IMS was “standardized” 17 years ago.
And bam, now when we run Kamailio we’ll get all the logic that rtpengine exposes when being called.
By setting cfgtrace we can enable a config trace as well, and as we step through our Kamailio config file for a SIP message, we can see what’s going on at every step.
So if you’re like me, you can use the debugger module, combined with reading through the source code, to prove every singletime that the issue is with my not understanding the function properly, and never the logic of the Kamailio module!
I generally do this with Python or via the Swagger UI for the Web UI, but here’s how we can create a fixed-line IMS subscriber in PyHSS, so we can register it with a softphone, without using EAP-AKA.
Firstly we create the AuC object for this password combo.
I started seeing this error the other day when running CDRsv1.GetCDRs on the CGrateS API:
SERVER_ERROR: unexpected end of JSON input
It seemed related to certain CDRs in the cdrs table of StoreDB.
After some digging, I found the stupid simple problem:
I’d written too much data to extra_fields, leading MySQL to cut off the data mid way through, meaning it couldn’t be reconstructed as JSON by CGrateS again.
Like the rounding issue I had, this wasn’t an issue with CGrateS but with MySQL.
Quick fix:
sudo mysql cgrates -e "ALTER TABLE cdrs MODIFY extra_fields LONGTEXT;"
And new fields can exceed this length without being cut off.
After we setup CgrateS the next thing we’d generally want to do would be to rate some traffic.
Of course, that could be realtime traffic, from Diameter, Radius, Kamailio, FreeSWITCH, Asterisk or whatever your case may be, but it could just as easily be CSV files, records from a database or a text file.
We’re going to be rating CDRs from simple CSV files with the date of the event, calling party, called party, and talk time, but of course your CDR exports will have a different format, and that’s to be expected – we tailor the Event Reader Service to match the format of the files we need.
The Event Reader Service, like everything inside CgrateS, is modular. ERS is a module we load that parses files using the rules we define, and creates Events that CgrateS can process and charge for, based on the rules we define.
But before I can tell you that story, I have to tell you this story…
Nick’s imaginary CSV factory
In the repo I’ve added a DummyCSV.csv, it’s (as you might have guessed) a CSV file.
This CSV file is like a million other CSV formats out there – We’ve got a CSV file with Start Time, End Time, Customer, Talk Time, Calling Party, Called Party, Animal (for reasons) and CallID to uniquely identify this CDR.
Protip: The Rainbow CSV VScode extension makes viewing/editing/querying CSV files in VScode much easier.
Call Start Time
Row 0
Call End Time
Row 1
Customer
Row 2
Talk Time
Row 3
Calling Party
Row 4
Called Party
Row 5
Animal
Row 6
CallID
Row 7
File Format
Next we need to feed this into CGrateS, and for that we’ll be using the Event Reporter Service.
JSON config files don’t make for riveting blog posts, but you’ve made it this far, so let’s power through.
ERS is setup in CGrateS’ JSON config file, where we’ll need to define one or more readers which are the the logic we define inside CGrateS to tell it what fields are what, where to find the files we need to import, and set all the parameters for the imports.
This means if we have a CSV file type we get from one of our suppliers with CDRs in it, we’d define a reader to parse that type of file. Likewise, if we’ve got a CSV of SMS traffic out of our SMSc, we’d need to define another reader to parse the CDRs in that format – Generally we’ll do a Reader for each file type we want to parse.
So let’s define a reader for this CSV spec we’ve just defined:
"ers": {
"enabled": true,
"readers": [
{
"id": "blog_example_csv_parser",
"enabled": true,
"run_delay": "-1",
"type": "*file_csv",
"opts": {
"csvFieldSeparator":",",
"csvLazyQuotes": true,
//csvLazyQuotes Counts the row length and if does not match this value declares an error
//-1 means to look at the first row and use that as the row length
"csvRowLength": -1
},
"source_path": "/var/spool/cgrates/blog_example_csv_parser/in",
"processed_path": "/var/spool/cgrates/blog_example_csv_parser/out",
"concurrent_requests": 1024, //How many files to process at the same time
"flags": [
"*cdrs",
"*log"
],
"tenant": "cgrates.org",
"filters": [
"*string:~*req.2:Nick", //Only process CDRs where Customer column == "Nick"
],
"fields":[]
}]}
This should hopefully be relatively simple (I’ve commented it as best I can).
The ID of the ERS object is just the name of this reader – you can name it anything you like, keeping in mind we can have multiple readers defined for different file formats we may want to read, and setting the ID just helps to differentiate them.
The run_delay of -1 means ERS will run as soon as a file is moved into the source_path directory, and the type is a CSV file – Note that’s moved not copied. We’ve got to move the file, not just copy it, as CGrateS waits for the inode notify.
In the opts section we set the specifics for the CSV we’re reading, field separator if how we’re separating the values in our CSV, and in our case, we’re using commas to delineate the fields, but if you were using a file using semicolons or another delineator, you’d adjust this.
Lastly we’ve got the paths, the source path is where we’ll need to move the files to get processed into, and the processed_path is where the processed files will end up.
For now I’ve set the flags to *log and *cdrs – By calling log we’ll make our lives a bit easier for debugging, and CDRs will send the event to the CDRs module to generate a rated CDR in CGrateS, which we could then use to bill a customer, a supplier, etc, and access via the API or exporting using Event Exporter Service.
Lastly under FilterS we’re able to define the filters that should define if we should process a row or not. You don’t know how much you need this feature until you need this feature. The filter rule I’ve included will only process lines where the Customer field in the CSV (row #2) is equal to “Nick”. You could use this to also filter only calls that have been answered, only calls to off-net, etc, etc – FilterS needs a blog post all on it’s own (and if you’re reading this in the future I may have already written one).
Alright, so far so good, we’ve just defined the metadata we need to do to read the file, but now how do we actually get down to parsing the lines in the file?
Well, that’s where the data in Fields: [] comes in.
If you’ve been following along the CgrateS in baby steps series, you’ll have rated a CDR using the API, that looked something like this:
ERS is going to use the same API to rate a CDR, calling more-or-less the same API, so we’re going to set the parameters that go into this from the CSV contents inside the fields:
"fields":[
//Type of Record (Voice)
{"tag": "ToR", "path": "*cgreq.ToR", "type": "*constant", "value": "*voice"},
//Category set to "call" to match RatingProfile_VoiceCalls from our RatingProfile
{"tag": "Category", "path": "*cgreq.Category", "type": "*constant", "value": "call"},
//RequestType is *rated as we won't be deducting from an account balance
{"tag": "RequestType", "path": "*cgreq.RequestType", "type": "*constant", "value": "*rated"},
]
That’s the static values out of the way, next up we’ll define our values we pluck from the CSV. We can get the value of each row from “~*req.ColumnNumber” where ColumnNumber is the column number starting from 0.
//Unique ID for this call - We get this from the CallID field in the CSV
{"tag": "OriginID", "path": "*cgreq.OriginID", "type": "*variable","value":"~*req.7"},
//Account is the Source of the call
{"tag": "Account", "path": "*cgreq.Account", "type": "*variable", "value": "~*req.4"},
//Destination is B Party Number - We use 'Called Party Number'
{"tag": "Destination", "path": "*cgreq.Destination", "type": "*variable", "value": "~*req.5"},
{"tag": "Subject", "path": "*cgreq.Subject", "type": "*variable", "value": "~*req.5"},
//Call Setup Time (In this case, CGrateS can already process this as a datetime object)
{"tag": "SetupTime", "path": "*cgreq.SetupTime", "type": "*variable", "value": "~*req.0"},
//Usage in seconds - We use 'Call duration'
{"tag": "Usage", "path": "*cgreq.Usage", "type": "*variable", "value": "~*req.3"},
//We can include extra columns with extra data - Like this one:
{"tag": "Animal", "path": "*cgreq.Animal", "type": "*variable", "value": "~*req.6"},
]
You’ll need to restart CGrateS after putting the config changes in, but your instance will probably fail to start as we’ll need to create the directories we specified CGrateS should monitor for incoming CSV files:
But before we can put this all into play, we’ll need to setup some rates. My previous posts have covered how to do this, so for that I’ve included a Python script to setup all the rates, which you can run once you’ve restarted CGrateS.
Alright, with that out of the way, we can test it out, move our Dummy.csv file to /var/spool/cgrates/blog_example_csv_parser/in and see what happens.
I run Ubuntu on my desktop and I mess with Kamailio a lot.
Recently I was doing some work with KEMI using Python, but installing the latest versions of Kamailio from the Debian Repos for Kamailio wasn’t working.
The following packages have unmet dependencies:
kamailio-python3-modules : Depends: libpython3.11 (>= 3.11.0) but 3.11.0~rc1-1~22.04 is to be installed
Kamailio’s Python modules expect libpython3.11 or higher, but Ubuntu 22.04 repos only contain the release candidate – not the final version:
If you’re setting up Kamailio for support for WebSocket and need to bind to TCP port 80 or TCP port 443, you may run into the issue that permission is denied to bind to these ports when you try and start the service.
Having rated CDRs in CGrateS is great, but in reality, you probably want to get them into a billing system, CSV file, S3 bucket, CRM, invoice, Grafana, SQL table, etc, etc.
The Event Exporter Service (EES (previously called CDRe)) handles exporting CDRs from CGrateS.
Like everything in CGrateS, it’s highly configurable, and, again, like everything in CGrateS, supports every combination of services you can think of, plus a stack you haven’t thought of.
CDRs can be exported one of two ways, in real time, as the CDR is generated (online), or after the fact, exporting from the database containing the CDRs (offline).
Exporting in realtime (online) is a great option if you don’t want (or need) to store the CDRs in CGrateS; if you’re just using CGrateS to rate calls and spit them into a seperate system, this is a fantastic option, as it allows your CGrateS instances to remain light and not get clogged up with lots of old CDRs – That said, of course you can export the CDRs in realtime and still store them in CGrateS, that’s also a totally valid approach as well.
The more traditional approach is offline CDR export, where periodically or when an event is triggered, you scrape up a pile of CDRs and send them to your external systems.
For both options, we’ll need to define at least one exporter in our cgrates.json config file. For this example we’ll define a HTTP POST that we will trigger for realtime (online) CDR exporting, and a CSV file we dump to periodically when called from the API.
So first things first, we enable the EES module in the config:
"ees": {
"enabled": true,
"exporters": [
]
}
We’ll start with defining one exporter, named CSVExporter, that will output files to a folder named “testCSV” in the /tmp/ directory, but you can plonk these files wherever you like:
We’ve got a lot of different types of export available to us, but type *file_csv is the easiest, so that’s where we’ll start.
Setting synchronous to true will mean we’ll only run one export job at a time, but it also means we’ll get back the result via the API, which will allow us to keep track of the ID of the last record we updated, so we don’t export the same record multiple times, more on this later.
Flags allows us to, if we wanted, bounce the event through AttributeS, for example, by adding *attributes to the flags, but in this case, it’s just logging to syslog.
Of course, just enabling ees won’t actually send calls to it, we’ll need to add “ees_conns“: [“*localhost”], to “apiers”: and “cdrs” so they know to bounce the events through it:
If you’ve already got CDRs on your system from our previous tutorial, fantastic, but if not, let’s get up and running with a quick and dirty script to define some destinations, a charger, an account balance and then use some of the balance to generate a CDR:
import cgrateshttpapi
import pprint
import uuid
import datetime
now = datetime.datetime.now()
CGRateS_Obj = cgrateshttpapi.CGRateS('localhost', 2080)
#Define Destinations
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":'cgrates.org',"ID":"Dest_AU_Mobile","Prefixes":["614"]}]})
#Load TariffPlan we just defined from StorDB to DataDB
CGRateS_Obj.SendData({"method":"APIerSv1.LoadTariffPlanFromStorDb","params":[{"TPid":'cgrates.org',"DryRun":False,"Validate":True,"APIOpts":None,"Caching":None}],"id":0})
#Define default Charger
print(CGRateS_Obj.SendData({"method": "APIerSv1.SetChargerProfile","params": [{"Tenant": "cgrates.org","ID": "DEFAULT",'FilterIDs': [],'AttributeIDs' : ['*none'],'Weight': 0,}]}))
account = "Nick_Test_123"
#Add a balance to the account with type *sms with 100 sms events
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV1.SetBalance","params": [{"Tenant": "cgrates.org","Account": account,"BalanceType": "*sms","DestinationIDs": 'Dest_NZ_Mobile;Dest_AU_Mobile',"Categories": "*any","Balance": {"ID": "100_SMS_Bundle_AU_NZ_Mobile","Value": 100,"Weight": 25}}]}))
#Process CDR Event for a single SMS
pprint.pprint(CGRateS_Obj.SendData({"method": "CDRsV2.ProcessExternalCDR","params": [{"OriginID": str(uuid.uuid1()),"ToR": "*sms","RequestType": "*pseudoprepaid","AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),"SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),"Tenant": "cgrates.org","Account": account,"Destination" : "61412345678","Usage": "1",}]}))
Right, with that out of the way, we should now have something in our CDRs table, a quick SQL query confirms this is the case:
So, as you may have guessed, we’ve called the ExportCDRs API endpoint, we’ve specified which ExporterIDs we want to reference (these link back to the objects in the config, and the one we have defined currently is named CSVExporter).
Setting Verbose: True means that CGrateS gives us back a lot of info from the API call, here’s what we get back:
Now that looks pretty positive, we got 12 events of SMS usage exported, which we can see in the file /tmp/testCSV/CSVExporter_21e9bc2.csv – and if we cat out the file, yeap, there’s all the CDRs.
But it’s a bit of a mess, there’s a lot of fields in there, so let’s adjust what goes into the CSV.
Let’s start by filtering what goes into the exporter, to only give us SMS events, of course you could adjust the filters here to target exporting only the records you want, based on anything you can define with Filters (and there’s a lot you can define with filters).
Now we’re only exporting SMS records, so let’s clean up the output of the CSV to just give us the data we want, which is the CDR ID, time, account, destination and usage.
Now after a restart of CGrateS, our exports look like this:
Stunning, truly beautiful, look at that output!
Right, well you may at this point have noticed a problem if you’ve run this more than once. The problem is that is every time we run this, we get all the CDRs since the beginning of time.
But where filtering by date/time falls down, is that if an offline CDR of a call on Monday, only got ingested on Tuesday, it would be missed by the export.
But, setting Verbose: True on the ExportCDRs API call gives us a handy trick, we’ve been told what the highest ID in the CDRs table we just exported in the response from the API in LastExpOrderID field.
If we jump over to the SQL database we use for StorDB, we can see that 33 is the ID of the highest CDR in the system.
So let’s try something, let’s run the exporter again, but this time let’s get all the CDRs where the ID is higher than 33:
#Process CDR Event for a single SMS
pprint.pprint(CGRateS_Obj.SendData({"method": "CDRsV2.ProcessExternalCDR","params": [{"OriginID": str(uuid.uuid1()),"ToR": "*sms","RequestType": "*pseudoprepaid","AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),"SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),"Tenant": "cgrates.org","Account": account,"Destination" : "61412345678","Usage": "1",}]}))
#Trigger export where the OrderID is above 33
result = CGRateS_Obj.SendData({"method":"APIerSv1.ExportCDRs","params":[
{"ExporterIDs": ["CSVExporter"],
"Verbose" : True,
"ExtraArgs" : {
"OrderIDStart" : int(33),
},
"Accounts" : [account]}
]})
pprint.pprint(result)
Boom, now if we have a look at the output we can see the export covered two records, and the last ID was 35.
So as long as we keep track of the LastExpOrderID value, and feed that as in input every time we run ExportCDRs, we can ensure we never miss a CDR, and never get the same CDR twice.
I got an email the other day asking a simple question:
How do I know if a subscriber is VoLTE roaming or not when they send an SMS to charge for it?
My immediate reaction was to look at the SIP headers, P-Access-Network-Info will tell you where the subscriber is located, end of.
Right?
Well not quite, this will tell the SMSc the location of the subscriber sending the SMS. If the PLMN in the P-Access-Network-Info != the home PLMN, the sub is roaming.
But does this information get passed to the OCS / OFCS?
The SMSc uses “Event based charging” to perform credit control, so let’s have a look at what AVPs are present in the Credit Control Request from the SMSc:
Hmm, the SMS-Information AVP (2000) contains a bunch of information about the SMS being sent, but I don’t see anything about the location of the sender in there.
Originator-Interface is just set to “SIP”, of course in a 2G/3G roaming scenario the Originator-SCCP-Address would be that of the Visited PLMN, but for us it is our SCCP address.
Maybe the standard allows for an additional optional AVP in the SMS-Information-AVP we’re missing? Let’s check TS 32.299:
Nope.
So how to deal with this?
While the standards aren’t totally clear on this, we added an IMS-Info AVP and inside that populated the Access-Network-Information directly from the SIP header, and then picked that off inside our OCS in order to apply the correct rules.
If you’re used to dealing with SIP, you’d expect to see:
Content-Type: application/sdp
This Content-Type multipart/mixed;boundary is totally valid in SIP, in fact RFC 5261 (Message Body Handling in the Session Initiation Protocol (SIP)) details the use of MIME in SIP, and the Geolocation extension uses this, as we see below from a 911 call example.
But while this extension is standardised, and having your SIP Body containing multipart MIME is legal, not everything supports this, including the FreeSWITCH bridge module, which just appends a new SDP body into the Mime Multipart
Site note: I noticed FreeSWITCH Bridge function just appends the new SIP body in the multipart MIME, leaving the original, SDP:
Okay, so how do we replace the MIME Multipart SIP body with a standard SDP?
#If the body is multipart then strip it and replace with a single part
if (has_body("multipart/mixed")) {
xlog("This has a multipart body");
if (filter_body("application/sdp")) {
remove_hf("Content-Type");
append_hf("Content-Type: application/sdp\r\n");
} else {
xlog("Body part application/sdp not found\n");
}
}
The docs describe AttributeS as a Key-Value-Store, but that’s probably selling it short – You can do some really cool stuff with AttributeS, and in this post, we’re going to learn about using AttributeS to transform stuff.
Note: Before we get started, I’d suggest copying this config file to use for testing.
Let’s look at a really basic example, where we add some data into AttributeS, match based on Account in CGrateS, and get back that data.
Well, for starters we’re calling the SetAttributeProfile endpoint, this is where we go to create / update Attribute Profiles, but in this case, because we’re hitting it for the first time with this ID, we’re creating a new entry called “ATTR_Nick_Key_Value_Example“, this will match any Contexts (more on them later) where the FilterIDs is a string, where the request Account, is equal to 1234.
Let’s run this against the CGrateS API and take a look at the result:
This tells us we matched the Attribute with the ID ATTR_Nick_Key_Value_Example, and inside Event we can see that ExampleKey was added with value ExampleValue.
Okay, you’re saying, well what was the point of that?
Well, what if as a key in the attributes, we had the password for the SIP account, which we passed to our SIP switch (Kamailio, FreeSWITCH or Asterisk for example), and used that to authenticate?
Now if the CGrateS Agent for your SIP Switch, includes the *attributes flag, and the call is coming from 1234, we’ll get back a key called “SIP_password” with the value “sosecretiputitonthewebsite”, which you can use to auth the SIP account.
We can also return multiple AttributeS, for example, we created two Attributes (ATTR_Nick_Password_Example and ATTR_Nick_Key_Value_Example) which match on the account 1234. This means we’ll get back the SIP Password from ATTR_Nick_Password_Example and the key:value we set in ATTR_Nick_Key_Value_Example:
The order can be controlled by the Weight flag in the attribute, and if you want to stop matching any other AttributeS rules after the current Attribute, you can set the Blocker=True flag when you create/update the Attribute.
Okay, I hear you saying, that’s all well and good, I can add arbitrary key/values to stuff. Here endeth the lesson right?
Well not quite, because we can add key/values, but we can also rewrite variables using AttributeS.
Let’s imagine we’ve got 3 phone numbers (DIDs) associated with an account inside CGrateS, for example’s sake let’s say we have 12340001, 12340002 and 12340003, and we want any calls from these numbers to be billed to a CGrateS account called “NickTest1234”.
Our SIP switch doesn’t need to know anything about “NickTest1234”, just the 3 DIDs it can use to call out from your SIP stack. But to do this, we’d need CGrateS to transform any events from these DIDs to replace the Account value inside CGrateS, with NickTest1234.
In the example code to go with this I’ve put together a simple for loop to add these – You can find the code on Github (link at the bottom).
So with these defined, let’s try and rate something, we’ll add a default Charger, and add an SMS balance, before simulating an SMS where the account is set to 12340003:
#Define default Charger
print(CGRateS_Obj.SendData({"method":"APIerSv1.SetChargerProfile","params":[{"Tenant":"cgrates.org","ID":"DEFAULT","FilterIDs":[],"AttributeIDs":["*none"],"Weight":0}]}))
#Add an SMS Balance
print(CGRateS_Obj.SendData({"method":"ApierV1.SetBalance","params":[{"Tenant":"cgrates.org","Account":"Nick_Test_123","BalanceType":"*sms","Categories":"*any","Balance":{"ID":"SMS_Balance_1","Value":"100","Weight":25}}],"id":13}))
import uuid
import datetime
now = datetime.datetime.now()
result = CGRateS_Obj.SendData({
"method": "CDRsV2.ProcessExternalCDR",
"params": [
{
"OriginID": str(uuid.uuid1()),
"ToR": "*sms",
"RequestType": "*pseudoprepaid",
"AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
"SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
"Tenant": "cgrates.org",
#This is going to be transformed to Nick_Test_123 by Attributes
"Account": "12340003",
"Usage": "1",
}
]
})
pprint.pprint(result)
Right, so all going well, here’s what you should see in the CDRs table:
Bingo! Despite the fact the Account in the ProcessExternalCDR was set to 12340003, and had no mention of “NickTest1234”, CGrateS transformed it to NickTest1234.
How did that happen? Well, inside our cgrates.json file we have set the cdrs and chargers modules to have a link to Attributes, which means that when we call CDRs or Chargers modules via the API, these will in turn bounce the data through AttributesS for any transformations.
This means we don’t need to run AttributeSv1.ProcessEvent ourselves, when we call CDRsV2.ProcessExternalCDR, the CDRs module will call AttributeSv1.ProcessEvent for us.
We can actually see this happening, using ngrep, which as you work more with CGrateS, is a tool you’ll get very familiar with, let’s take a peek:
sudo ngrep -t -W byline port 2012 -d lo
Now if we run the CDRsV2.ProcessExternalCDR again, we’ll see the CDRs module has called Attributes for us:
Boom, there it is, same as we ran, but it’s being handled by CGrateS for us.
If you look carefully you’ll see the context in the API request is set to “*cdrs”, this means the CDRs module is calling Attributes.
When we define each of our Attributes, as we did earlier in the post, we can set what contexts they are valid in, for example we may want to apply the transformation when called by CDRs, but not other modules, you can restrict that when you define the Attribute by setting “Contexts”: [“*cdrs”].
Okay, so we’ve done some account replacement, what else can we do?
Well, let’s look at some other use cases,
Here in Australia we’ve got a few valid dialing formats, you could dial E.164 format (Numbers look like: +61212341234), 0NSN format (Numbers look like: 02 1234 1234) or NSN format (Numbers look like: 1234 1234 assuming you’re in the 03 area code yourself). If we want to define all our Destinations in E.164 format, we’ll need to to normalise the format using AttributeS, so the numbers always come as E.164.
And then under AttributeS we’ve defined a rule to replace anything matching the 0NSN regex, to strip the first digit and append a 61, to put it in E.164 format, and in SN format as the second entry.
And there you have it folks; our number format standardized.
We can combo / cascade AttributeS rules together, with the aid of the Weight and Blocker flags in the API.
Let’s imagine the 61212341234 number has been ported from Operator1 to Operator2, and the Destinations we’ve defined in CGrateS for this prefix are currently set to DST_Operator1. But because this number has been ported we should use DST_Operator2, so we charge the Operator2, as this number has been ported.
This means we don’t need to duplicate destination definitions to show this number has been ported, as this will be updated as the call gets rated, so we just assign the Attribute to each ported number.
So let’s match where the Subject of the call is 61212341234 (even though we’re going to input the Subject as 12341234), and rewrite the Destination attribute to DST_Operator2:
From the results we can see we matched two AttributeS rules, the first, ATTR_0NSN_to_E164_02_Area_Code reformatted the Subject of the call from 12341234 to 61212341234, then the updated Subject was passed through to ATTR_Ported_61212341234, which updated the Destination attribute to DST_Operator2.
If you’re having issues, make sure you have loaded the config file, are running the latest version, and if in doubt (and not on a production system), this script will clear all the data for you so you can rule out anything interfering.
Android, being open source, allows us to see how this logic works, and it’s important for operators to understand this logic, as it’s what dictates the behavior in many scenarios.
It’s important to note that I’m not covering Apple here, this information is not publicly available to share for iOS devices, so I won’t be sharing anything on this – Apple has their own ecosystem to handle emergency calling, if you’re from an operator and reading this, I’d suggest getting in touch with your Apple account manager to discuss it, they’re always great to work with.
The Android Open Source Project has an “emergency number database”. This database has each of the emergency phone numbers and the corresponding service, for each country.
This file can be read at packages/services/Telephony/ecc/input/eccdata.txt on a phone with engineering mode.
Let’s take a look what’s in mainline Android for Australia:
