Category Archives: Mobile Networks

CGrateS – ActionTriggers

In our last post we looked at Actions and ActionPlans, and one of the really funky things we can do is setting ActionPlans to trigger on a time schedule or setting ActionTriggers to trigger on an event.

We’re going to build on the examples we had on the last post, so we’ll assume your code is up to the point where we’ve added a Signup Bonus to an account, using an ActionPlan we assigned when creating the account.

In this post, we’re going to create an action that charges $6, called “Action_Monthly_Charge“, and tie it to an ActionPlan called “ActionPlan_Monthly_Charge“, but to demo how this works rather than charging this Monthly, we’re going to charge it every minute.

Then with our balances ticking down, we’ll set up an ActionTrigger to trigger when the balance drops below $95, and alert us.

Defining the Monthly Charge Action

The Action for the Monthly charge will look much like the other actions we’ve defined, except the Identifier is *debit so we know we’re deducting from the balance, and we’ll log to the CDRs table too:

# Action to add a Monthly charge of $6
Action_Monthly_Charge = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Monthly_Charge",
          "Actions": [
              {
                'Identifier': '*debit',
                'BalanceType': '*monetary',
               'Units': 6,
               'Id': 'Action_Monthly_Charge_Debit',
               'Weight': 70},
              {
                  "Identifier": "*log",
                  "Weight": 60,
                  'Id' : "Action_Monthly_Charge_Log"
              },
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Recurring Charge\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Monthly_Charge))

Next we’ll need to wrap this up into an ActionPlan, this is where some of the magic happens. Inside the action plan we can set a once off time, or a recurring time, kinda like Cron.

We’re setting the time to *every_minute so things will happen quickly while we watch, this action will get triggered every 60 seconds. In real life of course, for a Monthly charge, we’d want to trigger this Action monthly, so we’d set this value to *monthly. If we wanted this to charge on the 2nd of the month we’d set the MonthDays to “2”, etc, etc.

# # Create ActionPlan using SetActionPlan to trigger the Action_Monthly_Charge
SetActionPlan_Daily_Action_Monthly_Charge_JSON = {
    "method": "ApierV1.SetActionPlan",
    "params": [{
        "Id": "ActionPlan_Monthly_Charge",
        "ActionPlan": [{
            "ActionsId": "Action_Monthly_Charge",
            "Years": "*any",
            "Months": "*any",
            "MonthDays": "*any",
            "WeekDays": "*any",
            "Time": "*every_minute",
            "Weight": 10
        }],
        "Overwrite": True,
        "ReloadScheduler": True
    }]
}
pprint.pprint(CGRateS_Obj.SendData(
    SetActionPlan_Daily_Action_Monthly_Charge_JSON))

Alright, but now what’s going to happen?

If you think the accounts will start getting debited every 60 seconds after applying this, you’d be wrong, we need to associate this ActionPlan with an Account first, this is how we control which accounts get which ActionPlans tied to them, to do this we’ll use the SetAccout API again we’ve been using to create accounts:

# Create the Account object inside CGrateS & assign ActionPlan_Signup_Bonus and ActionPlan_Monthly_Charge
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionPlanIds": ["ActionPlan_Signup_Bonus", "ActionPlan_Monthly_Charge"],
            "ActionPlansOverwrite": True,
            "ReloadScheduler":True
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

So what’s going to happen if we run this?

Well, for starters the ActionPlan named “ActionPlan_Signup_Bonus” is going to be triggered, as in the ActionPlan it’s Timing is set to *asap, so CGrateS will apply the corresponding Action (“Action_Add_Signup_Bonus“) right away, which will credit the account $99.

But a minute after that, we’ll trigger the ActionPlan named “ActionPlan_Monthly_Charge”, as the timing for this is set to *every_minute, when the Action “Action_Monthly_Charge” is triggered, it’s going to be deducting $6 from the balance.

We can check this by using the GetAccount API:

# Get Account Info
pprint.pprint(CGRateS_Obj.SendData({'method': 'ApierV2.GetAccount', 'params': [
              {"Tenant": "cgrates.org", "Account": str(Account)}]}))

You should see a balance of $99 to start with, and then after 60 seconds, it should be down to $93, and so on.

{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-17T14:57:20.71493633+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '3a896369-8107-4e32-bcef-2d078c981b8a',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-17T14:57:21.802521707+11:00'}}

Triggering Actions based on Balances with ActionTriggers

Okay, so we’ve set up recurring charges, now let’s get notified if the balance drops below $95, we’ll start, like we have before, with defining an Action, this will log to the CDRs table, HTTP post and write to syslog:


#Define a new Action to send an HTTP POST
Action_HTTP_Notify_95 = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_HTTP_Notify_95",
          "Actions": [
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Balance dipped below $95\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
              {
                  "Identifier": "*http_post_async",
                  "ExtraParameters": "http://10.177.2.135/95_remaining",
                  "ExpiryTime": "*unlimited",
                  "Weight": 700
              },
              {
                  "Identifier": "*log",
                  "Weight": 1200
              }
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_HTTP_Notify_95))

Now we’ll define an ActionTrigger to check if the balance is below $95 and trigger our newly created Action (“Action_HTTP_Notify_95“) when that condition is met:


#Define ActionTrigger
ActionTrigger_95_Remaining_JSON = {
    "method": "APIerSv1.SetActionTrigger",
    "params": [
        {
            "GroupID" : "ActionTrigger_95_Remaining",
            "ActionTrigger": 
                {
                    "BalanceType": "*monetary",
                    "Balance" : {
                        'BalanceType': '*monetary',
                        'ID' : "*default",
                        'BalanceID' : "*default",
                        'Value' : 95,
                        },
                    "ThresholdType": "*min_balance",
                    "ThresholdValue": 95,
                    "Weight": 10,
                    "ActionsID" : "Action_HTTP_Notify_95",
                },
            "Overwrite": True
        }
    ]
}
pprint.pprint(CGRateS_Obj.SendData(ActionTrigger_95_Remaining_JSON))

We’ve defined the ThresholdType of *min_balance, but we could equally set this to ThresholdType to *max_balance, *balance_expired or trigger when a certain Counter has been triggered enough times.

Adding an ActionTrigger to an Account

Again, like the ActionPlan we created before, before the ActionTrigger we just created will be used, we need to associate it with an Account, for this we’ll use the AddAccountActionTriggers API, specify the Account and the ActionTriggerID for the ActionTrigger we just created.


#Add ActionTrigger to Account 
Add_ActionTrigger_to_Account_JSON = {
    "method": "APIerSv1.AddAccountActionTriggers",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionTriggerIDs": ["ActionTrigger_95_Remaining"],
            "ActionTriggersOverwrite": True
        }
    ]
}
pprint.pprint(CGRateS_Obj.SendData(Add_ActionTrigger_to_Account_JSON))

If we run this all together, creating the account with the “ActionPlan_Signup_Bonus” will give the account a $99 Balance. But after 60 seconds, “ActionPlan_Monthly_Charge” will kick in, and every 60 seconds after that, at which point the balance will get to below $95 when CGrateS will trigger the ActionTriggerActionTrigger_95_Remaining” and get the HTTP POST to the HTTP endpoint and log entry:

We can check on this using the ApierV2.GetAccount method, where we’ll see the ActionTrigger we just defined.

Checking out the LastExecutionTime we can see if the ActionTrigger been triggered or not.

So using this technique, we can notify a customer when they’ve used a certain amount of their balance, but we can lock out Accounts who have spent more than their allocated spend limit by setting an Action that suspends the Account once it reaches a certain level. We notify customers when balance expires, or if a certain number of counters has been triggered.

As always I’ve put all the code used in this example, from start to finish, up on GitHub.

CGrateS – Actions & Action Plans

In our last post we added a series of different balances to an account, these were actions we took via the API specifically to add a balance.

But there’s a lot more actions we may want to do beyond just adding balance.

CGrateS has the concept of “Actions” which are, as the name suggests, things we want to do to the system.

Some example Actions would be:

  • Adding / Deducting / Resetting a balance
  • Adding a CDR log
  • Enable/Disable an account
  • Sending HTTP POST request or email notification
  • Deleting / suspending account
  • Transferring balances

We can run these actions on a timed basis, or when an event is triggered, and group Actions together to run multiple actions via an ActionTrigger, this means we can trigger these Actions, not just by sending an API request, but based on the state of the subscriber / account.

Let’s look at some examples,

We can define an Action named “Action_Monthly_Fee” to debit $12 from the monetary balance of an account, and add a CDR with the name “Monthly Account Fee” when it does so.
We can use ActionTriggers to run this every month on the account automatically.

We can define an Action named “Usage_Warning_10GB” to send an email to the Account owner to inform them they’ve used 10GB of usage, and use ActionTriggers to send this when the customer has used 10GB of their *data balance.

Using Actions

Note: The Python script I’ve used with all the examples in this post is available on GitHub here.

Let’s start by defining an Account, just as we have before:

# Create the Account object inside CGrateS
Account = "Nick_Test_123"
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account)
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Let’s start basic; to sweeten the deal for new Accounts, we’ll give them $99 of balance to use in the first month they have the service. Rather than hitting the AddBalance API, we’ll define an Action named “Action_Add_Signup_Bonus” to credit $99 of monetary balance to an account.

If you go back to our last post, you should know what we’d need to do to add this balance manually with the AddBalance API, but let’s look at how we can create the same balance add functionality using Actions:

#Add a Signup Bonus of $99 to the account with type *monetary expiring a month after it's added
Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 99,
                  "ExpiryTime": "*month",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 1200,
                  "ExtraParameters": "",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 10
              }
]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Alright, this should look pretty familiar if you’ve just come from Account Balances.
You’ll notice we’re no longer calling, SetBalance, we’re now calling SetActions, to create the ActionsId with the name “Action_Add_Signup_Bonus“.
In “Action_Add_Signup_Bonus” we’ve got an actions we’ll do when “Action_Add_Signup_Bonus” is called.
We can define multiple actions, but for now we’ve only got one action defined, which has the Identifier (which defines what the action does) set to *topup to add balance.
As you probably guessed, we’re triggering a top up, and setting the BalanceId, BalanceType, Units, ExpiryTime and BalanceWeight just as we would using SetBalance to add a balance.

So how do we use the Action we just created? Well, there’s a lot of options, but let’s start with the most basic – Via the API:

# Trigger ExecuteAction
Account_Action_trigger_JSON = {"method": "APIerSv1.ExecuteAction", "params": [
    {"Tenant": "cgrates.org", "Account": str(Account), "ActionsId": "Action_Add_Signup_Bonus"}]}
pprint.pprint(CGRateS_Obj.SendData(Account_Action_trigger_JSON))

Boom, we’ve called the ExecuteAction API call, to execute the Action named “Action_Add_Signup_Bonus“.

We can check on this with GetAccount again and check the results:

# Get Account Info
pprint.pprint(CGRateS_Obj.SendData({'method': 'ApierV2.GetAccount', 'params': [
              {"Tenant": "cgrates.org", "Account": str(Account)}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-15T10:27:52.865119544+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '01cfb471-ba38-453a-b0e2-8ddb397dfe9c',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-15T10:27:52.865144268+11:00'}}

Great start!

Making Actions Useful

Well congratulations, we took something we previously did with one API call (SetBalance), and we did it with two (SetAction and ExcecuteAction)!

But let’s start paying efficiency dividends,

When we add a balance, let’s also add a CDR log event so we’ll know the account was credited with the balance when we call the GetCDRs API call.

We’d just modify our SetActions to include an extra step:

Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
...
              }, 
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Your sign up Bonus\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 10
              }
]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Boom, now we’ll get a CDR created when the Action is triggered.

But let’s push this a bit more and add some more steps in the Action:

As well as adding balance and putting in a CDR to record what we did, let’s also send a notification to our customer via an HTTP API (BYO customer push notification system) and log to Syslog what’s going on.

# Add a Signup Bonus of $99 to the account with type *monetary expiring a month after it's added
Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 99,
                  "ExpiryTime": "*month",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 1200,
                  "ExtraParameters": "",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 90
              },
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Your sign up Bonus\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
              {
                  "Identifier": "*http_post_async",
                  "ExtraParameters": "http://10.177.2.135/example_endpoint",
                  "ExpiryTime": "*unlimited",
                  "Weight": 70
              },
              {
                  "Identifier": "*log",
                  "Weight": 60
              }
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Phew! That’s a big action, but if we execute the action again using ExecuteAction, we’ll get all these things happening at once:

Okay, now we’re getting somewhere!

ActionPlans

Having an Action we can trigger manually via the API is one thing, but being able to trigger it automatically is where it really comes into its own.

Let’s define an ActionPlan, that is going to call our Action named “Action_Add_Signup_Bonus” as soon as the ActionPlan is assigned to an Account.

# Create ActionPlan using SetActionPlan to trigger the Action_Signup_Bonus ASAP
SetActionPlan_Signup_Bonus_JSON = {
    "method": "ApierV1.SetActionPlan",
    "params": [{
        "Id": "ActionPlan_Signup_Bonus",
        "ActionPlan": [{
            "ActionsId": "Action_Add_Signup_Bonus",
            "Years": "*any",
            "Months": "*any",
            "MonthDays": "*any",
            "WeekDays": "*any",
            "Time": "*asap",
            "Weight": 10
        }],
        "Overwrite": True,
        "ReloadScheduler": True
    }]
}
pprint.pprint(CGRateS_Obj.SendData(SetActionPlan_Signup_Bonus_JSON))

So what have we done here? We’ve made an ActionPlan named “Action_Add_Signup_Bonus”, which, when associated with an account, will run the Action “Action_Add_Signup_Bonus” as soon as it’s tied to the account, thanks to the Time*asap“.

Now if we create or update an Account using the SetAccount method, we can set the ActionPlanIds to reference our “ActionPlan_Signup_Bonus” and it’ll be triggered straight away.

# Create the Account object inside CGrateS
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionPlanIds": ["ActionPlan_Signup_Bonus"],
            "ActionPlansOverwrite": True,
            "ReloadScheduler":True
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Now if we were to run a GetAccount API call, we’ll see the Account balance assigned that was created by the action Action_Add_Signup_Bonus which was triggered by ActionPlan assigned to the account:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-16T12:41:02.530985381+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '7bdbee5c-0888-4da2-b42f-5d6b8966ee2d',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-16T12:41:12.7236096+11:00'}}

But here’s where it gets interesting, in the ActionPlan we just defined the Time was set to “*asap“, which means the Action is triggered as soon as it was assigned to the account, but if we set the Time value to “*monthly“, the Action would get triggered every month, or *every_minute to trigger every minute, or *month_end to trigger at the end of every month.

Code for these examples is available here.

I’m trying to keep these posts shorter as there’s a lot to cover. Stick around for our next post, we’ll look at some more ActionTriggers to keep decreasing the balance of the account, and setting up ActionTriggers to send a notification to the customer to tell them when their balance is getting low, or any other event based Action you can think of!

Shiny things inside Cellular Diplexers

I recently ended up with a few Commscope RF combiners from a cell site, they’re not on frequencies that are of any use to us, so, let’s see what’s inside.

The units on the bench are Commscope Diplexer units, these ones allow you to put a signal between 694-862Mhz, and another signal between 880-960Mhz, on the same RF feeder up the tower.

It’s a nifty trick from the days where radio units lived at the bottom of the tower, but now with Remote Radio Units, and Active Antenna Units, it’s becoming increasingly uncommon to have radio units in the site hut, and more common to just run DC & fibre up the tower and power a radio unit right next to the antenna – This is especially important for higher frequencies where of course the feeder loss is greater.

Diplexer unit before it is maimed…

Anywho, that’s about all I know of them, after the liberal application of chemicals to remove the stickers and several burns from a heat gun, we started to get the unit open, to show the zillion adjustment bolts, and finely machined parts.

Thanks to Oliver for offering up the bench space when I rocked to up to their house with some stuff to pull apart.

CGrateS – Accounts & Balances

So far we’ve used CGrateS to rate a basic CDR and get a cost for it, but in the real world, we’d usually associate the cost with an account, which would represent a business or a person, who will ultimately be charged for using the service.

Note: I’ve put the code for all this in Github, if you’ve got issues following along, or don’t want to copy and paste the code from the website, you can grab the code here.

Creating an Account

Let’s start off by creating an account inside CGrateS – This is kinda pointless, but we’ll talk more about that later:

#Create the Account object inside CGrateS 
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123"
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Running this onto the API should create an account named “Nick_Test_123”, but let’s confirm that’s the case:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))

Running this will give us the information about the account we just created:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
OrderedDict([('id', None),
             ('result',
              OrderedDict([('ID', 'cgrates.org:Nick_Test_123'),
                           ('BalanceMap', None),
                           ('UnitCounters', None),
                           ('ActionTriggers', None),
                           ('AllowNegative', False),
                           ('Disabled', False),
                           ('UpdateTime',
                            '2023-10-09T16:53:37.524466041+11:00')])),
             ('error', None)])

That was easy!

There’s not really much to see on our account at this stage, other than the UpdateTime, there’s nothing really going on, we don’t have any Balances.

Adding Balance for Voice

Accounts exist for spending, so let’s add a balance to this account to send from.

We’ll use the SetBalance API to create a new balance with 5 minutes of talk time, that we can use for making a call, and talking, for (you guessed it) – 5 minutes, so and we’ll use the balance “5_minute_voice_balance” that we’ll create:

#Add a balance to the account with type *voice with 5 minutes of Talk Time
Create_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "5_minute_voice_balance",
                "Value": "5m",
                "Weight": 25
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Voice_Balance_JSON))

Now if we run the GetAccount API command again, we should see the new balance we just created:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': '37423d07-d99a-40b1-851a-981c3df02cb3',
                                       'Value': 300000000000,
                                       'Weight': 25}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T17:58:23.801531205+11:00'}}

So now we’ve got a new balance named ‘5_minute_voice_balance‘:

  • The type is *voice, because this balance is storing talk time
  • The weight of this balance is 25, this means this balance should take priority over any balances with a lower value than 25 (that’s right, we can (and will) do tiered balances)
  • The value is 300000000000 nanoseconds, which equates to 5 minutes (yes, that’s the correct number of zeros)

Okay, but Nick_Test_123 probably wants to make some calls, so let’s generate a 2.5 minute call event and check out what happens.


#Generate a new call event for a 2.5 minute (150 second) call
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "150s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

Alright, now we’ve got a call event, let’s call the GetAccount API again to check the balance:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': '37423d07-d99a-40b1-851a-981c3df02cb3',
                                       'Value': 150000000000,
                                       'Weight': 25}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T17:58:23.80925546+11:00'}}

And there you have it, we’ve used 150 seconds of our 300 second (5 minutes) of talk time in this balance, leaving with us 150000000000 nanoseconds (150 seconds) remaining!

And with that progress, now is a great time to pause and talk about some theory that’s really important to grasp!

Balance Types

But *voice is just one balance type – We can support multiple balance types; we’ve just given a balance of *voice for talk time, but we could also give a credit to the balance for *sms or *data, you name it (*generic) and cash (*monetary) and we can have multiple separate balances for each.

This means we can have one account with something like:

  • 100 minutes of Local / National Calls (Expires at the end of the month)
  • 40 minutes of Mobile Calls (Expires 24 hours after it’s been created)
  • 80 minutes of Mobile Calls (During “Happy Hour” from 6 to 7pm)
  • 50 minutes of International Calls (Expires in 30 days)

And not just voice balance, the same account could also have:

  • 1GB of Data usage
  • 50 SMS to on-net destinations
  • $200 of Cash (expiring never)

Phew! That’s a lot of balances, but we can do it all through CGrateS!

What Balance to Use

So if we’ve got a stack of balances defined, how does CGrateS know what balance to use?

Firstly CGrateS is going to evaluate the BalanceType, this is set on events, so if we get an event for *data CGrateS will check out the balances available for *data, and evaluate the balances by Weight, with the highest weight evaluated first.
If we get to the end of all the available balances for that BalanceType, CGrateS then evaluates *generic and then *monetary balances, again, ordered by Weight.

We can set what balance gets used based on the Destination; using DestinationIDs we can filter the Balance to only apply for calls to Local/National numbers, so a call to an International destination won’t use that balance.

We can also set an Expiry on the Balance, for example we can give a customer 30 days to use the balance, after which it expires and can’t be used, likewise we can set Timings so enable scenarios like a “Happy hour” with extra calls between 6pm and 7pm.

When we define a balance we can also set the Blocker flag to True, if this is set, it means CGrateS will not look evaluate any balances after reaching that balance.

Adding a Balance for Local / National & Mobile Calls

Let’s jump back into the practice, and define two new Balances; one for Local/National calls, and another for Mobile calls.
But first we’ll need to know what destinations are mobiles and what are local/national (fixed). We’ve covered setting Destinations previously, so let’s set up the Destinations:


CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_Fixed","Prefixes":["612", "613", "617", "618"]}]})
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_Mobile","Prefixes":["614"]}]})
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_TollFree","Prefixes":["6113", "6118"]}]})
#Load TariffPlan we just defiend from StorDB to DataDB
print(CGRateS_Obj.SendData({"method":"APIerSv1.LoadTariffPlanFromStorDb","params":[{"TPid":"cgrates.org","DryRun":False,"Validate":True,"APIOpts":None,"Caching":None}],"id":0}))

Alright, now let’s add a balance for our local/national (fixed) calls.

To do this, we’ll add two new balances, but we’ll need to differentiate this from the 5_minute_voice_balance we created earlier, and to achieve this we weill:

  • Set a higher Weight value than we have set on 5_minute_voice_balance (25) so this balance will get consumed before 5_minute_voice_balance does
  • Set the DestinationIDs to match the destinations (Dest_AU_Mobile for Mobile and Dest_AU_Fixed for Local/National) we want the balance to apply to

ProTip: When you we create our Balance we can set what Destinations we want to use this balance for, if you want to specify multiple balances, we can do it by setting the Balance names as a string delimited by semicolons, like “DestinationIDs”: “Dest_AU_Fixed;Dest_AU_Mobile;Dest_AU_TollFree”

We’ll also set a balance expiry, which we’ll cover shortly, but now let’s define out 100 minutes for Local/National expiring at the end of the month:

#Add a balance to the account with type *voice with 100 minutes of talk time to Local / National Destinations expiring at the end of the month
Create_Local_National_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "Local_National_100_minutes_voice_balance",
                "Value": "100m",
                "ExpiryTime": "*month_end",
                "Weight": 60,
                "DestinationIDs": "Dest_AU_Fixed",
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Local_National_Voice_Balance_JSON))

We’ll also add our 24 hours to use to use 40 minutes of talk to mobiles, and a GetAccount to check the result:

#Add a balance to the account with type *voice with 40 minutes of talk time to Mobile Destinations expiring in 24 hours
Create_Mobile_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "Mobile_40_minutes_voice_balance",
                "Value": "40m",
                "ExpiryTime": "*daily",
                "Weight": 60,
                "DestinationIDs": "Dest_AU_Mobile",
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Mobile_Voice_Balance_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

Alright, let’s try running that:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'ad9d8bdd-64df-430f-af9d-3fc0410fd16b',
                                       'Value': 150000000000,
                                       'Weight': 25},
                                      {'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': {'Dest_AU_Fixed': True},
                                       'Disabled': False,
                                       'ExpirationDate': '2023-10-31T23:59:59+11:00',
                                       'Factor': None,
                                       'ID': 'Local_National_100_minutes_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'e4a2c211-8112-4e40-b3e6-250863404cc9',
                                       'Value': 6000000000000,
                                       'Weight': 60},
                                      {'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': {'Dest_AU_Mobile': True},
                                       'Disabled': False,
                                       'ExpirationDate': '2023-10-15T18:15:11.521636734+11:00',
                                       'Factor': None,
                                       'ID': 'Mobile_40_minutes_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'd4cbf6d8-50a5-4c97-82c2-dfe9936ae8d1',
                                       'Value': 2400000000000,
                                       'Weight': 60}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T18:15:11.524242437+11:00'}}

Alright! We now have 3 balances defined!

Notice in the API in the expiry I put *daily and *month_end, but in the output it’s got a real date and time (I wrote this 14/10/23 around 07:00 UTC, hence why those dates are what they are).
I could have specified the date and time in the API of a specific time I wanted the balance to expire (You can too, just replace “*daily” with “2024-01-01T00:00:00Z” for example), but that’s a pain in the butt, especially considering most of the time these values will be something common.
The *month_end is a special “meta” value, there’s a heap of these that allow us to do things like “current time + 20 minutes” (+20m), this time next month (*monthly), “this time tomorrow” (*daily), or “this time next week” (+168h) – You can find the full list of special dates here.

From a product perspective, setting an expiry on balances means we can set credit to expire 2 years after the subscriber tops up, but the same logic can be used so a subscriber could purchase a 7 day addon pack, that expires in 7 days, or a monthly plan can automatically expire in 30 days.

Now if we call the ProcessExternalCDR API again with a call to a Mobile and a Fixed number, we’ll see the respective balances get deducted.


#Generate a new call event for a 2.5 minute (150 second) call to a mobile number
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61412341234",
            "Destination": "61412341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "30s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

#Generate a new call event for a 2.5 minute (150 second) call to a fixed line local/national number
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61212341234",
            "Destination": "61212341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "30s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

We should see the minutes reduced by 30 seconds for our Local_National_100_minutes_voice_balance and Mobile_40_minutes_voice_balance balances, while our 5_minute_voice_balance hasn’t been touched.

{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":"None",
   "Disabled":false,
   "ExpirationDate":"0001-01-01T00:00:00Z",
   "Factor":"None",
   "ID":"5_minute_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"29f21735-1d62-49b1-9c53-80eab6f7b005",
   "Value":150000000000,
   "Weight":25
},
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Fixed":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-31T23:59:59+11:00",
   "Factor":"None",
   "ID":"Local_National_100_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"54db4f60-342e-4738-aaf1-a1304badc41d",
   "Value":5970000000000,
   "Weight":60
},
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Mobile":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-15T18:32:34.888821482+11:00",
   "Factor":"None",
   "ID":"Mobile_40_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"501eb00e-e947-4675-926f-080911e66897",
   "Value":2370000000000,
   "Weight":60
}

One last thing we’ll try before we end, our Mobile_40_minutes_voice_balance has still got 39.5 minutes left, and our 5_minute_voice_balance has still got minutes remaining, so if we try and make a call that’s 2450 seconds (~41 minutes), we should consume all the remaining minutes in Mobile_40_minutes_voice_balance and the go onto consume the remaining 1 minute out of 5_minute_voice_balance.

Let’s test this theory!

#Generate a new call event for a 42 minute call to a mobile to use all of our Mobile_40_minutes_voice_balance and start consuming 5_minute_voice_balance
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61412341234",
            "Destination": "61412341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "2450s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

Let’s check the output:

{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":"None",
   "Disabled":false,
   "ExpirationDate":"0001-01-01T00:00:00Z",
   "Factor":"None",
   "ID":"5_minute_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"29f21735-1d62-49b1-9c53-80eab6f7b005",
   "Value":70000000000,
   "Weight":25
},
...
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Mobile":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-15T18:44:18.161474861+11:00",
   "Factor":"None",
   "ID":"Mobile_40_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"501eb00e-e947-4675-926f-080911e66897",
   "Value":0.0161939859,
   "Weight":60
}

Boom, there we have it! Used all of the minutes in Mobile_40_minutes_voice_balance and started eating into the 5_minute_voice_balance.

Alright, that was a long post! Sorry about that, and props for making it to the end, still so much to learn about CGrateS.

Tales from the Trenches: mode-set in AMR

This one was a bit of a head scratcher for me, but I’m always glad to learn something new.

The handset made a VoLTE call, and it’s SDP offer shows it can support AMR and AMR-WB:

        Media Attribute (a): rtpmap:116 AMR-WB/16000/1
        Media Attribute (a): fmtp:116 mode-set=0,1,2,3,4,5,6,7,8;mode-change-capability=2;max-red=220
        Media Attribute (a): rtpmap:118 AMR/8000/1
        Media Attribute (a): fmtp:118 mode-set=0,1,2,3,4,5,6,7;mode-change-capability=2;max-red=220
        Media Attribute (a): rtpmap:111 telephone-event/16000
        Media Attribute (a): fmtp:111 0-15

Okay, that’s pretty normal, I can see we have the mode-set parameter defined, which indicates what modes the handset supports for each codec.

In our problem scenario, the Media Gateway that the call was sent to responded with this SDP answer:

        Media Description, name and address (m): audio 24504 RTP/AVP 118 110
        Media Attribute (a): rtpmap:118 AMR/8000
        Media Attribute (a): fmtp:118 mode-set=7
        Media Attribute (a): rtpmap:110 telephone-event/8000
        Media Attribute (a): fmtp:110 0-15
        Media Attribute (a): ptime:20
        Media Attribute (a): sendrecv
        [Generated Call-ID: FA163E564B37-f4d-98f56700-735d25-65357ee0-9c488]

But we got an error about not available codecs and the call drops, what gives?

Both sides support AMR (Only the phone supports AMR-WB), and the Media Gateway, as the answerer, supports mode-set 7, which is supported by the UE, so we should be good?

Well, not quite:

If mode-set is specified, it MUST be abided, and frames encoded with modes outside of the subset MUST NOT be sent in any RTP payload or used in codec mode requests. If not present, all codec modes are allowed for the payload type.

RFC 4867 – RTP Payload Format for AMR and AMR-WB

Okay, I get it, the answerer (media gateway) only supports mode 7, but the UE supports all the modes, so we should be fine right?

Well, no.

Section 8.3.1 in the RFC goes on to say in the Offer-Answer Model Considerations:

The parameter [mode-set] is bi-directional, i.e., the restricted set applies to media both to be received and sent by the declaring entity. If a mode set was supplied in the offer, the answerer SHALL return the mode-set unmodified or reject the payload type. However, the answerer is free to choose a mode-set in the answer only if no mode-set was supplied in the offer for a unicast two-peer session.

And there is our problem, and why the call is getting rejected.

The Media Gateway (the answerer in this scenario) is sending back the mode-set it supports (7) but as the UE / handset (offerer) included the mode-set, the Media Gateway should either respond with the same mode set (if it supported all the requested modes) or reject it.

Instead we’re seeing the Media Gateway repond with the mode set, which it supports, which it should not do: The Media Gateway should either return the same mode-set (unmodified / unchanged) or reject it.

And boom, another ticket to another vendor…

Tales from the Trenches – Emergency Calling when Roaming

In my last post talking about the Emergency Calling Codes, I had a few comments asking about what about in roaming scenarios?

For example, an American visiting the UK, would have 911 on the Emergency Calling Codes list on their SIM card, but in the UK they dial 999 to reach emergency services.

There’s two angles to this, the first is if a roamer dials the emergency calling code of their home country, the other is if they dial the emergency calling code of the country they are in.

Let’s look at the first scenario, where the roamer dials the emergency calling code of their home country.

If our American in the UK abroad dials 911, that number is on the ECC list on the SIM, it’s still flagged as an emergency call, and just goes out with the standard urn:service:sos URN – The network never sees 911 or 999, just that it’s an SOS call that goes to the PSAP.

In this scenario, the fact the dialled number is not passed to the network is actually a positive, we get the intent that the user wants to reach emergency services, and route based on this.

But what if our American friend in need dials 999?
That’s the correct number for the end user to dial in the UK after all, but if that’s not in their ECC list on the SIM / device, it’d go through as a regular call right?

If the call does not get flagged as an emergency call on the UE this has its own set of complications and considerations:

S8-Home Routing for VoLTE means that as the UE doesn’t know this is an emergency call, the call will get routed back to the home network. This means the call doesn’t go to the E-CSCF in the visited network, and would probably just get a message saying the number they’ve dialed is unavailable, this would be exactly as if they dialed 999 at home in the US.

But we have a fix for this!
On each MME we can set a list of emergency numbers, which would allow our Britt’s phone to know on this network, what the emergency calling codes are, and route the 999 call to the local PSAP, rather than home routing it.

MME Emergency Number list Config

This information is jammed into the Emergency Number List IE in the NAS Attach Accept body.

This means our American visitor in the UK, would know about 999 from the ECC list configured in the roaming operator’s MME.

The purpose of this information element is to encode emergency number(s) for use within the country where the IE is received.

3GPP TS 24.008: 10.5.3.13 – Emergency Number List

Where this becomes more problematic is unauthenticated emergency calling.

For example, a our American visiting the UK, that is not roaming dials 999.

We’ll assume the UK and US operator don’t have a VoLTE roaming agreement because they’ve been kicking the can down the road when it comes to VoLTE roaming… This is super common scenario – last numbers I saw on this were last year with ~50 bilateral VoLTE agreements in place worldwide.

Because the phone is not attached to a local MME, the handset does not know that 999 is an emergency calling code (because it’s not on the SIM), after all, the only way it can get the Emergency Number List is from an MME, and not having been attached to an MME, means the phone does not have the ECC list for the country, so the the handset does not begin the emergency attach procedure to make the call.

Common sense prevails here, on the majority of phones and the majority of SIM profiles, codes like 112 or 911 are treated as emergency calls, but more obscure numbers, such as dialing 999 in the UK or 10111 for South African Police on a handset with US firmware, are not guaranteed to work. Generally dialing the Emergency Calling code in the home network would get you through to some emergency services (although as we talked about in the last post, this might get you routed to the wrong agency in countries where each agency has their own number).

A better way forward?

These days I don’t dial much (apart from if I’m making adjustments on the Step-by-Step exchange), when I call people I do it from contacts, hyperlinks, etc.

Emergency Dialler page in Android

There is mountains of research to suggest that asking people to remember codes and phone numbers, is a struggle. A tourist who finds themselves in Tunisia in need of assistance, is unlikely to remember that it’s 190 for an Ambulance, and 198 for Fire.

Perhaps the ECC list on a phone should populate a page of icons from the emergency page on the phone, with the universal icon for each agency, that sends to the URN for that service type?

Countries with a single PSAP could have the URNs for each service type routed to the same place, while countries with seperated PSAPs for each service type, can route accordingly.

Likewise if a country does have a centralised PSAP for all call types, knowing the type that is selected would be useful, for example if the user has pressed fire and is not responsive when the call is answered, the best unit to dispatch would probably be a fire engine.

Tales from the Trenches: The issue with Emergency Calling URNs in IMS Networks

A lot of countries have a single point of contact for emergency services; in Europe you’d call 112 in an emergency, 000 in Australia or 911 in the US. Calling this number in the country will get you the emergency services.

This means a caller can order an ambulance for smoke inhalation, and the fire brigade, in one call.

But that’s not the case in every country; many countries don’t have one number for the emergency services, they’ve got multiple; a phone number for police, a different number for fire brigade and a different number for an ambulance.

For example, in Brazil if you need the police, you call 190, while a for example, uses 193 as the emergency number for the fire department, the police can be reached at 190 or 191 depending on if it’s road policing or general, and medical emergencies are covered by 192. Other countries have similar setups.

This is all well and good if you’re in Brazil, and you call 192 for an ambulance, the phone sends a SIP INVITE with a Request URI of sip:[email protected], because we can put a rule into our E-CSCF to say if the number is 192 to route it to the answer point for ambulances – But that’s not often the case on emergency calls.

In IMS, handsets generally detect the number dialed is on the Emergency Calling Code (ECC) list from the USIM Card.

The use of the ECC list means the phone knows this is an emergency call, and this is really important. For countries that use AML this can trigger sending of the AML SMS that process, and Emergency Calls should always be allowed to be made, even without credit, a valid SIM card, or even a SIM in the phone at all.

But this comes with a cost; when a user dials 911, the phones doesn’t (generally) send a call to sip:[email protected] like it would with any other dialled number, but rather the SIP INVITE is sent to urn:service:sos which will be routed to the PSAP by the E-CSCF. When a call comes through to these URNs they’re given top priority in the network

This is all well and good in a country where it doesn’t matter which emergency service you called, because all emergency calls route to a single PSAP, but in a country with multiple numbers, it’s really important when you call and ambulance, your call doesn’t get routed to animal control.

That means the phone has to look at what emergency number you’ve dialed, and map the URN it sends the call to to match what you’ve actually requested.

Recently we’ve been helping an operator in a country with a numbering plan like this, and we’ve been finding the limits of the standards here.
So let’s start by looking at what the standards state:

IMS Emergency Calling is governed by TS 103.479 which in turn delegates to IETF RFC 5031, but for the calling number to URN translation, it’s pretty quiet.

Let’s look at what RFC 5031 allows for URNs:

  • urn:service:sos.ambulance
  • urn:service:sos.animal-control
  • urn:service:sos.fire
  • urn:service:sos.gas
  • urn:service:sos.marine
  • urn:service:sos.mountain
  • urn:service:sos.physician
  • urn:service:sos.poison
  • urn:service:sos.police

The USIM’s Emergency Calling Codes EF would be the perfect source of this data; for each emergency calling code defined, you’ve got a flag to indicate what it’s for, here’s what we’ve got available on the SIM Card:

  • Bit 1 Police
  • Bit 2 Ambulance
  • Bit 3 Fire Brigade
  • Bit 4 Marine Guard
  • Bit 5 Mountain Rescue
  • Bit 6 manually initiated eCall
  • Bit 7 automatically initiated eCall
  • Bit 8 is spare and set to “0”

So these could be mapped pretty easily you’d think, so if the call is made to an Emergency Calling Code flagged with Bit 4, the URN would go to urn:service:sos.mountain.

Alas from our research, we’ve found most OEMs send calls to the generic urn:service:sos, regardless of the dialled number and the ECC flags that are set on the SIM for that number.

One of the big chip vendors sends calls to an ECC flagged as Ambulance to urn:service:sos.fire, which is totally infuriating, and we’ve had to put a rule in our E-CSCF to handle this if the User Agent is set to one of their phones.

Is there room for improvement here? For sure! Emergency calling is super important, and time is of the essence, while animal control can probably transfer you to an ambulance, an emergency is by very nature time sensitive, and any time wasted can lead to worse outcomes.

While carrier bundles from the OEMs can handle this, the global ability to take any phone, from any country and call an emergency number is so important, that relying on a country-by-country approach here won’t suffice.

What could we do as an industry to address this?

Acknowledging that not all countries have a single point of contact for emergency service, introducing a simple mechanism in the UE SIP message to indicate what number (Emergency Calling Code) the user actually dialled would be invaluable here.

URNs are important, but knowing the dialed number when it comes to PSAP routing, is so important – This wouldn’t even need to be its own SIP header, it could just be thrown into the Contact header as another parameter.

Highly developed markets are often the first to embrace new tech (for us this means VoLTE and VoNR), but this means that these issues seen by less developed markets won’t appear until long after the standard has been set in stone, and often countries like this aren’t at the table of the standards bodies to discuss such requirements.

This easy, reasonable update to the standard, has the potential to save lives, and next time this comes up in a working group I’ll be advocating for a change.

Playing back AMR streams from Packet Captures

The other day I found myself banging my head on the table to diagnose an issue with Ringback tone on an SS7 link and the IMS.

On the IMS side, no RBT was heard, but I could see the Media Gateway was sending RTP packets to the TAS, and the TAS was sending it to the UE, but was there actual content in the RTP packets or was it just silence?

If this was PCM / G711 we’d be able to just playback in Wireshark, but alas we can’t do this for the AMR codec.

Filter the RTP stream out in Wireshark

Inside Wireshark I filtered each of the audio streams in one direction (one for the A-Party audio and one for the B-Party audio)

Then I needed to save each of the streams as a separate PCAP file (Not PCAPng).

Turn into AMR File

With the audio stream for one direction saved, we can turn it into an AMR file, using Juan Noguera (Spinlogic)’s AMR Extractor tool.

Clone the Repo from git, and then in the same directory run:

python3 pcap_parser.py -i AMR_B_Leg.pcap -o AMR_B_Leg.3ga

Playback with VLC / Audacity

I was able to play the file with VLC, and load it into Audacity to easily see that yes, the Ringback Tone was present in the AMR stream!

A look at Advanced Mobile Location SMS for Emergency Calls

Advanced Mobile Location (AML) is being rolled out by a large number of mobile network operators to provide accurate caller location to emergency services, so how does it work, what’s going on and what do you need to know?

Recently we’ve been doing a lot of work on emergency calling in IMS, and meeting requirements for NG-112 / e911, etc.

This led me to seeing my first Advanced Mobile Location (AML) SMS in the wild.

For those unfamiliar, AML is a fancy text message that contains the callers location, accuracy, etc, that is passed to emergency services when you make a call to emergency services in some countries.

It’s sent automatically by your handset (if enabled) when making a call to an emergency number, and it provides the dispatch operator with your location information, including extra metadata like the accuracy of the location information, height / floor if known, and level of confidence.

The standard is primarily driven by EENA, and, being backed by the European Union, it’s got almost universal handset support.

Google has their own version of AML called ELS, which they claim is supported on more than 99% of Android phones (I’m unclear on what this means for Harmony OS or other non-Google backed forks of Android), and Apple support for AML starts from iOS 11 onwards, meaning it’s supported on iPhones from the iPhone 5S onards,.

Call Flow

When a call is made to the PSAP based on the Emergency Calling Codes set on the SIM card or set in the OS, the handset starts collecting location information. The phone can pull this from a variety of sources, such as WiFi SSIDs visible, but the best is going to be GPS or one of it’s siblings (GLONASS / Galileo).

Once the handset has a good “lock” of a location (or if 20 seconds has passed since the call started) it bundles up all of this information the phone has, into an SMS and sends it to the PSAP as a regular old SMS.

The routing from the operator’s SMSc to the PSAP, and the routing from the PSAP to the dispatcher screen of the operator taking the call, is all up to implementation. For the most part the SMS destination is the emergency number (911 / 112) but again, this is dependent on the country.

Inside the SMS

To the user, the AML SMS is not seen, in fact, it’s actually forbidden by the standard to show in the “sent” items list in the SMS client.

On the wire, the SMS looks like any regular SMS, it can use GSM7 bit encoding as it doesn’t require any special characters.

Each attribute is a key / value pair, with semicolons (;) delineating the individual attributes, and = separating the key and the value.

Below is an example of an AML SMS body:

A"ML=1;lt=+54.76397;lg=-
0.18305;rd=50;top=20130717141935;lc=90;pm=W;si=123456789012345;ei=1234567890123456;mcc=234;mnc=30; ml=128

If you’ve got a few years of staring at Wireshark traces in Hex under your belt, then this will probably be pretty easy to get the gist of what’s going on, we’ve got the header (A”ML=1″) which denotes this is AML and the version is 1.

After that we have the latitude (lt=), longitude (lg=), radius (rd=), time of positioning (top=), level of confidence (lc=), positioning method (pm=) with G for GNSS, W for Wifi signal, C for Cell
or N for a position was not available, and so on.

AML outside the ordinary

Roaming Scenarios

If an emergency occurs inside my house, there’s a good chance I know the address, and even if I don’t know my own address, it’s probably linked to the account holder information from my telco anyway.

AML and location reporting for emergency calls is primarily relied upon in scenarios where the caller doesn’t know where they’re calling from, and a good example of this would be a call made while roaming.

If I were in a different country, there’s a much higher likelihood that I wouldn’t know my exact address, however AML does not currently work across borders.

The standard suggests disabling SMS when roaming, which is not that surprising considering the current state of SMS transport.

Without a SIM?

Without a SIM in the phone, calls can still be made to emergency services, however SMS cannot be sent.

That’s because the emergency calling standards for unauthenticated emergency calls, only cater for

This is a limitation however this could be addressed by 3GPP in future releases if there is sufficient need.

HTTPS Delivery

The standard was revised to allow HTTPS as the delivery method for AML, for example, the below POST contains the same data encoded for use in a HTTP transaction:

v=3&device_number=%2B447477593102&location_latitude=55.85732&location_longitude=-
4.26325&location_time=1476189444435&location_accuracy=10.4&location_source=GPS&location_certainty=83
&location_altitude=0.0&location_floor=5&device_model=ABC+ABC+Detente+530&device_imei=354773072099116
&device_imsi=234159176307582&device_os=AOS&cell_carrier=&cell_home_mcc=234&cell_home_mnc=15&cell_net
work_mcc=234&cell_network_mnc=15&cell_id=0213454321 

Implementation of this approach is however more complex, and leads to little benefit.

The operator must zero-rate the DNS, to allow the FQDN for this to be resolved (it resolves to a different domain in each country), and allow traffic to this endpoint even if the customer has data disabled (see what happens when your handset has PS Data Off ), or has run out of data.

Due to the EU’s stance on Net Neutrality, “Zero Rating” is a controversial topic that means most operators have limited implementation of this, so most fall back to SMS.

Other methods for sharing location of emergency calls?

In some upcoming posts we’ll look at the GMLC used for E911 Phase 2, and how the network can request the location from the handset.

Further Reading

https://eena.org/knowledge-hub/documents/aml-specifications-requirements/

VoLTE / IMS – Analysis Challenge

It’s challenge time, this time we’re going to be looking at an IMS PCAP, and answering some questions to test your IMS analysis chops!

Here’s the packet capture:

Easy Questions

  • What QCI value is used for the IMS bearer?
  • What is the registration expiry?
  • What is the E-UTRAN Cell ID the Subscriber is served by?
  • What is the AMBR of the IMS APN?

Intermediate Questions

  • Is this the first or subsequent registration?
  • What is the Integrity-Key for the registration?
  • What is the FQDN of the S-CSCF?
  • What Nonce value is used and what does it do?
  • What P-CSCF Addresses are returned?
  • What time would the UE need to re-register by in order to stay active?
  • What is the AA-Request in #476 doing?
  • Who is the(opens in a new tab)(opens in a new tab)(opens in a new tab) OEM of the handset?
  • What is the MSISDN associated with this user?

Hard Questions

  • What port is used for the ESP data?
  • Which encryption algorithm and algorithm is used?
  • How many packets are sent over the ESP tunnel to the UE?
  • Where should SIP SUBSCRIBE requests get routed?
  • What’s the model of phone?

The answers for each question are on the next page, let me know in the comments how you went, and if there’s any tricky ones!

Mobile IPv6 Tax?

Recently a Tweet from Dean Bubly got me thinking about how data is charged in cellular:

In the cellular world, subscribers are charged for data from the IP, transport and applications layers; this means you pay for the IP header, you pay for the TCP/UDP header, and you pay for the contents (the cat videos it contains).

This also means if an operator moves mobile subscribers from IPv4 to IPv6, there’s an extra 20 bytes the customer is charged for for every packet sent / received, which the customer is charged for – This is because the IPv6 header is longer than the IPv4 header.

Source: ServerFault - https://serverfault.com/questions/547768/ipv4-header-vs-ipv6-header-size

In most cases, mobile subs don’t get a choice as to if their connection is IPv4 or IPv6, but on a like for like basis, we can say that if a customer moves is on IPv6 every packet sent/received will have an extra 20 bytes of data consumed compared to IPv4.

This means subscribers use more data on IPv6, and this means they get charged for more data on IPv6.

For IoT applications, light users and PAYG users, this extra 20 bytes per packet could add up to something significant – But how much?

We can quantify this, but we’d need to know the number of packets sent on average, and the quantity of the data transferred, because the number of packets is the multiplier here.

So for starters I’ve left a phone on the desk, it’s registered to the network but just sitting in Idle mode – This is an engineering phone from an OEM, it’s just used for testing so doesn’t have anything loaded onto it in terms of apps, it’s not signed into any applications, or checking in the background, so I thought I’d try something more realistic.

So to get a clearer picture, I chucked a SIM in my regular everyday phone I use personally, registered it to the cellular lab I have here. For the next hour I sniffed the GTP traffic for the phone while it was sitting on my desk, not touching the phone, and here’s what I’ve got:

Overall the PCAP includes 6,417,732 bytes of data, but this includes the transport and GTP headers, meaning we can drop everything above it in our traffic calculations.

Everything except the data encapsulated in GTP can be dropped

For this I’ve got 14 bytes of ethernet, 20 bytes IP, 8 bytes UDP and 5 bytes for TZSP (this is to copy the traffic from the eNB to my local machine), then we’ve got the transport from the eNB to the SGW, 14 bytes of ethernet again, 20 bytes of IP , 8 bytes of UDP and 8 bytes of GTP then the payload itself. Phew.
All this means we can drop 97 bytes off every packet.

We have 16,889 packets, 6,417,732 bytes in total, minus 97 bytes from each gives us 1,638,233 of headers to drop (~1.6MB) giving us a total of 4.556 MB traffic to/from the phone itself.

This means my Android phone consumes 4.5 MB of cellular data in an hour while sitting on the desk, with 16,889 packets in/out.

Okay, now we’re getting somewhere!

So now we can answer the question, if each of these 16k packets was IPv6, rather than IPv4, we’d be adding another 20 bytes to each of them, 20 bytes x 16,889 packets gives 337,780 bytes (~0.3MB) to add to the total.

If this traffic was transferred via IPv6, rather than IPv4, we’d be looking at adding 20 bytes to each of the 16,889 packets, which would equate to 0.3MB extra, or about 7% overhead compared to IPv4.

But before you go on about what an outrage this IPv6 transport is, being charged for those extra bytes, that’s only one part of the picture.

There’s a reason operators are finally embracing IPv6, and it’s not to put an extra 7% of traffic on the network (I think if you asked most capacity planners, they’d say they want data savings, not growth).

IPv6 is, for lack of a better term, less rubbish than IPv4.

There’s a lot of drivers for IPv6, and some of these will reduce data consumption.
IPv6 is actually your stuff talking directly to the remote stuff, this means that we don’t need to rely on NAT, so no need to do NAT keepalives, and opening new sessions, which is going to save you data. If you’re running apps that need to keep a connection to somewhere alive, these data savings could negate your IPv6 overhead costs.

Will these potential data savings when using IPv6 outweigh the costs?

That’s going to depend on your use case.

If you’ve extremely bandwidth / data constrained, for example, you have an IoT device on an NTN / satellite connection, that was having to Push data every X hours via IPv4 because you couldn’t pull data from it as it had no public IP, then moving it to IPv6 so you can pull the data on the public IP, on demand, will save you data. That’s a win with IPv6.

If you’re a mobile user, watching YouTube, getting push notifications and using your phone like a normal human, probably not, but if you’re using data like a normal user, you’ve probably got a sizable data allowance that you don’t end up fully consuming, and the extra 20 bytes per packet will be nothing in comparison to the data used to watch a 2k video on your small phone screen.

DNS – TCP or UDP?

Ask someone with headphones and a lanyard in the halls of a datacenter what transport does DNS use, there’s a good chance the answer you’d get back is UDP Port 53.

But not always!

In scenarios where the DNS response is large (beyond 512 bytes) a DNS query will shift over to TCP for delivery.

How does the client know when to shift the request to TCP – After all, the DNS server knows how big the response is, but the client doesn’t.

The answer is the Truncated flag, in the response.

The DNS server sends back a response, but with the Truncated bit set, as per RFC 1035:

TC TrunCation – specifies that this message was truncated due to length greater than that permitted on the transmission channel.

RFC 1035

Here’s an example of the truncated bit being set in the DNS response.

The DNS client, upon receiving a response with the truncated bit set, should run the query again, this time using TCP for the transport.

One prime example of this is DNS NAPTR records used for DNS in roaming scenarios, where the response can quite often be quite large.

If it didn’t move these responses to TCP, you’d run the risk of MTU mismatches dropping DNS. In that half of my life has been spent debugging DNS issues, and the other half of my life debugging MTU issues, if I had MTU and DNS issues together, I’d be looking for a career change…

Improving WiFi Calling quality for WiFi Operators

I had a question recently on LinkedIn regarding how to preference Voice over WiFi traffic so that a network engineer operating the WiFi network can ensure the best quality of experience for Voice over WiFi.

Voice over WiFi is underpinned by the ePDG – Evolved Packet Data Gateway (this is a fancy IPsec tunnel we authenticate to using the SIM to drop our traffic into the P-CSCF over an unsecured connection). To someone operating a WiFi network, the question is how do we prioritise the traffic to the ePDGs and profile it?

ePDGs can be easily discovered through a simple DNS lookup, once you know the Mobile Network Code and Mobile Country code of the operators you want to prioritise, you can find the IPs really easily.

ePDG addresses take the form epdg.epc.mncXXX.mccYYY.pub.3gppnetwork.org so let’s look at finding the IPs for each of these for the operators in a country:

The first step is nailing down the mobile network code and mobile country codes of the operators you want to target, Wikipedia is a great source for this information.
Here in Australia we have the Mobile Country Code 505 and the big 3 operators all support Voice over WiFi, so let’s look at how we’d find the IPs for each.
Telstra has mobile network code (MNC) 01, in 3GPP DNS we always pad network codes to 3 digits, so that’ll be 001, and the mobile country code (MCC) for Australia is 505.
So to find the IPs for Telstra we’d run an nslookup for epdg.epc.mnc001.mcc505.pub.3gppnetwork.org – The list of IPs that are returned, are the IPs you’ll see Voice over WiFi traffic going to, and the IPs you should provide higher priority to:

For the other big operators in Australia epdg.epc.mnc002.mcc505.pub.3gppnetwork.org will get you Optus and epdg.epc.mnc003.mcc505.pub.3gppnetwork.org will get you VHA.

The same rules apply in other countries, you’d just need to update the MNC/MCC to match the operators in your country, do an nslookup and prioritise those IPs.

Generally these IPs are pretty static, but there will need to be a certain level of maintenance required to keep this list up to date by rechecking.

Happy WiFi Calling!

SMS Transport Wars?

There’s old joke about standards that the great thing about standards there’s so many to choose from.

SMS wasn’t there from the start of GSM, but within a year of the inception of 2G we had SMS, and we’ve had SMS, almost totally unchanged, ever since.

In a recent Twitter exchange, I was asked, what’s the best way to transport SMS?
As always the answer is “it depends” so let’s take a look together at where we’ve come from, where we are now, and how we should move forward.

How we got Here

Between 2G and 3G SMS didn’t change at all, but the introduction of 4G (LTE) caused a bit of a rethink regarding SMS transport.

Early builders of LTE (4G) networks launched their 4G offerings without 4G Voice support (VoLTE), with the idea that networks would “fall back” to using 2G/3G for voice calls.

This meant users got fast data, but to make or receive a call they relied on falling back to the circuit switched (2G/3G) network – Hence the name Circuit Switched Fallback.

Falling back to the 2G/3G network for a call was one thing, but some smart minds realised that if a phone had to fall back to a 2G/3G network every time a subscriber sent a text (not just calls) – And keep in mind this was ~2010 when SMS traffic was crazy high; then that would put a huge amount of strain on the 2G/3G layers as subs constantly flip-flopped between them.

To address this the SGs-AP interface was introduced, linking the 4G core (MME) with the 2G/3G core (MSC) to support this stage where you had 4G/LTE but only for data, SMS and calls still relied on the 2G/3G core (MSC).

The SGs-AP interface has two purposes;
One, It can tell a phone on 4G to fallback to 2G/3G when it’s got an incoming call, and two; it can send and receive SMS.

SMS traffic over this interface is sometimes described as SMS-over-NAS, as it’s transported over a signaling channel to the UE.

This also worked when roaming, as the MSC from the 2G/3G network was still used, so SMS delivery worked the same when roaming as if you were in the home 2G/3G network.

Enter VoLTE & IMS

Of course when VoLTE entered the scene, it also came with it’s own option for delivering SMS to users, using IP, rather than the NAS signaling. This removed the reliance on a link to a 2G/3G core (MSC) to make calls and send texts.

This was great because it allowed operators to build networks without any 2G/3G network elements and build a fully standalone LTE only network, like Jio, Rakuten, etc.

VoLTE didn’t change anything about the GSM 2G/3G SMS PDU, it just bundled it up in an SIP message body, this is often referred to as SMS-over-IP.

SMS-over-IP doesn’t address any of the limitations from 2G/3G, including limiting multipart messages to send payloads above 160 characters, and carries all the same limitations in order to be backward compatible, but it is over IP, and it doesn’t need 2G or 3G.

In roaming scenarios, S8 Home Routing for VoLTE enabled SMS to be handled when roaming the same way as voice calls, which made SMS roaming a doddle.

4G SMS: SMS over IP vs SMS over NAS

So if you’re operating a 4G network, should you deliver your SMS traffic using SMS-over-IP or SMS-over-NAS?

Generally, if you’ve been evolving your network over the years, you’ve got an MSC and a 2G/3G network, you still may do CSFB so you’ve probably ended up using SMS over NAS using the SGs-AP interface.
This method still relies on “the old ways” to work, which is fine until a discussion starts around sunsetting the 2G/3G networks, when you’d need to move calling to VoLTE, and SMS over NAS is a bit of a mess when it comes to roaming.

Greenfield operators generally opt for SMS over IP from the start, but this has its own limitations; SMS over IP is has awful efficiency which makes it unsuitable for use with NB-IoT applications which are bandwidth constrained, support for SMS over IP is generally limited to more expensive chipsets, so the bargain basement chips used for IoT often don’t support SMS over IP either, and integration of VoLTE comes with its own set of challenges regarding VoLTE enablement.

5G enters the scene (Nsmsf_SMService)

5G rolled onto the scene with the opportunity to remove the SMS over NAS option, and rely purely on SMS over IP (IMS); forcing the industry to standardise on an option alas this did not happen.

Instead 5GC introduces another delivery mechanism for SMS, just for 5GC without VoNR, the SMSf which can still send messages over the 5G NAS messaging.

This added another option for SMS delivery dependent on the access network used, and the Nsmsf_SMService interface does not support roaming.

Of course if you are using Voice over NR (VoNR) then like VoLTE, SMS is carried in a SIP message to the IMS, so this negates the need for the Nsmsf_SMService.

2G/3G Shutdown – Diameter to replace SGs-AP (SGd)

With the 2G/3G shutdown in the US operators who had up until this point been relying on SMS-over-NAS using the SGs-AP interface back to their MSCs were forced to make a decision on how to route SMS traffic, after the MSCs were shut down.

This landed with SMS-over-Diameter, where the 4G core (MME) communicates over Diameter with the SMSc.

The advantage of this approach is the Diameter protocol stack is the backbone of 4G roaming, and it’s not a stretch to get existing Diameter Routing Agents to start flicking SMS over Diameter messages between operators.

This has adoption by all the US operators, but we’re not seeing it so widely deployed in the rest of the world.

State of Play

OptionConditionsNotes
MAP2G/3G OnlyRelies on SS7 signaling and is very old
Supports roaming
SGs-AP (SMS-over-NAS)4G only relies on 2G/3GNeeds an MSC to be present in the network (generally because you have a 2G/3G network and have not deployed VoLTE)
Supports limited roaming
SMS over IP (IMS)4G / 5GNot supported on 2G/3G networks
Relies on a IMS enabled handset and network
Supports roaming in all S8 Home Routed scenarios
Device support limited, especially for IoT devices
Diameter SGd4G only / 5G NSAOnly works on 4G or 5G NSA
Better device support than 4G/5G
Supports roaming in some scenarios
Nsmsf_SMService5G standalone onlyOnly works on 5GC
Doesn’t support roaming
The convoluted world of SMS delivery options

A Way Forward:

While the SMS payload hasn’t changed in the past 31 years, how it is transported has opened up a lot of potential options for operators to use, with no clear winner, while SMS revenues and traffic volumes have continued to fall.

For better or worse, the industry needs to accept that SMS over NAS is an option to use when there is no IMS, and that in order to decommission 2G/3G networks, IMS needs to be embraced, and so SMS over IP (IMS) supported in all future networks, seems like the simple logical answer to move forward.

And with that clear path forward, we add in another wildcard…

Direct to device Satellite messes everything up…

Remember way back in this post when I said SMS over IP using IMS is a really really inefficient way of getting data? Well that hasn’t been a problem as we progressed up the generations of cellular tech as with each “G” we had more and more bandwidth than the last.

To throw a spanner in the works, let’s introduce NB-IoT and Non-Terrestrial Networks which rely on Non-IP-Data-Delivery.

These offer the ability to cover the globe with a low bandwidth / high latency service, that would ensure a subscriber is always just a message away, we’re seeing real world examples of these networks getting deployed for messaging applications already.

But, when you’ve only got a finite resource of bandwidth, and massive latencies to contend with, the all-IP architecture of IMS (VoLTE / VoNR) and it’s woeful inefficiency starts to really sting.

Of course there are potential workarounds here, Robust Header Correction (ROHC) can shrink this down, but it’s still going to rely on the 3 way handshake of TCP, TCP keepalive timers and IMS registrations, which in turn can starve the radio resources of the satellite link.

For NTN (Satelite) networks the case is being heavily made to rely on Non-IP-Data-Delivery, so the logical answer for these applications is to move the traffic back to SMS over NAS.

End Note

Even with SMS over 30 years old, we can still expect it to be a part of networks for years to come, even as WhatsApp / iMessage, etc, offer enhanced services. As to how it’s transported and the myriad of options here, I’m expecting that we’ll keep seeing a multi-transport mix long into the future.

For simple, cut-and-dried 4G/5G only network, IMS and SMS over IP makes the most sense, but for anything outside of that, you’ve got a toolbox of options for use to make a solution that best meets your needs.

Verify Android Signing Certificate for ARA-M Carrier Privileges in App

Part of the headache when adding the ARA-M Certificate to a SIM is getting the correct certificate loaded,

The below command calculates it the SHA-1 Digest we need to load as the App ID on the SIM card’s ARA-M or ARA-F applet:

apksigner verify --verbose --print-certs "yourapp.apk"

You can then flash this onto the SIM with PySIM:

pySIM-shell (MF/ADF.ARA-M)> aram_store_ref_ar_do --aid FFFFFFFFFFFF --device-app-id 40b01d74cf51bfb3c90b69b6ae7cd966d6a215d4 --android-permissions 0000000000000001 --apdu-always

What’s the maximum speed for LTE and 5G?

Even before 5G was released, the arms race to claim the “fastest” speeds on LTE, NSA and SA networks has continued, with pretty much every operator claiming a “first” or “fastest”.

I myself have the fastest 5G network available* but I thought I’d look at how big the values are we can put in for speed, these are the Maximum Bitrate Values (like AMBR) we can set on an APN/DNN, or on a Charging Rule.

*Measurement is of the fastest 5G network in an eastward facing office, operated by a person named Nick, in a town in Australia. Other networks operated by people other than those named Nick in eastward facing office outside of Australia were not compared.

The answer for Release 8 LTE is 4294967294 bytes per second, aka 4295 Mbps 4.295 Gbps.

Not bad, but why this number?

The Max-Requested-Bandwidth-DL AVP tells the PGW the max throughput allowed in bits per second. It’s a Unsigned32 so max value is 4294967294, hence the value.

But come release 15 some bright spark thought we may in the not to distant future break this barrier, so how do we go above this?

The answer was to bolt on another AVP – the “Extended-Max-Requested-BW-DL” AVP ( 554 ) was introduced, you might think that means the max speed now becomes 2x 4.295 Gbps but that’s not quite right – The units was shifted.

This AVP isn’t measuring bits per second it’s measuring kilobits per second.

So the standard Max-Requested-Bandwidth-DL AVP gives us 4.3 Gbps, while the Extended-Max-Requested-Bandwidth gives us a 4,295 Gbps.

We add the Extended-Max-Requested-Bandwidth AVP (4295 Gbps) onto the Max-Requested Bandwidth AVP (4.3 Gbps) giving us a total of 4,4299.3 Gbps.

So the short answer:

Pre release 15: 4.3 Gbps

Post release 15: 4,4299.3 Gbps

Using Wireshark to search a SIM

Today I was updating a SIM profile for work, the client is rebranding and we need to remove all references to their old brand from the SIM profile.

I’ve written about using Wireshark to view APDU traces on SIM cards before, but today I had a simple need, to find all files with the client’s brand name in them.

I started off just updating the SPN, OPN, etc, etc, but I had a suspicion there were still references.

I confirmed this pretty easily with Wireshark, first I started a trace in Wireshark of the APDUs: I enabled capturing on a USB Interface:

modprobe usbmon

Then we need to find where our card reader is connected, running ‘lsusb‘ lists all the USB devices, and you can see here’s mine on Bus 1, Device 49.

Then fired up Wireshark, selected USB Bus 01 to capture all the USB traffic on the bus.

Then I ran the “export” command in PySIM to read the contents of all the files on the SIM, and jumped back over to Wireshark. (PySIM decodes most files but not all – Whereas this method just looks for the bytes containing the string)

From the search menu in Wireshark I searched the packet bytes for the string containing the old brand name, and found two more EFs I’d missed.

For anyone playing along at home, using this method I found references to the old brand name in SMSP (which contains the network name) and ADN (Which had the customer support number as a contact with the old brand name).

Another great use for Wireshark!

Cisco ITP STP – Network Appearance

Short one,
The other day I needed to add a Network Appearance on an SS7/SS7 M3UA linkset.

Network Appearances on M3UA links are kinda like a port number, in that they allow you to distinguish traffic to the same point code, but handled by different logical entities.

When I added the NA parameter on the Linkset nothing happened.

If you’re facing the same you’ll need to set:

cs7 multi-instance

In the global config (this is the part I missed).

Then select the M3UA linkset you want to change and add the network-appearance parameter:

network-appearance 10

And bingo, you’ll start seeing it in your M3UA traffic:

Huawei BBU 3900 Architecture

Huawei BTS3900 eNB Configuration

Last year I purchased a cheap second hand Huawei macro base station – there’s lots of these on the market at the moment due to the fact they’re being replaced in many countries.

I’m using it in my lab environment, and as such the config I’ve got is very “bare bones” and basic. Keep in mind if you’re looking to deploy a Macro eNodeB in production, you may need more than just a blog post to get everything tuned and functioning properly…

In this post we’ll cover setting up a Huawei BTS3900 eNodeB from scratch, using the MML interface, without relying on the U2020 management tool.

Obviously the details I setup (IP Addressing, PLMN and RF parameters) are going to be different to what you’re configuring, so keep that in mind, where I’ve got my MME Addresses, site IDs, TACs, IP Addresses, RFUs, etc, you’ll need to substitute your own values.

A word on Cabinets

Typically these eNodeBs are shipped in cabinets, that contain the power supplies, alarm / environmental monitoring, power distribution, etc.

Early on in the setup process we’ll be setting the cabinet types we’ve got, and then later on we’ll tell the system what we have installed in which slots.

This is fine if you have a cabinet and know the type, but in my case at least I don’t have a cabinet manufactured by Huawei, just a rack with some kit mounted in it.

This is OK, but it leads to a few gotchas I need to add a cabinet (even though it doesn’t physically exist) and when I setup my RRUs I need to define what cabinet, slot and subrack it’s in, even though it isn’t in any. Keep this in mind as we go along and define the position of the equipment, that if you’re not using a real-world cabinet, the values mean nothing, but need to be kept consistent.

The Basics

Before we get started, familiarise yourself with the Huawei MML we’ll use for configuring the unit, and log into the Web UI and bring up an MML shell.

To begin we’ll need to setup the basics, by disabling DHCP and setting an local IP Address for the unit.

 SET DHCPSW: SWITCH=DISABLE;
 SET LOCALIP: IP="192.168.5.234", MASK="255.255.248.0";

Obviously your IP address details will be different.
Next we’ll add an eNodeB function, the LMPT / UMPT can have multiple functions and multiple eNodeBs hosted on the same hardware, but in our case we’re just going to configure one:

 ADD ENODEBFUNCTION: eNodeBFunctionName="LTE", ApplicationRef=1, eNodeBId=9527;
 SET NE: NENAME="HUAWEI", LOCATION="NewSite", DID="NewSite12345", SITENAME="NewSite1", USERLABEL="NewInitSite";
 ADD LOCATION: LOCATIONNAME="NewSite", GCDF=Degree, LATITUDEDEGFORMAT=0, LONGITUDEDEGFORMAT=0; 

Again, your eNodeB ID, location, site name, etc, are all going to be different, as will your location.

Next we’ll set the system to maintenance mode (MNTMODE), so we can make changes on the fly (this takes the eNB off the air, but we’re already off the air), you’ll need to adjust the start and end times to reflect the current time for the start time, and end time to be after you’re done setting all this up.

 SET MNTMODE: MNTMode=INSTALL, ST=2013&09&20&15&00&00, ET=2013&09&25&15&00&00, MMSetRemark="NewSite Install";

Next we’ll set the operator details, this is the PLMN of the eNodeB, and create a new tracking area.

 ADD CNOPERATOR: CnOperatorId=0, CnOperatorName="NickTest", CnOperatorType=CNOPERATOR_PRIMARY, Mcc="001", Mnc="01";
ADD CNOPERATORTA: TrackingAreaId=0, CnOperatorId=0, Tac=1;

Next we’ll be setting and populating the cabinets I mentioned earlier. I’ll be telling the unit it’s inside a APM30 (Cabinet 0), and in Cabinet Number 0, Subrack 0, is a BBU3900.

 //To modify the cabinet type, run the following command:
ADD CABINET:CN=0,TYPE=APM30;
//Add a BBU3900 subrack, run the following command:
ADD SUBRACK:CN=0,SRN=0,TYPE=BBU3900;
//To configure boards and RF datas, run the following commands:

And inside the BBU3900 there’s some cards of course, and each card has as slot, as per the drawing below.

In my environment I’ve got a LMPT in slot 7, and a LBBP in Slot 3. There’s a fan and a UPEU too, so:
We’ll add a board in Slot No. 7, of type LMPT,
We’ll add a board in Slot No. 3, of type LBBP working on FDD,
We’ll add a fan board in Slot No. 16, and a UPEU in Slot No. 18.

 ADD BRD:SN=7,BT=LMPT;
 ADD BRD:CN=0,SRN=0,SN=3,BT=LBBP,WM=TDD;
 ADD BRD:CN=0,SRN=0,SN=16,BT=FAN;
 ADD BRD:CN=0,SRN=0,SN=18,BT=UPEU;

Huawei publish design guides for which cards should be in which slots, the general rule is that your LMPT / UMPT card goes in Slot 7, with your BBP cards (UBBP or LBBP) in slots 3, then 2, then 1, then 0. Fans and UPEUs can only go in the slots designed to fit them, so that makes it a bit easier.

Next we’ll need to setup our RRUs, for this we’ll need to setup an RRU chain, which is the Huawei term for the CPRI links and add an RRU into it:

ADD RRUCHAIN:RCN=10,TT=CHAIN,BM=COLD,HSRN=70,HSN=0,HPN=0;

ADD RRU:CN=0,SRN=60,SN=0,TP=BRANCH,RCN=10,PS=0,RT=MPMU,RS=TDL,RXNUM=0,TXNUM=0;

With our RRU chains defined, we’ll need to setup our transport network to get the traffic back to the S-GW / MME:

SET ETHPORT: SN=7, SBT=BASE_BOARD, PA=COPPER, SPEED=AUTO, DUPLEX=AUTO;
ADD DEVIP: SN=7, SBT=BASE_BOARD, PT=ETH, PN=0, IP="10.10.10.67", MASK="255.255.255.0";
ADD IPRT: RTIDX=0, SN=7, SBT=BASE_BOARD, DSTIP="10.166.1.251", DSTMASK="255.255.255.255", RTTYPE=NEXTHOP, NEXTHOP="10.10.10.1"; 
ADD IPRT: RTIDX=1, SN=7, SBT=BASE_BOARD, DSTIP="10.4.3.3", DSTMASK="255.255.255.255", RTTYPE=NEXTHOP, NEXTHOP="10.10.10.1"; 
ADD IPRT: RTIDX=2, SN=7, SBT=BASE_BOARD, DSTIP="10.3.3.3", DSTMASK="255.255.255.255", RTTYPE=NEXTHOP, NEXTHOP="10.10.10.1";
ADD IPRT: RTIDX=3, SN=7, SBT=BASE_BOARD, DSTIP="10.60.60.60", DSTMASK="255.255.255.255", RTTYPE=NEXTHOP, NEXTHOP="10.10.10.1";
ADD OMCH: IP="10.10.10.67", MASK="255.255.255.0", PEERIP="10.166.1.251", PEERMASK="255.255.255.255", BEAR=IPV4, BRT=YES, RTIDX=0, BINDSECONDARYRT=NO, CHECKTYPE=NONE;
ADD VLANMAP: NEXTHOPIP="10.10.10.1", MASK="255.255.248.0", VLANMODE=SINGLEVLAN, VLANID=3721, SETPRIO=DISABLE; 
ADD SCTPTEMPLATE: SCTPTEMPLATEID=0, SWITCHBACKFLAG=ENABLE;
ADD SCTPHOST: SCTPHOSTID=0, IPVERSION=IPv4, SIGIP1V4="10.10.10.67", SIGIP1SECSWITCH=DISABLE, SIGIP2SECSWITCH=DISABLE, PN=2000, SCTPTEMPLATEID=0;
ADD SCTPPEER: SCTPPEERID=0, IPVERSION=IPv4, SIGIP1V4="10.3.3.3", SIGIP1SECSWITCH=DISABLE, SIGIP2SECSWITCH=DISABLE, PN=2000;
ADD USERPLANEHOST: UPHOSTID=0, IPVERSION=IPv4, LOCIPV4="10.10.10.67", IPSECSWITCH=DISABLE;
ADD EPGROUP: EPGROUPID=0;
ADD SCTPHOST2EPGRP: EPGROUPID=0, SCTPHOSTID=0; 
ADD SCTPPEER2EPGRP: EPGROUPID=0, SCTPPEERID=0;
ADD UPHOST2EPGRP: EPGROUPID=0, UPHOSTID=0;
ADD S1: S1Id=0, CnOperatorId=0, EpGroupCfgFlag=CP_UP_CFG, CpEpGroupId=0, UpEpGroupId=0;


We’ll need clocking and time as well, we’ll use NTP and GPS:

SET TIMESRC: TIMESRC=NTP; 
ADD NTPC: MODE=IPV4, IP="10.166.1.251", PORT=123, SYNCCYCLE=60, AUTHMODE=PLAIN; 
SET MASTERNTPS: MODE=IPV4, IP="10.166.1.251"; 
SET TZ: ZONET=GMT+0800, DST=NO;

ADD GPS: SRN=0, SN=7;
SET CLKMODE: MODE=MANUAL, CLKSRC=GPS, SRCNO=0;
SET CLKSYNCMODE:CLKSYNCMODE=TIME;

Next we’ll need to define a sector, sector equipment & cell, then link it to a sector equipment group:

ADD SECTOR:SECTORID=0,ANTNUM=2,ANT1CN=0,ANT1SRN=60,ANT1SN=255, ANT1N=R0A,ANT2CN=0,ANT2SRN=60,ANT2SN=255,ANT2N=R0B,CREATESECTOREQM=FALSE;

ADD SECTOREQM:SECTOREQMID=0,SECTORID=0,ANTNUM=2,ANT1CN=0, ANT1SRN=60,ANT1SN=255,ANT1N=R0A,ANTTYPE1=RXTX_MODE,ANT2CN=0,ANT2SRN=60,ANT2SN=255,ANT2N=R0B,ANTTYPE2=RXTX_MODE;

ADD CELL:LOCALCELLID=1,CELLNAME="CELL1",FREQBAND=41,ULEARFCNCFGIND=NOT_CFG,DLEARFCN=40340,ULBANDWIDTH=CELL_BW_N100,DLBANDWIDTH=CELL_BW_N100,CELLID=1,PHYCELLID=1,FDDTDDIND=CELL_TDD,SUBFRAMEASSIGNMENT=SA2,SPECIALSUBFRAMEPATTERNS=SSP5,ROOTSEQUENCEIDX=0,CUSTOMIZEDBANDWIDTHCFGIND=NOT_CFG,EMERGENCYAREAIDCFGIND=NOT_CFG,UEPOWERMAXCFGIND=NOT_CFG,MULTIRRUCELLFLAG=BOOLEAN_TRUE,MULTIRRUCELLMODE=MPRU_AGGREGATION, CPRICOMPRESSION=NORMAL_COMPRESSION,TXRXMODE=2T2R;

ADD EUSECTOREQMGROUP:LOCALCELLID=1,SECTOREQMGROUPID=1;
ADD EUSECTOREQMID2GROUP:LOCALCELLID=1,SECTOREQMGROUPID=1, SECTOREQMID=0;

Alright, now we can activate it:

//Modify the reference signal power.
MOD PDSCHCFG: LocalCellId=1, ReferenceSignalPwr=-81;

//Add an operator for the cell.
ADD CELLOP: LocalCellId=0, TrackingAreaId=0;

//Activate the cell.
ACT CELL: LocalCellId=1;

And lastly we can define some neighboring cells:

//Configure neighboring cells. 
ADD EUTRANINTERNFREQ: LocalCellId=1, DlEarfcn=3100, UlEarfcnCfgInd=NOT_CFG, CellReselPriorityCfgInd=NOT_CFG, SpeedDependSPCfgInd=NOT_CFG, MeasBandWidth=MBW100, PmaxCfgInd=NOT_CFG, QqualMinCfgInd=NOT_CFG;
ADD EUTRANEXTERNALCELL: Mcc="460", Mnc="02", eNodeBId=236, CellId=0, DlEarfcn=3100, UlEarfcnCfgInd=NOT_CFG, PhyCellId=236, Tac=33;
ADD EUTRANINTERFREQNCELL: LocalCellId=1, Mcc="460", Mnc="02", eNodeBId=236, CellId=0;

BSF Addresses

The Binding Support Function is used in 4G and 5G networks to allow applications to authenticate against the network, it’s what we use to authenticate for XCAP and for an Entitlement Server.

Rather irritatingly, there are two BSF addresses in use:

If the ISIM is used for bootstrapping the FQDN to use is:

bsf.ims.mncXXX.mccYYY.pub.3gppnetwork.org

But if the USIM is used for bootstrapping the FQDN is

bsf.mncXXX.mccYYY.pub.3gppnetwork.org

You can override this by setting the 6FDA EF_GBANL (GBA NAF List) on the USIM or equivalent on the ISIM, however not all devices honour this from my testing.