Category Archives: Notes

Virtualizing VMware to run inside Proxmox

Notes, Software, , ,

Like a lot of companies, we’re moving away from VMware, and in our case, shifting to Proxmox.

But that doesn’t mean we can get entirely away from VMware, but more that it’s not our hypervisor of choice anymore, and this means shifting our dev environments and lab off VMware to Proxmox first.

So today I sat down to try and shift everything to Proxmox, while keeping the VMware based VMs accessible until they can slowly die of bitrot.

A sane person would probably utilize Proxmox’s fancy new tool for migrating VMs from VMware to Proxmox, and it’s great, but in our case at least, it required logging into each VM and remapping NICs, etc, which is tricky on boxes I don’t have access to – Plus we need to keep some VMware capability for testing / labbing stuff up.

So I decided into install Proxmox onto the bare metal servers, and then create a VMware virtual machine inside the Proxmox stack, to host a VMware ESXi instance.

I started off inside VMware (Before installing any Proxmox) by moving all the VMs onto a single physical disk, which I then removed from the server, so as to not accidentally format the one disk I didn’t want to format.

Next I nuked the server and setup the new stack with Proxmox, which is a doddle, and not something I’ll cover.

Then I loaded a VMware ISO into Proxmox and started setting up the VM.

Now, nested virtualization is a real pain in the behind.

VMware doesn’t like not being run on bare metal, and it took me a good long amount of time to find the hardware config that I could setup in Proxmox that VMware would accept.

Create the VM in the Web UI; I found using a SATA drive worked while SCSI failed, so create a SATA based LVM image to use, and mount the datastore ISO.

Then edit /etc/pve/qemu-server/your_id.conf and replace the netX, args, boot and ostype to match the below:

args: -cpu host,+invtsc,-vmx
boot: order=ide2;net0
cores: 32
cpu: host
ide2: local:iso/VMware-VMvisor-Installer-8.0U2b-23305546.x86_64.iso,media=cdrom,size=620274K
memory: 128000
meta: creation-qemu=8.1.5,ctime=1717231453
name: esxi
net0: vmxnet3=BC:24:11:95:8A:50,bridge=vmbr0
numa: 0
ostype: l26
sata0: ssd-3600gb:vm-100-disk-1,size=32G
scsihw: pvscsi
smbios1: uuid=6066a9cd-6910-4902-abc7-dfb223042630
sockets: 1
vmgenid: 6fde194f-9932-463e-b38a-82d2e7e1f2dd

Now you can go and start the VM, but once you’ve got the VMware splash screen, you’ll need to press Shift + O to enter the boot options.

At the runweasle cdromBoot after it add allowLegacyCPU=true – This will allow ESXi to use our (virtual) CPU.

Next up you’ll install VMware ESXi just like you’ve probably done 100 times before (is this the last time?), and once it’s done installing, power off, we’ll have to make few changes to the VM definition file.

Then after install we need to change the boot order, by updating:

boot: order=sata0

And unmount the ISO:

ide2: none,media=cdrom

Now remember how I’d pulled the hard disk containing all the VMware VMs out so I couldn’t break it? Well, don’t drop that, because now we’re going to map that physical drive into the VM for VMware, so I can boot all those VMs.

I plugged in the drive and I used this to find the drive I’d just inserted:

fdisk -l

Which showed the drive I’d just added last, with it’s VMware file system.

So next we need to map this through the VM we just created inside Proxmox, so VMware inside Proxmox can access the VMware file system on the disk filled with all our old VMware VMs.

VM. VM. VM. The word has lost all meaning to me at this stage.

We can see the mount point of our physical disk; in our case is /dev/sdc so that’s what we’ll pass through to the VM.

Here’s my final .conf file for the Proxmox VM:

args: -cpu host
balloon: 1024
boot: order=sata0
cores: 32
ide2: none,media=cdrom
kvm: 1
memory: 128000
meta: creation-qemu=8.1.5,ctime=1717231453
name: esxi
net0: vmxnet3=BC:24:11:95:8A:50,bridge=vmbr0
numa: 0
ostype: l26
sata0: ssd-3600gb:vm-100-disk-1,size=32G
sata1: /dev/sda
scsihw: pvscsi
smbios1: uuid=6066a9cd-6910-4902-abc7-dfb223042630
sockets: 1
vmgenid: 6fde194f-9932-463e-b38a-82d2e7e1f2dd

Now I can boot up the VM, log into VMware and behold, our disk is visible:

Last thing we need to do is get it mounted as a VMware Datastore so we can boot all those VMs.

For that, we’ll enable SSH on the ESXi box and SSH into it.

We’ll use the esxcfg-volume -l command to get the UUID of our drive

[root@localhost:~] esxcfg-volume -l
Scanning for VMFS-6 host activity (4096 bytes/HB, 1024 HBs).
VMFS UUID/label: 6513c05a-43db8c20-db0f-c81f66ea471a/FatBoy
Can mount: Yes
Can resignature: Yes
Extent name: t10.ATA_____QEMU_HARDDISK___________________________QM00007_____________:1 range: 0 - 1716735 (MB)

In my case the drive is aptly named “FatBoy”, so we’ll grab the UUID up to the slash, and then use that as the Mount parameter:

[root@localhost:~] esxcfg-volume -M 6513c05a-43db8c20-db0f-c81f66ea471a
Persistently mounting volume 6513c05a-43db8c20-db0f-c81f66ea471a

And now, if everything has gone well, after logging into the Web UI, you’ll see this:

Then the last step is going to be re-registering all the VMs, you can do this by hand, by selecting the .vmx file and adding it.

Alternately, if you’re lazy like me, I wrote a little script to do the same thing:

[root@localhost:~] cat load_vms3.sh 


#!/bin/bash

# Datastore name
DATASTORE="FatBoi/"

# Log file to store the output
LOG_FILE="/var/log/register_vms.log"

# Clear the log file
> $LOG_FILE

echo "Starting VM registration process on datastore: $DATASTORE" | tee -a $LOG_FILE

# Check if datastore directory exists
if [ ! -d "/vmfs/volumes/$DATASTORE" ]; then
    echo "Datastore $DATASTORE does not exist!" | tee -a $LOG_FILE
    exit 1
fi

# Find all .vmx files in the datastore and register them
find /vmfs/volumes/$DATASTORE -type f -name "*.vmx" | while read VMX_PATH; do
    echo "Registering VM: $VMX_PATH" | tee -a $LOG_FILE
    vim-cmd solo/registervm "$VMX_PATH" | tee -a $LOG_FILE
done

echo "VM registration process completed." | tee -a $LOG_FILE

[root@localhost:~] sh load_vms3.sh

Now with all your VMs loaded, you should almost be ready to roll and power them all back on.

And here’s where I ran into another issue:

Luckily the Proxmox wiki had the answer:

For an Intel CPU here’s what you run on the Proxmox hypervisor:

echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf

But before we reboot the Hypervisor (Proxmox) we’ll have to reboot the VMware hypervisor too, because here’s something else to make you punch the screen:

Luckily we can fix this one globaly.

SSH into the VMware box, edit /etc/vmware/config.xml file and add:

vhv.enable = "FALSE"

Which will disable the performance counters.

Now power off the VMware VM, and reboot the Proxmox hypervisor, when it powers on again, Proxmox will allow nested virtualization, and when you power back on the VMware VM, you’ll have performance counters disabled, and then, you will be done.

Yeah, not a great use of my Saturday, but here we are…

DNS’ role in S8-Home Routing Roaming

EPC, IMS / VoLTE, LTE, Mobile Networks, Notes, RFCs & Standards, , , , ,

S8 Home Routing is a really simple concept, the traffic goes from the SGW in the visited PLMN to the PGW in the home PLMN, so the PCRF, OCS/OFCS, IMS, IP Addresses, etc, etc, are all in the home network, and this avoids huge amounts of complexity.

But in order for this to work, the visited network MME needs to find the PGW of the home network, and with over 700 roaming networks in commercial use, each one with potentially hundreds of unique APNs each routing to a different PGW, this is a tricky proposition.

If you’ve configured your PGW peers statically on your MME, that’s fine, but it doesn’t scale very well – And if you add an MVNO who wants their own PGW for serving their APN, well you’ll be adding some complexity there to, so what to do?

Well, the answer is DNS.

By taking the APN to be served, the home PLMN and the interface type desired, with some funky DNS queries, our MME can determine which PGW should be selected for a request.

Let’s take a look, for a UE from MNC XXX MCC YYY roaming into our network, trying to access the “IMS” APN.

Our MME knows the network code of the roaming subscriber from the IMSI is MNC XXX, MCC YYY, and that the UE is requesting the IMS APN.

So our MME crafts a DNS request for the NAPTR query for ims.apn.epc.mncXXX.mccYYY.3gppnetwork.org:

Because the domain is epc.mncXXX.mccYYY.3gppnetwork.org it’s routed to the authoritative DNS server in the home network, which sends back the response:

We’ve got a few peers to pick from, so we need to filter this list of Answers to only those that are relevant to us.

First we filter by the Service tag, whihc for each listed peer shows what services that peer supports.

But since we’re looking for S8, we need to find a peer who’s “Service” tag string contains:

x-3gpp-pgw:x-s8-gtp

We’re looking for two bits of info here, the presence of x-3gpp-pgw in the Service to indicate that this peer is a PGW and x-s8-gtp to indicate that this peer supports the S8 interface.

A service string like this:

x-3gpp-pgw:x-s5-gtp

Would be excluded as it only supports S5 not S8 (Even though they are largely the same interface, S8 is used in roaming).

It’s also not uncommon to see both services indicated as supported, in which case that peer could be selected too:

x-3gpp-pgw:x-s5-gtp:x-s8-gtp

(The answers in the screenshot include :x-gp which means the PGWs advertised are also co-located with a GGSN)

So with our answers whittled down to only those that meet our needs, we next use the Order and the Preference to pick our best candidate, this is the same as regular DNS selection logic.

From our candidate, we’ve also got the Regex Replacement, which allows our original DNS request to be re-written, which allows us to point at a single peer.

In our answer, we see the original request ims.apn.epc.mncXXX.mccYYY.3gppnetwork.org is to be re-written to topon.lb1.pgw01.epc.mncXXX.mccYYY.3gppnetwork.org.

This is the FQDN of the PGW we should use.

Now we know the FQND we should use, we just do an A-Record lookup (Or AAAA record lookup if it is IPv6) for that peer we are targeting, to turn that FQDN into an IP address we can use.

And then in comes the response:

So now our MME knows the IP of the PGW, it can craft a Create Session request where the F-TEID for the S8 interface has the PGW IP set on it that we selected.

For more info on this TS 129.303 (Domain Name System Procedures) is the definitive doc, but the GSMA’s IR.88 “LTE and EPC Roaming Guidelines” provides a handy reference.

vCenter – Partition Folder Isolation

Notes, Security, Software,

While vCenter doesn’t really do contexts / mutli-tenants / VPCs like the hyperscalers, there are simple (ish) ways to do context separation inside VMware vCenter.

This means you can have a user who only has access to say a folder of VMs, but not able to see VMs outside of that folder.

Create a new Role inside vCenter from Administration -> Roles -> Add

Give the role all Virtual Machine privileges:

Create a new account (Can be on AD if you’re not using Local accounts, this is just our lab so I’ve created it as a Local account) for the user. We’re using this account for Ansible so we’ve used “Demo-Ansible-User” as the username

Now create a folder for the group of VMs, or pick an existing folder you want to give access to.

Right click on the folder and select “Add Permission”.

We give permission to that user with that role on the folder and make sure you tick “Propagate to children” (I missed this step before and had to repeat it):

If you are using templates, make sure the template is either in the folder, or apply the same permission to the template, by right clicking on it, Add Permission, same as this.

Finally you should be able to log in as that user and see the template, and clone it from the web UI, or create VMs but only within that folder.

CGrateS – ActionTriggers

CGrateS, EPC, IMS / VoLTE, Mobile Networks, Notes, SDM, Software, Uncategorized, VoIP, , , ,

In our last post we looked at Actions and ActionPlans, and one of the really funky things we can do is setting ActionPlans to trigger on a time schedule or setting ActionTriggers to trigger on an event.

We’re going to build on the examples we had on the last post, so we’ll assume your code is up to the point where we’ve added a Signup Bonus to an account, using an ActionPlan we assigned when creating the account.

In this post, we’re going to create an action that charges $6, called “Action_Monthly_Charge“, and tie it to an ActionPlan called “ActionPlan_Monthly_Charge“, but to demo how this works rather than charging this Monthly, we’re going to charge it every minute.

Then with our balances ticking down, we’ll set up an ActionTrigger to trigger when the balance drops below $95, and alert us.

Defining the Monthly Charge Action

The Action for the Monthly charge will look much like the other actions we’ve defined, except the Identifier is *debit so we know we’re deducting from the balance, and we’ll log to the CDRs table too:

# Action to add a Monthly charge of $6
Action_Monthly_Charge = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Monthly_Charge",
          "Actions": [
              {
                'Identifier': '*debit',
                'BalanceType': '*monetary',
               'Units': 6,
               'Id': 'Action_Monthly_Charge_Debit',
               'Weight': 70},
              {
                  "Identifier": "*log",
                  "Weight": 60,
                  'Id' : "Action_Monthly_Charge_Log"
              },
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Recurring Charge\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Monthly_Charge))

Next we’ll need to wrap this up into an ActionPlan, this is where some of the magic happens. Inside the action plan we can set a once off time, or a recurring time, kinda like Cron.

We’re setting the time to *every_minute so things will happen quickly while we watch, this action will get triggered every 60 seconds. In real life of course, for a Monthly charge, we’d want to trigger this Action monthly, so we’d set this value to *monthly. If we wanted this to charge on the 2nd of the month we’d set the MonthDays to “2”, etc, etc.

# # Create ActionPlan using SetActionPlan to trigger the Action_Monthly_Charge
SetActionPlan_Daily_Action_Monthly_Charge_JSON = {
    "method": "ApierV1.SetActionPlan",
    "params": [{
        "Id": "ActionPlan_Monthly_Charge",
        "ActionPlan": [{
            "ActionsId": "Action_Monthly_Charge",
            "Years": "*any",
            "Months": "*any",
            "MonthDays": "*any",
            "WeekDays": "*any",
            "Time": "*every_minute",
            "Weight": 10
        }],
        "Overwrite": True,
        "ReloadScheduler": True
    }]
}
pprint.pprint(CGRateS_Obj.SendData(
    SetActionPlan_Daily_Action_Monthly_Charge_JSON))

Alright, but now what’s going to happen?

If you think the accounts will start getting debited every 60 seconds after applying this, you’d be wrong, we need to associate this ActionPlan with an Account first, this is how we control which accounts get which ActionPlans tied to them, to do this we’ll use the SetAccout API again we’ve been using to create accounts:

# Create the Account object inside CGrateS & assign ActionPlan_Signup_Bonus and ActionPlan_Monthly_Charge
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionPlanIds": ["ActionPlan_Signup_Bonus", "ActionPlan_Monthly_Charge"],
            "ActionPlansOverwrite": True,
            "ReloadScheduler":True
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

So what’s going to happen if we run this?

Well, for starters the ActionPlan named “ActionPlan_Signup_Bonus” is going to be triggered, as in the ActionPlan it’s Timing is set to *asap, so CGrateS will apply the corresponding Action (“Action_Add_Signup_Bonus“) right away, which will credit the account $99.

But a minute after that, we’ll trigger the ActionPlan named “ActionPlan_Monthly_Charge”, as the timing for this is set to *every_minute, when the Action “Action_Monthly_Charge” is triggered, it’s going to be deducting $6 from the balance.

We can check this by using the GetAccount API:

# Get Account Info
pprint.pprint(CGRateS_Obj.SendData({'method': 'ApierV2.GetAccount', 'params': [
              {"Tenant": "cgrates.org", "Account": str(Account)}]}))

You should see a balance of $99 to start with, and then after 60 seconds, it should be down to $93, and so on.

{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-17T14:57:20.71493633+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '3a896369-8107-4e32-bcef-2d078c981b8a',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-17T14:57:21.802521707+11:00'}}

Triggering Actions based on Balances with ActionTriggers

Okay, so we’ve set up recurring charges, now let’s get notified if the balance drops below $95, we’ll start, like we have before, with defining an Action, this will log to the CDRs table, HTTP post and write to syslog:


#Define a new Action to send an HTTP POST
Action_HTTP_Notify_95 = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_HTTP_Notify_95",
          "Actions": [
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Balance dipped below $95\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
              {
                  "Identifier": "*http_post_async",
                  "ExtraParameters": "http://10.177.2.135/95_remaining",
                  "ExpiryTime": "*unlimited",
                  "Weight": 700
              },
              {
                  "Identifier": "*log",
                  "Weight": 1200
              }
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_HTTP_Notify_95))

Now we’ll define an ActionTrigger to check if the balance is below $95 and trigger our newly created Action (“Action_HTTP_Notify_95“) when that condition is met:


#Define ActionTrigger
ActionTrigger_95_Remaining_JSON = {
    "method": "APIerSv1.SetActionTrigger",
    "params": [
        {
            "GroupID" : "ActionTrigger_95_Remaining",
            "ActionTrigger": 
                {
                    "BalanceType": "*monetary",
                    "Balance" : {
                        'BalanceType': '*monetary',
                        'ID' : "*default",
                        'BalanceID' : "*default",
                        'Value' : 95,
                        },
                    "ThresholdType": "*min_balance",
                    "ThresholdValue": 95,
                    "Weight": 10,
                    "ActionsID" : "Action_HTTP_Notify_95",
                },
            "Overwrite": True
        }
    ]
}
pprint.pprint(CGRateS_Obj.SendData(ActionTrigger_95_Remaining_JSON))

We’ve defined the ThresholdType of *min_balance, but we could equally set this to ThresholdType to *max_balance, *balance_expired or trigger when a certain Counter has been triggered enough times.

Adding an ActionTrigger to an Account

Again, like the ActionPlan we created before, before the ActionTrigger we just created will be used, we need to associate it with an Account, for this we’ll use the AddAccountActionTriggers API, specify the Account and the ActionTriggerID for the ActionTrigger we just created.


#Add ActionTrigger to Account 
Add_ActionTrigger_to_Account_JSON = {
    "method": "APIerSv1.AddAccountActionTriggers",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionTriggerIDs": ["ActionTrigger_95_Remaining"],
            "ActionTriggersOverwrite": True
        }
    ]
}
pprint.pprint(CGRateS_Obj.SendData(Add_ActionTrigger_to_Account_JSON))

If we run this all together, creating the account with the “ActionPlan_Signup_Bonus” will give the account a $99 Balance. But after 60 seconds, “ActionPlan_Monthly_Charge” will kick in, and every 60 seconds after that, at which point the balance will get to below $95 when CGrateS will trigger the ActionTrigger “ActionTrigger_95_Remaining” and get the HTTP POST to the HTTP endpoint and log entry:

We can check on this using the ApierV2.GetAccount method, where we’ll see the ActionTrigger we just defined.

Checking out the LastExecutionTime we can see if the ActionTrigger been triggered or not.

So using this technique, we can notify a customer when they’ve used a certain amount of their balance, but we can lock out Accounts who have spent more than their allocated spend limit by setting an Action that suspends the Account once it reaches a certain level. We notify customers when balance expires, or if a certain number of counters has been triggered.

As always I’ve put all the code used in this example, from start to finish, up on GitHub.

CGrateS – Actions & Action Plans

CGrateS, Mobile Networks, Notes, Python, SDM, Software, VoIP, ,

In our last post we added a series of different balances to an account, these were actions we took via the API specifically to add a balance.

But there’s a lot more actions we may want to do beyond just adding balance.

CGrateS has the concept of “Actions” which are, as the name suggests, things we want to do to the system.

Some example Actions would be:

  • Adding / Deducting / Resetting a balance
  • Adding a CDR log
  • Enable/Disable an account
  • Sending HTTP POST request or email notification
  • Deleting / suspending account
  • Transferring balances

We can run these actions on a timed basis, or when an event is triggered, and group Actions together to run multiple actions via an ActionTrigger, this means we can trigger these Actions, not just by sending an API request, but based on the state of the subscriber / account.

Let’s look at some examples,

We can define an Action named “Action_Monthly_Fee” to debit $12 from the monetary balance of an account, and add a CDR with the name “Monthly Account Fee” when it does so.
We can use ActionTriggers to run this every month on the account automatically.

We can define an Action named “Usage_Warning_10GB” to send an email to the Account owner to inform them they’ve used 10GB of usage, and use ActionTriggers to send this when the customer has used 10GB of their *data balance.

Using Actions

Note: The Python script I’ve used with all the examples in this post is available on GitHub here.

Let’s start by defining an Account, just as we have before:

# Create the Account object inside CGrateS
Account = "Nick_Test_123"
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account)
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Let’s start basic; to sweeten the deal for new Accounts, we’ll give them $99 of balance to use in the first month they have the service. Rather than hitting the AddBalance API, we’ll define an Action named “Action_Add_Signup_Bonus” to credit $99 of monetary balance to an account.

If you go back to our last post, you should know what we’d need to do to add this balance manually with the AddBalance API, but let’s look at how we can create the same balance add functionality using Actions:

#Add a Signup Bonus of $99 to the account with type *monetary expiring a month after it's added
Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 99,
                  "ExpiryTime": "*month",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 1200,
                  "ExtraParameters": "",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 10
              }
]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Alright, this should look pretty familiar if you’ve just come from Account Balances.
You’ll notice we’re no longer calling, SetBalance, we’re now calling SetActions, to create the ActionsId with the name “Action_Add_Signup_Bonus“.
In “Action_Add_Signup_Bonus” we’ve got an actions we’ll do when “Action_Add_Signup_Bonus” is called.
We can define multiple actions, but for now we’ve only got one action defined, which has the Identifier (which defines what the action does) set to *topup to add balance.
As you probably guessed, we’re triggering a top up, and setting the BalanceId, BalanceType, Units, ExpiryTime and BalanceWeight just as we would using SetBalance to add a balance.

So how do we use the Action we just created? Well, there’s a lot of options, but let’s start with the most basic – Via the API:

# Trigger ExecuteAction
Account_Action_trigger_JSON = {"method": "APIerSv1.ExecuteAction", "params": [
    {"Tenant": "cgrates.org", "Account": str(Account), "ActionsId": "Action_Add_Signup_Bonus"}]}
pprint.pprint(CGRateS_Obj.SendData(Account_Action_trigger_JSON))

Boom, we’ve called the ExecuteAction API call, to execute the Action named “Action_Add_Signup_Bonus“.

We can check on this with GetAccount again and check the results:

# Get Account Info
pprint.pprint(CGRateS_Obj.SendData({'method': 'ApierV2.GetAccount', 'params': [
              {"Tenant": "cgrates.org", "Account": str(Account)}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-15T10:27:52.865119544+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '01cfb471-ba38-453a-b0e2-8ddb397dfe9c',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-15T10:27:52.865144268+11:00'}}

Great start!

Making Actions Useful

Well congratulations, we took something we previously did with one API call (SetBalance), and we did it with two (SetAction and ExcecuteAction)!

But let’s start paying efficiency dividends,

When we add a balance, let’s also add a CDR log event so we’ll know the account was credited with the balance when we call the GetCDRs API call.

We’d just modify our SetActions to include an extra step:

Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
...
              }, 
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Your sign up Bonus\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 10
              }
]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Boom, now we’ll get a CDR created when the Action is triggered.

But let’s push this a bit more and add some more steps in the Action:

As well as adding balance and putting in a CDR to record what we did, let’s also send a notification to our customer via an HTTP API (BYO customer push notification system) and log to Syslog what’s going on.

# Add a Signup Bonus of $99 to the account with type *monetary expiring a month after it's added
Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 99,
                  "ExpiryTime": "*month",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 1200,
                  "ExtraParameters": "",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 90
              },
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Your sign up Bonus\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
              {
                  "Identifier": "*http_post_async",
                  "ExtraParameters": "http://10.177.2.135/example_endpoint",
                  "ExpiryTime": "*unlimited",
                  "Weight": 70
              },
              {
                  "Identifier": "*log",
                  "Weight": 60
              }
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Phew! That’s a big action, but if we execute the action again using ExecuteAction, we’ll get all these things happening at once:

Okay, now we’re getting somewhere!

ActionPlans

Having an Action we can trigger manually via the API is one thing, but being able to trigger it automatically is where it really comes into its own.

Let’s define an ActionPlan, that is going to call our Action named “Action_Add_Signup_Bonus” as soon as the ActionPlan is assigned to an Account.

# Create ActionPlan using SetActionPlan to trigger the Action_Signup_Bonus ASAP
SetActionPlan_Signup_Bonus_JSON = {
    "method": "ApierV1.SetActionPlan",
    "params": [{
        "Id": "ActionPlan_Signup_Bonus",
        "ActionPlan": [{
            "ActionsId": "Action_Add_Signup_Bonus",
            "Years": "*any",
            "Months": "*any",
            "MonthDays": "*any",
            "WeekDays": "*any",
            "Time": "*asap",
            "Weight": 10
        }],
        "Overwrite": True,
        "ReloadScheduler": True
    }]
}
pprint.pprint(CGRateS_Obj.SendData(SetActionPlan_Signup_Bonus_JSON))

So what have we done here? We’ve made an ActionPlan named “Action_Add_Signup_Bonus”, which, when associated with an account, will run the Action “Action_Add_Signup_Bonus” as soon as it’s tied to the account, thanks to the Time “*asap“.

Now if we create or update an Account using the SetAccount method, we can set the ActionPlanIds to reference our “ActionPlan_Signup_Bonus” and it’ll be triggered straight away.

# Create the Account object inside CGrateS
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionPlanIds": ["ActionPlan_Signup_Bonus"],
            "ActionPlansOverwrite": True,
            "ReloadScheduler":True
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Now if we were to run a GetAccount API call, we’ll see the Account balance assigned that was created by the action Action_Add_Signup_Bonus which was triggered by ActionPlan assigned to the account:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-16T12:41:02.530985381+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '7bdbee5c-0888-4da2-b42f-5d6b8966ee2d',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-16T12:41:12.7236096+11:00'}}

But here’s where it gets interesting, in the ActionPlan we just defined the Time was set to “*asap“, which means the Action is triggered as soon as it was assigned to the account, but if we set the Time value to “*monthly“, the Action would get triggered every month, or *every_minute to trigger every minute, or *month_end to trigger at the end of every month.

Code for these examples is available here.

I’m trying to keep these posts shorter as there’s a lot to cover. Stick around for our next post, we’ll look at some more ActionTriggers to keep decreasing the balance of the account, and setting up ActionTriggers to send a notification to the customer to tell them when their balance is getting low, or any other event based Action you can think of!

CGrateS – Accounts & Balances

CGrateS, IMS / VoLTE, Mobile Networks, Notes, Software, VoIP, , ,

So far we’ve used CGrateS to rate a basic CDR and get a cost for it, but in the real world, we’d usually associate the cost with an account, which would represent a business or a person, who will ultimately be charged for using the service.

Note: I’ve put the code for all this in Github, if you’ve got issues following along, or don’t want to copy and paste the code from the website, you can grab the code here.

Creating an Account

Let’s start off by creating an account inside CGrateS – This is kinda pointless, but we’ll talk more about that later:

#Create the Account object inside CGrateS 
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123"
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Running this onto the API should create an account named “Nick_Test_123”, but let’s confirm that’s the case:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))

Running this will give us the information about the account we just created:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
OrderedDict([('id', None),
             ('result',
              OrderedDict([('ID', 'cgrates.org:Nick_Test_123'),
                           ('BalanceMap', None),
                           ('UnitCounters', None),
                           ('ActionTriggers', None),
                           ('AllowNegative', False),
                           ('Disabled', False),
                           ('UpdateTime',
                            '2023-10-09T16:53:37.524466041+11:00')])),
             ('error', None)])

That was easy!

There’s not really much to see on our account at this stage, other than the UpdateTime, there’s nothing really going on, we don’t have any Balances.

Adding Balance for Voice

Accounts exist for spending, so let’s add a balance to this account to send from.

We’ll use the SetBalance API to create a new balance with 5 minutes of talk time, that we can use for making a call, and talking, for (you guessed it) – 5 minutes, so and we’ll use the balance “5_minute_voice_balance” that we’ll create:

#Add a balance to the account with type *voice with 5 minutes of Talk Time
Create_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "5_minute_voice_balance",
                "Value": "5m",
                "Weight": 25
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Voice_Balance_JSON))

Now if we run the GetAccount API command again, we should see the new balance we just created:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': '37423d07-d99a-40b1-851a-981c3df02cb3',
                                       'Value': 300000000000,
                                       'Weight': 25}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T17:58:23.801531205+11:00'}}

So now we’ve got a new balance named ‘5_minute_voice_balance‘:

  • The type is *voice, because this balance is storing talk time
  • The weight of this balance is 25, this means this balance should take priority over any balances with a lower value than 25 (that’s right, we can (and will) do tiered balances)
  • The value is 300000000000 nanoseconds, which equates to 5 minutes (yes, that’s the correct number of zeros)

Okay, but Nick_Test_123 probably wants to make some calls, so let’s generate a 2.5 minute call event and check out what happens.


#Generate a new call event for a 2.5 minute (150 second) call
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "150s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

Alright, now we’ve got a call event, let’s call the GetAccount API again to check the balance:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': '37423d07-d99a-40b1-851a-981c3df02cb3',
                                       'Value': 150000000000,
                                       'Weight': 25}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T17:58:23.80925546+11:00'}}

And there you have it, we’ve used 150 seconds of our 300 second (5 minutes) of talk time in this balance, leaving with us 150000000000 nanoseconds (150 seconds) remaining!

And with that progress, now is a great time to pause and talk about some theory that’s really important to grasp!

Balance Types

But *voice is just one balance type – We can support multiple balance types; we’ve just given a balance of *voice for talk time, but we could also give a credit to the balance for *sms or *data, you name it (*generic) and cash (*monetary) and we can have multiple separate balances for each.

This means we can have one account with something like:

  • 100 minutes of Local / National Calls (Expires at the end of the month)
  • 40 minutes of Mobile Calls (Expires 24 hours after it’s been created)
  • 80 minutes of Mobile Calls (During “Happy Hour” from 6 to 7pm)
  • 50 minutes of International Calls (Expires in 30 days)

And not just voice balance, the same account could also have:

  • 1GB of Data usage
  • 50 SMS to on-net destinations
  • $200 of Cash (expiring never)

Phew! That’s a lot of balances, but we can do it all through CGrateS!

What Balance to Use

So if we’ve got a stack of balances defined, how does CGrateS know what balance to use?

Firstly CGrateS is going to evaluate the BalanceType, this is set on events, so if we get an event for *data CGrateS will check out the balances available for *data, and evaluate the balances by Weight, with the highest weight evaluated first.
If we get to the end of all the available balances for that BalanceType, CGrateS then evaluates *generic and then *monetary balances, again, ordered by Weight.

We can set what balance gets used based on the Destination; using DestinationIDs we can filter the Balance to only apply for calls to Local/National numbers, so a call to an International destination won’t use that balance.

We can also set an Expiry on the Balance, for example we can give a customer 30 days to use the balance, after which it expires and can’t be used, likewise we can set Timings so enable scenarios like a “Happy hour” with extra calls between 6pm and 7pm.

When we define a balance we can also set the Blocker flag to True, if this is set, it means CGrateS will not look evaluate any balances after reaching that balance.

Adding a Balance for Local / National & Mobile Calls

Let’s jump back into the practice, and define two new Balances; one for Local/National calls, and another for Mobile calls.
But first we’ll need to know what destinations are mobiles and what are local/national (fixed). We’ve covered setting Destinations previously, so let’s set up the Destinations:


CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_Fixed","Prefixes":["612", "613", "617", "618"]}]})
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_Mobile","Prefixes":["614"]}]})
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_TollFree","Prefixes":["6113", "6118"]}]})
#Load TariffPlan we just defiend from StorDB to DataDB
print(CGRateS_Obj.SendData({"method":"APIerSv1.LoadTariffPlanFromStorDb","params":[{"TPid":"cgrates.org","DryRun":False,"Validate":True,"APIOpts":None,"Caching":None}],"id":0}))

Alright, now let’s add a balance for our local/national (fixed) calls.

To do this, we’ll add two new balances, but we’ll need to differentiate this from the 5_minute_voice_balance we created earlier, and to achieve this we weill:

  • Set a higher Weight value than we have set on 5_minute_voice_balance (25) so this balance will get consumed before 5_minute_voice_balance does
  • Set the DestinationIDs to match the destinations (Dest_AU_Mobile for Mobile and Dest_AU_Fixed for Local/National) we want the balance to apply to

ProTip: When you we create our Balance we can set what Destinations we want to use this balance for, if you want to specify multiple balances, we can do it by setting the Balance names as a string delimited by semicolons, like “DestinationIDs”: “Dest_AU_Fixed;Dest_AU_Mobile;Dest_AU_TollFree”

We’ll also set a balance expiry, which we’ll cover shortly, but now let’s define out 100 minutes for Local/National expiring at the end of the month:

#Add a balance to the account with type *voice with 100 minutes of talk time to Local / National Destinations expiring at the end of the month
Create_Local_National_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "Local_National_100_minutes_voice_balance",
                "Value": "100m",
                "ExpiryTime": "*month_end",
                "Weight": 60,
                "DestinationIDs": "Dest_AU_Fixed",
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Local_National_Voice_Balance_JSON))

We’ll also add our 24 hours to use to use 40 minutes of talk to mobiles, and a GetAccount to check the result:

#Add a balance to the account with type *voice with 40 minutes of talk time to Mobile Destinations expiring in 24 hours
Create_Mobile_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "Mobile_40_minutes_voice_balance",
                "Value": "40m",
                "ExpiryTime": "*daily",
                "Weight": 60,
                "DestinationIDs": "Dest_AU_Mobile",
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Mobile_Voice_Balance_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

Alright, let’s try running that:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'ad9d8bdd-64df-430f-af9d-3fc0410fd16b',
                                       'Value': 150000000000,
                                       'Weight': 25},
                                      {'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': {'Dest_AU_Fixed': True},
                                       'Disabled': False,
                                       'ExpirationDate': '2023-10-31T23:59:59+11:00',
                                       'Factor': None,
                                       'ID': 'Local_National_100_minutes_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'e4a2c211-8112-4e40-b3e6-250863404cc9',
                                       'Value': 6000000000000,
                                       'Weight': 60},
                                      {'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': {'Dest_AU_Mobile': True},
                                       'Disabled': False,
                                       'ExpirationDate': '2023-10-15T18:15:11.521636734+11:00',
                                       'Factor': None,
                                       'ID': 'Mobile_40_minutes_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'd4cbf6d8-50a5-4c97-82c2-dfe9936ae8d1',
                                       'Value': 2400000000000,
                                       'Weight': 60}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T18:15:11.524242437+11:00'}}

Alright! We now have 3 balances defined!

Notice in the API in the expiry I put *daily and *month_end, but in the output it’s got a real date and time (I wrote this 14/10/23 around 07:00 UTC, hence why those dates are what they are).
I could have specified the date and time in the API of a specific time I wanted the balance to expire (You can too, just replace “*daily” with “2024-01-01T00:00:00Z” for example), but that’s a pain in the butt, especially considering most of the time these values will be something common.
The *month_end is a special “meta” value, there’s a heap of these that allow us to do things like “current time + 20 minutes” (+20m), this time next month (*monthly), “this time tomorrow” (*daily), or “this time next week” (+168h) – You can find the full list of special dates here.

From a product perspective, setting an expiry on balances means we can set credit to expire 2 years after the subscriber tops up, but the same logic can be used so a subscriber could purchase a 7 day addon pack, that expires in 7 days, or a monthly plan can automatically expire in 30 days.

Now if we call the ProcessExternalCDR API again with a call to a Mobile and a Fixed number, we’ll see the respective balances get deducted.


#Generate a new call event for a 2.5 minute (150 second) call to a mobile number
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61412341234",
            "Destination": "61412341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "30s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

#Generate a new call event for a 2.5 minute (150 second) call to a fixed line local/national number
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61212341234",
            "Destination": "61212341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "30s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

We should see the minutes reduced by 30 seconds for our Local_National_100_minutes_voice_balance and Mobile_40_minutes_voice_balance balances, while our 5_minute_voice_balance hasn’t been touched.

{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":"None",
   "Disabled":false,
   "ExpirationDate":"0001-01-01T00:00:00Z",
   "Factor":"None",
   "ID":"5_minute_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"29f21735-1d62-49b1-9c53-80eab6f7b005",
   "Value":150000000000,
   "Weight":25
},
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Fixed":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-31T23:59:59+11:00",
   "Factor":"None",
   "ID":"Local_National_100_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"54db4f60-342e-4738-aaf1-a1304badc41d",
   "Value":5970000000000,
   "Weight":60
},
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Mobile":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-15T18:32:34.888821482+11:00",
   "Factor":"None",
   "ID":"Mobile_40_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"501eb00e-e947-4675-926f-080911e66897",
   "Value":2370000000000,
   "Weight":60
}

One last thing we’ll try before we end, our Mobile_40_minutes_voice_balance has still got 39.5 minutes left, and our 5_minute_voice_balance has still got minutes remaining, so if we try and make a call that’s 2450 seconds (~41 minutes), we should consume all the remaining minutes in Mobile_40_minutes_voice_balance and the go onto consume the remaining 1 minute out of 5_minute_voice_balance.

Let’s test this theory!

#Generate a new call event for a 42 minute call to a mobile to use all of our Mobile_40_minutes_voice_balance and start consuming 5_minute_voice_balance
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61412341234",
            "Destination": "61412341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "2450s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

Let’s check the output:

{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":"None",
   "Disabled":false,
   "ExpirationDate":"0001-01-01T00:00:00Z",
   "Factor":"None",
   "ID":"5_minute_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"29f21735-1d62-49b1-9c53-80eab6f7b005",
   "Value":70000000000,
   "Weight":25
},
...
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Mobile":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-15T18:44:18.161474861+11:00",
   "Factor":"None",
   "ID":"Mobile_40_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"501eb00e-e947-4675-926f-080911e66897",
   "Value":0.0161939859,
   "Weight":60
}

Boom, there we have it! Used all of the minutes in Mobile_40_minutes_voice_balance and started eating into the 5_minute_voice_balance.

Alright, that was a long post! Sorry about that, and props for making it to the end, still so much to learn about CGrateS.

GNS3 vCenter / ESXi – Allow Traffic

Notes, Software, , ,

The other day I setup GNS3 in the lab for some testing, we run vCenter for our server workloads, so I chucked the OVA on there.

One issue I ran into is that when linking a Cloud Component to a router, I simply could not get a path in/out of the router, I wasn’t learning MAC addresses and my ARP requests were going unanswered.

Wireshark showed the ARP requests going down that interface, and broadcast traffic from the rest of the network, so what gives?

The answer was pretty simple, on the vHost itself I needed to enable Promiscuous mode to allow L2 addresses that aren’t the VM, to be sent from within the VM.

Under Networking -> Port Groups -> the NICs you have assigned in GNS3:

Make sure Promiscuous mode, MAC address changes and Forged transmits are allowed – By default they’re denied on the vSwitch which it inherits from.

There’s obviously security concerns here, so think before you do, but that should have packets flowing.

Logging DSL Line Rate & SNR on a Draytek Modem

Australian Telco, Notes, Software, , , , ,

I am connected on a VDSL line, not by choice, but here we are.
DSL is many things, but consistent it not one of them, so I thought it’d be interesting to graph out the SNR and the line rate of the connection.

This is an NBN FTTN circuit, I run Mikrotiks for the routing, but I have a Draytek Vigor 130 that acts as a dumb modem and connects to the Tik.

Draytek exposes this info via SNMP, but the OIDs / MIBs are not part of the standard Prometheus snmp_exporter, so I’ve added them into snmp_exporter.yaml and restarted the snmp_exporter service.

draytek:
  walk:
  - 1.3.6.1.2.1.10.94.1.1.3.1.8
  - 1.3.6.1.2.1.10.94.1.1.3.1.4
  - 1.3.6.1.2.1.10.94.1.1.5.1.2.4
  - 1.3.6.1.2.1.10.94.1.1.4.1.2.4
  metrics:
  - name: Draytek_dsl_LineRate
    oid: 1.3.6.1.2.1.10.94.1.1.3.1.8
    type: gauge
    help: adslAtucCurrAttainableRate

  - name: Draytek_dsl_Linerate_Down
    oid: 1.3.6.1.2.1.10.94.1.1.4.1.2.4
    type: gauge
    help: Draytek_dsl_Linerate_Down

  - name: Draytek_dsl_Linerate_Up
    oid: 1.3.6.1.2.1.10.94.1.1.5.1.2.4
    type: gauge
    help: Draytek_dsl_Linerate_Up

  - name: Draytek_dsl_SNR
    oid: 1.3.6.1.2.1.10.94.1.1.3.1.4
    type: gauge
    help: adslAturCurrSnrMgn

Then I added this as a target in Prometheus:

  - job_name: Draytek Logger
    scrape_interval: 1m
    scrape_timeout: 30s
    static_configs:
          - targets: ['10.0.2.1']  # My modem

    metrics_path: /snmp
    params:
      module: ['draytek']
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: localhost:9116  # SNMP exporter address

And then from Grafana I can quantify exactly how bad my line is over time!

Only two dropouts today!

Mobile IPv6 Tax?

5G SA, EPC, LTE, Mobile Networks, Notes, RFCs & Standards, ,

Recently a Tweet from Dean Bubly got me thinking about how data is charged in cellular:

In the cellular world, subscribers are charged for data from the IP, transport and applications layers; this means you pay for the IP header, you pay for the TCP/UDP header, and you pay for the contents (the cat videos it contains).

This also means if an operator moves mobile subscribers from IPv4 to IPv6, there’s an extra 20 bytes the customer is charged for for every packet sent / received, which the customer is charged for – This is because the IPv6 header is longer than the IPv4 header.

Source: ServerFault - https://serverfault.com/questions/547768/ipv4-header-vs-ipv6-header-size

In most cases, mobile subs don’t get a choice as to if their connection is IPv4 or IPv6, but on a like for like basis, we can say that if a customer moves is on IPv6 every packet sent/received will have an extra 20 bytes of data consumed compared to IPv4.

This means subscribers use more data on IPv6, and this means they get charged for more data on IPv6.

For IoT applications, light users and PAYG users, this extra 20 bytes per packet could add up to something significant – But how much?

We can quantify this, but we’d need to know the number of packets sent on average, and the quantity of the data transferred, because the number of packets is the multiplier here.

So for starters I’ve left a phone on the desk, it’s registered to the network but just sitting in Idle mode – This is an engineering phone from an OEM, it’s just used for testing so doesn’t have anything loaded onto it in terms of apps, it’s not signed into any applications, or checking in the background, so I thought I’d try something more realistic.

So to get a clearer picture, I chucked a SIM in my regular everyday phone I use personally, registered it to the cellular lab I have here. For the next hour I sniffed the GTP traffic for the phone while it was sitting on my desk, not touching the phone, and here’s what I’ve got:

Overall the PCAP includes 6,417,732 bytes of data, but this includes the transport and GTP headers, meaning we can drop everything above it in our traffic calculations.

Everything except the data encapsulated in GTP can be dropped

For this I’ve got 14 bytes of ethernet, 20 bytes IP, 8 bytes UDP and 5 bytes for TZSP (this is to copy the traffic from the eNB to my local machine), then we’ve got the transport from the eNB to the SGW, 14 bytes of ethernet again, 20 bytes of IP , 8 bytes of UDP and 8 bytes of GTP then the payload itself. Phew.
All this means we can drop 97 bytes off every packet.

We have 16,889 packets, 6,417,732 bytes in total, minus 97 bytes from each gives us 1,638,233 of headers to drop (~1.6MB) giving us a total of 4.556 MB traffic to/from the phone itself.

This means my Android phone consumes 4.5 MB of cellular data in an hour while sitting on the desk, with 16,889 packets in/out.

Okay, now we’re getting somewhere!

So now we can answer the question, if each of these 16k packets was IPv6, rather than IPv4, we’d be adding another 20 bytes to each of them, 20 bytes x 16,889 packets gives 337,780 bytes (~0.3MB) to add to the total.

If this traffic was transferred via IPv6, rather than IPv4, we’d be looking at adding 20 bytes to each of the 16,889 packets, which would equate to 0.3MB extra, or about 7% overhead compared to IPv4.

But before you go on about what an outrage this IPv6 transport is, being charged for those extra bytes, that’s only one part of the picture.

There’s a reason operators are finally embracing IPv6, and it’s not to put an extra 7% of traffic on the network (I think if you asked most capacity planners, they’d say they want data savings, not growth).

IPv6 is, for lack of a better term, less rubbish than IPv4.

There’s a lot of drivers for IPv6, and some of these will reduce data consumption.
IPv6 is actually your stuff talking directly to the remote stuff, this means that we don’t need to rely on NAT, so no need to do NAT keepalives, and opening new sessions, which is going to save you data. If you’re running apps that need to keep a connection to somewhere alive, these data savings could negate your IPv6 overhead costs.

Will these potential data savings when using IPv6 outweigh the costs?

That’s going to depend on your use case.

If you’ve extremely bandwidth / data constrained, for example, you have an IoT device on an NTN / satellite connection, that was having to Push data every X hours via IPv4 because you couldn’t pull data from it as it had no public IP, then moving it to IPv6 so you can pull the data on the public IP, on demand, will save you data. That’s a win with IPv6.

If you’re a mobile user, watching YouTube, getting push notifications and using your phone like a normal human, probably not, but if you’re using data like a normal user, you’ve probably got a sizable data allowance that you don’t end up fully consuming, and the extra 20 bytes per packet will be nothing in comparison to the data used to watch a 2k video on your small phone screen.

CGrateS MySQL Rounding Error

CGrateS, Notes, VoIP, , ,

I put my rates in with a stack of decimal points, because accuracy matters!

But when I manually calculated the outputted costs associated with each transaction, I seemed to have some rounding errors.

So what was the issue?

The schema in MySQL was set to DECIMAL(10,4) which gives us 10 digits after the decimal point and 4 digits after.

A quick alter table and a reimport of the rates and I was on my way!

"alter table tp_rates modify column  rate DECIMAL(10,10);"

Lesson learned and hopefully of use to any other CGrateS users who may be using MySQL as a StoreDB.

Charging problem on Fluke 990 CopperPro

Australian Telco, Notes, , ,

I have a bit of a thing for Test Sets…

Arguably, the most capable (toolbox-carryable) test set out there is the Fluke 990 CopperPro, combined TDR, Butt Set, Toner, RFL, Noise meter, etc, etc, it’s a really cool gizmo (And probably worthy of a blog post in itself someday).

Alas mine had stopped taking much of a charge, you’d plug it into the charger, and after 20 minutes of use, it’d report low power and shut down.

Sitting on the charger the Fluke would show the red charging light for about 20 minutes, then the charging light would turn off.

If I unplugged and replugged the charger I’d get red charging light for another few minutes, then turn off again.

Naively, I ruled it to be an issue with the battery (Which is close to 10 years old), and ordered a new one.

But with the new battery in hand, the same issue.

Stupidly, it turns out I was using a 15v charger, technically the unit supports a 15v charger, but it seems after a period, the internal voltage regulator overheats and shuts off, meaning the battery never got a good charge.

Swapping it with a 12v charger and I’m charging without issues.

But not before I ended up ordering a new battery, so now with the new battery I’m getting 15 hours of runtime out of a charge, and still squeezing 7 out of 8 year old battery it originally came with.

Dumb mistake but hopefully an easy fix for anyone with the same issue.

2022 Telecom Reading List

Notes,

Here’s the list of telecom related books I’ve read in the past year or have plans to read this year. (And here’s the reading list from last year)

Voice Across The Sea / Arthur C. Clarke

I recently finished this one, the book covers the developments of early long distance submarine cables, right up until the 1950s, it’s a good mix of human interest and technological achievement.

I got introduced to Clarke from The Idea Factory – Bell Labs and the Great Age of American Innovation which I read last year, and while it may have been written in the 1950s, this book covers the early days of Gutta-Percha wrapped steel cables being thrown in the ocean, and several failed attempts at linking the Atlantic, up to the Coax systems that were eventually replaced by Fibre after the book was published.

As well as his scientific achievements, and this book, Clarke is perhaps best remembered as the author of 2001: A Space Odyssey.

I’d suggest reading The Undersea Network after Voice Across the Sea, as it somewhat bookends this.

Tubes: A Journey to the Center of the Internet / Andrew Blum

A well written introduction to “The Internet”. If you know AS numbers off the top of your head, you may find it lacking in technical detail, but this book isn’t meant to be a guide on IP Transit Engineering, but rather how the internet was built and continues to run, and the people behind it more than the tech.

The Undersea Network / Nicole Starosielski

A book about the Undersea optic fibre networks, cables, the people behind them and “Turbulent ecologies”.

Girdle Round the Earth – The History of Cable & Wireless / Hugh Barty-King

Having read Tubes, the Undersea Network and Voice Across the Sea, it made sense to learn more about Cable and Wireless’ history, this one has only just arrived, so I’ve yet to finish read it.

FreeSWITCH 1.8 / Anthony Minessale & Giovanni Maruzzelli

My skills were getting rusty having not used FreeSWITCH for a while, and I found this book just as useful on whatever read I’m up to now as it was when I first read it.

Kitten Clone – Inside Alcatel-Lucent / Douglas Coupland

Strange title for a book, but this short read covers Alcatel Lucent’s history, before the NSN / Nokia merger.

Signaling System 7: Basics / Lawrence Harte

A much lighter read than Signaling System No. 7 (SS7/C7), this is a good introduction to the topic, and has been a really good reference for the Demystifying SS7 & Sigtran Networks (With Labs!) series I’ve been writing.

Signaling System No. 7 (SS7/C7): Protocol, Architecture, and Services / Lee Dryburgh, Jeff Hewett

I pick this book up whenever I need a more detail on part of SS7, and I’ve been making an effort to go through the whole thing. It’s very thorough and covers all the “gotchas”.

Ubuntu Cloned VMs getting Duplicate IPs (and yes – the MAC Addresses are unique)

Notes, Software, Uncategorized

So I run a lot of VMs. It’s not unusual when I’m automating something with Ansible or setting up a complex lab to be running 20+ VMs at a time, and often I’ll create a base VM and clone it a dozen times.

Alas, Ubuntu 20.04 has some very irritating default behaviour, where even if the MAC addresses of these cloned VMs differ they get the same IP Address from DHCP.

That’s because by default Netplan doesn’t use the MAC address as the identifier when requesting a DHCP lease. And if you’ve cloned a VM the identifier it does use doesn’t change even if you do change the MAC address…

Irritating, but easily fixed!

Editing the netplan config:

network:
  ethernets:
    eth0:
      dhcp4: true
      dhcp-identifier: mac
  version: 2

Run a netplan-apply and you’re done.

Now you can clone that VM as many times as you like and each will get it’s own unique IP address.

Diff + Wireshark

Notes, RFCs & Standards, Software

Supposedly Archimedes once said:

Give me a lever long enough and a fulcrum on which to place it, and I shall move the world.

For me, the equivalent would be:

Give me a packet capture of the problem occurring and a standards document against which to compare it, and I shall debug the networking world.

And if you’re like me, there’s a good chance when things are really not going your way, you roll up your sleeves, break out Wireshark and the standards docs and begin the painstaking process of trying to work out what’s not right.

Today’s problem involved a side by side comparison between a pcap of a known good call, and one which is failing, so I just had to compare the two, which is slow and fairly error-prone,

So I started looking for something to diff PCAPs easily. The data I was working with was ASN.1 encoded so I couldn’t export as text like you can with HTTP or SIP based protocols and compare it that way.

In the end I stumbled across something even better to compare frames from packet captures side by side, with the decoding intact!

Turns out yo ucan copy the values including decoding from within Wireshark, which means you can then just paste the contents into a diff tool (I’m using the fabulous Meld on Linux, but any diff tool will do including diff itself) and off you go, side-by-side comparison.

Select the first packet/frame you’re interested in (or even just the section), expand the subkeys, right click, copy “All Visible items”. This copy contains all the decoded data, not just the raw bytes, which is what makes it so great.

Next paste it into your diff tool of choice, repeat with the one to compare against, scroll past the data you know is going to be different (session IDs, IPs, etc) and ta-da, there’s the differences.

Video of the whole process below:

Offtopic – HDMI & USB over Twisted Pair

Notes

I wanted to be able to use my desktop computer which lives in my office, on the TV in the living room.

Long HDMI cables would involve me climbing around under the house, and making more holes in the walls, and most wireless keyboard/mouse combos wouldn’t reach that far and USB has a limit of 5 meters.

So instead I put together a rather simple solution that I’m quite happy with.

I ran a lot of Cat5 in the house a while ago, and I’ve got 4 Cat5e sockets behind the TV, and a patch panel at my desk.

I purchased online a HDMI over RJ45/Cat5e adapter, and a USB over RJ45/Cat5e adapter online, for about $5 each.

These are passive devices (baluns) meaning they aren’t converting anything to IP or Ethernet for transport.

This means no additional latency (beyond velocity factor of the cable and the distance, but I digress), so it’s not a remote-desktop experience, it’s like sitting in front of a screen, because that’s what it is.

Very happy with the results!

Being mean to Mikrotiks – Pushing SMB File Share

Linux, Notes, Software,

I’d tried in the past to use the USB port on the Mikrotik, an external HDD and the SMB server in RouterOS, to act as a simple NAS for sharing files on the home network. And the performance was terrible.

This is because the device is a Router. Not a NAS (duh). And everything I later read online confirmed that yes, this is a router, not a NAS so stop trying.

But I recently got a new Mikrotik CRS109, so now I have a new Mikrotik, how bad is the SMB file share performance?

To test this I’ve got a USB drive with some files on it, an old Mikrotik RB915G and the new Mikrotik CRS109-8G-1S-2HnD-IN, and compared the time it takes to download a file between the two.

Mikrotik Routerboard RB951G

While pulling a 2Gb file of random data from a FAT formatted flash drive, I achieved a consistent 1.9MB/s (15.2 Mb/s)

nick@oldfaithful:~$ smbget smb://10.0.1.1/share1/2Gb_file.bin
 Password for [nick] connecting to //share1/10.0.1.1: 
 Using workgroup WORKGROUP, user nick
 smb://10.0.1.1/share1/2Gb_file.bin                                                                                                                                              
 Downloaded 2.07GB in 1123 seconds

Mikrotik CRS109

In terms of transfer speed, a consistent 2.8MB/s (22.4 Mb/s)

nick@oldfaithful:~$ smbget smb://10.0.1.1/share1/2Gb_file.bin
 Password for [nick] connecting to //share1/10.0.1.1: 
 Using workgroup WORKGROUP, user nick
 smb://10.0.1.1/share1/2Gb_file.bin                                              
 Downloaded 2.07GB in 736 seconds

Profiler shows 25% CPU usage on “other”, which drops down as soon as the file transfers stop,

So better, but still not a NAS (duh).

The Verdict

Still not a NAS. So stop trying to use it as a NAS.

As my download speed is faster than 22Mbps I’d just be better to use cloud storage.

GIF showing using Redis-CLI to get a value

Adding SNMP to anything with Redis and Python

Notes, Python, Software, , , ,

I’ve been adding SNMP support to an open source project I’ve been working on (PyHSS) to generate metrics / performance statistics from it, and this meant staring down SNMP again, but this time I’ve come up with a novel way to handle SNMP, that made it much less painful that normal.

The requirement was simple enough, I already had a piece of software I’d written in Python, but I had a need to add an SNMP server to get information about that bit of software.

For a little more detail – PyHSS handles Device Watchdog Requests already, but I needed a count of how many it had handled, made accessible via SNMP. So inside the logic that does this I just increment a counter in Redis;

#Device Watchdog Answer
    def Answer_280(self, packet_vars, avps):                                                      
        self.redis_store.incr('Answer_280_attempt_count')

In the code example above I just add 1 (increment) the Redis key ‘Answer_280_attempt_count’.

The beauty is that that this required minimal changes to the rest of my code – I just sprinkled in these statements to increment Redis keys throughout my code.

Now when that existing function is run, the Redis key “Answer_280_attempt_count” is incremented.

So I ran my software and the function I just added the increment to was called a few times, so I jumped into redis-cli to check on the values;

GIF showing using Redis-CLI to get a value

And just like that we’ve done all the heavy lifting to add SNMP to our software.

For anything else we want counters on, add the increment to your code to store a counter in Redis with that information.

So next up we need to expose our Redis keys via SNMP,

For this, I took a simple SNMP server example from Stackoverflow, to set the output of a MIB tree, and simply bolted in getting a bit of data from, code below:

#Pulled from https://stackoverflow.com/questions/58909285/how-to-add-variable-in-the-mib-tree

from pysnmp.entity import engine, config
from pysnmp.entity.rfc3413 import cmdrsp, context
from pysnmp.carrier.asyncore.dgram import udp
from pysnmp.smi import instrum, builder
from pysnmp.proto.api import v2c
import datetime
import redis


import redis
redis_store = redis.Redis(host='localhost', port=6379, db=0)
# Create SNMP engine
snmpEngine = engine.SnmpEngine()

# Transport setup

# UDP over IPv4
config.addTransport(
    snmpEngine,
    udp.domainName,
    udp.UdpTransport().openServerMode(('127.0.0.1', 1161))
)

# SNMPv3/USM setup

# user: usr-md5-none, auth: MD5, priv NONE
config.addV3User(
    snmpEngine, 'usr-md5-none',
    config.usmHMACMD5AuthProtocol, 'authkey1'
)
# Allow full MIB access for each user at VACM
config.addVacmUser(snmpEngine, 3, 'usr-md5-none', 'authNoPriv', (1, 3, 6, 1, 2, 1), (1, 3, 6, 1, 2, 1))


# SNMPv2c setup

# SecurityName <-> CommunityName mapping.
config.addV1System(snmpEngine, 'my-area', 'public')

# Allow full MIB access for this user / securityModels at VACM
config.addVacmUser(snmpEngine, 2, 'my-area', 'noAuthNoPriv', (1, 3, 6, 1, 2, 1), (1, 3, 6, 1, 2, 1))

# Get default SNMP context this SNMP engine serves
snmpContext = context.SnmpContext(snmpEngine)


# Create an SNMP context with default ContextEngineId (same as SNMP engine ID)
snmpContext = context.SnmpContext(snmpEngine)

# Create multiple independent trees of MIB managed objects (empty so far)
mibTreeA = instrum.MibInstrumController(builder.MibBuilder())
mibTreeB = instrum.MibInstrumController(builder.MibBuilder())

# Register MIB trees at distinct SNMP Context names
snmpContext.registerContextName(v2c.OctetString('context-a'), mibTreeA)
snmpContext.registerContextName(v2c.OctetString('context-b'), mibTreeB)

mibBuilder = snmpContext.getMibInstrum().getMibBuilder()

MibScalar, MibScalarInstance = mibBuilder.importSymbols(
    'SNMPv2-SMI', 'MibScalar', 'MibScalarInstance'
)
class MyStaticMibScalarInstance(MibScalarInstance):
    def getValue(self, name, idx):
        currentDT = datetime.datetime.now()
        return self.getSyntax().clone(
            'Hello World!! It\'s currently: ' + str(currentDT)
        )

class AnotherStaticMibScalarInstance(MibScalarInstance):
    def getValue(self, name, idx):
        return self.getSyntax().clone('Ahoy hoy?')

class Answer_280_attempt_count(MibScalarInstance):
    def getValue(self, name, idx):
        return self.getSyntax().clone(redis_store.get('Answer_280_attempt_count'))


mibBuilder.exportSymbols(
    '__MY_MIB', MibScalar((1, 3, 6, 1, 2, 1, 1, 1), v2c.OctetString()),
    MyStaticMibScalarInstance((1, 3, 6, 1, 2, 1, 1, 1), (0,), v2c.OctetString()),
    AnotherStaticMibScalarInstance((1, 3, 6, 1, 2, 1, 1, 1), (0,1), v2c.OctetString()),
    Answer_280_attempt_count((1, 3, 6, 1, 2, 1, 1, 1), (0,2), v2c.Integer32())
)

# Register SNMP Applications at the SNMP engine for particular SNMP context
cmdrsp.GetCommandResponder(snmpEngine, snmpContext)
cmdrsp.SetCommandResponder(snmpEngine, snmpContext)
cmdrsp.NextCommandResponder(snmpEngine, snmpContext)
cmdrsp.BulkCommandResponder(snmpEngine, snmpContext)

# Register an imaginary never-ending job to keep I/O dispatcher running forever
snmpEngine.transportDispatcher.jobStarted(1)

# Run I/O dispatcher which would receive queries and send responses
try:
    snmpEngine.transportDispatcher.runDispatcher()

except:
    snmpEngine.transportDispatcher.closeDispatcher()
    raise

While PySNMP can be a bit much to wrap your head around, all you need to know:

V2 community string set in:

config.addV1System(snmpEngine, 'my-area', 'public')

Create an additional class from the template below for each of your Redis keys you wish to expose;

class something_else_from_Redis(MibScalarInstance):
    def getValue(self, name, idx):
        return self.getSyntax().clone(redis_store.get('something_else_from_Redis'))

Renaming the class and replacing the redis_store.get() value with the Redis key you want to pull,

And finally inside mibBuilder.exportSymbols() add each of the new classes you added and the OID for each;

    Answer_280_attempt_count((1, 3, 6, 1, 2, 1, 1, 1), (0,2), v2c.Integer32())
    something_else_from_Redis((1, 3, 6, 1, 2, 1, 1, 1), (0,3), v2c.Integer32())

Then when you run it, presto, you’re exposing that data via SNMP.

You can verify it through SNMP walk or start integrating it into your NMS, in the above example OID 1.3.6.1.2.1.1.1.0.2, contains the value of Answer_280_attempt_count from Redis, and with that, you’re exposing info via SNMP, all while not really having to think about SNMP.

*Ok, you still have to sort which OIDs you assign for what, but you get the idea.

My 2021 Telco Reading List for the Holidays

Australian Telco, Notes,

Here’s the list of books I’ve got for the holiday period:

5G Core Networks: Powering Digitalization

A good technical overview of the 5GC architecture, covering the actual elements and their interfaces / reference points, without any talk of robotic surgery.

Clear Across Australia (A history of Telecommunications)

Ann Moyal

This one is an actual hardback book that came in the mail, not just delivered to my ebook reader!

It covers the history of telecommunications in Australia, right from Charles Todd’s Overland Telegraph Line, to the first talk of “Integrated Digital Network” (Later ISDN).

Sonet / SDH Demystified

Steven Shepard

An overview of SDH/PDH networks, for the somewhat beginner.

The Idea Factory

Jon Gertner

A history of Bell Labs and it’s many, many, many achievements.

LTE (4G) – TMSI & GUTI

EPC, EUTRAN, LTE, Mobile Networks, Notes, RF, SDM, , , ,

We’ve already touched on how subscribers are authenticated to the network, how the network is authenticated to subscribers and how the key hierarchy works for encryption of user data and control plane data.

If the IMSI was broadcast in the clear over the air, anyone listening would have the unique identifier of the subscriber nearby and be able to track their movements.

We want to limit the use of the IMSI over the air to a minimum.

During the first exchange the terminal is forced to send it’s IMSI, it’s the only way we can go about authenticating to the network, but once the terminal is authenticated and encryption of the radio link has been established, the network allocates a temporary identifier to the terminal, called the Temporary Mobile Subscriber Identity (TMSI) by the serving MME.

The TMSI is given to the terminal once encryption is setup, so only the network and the terminal know the mapping between IMSI and TMSI.

The TMSI is used for all future communication between the Network and the Terminal, hiding the IMSI.

The TMSI can be updated / changed at regular intervals to ensure the IMSI-TMSI mapping cannot be ascertained by a process of elimination.

The TMSI is short – only 4 bytes long – and this only has significance for the serving MME.

For the network to ascertain what MME is serving what TMSI the terminal is also assigned a Globally Unique Temporary (UE) Identity (GUTI), to identify the MME that knows the TMSI to IMSI mapping.

The GUTI is made up of the MNC/MCC combination, then an MME group ID to identify the MME group the serving MME is in, a MME code to identify the MME that allocated the TMSI and finally the TMSI itself.

The decision to use the TMSI or GUTI in a dialog is dependant on the needs of the dialog and what information both sides have. For example in an MME change the GUTI is needed so the original IMSI can be determined by the new MME, while in a normal handover the TMSI is enough.

IMTx: NET02x (4G Network Essentials) – Mobility Management – 3. Processing Location Updates

EPC, EUTRAN, LTE, Mobile Networks, Notes, RF, , , , ,

These are my lecture notes from IMT’s NET02x (4G Network Essentials) course, I thought I’d post them here as they may be useful to someone. You can find my complete notes here.

Let’s look at how the Tracking Area Updates work from the point of view of the network.

Let’s take an example of a UE which has been sent the Tracking Area List TA0 and TA1, which is currently in ECM_IDLE state served by eNBs in Tracking Area 1.

The UE is moving towards another eNB in Tracking Area 2. As the UE listens on the Broadcast Channel the power of the new eNB overtakes that of the previous eNB, but the UE notes the Tracking Area of the new eNB, which is not on the UE’s Tracking Area List.

So the UE must make a Tracking Area Update to inform the network.

The first thing to do is to establish a radio connection.

Once the radio connection is setup a S1-AP connection is setup, upon which an NAS message – EMM Tracking Area Update Request is sent which contains the GUIT and old Tracking Area ID, which is sent to the MME.

The MME then sends back a new Tracking Area List for the UE and new TMSI to update the GUTI of the subscriber.

The UE updates it’s GUTI, updates it’s Tracking Area List, sends an EMM TRACKING AREA UPDATE COMPLETE and the UE returns to ECM_IDLE state.

Further Reading