Category Archives: Notes

CGrateS – SERVER_ERROR: unexpected end of JSON input

CGrateS, Notes, Software, VoIP

I started seeing this error the other day when running CDRsv1.GetCDRs on the CGrateS API:

SERVER_ERROR: unexpected end of JSON input

It seemed related to certain CDRs in the cdrs table of StoreDB.

After some digging, I found the stupid simple problem:

I’d written too much data to extra_fields, leading MySQL to cut off the data mid way through, meaning it couldn’t be reconstructed as JSON by CGrateS again.

Like the rounding issue I had, this wasn’t an issue with CGrateS but with MySQL.

Quick fix:

sudo mysql cgrates -e "ALTER TABLE cdrs MODIFY extra_fields LONGTEXT;"

And new fields can exceed this length without being cut off.

Mobile Network Code – 2 or 3 Digits?

Mobile Networks, Notes, SDM, SIM Cards, ,

Every mobile network broadcasts a Public Mobile Network Code – aka a PLMN. This 6 octet value is used to identify the network (Although this gets murky with shared codes and Private networks and when OEMs make them for codes they don’t own).

It’s made up of a Mobile Country Code followed by a Mobile Network code.

One of the guys at work asked a seemingly simple question, is the PLMN with MCC 505 and MNC 57 the same as MCC 505 MNC 057 – It’s on 6 octets after all.

So is Mobile Network Code 57 the same as Mobile Network Code 057 in the PLMN code?

The answer is no, and it’s a massive pain in the butt.

All countries use 3 digit Mobile Country Codes, so Australia, is 505. That part is easy.

The tricky part is that some countries (Like Australia) use 2 digit Mobile Network Codes, while others (Like the US) use 3 digit mobile network codes.

This means our 6 digit PLMN has to get padded when encoding a 2 digit Mobile Network Code, which is a pain, but also that by looking at the IMSI alone, you don’t know if the PLMN is 2 digit or 3 digit – IMSI 5055710000001 could be parsed as MCC 505, MNC 571 or MCC 505, MNC 57.

Why would you do this? Why would a regulator opt to have 1/10th the addressable size of network codes – I don’t know, and I haven’t been able to find an answer – If you know please drop a comment, I’d love to know.

So how do we handle this?

There are files in the SIM profile to indicate the length of the MNC, the Administrative Domain EF on the SIM allow us to indicate if the MNC is 2 digit or 3 digit, and the HPLMNwAct and the other *PLMN* EFs encode the PLMN as 6 digit, with or without padding, to allow differentiation.

That’s all well and good from a SIM perspective, but less useful for scenarios where you might be the Visited PLMN for example, and only see the IMSI of a Subscriber.

We worked on a project in a country that mixed both 2 digit and 3 digit Mobile Network Codes, under the same Mobile Country Code. Certain Qualcomm phones would do very very strange things, and it took us a long time and a lot of SIM OTA to resolve the issue, but that’s a story for another day…

private_data_dir on Ansible Runners called from Python

Linux, Notes,

We’ve got a web based front end in our CRM which triggers Ansible Playbooks for provisioning of customer services, this works really well, except I’d been facing a totally baffling issue of late.

Ansible Plays (Provisioning jobs) would have variables set that they inherited from other Ansible Plays, essentially if I set a variable in Play X and then ran Play Y, the variable would still be set.

I thought this was an issue with our database caching the play data showing the results from a previous play, that wasn’t the case.

Then I thought our API that triggers this might be passing extra variables in that it had cached, wasn’t the case.

In the end I ran the Ansible function call in it’s simplest possible form, with no API, no database, nothing but plain vanilla Ansible called from Python

    # Run the actual playbook
    r = ansible_runner.run(
        private_data_dir='/tmp/',
        playbook=playbook_path,
        extravars=extra_vars,
        inventory=inventory_path,
        event_handler=event_handler_obj.event_handler,
        quiet=False
    )

And I still I had the same issue of variables being inherited.

So what was the issue? Well the title gives it away, the private_data_dir parameter creates a folder in that directory, called env/extravars which a JSON file lives with all the vars from all the provisioning runs.

Removing the parameter from the Python function call resolved my issue, but did not give me back the half a day I spent debugging this…

Tales from the Trenches – SMS Data Coding Scheme 0

IMS / VoLTE, Mobile Networks, Notes, RFCs & Standards, , , , ,

The Data Coding Scheme (DCS or TP-DCS) header in an SMS body indicates what encoding is used in that message.

It means if we’re using UCS-2 (UTF16) special characters like Emojis etc, in in our message, the phone knows to decode the data in the message body using UTF, because the Data Coding Scheme (DCS) header indicates the contents are encoded in UTF.

Likewise, if we’re not using any fancy characters in our message and the message is encoded as plain old GSM7, we set set the DCS to 0 to indicate this is using GSM7.

From my experience, I’d always assumed that DCS0 (Default) == GSM7, but today I learned, that’s not always the case. Some SMSc entities treat DCS0 as Latin.

Let me explain why this is stupid and why I wasted a lot of time on this.

We can indicate that a message is encoded as Latin by setting the DCS to 0x03:

We cannot indicate that the message is encoded as GSM7 through anything other than the default alphabet (DCS 0).

Latin has it’s own encoding flag, if I wanted the message treated as Latin, I’d indicate the message encoding is Latin in the DCS bit!

I spent a bunch of time trying to work out why a customer was having issues getting messages to subscribers on another operator, and it turned out the other operator treats messages we send to them on SMPP with DCS0 as Latin encoding, and then cracks the sads when trying to deliver it.

The above diff shows the message we send (Right), and the message they dry to deliver (left).

Well, lesson learned…

Kamailio – NDB_Redis – undefined symbol: redisvCommand

Kamailio, Notes, VoIP, ,

I ran into this issue the other day while compiling Kamailio from source:

Jul 03 23:49:35 kamailio[305199]: ERROR: <core> [core/sr_module.c:608]: ksr_load_module(): could not open module </usr/local/lib64/kamailio/modules/ndb_redis.so>: /usr/local/lib64/kamailio/modules/ndb_redis.so: undefined symbol: redisvCommand
Jul 03 23:49:35 kamailio[305199]: CRITICAL: <core> [core/cfg.y:4024]: yyerror_at(): parse error in config file /etc/kamailio/kamailio.cfg, line 487, column 12-22: failed to load module

So what was going on?

Kamailio’s NDB Redis module relies on libhiredis which was installed, but the path it installs to wasn’t in the LD_LIBRARY_PATH environment variable.

So I added /usr/lib/x86_64-linux-gnu to LD_LIBRARY_PATH, recompiled and presto, the error is gone!

export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH

Virtualizing VMware to run inside Proxmox

Notes, Software, , ,

Like a lot of companies, we’re moving away from VMware, and in our case, shifting to Proxmox.

But that doesn’t mean we can get entirely away from VMware, but more that it’s not our hypervisor of choice anymore, and this means shifting our dev environments and lab off VMware to Proxmox first.

So today I sat down to try and shift everything to Proxmox, while keeping the VMware based VMs accessible until they can slowly die of bitrot.

A sane person would probably utilize Proxmox’s fancy new tool for migrating VMs from VMware to Proxmox, and it’s great, but in our case at least, it required logging into each VM and remapping NICs, etc, which is tricky on boxes I don’t have access to – Plus we need to keep some VMware capability for testing / labbing stuff up.

So I decided into install Proxmox onto the bare metal servers, and then create a VMware virtual machine inside the Proxmox stack, to host a VMware ESXi instance.

I started off inside VMware (Before installing any Proxmox) by moving all the VMs onto a single physical disk, which I then removed from the server, so as to not accidentally format the one disk I didn’t want to format.

Next I nuked the server and setup the new stack with Proxmox, which is a doddle, and not something I’ll cover.

Then I loaded a VMware ISO into Proxmox and started setting up the VM.

Now, nested virtualization is a real pain in the behind.

VMware doesn’t like not being run on bare metal, and it took me a good long amount of time to find the hardware config that I could setup in Proxmox that VMware would accept.

Create the VM in the Web UI; I found using a SATA drive worked while SCSI failed, so create a SATA based LVM image to use, and mount the datastore ISO.

Then edit /etc/pve/qemu-server/your_id.conf and replace the netX, args, boot and ostype to match the below:

args: -cpu host,+invtsc,-vmx
boot: order=ide2;net0
cores: 32
cpu: host
ide2: local:iso/VMware-VMvisor-Installer-8.0U2b-23305546.x86_64.iso,media=cdrom,size=620274K
memory: 128000
meta: creation-qemu=8.1.5,ctime=1717231453
name: esxi
net0: vmxnet3=BC:24:11:95:8A:50,bridge=vmbr0
numa: 0
ostype: l26
sata0: ssd-3600gb:vm-100-disk-1,size=32G
scsihw: pvscsi
smbios1: uuid=6066a9cd-6910-4902-abc7-dfb223042630
sockets: 1
vmgenid: 6fde194f-9932-463e-b38a-82d2e7e1f2dd

Now you can go and start the VM, but once you’ve got the VMware splash screen, you’ll need to press Shift + O to enter the boot options.

At the runweasle cdromBoot after it add allowLegacyCPU=true – This will allow ESXi to use our (virtual) CPU.

Next up you’ll install VMware ESXi just like you’ve probably done 100 times before (is this the last time?), and once it’s done installing, power off, we’ll have to make few changes to the VM definition file.

Then after install we need to change the boot order, by updating:

boot: order=sata0

And unmount the ISO:

ide2: none,media=cdrom

Now remember how I’d pulled the hard disk containing all the VMware VMs out so I couldn’t break it? Well, don’t drop that, because now we’re going to map that physical drive into the VM for VMware, so I can boot all those VMs.

I plugged in the drive and I used this to find the drive I’d just inserted:

fdisk -l

Which showed the drive I’d just added last, with it’s VMware file system.

So next we need to map this through the VM we just created inside Proxmox, so VMware inside Proxmox can access the VMware file system on the disk filled with all our old VMware VMs.

VM. VM. VM. The word has lost all meaning to me at this stage.

We can see the mount point of our physical disk; in our case is /dev/sdc so that’s what we’ll pass through to the VM.

Here’s my final .conf file for the Proxmox VM:

args: -cpu host
balloon: 1024
boot: order=sata0
cores: 32
ide2: none,media=cdrom
kvm: 1
memory: 128000
meta: creation-qemu=8.1.5,ctime=1717231453
name: esxi
net0: vmxnet3=BC:24:11:95:8A:50,bridge=vmbr0
numa: 0
ostype: l26
sata0: ssd-3600gb:vm-100-disk-1,size=32G
sata1: /dev/sda
scsihw: pvscsi
smbios1: uuid=6066a9cd-6910-4902-abc7-dfb223042630
sockets: 1
vmgenid: 6fde194f-9932-463e-b38a-82d2e7e1f2dd

Now I can boot up the VM, log into VMware and behold, our disk is visible:

Last thing we need to do is get it mounted as a VMware Datastore so we can boot all those VMs.

For that, we’ll enable SSH on the ESXi box and SSH into it.

We’ll use the esxcfg-volume -l command to get the UUID of our drive

[root@localhost:~] esxcfg-volume -l
Scanning for VMFS-6 host activity (4096 bytes/HB, 1024 HBs).
VMFS UUID/label: 6513c05a-43db8c20-db0f-c81f66ea471a/FatBoy
Can mount: Yes
Can resignature: Yes
Extent name: t10.ATA_____QEMU_HARDDISK___________________________QM00007_____________:1 range: 0 - 1716735 (MB)

In my case the drive is aptly named “FatBoy”, so we’ll grab the UUID up to the slash, and then use that as the Mount parameter:

[root@localhost:~] esxcfg-volume -M 6513c05a-43db8c20-db0f-c81f66ea471a
Persistently mounting volume 6513c05a-43db8c20-db0f-c81f66ea471a

And now, if everything has gone well, after logging into the Web UI, you’ll see this:

Then the last step is going to be re-registering all the VMs, you can do this by hand, by selecting the .vmx file and adding it.

Alternately, if you’re lazy like me, I wrote a little script to do the same thing:

[root@localhost:~] cat load_vms3.sh 


#!/bin/bash

# Datastore name
DATASTORE="FatBoi/"

# Log file to store the output
LOG_FILE="/var/log/register_vms.log"

# Clear the log file
> $LOG_FILE

echo "Starting VM registration process on datastore: $DATASTORE" | tee -a $LOG_FILE

# Check if datastore directory exists
if [ ! -d "/vmfs/volumes/$DATASTORE" ]; then
    echo "Datastore $DATASTORE does not exist!" | tee -a $LOG_FILE
    exit 1
fi

# Find all .vmx files in the datastore and register them
find /vmfs/volumes/$DATASTORE -type f -name "*.vmx" | while read VMX_PATH; do
    echo "Registering VM: $VMX_PATH" | tee -a $LOG_FILE
    vim-cmd solo/registervm "$VMX_PATH" | tee -a $LOG_FILE
done

echo "VM registration process completed." | tee -a $LOG_FILE

[root@localhost:~] sh load_vms3.sh

Now with all your VMs loaded, you should almost be ready to roll and power them all back on.

And here’s where I ran into another issue:

Luckily the Proxmox wiki had the answer:

For an Intel CPU here’s what you run on the Proxmox hypervisor:

echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf

But before we reboot the Hypervisor (Proxmox) we’ll have to reboot the VMware hypervisor too, because here’s something else to make you punch the screen:

Luckily we can fix this one globaly.

SSH into the VMware box, edit /etc/vmware/config.xml file and add:

vhv.enable = "FALSE"

Which will disable the performance counters.

Now power off the VMware VM, and reboot the Proxmox hypervisor, when it powers on again, Proxmox will allow nested virtualization, and when you power back on the VMware VM, you’ll have performance counters disabled, and then, you will be done.

Yeah, not a great use of my Saturday, but here we are…

DNS’ role in S8-Home Routing Roaming

EPC, IMS / VoLTE, LTE, Mobile Networks, Notes, RFCs & Standards, , , , ,

S8 Home Routing is a really simple concept, the traffic goes from the SGW in the visited PLMN to the PGW in the home PLMN, so the PCRF, OCS/OFCS, IMS, IP Addresses, etc, etc, are all in the home network, and this avoids huge amounts of complexity.

But in order for this to work, the visited network MME needs to find the PGW of the home network, and with over 700 roaming networks in commercial use, each one with potentially hundreds of unique APNs each routing to a different PGW, this is a tricky proposition.

If you’ve configured your PGW peers statically on your MME, that’s fine, but it doesn’t scale very well – And if you add an MVNO who wants their own PGW for serving their APN, well you’ll be adding some complexity there to, so what to do?

Well, the answer is DNS.

By taking the APN to be served, the home PLMN and the interface type desired, with some funky DNS queries, our MME can determine which PGW should be selected for a request.

Let’s take a look, for a UE from MNC XXX MCC YYY roaming into our network, trying to access the “IMS” APN.

Our MME knows the network code of the roaming subscriber from the IMSI is MNC XXX, MCC YYY, and that the UE is requesting the IMS APN.

So our MME crafts a DNS request for the NAPTR query for ims.apn.epc.mncXXX.mccYYY.3gppnetwork.org:

Because the domain is epc.mncXXX.mccYYY.3gppnetwork.org it’s routed to the authoritative DNS server in the home network, which sends back the response:

We’ve got a few peers to pick from, so we need to filter this list of Answers to only those that are relevant to us.

First we filter by the Service tag, whihc for each listed peer shows what services that peer supports.

But since we’re looking for S8, we need to find a peer who’s “Service” tag string contains:

x-3gpp-pgw:x-s8-gtp

We’re looking for two bits of info here, the presence of x-3gpp-pgw in the Service to indicate that this peer is a PGW and x-s8-gtp to indicate that this peer supports the S8 interface.

A service string like this:

x-3gpp-pgw:x-s5-gtp

Would be excluded as it only supports S5 not S8 (Even though they are largely the same interface, S8 is used in roaming).

It’s also not uncommon to see both services indicated as supported, in which case that peer could be selected too:

x-3gpp-pgw:x-s5-gtp:x-s8-gtp

(The answers in the screenshot include :x-gp which means the PGWs advertised are also co-located with a GGSN)

So with our answers whittled down to only those that meet our needs, we next use the Order and the Preference to pick our best candidate, this is the same as regular DNS selection logic.

From our candidate, we’ve also got the Regex Replacement, which allows our original DNS request to be re-written, which allows us to point at a single peer.

In our answer, we see the original request ims.apn.epc.mncXXX.mccYYY.3gppnetwork.org is to be re-written to topon.lb1.pgw01.epc.mncXXX.mccYYY.3gppnetwork.org.

This is the FQDN of the PGW we should use.

Now we know the FQND we should use, we just do an A-Record lookup (Or AAAA record lookup if it is IPv6) for that peer we are targeting, to turn that FQDN into an IP address we can use.

And then in comes the response:

So now our MME knows the IP of the PGW, it can craft a Create Session request where the F-TEID for the S8 interface has the PGW IP set on it that we selected.

For more info on this TS 129.303 (Domain Name System Procedures) is the definitive doc, but the GSMA’s IR.88 “LTE and EPC Roaming Guidelines” provides a handy reference.

vCenter – Partition Folder Isolation

Notes, Security, Software,

While vCenter doesn’t really do contexts / mutli-tenants / VPCs like the hyperscalers, there are simple (ish) ways to do context separation inside VMware vCenter.

This means you can have a user who only has access to say a folder of VMs, but not able to see VMs outside of that folder.

Create a new Role inside vCenter from Administration -> Roles -> Add

Give the role all Virtual Machine privileges:

Create a new account (Can be on AD if you’re not using Local accounts, this is just our lab so I’ve created it as a Local account) for the user. We’re using this account for Ansible so we’ve used “Demo-Ansible-User” as the username

Now create a folder for the group of VMs, or pick an existing folder you want to give access to.

Right click on the folder and select “Add Permission”.

We give permission to that user with that role on the folder and make sure you tick “Propagate to children” (I missed this step before and had to repeat it):

If you are using templates, make sure the template is either in the folder, or apply the same permission to the template, by right clicking on it, Add Permission, same as this.

Finally you should be able to log in as that user and see the template, and clone it from the web UI, or create VMs but only within that folder.

CGrateS – ActionTriggers

CGrateS, EPC, IMS / VoLTE, Mobile Networks, Notes, SDM, Software, Uncategorized, VoIP, , , ,

In our last post we looked at Actions and ActionPlans, and one of the really funky things we can do is setting ActionPlans to trigger on a time schedule or setting ActionTriggers to trigger on an event.

We’re going to build on the examples we had on the last post, so we’ll assume your code is up to the point where we’ve added a Signup Bonus to an account, using an ActionPlan we assigned when creating the account.

In this post, we’re going to create an action that charges $6, called “Action_Monthly_Charge“, and tie it to an ActionPlan called “ActionPlan_Monthly_Charge“, but to demo how this works rather than charging this Monthly, we’re going to charge it every minute.

Then with our balances ticking down, we’ll set up an ActionTrigger to trigger when the balance drops below $95, and alert us.

Defining the Monthly Charge Action

The Action for the Monthly charge will look much like the other actions we’ve defined, except the Identifier is *debit so we know we’re deducting from the balance, and we’ll log to the CDRs table too:

# Action to add a Monthly charge of $6
Action_Monthly_Charge = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Monthly_Charge",
          "Actions": [
              {
                'Identifier': '*debit',
                'BalanceType': '*monetary',
               'Units': 6,
               'Id': 'Action_Monthly_Charge_Debit',
               'Weight': 70},
              {
                  "Identifier": "*log",
                  "Weight": 60,
                  'Id' : "Action_Monthly_Charge_Log"
              },
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Recurring Charge\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Monthly_Charge))

Next we’ll need to wrap this up into an ActionPlan, this is where some of the magic happens. Inside the action plan we can set a once off time, or a recurring time, kinda like Cron.

We’re setting the time to *every_minute so things will happen quickly while we watch, this action will get triggered every 60 seconds. In real life of course, for a Monthly charge, we’d want to trigger this Action monthly, so we’d set this value to *monthly. If we wanted this to charge on the 2nd of the month we’d set the MonthDays to “2”, etc, etc.

# # Create ActionPlan using SetActionPlan to trigger the Action_Monthly_Charge
SetActionPlan_Daily_Action_Monthly_Charge_JSON = {
    "method": "ApierV1.SetActionPlan",
    "params": [{
        "Id": "ActionPlan_Monthly_Charge",
        "ActionPlan": [{
            "ActionsId": "Action_Monthly_Charge",
            "Years": "*any",
            "Months": "*any",
            "MonthDays": "*any",
            "WeekDays": "*any",
            "Time": "*every_minute",
            "Weight": 10
        }],
        "Overwrite": True,
        "ReloadScheduler": True
    }]
}
pprint.pprint(CGRateS_Obj.SendData(
    SetActionPlan_Daily_Action_Monthly_Charge_JSON))

Alright, but now what’s going to happen?

If you think the accounts will start getting debited every 60 seconds after applying this, you’d be wrong, we need to associate this ActionPlan with an Account first, this is how we control which accounts get which ActionPlans tied to them, to do this we’ll use the SetAccout API again we’ve been using to create accounts:

# Create the Account object inside CGrateS & assign ActionPlan_Signup_Bonus and ActionPlan_Monthly_Charge
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionPlanIds": ["ActionPlan_Signup_Bonus", "ActionPlan_Monthly_Charge"],
            "ActionPlansOverwrite": True,
            "ReloadScheduler":True
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

So what’s going to happen if we run this?

Well, for starters the ActionPlan named “ActionPlan_Signup_Bonus” is going to be triggered, as in the ActionPlan it’s Timing is set to *asap, so CGrateS will apply the corresponding Action (“Action_Add_Signup_Bonus“) right away, which will credit the account $99.

But a minute after that, we’ll trigger the ActionPlan named “ActionPlan_Monthly_Charge”, as the timing for this is set to *every_minute, when the Action “Action_Monthly_Charge” is triggered, it’s going to be deducting $6 from the balance.

We can check this by using the GetAccount API:

# Get Account Info
pprint.pprint(CGRateS_Obj.SendData({'method': 'ApierV2.GetAccount', 'params': [
              {"Tenant": "cgrates.org", "Account": str(Account)}]}))

You should see a balance of $99 to start with, and then after 60 seconds, it should be down to $93, and so on.

{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-17T14:57:20.71493633+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '3a896369-8107-4e32-bcef-2d078c981b8a',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-17T14:57:21.802521707+11:00'}}

Triggering Actions based on Balances with ActionTriggers

Okay, so we’ve set up recurring charges, now let’s get notified if the balance drops below $95, we’ll start, like we have before, with defining an Action, this will log to the CDRs table, HTTP post and write to syslog:


#Define a new Action to send an HTTP POST
Action_HTTP_Notify_95 = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_HTTP_Notify_95",
          "Actions": [
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Balance dipped below $95\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
              {
                  "Identifier": "*http_post_async",
                  "ExtraParameters": "http://10.177.2.135/95_remaining",
                  "ExpiryTime": "*unlimited",
                  "Weight": 700
              },
              {
                  "Identifier": "*log",
                  "Weight": 1200
              }
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_HTTP_Notify_95))

Now we’ll define an ActionTrigger to check if the balance is below $95 and trigger our newly created Action (“Action_HTTP_Notify_95“) when that condition is met:


#Define ActionTrigger
ActionTrigger_95_Remaining_JSON = {
    "method": "APIerSv1.SetActionTrigger",
    "params": [
        {
            "GroupID" : "ActionTrigger_95_Remaining",
            "ActionTrigger": 
                {
                    "BalanceType": "*monetary",
                    "Balance" : {
                        'BalanceType': '*monetary',
                        'ID' : "*default",
                        'BalanceID' : "*default",
                        'Value' : 95,
                        },
                    "ThresholdType": "*min_balance",
                    "ThresholdValue": 95,
                    "Weight": 10,
                    "ActionsID" : "Action_HTTP_Notify_95",
                },
            "Overwrite": True
        }
    ]
}
pprint.pprint(CGRateS_Obj.SendData(ActionTrigger_95_Remaining_JSON))

We’ve defined the ThresholdType of *min_balance, but we could equally set this to ThresholdType to *max_balance, *balance_expired or trigger when a certain Counter has been triggered enough times.

Adding an ActionTrigger to an Account

Again, like the ActionPlan we created before, before the ActionTrigger we just created will be used, we need to associate it with an Account, for this we’ll use the AddAccountActionTriggers API, specify the Account and the ActionTriggerID for the ActionTrigger we just created.


#Add ActionTrigger to Account 
Add_ActionTrigger_to_Account_JSON = {
    "method": "APIerSv1.AddAccountActionTriggers",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionTriggerIDs": ["ActionTrigger_95_Remaining"],
            "ActionTriggersOverwrite": True
        }
    ]
}
pprint.pprint(CGRateS_Obj.SendData(Add_ActionTrigger_to_Account_JSON))

If we run this all together, creating the account with the “ActionPlan_Signup_Bonus” will give the account a $99 Balance. But after 60 seconds, “ActionPlan_Monthly_Charge” will kick in, and every 60 seconds after that, at which point the balance will get to below $95 when CGrateS will trigger the ActionTrigger “ActionTrigger_95_Remaining” and get the HTTP POST to the HTTP endpoint and log entry:

We can check on this using the ApierV2.GetAccount method, where we’ll see the ActionTrigger we just defined.

Checking out the LastExecutionTime we can see if the ActionTrigger been triggered or not.

So using this technique, we can notify a customer when they’ve used a certain amount of their balance, but we can lock out Accounts who have spent more than their allocated spend limit by setting an Action that suspends the Account once it reaches a certain level. We notify customers when balance expires, or if a certain number of counters has been triggered.

As always I’ve put all the code used in this example, from start to finish, up on GitHub.

CGrateS – Actions & Action Plans

CGrateS, Mobile Networks, Notes, Python, SDM, Software, VoIP, ,

In our last post we added a series of different balances to an account, these were actions we took via the API specifically to add a balance.

But there’s a lot more actions we may want to do beyond just adding balance.

CGrateS has the concept of “Actions” which are, as the name suggests, things we want to do to the system.

Some example Actions would be:

  • Adding / Deducting / Resetting a balance
  • Adding a CDR log
  • Enable/Disable an account
  • Sending HTTP POST request or email notification
  • Deleting / suspending account
  • Transferring balances

We can run these actions on a timed basis, or when an event is triggered, and group Actions together to run multiple actions via an ActionTrigger, this means we can trigger these Actions, not just by sending an API request, but based on the state of the subscriber / account.

Let’s look at some examples,

We can define an Action named “Action_Monthly_Fee” to debit $12 from the monetary balance of an account, and add a CDR with the name “Monthly Account Fee” when it does so.
We can use ActionTriggers to run this every month on the account automatically.

We can define an Action named “Usage_Warning_10GB” to send an email to the Account owner to inform them they’ve used 10GB of usage, and use ActionTriggers to send this when the customer has used 10GB of their *data balance.

Using Actions

Note: The Python script I’ve used with all the examples in this post is available on GitHub here.

Let’s start by defining an Account, just as we have before:

# Create the Account object inside CGrateS
Account = "Nick_Test_123"
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account)
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Let’s start basic; to sweeten the deal for new Accounts, we’ll give them $99 of balance to use in the first month they have the service. Rather than hitting the AddBalance API, we’ll define an Action named “Action_Add_Signup_Bonus” to credit $99 of monetary balance to an account.

If you go back to our last post, you should know what we’d need to do to add this balance manually with the AddBalance API, but let’s look at how we can create the same balance add functionality using Actions:

#Add a Signup Bonus of $99 to the account with type *monetary expiring a month after it's added
Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 99,
                  "ExpiryTime": "*month",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 1200,
                  "ExtraParameters": "",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 10
              }
]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Alright, this should look pretty familiar if you’ve just come from Account Balances.
You’ll notice we’re no longer calling, SetBalance, we’re now calling SetActions, to create the ActionsId with the name “Action_Add_Signup_Bonus“.
In “Action_Add_Signup_Bonus” we’ve got an actions we’ll do when “Action_Add_Signup_Bonus” is called.
We can define multiple actions, but for now we’ve only got one action defined, which has the Identifier (which defines what the action does) set to *topup to add balance.
As you probably guessed, we’re triggering a top up, and setting the BalanceId, BalanceType, Units, ExpiryTime and BalanceWeight just as we would using SetBalance to add a balance.

So how do we use the Action we just created? Well, there’s a lot of options, but let’s start with the most basic – Via the API:

# Trigger ExecuteAction
Account_Action_trigger_JSON = {"method": "APIerSv1.ExecuteAction", "params": [
    {"Tenant": "cgrates.org", "Account": str(Account), "ActionsId": "Action_Add_Signup_Bonus"}]}
pprint.pprint(CGRateS_Obj.SendData(Account_Action_trigger_JSON))

Boom, we’ve called the ExecuteAction API call, to execute the Action named “Action_Add_Signup_Bonus“.

We can check on this with GetAccount again and check the results:

# Get Account Info
pprint.pprint(CGRateS_Obj.SendData({'method': 'ApierV2.GetAccount', 'params': [
              {"Tenant": "cgrates.org", "Account": str(Account)}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-15T10:27:52.865119544+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '01cfb471-ba38-453a-b0e2-8ddb397dfe9c',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-15T10:27:52.865144268+11:00'}}

Great start!

Making Actions Useful

Well congratulations, we took something we previously did with one API call (SetBalance), and we did it with two (SetAction and ExcecuteAction)!

But let’s start paying efficiency dividends,

When we add a balance, let’s also add a CDR log event so we’ll know the account was credited with the balance when we call the GetCDRs API call.

We’d just modify our SetActions to include an extra step:

Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
...
              }, 
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Your sign up Bonus\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 10
              }
]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Boom, now we’ll get a CDR created when the Action is triggered.

But let’s push this a bit more and add some more steps in the Action:

As well as adding balance and putting in a CDR to record what we did, let’s also send a notification to our customer via an HTTP API (BYO customer push notification system) and log to Syslog what’s going on.

# Add a Signup Bonus of $99 to the account with type *monetary expiring a month after it's added
Action_Signup_Bonus = {
    "id": "0",
    "method": "ApierV1.SetActions",
    "params": [
        {
          "ActionsId": "Action_Add_Signup_Bonus",
          "Actions": [
              {
                  "Identifier": "*topup",
                  "BalanceId": "Balance_Signup_Bonus",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 99,
                  "ExpiryTime": "*month",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 1200,
                  "ExtraParameters": "",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 90
              },
              {
                  "Identifier": "*cdrlog",
                  "BalanceId": "",
                  "BalanceUuid": "",
                  "BalanceType": "*monetary",
                  "Directions": "*out",
                  "Units": 0,
                  "ExpiryTime": "",
                  "Filter": "",
                  "TimingTags": "",
                  "DestinationIds": "",
                  "RatingSubject": "",
                  "Categories": "",
                  "SharedGroups": "",
                  "BalanceWeight": 0,
                  "ExtraParameters": "{\"Category\":\"^activation\",\"Destination\":\"Your sign up Bonus\"}",
                  "BalanceBlocker": "false",
                  "BalanceDisabled": "false",
                  "Weight": 80
              },
              {
                  "Identifier": "*http_post_async",
                  "ExtraParameters": "http://10.177.2.135/example_endpoint",
                  "ExpiryTime": "*unlimited",
                  "Weight": 70
              },
              {
                  "Identifier": "*log",
                  "Weight": 60
              }
          ]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))

Phew! That’s a big action, but if we execute the action again using ExecuteAction, we’ll get all these things happening at once:

Okay, now we’re getting somewhere!

ActionPlans

Having an Action we can trigger manually via the API is one thing, but being able to trigger it automatically is where it really comes into its own.

Let’s define an ActionPlan, that is going to call our Action named “Action_Add_Signup_Bonus” as soon as the ActionPlan is assigned to an Account.

# Create ActionPlan using SetActionPlan to trigger the Action_Signup_Bonus ASAP
SetActionPlan_Signup_Bonus_JSON = {
    "method": "ApierV1.SetActionPlan",
    "params": [{
        "Id": "ActionPlan_Signup_Bonus",
        "ActionPlan": [{
            "ActionsId": "Action_Add_Signup_Bonus",
            "Years": "*any",
            "Months": "*any",
            "MonthDays": "*any",
            "WeekDays": "*any",
            "Time": "*asap",
            "Weight": 10
        }],
        "Overwrite": True,
        "ReloadScheduler": True
    }]
}
pprint.pprint(CGRateS_Obj.SendData(SetActionPlan_Signup_Bonus_JSON))

So what have we done here? We’ve made an ActionPlan named “Action_Add_Signup_Bonus”, which, when associated with an account, will run the Action “Action_Add_Signup_Bonus” as soon as it’s tied to the account, thanks to the Time “*asap“.

Now if we create or update an Account using the SetAccount method, we can set the ActionPlanIds to reference our “ActionPlan_Signup_Bonus” and it’ll be triggered straight away.

# Create the Account object inside CGrateS
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": str(Account),
            "ActionPlanIds": ["ActionPlan_Signup_Bonus"],
            "ActionPlansOverwrite": True,
            "ReloadScheduler":True
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Now if we were to run a GetAccount API call, we’ll see the Account balance assigned that was created by the action Action_Add_Signup_Bonus which was triggered by ActionPlan assigned to the account:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*monetary': [{'Blocker': False,
                                          'Categories': {},
                                          'DestinationIDs': {},
                                          'Disabled': False,
                                          'ExpirationDate': '2023-11-16T12:41:02.530985381+11:00',
                                          'Factor': None,
                                          'ID': 'Balance_Signup_Bonus',
                                          'RatingSubject': '',
                                          'SharedGroups': {},
                                          'TimingIDs': {},
                                          'Timings': None,
                                          'Uuid': '7bdbee5c-0888-4da2-b42f-5d6b8966ee2d',
                                          'Value': 99,
                                          'Weight': 1200}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-16T12:41:12.7236096+11:00'}}

But here’s where it gets interesting, in the ActionPlan we just defined the Time was set to “*asap“, which means the Action is triggered as soon as it was assigned to the account, but if we set the Time value to “*monthly“, the Action would get triggered every month, or *every_minute to trigger every minute, or *month_end to trigger at the end of every month.

Code for these examples is available here.

I’m trying to keep these posts shorter as there’s a lot to cover. Stick around for our next post, we’ll look at some more ActionTriggers to keep decreasing the balance of the account, and setting up ActionTriggers to send a notification to the customer to tell them when their balance is getting low, or any other event based Action you can think of!

CGrateS – Accounts & Balances

CGrateS, IMS / VoLTE, Mobile Networks, Notes, Software, VoIP, , ,

So far we’ve used CGrateS to rate a basic CDR and get a cost for it, but in the real world, we’d usually associate the cost with an account, which would represent a business or a person, who will ultimately be charged for using the service.

Note: I’ve put the code for all this in Github, if you’ve got issues following along, or don’t want to copy and paste the code from the website, you can grab the code here.

Creating an Account

Let’s start off by creating an account inside CGrateS – This is kinda pointless, but we’ll talk more about that later:

#Create the Account object inside CGrateS 
Create_Account_JSON = {
    "method": "ApierV2.SetAccount",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123"
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Account_JSON))

Running this onto the API should create an account named “Nick_Test_123”, but let’s confirm that’s the case:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))

Running this will give us the information about the account we just created:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
OrderedDict([('id', None),
             ('result',
              OrderedDict([('ID', 'cgrates.org:Nick_Test_123'),
                           ('BalanceMap', None),
                           ('UnitCounters', None),
                           ('ActionTriggers', None),
                           ('AllowNegative', False),
                           ('Disabled', False),
                           ('UpdateTime',
                            '2023-10-09T16:53:37.524466041+11:00')])),
             ('error', None)])

That was easy!

There’s not really much to see on our account at this stage, other than the UpdateTime, there’s nothing really going on, we don’t have any Balances.

Adding Balance for Voice

Accounts exist for spending, so let’s add a balance to this account to send from.

We’ll use the SetBalance API to create a new balance with 5 minutes of talk time, that we can use for making a call, and talking, for (you guessed it) – 5 minutes, so and we’ll use the balance “5_minute_voice_balance” that we’ll create:

#Add a balance to the account with type *voice with 5 minutes of Talk Time
Create_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "5_minute_voice_balance",
                "Value": "5m",
                "Weight": 25
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Voice_Balance_JSON))

Now if we run the GetAccount API command again, we should see the new balance we just created:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': '37423d07-d99a-40b1-851a-981c3df02cb3',
                                       'Value': 300000000000,
                                       'Weight': 25}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T17:58:23.801531205+11:00'}}

So now we’ve got a new balance named ‘5_minute_voice_balance‘:

  • The type is *voice, because this balance is storing talk time
  • The weight of this balance is 25, this means this balance should take priority over any balances with a lower value than 25 (that’s right, we can (and will) do tiered balances)
  • The value is 300000000000 nanoseconds, which equates to 5 minutes (yes, that’s the correct number of zeros)

Okay, but Nick_Test_123 probably wants to make some calls, so let’s generate a 2.5 minute call event and check out what happens.


#Generate a new call event for a 2.5 minute (150 second) call
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "150s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

Alright, now we’ve got a call event, let’s call the GetAccount API again to check the balance:

#Print the Account Information
pprint.pprint(CGRateS_Obj.SendData({'method':'ApierV2.GetAccount','params':[{"Tenant":"cgrates.org","Account": "Nick_Test_123"}]}))
{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': '37423d07-d99a-40b1-851a-981c3df02cb3',
                                       'Value': 150000000000,
                                       'Weight': 25}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T17:58:23.80925546+11:00'}}

And there you have it, we’ve used 150 seconds of our 300 second (5 minutes) of talk time in this balance, leaving with us 150000000000 nanoseconds (150 seconds) remaining!

And with that progress, now is a great time to pause and talk about some theory that’s really important to grasp!

Balance Types

But *voice is just one balance type – We can support multiple balance types; we’ve just given a balance of *voice for talk time, but we could also give a credit to the balance for *sms or *data, you name it (*generic) and cash (*monetary) and we can have multiple separate balances for each.

This means we can have one account with something like:

  • 100 minutes of Local / National Calls (Expires at the end of the month)
  • 40 minutes of Mobile Calls (Expires 24 hours after it’s been created)
  • 80 minutes of Mobile Calls (During “Happy Hour” from 6 to 7pm)
  • 50 minutes of International Calls (Expires in 30 days)

And not just voice balance, the same account could also have:

  • 1GB of Data usage
  • 50 SMS to on-net destinations
  • $200 of Cash (expiring never)

Phew! That’s a lot of balances, but we can do it all through CGrateS!

What Balance to Use

So if we’ve got a stack of balances defined, how does CGrateS know what balance to use?

Firstly CGrateS is going to evaluate the BalanceType, this is set on events, so if we get an event for *data CGrateS will check out the balances available for *data, and evaluate the balances by Weight, with the highest weight evaluated first.
If we get to the end of all the available balances for that BalanceType, CGrateS then evaluates *generic and then *monetary balances, again, ordered by Weight.

We can set what balance gets used based on the Destination; using DestinationIDs we can filter the Balance to only apply for calls to Local/National numbers, so a call to an International destination won’t use that balance.

We can also set an Expiry on the Balance, for example we can give a customer 30 days to use the balance, after which it expires and can’t be used, likewise we can set Timings so enable scenarios like a “Happy hour” with extra calls between 6pm and 7pm.

When we define a balance we can also set the Blocker flag to True, if this is set, it means CGrateS will not look evaluate any balances after reaching that balance.

Adding a Balance for Local / National & Mobile Calls

Let’s jump back into the practice, and define two new Balances; one for Local/National calls, and another for Mobile calls.
But first we’ll need to know what destinations are mobiles and what are local/national (fixed). We’ve covered setting Destinations previously, so let’s set up the Destinations:


CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_Fixed","Prefixes":["612", "613", "617", "618"]}]})
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_Mobile","Prefixes":["614"]}]})
CGRateS_Obj.SendData({'method':'ApierV2.SetTPDestination','params':[{"TPid":"cgrates.org","ID":"Dest_AU_TollFree","Prefixes":["6113", "6118"]}]})
#Load TariffPlan we just defiend from StorDB to DataDB
print(CGRateS_Obj.SendData({"method":"APIerSv1.LoadTariffPlanFromStorDb","params":[{"TPid":"cgrates.org","DryRun":False,"Validate":True,"APIOpts":None,"Caching":None}],"id":0}))

Alright, now let’s add a balance for our local/national (fixed) calls.

To do this, we’ll add two new balances, but we’ll need to differentiate this from the 5_minute_voice_balance we created earlier, and to achieve this we weill:

  • Set a higher Weight value than we have set on 5_minute_voice_balance (25) so this balance will get consumed before 5_minute_voice_balance does
  • Set the DestinationIDs to match the destinations (Dest_AU_Mobile for Mobile and Dest_AU_Fixed for Local/National) we want the balance to apply to

ProTip: When you we create our Balance we can set what Destinations we want to use this balance for, if you want to specify multiple balances, we can do it by setting the Balance names as a string delimited by semicolons, like “DestinationIDs”: “Dest_AU_Fixed;Dest_AU_Mobile;Dest_AU_TollFree”

We’ll also set a balance expiry, which we’ll cover shortly, but now let’s define out 100 minutes for Local/National expiring at the end of the month:

#Add a balance to the account with type *voice with 100 minutes of talk time to Local / National Destinations expiring at the end of the month
Create_Local_National_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "Local_National_100_minutes_voice_balance",
                "Value": "100m",
                "ExpiryTime": "*month_end",
                "Weight": 60,
                "DestinationIDs": "Dest_AU_Fixed",
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Local_National_Voice_Balance_JSON))

We’ll also add our 24 hours to use to use 40 minutes of talk to mobiles, and a GetAccount to check the result:

#Add a balance to the account with type *voice with 40 minutes of talk time to Mobile Destinations expiring in 24 hours
Create_Mobile_Voice_Balance_JSON = {
    "method": "ApierV1.SetBalance",
    "params": [
        {
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "BalanceType": "*voice",
            "Categories": "*any",
            "Balance": {
                "ID": "Mobile_40_minutes_voice_balance",
                "Value": "40m",
                "ExpiryTime": "*daily",
                "Weight": 60,
                "DestinationIDs": "Dest_AU_Mobile",
            }
        }
    ]
}
print(CGRateS_Obj.SendData(Create_Mobile_Voice_Balance_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

Alright, let’s try running that:

{'method': 'ApierV2.GetAccount', 'params': [{'Tenant': 'cgrates.org', 'Account': 'Nick_Test_123'}]}
{'error': None,
 'id': None,
 'result': {'ActionTriggers': None,
            'AllowNegative': False,
            'BalanceMap': {'*voice': [{'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': None,
                                       'Disabled': False,
                                       'ExpirationDate': '0001-01-01T00:00:00Z',
                                       'Factor': None,
                                       'ID': '5_minute_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'ad9d8bdd-64df-430f-af9d-3fc0410fd16b',
                                       'Value': 150000000000,
                                       'Weight': 25},
                                      {'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': {'Dest_AU_Fixed': True},
                                       'Disabled': False,
                                       'ExpirationDate': '2023-10-31T23:59:59+11:00',
                                       'Factor': None,
                                       'ID': 'Local_National_100_minutes_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'e4a2c211-8112-4e40-b3e6-250863404cc9',
                                       'Value': 6000000000000,
                                       'Weight': 60},
                                      {'Blocker': False,
                                       'Categories': None,
                                       'DestinationIDs': {'Dest_AU_Mobile': True},
                                       'Disabled': False,
                                       'ExpirationDate': '2023-10-15T18:15:11.521636734+11:00',
                                       'Factor': None,
                                       'ID': 'Mobile_40_minutes_voice_balance',
                                       'RatingSubject': '',
                                       'SharedGroups': None,
                                       'TimingIDs': None,
                                       'Timings': None,
                                       'Uuid': 'd4cbf6d8-50a5-4c97-82c2-dfe9936ae8d1',
                                       'Value': 2400000000000,
                                       'Weight': 60}]},
            'Disabled': False,
            'ID': 'cgrates.org:Nick_Test_123',
            'UnitCounters': None,
            'UpdateTime': '2023-10-14T18:15:11.524242437+11:00'}}

Alright! We now have 3 balances defined!

Notice in the API in the expiry I put *daily and *month_end, but in the output it’s got a real date and time (I wrote this 14/10/23 around 07:00 UTC, hence why those dates are what they are).
I could have specified the date and time in the API of a specific time I wanted the balance to expire (You can too, just replace “*daily” with “2024-01-01T00:00:00Z” for example), but that’s a pain in the butt, especially considering most of the time these values will be something common.
The *month_end is a special “meta” value, there’s a heap of these that allow us to do things like “current time + 20 minutes” (+20m), this time next month (*monthly), “this time tomorrow” (*daily), or “this time next week” (+168h) – You can find the full list of special dates here.

From a product perspective, setting an expiry on balances means we can set credit to expire 2 years after the subscriber tops up, but the same logic can be used so a subscriber could purchase a 7 day addon pack, that expires in 7 days, or a monthly plan can automatically expire in 30 days.

Now if we call the ProcessExternalCDR API again with a call to a Mobile and a Fixed number, we’ll see the respective balances get deducted.


#Generate a new call event for a 2.5 minute (150 second) call to a mobile number
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61412341234",
            "Destination": "61412341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "30s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

#Generate a new call event for a 2.5 minute (150 second) call to a fixed line local/national number
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61212341234",
            "Destination": "61212341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "30s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

We should see the minutes reduced by 30 seconds for our Local_National_100_minutes_voice_balance and Mobile_40_minutes_voice_balance balances, while our 5_minute_voice_balance hasn’t been touched.

{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":"None",
   "Disabled":false,
   "ExpirationDate":"0001-01-01T00:00:00Z",
   "Factor":"None",
   "ID":"5_minute_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"29f21735-1d62-49b1-9c53-80eab6f7b005",
   "Value":150000000000,
   "Weight":25
},
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Fixed":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-31T23:59:59+11:00",
   "Factor":"None",
   "ID":"Local_National_100_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"54db4f60-342e-4738-aaf1-a1304badc41d",
   "Value":5970000000000,
   "Weight":60
},
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Mobile":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-15T18:32:34.888821482+11:00",
   "Factor":"None",
   "ID":"Mobile_40_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"501eb00e-e947-4675-926f-080911e66897",
   "Value":2370000000000,
   "Weight":60
}

One last thing we’ll try before we end, our Mobile_40_minutes_voice_balance has still got 39.5 minutes left, and our 5_minute_voice_balance has still got minutes remaining, so if we try and make a call that’s 2450 seconds (~41 minutes), we should consume all the remaining minutes in Mobile_40_minutes_voice_balance and the go onto consume the remaining 1 minute out of 5_minute_voice_balance.

Let’s test this theory!

#Generate a new call event for a 42 minute call to a mobile to use all of our Mobile_40_minutes_voice_balance and start consuming 5_minute_voice_balance
Process_External_CDR_JSON = {
    "method": "CDRsV2.ProcessExternalCDR",
    "params": [
        {
            "OriginID": str(uuid.uuid1()),
            "ToR": "*voice",
            "RequestType": "*pseudoprepaid",
            "AnswerTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "SetupTime": now.strftime("%Y-%m-%d %H:%M:%S"),
            "Subject": "61412341234",
            "Destination": "61412341234",
            "Tenant": "cgrates.org",
            "Account": "Nick_Test_123",
            "Usage": "2450s",
        }
    ]
}
print(CGRateS_Obj.SendData(Process_External_CDR_JSON))

# Get Account Info Again
pprint.pprint(CGRateS_Obj.SendData({"method": "ApierV2.GetAccount", "params": [
              {"Tenant": "cgrates.org", "Account": "Nick_Test_123"}]}))

Let’s check the output:

{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":"None",
   "Disabled":false,
   "ExpirationDate":"0001-01-01T00:00:00Z",
   "Factor":"None",
   "ID":"5_minute_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"29f21735-1d62-49b1-9c53-80eab6f7b005",
   "Value":70000000000,
   "Weight":25
},
...
{
   "Blocker":false,
   "Categories":"None",
   "DestinationIDs":{
      "Dest_AU_Mobile":true
   },
   "Disabled":false,
   "ExpirationDate":"2023-10-15T18:44:18.161474861+11:00",
   "Factor":"None",
   "ID":"Mobile_40_minutes_voice_balance",
   "RatingSubject":"",
   "SharedGroups":"None",
   "TimingIDs":"None",
   "Timings":"None",
   "Uuid":"501eb00e-e947-4675-926f-080911e66897",
   "Value":0.0161939859,
   "Weight":60
}

Boom, there we have it! Used all of the minutes in Mobile_40_minutes_voice_balance and started eating into the 5_minute_voice_balance.

Note: I’ve put the code for all this in Github, if you’ve got issues following along, or don’t want to copy and paste the code from the website, you can grab the code here.

Alright, that was a long post! Sorry about that, and props for making it to the end, still so much to learn about CGrateS.

GNS3 vCenter / ESXi – Allow Traffic

Notes, Software, , ,

The other day I setup GNS3 in the lab for some testing, we run vCenter for our server workloads, so I chucked the OVA on there.

One issue I ran into is that when linking a Cloud Component to a router, I simply could not get a path in/out of the router, I wasn’t learning MAC addresses and my ARP requests were going unanswered.

Wireshark showed the ARP requests going down that interface, and broadcast traffic from the rest of the network, so what gives?

The answer was pretty simple, on the vHost itself I needed to enable Promiscuous mode to allow L2 addresses that aren’t the VM, to be sent from within the VM.

Under Networking -> Port Groups -> the NICs you have assigned in GNS3:

Make sure Promiscuous mode, MAC address changes and Forged transmits are allowed – By default they’re denied on the vSwitch which it inherits from.

There’s obviously security concerns here, so think before you do, but that should have packets flowing.

Logging DSL Line Rate & SNR on a Draytek Modem

Australian Telco, Notes, Software, , , , ,

I am connected on a VDSL line, not by choice, but here we are.
DSL is many things, but consistent it not one of them, so I thought it’d be interesting to graph out the SNR and the line rate of the connection.

This is an NBN FTTN circuit, I run Mikrotiks for the routing, but I have a Draytek Vigor 130 that acts as a dumb modem and connects to the Tik.

Draytek exposes this info via SNMP, but the OIDs / MIBs are not part of the standard Prometheus snmp_exporter, so I’ve added them into snmp_exporter.yaml and restarted the snmp_exporter service.

draytek:
  walk:
  - 1.3.6.1.2.1.10.94.1.1.3.1.8
  - 1.3.6.1.2.1.10.94.1.1.3.1.4
  - 1.3.6.1.2.1.10.94.1.1.5.1.2.4
  - 1.3.6.1.2.1.10.94.1.1.4.1.2.4
  metrics:
  - name: Draytek_dsl_LineRate
    oid: 1.3.6.1.2.1.10.94.1.1.3.1.8
    type: gauge
    help: adslAtucCurrAttainableRate

  - name: Draytek_dsl_Linerate_Down
    oid: 1.3.6.1.2.1.10.94.1.1.4.1.2.4
    type: gauge
    help: Draytek_dsl_Linerate_Down

  - name: Draytek_dsl_Linerate_Up
    oid: 1.3.6.1.2.1.10.94.1.1.5.1.2.4
    type: gauge
    help: Draytek_dsl_Linerate_Up

  - name: Draytek_dsl_SNR
    oid: 1.3.6.1.2.1.10.94.1.1.3.1.4
    type: gauge
    help: adslAturCurrSnrMgn

Then I added this as a target in Prometheus:

  - job_name: Draytek Logger
    scrape_interval: 1m
    scrape_timeout: 30s
    static_configs:
          - targets: ['10.0.2.1']  # My modem

    metrics_path: /snmp
    params:
      module: ['draytek']
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: localhost:9116  # SNMP exporter address

And then from Grafana I can quantify exactly how bad my line is over time!

Only two dropouts today!

Mobile IPv6 Tax?

5G SA, EPC, LTE, Mobile Networks, Notes, RFCs & Standards, ,

Recently a Tweet from Dean Bubly got me thinking about how data is charged in cellular:

In the cellular world, subscribers are charged for data from the IP, transport and applications layers; this means you pay for the IP header, you pay for the TCP/UDP header, and you pay for the contents (the cat videos it contains).

This also means if an operator moves mobile subscribers from IPv4 to IPv6, there’s an extra 20 bytes the customer is charged for for every packet sent / received, which the customer is charged for – This is because the IPv6 header is longer than the IPv4 header.

Source: ServerFault - https://serverfault.com/questions/547768/ipv4-header-vs-ipv6-header-size

In most cases, mobile subs don’t get a choice as to if their connection is IPv4 or IPv6, but on a like for like basis, we can say that if a customer moves is on IPv6 every packet sent/received will have an extra 20 bytes of data consumed compared to IPv4.

This means subscribers use more data on IPv6, and this means they get charged for more data on IPv6.

For IoT applications, light users and PAYG users, this extra 20 bytes per packet could add up to something significant – But how much?

We can quantify this, but we’d need to know the number of packets sent on average, and the quantity of the data transferred, because the number of packets is the multiplier here.

So for starters I’ve left a phone on the desk, it’s registered to the network but just sitting in Idle mode – This is an engineering phone from an OEM, it’s just used for testing so doesn’t have anything loaded onto it in terms of apps, it’s not signed into any applications, or checking in the background, so I thought I’d try something more realistic.

So to get a clearer picture, I chucked a SIM in my regular everyday phone I use personally, registered it to the cellular lab I have here. For the next hour I sniffed the GTP traffic for the phone while it was sitting on my desk, not touching the phone, and here’s what I’ve got:

Overall the PCAP includes 6,417,732 bytes of data, but this includes the transport and GTP headers, meaning we can drop everything above it in our traffic calculations.

Everything except the data encapsulated in GTP can be dropped

For this I’ve got 14 bytes of ethernet, 20 bytes IP, 8 bytes UDP and 5 bytes for TZSP (this is to copy the traffic from the eNB to my local machine), then we’ve got the transport from the eNB to the SGW, 14 bytes of ethernet again, 20 bytes of IP , 8 bytes of UDP and 8 bytes of GTP then the payload itself. Phew.
All this means we can drop 97 bytes off every packet.

We have 16,889 packets, 6,417,732 bytes in total, minus 97 bytes from each gives us 1,638,233 of headers to drop (~1.6MB) giving us a total of 4.556 MB traffic to/from the phone itself.

This means my Android phone consumes 4.5 MB of cellular data in an hour while sitting on the desk, with 16,889 packets in/out.

Okay, now we’re getting somewhere!

So now we can answer the question, if each of these 16k packets was IPv6, rather than IPv4, we’d be adding another 20 bytes to each of them, 20 bytes x 16,889 packets gives 337,780 bytes (~0.3MB) to add to the total.

If this traffic was transferred via IPv6, rather than IPv4, we’d be looking at adding 20 bytes to each of the 16,889 packets, which would equate to 0.3MB extra, or about 7% overhead compared to IPv4.

But before you go on about what an outrage this IPv6 transport is, being charged for those extra bytes, that’s only one part of the picture.

There’s a reason operators are finally embracing IPv6, and it’s not to put an extra 7% of traffic on the network (I think if you asked most capacity planners, they’d say they want data savings, not growth).

IPv6 is, for lack of a better term, less rubbish than IPv4.

There’s a lot of drivers for IPv6, and some of these will reduce data consumption.
IPv6 is actually your stuff talking directly to the remote stuff, this means that we don’t need to rely on NAT, so no need to do NAT keepalives, and opening new sessions, which is going to save you data. If you’re running apps that need to keep a connection to somewhere alive, these data savings could negate your IPv6 overhead costs.

Will these potential data savings when using IPv6 outweigh the costs?

That’s going to depend on your use case.

If you’ve extremely bandwidth / data constrained, for example, you have an IoT device on an NTN / satellite connection, that was having to Push data every X hours via IPv4 because you couldn’t pull data from it as it had no public IP, then moving it to IPv6 so you can pull the data on the public IP, on demand, will save you data. That’s a win with IPv6.

If you’re a mobile user, watching YouTube, getting push notifications and using your phone like a normal human, probably not, but if you’re using data like a normal user, you’ve probably got a sizable data allowance that you don’t end up fully consuming, and the extra 20 bytes per packet will be nothing in comparison to the data used to watch a 2k video on your small phone screen.

CGrateS MySQL Rounding Error

CGrateS, Notes, VoIP, , ,

I put my rates in with a stack of decimal points, because accuracy matters!

But when I manually calculated the outputted costs associated with each transaction, I seemed to have some rounding errors.

So what was the issue?

The schema in MySQL was set to DECIMAL(10,4) which gives us 10 digits after the decimal point and 4 digits after.

A quick alter table and a reimport of the rates and I was on my way!

"alter table tp_rates modify column  rate DECIMAL(10,10);"

Lesson learned and hopefully of use to any other CGrateS users who may be using MySQL as a StoreDB.

Charging problem on Fluke 990 CopperPro

Australian Telco, Notes, , ,

I have a bit of a thing for Test Sets…

Arguably, the most capable (toolbox-carryable) test set out there is the Fluke 990 CopperPro, combined TDR, Butt Set, Toner, RFL, Noise meter, etc, etc, it’s a really cool gizmo (And probably worthy of a blog post in itself someday).

Alas mine had stopped taking much of a charge, you’d plug it into the charger, and after 20 minutes of use, it’d report low power and shut down.

Sitting on the charger the Fluke would show the red charging light for about 20 minutes, then the charging light would turn off.

If I unplugged and replugged the charger I’d get red charging light for another few minutes, then turn off again.

Naively, I ruled it to be an issue with the battery (Which is close to 10 years old), and ordered a new one.

But with the new battery in hand, the same issue.

Stupidly, it turns out I was using a 15v charger, technically the unit supports a 15v charger, but it seems after a period, the internal voltage regulator overheats and shuts off, meaning the battery never got a good charge.

Swapping it with a 12v charger and I’m charging without issues.

But not before I ended up ordering a new battery, so now with the new battery I’m getting 15 hours of runtime out of a charge, and still squeezing 7 out of 8 year old battery it originally came with.

Dumb mistake but hopefully an easy fix for anyone with the same issue.

2022 Telecom Reading List

Notes,

Here’s the list of telecom related books I’ve read in the past year or have plans to read this year. (And here’s the reading list from last year)

Voice Across The Sea / Arthur C. Clarke

I recently finished this one, the book covers the developments of early long distance submarine cables, right up until the 1950s, it’s a good mix of human interest and technological achievement.

I got introduced to Clarke from The Idea Factory – Bell Labs and the Great Age of American Innovation which I read last year, and while it may have been written in the 1950s, this book covers the early days of Gutta-Percha wrapped steel cables being thrown in the ocean, and several failed attempts at linking the Atlantic, up to the Coax systems that were eventually replaced by Fibre after the book was published.

As well as his scientific achievements, and this book, Clarke is perhaps best remembered as the author of 2001: A Space Odyssey.

I’d suggest reading The Undersea Network after Voice Across the Sea, as it somewhat bookends this.

Tubes: A Journey to the Center of the Internet / Andrew Blum

A well written introduction to “The Internet”. If you know AS numbers off the top of your head, you may find it lacking in technical detail, but this book isn’t meant to be a guide on IP Transit Engineering, but rather how the internet was built and continues to run, and the people behind it more than the tech.

The Undersea Network / Nicole Starosielski

A book about the Undersea optic fibre networks, cables, the people behind them and “Turbulent ecologies”.

Girdle Round the Earth – The History of Cable & Wireless / Hugh Barty-King

Having read Tubes, the Undersea Network and Voice Across the Sea, it made sense to learn more about Cable and Wireless’ history, this one has only just arrived, so I’ve yet to finish read it.

FreeSWITCH 1.8 / Anthony Minessale & Giovanni Maruzzelli

My skills were getting rusty having not used FreeSWITCH for a while, and I found this book just as useful on whatever read I’m up to now as it was when I first read it.

Kitten Clone – Inside Alcatel-Lucent / Douglas Coupland

Strange title for a book, but this short read covers Alcatel Lucent’s history, before the NSN / Nokia merger.

Signaling System 7: Basics / Lawrence Harte

A much lighter read than Signaling System No. 7 (SS7/C7), this is a good introduction to the topic, and has been a really good reference for the Demystifying SS7 & Sigtran Networks (With Labs!) series I’ve been writing.

Signaling System No. 7 (SS7/C7): Protocol, Architecture, and Services / Lee Dryburgh, Jeff Hewett

I pick this book up whenever I need a more detail on part of SS7, and I’ve been making an effort to go through the whole thing. It’s very thorough and covers all the “gotchas”.

Ubuntu Cloned VMs getting Duplicate IPs (and yes – the MAC Addresses are unique)

Notes, Software, Uncategorized

So I run a lot of VMs. It’s not unusual when I’m automating something with Ansible or setting up a complex lab to be running 20+ VMs at a time, and often I’ll create a base VM and clone it a dozen times.

Alas, Ubuntu 20.04 has some very irritating default behaviour, where even if the MAC addresses of these cloned VMs differ they get the same IP Address from DHCP.

That’s because by default Netplan doesn’t use the MAC address as the identifier when requesting a DHCP lease. And if you’ve cloned a VM the identifier it does use doesn’t change even if you do change the MAC address…

Irritating, but easily fixed!

Editing the netplan config:

network:
  ethernets:
    eth0:
      dhcp4: true
      dhcp-identifier: mac
  version: 2

Run a netplan-apply and you’re done.

Now you can clone that VM as many times as you like and each will get it’s own unique IP address.

Diff + Wireshark

Notes, RFCs & Standards, Software

Supposedly Archimedes once said:

Give me a lever long enough and a fulcrum on which to place it, and I shall move the world.

For me, the equivalent would be:

Give me a packet capture of the problem occurring and a standards document against which to compare it, and I shall debug the networking world.

And if you’re like me, there’s a good chance when things are really not going your way, you roll up your sleeves, break out Wireshark and the standards docs and begin the painstaking process of trying to work out what’s not right.

Today’s problem involved a side by side comparison between a pcap of a known good call, and one which is failing, so I just had to compare the two, which is slow and fairly error-prone,

So I started looking for something to diff PCAPs easily. The data I was working with was ASN.1 encoded so I couldn’t export as text like you can with HTTP or SIP based protocols and compare it that way.

In the end I stumbled across something even better to compare frames from packet captures side by side, with the decoding intact!

Turns out yo ucan copy the values including decoding from within Wireshark, which means you can then just paste the contents into a diff tool (I’m using the fabulous Meld on Linux, but any diff tool will do including diff itself) and off you go, side-by-side comparison.

Select the first packet/frame you’re interested in (or even just the section), expand the subkeys, right click, copy “All Visible items”. This copy contains all the decoded data, not just the raw bytes, which is what makes it so great.

Next paste it into your diff tool of choice, repeat with the one to compare against, scroll past the data you know is going to be different (session IDs, IPs, etc) and ta-da, there’s the differences.

Video of the whole process below:

Offtopic – HDMI & USB over Twisted Pair

Notes

I wanted to be able to use my desktop computer which lives in my office, on the TV in the living room.

Long HDMI cables would involve me climbing around under the house, and making more holes in the walls, and most wireless keyboard/mouse combos wouldn’t reach that far and USB has a limit of 5 meters.

So instead I put together a rather simple solution that I’m quite happy with.

I ran a lot of Cat5 in the house a while ago, and I’ve got 4 Cat5e sockets behind the TV, and a patch panel at my desk.

I purchased online a HDMI over RJ45/Cat5e adapter, and a USB over RJ45/Cat5e adapter online, for about $5 each.

These are passive devices (baluns) meaning they aren’t converting anything to IP or Ethernet for transport.

This means no additional latency (beyond velocity factor of the cable and the distance, but I digress), so it’s not a remote-desktop experience, it’s like sitting in front of a screen, because that’s what it is.

Very happy with the results!