Turns out it was great training for a career in telecom; I learned basic rigging, working at heights, electrical work, patching rats nests of cables and the shared camaraderie that comes from having stayed up all night working on something no one would ever notice, unless you hadn’t done it.
At the time the Strand 520i lighting console was the coolest thing ever, it had support for 4 DMX universes (2048 channels – Who could ever need more than that!?) and cost more than a new car.
One late Friday night browsing online I found one for sale by a University in the state I live in, for $100 or best offer. You better believe I smashed the buy now button so hard my mouse almost broke. I was going to own my very own Strand 520.
I spent the weekend reading through the old manuals, remembering how to use all the features, then dragged my partner for the road trip the following Monday morning to pick it up and bring it home.
But before I could do anything fun I had to find a PS/2 keyboard and a VGA screen, which took me a few more days (a visit to the tip was the easiest way to get a hold of them).
Then I needed something to receive DMX – I found everything now uses ArtNet (DMX over IP) and there’s visualisers for simulating an arena / stage lighting setup, but all take ArtNet now, so I ordered a DMX to ArtNet converter.
Inside the unit is pretty much a standard PC (An “OG” Pentium in the Strand 520) with an ISA card for all the lighting control stuff.
The clicking hard drive managed to boot, but I didn’t think it’d last long, having been made more than 25 years prior. So I created a disk image and copied the file system onto a CF card using a CF to IDE adapter. This trick meant it booted faster than ever before.
Clicky HDD before it became a CF
One thing I’d read about online was the VARTA battery had a tendency to leak battery acid all over the PCB. This one had yet to fully spill its guts, but was looking a bit bulgey and had started to leak a little already.
The battery (I’d read) was only for storing info in a power loss scenario, and if the battery isn’t present it just slows the boot time as everything has to be read from disk, so I took the leap of faith and cut the battery out, and lo, it still all boots.
Slightly furry VARTA battery
So now I was OK to get the desk online properly, but was getting semi-regular lock ups where DMX would stop outputting and inputs on the console were not read, but the underlying PC was still working.
I spent a lot time debugging this. BIOS settings, interrupts, I dived into how ISA works, replaced the battery I removed with a brand new one, and at one point I broke out the oscilloscope, but nothing worked.
Around the same time I noticed the Ethernet port (BNC!) would work if I just ran plain DOS (could ping from DOS but when the application started the NIC would go dead), which made me think I may be facing a hardware fault with the “CS” – the show processor for the console which is what the motherboard connects to via ISA.
The desk itself with the face off, and the CF adapter being mounted
Alas being almost 30 years old (this unit was made in 1996) there aren’t a great number of them around to test with, so I could try swapping out the “CS” board. What I did find was another complete console on eBay for $50 but it was in the UK, and they weigh a lot. Shipping this thing was not an option.
But for a bit of of extra cash the seller was willing to crack the case open and strip out the two main boards and post me just those. This had added bonus that the motherboard and CPU of the board sent from the UK was a 520i meaning it has the Pentium II processor – This Strand 520 was now going to be a Strand 520i.
A month later a box appeared at my door containing the boards, but the battery on the CS board from the UK had well and truly spilled its guts, leaving some toxic sludge around all the components nearby. A can of PCB cleaner and a toothbrush (which I will not be using to brush teeth with anymore) and I’d cleaned it up as best I could, but the fan output from the board was well and truly dead, with some of the SMD components just eaten by the acid.
So I put everything back into the case and wired it up. The mounts for the Motherboard were slightly different, and the software that is used for the 520i is different from the 520 (without the i).
The HDD from the UK was unable to boot, but I was able to get it to spin up enough to copy off the ~5Mb of files I needed, then I did a fresh install of MS DOS and copied the installer for the StrandOS.
Finally fully functional
Finally I had a stable working console. Not just that but the Strand Networker application was now available to me. So I plugged into the 10Mbps connection and set the console to output to Network as well as DMX.
Enabling the “networker” for network DMX transmission
I cranked open Wireshark and there was a mystery signal sent to the broadcast address on UDP…
I patched a single DMX channel and changed the value and when I viewed the data in Wireshark I could see a hex representation of the DMX 0-255 value.
Easy I thought to myself, it’ll just be a grid of channels, each with their value as hex. Ha! I was wrong.
Turns out Strand Shownet used a conditional form of “Run Length Encoding” compression, where if you’ve got channels 1 through 5 at 50% rather than encoding this is 5 bytes each showing 0x80, it uses 2 bytes, to indicate 5 sequential channels (run length) and then the value (0x80). Then there’s another bit to denote how many forward places to move and if the next channel is using RLE or not.
The code got messy; it’s not the best thing I’ve ever written but it works for 2 full universes of DMX (I need to spend more time to understand where the channel encoding overflow happens as I end up a few channels ahead of where I should be on universe 3 and above).
The code is available on Github and I’d love to know if anyone’s using it with these old dinosaurs!
Last year we deployed some Hughes HL1120W OneWeb terminals in one of the remote cellular networks we support.
Unfortunately it was failing to meet our expectations in terms of performance and reliability – We were seeing multiple dropouts every few hours, for between 30 seconds and ~3 minutes at a time, and while our reseller was great, we weren’t really getting anywhere with Eutelsat in terms of understanding why it wasn’t working.
Luckily for us, Hughes (who manufacture the OneWeb terminals) have an unprotected API (*facepalm*) from which we can scrape all the information about what the terminal sees.
As that data is in an API we have to query, I knocked up a quick Python script to poll the API and convert the data from the API into Prometheus data so we could put it into Grafana and visualise what’s going on with the terminals and the constellation.
After getting all this into Grafana and combining it with the ICMP Blackbox exporter (we configured Blackbox to send HTTP requests and ICMP pings out of each of the different satellite terminals we had (a mix of OneWeb and others)) we could see a pattern emerging where certain “birds” (satellites) that passed overhead would come with packet loss and dropouts.
It was the same satellites each time that led to the drops, which allowed us to pinpoint to say when we see this satellite coming over the horizon, we know there’s going to be some packet loss.
In the end Eutelsat acknowledged they had two faulty satellites in the orbit we are using, hence seeing the dropouts, and they are currently working on resolving this (but that actually does require rockets, so we’re left without a usable service for the time being) but it was a fun problem to diagnose and a good chance to learn more about space.
Packet loss on the two OneWeb terminals (Not seen on other constellation) correlated with a given satellite pass
The repo has instructions for use and the Grafana templates we used.
At one point I started playing with the OneWeb Ephemeris data so I could calculate the azimuth and elevation of each of the birds from our relative position, and work out distances and angles from the terminal. The maths was kinda fun, but oddly the datetimes in the OneWeb ephemeris data set seems to be about 10 years and 10 days behind the current datetime – Possibly this gives an insight into OneWeb’s two day outage at the start of the year due to their software not handling leap years.
Despite all these teething issues I’m still optimistic about OneWeb, Kupler and Qianfan (Thousand Sails) opening up the LEO market and covering more people in more places.
Update: Thanks to Scott via email who sent this: One note, there’s a difference between GPS time and Unix time of about 10 years 5 days. This is due to a) the Unix epoch starting 1970-01-01 and the gps epoch starting 1980-01-05 and b) gps time is not adjusted for leap seconds, and ends up being offset by an integer number of seconds.
Namespace(k='11111111111111111111111111111111', op='22222222222222222222222222222222', opc=None) Generating OPc key from OP & K Generating Multimedia Authentication Vector Input K: b'11111111111111111111111111111111' Input OPc: b'2f3466bd1bea1ac9a8e1ab05f6f43245' Input AMF: b'\x80\x00'
Of course, being open source, you can grab the functions out of this and make a little script to convert everything in a CSV or whatever format your key data is in.
So what about OPc to OP? Well, this is a one-way transaction, we can’t get the OP Key from an OPc & Ki.
The other side of the SCTP connection didn’t like my SCTP parameters.
My SCTP INIT looked like this:
By default, Linux includes support for the ECN and SupportedAddressTypes parameters in the SCTP INIT.
But the other side did not like this, it sent back an ERROR stating that:
Okay, apparently it doesn’t like the fact that we support Forward Transmission Sequence Numbers – So how to turn it off?
I’m using P1Sec’s PySCTP library to interact with the SCTP stack, and I couldn’t find any referneces to this in the code, but then I rememberd that PySCTP is just a wrapper for libsctp so I should be able to control it from there.
Inside /proc/sys/net/sctp we can see all the parameters we can control.
To disable the Forward TSN I need to disable the feature that controls it – Forward Transmission Sequence numbers are introduced in the Partial Reliability Extension (RFC 3758). So it was just a matter of disabling that with:
I run Ubuntu on my desktop and I mess with Kamailio a lot.
Recently I was doing some work with KEMI using Python, but installing the latest versions of Kamailio from the Debian Repos for Kamailio wasn’t working.
The following packages have unmet dependencies:
kamailio-python3-modules : Depends: libpython3.11 (>= 3.11.0) but 3.11.0~rc1-1~22.04 is to be installed
Kamailio’s Python modules expect libpython3.11 or higher, but Ubuntu 22.04 repos only contain the release candidate – not the final version:
In our last post we added a series of different balances to an account, these were actions we took via the API specifically to add a balance.
But there’s a lot more actions we may want to do beyond just adding balance.
CGrateS has the concept of “Actions” which are, as the name suggests, things we want to do to the system.
Some example Actions would be:
Adding / Deducting / Resetting a balance
Adding a CDR log
Enable/Disable an account
Sending HTTP POST request or email notification
Deleting / suspending account
Transferring balances
We can run these actions on a timed basis, or when an event is triggered, and group Actions together to run multiple actions via an ActionTrigger, this means we can trigger these Actions, not just by sending an API request, but based on the state of the subscriber / account.
Let’s look at some examples,
We can define an Action named “Action_Monthly_Fee” to debit $12 from the monetary balance of an account, and add a CDR with the name “Monthly Account Fee” when it does so. We can use ActionTriggers to run this every month on the account automatically.
We can define an Action named “Usage_Warning_10GB” to send an email to the Account owner to inform them they’ve used 10GB of usage, and use ActionTriggers to send this when the customer has used 10GB of their *data balance.
Let’s start basic; to sweeten the deal for new Accounts, we’ll give them $99 of balance to use in the first month they have the service. Rather than hitting the AddBalance API, we’ll define an Action named “Action_Add_Signup_Bonus” to credit $99 of monetary balance to an account.
If you go back to our last post, you should know what we’d need to do to add this balance manually with the AddBalance API, but let’s look at how we can create the same balance add functionality using Actions:
#Add a Signup Bonus of $99 to the account with type *monetary expiring a month after it's added
Action_Signup_Bonus = {
"id": "0",
"method": "ApierV1.SetActions",
"params": [
{
"ActionsId": "Action_Add_Signup_Bonus",
"Actions": [
{
"Identifier": "*topup","BalanceId": "Balance_Signup_Bonus",
"BalanceUuid": "",
"BalanceType": "*monetary",
"Directions": "*out",
"Units": 99,
"ExpiryTime": "*month",
"Filter": "",
"TimingTags": "",
"DestinationIds": "",
"RatingSubject": "",
"Categories": "",
"SharedGroups": "",
"BalanceWeight": 1200,
"ExtraParameters": "",
"BalanceBlocker": "false",
"BalanceDisabled": "false",
"Weight": 10
}
]}]}
pprint.pprint(CGRateS_Obj.SendData(Action_Signup_Bonus))
Alright, this should look pretty familiar if you’ve just come from Account Balances. You’ll notice we’re no longer calling, SetBalance, we’re now calling SetActions, to create the ActionsId with the name “Action_Add_Signup_Bonus“. In “Action_Add_Signup_Bonus” we’ve got an actions we’ll do when “Action_Add_Signup_Bonus” is called. We can define multiple actions, but for now we’ve only got one action defined, which has the Identifier (which defines what the action does) set to *topup to add balance. As you probably guessed, we’re triggering a top up, and setting the BalanceId, BalanceType, Units, ExpiryTime and BalanceWeight just as we would using SetBalance to add a balance.
So how do we use the Action we just created? Well, there’s a lot of options, but let’s start with the most basic – Via the API:
Boom, now we’ll get a CDR created when the Action is triggered.
But let’s push this a bit more and add some more steps in the Action:
As well as adding balance and putting in a CDR to record what we did, let’s also send a notification to our customer via an HTTP API (BYO customer push notification system) and log to Syslog what’s going on.
So what have we done here? We’ve made an ActionPlan named “Action_Add_Signup_Bonus”, which, when associated with an account, will run the Action “Action_Add_Signup_Bonus” as soon as it’s tied to the account, thanks to the Time “*asap“.
Now if we create or update an Account using the SetAccount method, we can set the ActionPlanIds to reference our “ActionPlan_Signup_Bonus” and it’ll be triggered straight away.
Now if we were to run a GetAccount API call, we’ll see the Account balance assigned that was created by the action Action_Add_Signup_Bonus which was triggered by ActionPlan assigned to the account:
But here’s where it gets interesting, in the ActionPlan we just defined the Time was set to “*asap“, which means the Action is triggered as soon as it was assigned to the account, but if we set the Time value to “*monthly“, the Action would get triggered every month, or *every_minute to trigger every minute, or *month_end to trigger at the end of every month.
I’m trying to keep these posts shorter as there’s a lot to cover. Stick around for our next post, we’ll look at some more ActionTriggers to keep decreasing the balance of the account, and setting up ActionTriggers to send a notification to the customer to tell them when their balance is getting low, or any other event based Action you can think of!
Next we’ll need to define our rt_pyform config, this is a super simple 3 line config file that specifies the path of what we’re doing:
DirectoryPath = "." # Directory to search
ModuleName = "script" # Name of python file. Note there is no .py extension
FunctionName = "transform" # Python function to call
The DirectoryPath directive specifies where we should search for the Python code, and ModuleName is the name of the Python script, lastly we have FunctionName which is the name of the Python function that does the rewriting.
Now let’s write our Python function for the transformation.
The Python function much have the correct number of parameters, must return a string, and must use the name specified in the config.
The following is an example of a function that prints out all the values it receives:
Note the order of the arguments and that return is of the same type as the AVP value (string).
We can expand upon this and add conditionals, let’s take a look at some more complex examples:
def transform(appId, flags, cmdCode, HBH_ID, E2E_ID, AVP_Code, vendorID, value):
print('[PYTHON]')
print(f'|-> appId: {appId}')
print(f'|-> flags: {hex(flags)}')
print(f'|-> cmdCode: {cmdCode}')
print(f'|-> HBH_ID: {hex(HBH_ID)}')
print(f'|-> E2E_ID: {hex(E2E_ID)}')
print(f'|-> AVP_Code: {AVP_Code}')
print(f'|-> vendorID: {vendorID}')
print(f'|-> value: {value}')
#IMSI Translation - if App ID = 16777251 and the AVP being evaluated is the Username
if (int(appId) == 16777251) and int(AVP_Code) == 1:
print("This is IMSI '" + str(value) + "' - Evaluating transformation")
print("Original value: " + str(value))
value = str(value[::-1]).zfill(15)
The above look at if the App ID is S6a, and the AVP being checked is AVP Code 1 (Username / IMSI ) and if so, reverses the username, so IMSI 1234567 becomes 7654321, the zfill is just to pad with leading 0s if required.
Now let’s do another one for a Realm Rewrite:
def transform(appId, flags, cmdCode, HBH_ID, E2E_ID, AVP_Code, vendorID, value):
#Print Debug Info
print('[PYTHON]')
print(f'|-> appId: {appId}')
print(f'|-> flags: {hex(flags)}')
print(f'|-> cmdCode: {cmdCode}')
print(f'|-> HBH_ID: {hex(HBH_ID)}')
print(f'|-> E2E_ID: {hex(E2E_ID)}')
print(f'|-> AVP_Code: {AVP_Code}')
print(f'|-> vendorID: {vendorID}')
print(f'|-> value: {value}')
#Realm Translation
if int(AVP_Code) == 283:
print("This is Destination Realm '" + str(value) + "' - Evaluating transformation")
if value == "epc.mnc001.mcc001.3gppnetwork.org":
new_realm = "epc.mnc999.mcc999.3gppnetwork.org"
print("translating from " + str(value) + " to " + str(new_realm))
value = new_realm
else:
#If the Realm doesn't match the above conditions, then don't change anything
print("No modification made to Realm as conditions not met")
print("Updated Value: " + str(value))
In the above block if the Realm is set to epc.mnc001.mcc001.3gppnetwork.org it is rewritten to epc.mnc999.mcc999.3gppnetwork.org, hopefully you can get a handle on the sorts of transformations we can do with this – We can translate any string type AVPs, which allows for hostname, realm, IMSI, Sh-User-Data, Location-Info, etc, etc, to be rewritten.
Sometimes FreeSWITCH XML dialplan is a bit cumbersome to do more complex stuff, particularly to do with interacting with APIs, etc. But we have the option of using scripts written in Python3 to achieve our outcomes and pass variables to/from the dialplan and perform actions as if we were in the dialplan.
For starters we’ll need to install the module and enable it, here’s the StackOverflow thread that got me looking at it where I share the setup steps.
Here is a very simple example I’ve put together to show how we interact with Python3 in FreeSWITCH:
We’ll create a script in /usr/share/freeswitch/scripts/ and call it “CallerName.py”
from freeswitch import *
import sys
def handler(session,args):
#Get Variables from FreeSWITCH
user_name = str(session.getVariable("user_name"))
session.execute("log", "Call from Username: " + str(user_name))
#Check if Username is equal to Nick
if user_name == "Nick":
session.execute("log", "Username is Nick!")
#Convert the Username to Uppercase
session.execute("set", "user_name=" + str(user_name).upper())
#And return to the dialplan
return
else:
#If no matches then log the error
session.execute("log", "CRIT Username is not Nick - Hanging up the call")
#And reject the call
session.execute("hangup", "CALL_REJECTED")
Once we’ve created and saved the file, we’ll need to ensure it is owned by and executable by service user:
After adding this to the dialplan, we’ll need to run a “reloadxml” to reload the dialplan, and now when these actions are hit, the Python script we created will be called, and if the user_name variable is set to “nick” it will be changed to “NICK”, and if it it isn’t, the call will be hung up with a “CALL_REJECTED” response.
Obviously this is a very basic scenario, but I’m using it for things like ACLs from an API, and dynamic call routing, using the familiar and easy to work with Python interpreter.
A lesson learned a long time ago in Net Eng, is that packet captures (seldom) lie, and the answers are almost always in the packets.
The issue is just getting those packets.
The Problem
But if you’re anything like me, you’re working on remote systems from your workstation, and trying to see what’s going on.
For me this goes like this:
SSH into machine in question
Start TCPdump
Hope that I have run it for long enough to capture the event of interest
Stop TCPdump
Change permissions on PCAP file created so I can copy it
SFTP into the machine in question
Transfer the PCAP to my local machine
View the PCAP in Wireshark
Discover I had not run the PCAP for long enough and repeat
Being a Mikrotik user I fell in love with the remote packet sniffer functionality built into them, where the switch/router will copy packets matching a filter and just stream them to the IP of my workstation.
If only there was something I could use to get this same functionality on remote machines – without named pipes, X11 forwarding or any of the other “hacky” solutions…
The Solution
Introducing Scratch’n’Sniff, a simple tcpdump front end that encapsulates all the filtered traffic of interest in TZSP the same as Mikrotiks do, and stream it (in real time) to your local machine for real time viewing in Wireshark.
Using it is very simple:
Capture all traffic on port 5060 on interface enp0s25 and send it to 10.0.1.252 python3 scratchnsniff.py --dstip 10.0.1.252 --packetfilter 'port 5060' --interface enp0s25
Capture all sctp and icmp traffic on interface lo and send it to 10.98.1.2: python3 scratchnsniff.py --dstip 10.98.1.2 --packetfilter 'sctp or icmp' --interface lo
If you’re keen to try it out you can grab it from GitHub – Scratch’n’Sniff and start streaming packets remotely.
If you’re trying to wean yourself away from SQL for everything database related, NoSQL based options like CouchDB are fantastic, but require some rewiring of the brain.
Searching in these databases, without SQL, was a learning curve.
I’m interacting with CouchDB using Python, and while you can create views to help find data, sometimes I knew one of the keys in the document and the value, I wanted to find all documents matching that.
In SQL world I’d do a SELECT WHERE and find the data I wanted, but in CouchDB it’s a bit harder to find.
Using the db.find() function with the {selector} as an input you can filter for all records that have the key value pair you’re looking for.
for doc in db.find({'selector': {'keyyouwanttofind': 'valueofkeyyouwanttofind'}}):
print(doc)
I recently had a bunch of antennas profiles in .msi format, which is the Planet format for storing antenna radiation patterns, but I’m working in Forsk Atoll, so I needed to convert them,
To load these into Atoll, you need to create a .txt file with each of the MSI files in each of the directories, I could do this by hand, but instead I put together a simple Python script you point at the folder full of your MSI files, and it creates the index .txt file containing a list of files, with the directory name.txt, just replace path with the path to your folder full of MSI files,
#Atoll Index Generator
import os
path = "C:\Users\Nick\Desktop\Antennas\ODV-065R15E-G"
antenna_folder = path.split('\\')[-1]
f = open(path + '\\' + 'index_' + str(antenna_folder) + '.txt', 'w+')
files = os.listdir(path)
for individual_file in files:
if individual_file[-4:] == ".msi":
print(individual_file)
f.write(individual_file + "\n")
f.close()
Chances are if you’re reading this, you’re trying to work out what Telephony Binary-Coded Decimal encoding is. I got you.
Again I found myself staring at encoding trying to guess how it worked, reading references that looped into other references, in this case I was encoding MSISDN AVPs in Diameter.
How to Encode a number using Telephony Binary-Coded Decimal encoding?
First, Group all the numbers into pairs, and reverse each pair.
So a phone number of 123456, becomes:
214365
Because 1 & 2 are swapped to become 21, 3 & 4 are swapped to become 34, 5 & 6 become 65, that’s how we get that result.
TBCD Encoding of numbers with an Odd Length?
If we’ve got an odd-number of digits, we add an F on the end and still flip the digits,
For example 789, we add the F to the end to pad it to an even length, and then flip each pair of digits, so it becomes:
87F9
That’s the abbreviated version of it. If you’re only encoding numbers that’s all you’ll need to know.
Detail Overload
Because the numbers 0-9 can be encoded using only 4 bits, the need for a whole 8 bit byte to store this information is considered excessive.
For example 1 represented as a binary 8-bit byte would be 00000001, while 9 would be 00001001, so even with our largest number, the first 4 bits would always going to be 0000 – we’d only use half the available space.
So TBCD encoding stores two numbers in each Byte (1 number in the first 4 bits, one number in the second 4 bits).
To go back to our previous example, 1 represented as a binary 4-bit word would be 0001, while 9 would be 1001. These are then swapped and concatenated, so the number 19 becomes 1001 0001 which is hex 0x91.
Let’s do another example, 82, so 8 represented as a 4-bit word is 1000 and 2 as a 4-bit word is 0010. We then swap the order and concatenate to get 00101000 which is hex 0x28 from our inputted 82.
Final example will be a 3 digit number, 123. As we saw earlier we’ll add an F to the end for padding, and then encode as we would any other number,
F is encoded as 1111.
1 becomes 0001, 2 becomes 0010, 3 becomes 0011 and F becomes 1111. Reverse each pair and concatenate 00100001 11110011 or hex 0x21 0xF3.
Special Symbols (#, * and friends)
Because TBCD Encoding was designed for use in Telephony networks, the # and * symbols are also present, as they are on a telephone keypad.
Astute readers may have noticed that so far we’ve covered 0-9 and F, which still doesn’t use all the available space in the 4 bit area.
The extended DTMF keys of A, B & C are also valid in TBCD (The D key was sacrificed to get the F in).
Symbol
4 Bit Word
*
1 0 1 0
#
1 0 1 1
a
1 1 0 0
b
1 1 0 1
c
1 1 1 0
So let’s run through some more examples,
*21 is an odd length, so we’ll slap an F on the end (*21F), and then encoded each pair of values into bytes, so * becomes 1010, 2 becomes 0010. Swap them and concatenate for our first byte of 00101010 (Hex 0x2A). F our second byte 1F, 1 becomes 0001 and F becomes 1111. Swap and concatenate to get 11110001 (Hex 0xF1). So *21 becomes 0x2A 0xF1.
And as promised, some Python code from PyHSS that does it for you:
def TBCD_special_chars(self, input):
if input == "*":
return "1010"
elif input == "#":
return "1011"
elif input == "a":
return "1100"
elif input == "b":
return "1101"
elif input == "c":
return "1100"
else:
print("input " + str(input) + " is not a special char, converting to bin ")
return ("{:04b}".format(int(input)))
def TBCD_encode(self, input):
print("TBCD_encode input value is " + str(input))
offset = 0
output = ''
matches = ['*', '#', 'a', 'b', 'c']
while offset < len(input):
if len(input[offset:offset+2]) == 2:
bit = input[offset:offset+2] #Get two digits at a time
bit = bit[::-1] #Reverse them
#Check if *, #, a, b or c
if any(x in bit for x in matches):
new_bit = ''
new_bit = new_bit + str(TBCD_special_chars(bit[0]))
new_bit = new_bit + str(TBCD_special_chars(bit[1]))
bit = str(int(new_bit, 2))
output = output + bit
offset = offset + 2
else:
bit = "f" + str(input[offset:offset+2])
output = output + bit
print("TBCD_encode output value is " + str(output))
return output
def TBCD_decode(self, input):
print("TBCD_decode Input value is " + str(input))
offset = 0
output = ''
while offset < len(input):
if "f" not in input[offset:offset+2]:
bit = input[offset:offset+2] #Get two digits at a time
bit = bit[::-1] #Reverse them
output = output + bit
offset = offset + 2
else: #If f in bit strip it
bit = input[offset:offset+2]
output = output + bit[1]
print("TBCD_decode output value is " + str(output))
return output
One feature I’m pretty excited to share is the addition of a single config file for defining how PyHSS functions,
In the past you’d set variables in the code or comment out sections to change behaviour, which, let’s face it – isn’t great.
Instead the config.yaml file defines the PLMN, transport time (TCP or SCTP), the origin host and realm.
We can also set the logging parameters, SNMP info and the database backend to be used,
HSS Parameters
hss:
transport: "SCTP"
#IP Addresses to bind on (List) - For TCP only the first IP is used, for SCTP all used for Transport (Multihomed).
bind_ip: ["10.0.1.252"]
#Port to listen on (Same for TCP & SCTP)
bind_port: 3868
#Value to populate as the OriginHost in Diameter responses
OriginHost: "hss.localdomain"
#Value to populate as the OriginRealm in Diameter responses
OriginRealm: "localdomain"
#Value to populate as the Product name in Diameter responses
ProductName: "pyHSS"
#Your Home Mobile Country Code (Used for PLMN calcluation)
MCC: "999"
#Your Home Mobile Network Code (Used for PLMN calcluation)
MNC: "99"
#Enable GMLC / SLh Interface
SLh_enabled: True
logging:
level: DEBUG
logfiles:
hss_logging_file: log/hss.log
diameter_logging_file: log/diameter.log
database_logging_file: log/db.log
log_to_terminal: true
database:
mongodb:
mongodb_server: 127.0.0.1
mongodb_username: root
mongodb_password: password
mongodb_port: 27017
Stats Parameters
redis:
enabled: True
clear_stats_on_boot: False
host: localhost
port: 6379
snmp:
port: 1161
listen_address: 127.0.0.1
A seperate server (hss_sctp.py) is run to handle SCTP connections, and if you’re looking for Multihoming, we got you dawg – Just edit the config file and set the bind_ip list to include each of your IPs to multi home listen on.
This is part 3 of an n part tutorial series on working with SIM cards.
So in our last post we took a whirlwind tour of what an APDU does, is, and contains.
Interacting with a card involves sending the APDU data to the card as hex, which luckily isn’t as complicated as it seems.
While reading what the hex should look like on the screen is all well and good, actually interacting with cards is the name of the game, so that’s what we’ll be doing today, and we’ll start to abstract some of the complexity away.
Getting Started
To follow along you will need:
A Smart Card reader – SIM card / Smart Card readers are baked into some laptops, some of those multi-card readers that read flash/SD/CF cards, or if you don’t have either of these, they can be found online very cheaply ($2-3 USD).
A SIM card – No need to worry about ADM keys or anything fancy, one of those old SIM cards you kept in the draw because you didn’t know what to do with them is fine, or the SIM in our phone if you can find the pokey pin thing. We won’t go breaking anything, promise.
You may end up fiddling around with the plastic adapters to change the SIM form factor between regular smart card, SIM card (standard), micro and nano.
USB SIM / Smart Card reader supports all the standard form factors makes life a lot easier!
To keep it simple, we’re not going to concern ourselves too much with the physical layer side of things for interfacing with the card, so we’ll start with sending raw APDUs to the cards, and then we’ll use some handy libraries to make life easier.
PCSC Interface
To abstract away some complexity we’re going to use the industry-standard PCSC (PC – Smart Card) interface to communicate with our SIM card. Throughout this series we’ll be using a few Python libraries to interface with the Smart Cards, but under the hood all will be using PCSC to communicate.
pyscard
I’m going to use Python3 to interface with these cards, but keep in mind you can find similar smart card libraries in most common programming languages.
At this stage as we’re just interfacing with Smart Cards, our library won’t have anything SIM-specific (yet).
We’ll use pyscard to interface with the PCSC interface. pyscard supports Windows and Linux and you can install it using PIP with:
pip install pyscard
So let’s get started by getting pyscard to list the readers we have available on our system:
#!/usr/bin/env python3
from smartcard.System import *
print(readers())
Running this will output a list of the readers on the system:
Here we can see the two readers that are present on my system (To add some confusion I have two readers connected – One built in Smart Card reader and one USB SIM reader):
(If your device doesn’t show up in this list, double check it’s PCSC compatible, and you can see it in your OS.)
So we can see when we run readers() we’re returned a list of readers on the system.
I want to use my USB SIM reader (The one identified by Identiv SCR35xx USB Smart Card Reader CCID Interface 00 00), so the next step will be to start a connection with this reader, which is the first in the list.
So to make life a bit easier we’ll store the list of smart card readers and access the one we want from the list;
#!/usr/bin/env python3
from smartcard.System import *
r = readers()
connection = r[0].createConnection()
connection.connect()
So now we have an object for interfacing with our smart card reader, let’s try sending an APDU to it.
Actually Doing something Useful
Today we’ll select the EF that contains the ICCID of the card, and then we will read that file’s binary contents.
This means we’ll need to create two APDUs, one to SELECT the file, and the other to READ BINARY to get the file’s contents.
We’ll set the instruction byte to A4 to SELECT, and B0 to READ BINARY.
Table of Instruction bytes from TS 102 221
APDU to select EF ICCID
The APDU we’ll send will SELECT (using the INS byte value of A4 as per the above table) the file that contains the ICCID.
Each file on a smart card has been pre-created and in the case of SIM cards at least, is defined in a specification.
For this post we’ll be selecting the EF ICCID, which is defined in TS 102 221.
Information about EF-ICCID from TS 102 221
To select it we will need it’s identifier aka File ID (FID), for us the FID of the ICCID EF is 2FE2, so we’ll SELECT file 2FE2.
Parameter 1 – Selection Control (Limit search options)
00 (Select by File ID)
P2
Parameter 1 – More selection options
04 (No data returned)
Lc
Length of Data
02 (2 bytes of data to come)
Data
File ID of the file to Select
2FE2 (File ID of ICCID EF)
So that’s our APDU encoded, it’s final value will be A0 A4 00 04 02 2FE2
So let’s send that to the card, building on our code from before:
#!/usr/bin/env python3
from smartcard.System import *
from smartcard.util import *
r = readers()
connection = r[0].createConnection()
connection.connect()
print("Selecting ICCID File")
data, sw1, sw2 = connection.transmit(toBytes('00a40004022fe2'))
print("Returned data: " + str(data))
print("Returned Status Word 1: " + str(sw1))
print("Returned Status Word 2: " + str(sw2))
If we run this let’s have a look at the output we get,
We got back:
Selecting ICCID File
Returned data: []
Returned Status Word 1: 97
Returned Status Word 2: 33
So what does this all mean?
Well for starters no data has been returned, and we’ve got two status words returned, with a value of 97 and 33.
We can lookup what these status words mean, but there’s a bit of a catch, the values we’re seeing are the integer format, and typically we work in Hex, so let’s change the code to render these values as Hex:
#!/usr/bin/env python3
from smartcard.System import *
from smartcard.util import *
r = readers()
connection = r[0].createConnection()
connection.connect()
print("Selecting ICCID File")
data, sw1, sw2 = connection.transmit(toBytes('00a40004022fe2'))
print("Returned data: " + str(data))
print("Returned Status Word 1: " + str(hex(sw1)))
print("Returned Status Word 2: " + str(hex(sw2)))
Now we’ll get this as the output:
Selecting ICCID File Returned data: [] Returned Status Word 1: 0x61 Returned Status Word 2: 0x1e
Status Word 2 contains a value of 1e which tells us that there are 30 bytes of extra data available with additional info about the file. (We’ll cover this in a later post).
So now we’ve successfully selected the ICCID file.
Keeping in mind with smart cards we have to select a file before we can read it, so now let’s read the binary contents of the file we selected;
The READ BINARY command is used to read the binary contents of a selected file, and as we’ve already selected the file 2FE2 that contains our ICCID, if we run it, it should return our ICCID.
If we consult the table of values for the INS (Instruction) byte we can see that the READ BINARY instruction byte value is B0, and so let’s refer to the spec to find out how we should format a READ BINARY instruction:
Code
Meaning
Value
CLA
Class bytes – Coding options
A0 (ISO 7816-4 coding)
INS
Instruction (Command) to be called
B0 (READ BINARY)
P1
Parameter 1 – Coding / Offset
00 (No Offset)
P2
Parameter 2 – Offset Low
00
Le
How many bytes to read
0A (10 bytes of data to come)
We know the ICCID file is 10 bytes from the specification, so the length of the data to return will be 0A (10 bytes).
Let’s add this new APDU into our code and print the output:
#!/usr/bin/env python3
from smartcard.System import *
from smartcard.util import *
r = readers()
connection = r[0].createConnection()
connection.connect()
print("Selecting ICCID File")
data, sw1, sw2 = connection.transmit(toBytes('00a40000022fe2'))
print("Returned data: " + str(data))
print("Returned Status Word 1: " + str(hex(sw1)))
print("Returned Status Word 2: " + str(hex(sw2)))
And we have read the ICCID of the card.
Phew.
That’s the hardest thing we’ll need to do over.
From now on we’ll be building the concepts we covered here to build other APDUs to get our cards to do useful things. Now you’ve got the basics of how to structure an APDU down, the rest is just changing values here and there to get what you want.
In our next post we’ll read a few more files, write some files and delve a bit deeper into exactly what it is we are doing.
I’ve been adding SNMP support to an open source project I’ve been working on (PyHSS) to generate metrics / performance statistics from it, and this meant staring down SNMP again, but this time I’ve come up with a novel way to handle SNMP, that made it much less painful that normal.
The requirement was simple enough, I already had a piece of software I’d written in Python, but I had a need to add an SNMP server to get information about that bit of software.
For a little more detail – PyHSS handles Device Watchdog Requests already, but I needed a count of how many it had handled, made accessible via SNMP. So inside the logic that does this I just increment a counter in Redis;
In the code example above I just add 1 (increment) the Redis key ‘Answer_280_attempt_count’.
The beauty is that that this required minimal changes to the rest of my code – I just sprinkled in these statements to increment Redis keys throughout my code.
Now when that existing function is run, the Redis key “Answer_280_attempt_count” is incremented.
So I ran my software and the function I just added the increment to was called a few times, so I jumped into redis-cli to check on the values;
And just like that we’ve done all the heavy lifting to add SNMP to our software.
For anything else we want counters on, add the increment to your code to store a counter in Redis with that information.
So next up we need to expose our Redis keys via SNMP,
Then when you run it, presto, you’re exposing that data via SNMP.
You can verify it through SNMP walk or start integrating it into your NMS, in the above example OID 1.3.6.1.2.1.1.1.0.2, contains the value of Answer_280_attempt_count from Redis, and with that, you’re exposing info via SNMP, all while not really having to think about SNMP.
*Ok, you still have to sort which OIDs you assign for what, but you get the idea.
Australia is a strange country; As a kid I was scared of dogs, and in response, our family got a dog.
This year started off with adventures working with ASN.1 encoded data, and after a week of banging my head against the table, I was scared of ASN.1 encoding.
But now I love dogs, and slowly, I’m learning to embrace ASN.1 encoding.
What is ASN.1?
ASN.1 is an encoding scheme.
The best analogy I can give is to image a sheet of paper with a form on it, the form has fields for all the different bits of data it needs,
Each of the fields on the form has a data type, and the box is sized to restrict input, and some fields are mandatory.
Now imagine you take this form and cut a hole where each of the text boxes would be.
We’ve made a key that can be laid on top of a blank sheet of paper, then we can fill the details through the key onto the blank paper and reuse the key over and over again to fill the data out many times.
When we remove the key off the top of our paper, and what we have left on the paper below is the data from the form. Without the key on top this data doesn’t make much sense, but we can always add the key back and presto it’s back to making sense.
While this may seem kind of pointless let’s look at the advantages of this method;
The data is validated by the key – People can’t put a name wherever, and country code anywhere, it’s got to be structured as per our requirements. And if we tried to enter a birthday through the key form onto the paper below, we couldn’t.
The data is as small as can be – Without all the metadata on the key above, such as the name of the field, the paper below contains only the pertinent information, and if a field is left blank it doesn’t take up any space at all on the paper.
It’s these two things, rigidly defined data structures (no room for errors or misinterpretation) and the minimal size on the wire (saves bandwidth), that led to 3GPP selecting ASN.1 encoding for many of it’s protocols, such as S1, NAS, SBc, X2, etc.
It’s also these two things that make ASN.1 kind of a jerk; If the data structure you’re feeding into your ASN.1 compiler does not match it will flat-out refuse to compile, and there’s no way to make sense of the data in its raw form.
But working with a super simple ASN.1 definition you’ve created is one thing, using the 3GPP defined ASN.1 definitions is another,
With the aid of the fantastic PyCrate library, which is where the real magic happens, and this was the nut I cracked this week, compiling a 3GPP ASN.1 definition and communicating a standards-based protocol with it.
I’ve written a playbook that provisions some server infrastructure, however one of the steps is to change the hostname.
A common headache when changing the hostname on a Linux machine is that if the hostname you set for the machine, isn’t in the machine’s /etc/hosts file, then when you run sudo su or su, it takes a really long time before it shows you the prompt as the machine struggles to do a DNS lookup for it’s own hostname and fails,
This becomes an even bigger problem when you’re using Ansible to setup these machines, Ansible times out when changing the hostname;
Simple fix, edit the /etc/ansible/ansible.cfg file and include
# SSH timeout
timeout = 300
And that’s it.
Want more telecom goodness?
I have a good old fashioned RSS feed you can subscribe to.
