I’d tried in the past to use the USB port on the Mikrotik, an external HDD and the SMB server in RouterOS, to act as a simple NAS for sharing files on the home network. And the performance was terrible.
This is because the device is a Router. Not a NAS (duh). And everything I later read online confirmed that yes, this is a router, not a NAS so stop trying.
But I recently got a new Mikrotik CRS109, so now I have a new Mikrotik, how bad is the SMB file share performance?
To test this I’ve got a USB drive with some files on it, an old Mikrotik RB915G and the new Mikrotik CRS109-8G-1S-2HnD-IN, and compared the time it takes to download a file between the two.
Mikrotik Routerboard RB951G
While pulling a 2Gb file of random data from a FAT formatted flash drive, I achieved a consistent 1.9MB/s (15.2 Mb/s)
nick@oldfaithful:~$ smbget smb://10.0.1.1/share1/2Gb_file.bin
Password for [nick] connecting to //share1/10.0.1.1:
Using workgroup WORKGROUP, user nick
smb://10.0.1.1/share1/2Gb_file.bin
Downloaded 2.07GB in 1123 seconds
Mikrotik CRS109
In terms of transfer speed, a consistent 2.8MB/s (22.4 Mb/s)
nick@oldfaithful:~$ smbget smb://10.0.1.1/share1/2Gb_file.bin
Password for [nick] connecting to //share1/10.0.1.1:
Using workgroup WORKGROUP, user nick
smb://10.0.1.1/share1/2Gb_file.bin
Downloaded 2.07GB in 736 seconds
Profiler shows 25% CPU usage on “other”, which drops down as soon as the file transfers stop,
So better, but still not a NAS (duh).
The Verdict
Still not a NAS. So stop trying to use it as a NAS.
As my download speed is faster than 22Mbps I’d just be better to use cloud storage.
A seperate server (hss_sctp.py) is run to handle SCTP connections, and if you’re looking for Multihoming, we got you dawg – Just edit the config file and set the bind_ip list to include each of your IPs to multi home listen on.
Ok, admittedly I haven’t actually seen “When a Stranger Calls”, or the less popular sequel “When a stranger Redials” (Ok may have made the last one up).
But the premise (as I read Wikipedia) is that the babysitter gets the call on the landline, and the police trace the call as originating from the landline.
But you can’t phone yourself, that’s not how local loops work – When the murderer goes off hook it loops the circuit, which busys it. You could apply ring current to the line I guess externally but unless our murder has a Ring generator or has setup a PBX inside the house, the call probably isn’t coming from inside the house.
On Topic – The GMLC
The GMLC (Gateway Mobile Location Centre) is a central server that’s used to locate subscribers within the network on different RATs (GSM/UMTS/LTE/NR).
The GMLC typically has interfaces to each of the radio access technologies, there is a link between the GMLC and the CS network elements (used for GSM/UMTS) such as the HLR, MSC & SGSN via Lh & Lg interfaces, and a link to the PS network elements (LTE/NR) via Diameter based SLh and SLg interfaces with the MME and HSS.
The GMLC’s tentacles run out to each of these network elements so it can query them as to a subscriber’s location,
LTE Call Flow
To find a subscriber’s location in LTE Diameter based signaling is used, to query the MME which in turn queries, the eNodeB to find the location.
But which MME to query?
The SLh Diameter interface is used to query the HSS to find out which MME is serving a particular Subscriber (identified by IMSI or MSISDN).
The LCS-Routing-Info-Request is sent by the GMLC to the HSS with the subscriber identifier, and the LCS-Routing-Info-Response is returned by the HSS to the GMLC with the details of the MME serving the subscriber.
Now we’ve got the serving MME, we can use the SLgDiameter interface to query the MME to the location of that particular subscriber.
The MME can report locations to the GMLC periodically, or the GMLC can request the MME provide a location at that point. For the GMLC to request a subscriber’s current location a Provide-Location-Request is set by the GMLC to the MME with the subscriber’s IMSI, and the MME responds after querying the eNodeB and optionally the UE, with the location info in the Provide-Location-Response.
(I’m in the process of adding support for these interfaces to PyHSS and all going well will release some software shortly to act at a GMLC so people can use this.)
Finding the actual Location
There are a few different ways the actual location of the UE is determined,
At the most basic level, Cell Global Identity (CGI) gives the identity of the eNodeB serving a user. If you’ve got a 3 sector site each sector typically has its own Cell Global Identity, so you can determine to a certain extent, with the known radiation pattern, bearing and location of the sector, in which direction a subscriber is. This happens on the network side and doesn’t require any input from the UE. But if we query the UE’s signal strength, this can then be combined with existing RF models and the signal strength reported by the UE to further pinpoint the user with a bit more accuracy. (Uplink and downlink cell coverage based positioning methods) Barometric pressure and humidity can also be reported by the base station as these factors will impact resulting signal strengths.
Timing Advance (TA) and Time of Arrival (TOA) both rely on timing signals to/from a UE to determine it’s distance from the eNodeB. If the UE is only served by a single cell this gives you a distance from the cell and potentially an angle inside which the subscriber is. This becomes far more useful with 3 or more eNodeBs in working range of the UE, where you can “triangulate” the UE’s location. This part happens on the network side with no interaction with the UE. If the UE supports it, EUTRAN can uses Enhanced Observed Time Difference (E-OTD) positioning method, which does TOD calcuation does this in conjunction with the UE.
GPS Assisted (A-GPS) positioning gives good accuracy but requires the devices to get it’s current location using the GPS, which isn’t part of the baseband typically, so isn’t commonly implimented.
Uplink Time Difference of Arrival (UTDOA) can also be used, which is done by the network.
So why do we need to get Subscriber Locations?
The first (and most noble) use case that springs to mind is finding the location of a subscriber making a call to emergency services. Often upon calling an emergency services number the GMLC is triggered to get the subscriber’s location in case the call is cut off, battery dies, etc.
But GMLCs can also be used for lots of other purposes, marketing purposes (track a user’s location and send targeted ads), surveillance (track movements of people) and network analytics (look at subscriber movement / behavior in a specific area for capacity planning).
Different countries have different laws regulating access to the subscriber location functions.
Hack to disable Location Reporting on Mobile Networks
If you’re wondering how you can disable this functionality, you can try the below hack to ensure that your phone does not report your location.
Press the power button on your phone
Turn it off
In reality, no magic super stealth SIM cards, special phones or fancy firmware will prevent the GMLC from finding your location. So far none of the “privacy” products I’ve looked at have actually done anything special at the Baseband level. Most are just snakeoil.
For as long as your device is connected to the network, the passive ways of determining location, such as Uplink Time Difference of Arrival (UTDOA) and the CGI are going to report your location.
Every now and then when looking into a problem I have to really stop and think about how things work low down, that I haven’t thought about for a long time, and MTU is one of those things.
I faced with an LTE MTU issue recently I thought I’d go back and brush up on my MTU knowhow and do some experimenting.
Note: This is an IPv4 discussion, IPv6 does not support fragmentation.
The very, very basics
MTU is the Maximum Transmission Unit.
In practice this is the largest datagram the layer can handle, and more often than not, this is based on a physical layer constraint, in that different physical layers can only stuff so much into a frame.
“The Internet” from a consumer perspective typically has an MTU of 1500 bytes or perhaps a bit under depending on their carrier, such as 1472 bytes. SANs in data centers typically use an MTU of around 9000 bytes, Out of the box, most devices if you don’t specify, will use an MTU of 1500 bytes.
As a general rule, service providers typically try to offer an MTU as close to 1500 as possible.
Messages that are longer than the Maximum Transmission Unit need to be broken up in a process known as “Fragmenting”. Fragmenting allows large frames to be split into smaller frames to make their way across hops with a lower MTU.
All about Fragmentation
So we can break up larger packets into smaller ones by Fragmenting them, so case closed on MTU right? Sadly not.
Fragmentation leads to reduced efficiency – Fragmenting frames takes up precious CPU cycles on the router performing it, and each time a frame is broken up, additional overhead is added by the device breaking it up, and by the receiver to reassemble it.
Fragmentation can happen multiple times across a path (Multi-Stage Fragmentation). For example if a frame is sent with a length of 9000 bytes, and needs to traverse a hop with an MTU of 4000, it would need to be fragmented (broken up) into 3 frames (Frame 1 and Frame 2 would be ~4000 bytes long and frame 3 would be ~1000 bytes long). If it then needs to traverse another hop with an MTU of 1500, then the 3 fragmented frame would each need to be further fragmented, with the first frame of ~4000 bytes being split up into 3 more fragmented frames. Lost track of what just happened? Spare a thought for the routers having to to do the fragmentation and the recipient having to reassemble their packets.
Fragmented frames are reassembled by the end recipient, other devices along the transmission path don’t reassemble packets.
In the end it boils down to this trade off: The larger the packet can be, the more user data we can stuff into each one as a percentage of the overall data. We want the percentage of user data for each packet to be as high as can be. This means we want to use the largest MTU possible, without having to fragment packets.
Overhead eats into our MTU
A 1500 byte MTU that has to be encapsulated in IPsec, GTP or PPP, is no longer a 1500 byte MTU as far as the customer is concerned.
Any of these encapsulation techniques add overhead, which shrinks the MTU available to the end customer.
This means we’ve got 50 bytes of transmission / transport overhead. This will be important later on!
How do subscribers know what to use as MTU?
Typically when a subscriber buys a DSL service or HFC connection, they’ll either get a preconfigured router from their carrier, or they will be given a list of values to use that includes MTU.
LTE and 5G on the other hand tell us the value we should use.
Inside the Protocol Configuration Options in the NAS PDU, the UE requests the MTU and DNS server to be used, and is provided back from the network.
This MTU value is actually set on the MME, not the P-GW. As the MME doesn’t actually know the maximum MTU of the network, it’s up to the operator to configure this to be a value that represents the network.
Why this Matters for LTE & 5G Transmission
As we covered earlier, fragmentation is costly. If we’re fragmenting packets we are:
Wasting resources on our transmission network / core networks – as we fragment Subscriber packets it’s taking up compute resources and therefore limiting throughput
Wasting radio resources as additional overhead is introduced for fragmented packets, and additional RBs need to be scheduled to handle the fragmented packets
To test this I’ve setup a scenario in the lab, and we’ll look at the packet captures to see how the MTU is advertised, and see how big we can make our MTU on the subscriber side.
I never cease to be amazed as to what I can do with Wireshark.
While we’re working with Smart Card readers and SIM cards, capturing and Decoding USB traffic to see what APDUs are actually being sent can be super useful, so in this post we’ll look at how we can use Wireshark to sniff the USB traffic to view APDUs being sent to smart cards from other software.
For the purposes of this post I’ll be reading the SIM cards with pySim, but in reality it’ll work with any proprietary SIM software, allowing you to see what’s actually being said to the card by your computer.
If you want to see what’s being sent between your phone and SIM card, the Osmocom SIMtrace is the device for you (And yes it also uses Wireshark for viewing this data!).
Ok, that’s all the prerequisites sorted, next we need to find the bus and device ID of our smart card reader,
We can get this listed with
lsusb
Here you can see I have a Smart Card reader on Bus 1 device 03 and another on Bus 2 device 10.
The reader I want to use is the “SCM Microsystems, Inc. SCR35xx USB Smart Card Reader” so I’ll jott down Bus 2 device 10. Yours will obviously be different, but you get the idea.
Finding the USB traffic in Wireshark
Next we’ll fire up Wireshark, if you’ve got your permissions right and followed along, you should see a few more interfaces starting with usbmonX in the capture list.
Because the device I want to capture from is on Bus 2, we’ll select usbmon2 and start capturing,
As you can see we’ve got a bit of a firehose of data, and we only care about device 10 on bus 2, so let’s filter for that.
So let’s generate some data and then filter for it, to generate some data I’m going to run pySim-read to read the data on a smart card that’s connected to my PC, and then filter to only see traffic on that USB device,
In my case as the USB device is 10 it’s got two sub addresses, so I’ll filter for USB Bus 2, device 10 sub-address 1 and 2, so the filter I’ll use is:
usb.addr=="2.10.1" or usb.addr=="2.10.2"
But this doesn’t really show us much, so let’s tell Wireshark this is PCSC/UCCID data to decode it as such;
So we’ll select some of this traffic -> Decode as -> USBCCID
Still not seeing straight APDUs, so let’s tell Wireshark one more bit of information – That we want to decode this information as GSM SIM data;
Again, we’ll select the data part of the USBCCID traffic -> Decode As -> GSM_SIM
And bingo, just like that we can now filter by gsm_sim and see the APDUs being sent / received.
I was working with someone the other day on a problem over a video call and sharing my screen on a Linux box.
They watched as I did what I do often and Grep’ed through my command history to find a command I’d run before but couldn’t remember the specifics of off the top of my head.
Something like this:
How I’ve always found commands from my history
Then the person I was working with told me to try pressing Control + R and typing the start of what I was looking for.
My head exploded.
Searching through Bash command history
From here you can search through your command history and scroll between matching entries,
I cannot believe it’s taken me this long to learn!
This is part 3 of an n part tutorial series on working with SIM cards.
So in our last post we took a whirlwind tour of what an APDU does, is, and contains.
Interacting with a card involves sending the APDU data to the card as hex, which luckily isn’t as complicated as it seems.
While reading what the hex should look like on the screen is all well and good, actually interacting with cards is the name of the game, so that’s what we’ll be doing today, and we’ll start to abstract some of the complexity away.
Getting Started
To follow along you will need:
A Smart Card reader – SIM card / Smart Card readers are baked into some laptops, some of those multi-card readers that read flash/SD/CF cards, or if you don’t have either of these, they can be found online very cheaply ($2-3 USD).
A SIM card – No need to worry about ADM keys or anything fancy, one of those old SIM cards you kept in the draw because you didn’t know what to do with them is fine, or the SIM in our phone if you can find the pokey pin thing. We won’t go breaking anything, promise.
You may end up fiddling around with the plastic adapters to change the SIM form factor between regular smart card, SIM card (standard), micro and nano.
USB SIM / Smart Card reader supports all the standard form factors makes life a lot easier!
To keep it simple, we’re not going to concern ourselves too much with the physical layer side of things for interfacing with the card, so we’ll start with sending raw APDUs to the cards, and then we’ll use some handy libraries to make life easier.
PCSC Interface
To abstract away some complexity we’re going to use the industry-standard PCSC (PC – Smart Card) interface to communicate with our SIM card. Throughout this series we’ll be using a few Python libraries to interface with the Smart Cards, but under the hood all will be using PCSC to communicate.
pyscard
I’m going to use Python3 to interface with these cards, but keep in mind you can find similar smart card libraries in most common programming languages.
At this stage as we’re just interfacing with Smart Cards, our library won’t have anything SIM-specific (yet).
We’ll use pyscard to interface with the PCSC interface. pyscard supports Windows and Linux and you can install it using PIP with:
pip install pyscard
So let’s get started by getting pyscard to list the readers we have available on our system:
#!/usr/bin/env python3
from smartcard.System import *
print(readers())
Running this will output a list of the readers on the system:
Here we can see the two readers that are present on my system (To add some confusion I have two readers connected – One built in Smart Card reader and one USB SIM reader):
(If your device doesn’t show up in this list, double check it’s PCSC compatible, and you can see it in your OS.)
So we can see when we run readers() we’re returned a list of readers on the system.
I want to use my USB SIM reader (The one identified by Identiv SCR35xx USB Smart Card Reader CCID Interface 00 00), so the next step will be to start a connection with this reader, which is the first in the list.
So to make life a bit easier we’ll store the list of smart card readers and access the one we want from the list;
#!/usr/bin/env python3
from smartcard.System import *
r = readers()
connection = r[0].createConnection()
connection.connect()
So now we have an object for interfacing with our smart card reader, let’s try sending an APDU to it.
Actually Doing something Useful
Today we’ll select the EF that contains the ICCID of the card, and then we will read that file’s binary contents.
This means we’ll need to create two APDUs, one to SELECT the file, and the other to READ BINARY to get the file’s contents.
We’ll set the instruction byte to A4 to SELECT, and B0 to READ BINARY.
Table of Instruction bytes from TS 102 221
APDU to select EF ICCID
The APDU we’ll send will SELECT (using the INS byte value of A4 as per the above table) the file that contains the ICCID.
Each file on a smart card has been pre-created and in the case of SIM cards at least, is defined in a specification.
For this post we’ll be selecting the EF ICCID, which is defined in TS 102 221.
Information about EF-ICCID from TS 102 221
To select it we will need it’s identifier aka File ID (FID), for us the FID of the ICCID EF is 2FE2, so we’ll SELECT file 2FE2.
Parameter 1 – Selection Control (Limit search options)
00 (Select by File ID)
P2
Parameter 1 – More selection options
04 (No data returned)
Lc
Length of Data
02 (2 bytes of data to come)
Data
File ID of the file to Select
2FE2 (File ID of ICCID EF)
So that’s our APDU encoded, it’s final value will be A0 A4 00 04 02 2FE2
So let’s send that to the card, building on our code from before:
#!/usr/bin/env python3
from smartcard.System import *
from smartcard.util import *
r = readers()
connection = r[0].createConnection()
connection.connect()
print("Selecting ICCID File")
data, sw1, sw2 = connection.transmit(toBytes('00a40004022fe2'))
print("Returned data: " + str(data))
print("Returned Status Word 1: " + str(sw1))
print("Returned Status Word 2: " + str(sw2))
If we run this let’s have a look at the output we get,
We got back:
Selecting ICCID File
Returned data: []
Returned Status Word 1: 97
Returned Status Word 2: 33
So what does this all mean?
Well for starters no data has been returned, and we’ve got two status words returned, with a value of 97 and 33.
We can lookup what these status words mean, but there’s a bit of a catch, the values we’re seeing are the integer format, and typically we work in Hex, so let’s change the code to render these values as Hex:
#!/usr/bin/env python3
from smartcard.System import *
from smartcard.util import *
r = readers()
connection = r[0].createConnection()
connection.connect()
print("Selecting ICCID File")
data, sw1, sw2 = connection.transmit(toBytes('00a40004022fe2'))
print("Returned data: " + str(data))
print("Returned Status Word 1: " + str(hex(sw1)))
print("Returned Status Word 2: " + str(hex(sw2)))
Now we’ll get this as the output:
Selecting ICCID File Returned data: [] Returned Status Word 1: 0x61 Returned Status Word 2: 0x1e
Status Word 2 contains a value of 1e which tells us that there are 30 bytes of extra data available with additional info about the file. (We’ll cover this in a later post).
So now we’ve successfully selected the ICCID file.
Keeping in mind with smart cards we have to select a file before we can read it, so now let’s read the binary contents of the file we selected;
The READ BINARY command is used to read the binary contents of a selected file, and as we’ve already selected the file 2FE2 that contains our ICCID, if we run it, it should return our ICCID.
If we consult the table of values for the INS (Instruction) byte we can see that the READ BINARY instruction byte value is B0, and so let’s refer to the spec to find out how we should format a READ BINARY instruction:
Code
Meaning
Value
CLA
Class bytes – Coding options
A0 (ISO 7816-4 coding)
INS
Instruction (Command) to be called
B0 (READ BINARY)
P1
Parameter 1 – Coding / Offset
00 (No Offset)
P2
Parameter 2 – Offset Low
00
Le
How many bytes to read
0A (10 bytes of data to come)
We know the ICCID file is 10 bytes from the specification, so the length of the data to return will be 0A (10 bytes).
Let’s add this new APDU into our code and print the output:
#!/usr/bin/env python3
from smartcard.System import *
from smartcard.util import *
r = readers()
connection = r[0].createConnection()
connection.connect()
print("Selecting ICCID File")
data, sw1, sw2 = connection.transmit(toBytes('00a40000022fe2'))
print("Returned data: " + str(data))
print("Returned Status Word 1: " + str(hex(sw1)))
print("Returned Status Word 2: " + str(hex(sw2)))
And we have read the ICCID of the card.
Phew.
That’s the hardest thing we’ll need to do over.
From now on we’ll be building the concepts we covered here to build other APDUs to get our cards to do useful things. Now you’ve got the basics of how to structure an APDU down, the rest is just changing values here and there to get what you want.
In our next post we’ll read a few more files, write some files and delve a bit deeper into exactly what it is we are doing.
I’ve been adding SNMP support to an open source project I’ve been working on (PyHSS) to generate metrics / performance statistics from it, and this meant staring down SNMP again, but this time I’ve come up with a novel way to handle SNMP, that made it much less painful that normal.
The requirement was simple enough, I already had a piece of software I’d written in Python, but I had a need to add an SNMP server to get information about that bit of software.
For a little more detail – PyHSS handles Device Watchdog Requests already, but I needed a count of how many it had handled, made accessible via SNMP. So inside the logic that does this I just increment a counter in Redis;
In the code example above I just add 1 (increment) the Redis key ‘Answer_280_attempt_count’.
The beauty is that that this required minimal changes to the rest of my code – I just sprinkled in these statements to increment Redis keys throughout my code.
Now when that existing function is run, the Redis key “Answer_280_attempt_count” is incremented.
So I ran my software and the function I just added the increment to was called a few times, so I jumped into redis-cli to check on the values;
And just like that we’ve done all the heavy lifting to add SNMP to our software.
For anything else we want counters on, add the increment to your code to store a counter in Redis with that information.
So next up we need to expose our Redis keys via SNMP,
Then when you run it, presto, you’re exposing that data via SNMP.
You can verify it through SNMP walk or start integrating it into your NMS, in the above example OID 1.3.6.1.2.1.1.1.0.2, contains the value of Answer_280_attempt_count from Redis, and with that, you’re exposing info via SNMP, all while not really having to think about SNMP.
*Ok, you still have to sort which OIDs you assign for what, but you get the idea.
It’s 2021, and everyone loves Containers; Docker & Kubernetes are changing how software is developed, deployed and scaled.
And yet so much of the Telco world still uses bare metal servers and dedicated hardware for processing.
So why not use Containers or VMs more for VoIP applications?
Disclaimer – When I’m talking VoIP about VoIP I mean the actual Voice over IP, that’s the Media Stream, RTP, the Audio, etc, not the Signaling (SIP). SIP is fine with Containers, it’s the media that has a bad time and that this post focuses on,
Virtualization Fundamentals
Once upon a time in Development land every application ran on it’s own server running in a DC / Central Office.
This was expensive to deploy (buying servers), operate (lots of power used) and maintain (lots of hardware to keep online).
Each server was actually sitting idle for a large part of the time, with the application running on it only using a some of the available resources some of the time.
One day Virtualization came and suddenly 10 physical servers could be virtualized into 10 VMs.
These VMs still need to run on servers but as each VM isn’t using 100% of it’s allocated resources all the time, instead of needing 10 servers to run it on you could run it on say 3 servers, and even do clever things like migrate VMs between servers if one were to fail.
VMs share the resources of the server it’s running on.
A server running VMs (Hypervisor) is able to run multiple VMs by splitting the resources between VMs.
If a VM A wants to run an operation at the same time a VM B & VM C, the operations can’t be run on each VM at the same time* so the hypervisor will queue up the requests and schedule them in, typically based on first-in-first out or based on a resource priority policy on the Hypervisor.
This is fine for a if VM A, B & C were all Web Servers. A request coming into each of them at the same time would see the VM the Hypervisor schedules the resources to respond to the request slightly faster, with the other VMs responding to the request when the hypervisor has scheduled the resources to the respective VM.
VoIP is an only child
VoIP has grown up on dedicated hardware. It’s an only child that does not know how to share, because it’s never had to.
Having to wait for resources to be scheduled by the Hypervisor to to VM in order for it to execute an operation is fine and almost unnoticeable for web servers, it can have some pretty big impacts on call quality.
If we’re running RTPproxy or RTPengine in order to relay media, scheduling delays can mean that the media stream ends up “bursty”.
RTP packets needing relaying are queued in the buffer on the VM and only relayed when the hypervisor is able to schedule resources, this means there can be a lot of packet-delay-variation (PDV) and increased latency for services running on VMs.
VMs and Containers both have this same fate, DPDK and SR-IOV assist in throughput, but they don’t stop interrupt headaches.
VMs that deprive other VMs on the same host of resources are known as “Noisy neighbors”.
The simple fix for all these problems? There isn’t one.
Each of these issues can be overcome, dedicating resources, to a specific VM or container, cleverly distributing load, but it is costly in terms of resources and time to tweak and implement, and some of these options undermine the value of virtualization or containerization.
As technology marches forward we have scenarios where Kubernetes can expose FPGA resources to pass them through to Pods, but right now, if you need to transcode more than ~100 calls efficiently, you’re going to need a hardware device.
And while it can be done by throwing more x86 / ARM compute resources at the problem, hardware still wins out as cheaper in most instances.
Australia is a strange country; As a kid I was scared of dogs, and in response, our family got a dog.
This year started off with adventures working with ASN.1 encoded data, and after a week of banging my head against the table, I was scared of ASN.1 encoding.
But now I love dogs, and slowly, I’m learning to embrace ASN.1 encoding.
What is ASN.1?
ASN.1 is an encoding scheme.
The best analogy I can give is to image a sheet of paper with a form on it, the form has fields for all the different bits of data it needs,
Each of the fields on the form has a data type, and the box is sized to restrict input, and some fields are mandatory.
Now imagine you take this form and cut a hole where each of the text boxes would be.
We’ve made a key that can be laid on top of a blank sheet of paper, then we can fill the details through the key onto the blank paper and reuse the key over and over again to fill the data out many times.
When we remove the key off the top of our paper, and what we have left on the paper below is the data from the form. Without the key on top this data doesn’t make much sense, but we can always add the key back and presto it’s back to making sense.
While this may seem kind of pointless let’s look at the advantages of this method;
The data is validated by the key – People can’t put a name wherever, and country code anywhere, it’s got to be structured as per our requirements. And if we tried to enter a birthday through the key form onto the paper below, we couldn’t.
The data is as small as can be – Without all the metadata on the key above, such as the name of the field, the paper below contains only the pertinent information, and if a field is left blank it doesn’t take up any space at all on the paper.
It’s these two things, rigidly defined data structures (no room for errors or misinterpretation) and the minimal size on the wire (saves bandwidth), that led to 3GPP selecting ASN.1 encoding for many of it’s protocols, such as S1, NAS, SBc, X2, etc.
It’s also these two things that make ASN.1 kind of a jerk; If the data structure you’re feeding into your ASN.1 compiler does not match it will flat-out refuse to compile, and there’s no way to make sense of the data in its raw form.
But working with a super simple ASN.1 definition you’ve created is one thing, using the 3GPP defined ASN.1 definitions is another,
With the aid of the fantastic PyCrate library, which is where the real magic happens, and this was the nut I cracked this week, compiling a 3GPP ASN.1 definition and communicating a standards-based protocol with it.
One of the key advantages of SCTP over TCP is the support for Multihoming,
From an application perspective, this enables one “socket”, to be shared across multiple IP Addresses, allowing multiple IP paths and physical NICs.
Through multihoming we can protect against failures in IP Routing and physical links, from a transport layer protocol.
So let’s take a look at how this actually works,
For starters there’s a few ways multihoming can be implemented, ideally we have multiple IPs on both ends (“client” and “server”), but this isn’t always achievable, so SCTP supports partial multi-homing, where for example the client has only one IP but can contact the server on multiple IP Addresses, and visa-versa.
The below image (Courtesy of Wikimedia) shows the ideal scenario, where both the client and the server have multiple IPs they can be reached on.
This would mean a failure of any one of the IP Addresses or the routing between them, would see the other secondary IP Addresses used for Transport, and the application not even necessarily aware of the interruption to the primary IP Path.
The Process
For starters, our SCTP Client/Server will each need to be aware of the IPs that can be used,
This is advertised in the INIT message, sent by the “client” to a “server” when the SCTP session is established.
SCTP INIT sent by the client at 10.0.1.185, but advertising two IPs
In the above screenshot we can see the two IPs for SCTP to use, the primary IP is the first one (10.0.1.185) and also the from IP, and there is just one additional IP (10.0.1.187) although there could be more.
In a production environment you’d want to ensure each of your IPs is in a different subnet, with different paths, hardware and routes.
So the INIT is then responded to by the client with an INIT_ACK, and this time the server advertises it’s IP addresses, the primary IP is the From IP address (10.0.1.252) and there is just one additional IP of 10.0.1.99,
SCTP INIT ACK showing Server’s Multi-homed IP Options
Next up we have the cookie exchange, which is used to protect against synchronization attacks, and then our SCTP session is up.
So what happens at this point? How do we know if a path is up and working?
Well the answer is heartbeat messages,
Sent from each of the IPs on the client to each of the IPs on the server, to make sure that there’s a path from every IP, to every other IP.
SCTP Heartbeats from each local IP to each remote IP
This means the SCTP stacks knows if a path fails, for example if the route to IP 10.0.1.252 on the server were to fail, the SCTP stack knows it has another option, 10.0.1.99, which it’s been monitoring.
So that’s multi-homed SCTP in action – While a lot of work has historically been done with LACP for aggregating multiple NICs together, and VRRP for ensuring a host is alive, SCTP handles this in a clean and efficient way.
I’ve attached a PCAP showing multi-homing on a Diameter S6a (HSS) interface between an MME and a HSS.
So dedicated appliances are dead and all our network functions are VMs or Containers, but there’s a performance hit when going virtual as the L2 processing has to be handled by the Hypervisor before being passed onto the relevant VM / Container.
If we have a 10Gb NIC in our server, we want to achieve a 10Gbps “Line Speed” on the Network Element / VNF we’re running on.
When we talked about appliances if you purchased an P-GW with 10Gbps NIC, it was a given you could get 10Gbps through it (without DPI, etc), but when we talk about virtualized network functions / network elements there’s a very real chance you won’t achieve the “line speed” of your interfaces without some help.
When you’ve got a Network Element like a S-GW, P-GW or UPF, you want to forward packets as quickly as possible – bottlenecks here would impact the user’s achievable speeds on the network.
To speed things up there are two technologies, that if supported by your software stack and hardware, allows you to significantly increase throughput on network interfaces, DPDK & SR-IOV.
DPDK – Data Plane Development Kit
Usually *Nix OSs handle packet processing on the Kernel level. As I type this the packets being sent to this WordPress server by Firefox are being handled by the Linux 5.8.0-36-generic kernel running on my machine.
The problem is the kernel has other things to do (interrupts), meaning increased delay in processing (due to waiting for processing capability) and decreased capacity.
DPDK shunts this processing to the “user space” meaning your application (the actual magic of the VNF / Network Element) controls it.
To go back to me writing this – If Firefox and my laptop supported DPDK, then the packets wouldn’t traverse the Linux kernel at all, and Firefox would be talking directly to my NIC. (Obviously this isn’t the case…)
So DPDK increases network performance by shifting the processing of packets to the application, bypassing the kernel altogether. You are still limited by the CPU and Memory available, but with enough of each you should reach very near to line speed.
SR-IOV – Single Root Input Output Virtualization
Going back to the me writing this analogy I’m running Linux on my laptop, but let’s imagine I’m running a VM running Firefox under Linux to write this.
If that’s the case then we have an even more convolted packet processing chain!
I type the post into Firefox which sends the packets to the Linux kernel, which waits to be scheduled resources by the hypervisor, which then process the packets in the hypervisor kernel before finally making it onto the NIC.
We could add DPDK which skips some of these steps, but we’d still have the bottleneck of the hypervisor.
With PCIe passthrough we could pass the NIC directly to the VM running the Firefox browser window I’m typing this, but then we have a problem, no other VMs can access these resources.
SR-IOV provides an interface to passthrough PCIe to VMs by slicing the PCIe interface up and then passing it through.
My VM would be able to access the PCIe side of the NIC, but so would other VMs.
So that’s the short of it, SR-IOR and DPDK enable better packet forwarding speeds on VNFs.
While we’ve already covered the inputs required by the authentication elements of the core network (The HSS in LTE/4G, the AuC in UMTS/3G and the AUSF in 5G) to generate an output, it’s worth noting that the Confidentiality Algorithms used in the process determines the output.
This means the Authentication Vector (Also known as an F1 and F1*) generated for a subscriber using Milenage Confidentiality Algorithms will generate a different output to that of Confidentiality Algorithms XOR or Comp128.
To put it another way – given the same input of K key, OPc Key (or OP key), SQN & RAND (Random) a run with Milenage (F1 and F1* algorithm) would yield totally different result (AUTN & XRES) to the same inputs run with a simple XOR.
Technically, as operators control the network element that generates the challenges, and the USIM that responds to them, it is an option for an operator to implement their own Confidentiality Algorithms (Beyond just Milenage or XOR) so long as it produced the same number of outputs. But rolling your own cryptographic anything is almost always a terrible idea.
So what are the differences between the Confidentiality Algorithms and which one to use? Spoiler alert, the answer is Milenage.
Milenage
Milenage is based on AES (Originally called Rijndael) and is (compared to a lot of other crypto implimentations) fairly easy to understand,
AES is very well studied and understood and unlike Comp128 variants, is open for anyone to study/analyse/break, although AES is not without shortcomings, it’s problems are at this stage, fairly well understood and mitigated.
There are a few clean open source examples of Milenage implementations, such as this C example from FreeBSD.
XOR
It took me a while to find the specifications for the XOR algorithm – it turns out XOR is available as an alternate to Milenage available on some SIM cards for testing only, and the mechanism for XOR Confidentiality Algorithm is only employed in testing scenarios, not designed for production.
Instead of using AES under the hood like Milenage, it’s just plan old XOR of the keys.
Comp128 was originally a closed source algorithm, with the maths behind it not publicly available to scrutinise. It is used in GSM A3 and A5 functions, akin to the F1 and F1* in later releases.
Due to its secretive nature it wasn’t able to be studied or analysed prior to deployment, with the idea that if you never said how your crypto worked no one would be able to break it. Spoiler alert; public weaknesses became exposed as far back as 1998, which led to Toll Fraud, SIM cloning and eventually the development of two additional variants, with the original Comp128 renamed Comp128-1, and Comp128-2 (stronger algorithm than the original addressing a few of its flaws) and Comp128-3 (Same as Comp128-2 but with a 64 bit long key generated).
Instead of going to all the effort of creating VMs (or running Ansible playbooks) we can use Docker and docker-compose to create a test environment with multiple Asterisk instances to dispatch traffic to from Kamailio.
I covered the basics of using Kamailio with Docker in this post, which runs a single Kamailio instance inside Docker with a provided config file, but in this post we’ll use docker-compose to run multiple Asterisk instances and setup Kamailio to dispatch traffic to them.
I am a big Kubernetes fan, and yes, all this can be done in Kubernetes, and would be a better fit for a production environment, but for a development environment it’s probably overkill.
#Copy the config file onto the Filesystem of the Docker instance
COPY dispatcher.list /etc/kamailio/
COPY kamailio.cfg /etc/kamailio/
The Kamailio config we’re using is very similar to the Dispatcher example but with a few minor changes to the timers and setting it to use the Dispatcher data from a text file instead of a database. If you have a look at the contents of dispatcher.list you’ll see three entries; dispatcher_w_docker_asterisk_1, dispatcher_w_docker_asterisk_2 & dispatcher_w_docker_asterisk_3. These will be the hostnames of the 3 Asterisk instances we’ll create.
Next up we’ll take a look at the docker-compose file, which defines how our environment will be composed, and defines which containers will be run
The docker-compose file contains definitions about the Containers we want to run, for this example we’ll run several Asterisk instances and a single Kamailio instance.
The replicas: 6 parameter is ignored by standard docker-compose up command, but will be used if you’re using Docker swarm, otherwise we’ll manually set the number of replicas when we run the command.
So with that defined let’s define our Kamailio service;
So far with most of our discussions about Kamailio we’ve talked about routing the initial SIP request (INVITE, REGISTER, SUBSCRIBE, etc), but SIP is not a one-message protocol, there’s a whole series of SIP messages that go into a SIP Dialog.
Sure the call may start with an INVITE, but there’s the 180 RINGING, the 200 OK and the ACK that go into getting the call actually established, and routing these in-dialog messages is just as important as routing the first INVITE.
When we’ve talked about SIP routing it’s all happened in the request_route {} block:
request_route {
xlog("Received $rm to $ru - Forwarding");
append_hf("X-Proxied: You betcha\r\n");
#Forward to new IP
forward("192.168.1.110");
}
In the example above we statelessly forward any initial requests to the IP 192.168.1.110.
We can add an onreply_route{} block to handle any replies from 192.168.1.110 back to the originator.
But why would we want to?
Some simple answers would be to do some kind of manipulation to the message – say to strip a Caller ID if CLIP is turned off, or to add a custom SIP header containing important information, etc.
onreply_route{
xlog("Got a reply $rs");
append_hf("X-Proxied: For the reply\r\n");
}
Let’s imagine a scenario where the destination our SIP proxy is relaying traffic to (192.168.1.110) starts responding with 404 error.
We could detect this in our onreply_route{} and do something about it.
onreply_route{
xlog("Got a reply $rs");
if($rs == 404) {
#If remote destination returns 404
xlog("Got a 404 for $rU");
#Do something about it
}
}
In the 404 example if we were using Dispatcher it’s got easily accessed logic to handle these scenarios a bit better than us writing it out here, but you get the idea.
There are a few other special routes like onreply_route{}, failure routes and event routes, etc.
Hopefully now you’ll have a better idea of how and when to use onreply_route{} in Kamailio.
The 900 and 1800 pair telecom distribution pillars (aka cabinets) are still a familiar sight almost everywhere in Australia where copper networks are still used, however prior to the early 1970s they were only deployed in metropolitan areas, and apparently one of the concerns of deploying them in rural areas was that they’d be shot at.
June 1966 issue of the Telecommunications Journal of Australia (TJA) has an article titled “Aluminum Distribution Cabinets – Resistance to Rifle Fire” is below, click to get the image full size.
At the short range, the vandal is expected to have either sufficient common sense or hard earned experience to realise the danger of ricochet.
Our BTS is going to need an accurate clock source in order to run, so without access to crazy accurate Timing over Packet systems or TDM links to use as reference sources, I’ve opted to use the GPS/GLONASS receiver built into the LMPT card.
Add new GPS with ID 0 on LMPT in slot 7 of cabinet 1:
Check GPS has sync (May take some time) using the Display GPS command;
DSP GPS: GN=0;
Assuming you’ve got an antenna connected and can see the sky, after ~10 minutes running the DSP GPS:; command again should show you an output like this:
+++ 4-PAL0089624 2020-11-28 01:06:55
O&M #806355684
%%DSP GPS: GN=0;%%
RETCODE = 0 Operation succeeded.
Display GPS State
-----------------
GPS Clock No. = 0
GPS Card State = Normal
GPS Card Type = M12M
GPS Work Mode = GPS
Hold Status = UNHOLDED
GPS Satellites Traced = 4
GLONASS Satellites Traced = 0
BDS Satellites Traced = 0
Antenna Longitude(1e-6 degree) = 144599999
Antenna Latitude(1e-6 degree) = -37000000
Antenna Altitude(m) = 613
Antenna Angle(degree) = 5
Link Active State = Activated
Feeder Delay(ns) = 15
GPS Version = NULL
(Number of results = 1)
--- END
Showing the GPS has got sync and a location fix,
Next we set BTS to use GPS as time source,
SET TIMESRC: TIMESRC=GPS;
Finally we’ll verify the Time is in sync on the BTS using the list time command:
DSP TIME:;
+++ 4-PAL0089624 2020-11-28 01:09:22
O&M #806355690
%%DSP TIME:;%%
RETCODE = 0 Operation succeeded.
Time Information
----------------
Time = 2020-11-28 01:09:22 GMT+00:00
--- END
Optionally you may wish to add a timezone, using the SET TZ:; command, but I’ve opted to keep it in UTC for simplicity.
In our last post we covered the file system structure of a smart card and the basic concepts of communication with cards. In this post we’ll look at what happens on the application layer, and how to interact with a card.
For these examples I’ll be using SIM cards, because admit it, you’ve already got a pile sitting in a draw, and this is a telco blog after all. You won’t need the ADM keys for the cards, we’ll modify files we’ve got write access to by default.
Commands & Instructions
So to do anything useful with the card we need issue commands / instructions to the card, to tell it to do things. Instructions like select this file, read it’s contents, update the contents to something else, verify my PIN, authenticate to the network, etc.
The term Command and Instruction are used somewhat interchangeably in the spec, I realise that I’ve done the same here to make it just as confusing, but instruction means the name of the specific command to be called, and command typically means the APDU as a whole.
The “Generic Commands” section of 3GPP TS 31.101 specifies the common commands, so let’s take a look at one.
The creatively named SELECT command/instruction is used to select the file we want to work with. In the SELECT command we’ll include some parameters, like where to find the file, so some parameters are passed with the SELECT Instruction to limit the file selection to a specific area, etc, the length of the file identifier to come, and the identifier of the file.
The card responds with a Status Word, returned by the card, to indicate if it was successful. For example if we selected a file that existed and we had permission to select, we’d get back a status word indicating the card had successfully selected the file. Status Words are 2 byte responses that indicate if the instruction was successful, but also the card has data it wants to send to the terminal as a result of the instruction, how much data the terminal should expect.
So if we just run a SELECT command, telling the card to select a file, we’ll get back a successful response from the card with a data length. Next need to get that data from the card. As the card can’t initiate communication, the GET RESPONSE instruction is sent to the card to get the data from the card, along with the length of the data to be returned.
The GET RESPONSE instruction/command is answered by the card with an APDU containing the data the card has to send, and the last 2 bytes contain the Status Word indicating if it was successful or not.
APDUs
So having covered the physical and link layers, we now move onto the Application Layer – where the magic happens.
Smart card communications is strictly master-slave based when it comes to the application layer.
The terminal sends a command to the card, which in turn sends back a response. Command -> Response, Command -> Response, over and over.
These commands are contained inside APplication Data Units (APDUs).
So let’s break down a simple APDU as it appears on the wire, so to speak.
The first byte of our command APDU is taken up with a header called the class byte, abbreviated to CLA. This specifies class coding, secure messaging options and channel options.
In the next byte we specify the Instruction for the command, that’s the task / operation we want the card to perform, in the spec this is abbreviated to INS.
The next two bytes, called P1 & P2 (Parameter 1 & Parameter 2) specify the parameters of how the instruction is to be to be used.
Next comes Lc – Length of Command, which specifies the length of the command data to follow,
Datacomes next, this is instruction data of the length specified in Lc.
Finally an optional Le – Length of expected response can be added to specify how long the response from the card should be.
Crafting APDUs
So let’s encode our own APDU to send to a card, for this example we’ll create the APDU to tell the card to select the Master File (MF) – akin to moving to the root directory on a *nix OS.
For this we’ll want a copy of ETSI TS 102 221 – the catchily named “Smart cards; UICC-Terminal interface; Physical and logical characteristics” which will guide in the specifics of how to format the command, because all the commands are encoded in hexadecimal format.
So here’s the coding for a SELECT command from section 11.1.1.1 “SELECT“,
For the CLA byte in our example we’ll indicate in our header that we’re using ISO 7816-4 encoding, with nothing fancy, which is denoted by the byte A0.
For the next but we’ve got INS (Instruction) which needs to be set to the hex value for SELECT, which is represented by the hex value A4, so our second byte will have that as it’s value.
The next byte is P1, which specifies “Selection Control”, the table in the specification outlines all the possible options, but we’ll use 00 as our value, meaning we’ll “Select DF, EF or MF by file id”.
The next byte P2 specifies more selection options, we’ll use “First or only occurrence” which is represented by 00.
The Lc byte defines the length of the data (file id) we’re going to give in the subsequent bytes, we’ve got a two byte File ID so we’ll specify 2 (represented by 02).
Finally we have the Data field, where we specify the file ID we want to select, for the example we’ll select the Master File (MF) which has the file ID ‘3F00‘, so that’s the hex value we’ll use.
So let’s break this down;
Code
Meaning
Value
CLA
Class bytes – Coding options
A0 (ISO 7816-4 coding)
INS
Instruction (Command) to be called
A4 (SELECT)
P1
Parameter 1 – Selection Control (Limit search options)
00 (Select by File ID)
P2
Parameter 1 – More selection options
00 (First occurrence)
Lc
Length of Data
02 (2 bytes of data to come)
Data
File ID of the file to Select
3F00 (File ID of master file)
So that’s our APDU encoded, it’s final value will be A0 A4 00 00 02 3F00
So there we have it, a valid APDU to select the Master File.
In the next post we’ll put all this theory into practice and start interacting with a real life SIM cards using PySIM, and take a look at the APDUs with Wireshark.
Want more telecom goodness?
I have a good old fashioned RSS feed you can subscribe to.