If you’ve ever worked in roaming, you’ll probably have had the misfortune of dealing with Transferred Account Procedures aka TAP files.
It’s used for billing a 2G GSM call right up to 5G data usage, if you use a service while roaming, somewhere in the world there’s a TAP file with your usage in it.
A brief history of TAP
TAP was originally specified by the GSMA in 1991 as a standard CDR interchange format between operators, for use in roaming scenarios.
Notice I said GSMA – Not 3GPP – This means there’s no 3GPP TS docs for this, it’s defined by the industry lobby group’s members, rather than the standards body.
So what does this actually mean? Well, if you’re MNO A and a customer from MNO B roams into your network, all the calls, SMS and data consumed by the roaming subscriber from MNO B will need to be billed to MNO B, by you, MNO A.
If a network operator wants to get paid for traffic used on their network by roaming subscribers, they’d better send out a TAP file to the roamer’s home network.
TAP is the file format generated my MNO A and sent to MNO B, containing all the usage charges that subscribers from MNO B have racked up while roaming into your network.
These are broken down into “Transactions” (CDRs), for events like making a call, connecting a PDN session and consuming data, or sending a text.
In the beginning of time, GSM provided only voice calling service. This meant that the only services a subscriber could consume while roaming was just making/receiving voice calls which were billed at the end of each month. – This meant billing was equally simple, every so often the visisted network would send the TAP files for the voice calls made by subscribers visited other networks, to the home networks, which would markup those charges, and add them onto the monthly invoice for each subscriber who was roaming.
But of course today, calling accounts for a tiny amount of usage on the network, but this happened gradually while passing through the introduction of SMS, CAMEL services, prepaid services, mobile data, etc.
For all these services that could be offered, the TAP format had to evolve to handle each of these scenarios.
As we move towards a flat IP architecture, where voice calls and SMS sent while roaming are just data, TAP files for 4G and 5G networks only need to show data transactions, so the call objects, CAMEL parameters and SMS objects are all falling by the wayside.
What’s inside a TAP File
TAP uses the most beloved of formats – ASN1 to encode the data. This means it is strictly formatted and rigidly specified.
Each file contains a Sequence Number which is a monotonically increasing number, which allows the receiver to know if any files have been missed between the file that’s being currently parsed, an the previous file.
They also have a recipient and sender TADIG code, which is a code allocated by GSMA that uniquely identifies the sender and the recipient of the file.
The TAP records exist in one of two common format, Notification Records and transferBatch records.
These files are exchanged between operators, in practice this means “Dumped on an FTP server as agreed between the two”.
TAP Notification Records
Notifications are the simplest of TAP records and are used when there aren’t any CDRs for roaming events during the time period the TAP file covers.
These are essentially blank TAP files generated by the visited network to let the home network know it’s still there, but there are no roaming subs consuming services in that period.
Notification files are really simple, let’s take a look as one shown as JSON:
{
"notification": {
"fileAvailableTimeStamp": {
"localTimeStamp": "2023/03/21 15:44:18",
"utcTimeOffset": "-0900"
},
"fileCreationTimeStamp": {
"localTimeStamp": "2023/03/21 15:44:18",
"utcTimeOffset": "-0900"
},
"fileSequenceNumber": "00003",
"recipient": "RECV",
"releaseVersionNumber": 12,
"sender": "SEND",
"specificationVersionNumber": 3,
"transferCutOffTimeStamp": {
"localTimeStamp": "2023/03/21 15:44:18",
"utcTimeOffset": "-0900"
}
}
}
Pretty simple!
TAP transferBatch Records
When we have services to bill and records to charge, that’s when instead we generate a transferBatch record.
It looks something like this:
There’s a lot going on in here, so let’s break it down section by section.
accountingInfo
The accountingInfo section specifies the currency, exchange rate parameters.
Keep in mind a TAP record generated by an operator in the US, would use USD, while the receiver of the file may be a European MNO dealing in EUR.
This gets even more complicated if you’re dealing with more obscure currencies where an intermediary currency is used, that’s where we bring in SDRs (“Special Drawing Right”) that map to the dollar value to be charged, kinda – the roaming agreement defines how many SDRs are in a dollar, in the example below we’re not using any, but you do see it.
When it comes to numbers and decimal places, TAP doesn’t exactly make it easy.
Significant Digits are defined by counting the first number before the decimal point and all the numbers to the right of the decimal point, so for example the number 1.234 would be 4 significant digits (1 digit before the decimal point and 3 digits after it).
Decimal Places are not actually supported for the Value fields in the TAP file. This is tricky because especially today when roaming tariffs are quite low, these values can be quite small, and we need to represent them as an integer number. TAP defines decimal places as the number of digits after the decimal place.
When it comes to the maximum number of decimal places, this actually impacts the maximum number we can store in the field – as ASN1 strictly enforce what we put in it.
"accountingInfo": {
"currencyConversionInfo": [
{
"exchangeRate": 1,
"exchangeRateCode": 1,
"numberOfDecimalPlaces": 0
}
],
"localCurrency": "USD",
"tapCurrency": "USD",
"tapDecimalPlaces": 5
},auditControlInfo
The auditControlInfo section contains the number of CDRs (callEventDetailsCount) contained in the TAP file, the timestamp of the first and last CDR in the file, the total charge and any tax charged.
All of the currency information was provided in the accountingInfo so this is just giving us our totals.
"auditControlInfo": {
"callEventDetailsCount": 6,
"earliestCallTimeStamp": {
"localTimeStamp": "2023/01/24 18:53:35",
"utcTimeOffset": "-0900"
},
"latestCallTimeStamp": {
"localTimeStamp": "2023/02/14 20:46:11",
"utcTimeOffset": "-0900"
},
"totalCharge": 17920,
"totalDiscountValue": 0,
"totalTaxValue": 0
},A CDR has 30 days from the time it was generated / service consumed by the roamer, to be baked into a TAP file. After this we can no longer charge for it, so it’s important that the earliestCallTimeStamp is not more than 30 days before the fileCreationTimeStamp seen in batchControlInfo.
batchControlInfo
The batchControlInfo section specifies the time the TAP file became available for transfer, the time the file was created (usually the same), the sequence number and the sender / recipient TADIG codes.
As mentioned earlier, we track sequence number so the receiver can know if a TAP file has been missed; for example if you’ve got TAP file 1 and TAP file 3 comes in, you can determine you’ve missed TAP file 2.
"batchControlInfo": {
"fileAvailableTimeStamp": {
"localTimeStamp": "2023/03/22 20:14:42",
"utcTimeOffset": "-0900"
},
"fileCreationTimeStamp": {
"localTimeStamp": "2023/03/22 20:14:42",
"utcTimeOffset": "-0900"
},
"fileSequenceNumber": "00001",
"recipient": "RECV",
"releaseVersionNumber": 12,
"sender": "SEND",
"specificationVersionNumber": 3,
"transferCutOffTimeStamp": {
"localTimeStamp": "2023/03/22 20:14:42",
"utcTimeOffset": "-0900"
}
},callEventDetails / gprsCall
Now we’re getting to the meat & potatoes of our TAP record, the CDRs themselves.
In LTE networks these are just records of data consumption, so let’s take a look inside the gprsCall records under callEventDetails:
In the gprsBasicCallInformation we’ve got as the name suggests the basic info about the data usage event. The time when the session started, the charging ID, the IMSI and the MSISDN of the subscriber to charge, along with their IP and the APN used.
"callEventDetails" { "gprsCall": {
"gprsBasicCallInformation": {
"callEventStartTimeStamp": {
"localTimeStamp": "2023/01/24 18:53:56",
"utcTimeOffsetCode": 1
},
"chargingId": 18290321,
"gprsChargeableSubscriber": {
"chargeableSubscriber": {
"simChargeableSubscriber": {
"imsi": "001010000000002",
"msisdn": "142151232"
}
},
"pdpAddress": "f21a:b5b9:a0b1:e568:a531:91f5:dc5e:899e"
},
"gprsDestination": {
"accessPointNameNI": "internet",
"accessPointNameOI": "mnc001.mcc001.gprs"
},
"totalCallEventDuration": 84
},
Next up we have the gprsLocationInformation – rates and tariffs may be set based on the location of the subscriber, so we need to identify the area the sub was using the services to select correct tariff / rate for traffic in this destination.
The recEntity is the index number of the SGW / PGW used for the transaction (more on that later).
"callEventDetails" { "gprsCall": {
"gprsLocationInformation": {
"geographicalLocation": {
"servingBid": "0002",
"servingLocationDescription": "Nick Lab AU"
},
"gprsNetworkLocation": {
"cellId": 00002,
"locationArea": 100,
"recEntity": [
4,
2
]
}
},
Next we have the gprsServiceUsed which, again as the name suggests, details the services used and the charge.
chargeDetailList contains the charged data (Made up of dataVolumeIncoming + dataVolumeOutgoing) and the cost.
The chargeableUnits indicates the actual data consumed, however most roaming agreements will standardise on some level of rounding, for example rounding up to the nearest Kilobyte (1024 bytes), so while a sub may consume 1025 bytes of data, they’d be billed for 2045 bytes of data. The data consumed is indicated in the chargeableUnits which indicates how much data was actually consumed, before any rounding policies where applied, while the amount that is actually charged (When taking into account rounding policies) is indicated inside Charged Units.
In the example below data usage is rounded up to the nearest 1024 bytes, 134390 bytes rounds up to the nearest 1024 gives you 135168 bytes.
As this is data we’re talking bytes, but not all bytes are created equal!
VoLTE traffic, using a QCI1 bearer is more valuable than QCI 9 cat videos, and TAP records take this into account in the Call Type Groups, each of which has a different price – Call Type Level 1 indicates the type of traffic, for S8 Home Routed LTE Traffic this is 10 (HGGSN/HP-GW), while Call Type Level 2 indicates the type of traffic as mapped to QCI values:
20 Unspecified/default LTE QCIs 21 LTE QCI 1 Conversational 22 LTE QCI 2 Conversational 23 LTE QCI 3 Conversational 24 LTE QCI 4 Streaming 25 LTE QCI 5 Interactive (specialised for signalling) 26 LTE QCI 6 Interactive 27 LTE QCI 7 Interactive 28 LTE QCI 8 Interactive 29 LTE QCI 9 Background
So Call Type Level 2 set to 20 indicates that this is “20 Unspecified/default LTE QCIs”, and Call Type Level 3 can be set to any value based on a defined inter-operator tariff.
Here’s the example as JSON:
"callEventDetails" { "gprsCall": {
"gprsServiceUsed": {
"chargeInformationList": [
{
"callTypeGroup": {
"callTypeLevel1": 10,
"callTypeLevel2": 20,
"callTypeLevel3": 0
},
"chargeDetailList": [
{
"charge": 1320,
"chargeType": "3030",
"chargeableUnits": 134390,
"chargedUnits": 135168
}
],
"chargedItem": "58",
"exchangeRateCode": 1
}
],
"dataVolumeIncoming": 65536,
"dataVolumeOutgoing": 69632
},
"operatorSpecInformation": [
"RAT:6",
]
}
}
These gprsCall objects are added for each CDR in the TAP file (as indicated by callEventDetailsCount).
networkInfo
Lastly we have the networkInfo which contains the recEntityInfo which gives each network element used in the transaction an ID that can be referenced.
"networkInfo": {
"recEntityInfo": [
{
"recEntityCode": 1,
"recEntityId": "1.2.3.4",
"recEntityType": 7
},
{
"recEntityCode": 2,
"recEntityId": "1.2.3.5",
"recEntityType": 8
},
],
"utcTimeOffsetInfo": [
{
"utcTimeOffset": "-0900",
"utcTimeOffsetCode": 1
}
]
}
recEntityType 7 means a PGW and contains the IP of the PGW in the Home PLMN, while recEntityType 8 means SGW and is the SGW in the Visited PLMN.
So this means if we reference recEntityCode 2 in a gprsCall, that we’re referring to an SGW at 1.2.3.5.
Lastly also got the utcTimeOffsetInfo to indicate the timezones used and assign a unique code to it.
Using the Records
We as humans? These records aren’t meant for us.
They’re designed to be generated by the Visited PLMN and sent to to the home PLMN, which ingests it and pays the amount specified in the time agreed.
Generally this is an FTP server that the TAP records get dumped into, and an automated bank transfer job based on the totals for the TAP records.
Testing of the TAP records is called “TADIG Testing” and it’s something we’ll go into another day, but in essence it’s validating that the output and contents of the files meet what both operators think is the contract pricing and specifications.
So that’s it! That’s what’s in a TAP record, what it does and how we use it!
GSMA are introducing BCE – Billing & Charging Evolution, a new standard, designed to last for the next 30+ years like TAP has. It’s still in its early days, but that’s the direction the GSMA has indicated it would like to go.
All Data Sessions in TAP fill appear under GPRS Teleservice. How does TAP indicate the differences between data sessions in 2G, 3G, EDGE, 4G, LTE, or VoLTE networks?
Any operator charging separate rates for the different data technologies?