Tag Archives: GTP

Best Practices for SGW & PGW Deployment Architectures for Roaming

IMS / VoLTE, Mobile Networks, SDM, , , , , , , , ,

The S8 Home Routing approach for LTE Roaming works really well, as more and more operators are switching off their legacy circuit switched 2G/3G networks and shifting to LTE & VoLTE for roaming, we’re seeing more an more S8-HR deployments.

When LTE was being standardised in 2008, Local Breakout (LBO) and S8 Home Routing were both considered options for how roaming may look. Fast forward to today, and S8 Home routing is the only way roaming is done for modern deployments.

In light of this, there are some “best practices” in an “all S8 Home Routed” world, we’ve developed, that I thought I’d share.

The Basics

When roaming, the SGW in the Visited Network, sends user traffic back to the PGW in the Home Network.

This means Online/Offline charging, IMS, PCRF, etc, is all done in the Home PLMN. As long as data packets can get from the SGW in the Visited PLMN to the PGW in the Home PLMN, and authentication flows from the Visited MME to the HSS in the Home PLMN, you’re golden.

The Constraints

Of course real networks don’t look as simple as this, in reality a roaming scenario for a visited network has a lot more nodes, which need to be

Building Distributed Packet Core & IMS

Virtualization (VNF / CNF) has led operators away from “big iron” hardware for Packet Core & IMS nodes, towards software based solutions, which in turn offer a lot more flexibility.

Best practice for design of User Plane is to keep the the latency down, by bringing the user plane closer to the user (the idea of “Edge” UPFs in 5GC is a great example of this), and the move away from “big iron” in central locations for SGW and PGW nodes has been the trend for the past decade.

So to achieve these goals in the networks we build, we geographically distribute the core network.

This means we’ve got quite a few S-GW, P-GW, MME & HSS instances across the network.
There’s some real advantages to this approach:

From a redundancy perspective this allows us to “spread the load” and build far more resilient networks. A network with 20 smaller HSS instances spread around the country, is far more resilient than 2 massive ones, regardless of how many power feeds or redundant disks it may have.

This allows us to be more resource efficient. MNOs have always provisioned excess capacity to cater for the loss of a node. If we have 2 MMEs serving a country, then each node has to have at least 50% capacity free, so if one MME were to fail, the other MME could handle the additional load it from it’s dead friend. This is costly for resources. Having 20 MMEs means each MME has to have 5% capacity free, to handle the loss of one MME in the pool.

It also forces our infrastructure teams to manage infrastructure “as cattle” rather than pets. These boxes don’t get names or lovingly crafted, they’re automatically spun up and destroyed without thinking about it.

For security, we only use internal IP addresses for the nodes in our packet core, this provides another layer of protection for the “crown jewels” of our network, so no one messing with BGP filtering can accidentally open the flood gates to our core, as one US operator learned leaving a GGSN open to the world leading to the private information for 100 million customers being leaked.

What this all adds to, is of course, the end user experience.
For the end subscriber / customer, they get a better experience thanks to the reduced latency the connection provides, better uptime and faster call setup / SMS delivery, and less cost to deliver services.

I love this approach and could prothletise about it all day, but in a roaming context this presents some challenges.

The distributed networks we build are in a constant state of flux, new capacity is being provisioned in some areas, nodes things decommissioned in others, and our our core nodes are only reachable on internal IPs, so wouldn’t be reachable by roaming networks.

Our Distributed-Core Roaming Solution

To resolve this we’ve taken a novel approach, we’ve deployed a pair of S-GWs we call the “Roaming SGWs”, and a pair of P-GWs we call the “Roaming PGWs”, these do have public IPs, and are dedicated for use only by roaming traffic.

We really like this approach for a few reasons:

It allows us to be really flexible do what we want inside the network, without impacting roaming customers or operators who use our network for roaming. All the benefits I described from the distributed architectures can still be realised.

From a security standpoint, only these SGW/PGW pairs have public IPs, all the others are on internal IPs. This good for security – Our core network is the ‘crown jewels’ of the network and we only expose an edge to other providers. Even though IPX networks are supposed to be secure, one of the largest IPX providers had their systems breached for 5 years before it was detected, so being almost as distrustful of IPX traffic as Internet traffic is a good thing.
This allows us to put these PGWs / SGWs at the “edge” of our network, and keep all our MMEs, as well as our on-net PGW and SGWs, on internal IPs, safe and secure inside our network.

For charging on the SGWs, we only need to worry about collecting CDRs from one set of SGWs (to go into the TAP files we use to bill the other operators), rather than running around hoovering up SGW CDRs from large numbers of Serving Gateways, which may get blown away and replaced without warning.

Of course, there is a latency angle to this, for international roaming, the traffic has to cross the sea / international borders to get to us. By putting it at the edge we’re seeing increased MOS on our calls, as the traffic is as close to the edge of the network as can be.

Caveat: Increased S11 Latency on Core Network sites over Satellite

This is probably not relevant to most operators, but some of our core network sites are fed only by satellite, and the move to this architecture shifted something: Rather than having latency on the S8 interface from the SGW to the PGW due to the satellite hop, we’ve got latency between the MME and the SGW due to the satellite hop.

It just shifts where in the chain the latency lies, but it did lead to us having to boost some timers in the MME and out of sequence deliver detection, on what had always been an internal interface previously.

Evolution to 5G Standalone Roaming

This approach aligns to the Home Routed options for 5G-SA roaming; UPF chaining means that the roaming traffic can still be routed, as seems to be the way the industry is going.

SA roaming is in its infancy, without widely deployed SA networks, we’re not going to see common roaming using SA for a good long while, but I’ll be curious to see if this approach becomes the de facto standard going forward.

Where to from here?

We’re pretty happy with this approach in the networks we’ve been building.

So far it’s made IREG testing easier as we’ve got two fixed points the IPX needs to hit (The DRAs and the SGWs) rather than a wide range of networks.

Operators with a vast number of APNs they need to drop into different VRFs may have to do some traffic engineering here – Our operations are generally pretty flat, but I can see where this may present some challenges for established operators shifting their traffic.

I’d be keen to hear if other operators are taking this approach and if they’ve run into any issues, or any issues others can see in this, feel free to drop a comment below.

What’s the maximum speed for LTE and 5G?

5G SA, EPC, LTE, Mobile Networks, SDM, , , ,

Even before 5G was released, the arms race to claim the “fastest” speeds on LTE, NSA and SA networks has continued, with pretty much every operator claiming a “first” or “fastest”.

I myself have the fastest 5G network available* but I thought I’d look at how big the values are we can put in for speed, these are the Maximum Bitrate Values (like AMBR) we can set on an APN/DNN, or on a Charging Rule.

*Measurement is of the fastest 5G network in an eastward facing office, operated by a person named Nick, in a town in Australia. Other networks operated by people other than those named Nick in eastward facing office outside of Australia were not compared.

The answer for Release 8 LTE is 4294967294 bytes per second, aka 4295 Mbps 4.295 Gbps.

Not bad, but why this number?

The Max-Requested-Bandwidth-DL AVP tells the PGW the max throughput allowed in bits per second. It’s a Unsigned32 so max value is 4294967294, hence the value.

But come release 15 some bright spark thought we may in the not to distant future break this barrier, so how do we go above this?

The answer was to bolt on another AVP – the “Extended-Max-Requested-BW-DL” AVP ( 554 ) was introduced, you might think that means the max speed now becomes 2x 4.295 Gbps but that’s not quite right – The units was shifted.

This AVP isn’t measuring bits per second it’s measuring kilobits per second.

So the standard Max-Requested-Bandwidth-DL AVP gives us 4.3 Gbps, while the Extended-Max-Requested-Bandwidth gives us a 4,295 Gbps.

We add the Extended-Max-Requested-Bandwidth AVP (4295 Gbps) onto the Max-Requested Bandwidth AVP (4.3 Gbps) giving us a total of 4,4299.3 Gbps.

So the short answer:

Pre release 15: 4.3 Gbps

Post release 15: 4,4299.3 Gbps

GTP Extension Headers (PDU session user plane protocol) in 5GC

5G SA, Mobile Networks, ,

The GPRS Tunneling Protocol is one of the last common bits of signaling seen in 5G networks, having existed since GPRS was standardized in 1998, and 23 years later, it’s still in use on the user plane.

But networks evolve, and 5G Networks required some extensions to GTP to support these on the N9 and N3 reference points. (UPF to UPF and UPF to gNodeB / Access Network).

3GPP TS 38.415 outlines the PDU session user plane protocol used in 5GC.

The Need for GTP Header Extensions

As increasingly complex QoS capabilities are introduced into 5GC, there is a need to signal certain information on a per-packet basis.

In previous generations of mobile network, traffic could be differentiated with different Tunnel Endpoint Identifiers (TEIDs) but not on a per-packet basis,

The expansion of QoS in 5GC means the UPF of gNodeB may need to set the QoS Flow Identifier per-packet, include delay measurements or signal that Reflective QoS is being used per packet, for this, you need to extend GTP.

Fortunately GTP has support for Extension Headers and this has been leveraged to add the PDU Session Container in the Extension Header of a GTP packet.

In here you can set on a per packet basis:

  • QoS Flow Identifier (QFI) – Used to identify the QoS flow to be used (Pretty self explanatory)
  • Reflective QoS Indicator (RQI) – To indicate reflective QoS is supported for the encapsulated packet
  • Paging Policy Presence (PPP) – To indicate support for Paging Policy Indicator (PPI)
  • Paging Policy Indicator (PPI) – Sets parameters of paging policy differentiation to be applied
  • QoS Monitoring Packet – Indicates packet is used for QoS Monitoring and DL & UL Timestamps to come
  • UL/DL Sending Time Stamps – 64 bit timestamp generated at the time the UPF or UE encodes the packet
  • UL/DL Received Time Stamps – 64 bit timestamp generated at the time the UPF or UE received the packet
  • UL/DL Delay Indicators – Indicates Delay Results to come
  • UL/DL Delay Results – Delay measurement results
  • Sequence Number Presence – Indicates if QFI sequence number to come
  • UL/DL QFI Sequence Number – Sequence number as assigned by the UPF or gNodeB

LTE EPC: Serving Gateway (S-GW) Basic Function

EPC, LTE, Mobile Networks, , , , ,

As our subscribers are mobile, moving between base stations / cells, the destination of the incoming GTP-U packets needs to be updated every time the subscriber moves from one cell to another.

If you’re not familiar with GTP take a read of this primer.

As we covered in the last post, the Packet Gateway (P-GW) handles decapsulating and encapsulating this traffic into GTP from external networks, and vise-versa. The Packet Gateway sends the traffic onto a Serving Gateway, that forwards the GTP-U traffic onto the eNodeB serving the user.

So why not just route the traffic from the Packet Gateway directly to the eNodeB?

As our users are inherently mobile, the signalling load to keep updating the destination of the incoming GTP-U traffic to the correct eNB, would put an immense load on the P-GW. So an intermediary gateway – the Serving Gateway (S-GW), is introduced.

The S-GW handles the mobility between cells, and takes the load of the P-GW. The P-GW just hands the traffic to a S-GW and let’s the S-GW handle the mobility.

It’s worth keeping in mind that most LTE connections are not “always on”. Subscribers (UEs) go into “Idle Mode”, in which the data connection and the radio connection is essentially paused, and able to be bought back at a moments notice (this allows us to get better battery life on the UE and better frequency utilisation).

When a user enters Idle Mode, an incoming packet needs to be buffered until the Subscriber/UE can get paged and come back online. Again this function is handled by the S-GW; buffering packets until the UE comes available then forwarding them on.

CUPS – Control and User Plane Separation in LTE & NR with PFCP (Sx & N4)

5G SA, EPC, LTE, Mobile Networks, , , , , ,

3GPP release 14 introduced the concept of CUPS – Control & User Plane Separation, and the Sx interface, this allows the control plane (GTP-C) functionality and the user plane (GTP-U) functionality to be separated, and run in a distributed fashion, allowing the node a user’s GTP-U traffic flows through to be in a different location to where the Control / Signalling traffic (GTP-C) flows.

In practice that means for an LTE EPC this means we split our P-GW and S-GW into a minimum of two network elements each.

A P-GW is split and becomes a P-GW-C that handles the P-GW Control Plane traffic (GTPv2-C) and a P-GW-U speaking GTP-U for our User Plane traffic. But the split doesn’t need to stop there, one P-GW-C could control multiple P-GW-Us, routing the user plane traffic. Sames goes for S-GW being split into S-GW-C and S-GW-U,

This would mean we could have a P-GW-U located closer to a eNB / User to reduce latency, by allowing GTP-U traffic to break out on a node closer to the user.

It also means we can scale better too, if we need to handle more data traffic, but not necessarily more control plane traffic, we can just add more P-GW-U nodes to handle this.

To manage this a new protocol was defined – PFCP – the Packet Forwarding Control Protocol. For LTE this is refereed to as the Sx reference point, it’ also reused in 5G-SA as the N4 reference point.

When a GTP-C “Create Session Request” comes into a P-GW for example from an S-GW, a PFCP “Session Establishment Request” is sent by the P-GW-C to the P-GW-U with much the same information that was in the GTP-C request, to setup the session.

So why split the Control and User Plane traffic if you’re going to just relay the GTP-C traffic in a different format?

That was my first question – the answer is that keeping the GTP-C interface ensures backward compatibility with older MMEs, PCRFs, charging systems, and allows a staged roll out and bolting on extra capacity.

GTP-C disappears entirely in 5G Standalone architecture and is replaced with the N4 interface, which uses PFCP for the Control Plan and GTP-U again.

Here’s a capture from Open5Gs core showing GTPv2C and PFCP in play.

CUPS-PFCP-and-GTPv2cDownload

Getting TEID up with GTP Tunnels

EPC, GSM, LTE, Mobile Networks, Uncategorized, , , , , ,

If you’re using an GSM / GPRS, UMTS, LTE or NR network, there’s a good chance all your data to and from the terminal is encapsulated in GTP.

GTP encapsulates user’s data into a GTP PDU / packet that can be redirected easily. This means as users of the network roam around from one part of the network to another, the destination IP of the GTP tunnel just needs to be updated, but the user’s IP address doesn’t change for the duration of their session as the user’s data is in the GTP payload.

One thing that’s a bit confusing is the TEID – Tunnel Endpoint Identifier.

Each tunnel has a sender TEID and transmitter TEID pair, as setup in the Create Session Request / Create Session Response, but in our GTP packet we only see one TEID.

There’s not much to a GTP-U header; at 8 bytes in all it’s pretty lightweight. Flags, message type and length are all pretty self explanatory. There’s an optional sequence number, the TEID value and the payload itself.

So the TEID identifies the tunnel, but it’s worth keeping in mind that the TEID only identifies a data flow from one Network Element to another, for example eNB to S-GW would have one TEID, while S-GW to P-GW would have another TEID.

Each tunnel has two TEIDs, a sending TEID and a receiving TEID. For some reason (Minimize overhead on backhaul maybe?) only the sender TEID is included in the GTP header;

This means a packet that’s coming from a mobile / UE will have one TEID, while a packet that’s going to the same mobile / UE will have a different TEID.

Mapping out TIEDs is typically done by looking at the Create Session Request / Responses, the Create Session Request will have one TIED, while the Create Session Response will have a different TIED, thus giving you your TIED pair.

GSM with Osmocom: GPRS & Packet Data

GSM, Mobile Networks, RF, , , ,

So far we’ve focused on building a plain “2G” (voice and SMS only) network, which was all consumers expected twenty years ago.

As the number of users accessing the internet through DSL, Dial Up & ISDN grew, the idea of getting this data “on the go” became more appealing. TCP/IP was becoming the dominant standard for networking, the first 802.11 WiFi spec had recently been published and demand for mobile data was growing.

There’s a catch however – TCP/IP was never designed to be mobile.

An IP address exists in a single location.

(Disclaimer: While you can “move” a subnet by advertising itself out in a different location via BGP peering relationships with other operators, it’s cumbersome, can only be done per /24 or larger, and most importantly it’s painfully slow. IPv6 has MIPv6 which attempts to fix some of these points, but that’s outside of this scope.)

GPRS addressed the mobility issue by having a single fixed point the IP Address is assigned to (the Gateway GPRS Support Node), which encapsulates IP traffic to/from a mobile user into GTP Packet (GPRS Tunnelling Protocol), like GRE or any of the other common routing encapsulation protocols, allowing the traffic to be rerouted to different destinations as the users move from being served by one BTS to another BTS.

I’ve written about GTP here if you’d like to learn more.

So now we’ve got a method of encapsulating our data we’ve got to work out how to get that data out over the air.

BTS Time Slots

Way back when we were first setting up our BSC and adding our BTS(s) you will have configured timeslots for each BTS configured on your BSC.

Chances are if you’ve been following along with this tutorial, that you configured the first time slot (timeslot 0) as a CCCH+SDCCH4, meaning Common Control Channel and 4 standalone dedicated control channels, and all the subsequent timeslots (timeslot 1 – 7) as Traffic Channels (full rate) – TCH/F.

This works well if we’re only carrying voice, but to carry data we need timeslots to put the data traffic on.

For this we’ll re assign a timeslot we were using on our BSC as a voice traffic channel (TCH/F) as a PDCH – a Packet Data Channel.

This means that on the BSC your timeslot config will look something like this:

   timeslot 6
    phys_chan_config PDCH
    hopping enabled 0
   timeslot 7
    phys_chan_config PDCH
    hopping enabled 0

In the above example I’ve assigned two timeslots for Packet Data Channels,

The more timeslots you allocate for data, the more bandwidth available, but the fewer voice resources available.

(Most GSM networks today have few data timeslots as more recent RATs like 3G/4G are taking the data traffic, and GSM is used primarily for voice and low bandwidth M2M communications)

GPRS and EDGE

GPRS comes in two flavors, GPRS and EDGE.

GPRS (General Packet Radio Services) was the first of the two, standardised in R97, and allowed users to reach a downlink speeds of up to 171Kbps using GMSK on the air interface to encode the data.

Users quickly expected more speed, so EDGE (Enhanced Data rates for Global Evolution) was standardised, from a core perspective it was the same, but from a BTS / Air interface perspective it relied on 8PSK instead of GMSK allowed users to reach a blistering 384Kbps on the downlink.

These speeds are the theoretical maximums.

As the difference between GPRS and EDGE is encoding on the air interface, from a core perspective it’s treated the same way, however as our BTS gets all it’s brains from the BSC, we’ll need to specify if the BTS should use EDGE or GPRS it in the BSC’s BTS config.

BSC Config

On the BSC for each BTS we want to enable for packet data, we’ll need to define the parameters.

There’s two other values we’ll introduce when setting this up,

The first is NSEI – the Network Service Entity Identifier, which is the identifier of the BTS’s Packet Control Unit, like the cell identity.

The second value we’ll touch on is the BVCI – the BSSGP Virtual Connections Identifier, which is used for addressing between the BTS PCU and the SGSN.

bts 0
  gprs mode egprs
  gprs 11bit_rach_support_for_egprs 0
  gprs routing area 0
  gprs network-control-order nc0
  gprs cell bvci 2
  gprs cell timer blocking-timer 3
  gprs cell timer blocking-retries 3
  gprs cell timer unblocking-retries 3
  gprs cell timer reset-timer 3
  gprs cell timer reset-retries 3
  gprs cell timer suspend-timer 10
  gprs cell timer suspend-retries 3
  gprs cell timer resume-timer 10
  gprs cell timer resume-retries 3
  gprs cell timer capability-update-timer 10
  gprs cell timer capability-update-retries 3
  gprs nsei 101
  gprs ns timer tns-block 3
  gprs ns timer tns-block-retries 3
  gprs ns timer tns-reset 3
  gprs ns timer tns-reset-retries 3
  gprs ns timer tns-test 30
  gprs ns timer tns-alive 3
  gprs ns timer tns-alive-retries 10
  gprs nsvc 0 nsvci 101
  gprs nsvc 0 local udp port 23001
  gprs nsvc 0 remote udp port 23000
  gprs nsvc 0 remote ip 10.0.1.201

The OsmoBSC docs cover each of these values, they’re essentially defaults.

There are quite a few changes required on the BSC for each BTS we’re setting this up for. Instead of giving you info on what fields to change, here’s the diffs.

In the next post we’ll cover the GGSN and the SGSN and then getting a device on “the net”.

GTPv2 – F-TEID Interface Types

EPC, LTE, Mobile Networks, Python, RFCs & Standards, , ,

I’ve been working on a ePDG for VoWiFi access to my IMS core.

This has led to a bit of a deep dive into GTP (easy enough) and GTPv2 (Bit harder).

The Fully Qualified Tunnel Endpoint Identifier includes an information element for the Interface Type, identified by a two digit number.

Here we see S2b is 32

In the end I found the answer in 3GPP TS 29.274, but thought I’d share it here.

0S1-U eNodeB GTP-U interface
1S1-U SGW GTP-U interface
2S12 RNC GTP-U interface
3S12 SGW GTP-U interface
4S5/S8 SGW GTP-U interface
5S5/S8 PGW GTP-U interface
6S5/S8 SGW GTP-C interface
7S5/S8 PGW GTP-C interface
8S5/S8 SGW PMIPv6 interface (the 32 bit GRE key is encoded in 32 bit TEID field and since alternate CoA is
not used the control plane and user plane addresses are the same for PMIPv6)
9S5/S8 PGW PMIPv6 interface (the 32 bit GRE key is encoded in 32 bit TEID field and the control plane and
user plane addresses are the same for PMIPv6)
10S11 MME GTP-C interface
11S11/S4 SGW GTP-C interface
12S10 MME GTP-C interface
13S3 MME GTP-C interface
14S3 SGSN GTP-C interface
15S4 SGSN GTP-U interface
16S4 SGW GTP-U interface
17S4 SGSN GTP-C interface
18S16 SGSN GTP-C interface
19eNodeB GTP-U interface for DL data forwarding
20eNodeB GTP-U interface for UL data forwarding
21RNC GTP-U interface for data forwarding
22SGSN GTP-U interface for data forwarding
23SGW GTP-U interface for DL data forwarding
24Sm MBMS GW GTP-C interface
25Sn MBMS GW GTP-C interface
26Sm MME GTP-C interface
27Sn SGSN GTP-C interface
28SGW GTP-U interface for UL data forwarding
29Sn SGSN GTP-U interface
30S2b ePDG GTP-C interface
31S2b-U ePDG GTP-U interface
32S2b PGW GTP-C interface
33S2b-U PGW GTP-U interface

I also found how this data is encoded on the wire is a bit strange,

In the example above the Interface Type is 7,

This is encoded in binary which give us 111.

This is then padded to 6 bits to give us 000111.

This is prefixed by two additional bits the first denotes if IPv4 address is present, the second bit is for if IPv6 address is present.

Bit 1Bit 2Bit 3-6
IPv4 Address Present IPv4 Address PresentInterface Type
11 000111

This is then encoded to hex to give us 87

Here’s my Python example;

interface_type = int(7)
interface_type = "{0:b}".format(interface_type).zfill(6)   #Produce binary bits
ipv4ipv6 = "10" #IPv4 only
interface_type = ipv4ipv6 + interface_type #concatenate the two
interface_type  = format(int(str(interface_type), 2),"x") #convert to hex

Why GTP for Mobile Networks?

EPC, LTE, Mobile Networks, RFCs & Standards, , , , ,

Let’s take a look at GTP, the workhorse of mobile user plane packet data.

This post covers all generations of mobile data (2.5 -> 5G), so I’m using generic terms.

GSM, UMTS, LTE & NR all have one protocol in common – GTP – The GPRS Tunneling Protocol.

So why do every generation of mobile data networks from GSM/GPRS in 2000, to 5G NR Standalone in 2020, rely on this one protocol for transporting user data?

So Why GTP?

GTP – the GPRS Tunnelling Protocol, is what encapsulates and tunnels IP packets from the internet / packet data network, to and from the User.

So why encapsulate the packets? What if the Base Station had access to the internet and routed the traffic to the users?

Let’s say we did that, we’d have to have large pools of IP addresses available at each Base Station and when a user connected they’d be assigned an IP Address and traffic for these users would be routed to the Base Station which would forward it onto the user.

This would work well until a user moves from one Base Station to another, when they’d have to get a new IP Address allocated.

TCP/IP was never designed to be mobile, an IP address only exists in a single location.

Breaking out traffic directly from a base station would have other issues, such as no easy way to enforce QoS or traffic policies, meter usage, etc.

How to fix IP’s lack of mobility? GTP.

GTP addressed the mobility issue by having a single fixed point the IP Address is assigned to (In GSM/GRPS/UMTS this is the Gateway GPRS Support Node, in LTE this is the P-GW and in 5G-SA this is the UPF), which encapsulates IP traffic to/from a mobile user into GTP Packet.

You can think of GTP like GRE or any of the other common encapsulation protocols, wrapping up the IP packets into a GTP packet which we can rerouted to different Base Stations as the users move from being served by one Base Station to another.

This easy redirecting / rerouting of user traffic is why GTP is used for NR (5G), LTE (4G), UMTS (3G) & GPRS (2.5G) architectures.

GTP Packets

When looking at a GTP packet of user data you’d be forgiven for thinking nothing much goes on,

Example GTP packet containing a DNS query

Like in most tunneling / encapsulation protocols we’ve got the original network / protocol stack of IPv4 and UDP, and a payload of a GTP packet.

The packet itself is pretty bare bones, there’s flags, denoting a few basics like version number, the message type (T-PDU), the length of the GTP packet and it’s payload (used for delineating the end of the payload), a sequence number an a Tunnel Endpoint Identifier (TEID).

In the payload, we can see the network / protocol stack and application layer of the contents of the GTP packet.

From a mobility standpoint, the beauty of GTP is that it takes IP packets and puts them into a media stream of sorts, with out of band signalling, this means we can change the parameters of our GTP stream easily without touching the encapsulated IP Packet.

When a UE moves from one base station to another, all that has to happen is the destination the GTP packets are sent to is changed from the old base station to the new base station. This is signalled using GTP-C in GPRS/UMTS, GTPv2-C in LTE and HTTP in 5G-SA.

Traffic to and from the UE would look the same as the screenshot above, the only difference would be the first IPv4 address would be different, but the IPv4 address in the GTP tunnel would be the same.

IMTx: NET02x (4G Network Essentials) – Management of Data Flows – 7. NAS and Global View of Protocol Stack

EPC, EUTRAN, LTE, Mobile Networks, Notes, RF, , , , , ,

These are my lecture notes from IMT’s NET02x (4G Network Essentials) course, I thought I’d post them here as they may be useful to someone. You can find my complete notes here.

The LTE architecture compartmentalises the roles in the mobile network.

For example the eNB concentrates on radio connection management, while the MME focuses on security and mobility.

Non Access Stratum (NAS) messages are exchanged between the terminal and the MME.

Access Stratum (AS) messages are exchanged over the air between the UE and the eNB. It contains all the radio related information.

The eNB must map the NAS messages from an MME to a LCID and RNTI and transmit them over the air, and vice-versa. The eNB forwards this data without ever analyzing it.

Transport of S1 messages is carried over SCTP which I’ve spoken about in the past.

The above image overlaps with the radio version we talked about earlier.

IMTx: NET02x (4G Network Essentials) – Management of Data Flows – 5/6. S1AP Connection

EPC, EUTRAN, LTE, Mobile Networks, Notes, RF, , , , , ,

These are my lecture notes from IMT’s NET02x (4G Network Essentials) course, I thought I’d post them here as they may be useful to someone. You can find my complete notes here.

Each MME can manage millions of UEs.

To handle this load the requirements of each subscriber for the MME must be as minimal and simple as possible so as to scale easily.

For each UE in the network a connection is setup between the UE and the MME.

This is done over the S1-AP’s Control Plane interface (sometimes calls S1-Control Plane or S1-CP) which carries control plane data to & from the UE via the eNB to the MME.

S1-CP is connection-oriented, meaning each UE has it’s own connection to the MME, so there are as many S1-CP connections to the MME as UE’s connected.

Each of these S1-CP connections is identified by a pair of unique connection IDs. The eNB keeps track of the connection IDs for each UE connected and hands this information off each time the UE moves to a different eNB.

The eNB keeps a lookup table between the RNTI of the UE and the LCID – the Logical Channel Identifier. This means that the eNB knows the sent and received ID of the S1-CP connection for each UE, and is able to translate that into the RNTI and LCID used to send the data over the air interface to the UE.

S1-CP Connect (Attach Procedure)

As we discussed in radio interfaces, when a UE connects to the network it is assigned an RNTI to identify it on the radio interface and allocate radio resources to it.

Once the RNTI is confirmed by both the eNB and the UE, a EMM Attach Request, which is put into an RRC Message called RRCConnectionSetupComplete.

The eNB must next choose a serving MME for this UE. It picks one based on it’s defined logic, and sends a S1-AP Intial UE Message (EMM Attach Request) to the MME along with the eNB’s connection identity assigned for this connection.

The MME stores the connection identity assigned by the eNB and chooses it’s own connection identity for it’s side, and sends back an S1AP Downlink NAS Transport response with both connection identities and the response for the attach request (This will be an EMM Authentication Request).

The eNB then stores the connection identity pair and the associated RNTI and LCID for the UE, and forwards the EMM Authentication Request to the RNTI of the UE via RRC.

The UE will pass the authentication challenge input parameters to the USIM which will generate a response. The UE will send the output of this response in a EMM Authentication Responseto the eNB, which will look at the RNTI and LCID received and consult the table to find the Connection Identifiers and IP of the serving MME for this UE.

S1AP Connect procedure

IMTx: NET02x (4G Network Essentials) – Management of Data Flows – 4. Transmitting Packets in a Tunnel

EPC, EUTRAN, LTE, Mobile Networks, RF, , , ,

These are my lecture notes from IMT’s NET02x (4G Network Essentials) course, I thought I’d post them here as they may be useful to someone. You can find my complete notes here.

Establishing a GTP Tunnel

When a new tunnel is setup between two nodes, GTP-C will be used to setup the tunnel and the both ends of the tunnel will allocate a their own locally unique TEID to the tunnel.

Let’s take a look at setting up a GTP tunnel between a S-GW and a P-GW, initiated by the S-GW.

The process will start with the S-GW sending the P-GW a GTP-C tunnel establishment request and include the TEID the S-GW has allocated for it’s end of the tunnel (using TEID 102 in this example), sent from the S-GW to the P-GW.

The P-GW will receive this packet. When it does it will allocate a new TEID for this tunnel for it’s side (In this case it’s 16538), store the sender’s address and received TEID, and link local TEID 16538 with S-GW/102.

An ACK is sent from the P-GW to the S-GW with both TEID values.

Finally the S-GW stores the senders’ address, the received TEID and the link 102-PGW address 16538.


Now the exchange is complete the S-GW and the P-GW each know the TEID of it’s local side of the tunnel, and the remote side of the tunnel.

TEID Management Tables

After GTP tunnels are setup a management table is populated defining the forward rules for that traffic.

For example a packet coming in on TEID 103 would, according to the table forward to TEID 102. TEID 102 sends traffic to the P-GW’s IP using remote TEID 16538.

The same rules for uplink are applied for downlink.

Each tunnel has pair of TEIDs a local TEID and a remote TEID.

Because it’s such a simple table it can be updated very easily and scales well.

Different QoS parameters can be assigned to each tunnel, called a data bearer.

IMTx: NET02x (4G Network Essentials) – Management of Data Flows – 3. Identifying & Managing Tunnels

EPC, EUTRAN, LTE, Mobile Networks, Notes, RF, , , ,

These are my lecture notes from IMT’s NET02x (4G Network Essentials) course, I thought I’d post them here as they may be useful to someone. You can find my complete notes here.

As we’ve talked about traffic to and from UEs is encapsulated in GTP-U tunnels, with the idea that by encapsulating data destined for a UE it can be routed to the correct destination (eNB serving UE) transparently and efficiently.

As all traffic destined for a UE will come to the P-GW, the P-GW must be able to quickly determine which eNB and S-GW to send the encapsulated data too.

The encapsulated data is logically grouped into tunnels between each node.

A GTP tunnel exists between the S-GW and the P-GW, another GTP tunnel exists between the S-GW and the eNB.

Each tunnel between the eNB and the S-GW, and each tunnel between S-GW and P-GW, is allocated a unique 32 bit value called a Tunnel Endpoint Identifier (TEID) allocated by the node that corresponds to each end of the tunnel and each TEID is locally unique to that node.

For each packet of user data (GTP-U) sent through a GTP tunnel the TEID allocated by the receiver is put in the GTP header by the sender.

The destinations of the tunnels can be updated, for example if a UE moves to a different eNB, the tunnel between the S-GW and the eNB can be quickly updated to point at the new eNB.

If the UE moves to a different eNB only the tunnel between the S-GW and the eNB needs to be updated

Each end of the tunnel is associated with a TEID, and each time a GTP packet is sent through the tunnel it includes the TEID of the remote end (reciever) in the GTP header.

IMTx: NET02x (4G Network Essentials) – Management of Data Flows – 2. GTP Protocol

EPC, EUTRAN, LTE, Mobile Networks, Notes, RF, , ,

These are my lecture notes from IMT’s NET02x (4G Network Essentials) course, I thought I’d post them here as they may be useful to someone. You can find my complete notes here.

When a packet arrives from an external network, like the internet, it is routed to the P-GW.

The P-GW takes this packet and places it in another IP packet (encapsulates it) and then forwards the encapsulated data to the Serving-Gateway.

The S-GW then takes the encapsulated data it just recieved and sends it on inside another IP packet to the eNB.

The encapsulated data sent from the P-GW to the S-GW, and the S-GW to the eNB, is carried by UDP, even if the traffic inside is TCP.

Communication between these elements can be done using internal addressing, and this addressing information will never be visible to the UE or the external networks, and only the P-GW needs to be reachable from the external networks.

This encapsulation is done using GTP – the GPRS Tunneling Protocol.

Specifically IP traffic to and from the UE is contained in GTP-U (User data) packets.

The control data for GTP is contained in GTP-C packets, which sets up tunnels for the GTP traffic to flow through (more on that later).

To summarize, user IP packets are encapsulated into GTP-U packets, which are a transported by UDP between the different nodes (S-GW and eNB)

IMTx: NET02x (4G Network Essentials) – Management of Data Flows – 1. Principle of Encapsulation

EPC, EUTRAN, LTE, Mobile Networks, RF, , , , , ,

These are my lecture notes from IMT’s NET02x (4G Network Essentials) course, I thought I’d post them here as they may be useful to someone. You can find my complete notes here.

Mobile networks are by definition, mobile.

In a fixed network, if I were to connect my laptop to my network at home, I’d get a different address to if I plug it in at work.

In a mobile network UEs are often moving, but we can’t keep changing the IP address – that would lead to all sorts of issues.

The UE must maintain the same IP address, at least for the duration of their session.

Instead the IP address of a UE is allocated by the P-Gateway (P-GW) when the UE attaches.

The IP the P-GW allocates to the UE is in a subnet managed by the P-GW – that IP prefix is associated with the P-GW, so traffic is sent to the P-GW to get to the UE.

Therefore all traffic destined for the UE from external networks will be sent to the P-GW first.

Because the UE is mobile and changing places inside the network, we need a way to keep the UE’s IP even though it’s moving around, something that by default TCP/IP networks don’t cater for very well.

To achieve this we use a technique known as encapsulation where we take the complete IP packet for the UE, and instead of forwarding it on to the UE on a Layer 3 or Layer 2 level, we bundle the whole packet up and put it inside a new IP packet we can forward on anywhere regardless of what’s insude, until it gets to the eNB the UE is on when it can be unpacked and sent to the UE, which is unaware of all the steps / hops it went through.

The concept is very similar to GRE to a VPN (PPTP, IPsec, L2TP, etc) where the user’s IP packets are encapsulated inside another IP packet.

We encapsulate the data using GTP – GPRS Tunneling Protocol.

From a Layer 3 perspective the fact the contents of the GTP packet (another IP packet) is irrelevant, and it’s just handled like any other IP packet being sent from the P-GW to the S-GW.

Getting traffic to the UE

When a packet is sent to the UE’s IP Address from an external destination, it’s first sent to the P-GW, as the P-GW manages that IP prefix.

The P-GW identifies the packet as being destined for a UE, so it encapsulates the entire packet for the UE, by wrapping it up inside a GTP packet.

UE’s IP Packet encapsulated by the P-GW and sent to the IP of the S-Gw

The P-GW then forwards the GTP packet to the serving Serving-Gateway (S-GW)’s IP Address, so the S-GW can forward it to the eNB to get to the UE.

(The P-GW forwards the traffic to the S-GW so the S-GW can get it to the correct eNB for the UE, the reason for having two nodes to manage this is so it can scale better)

Once the traffic gets to the the eNB serving the UE, the eNB de-encapsulates the data, so it’s now got an IP packet with the destination IP of the UE.

It takes the data and puts it onto a transport block sent to the specific RNTI of the UE.

The hops between the UE and the P-GW are transparent to the UE – it doesn’t see the IP Address of the eNB or the S-GW, or any of the routers in between.

Further Reading

Wikipedia – GPRS Tunneling Protocol

3GPP Specs for LTE’s implementation of GTP

Packet capture of some GTP packets