Tag Archives: SCTP

SCTP Parameter Tuning

EPC, LTE, Mobile Networks, RFCs & Standards,

There’s a lot to like about SCTP. No head of line blocking, MTU issues, sequenced, acknowledged delivery of messages, not to mention Multi-Homing and message bundling.

But if you really want to get the most bang for your buck, you’ll need to tune your SCTP parameters to match the network conditions.

While tuning the parameters per-association would be time consuming, most SCTP stacks allow you to set templates for SCTP parameters, for example you would have a different set of parameters for the SCTP stacks inside your network, compared to SCTP stacks for say a roaming scenario or across microwave links.

IETF kindly provides a table with their recommended starting values for SCTP parameter tuning:

RTO.Initial3 seconds
RTO.Min1 second
RTO.Max60 seconds
Max.Burst4
RTO.Alpha1/8
RTO.Beta1/4
Valid.Cookie.Life60 seconds
Association.Max.Retrans10 attempts
Path.Max.Retrans5 attempts (per destination address)
Max.Init.Retransmits8 attempts
HB.interval30 seconds
HB.Max.Burst1
IETF – RFC4960: SCTP – Suggested Protocol Parameter Values

But by adjusting the Max Retrans and Retransmission Timeout (RTO) values, we can detect failures on the network more quickly, and reduce the number of packets we’ll loose should we have a failure.

We begin with the engineered round-trip time (RTT) – that is made up of the time it takes to traverse the link, processing time for the remote SCTP stack and time for the response to traverse the link again. For the examples below we’ll take an imaginary engineered RTT of 200ms.

RTO.min is the minimum retransmission timeout.
If this value is set too low then before the other side has had time to receive the request, process it and send a response, we’ve already retransmitted it.

This should be set to the round trip delay plus processing needed to send and acknowledge a packet plus some allowance for variability due to jitter; a value of 1.15 times the Engineered RTT is often chosen

So for us, 200 * 1.15 = 230ms RTO.min value.

RTO.max is the maximum amount of time we should wait before transmitting a request.
Typically three times the Engineered RTT.

So for us, 200 * 3 = 600ms RTO.min value.

Path.Max.Retransmissions is the maximum number of retransmissions to be sent down a path before the path is considered to be failed.
For example if we loose a transmission path on a multi-homed server, how many retransmissions along that path should we send until we consider it to be down?

Values set are dependant on if you’re multi-homing or not (you can be more picky if you are) and the level of acceptable packet loss in your transmission link.

Typical values are 4 Retransmissions (per destination address) for a Single-Homed association, and 2 Retransmissions (per destination address) for a Multi-Homed association.

Association.Max.Retransmissions is the maximum number of retransmissions for an association. If a transmission link in a multi-homed SCTP scenario were to go down, we would pass the Path.Max.Retransmissions value and the SCTP stack would stop sending traffic out that path, and try another, but what if the remote side is down? In that scenario all our paths would fail, so we need another counter – Path.Max.Retransmissions to count the total number of retransmissions to an association / destination. When the Association.Max.Retransmissions is reached the association is considered down.

In practice this value would be the number of paths, multiplied by the Path.Max.Retransmissions.

SCTP Multihoming

Mobile Networks, RFCs & Standards, ,

One of the key advantages of SCTP over TCP is the support for Multihoming,

From an application perspective, this enables one “socket”, to be shared across multiple IP Addresses, allowing multiple IP paths and physical NICs.

Through multihoming we can protect against failures in IP Routing and physical links, from a transport layer protocol.

So let’s take a look at how this actually works,

For starters there’s a few ways multihoming can be implemented, ideally we have multiple IPs on both ends (“client” and “server”), but this isn’t always achievable, so SCTP supports partial multi-homing, where for example the client has only one IP but can contact the server on multiple IP Addresses, and visa-versa.

The below image (Courtesy of Wikimedia) shows the ideal scenario, where both the client and the server have multiple IPs they can be reached on.

Arkrishna, CC BY-SA 4.0 <https://creativecommons.org/licenses/by-sa/4.0>, via Wikimedia Commons

This would mean a failure of any one of the IP Addresses or the routing between them, would see the other secondary IP Addresses used for Transport, and the application not even necessarily aware of the interruption to the primary IP Path.

The Process

For starters, our SCTP Client/Server will each need to be aware of the IPs that can be used,

This is advertised in the INIT message, sent by the “client” to a “server” when the SCTP session is established.

SCTP INIT sent by the client at 10.0.1.185, but advertising two IPs

In the above screenshot we can see the two IPs for SCTP to use, the primary IP is the first one (10.0.1.185) and also the from IP, and there is just one additional IP (10.0.1.187) although there could be more.

In a production environment you’d want to ensure each of your IPs is in a different subnet, with different paths, hardware and routes.

So the INIT is then responded to by the client with an INIT_ACK, and this time the server advertises it’s IP addresses, the primary IP is the From IP address (10.0.1.252) and there is just one additional IP of 10.0.1.99,

SCTP INIT ACK showing Server’s Multi-homed IP Options

It’s worth noting that according to RFC 4960 Multi-homing is Optional and so is the IP Address Header, if it’s not advertised the sender is single-homed.

Next up we have the cookie exchange, which is used to protect against synchronization attacks, and then our SCTP session is up.

So what happens at this point? How do we know if a path is up and working?

Well the answer is heartbeat messages,

Sent from each of the IPs on the client to each of the IPs on the server, to make sure that there’s a path from every IP, to every other IP.

SCTP Heartbeats from each local IP to each remote IP

This means the SCTP stacks knows if a path fails, for example if the route to IP 10.0.1.252 on the server were to fail, the SCTP stack knows it has another option, 10.0.1.99, which it’s been monitoring.

So that’s multi-homed SCTP in action – While a lot of work has historically been done with LACP for aggregating multiple NICs together, and VRRP for ensuring a host is alive, SCTP handles this in a clean and efficient way.

I’ve attached a PCAP showing multi-homing on a Diameter S6a (HSS) interface between an MME and a HSS.

SCTP_MultiHomed_4

IMTx: NET02x (4G Network Essentials) – Management of Data Flows – 7. NAS and Global View of Protocol Stack

EPC, EUTRAN, LTE, Mobile Networks, Notes, RF, , , , , ,

These are my lecture notes from IMT’s NET02x (4G Network Essentials) course, I thought I’d post them here as they may be useful to someone. You can find my complete notes here.

The LTE architecture compartmentalises the roles in the mobile network.

For example the eNB concentrates on radio connection management, while the MME focuses on security and mobility.

Non Access Stratum (NAS) messages are exchanged between the terminal and the MME.

Access Stratum (AS) messages are exchanged over the air between the UE and the eNB. It contains all the radio related information.

The eNB must map the NAS messages from an MME to a LCID and RNTI and transmit them over the air, and vice-versa. The eNB forwards this data without ever analyzing it.

Transport of S1 messages is carried over SCTP which I’ve spoken about in the past.

The above image overlaps with the radio version we talked about earlier.

IMTx: NET02x (4G Network Essentials) – Management of Data Flows – 5/6. S1AP Connection

EPC, EUTRAN, LTE, Mobile Networks, Notes, RF, , , , , ,

These are my lecture notes from IMT’s NET02x (4G Network Essentials) course, I thought I’d post them here as they may be useful to someone. You can find my complete notes here.

Each MME can manage millions of UEs.

To handle this load the requirements of each subscriber for the MME must be as minimal and simple as possible so as to scale easily.

For each UE in the network a connection is setup between the UE and the MME.

This is done over the S1-AP’s Control Plane interface (sometimes calls S1-Control Plane or S1-CP) which carries control plane data to & from the UE via the eNB to the MME.

S1-CP is connection-oriented, meaning each UE has it’s own connection to the MME, so there are as many S1-CP connections to the MME as UE’s connected.

Each of these S1-CP connections is identified by a pair of unique connection IDs. The eNB keeps track of the connection IDs for each UE connected and hands this information off each time the UE moves to a different eNB.

The eNB keeps a lookup table between the RNTI of the UE and the LCID – the Logical Channel Identifier. This means that the eNB knows the sent and received ID of the S1-CP connection for each UE, and is able to translate that into the RNTI and LCID used to send the data over the air interface to the UE.

S1-CP Connect (Attach Procedure)

As we discussed in radio interfaces, when a UE connects to the network it is assigned an RNTI to identify it on the radio interface and allocate radio resources to it.

Once the RNTI is confirmed by both the eNB and the UE, a EMM Attach Request, which is put into an RRC Message called RRCConnectionSetupComplete.

The eNB must next choose a serving MME for this UE. It picks one based on it’s defined logic, and sends a S1-AP Intial UE Message (EMM Attach Request) to the MME along with the eNB’s connection identity assigned for this connection.

The MME stores the connection identity assigned by the eNB and chooses it’s own connection identity for it’s side, and sends back an S1AP Downlink NAS Transport response with both connection identities and the response for the attach request (This will be an EMM Authentication Request).

The eNB then stores the connection identity pair and the associated RNTI and LCID for the UE, and forwards the EMM Authentication Request to the RNTI of the UE via RRC.

The UE will pass the authentication challenge input parameters to the USIM which will generate a response. The UE will send the output of this response in a EMM Authentication Responseto the eNB, which will look at the RNTI and LCID received and consult the table to find the Connection Identifiers and IP of the serving MME for this UE.

S1AP Connect procedure

Kamailio Bytes – SCTP

Linux, RFCs & Standards, VoIP, , ,

I’ve talked about how cool SCTP is in the past, so I thought I’d describe how easy it is to start using SCTP as the Transport protocol in Kamailio.

I’m working on a Debian based system, and I’ll need to install libsctp-dev to use the SCTP module.

apt-get install libsctp-dev

Next we’ll edit the Kamailio config to load module sctp in the loadmodules section:

...
loadmodule "sctp.so"
...

Now we’ll start listening on SCTP, so where your current listen= entries are we’ll add one:

listen=sctp:0.0.0.0:5060

I’ve loaded Dispatcher for this example, and we’ll add a new entry to Dispatcher so we can ping ourselves.

We’ll use kamctl to add a new dispatcher entry of our loopback IP (127.0.0.1) but using SCTP as the transport.

kamctl dispatcher add 1 'sip:127.0.0.1:5060;transport=sctp' 0 0 '' 'Myself SCTP'

Now I’ll restart Kamailio and check kamcmd:

kamcmd dispatcher.list

All going well you’ll see the entry as up in Dispatcher:

And firing up tcpdump should show you that sweet SCTP traffic:

tcpdump -i lo -n sctp

Sadly by default TCPdump doesn’t show our SIP packets as they’re in SCTP, you can still view this in Wireshark though:

Here’s a copy of the packet capture I took:

SCTP Packet CaptureDownload

I’ve put a copy of my basic config on GitHub.

Now get out there and put SCTP into the real world!

SCTP (And why you should be using it)

RFCs & Standards, , ,

XKCD said it best:

XKCD Standards

Sometimes standards are created that are superior in some scenarios, and just don’t get enough love.

To me Stream Control Transmission Protocol (SCTP) is one of those, and it’s really under-utilised in Voice.

Defined by the SIGTRAN working group in 2000 while working to transport SS7 over IP, SCTP takes all the benefits of TCP, mixes in some of the benefits of UDP (No head of line blocking) and mutihoming support, and you’ve got yourself a humdinger of a Transmission Protocol.

Advantages

Reliable Transmission

Like TCP, SCTP includes a reliable transmission mechanism that ensures packets are delivered and retries if they’re not.

Multi Homing

SCTP’s multi homing allows a single connection to be split across multiple paths. This means if you had two paths between Melbourne and Sydney, you could be sending data down both simultaneously.

This means a loss of one transmission path results in the data being sent down another available transmission path.

If you’re doing this using TCP you’d have to wait for the TCP session to expire, BGP to update and then try again. Not so with SCTP.

No Head of Line Blocking

An error / discard with a packet in a TCP stream requires a re-transmission, blocking anything else in that stream from getting through until the error/discarded packet is sorted out. This is referred to as “head of line blocking” and is generally avoided by switching to UDP but that looses the reliability.

4 Way Handshake

Compared to TCP’s 3-way handshake which is susceptible to SYN flooding.

Deployment

If you’ve got a private network, chances are it can support SCTP.

There’s built in SCTP support in almost all Linux kernels since 2002, Cisco iOS and VxWorks all have support, and there’s 3rd party drivers for OSX and Windows.

SCTP is deployed in 3GPP’s LTE / EPC protocol stack for communication over S1-AP and X2 interfaces, meaning if you’ve got a LTE enabled mobile you’re currently using it, not that you’d see the packets.

You’ll find SCTP in SIGTRAN implementations and some TDM-IP gateways, Media Gateways, protocol converters etc, but it’s not widely deployed outside of this.