Kamailio Bytes – HTable

HTable is Kamailio’s implimentation of Hash Tables a database-like data structure that runs in memory and is very quick.

It’s uses only become apparent when you’ve become exposed to it.

Let’s take an example of protecting against multiple failed registration attempts.

We could create a SQL database called registration attempts, and each time one failed log the time and attempted username.

Then we could set it so before we respond to traffic we query the database, find out how many rows there are that match the username being attempted and if it’s more than a threshold we set we send back a rate limit response.

The problem is that’s fairly resource intensive, the SQL data is read and written from disks and is slow to do both.

Enter HTable, which achieves the same thing with an in-memory database, that’s lightning fast.

Basic Setup

We’ll need to load htable and create an htable called Table1 to store data in:

loadmodule "htable.so"
...
modparam("htable", "htable", "MessageCount=>size=12;initval=0")

Now we’ve initialised a new htable called MessageCount with a size of 12 bytes, and an initial value of 0.

Basic Usage

Now we can put some code in our request_route{} block to increment the MessageCount htable entry each time a new message is received.

request_route {
         $sht(MessageCount=>test) = $sht(MessageCount=>test) + 1;
         xlog("MessageCount is $sht(MessageCount=>test)");
}

$sht(MessageCount=>test) is the logical link to the Htable called MessageCount with a key named test. We’re making that equal itself + 1.

We’re then outputting the content of $sht(MessageCount=>test) to xlog too so we can see it’s value in Syslog.

Now each time a new dialog is started the MessageCount htable key “test” will be incremented.

We can confirm this in Syslog:

ERROR: : MessageCount is 1
ERROR: : MessageCount is 2

We can also check this in kamcmd too:

htable.dump MessageCount

Here we can see in MessageCount there is one key named “test” with a value of 6, and it’s an integer. (You can also store Strings in HTable).

So that’s all well and pointless, but let’s do make it a bit more useful, report on how many SIP transactions we get per IP. Instead of storing our values with the name key “test” we’ll name it based on the Source IP of the message, which lives in Psedovariable $si for Source IP Address.

request_route {
        $sht(MessageCount=>$si) = $sht(MessageCount=>$si) + 1;
        xlog("MessageCount is $sht(MessageCount=>$si)");
}

Now let’s dump the MessageCount again:

htable.dump MessageCount

Done.

Now we can see a count of how many transactions each IP has.

Doing Useful Things

One of the most obvious usage examples of HTable usage is rate limiting authentication attempts. So let’s do that.

We’ll need to create a new htable to contain our AuthCount table:

modparam("htable", "htable", "AuthCount=>size=12;initval=0;autoexpire=360")

I’m calling the boilerplate AUTH block, and I’ve added some logic to increment the AuthCount for each failed auth attempt, and reset it to $null if authentication is successful, thus resetting the counter for that IP Address.

if (is_method("REGISTER") || from_uri==myself) {
        # authenticate requests
        if (!auth_check("$fd", "subscriber", "1")) {
                auth_challenge("$fd", "0");
                $sht(AuthCount=>$si) = $sht(AuthCount=>$si) + 1;
                exit;
        }
        # user authenticated - remove auth header
        if(!is_method("REGISTER|PUBLISH"))
                consume_credentials();
                $sht(AuthCount=>$si) = $null;
}

Now we’ve done that we need to actually stop the traffic if it’s failed too many times. I’ve added the below check into REQINIT block, which I call at the start of processing:

if($sht(AuthCount=>$si) > 5){
        xlog("$si is back again, rate limiting them...");
        sl_send_reply("429", "Rate limiting");
        exit;
}

Now if AuthCount is more than 5, it’ll respond with a Rate Limiting response.

Because in our modparam() setup for AuthCount we set an expiry, after 360 seconds (10 minutes), after 10 minutes all will be forgiven and our blocked UA can register again.

Advanced Usage / Notes

So now we’ve got Kamailio doing rate limiting, it’s probably worth mentioning the Pike module, which can also be used.

You’ll notice if you reboot Kamailio all the htable values are lost, that’s because the hashes are stored in memory, so aren’t persistent.

You have a few options for making this data persistent,

By using DMQ you can Sync data between Kamailio instances including htable values.

modparam("htable", "enable_dmq", 1)

You can also sync it to a database backend:

modparam("htable", "db_url", "mysql://kamailio:kamailiorw@localhost/kamailio")

kamcmd can view, modify & manipulate htable values.

As we’ve seen before we can dump the contents of an htable using:

kamcmd htable.dump MessageCount

We can also add new entries & modify existing ones:

kamcmd htable.seti MessageCount ExampleAdd s:999

htable.seti is for setting integer values, we can also use htable.sets to set string values:

htable.sets MessageCount ExampleAdd Iamastring

We can also delete values from here too, which can be super useful for unblocking destinations manually:

htable.delete MessageCount ExampleAdd

As always code from this example is on GitHub. (Please don’t use it in production without modification, Authentication is only called on Register, and it’s just built upon the previous tutorials).

Kamailio documentation for HTable module.