Using Exablaze Hardware for risk monitoring

In this blog I want to go over how some of our hardware can be used for logging and risk analysis. The idea is to give you a feel for how you can do logging and risk analysis with very little latency overhead. The solutions I'm presenting here make use of both the ExaNIC X4 and the ExaLINK ultra low latency switch.

Logging and risk analysis

The purpose of logging is to keep an accurate record of all messages sent between two or more systems. Ideally, these messages will be monitored and stored by an independent system or server. This record of messages can then be used in a number of ways, perhaps the most useful of which is in replay. The idea is that with an accurate record of what happened, and when, you can reconstruct events from the past and learn from them.

Having a live and accurate record of messages sent between your trading servers and the exchange also allows you to have an independent system monitor your open positions.

When you're trading though, one thing you definitely don't want is the logging and risk analysis system getting in the way or slowing down your trades. This is where our hardware can help - you can definitely keep accurate log of all events without impacting on your overall latency budget.

ExaLINK port mirroring

Let's look at how the ExaLINK can be used in logging and risk control applications, with particular focus on a typical trading environment. Let's assume you have a trading system that includes a number of clients, all sending orders to a single front end processor (FEP) or gateway that is responsible for validating orders prior to sending them to the exchange.

Let's say we want to have a log of:

  • All messages sent to and from the exchange
  • All messages sent from any client to the FEP

We can create these logs using the ExaLINK, configuring it to mirror ports. The easiest way to do this is to patch two ports together through the ExaLINK (this results in under 5 ns of latency - equivalent to less than a meter of fibre). Then, mirror the port you want to log out of a port connected to the logging server.

In the diagram below, we're taking client orders into one port on the ExaLINK, which we then patch through to the FEP. We also mirror these client orders through to a logging/risk machine. Likewise, we also take orders sent from the FEP through the ExaLINK, prior to being routed to the exchange. We then mirror FEP orders to the logging/risk machine.

Diagram showing how ExaLINK can be used to connect to a logging server

This way the logging machine can see all traffic being sent from the client to the FEP, and from the FEP to the exchange all with virtually no latency penalty - under 5 ns per patch through the ExaLINK. By using ExaNIC network cards in both the FEP and the logging/risk machine, we can very accurately timestamp all packets as they pass through the network - to 6 ns resolution.

One big advantage of this setup is that the ExaLINK can act as a very fast kill switch - that is, if a client is sending too many invalid orders it can be disconnected at the physical layer from the rest of the network. This can be done programmatically through our API or SNMP

Bringing in the ExaNIC

Let's look in a little more detail at the ExaNIC low latency network interface card. One of the more interesting features of this card is port mirroring, which allows you to replicate any traffic transmitted or received on the first 3 ports out of the 4th port. This is all done in hardware on the card and has no effect on the latency. This has some obvious advantages when you're trying to log traffic sent by a trading server!

Let's look at the same example we discussed above, but this time focussing just on the FEP. This time, we're using the ExaNIC X4 in the FEP to:

  • Receive client orders on port 1
  • Transmit orders to the exchange on port 2
  • Log received or transmitted packets out of port 3

We can get the ExaNIC to replicate all transmitted and received traffic on ports 1 and 2 out of port 3 using our exanic-config utility as follows:

$ exanic-config exanic0:1 mirror-tx on
$ exanic-config exanic0:1 mirror-rx on
$ exanic-config exanic0:2 mirror-tx on
$ exanic-config exanic0:2 mirror-rx on

Now anything that port 1 or 2 transmits or receives will be replicated, by hardware, out of port 3 to our logging machine.

Diagram showing how mirroring can be used to connect to a logging server

This works well in instances where there are less clients than would necessitate a full low latency switch like the ExaNIC.

As usual if you have any questions or want more information about anything we've discussed in this blog, get in touch!