MikroTik Fasttrack with IPsec

[Andy]

Fasttrack is a new feature introduced in RouterOS v6.29 that allows you to forward packages in a way that they are not handled by the Linux Kernel which greatly improves the throughput of your router as well as lowering the CPU load.

Fasttrack allows all packages that have the state Established or Related to bypass the Kernel and be directly forwarded to the target. So, once a connection is marked as established or related, it won’t go through any firewalling or processing and will directly forwarded to the target. Of course – a connection gains the state of established or related once it went through the firewall so it will still be secure.

But there’s a known issue that Fasttrack will not work with IPsec connections, it will result in a rather wonky experience or very unstable IPsec connection. So if you have IPsec connections in your MikroTik but want to take the advantages of Fasttrack, here’s the resolution for you!

Firstly, we want to mark all IPsec connections:

/ip firewall mangle add action=mark-connection chain=forward comment="mark ipsec connections" ipsec-policy=out,ipsec new-connection-mark=ipsec
/ip firewall mangle add action=mark-connection chain=forward comment="mark ipsec connections" ipsec-policy=in,ipsec new-connection-mark=ipsec

Secondly, we want to create a new Fasttrack rule but excluding all IPsec connections that we have marked with the two mangle rules above:

/ip firewall filter add chain=forward action=fasttrack-connection connection-state=established,related connection-mark=!ipsec
/ip firewall filter add chain=forward action=accept connection-state=established,related

If any, don’t forget to delete your old Fasttrack rule.

I’ve been using this rule in my live IPsec site-to-site tunnel and the result is just simply amazing!

Let me know if you have any question or comment by posting down below. Cheers!

 

[Lsi]

This doesn't seem to be a bug anymore, as the L2TP IPsec performance on my hEX via SMB in a SHA1 AES-128 tunnel (tunnel MTU 1400) with a Windows client caps around 10Mb (on a 90Mb fiber business connection) with this set of mangle & firewall rules enabled or disabled. Web traffic tested via Speedtest.net does not cap, as it scales to around 75Mb, if I allow Internet traffic through the tunnel.

Are the IPsec issues fixed in modern RouterOS? These tests were on RouterOS 6.47.1 and 6.47.2.

Lower end Mikrotik routers (eg: RB951G) actually produce higher SMB throughput in the same cfg on the same versions of RouterOS, despite having less cores and no IPsec offload capability.

If you have any suggestion to speeding up SMB throughput across the tunnel that would be appreciated, as I am looking to migrate from similar class Ubiquiti Edgerouter products which cap both types of traffic at 35-40Mb through an identically configured L2TP IPsec tunnel.

 

[Lsi]

Those throughput #'s on the Ubiquitis are with working IPsec offloading (Cavium-based Edgerouter Lite) and broken / disabled IPsec offloading (Mediatek-based Edgerouter X) tested on 600/100 cable connections. All WANs involved have an optimal MTU of 1500.

 

[Andy]

Hi @Lsi,

Thank you for bringing this exciting discussion.

When we tested, that problem appears on all MikroTik models, even with our CCR (Tile) routers.

So we created the rules above and apply them as our standard setup.

Are the IPsec issues fixed in modern RouterOS? These tests were on RouterOS 6.47.1 and 6.47.2.

At this stage, we haven't gone RouterOS 6.47, as we're using the Long-term, so our production routers are running on v6.45.9 at the moment.

From my point of view, the problem seems caused by MikroTik configuration being 'vanilla', hence a supplementary rule is required for IPsec with Fasttrack, or it could be also a bug indeed, which they might fix in the later version.

Also, it's worth mentioning that we don't think it's an architectural problem, we've tested MIPSBE, MIPS, ARM, Intel, Tile MikroTik routers, and all of them appears having the same issues.

 

[Lsi]

To summarize what I was trying to say, on 6.47 these rules don't seem to be needed or have any performance benefit in this L2TP IPsec server throughput issue scenario. So it is probably "fixed" in the sense that you don't have to disable IPsec fasttrack with your rules here to ensure normal performance levels.

The SMB throughput issue points to a different severe performance bug somewhere (re 10Mb cap & ~7x slower than web traffic). I had assumed that IPsec offloading was working to some degree--since web traffic is ~2x faster over the IPsec tunnel compared to similar Ubiquiti products, but this could just be much more efficient firmware design by MikroTik. The similar class Ubiquitis are performing at this lower (but consistent) 35-40Mb level with working IPsec offload (Cavium) or with broken IPsec offloading but a more powerful processor (Mediatek).

So using your rules seems to show that the hEX on 6.47 firmware is automatically disabling fasttracked IPsec, but tunnel throughput varies widely depending on content type. If they could fix the tunnel performance to operate consistently at 60-70Mb without fixing fasttrack, I would be happy with those results considering its price point.