Improved FreeNAS SCP on X10SDV-4C+-TLN4F

FreeNAS is great and supports a wide variety of hardware, I highly suggest using it.

I setup FreeNAS on Supermicro's X10SDV-4C+-TLN4F and noticed slow scp transfers of 1MB/s. Horrendous, but with some debugging I discovered the changes below saturate an nfs transfer on a 1Gbps network at ~113MB/s.

Most of this article is mentioned in my questions on the FreeNAS forum or this submitted FreeNAS issue.

Don't use openssh cipher chacha20-poly1305@openssh.com

OpenBSD or FreeNAS has some issue where the chacha20 cipher is very slow on this specific hardware. So don't use that cipher.

In FreeNAS navigate to Services -> edit SSH -> Advanced -> Extra options and use:

Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com  

After this change I was up to 30MB/s-50MB/s.

Use bigger buffers for networking

To get at the 113MB/s rate there are a few sysctls that I needed to enable. But note that the autotune sucks so please don't use that. Also note the below sysctls are for My Hardware and not necessarily yours -- so I suggest using command line before making these changes permanent!!! These options were discovered by reading the FreeNAS manual section on networking.

Really, applying these in the FreeNAS GUI out of order may break all network connections. Try the CLI first (or backup freenas config) or you risk making changes without the GUI.

Apply these sysctls in this order:

sysctl kern.ipc.maxsockbuf=33554432  
sysctl net.inet.tcp.recvbuf_auto=1  
sysctl net.inet.tcp.sendbuf_auto=1  
sysctl net.inet.tcp.sendbuf_max=33554432  
sysctl net.inet.tcp.recvbuf_max=33554432  
sysctl net.inet.tcp.recvspace=4194304  
sysctl net.inet.tcp.sendspace=4194304  

Test another transfer and I hope you see 113MB/s!


Powered by Digital Ocean

Click the image for a $10 referral