FreeNAS is great and supports a wide variety of hardware, I highly suggest using it.
I setup FreeNAS on Supermicro's X10SDV-4C+-TLN4F and noticed slow scp transfers of 1MB/s. Horrendous, but with some debugging I discovered the changes below saturate an nfs transfer on a 1Gbps network at ~113MB/s.
Most of this article is mentioned in my questions on the FreeNAS forum or this submitted FreeNAS issue.
Don't use openssh cipher chacha20-poly1305@openssh.com
OpenBSD or FreeNAS has some issue where the chacha20 cipher is very slow on this specific hardware. So don't use that cipher.
In FreeNAS navigate to Services
-> edit SSH
-> Advanced
-> Extra options
and use:
Ciphers aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
After this change I was up to 30MB/s-50MB/s.
Use bigger buffers for networking
To get at the 113MB/s rate there are a few sysctls that I needed to enable. But note that the autotune
sucks so please don't use that. Also note the below sysctls are for My Hardware and not necessarily yours -- so I suggest using command line before making these changes permanent!!! These options were discovered by reading the FreeNAS manual section on networking.
Really, applying these in the FreeNAS GUI out of order may break all network connections. Try the CLI first (or backup freenas config) or you risk making changes without the GUI.
Apply these sysctls in this order:
sysctl kern.ipc.maxsockbuf=33554432
sysctl net.inet.tcp.recvbuf_auto=1
sysctl net.inet.tcp.sendbuf_auto=1
sysctl net.inet.tcp.sendbuf_max=33554432
sysctl net.inet.tcp.recvbuf_max=33554432
sysctl net.inet.tcp.recvspace=4194304
sysctl net.inet.tcp.sendspace=4194304
Test another transfer and I hope you see 113MB/s!