Fast Stream Ciphers

A quick post. 

We have some home made cheapy frankenstorage, thats based around a low end celeron class desktop CPU. This is fine for moderate raid work, but recently I had to shovel a few terabytes between them, over the network. Making that run quickly enough led to a bit of experimentation with the speeds of the different ciphers.

Normally I’d reach for nc at this point and run something like this on the sender

# export GZIP="-4"
# tar czf - /partition_to_be_backed_up | pv | nc remote_host 31337 

with an nc listener outputting to tar in the right place to receive the data on the remote_host, with an optional pv to get the output rate.  

The -4 option to gzip can make a large difference on underpowered systems. At a trade off of about 10% of the compressed space, you can be up to 10x quicker than gzip -9. 

This system only had two cores. There are other options – like pigz for multicore machines. 

However, this needed ssh. 

Since ssh dropped the none cipher as a default compiled in option most people go with the default and just bear the delay. 

A bit of testing gave me the following results however;

Cipher Name      Transfer Rate recorded (kbit/sec)
3des (default) 237209.7
des 242359.0
blowfish 242359.0
arcfour 326436.8

So with a combination of fast gzip and arcfour, we get almost 100kbit/sec faster. In this case, that lopped several hours off the transfer time. 

Since taking the notes for this post, I’ve found the following blog post which goes into much more detail.

http://blog.famzah.net/2010/06/11/openssh-ciphers-performance-benchmark/

Their cpu under test is much slower than the one we had, but there is still benefit to be had by choosing the correct cipher with the ‘-c’ option to scp.

Any opinions, news, research, analyses, prices or other information ("information") contained on this Blog, constitutes marketing communication and it has not been prepared in accordance with legal requirements designed to promote the independence of investment research. Further, the information contained within this Blog does not contain (and should not be construed as containing) investment advice or an investment recommendation, or an offer of, or solicitation for, a transaction in any financial instrument. LMAX Group has not verified the accuracy or basis-in-fact of any claim or statement made by any third parties as comments for every Blog entry.

LMAX Group will not accept liability for any loss or damage, including without limitation to, any loss of profit, which may arise directly or indirectly from use of or reliance on such information. No representation or warranty is given as to the accuracy or completeness of the above information. While the produced information was obtained from sources deemed to be reliable, LMAX Group does not provide any guarantees about the reliability of such sources. Consequently any person acting on it does so entirely at his or her own risk. It is not a place to slander, use unacceptable language or to promote LMAX Group or any other FX and CFD provider and any such postings, excessive or unjust comments and attacks will not be allowed and will be removed from the site immediately.