I think a more accurate title would be "How to reduce memory usage 10x in almost anything that uses OpenSSL." Bottom line: All those bookkeeping data structures for compression take up a lot of space. Disable SSL compression and you can go from 1MB per connection down to 100kB.

This is a really good writeup. I have never used Instruments before, and while I'm not an expert in OpenSSL, there was enough detail to point me in the right direction when dealing with this in the future. Thanks.

Don't forget to enable SSL_MODE_RELEASE_BUFFERS with OpenSSL 1.0.0 too.

Ah, good point, Node already does this, but I'll append the post to mention it.

The article also covers the same issue in Python and shows how to drastically reduce memory usage. Pretty awesome stuff!

This is great not because Python is my language of choice but because this shows how to generalize it to other cases and not stay in node land. I wish more articles would take this approach.

I'd like NodeJS to add a tls.createCTX() method that generates an OpenSSL CTX object that can be manipulated and passed as an optional parameter to the various other tls methods.

This would allow great flexibility in the TLS configuration - everything from CA paths to CRL checking to RFC 5077 support could be configured by the application once and the settings reused for all qualifying subsequent connections. Compare with, e.g., AnyEvent::TLS (http://search.cpan.org/~mlehmann/AnyEvent-5.31/lib/AnyEvent/...).

I love posts like this. There are too many blog posts that make it to the top of HN that are just complaining or speculating of some type. I would much rather read a detailed write-up on the discovery and solution to a problem.

Summary: when using OpenSSL with a lot of connections, use a shared SSL_CTX or you'll have a large per-connection overhead.

Eep! Skipping compression before you encrypt is a bad idea. Using it saves bandwidth, & makes attacks much more complicated.

Can Node.js do some compression before it hands off data to OpenSSL?

Probably if you use something like node-compress or node-zippy.

If you want to debug an issue like this on Linux, the Massif tool (part of Valgrind) is a decent heap profiler.

What is the windows variation to those/Instruments.app? (generally speaking, I am aware that it might not apply to debugging node apps running on top of cygwin)

It is interesting, how he is using OpenSSL, Node.js, Python, Twisted framework and still manages to take a swing at OSS development tools in favor of closed ones.

Does this issue even exists on serious operating systems?