How we cut down memory usage by 82%

redis-at-scale

RAM memory in comparison to disk storage is not so cheap. Having big Redis cluster could lead to some expenses on such memory. Thanks to lessons we learned on our previous projects we can share with you some optimizing techniques. Nice fact is, they are based on application-side so they won’t affect performance of your database servers.

Compress your data!

It’s a big win if you compress data before saving to Redis.

In one of our Redis instance we store mid-large json objects. Thanks to PHP gzcompress function which internally uses ZLIB library we were able to reduce memory usage by 82% – from about 340 GB to 60 GB. Due to that, now we need only one server with 64GB of RAM instead of having instance with 512 GB which would be likely 3 times more expensive. It was the way we could scale up and avoid premature horizontally scaling.

Test

Before using compression we’ve made some tests which let us to chose the best algorithm for us. We prepared test cases for gzcompress and bzcompress algorithms with different compression levels.

They made 500 iterations of the test with ~30KB JSON objects and:

  • calculated compression ratios
  • counted overhead time of the algorithms

Results

Gzcompress proved to be about 6 times faster than bzcompress when compressing and decompressing data.

Compress/Uncompress times

Compress/Uncompress times in comparison with different algorithms

Compression ratio was similar for both algorithms.

Compression ratio in comparision with different algorithms

Compression ratio in comparision with different algorithms

So decision was easy – we used gzcompress because of its speed and good compression ratios.

The CPU overhead in comparison to memory saving is small. Also compression operations are running on application server, so they aren’t affecting in any way Redis performance.

Remember that using this approach is not always appropriate. You shouldn’t do it when your data doesn’t compress well:

  • Isn’t well-structured – JSON, XML is good at compression because of repeated characters, tags
  • You have millions of small keys – which e.g. stores some numbers, short words

Of course you should make tests on your own using data stubs which you would store in the database.

Use short keys

Easy trick to implement but can also reduce amount of required memory.

Think of it: if you’ve 100,000,000 keys named like

my/fancy/key-name/... (18 characters)

the overhead for just storing names is rather big in comparision to possible another format like:

m/f/kn/...   (7 characters)

you save 11 characters (11 bytes), which at big scale could let you to save

100,000,000 * 11 bytes = 1 GB of RAM memory!

Using hasmaps

The cool thing in Redis is that, it can handle data types above then standard key-value memcached model.

You must be aware that every key in Redis comes with some additional metadata. So when you’ll store objects like “key=value” that doesn’t mean you will use 8 bytes per key.

redis redis:6379> DEBUG OBJECT my/key
Value at:0x7f36f2980900 refcount:1 encoding:raw serializedlength:4 lru:463740 lru_seconds_idle:1215660

Redis has to e.g. store some info for LRU algorithm.

Thanks to hashes you can save some space. The idea was nicely described by Instagram engineers. Mainly you can create your own, simple memcache in a Redis using hashes 😉 Values stored in hashes don’t come with additional metadata. Internally Redis uses simple dictionary structure for storing them.

More info about this technique can be find at newly added memory optimization tips page on Redis.io.

Last thoughts…

As you can see, some micro optimizations could let you to save much amount of RAM memory which is not so cheap. But as shown on examples – they will be noticeable at larger scale.

In fact, such tips can be used not only in Redis, also other databases can be affected. For e.g. MongoDB need to store duplicated attribute names for every object- using short names will also be effective.


Are you looking for experienced team to scale-out your application? Check Octivi!

Looking to scale-out your
web application?

Hire Octivi!

Antoni is a Software Architect and Scrum Master at Octivi. He is responsible for software architecture of our key projects, he also holds Professional Scrum Master certificate.

  • Nice feedbacks! Your article will be featured in our next @redisweekly https://redisweekly.com !

  • Jay Johnston

    I compress my JSON in Redis as well and use Google’s Snappy library rather than zlib. I ran tests with my data and Snappy was significantly faster at decompression than zlib. It doesn’t compress as heavily, so the resulting data is slightly larger, but decompression speed was more important for my application.

    • Thanks @Jay for great comment.
      Aside with compression we were investigating other ways to store our serialized blobs – lightweight alternatives to JSON:
      – There’s BSON (http://bsonspec.org/) internally used by MongoDB, which results with slightly smaller data space.
      – UBJSON says that the format led to optimize >30% space, they even described some memory optimization tips on their page – http://ubjson.org/#size

      But in our case the performance wasn’t a problem – the main thing was to reduce memory usage on our Redis servers – we use them as a long-term storage, not a cache.

      The optimization from ~340GB to 60GB let us to have only two 64GB Redis servers instead of >four< 256GB. As you can see, that the costs optimization is significant!

      Btw. Why the numbers of servers are doubled? Because of the redundancy – we keep them in master/slave configuration for high availability 🙂