No, you read that correctly.
For those unfamiliar with Brotli compression – Brotli is a compression format that provides roughly a 30-50% higher compression ratio over the industry-standard GZIP. Brotli servers your website faster than ever before.
Over the past few months though, we’ve conducted hundreds of hours of testing and development. On this day, 7 years old, Google launched its ground-breaking brotli compression format. Today, we’re excited to launch what we hope will be the future compression standard of the hosting industry: Broccoli Compression.
There are two problems with the current Brotli protocol:
- It’s slow. Compression levels beyond 9 take milliseconds.
- It could be more efficient and compress content even further.
For reference, here’s the number of seconds it takes to compress some of Trinculo’s lines from Shakespeare’s “The Tempest.”
As you can see – compression beyond level 9 starts to take exponentially more time.
To achieve sane compression times for web requests, level 9 is generally recommended. Here are the file sizes, in bytes, after compression
The original file is 3443 bytes.
As you can see, using level 9 provides almost identical compression ratios to GZip’s highest and second-highest compression level while maintaining approximately the same compression time. While the file used for testing is quite small, this graph provides a good picture of the results you can expect.
To tackle this issue, our team tried and created hundreds of algorithms. Ultimately, ours came out something like this:
A grossly simplified visual with no detail, but you get the idea. And why we named it broccoli.
But wait. To understand how broccoli compression works, we have to explain how compression works in general.
Compression works to reduce the size of files by exploiting statistical redundancy in data (lossless compression) and applying transformations to data to strip less visible aspects of data (lossy). Yes. That’s copied from Wikipedia.
Our algorithm utilizes binary decision trees to decide what data can, and can’t be removed. Previously, binary decision trees weren’t an efficient solution. However, our optimizations have made it a reality.
This ultra-efficient compression method allows up to a 99% reduction in file size, and a 41x speedup over Brotli and GZip.
Here’s a comparison of Brotli, GZip, and Broccolli:
And here’s how long compression takes:
# stat shake.txt && mtime broccoli -q 39 -f shake.txt && stat shake.txt File: ‘shake.txt’ Size: 3443 Blocks: 8 IO Block: 4096 regular file Device: 805h/2053d Inode: 340452361 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Context: unconfined_u:object_r:unlabeled_t:s0 Access: 2021-03-31 21:09:37.834255800 -0500 Modify: 2021-03-31 21:09:37.834255800 -0500 Change: 2021-03-31 21:09:37.834255800 -0500 Birth: - .002131299 File: ‘shake.txt’ Size: 207 Blocks: 8 IO Block: 4096 regular file Device: 805h/2053d Inode: 340452361 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Context: unconfined_u:object_r:unlabeled_t:s0 Access: 2021-03-31 21:09:37.834255800 -0500 Modify: 2021-03-31 21:11:20.798943276 -0500 Change: 2021-03-31 21:11:20.798943276 -0500 Birth: -
That’s right. Just 14% of the maximum level of Brotli compression, and 6% of the original file. And it takes less time than all the other methods.
What is even better – since brotli and broccoli are so similar, there’s no concern for browser compatibility. Any browser supporting brotli, will automatically work with broccoli compression!
We’ve released and fully open-sourced our compression library here (highly recommend checking that out!) and will be releasing NGINX and Apache modules so everyone can take advantage of the massive performance boost.
Broccoli compression has been rolled out to every single site hosted by CynderHost. All users should see significant performance boosts.
If you’re not hosted with CynderHost and would like to massively speed up your site, we recommend asking your hosting provider when they plan on supporting Broccoli compression today!
PS: If anyone is wondering if it boosts your PSI score, it does.