Categories
Announcements Speed

Announcing Support For Broccoli Compression

Instant 100 Pagespeed Scores. 99% filesize reduction. Zero latency. #Broccoli is the future.

No, you read that correctly.

For those unfamiliar with Brotli compression – Brotli is a compression format that provides roughly a 30-50% higher compression ratio over the industry-standard GZIP. Brotli servers your website faster than ever before.

Over the past few months though, we’ve conducted hundreds of hours of testing and development. On this day, 7 years old, Google launched its ground-breaking brotli compression format. Today, we’re excited to launch what we hope will be the future compression standard of the hosting industry: Broccoli Compression.

There are two problems with the current Brotli protocol:

  1. It’s slow. Compression levels beyond 9 take milliseconds.
  2. It could be more efficient and compress content even further.

For reference, here’s the number of seconds it takes to compress some of Trinculo’s lines from Shakespeare’s “The Tempest.”

As you can see – compression beyond level 9 starts to take exponentially more time.

To achieve sane compression times for web requests, level 9 is generally recommended. Here are the file sizes, in bytes, after compression

The original file is 3443 bytes.

As you can see, using level 9 provides almost identical compression ratios to GZip’s highest and second-highest compression level while maintaining approximately the same compression time. While the file used for testing is quite small, this graph provides a good picture of the results you can expect.

To tackle this issue, our team tried and created hundreds of algorithms. Ultimately, ours came out something like this:

Our marketing team made us add some colors.

A grossly simplified visual with no detail, but you get the idea. And why we named it broccoli.

But wait. To understand how broccoli compression works, we have to explain how compression works in general.

Compression works to reduce the size of files by exploiting statistical redundancy in data (lossless compression) and applying transformations to data to strip less visible aspects of data (lossy). Yes. That’s copied from Wikipedia.

Our algorithm utilizes binary decision trees to decide what data can, and can’t be removed. Previously, binary decision trees weren’t an efficient solution. However, our optimizations have made it a reality.

This ultra-efficient compression method allows up to a 99% reduction in file size, and a 41x speedup over Brotli and GZip.

Here’s a comparison of Brotli, GZip, and Broccolli:

And here’s how long compression takes:

# stat shake.txt &&  mtime broccoli -q 39 -f shake.txt && stat shake.txt
  File: ‘shake.txt’
  Size: 3443            Blocks: 8          IO Block: 4096   regular file
Device: 805h/2053d      Inode: 340452361   Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: unconfined_u:object_r:unlabeled_t:s0
Access: 2021-03-31 21:09:37.834255800 -0500
Modify: 2021-03-31 21:09:37.834255800 -0500
Change: 2021-03-31 21:09:37.834255800 -0500
 Birth: -
.002131299
  File: ‘shake.txt’
  Size: 207             Blocks: 8          IO Block: 4096   regular file
Device: 805h/2053d      Inode: 340452361   Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: unconfined_u:object_r:unlabeled_t:s0
Access: 2021-03-31 21:09:37.834255800 -0500
Modify: 2021-03-31 21:11:20.798943276 -0500
Change: 2021-03-31 21:11:20.798943276 -0500
 Birth: -

That’s right. Just 14% of the maximum level of Brotli compression, and 6% of the original file. And it takes less time than all the other methods.

Mind-blowing, no?

What is even better – since brotli and broccoli are so similar, there’s no concern for browser compatibility. Any browser supporting brotli, will automatically work with broccoli compression!

We’ve released and fully open-sourced our compression library here (highly recommend checking that out!) and will be releasing NGINX and Apache modules so everyone can take advantage of the massive performance boost.

Broccoli compression has been rolled out to every single site hosted by CynderHost. All users should see significant performance boosts.

If you’re not hosted with CynderHost and would like to massively speed up your site, we recommend asking your hosting provider when they plan on supporting Broccoli compression today!

PS: If anyone is wondering if it boosts your PSI score, it does.

PSI scores don’t lie.

While broccoli compression isn’t a real thing, our constant drive to make websites faster is. Check out our Hosting and Caching Engine. Happy 4/1st.

Leave a Reply

Your email address will not be published. Required fields are marked *