How to GZip Drupal 6.x's aggregated CSS and JS files

compression ratio illustration
Cut the size in half

Why

I was tired of waiting for this to happen. It's simple stuff and the effect is pretty big if your content primarily consists of text. Well, it's not like this affects the text - that's already compressed by Drupal if you allow it to do so - what I mean is the compression ratio. If you serve megabytes of images, sound, and video, shrinking those few percent of JS and CSS won't have much of an effect.

However, if your site is more like this one you might be able to cut the size of the site in half. For people with a non-primed cache, that is. That's a really big plus if you get a sudden surge of visits. Nowadays the typical resource usage pattern seems to go from one spike to the next with rather little usage in-between. Take a look a Esoteric Curio's excellent article if you're interested in some more details.

As I previously pointed out even your regular visitors often won't have a primed cache, because the cache is simply too small for today's bloaty web. By the time they visit your site again the cache was already overwritten most of the time. Server-sided caching and compression to the rescue!

Drupal already handles caching of pages and blocks, which already helps a lot. It's also able to utilize GZip compression for the markup, which again is cached on the server side. CSS and JS files can be also aggregated, which reduces the number of HTTP requests, which in turn also improves loading times.

However, the next logical step - compressing those aggregated CSS and JS files - was omitted. Fortunately adding this feature is very easy if you don't ever want to turn it off. Well, why would you? :)

How

Modify includes/common.inc

Step 1 - Locate the following line (search for "file_save"):

file_save_data($data, $csspath .'/'. $filename, FILE_EXISTS_REPLACE);

Step 2 - Put the following line below it:

file_save_data(gzencode($data,9), $csspath .'/'. $filename . '.gz', FILE_EXISTS_REPLACE);

Step 3 - Locate the following line (search for "file_save"):

file_save_data($contents, $jspath .'/'. $filename, FILE_EXISTS_REPLACE);

Step 4 - Put the following line below it:

file_save_data(gzencode($contents,9), $jspath .'/'. $filename .'.gz', FILE_EXISTS_REPLACE);

The second parameter of gzencode is the compression level. 9 offers the best compression, but it's also the slowest. Since compression happens very rarely using a level of 9 is alright. A level of 7 or even 5 would be almost as good, while only taking a fraction of the time. But as I said, it literally runs only once in a blue moon - going for the extreme is really alright.

Keep in mind to add those two lines again after upgrading Drupal.

Modify .htaccess

Add the following before the "<IfModule mod_rewrite.c>" block:

<Files *.js.gz>
  AddEncoding gzip .js
  ForceType application/x-javascript
</Files>

<Files *.css.gz>
  AddEncoding gzip .css
  ForceType text/css
</Files>

Insert the following at the very end of the "<IfModule mod_rewrite.c>" block (before "</IfModule>"):

RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)\.css $1.css.gz [L,QSA]

RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)\.js $1.js.gz [L,QSA]

mod_negotiate could have been used instead of mod_rewrite, but the former looked a bit less feasible from my point of view. Well, feel free to experiment with that stuff.

Results

On my rather plain website the results are pretty impressive as illustrated by figure 1:

Compression ratio diagram
Figure 1: Compression ratio

The document, external document, images, and the external JavaScript were already heavily compressed. Also compressing the aggregated JavaScript and CSS files saves about 28% on the front page and a whopping 44% on that specific blog entry.

This blog post uses about 30kb for additional images, which means that compression of those aggregated files saves about 34%. That's still pretty awesome, isn't it?

Update 07/10/2008: I forgot a backslash in both rewrite rules. It's now fixed in the .htaccess excerpt above. This mistake caused paths like "sh_css.min.js" to be rewritten as "sh.css.gz" instead of "sh_css.min.js.gz". Whoops. My thanks go to some random guy/gal from the Czech Republic who made it show up in the logs. :)

Comments

Awesome

This works perfect, thanks for sharing!

don't hack core

This is a lot of work when you can simply enable gzip compression in Apache.

Not always an option

With mod_deflate the content will be compressed in real time. Over and over again, which of course results in higher CPU usage. The method I'm using does that only once whenever it's necessary. The result is stored on disc and the same static file is served a zillion times.

It's a very nice solution, which really helps a lot on cheap shared hosts (such as this one).

Hacking the core is of course not desirable, but in this case the change is very simple and I'm willing to redo that as often as necessary. Drupal will have this feature in the future, but if it takes too long I might be forced to switch hosts sooner than actually necessary.

That's why I decided to do it. It also serves as a nice example of the benefits and it's also simple enough to quickly evaluate the effects on your own; everyone can do it in a couple of minutes. Oh and the diagram also makes the benefits crystal-clear.

Have an answer for 5.x?

I still am using 5.7, I would REALLY appreciate a mod for 5.7. I tried just doing this mod, but it did not all work. The files we're gzipped, but I can't figure out how to get it read it. The end result is my page without my css! Could you come up with a mod?

Hm

5.x doesn't seem to aggregate JS files, but it does aggregate CSS files at least.

Well, you shouldn't see unstyled pages ever. Therefore I'll explain briefly how it's supposed to work:

Whenever CSS or JS files are aggregated they are written to disc. This part is left unchanged. We'll still need the uncompressed files for those clients which don't support GZip compression. However, this is also the right spot to create a gzipped version for more modern clients. That's why we'll create a gzipped copy right below it.

So, now (after clearing the cached data and triggering a rebuild by requesting some pages) we should be able to see one or more aggregate files together with their gzipped copies over in the files/css or files/js directory (or wherever you told Drupal to store those files).

The actual logic is handled by .htaccess. If a CSS or JS file is requested, the request will be internally redirected to the gzipped copy. But that only happens if both rewrite conditions are met:

  1. the client accepts gzipped content (as indicated by the Accept-Encoding header) and
  2. a gzipped copy actually exists

I forgot...

You can aggregate JS files in 5.x with the JavaScript Aggregator module.

Patch

A patch would be awesome, to maintain Drupal files after upgrades! -- Gurpartap Singh

Blank page

drupal 6.3 I get a blank page after making the above modifications

re: Blank page

Sorry for the slightly late reply. This site currently runs Drupal 6.3 and it works fine. There weren't any changes from 6.0 to 6.3 which could have broken this hack. After all it only creates a gzipped copy of those aggregated files whenever they are created.

I suggest to check your .htaccess. Re-read my instructions carefully and ensure that you really didn't do anything wrong there.

Edit: Also works with 6.4 ;)

Fantastic! works in 6.6

My site is hosted by GoDaddy (shared-Linux). After applying your code my pages literally flew.
- admin/settings/performance: normal caching, page compression off (will experiment with on), optimized CSS/JS

verified: page compression disabled seems much better than enabled (YSlow), very newbie dev, no savvy on making hi-tech measurements for the moment

Thanks a lot for sharing the expertise.
Edwin

Hi, Thanks for sharing

Hi,

Thanks for sharing this.

I have done what you said. However, I installed drupal CMS at subfolder (http://example.com/drupal/) and your solution did not work.

Do I need to add/ change some things to let your code works on my drupal site?

re: Hi, Thanks for sharing

>[...] your solution did not work. Do I need to add/ change some things to let your code works on my drupal site?

How should I know? :)

My local test installs reside within sub-dirs and it does work fine there.

Did you verify that your PHP version comes with zlib support? Does .htaccess work? Are there any error messages? Where there any JS/CSS files aggregated after you added this stuff? (That's the only time those 2 lines are executed) Did you check if there were any *.gz files inside that temp dir? Did you check for compression as anonymous user? (You won't get the cached stuff otherwise.)

Hi Jh, I verified that the

Hi Jh,

I verified that the PHP version comes with zlib support, .htaccess works, CSS/JS files aggregated after I added this stuff resides in mysite/drupal/sites/default/files/css(js).

I did not see any *.gz files inside "mysite/drupal".

I use drupal 6.6 and the same theme as you are using (garland). After I have enabled "aggregating css and js files" options, I got 2 css files with one has 25.6 kb and the other has 876 bytes. I then applied your trick and I saw that 2 css files did not decrease in size. They are still 25.6 kb and 876 bytes.

I post two files are common.inc and .htaccess as below and I hope you could help me to check if I put your codes in wrong place.

common.inc file:
....
// Create the CSS file.
file_save_data($data, $csspath .'/'. $filename, FILE_EXISTS_REPLACE);
file_save_data(gzencode($data,9), $csspath .'/'. $filename . '.gz', FILE_EXISTS_REPLACE);
}
return $csspath .'/'. $filename;
}
....
// Create the JS file.
file_save_data($contents, $jspath .'/'. $filename, FILE_EXISTS_REPLACE);
file_save_data(gzencode($contents,9), $jspath .'/'. $filename .'.gz', FILE_EXISTS_REPLACE);
}

return $jspath .'/'. $filename;
}

.htaccess file:

# PHP 5, Apache 1 and 2.

php_value magic_quotes_gpc 0
php_value register_globals 0
php_value session.auto_start 0
php_value mbstring.http_input pass
php_value mbstring.http_output pass
php_value mbstring.encoding_translation 0

# Requires mod_expires to be enabled.

# Enable expirations.
ExpiresActive On

# Cache all files for 2 weeks after access (A).
ExpiresDefault A1209600

# Do not cache dynamically generated pages.
ExpiresByType text/html A1

# Various rewrite rules.

AddEncoding gzip .js
ForceType application/x-javascript


AddEncoding gzip .css
ForceType text/css

RewriteEngine on
# If your site can be accessed both with and without the 'www.' prefix, you
# can use one of the following settings to redirect users to your preferred
# URL, either WITH or WITHOUT the 'www.' prefix. Choose ONLY one option:
#
# To redirect all users to access the site WITH the 'www.' prefix,
# adapt and uncomment the following:
# RewriteCond %{HTTP_HOST} ^.....\.com$ [NC]
#
# To redirect all users to access the site WITHOUT the 'www.' prefix,
# uncomment and adapt the following:
# RewriteCond %{HTTP_HOST} ^www\.....\.com$ [NC]
# RewriteRule ^(.*)$ ..../$1 [L,R=301]
#
#
# Modify the RewriteBase if you are using Drupal in a subdirectory or in a
# VirtualDocumentRoot and the rewrite rules are not working properly.
# For example if your site is at http://example.com/drupal uncomment and
# modify the following line:
# RewriteBase /drupal
#
# If your site is running in a VirtualDocumentRoot at.....,
# uncomment the following line:
# RewriteBase /
# Rewrite URLs of the form 'x' to the form 'index.php?q=x'.
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)\.css $1.css.gz [L,QSA]

RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)\.js $1.js.gz [L,QSA]

# $Id: .htaccess,v 1.90.2.1 2008/07/08 09:33:14 goba Exp $

Thank you very much for reading my questions.

I'm looking forward your response

Li.

P.S.
In case you want to know my email for contacting, please use this email address: licoi123[that-a]yahoo[that-dot]com. My name is Li. Thank you.

re: Hi Jh, I verified that the

>After I have enabled "aggregating css and js files" options, I got 2 css files with one has 25.6 kb and the other has 876
>bytes. I then applied your trick and[...]

Ye, well... that's the reason. JS and CSS files are only aggregated if necessary. Since those very files are already aggregated those aggregate methods (and thus the gzipping code) won't be executed. Note that this is the desired behavior. GZipping at level 9 is somewhat slow, but if it's only done once in a blue moon (if there were any changes, that is) everything is alright.

So, if you want the interpreter to enter this branch where the compression happens you can for example change some of the JS/CSS files. You can also disable aggregation, clear the cache, delete the aggregated files, enable aggregation again, and then hit the site as anonymous user. However, using the latter option may result in a few 404s from users who take a look at search-engine-sided cached pages (since those specific aggregated files will be gone then). In order to avoid this hypothetical non-issue you can temporarily rename those aggregated files instead of deleting them.

HTH

Doing this for JS on version 5

It's possible to enable this on drupal 5 with the javascript_aggregator module, by adding the required line inside the javascript_aggregator.module. Works a treat, thanks.

Please help to make it work for Drupal 5

Hi,

I tried this for Drupal 5, but it didn't work.

I inserted only file_save_data(gzencode($data,9), $csspath .'/'. $filename . '.gz', FILE_EXISTS_REPLACE); in common.inc (because there's no file_save for javasctrips in Drupal 5.

Then I inserted the required lines in htaccess file. (But I already had quite a lot of custom code there from previous days).

The result was that nothing really changed. I go to /files/css and there are no gzip files there.

Could you possibly e-mail me your version of Drupal 5 htaccess on kronstadt.films[the-@]gmail.com ??

If I could get this to work, I would be eternally gratefull -- I've spent months and months trying to find a solution to this (see for example http://drupal.org/node/248909).

Many many thanks

Drupalina

re: Please help to make it work for Drupal 5

>I go to /files/css and there are no gzip files there.

As I explained earlier, compressed versions won't be created without a reason. You see... the aggregation stuff isn't executed all the time. Add a comment to your main style sheet, hit the page as anon user (press ctrl+f5 for being sure), and then check the directory if there is a new *.css and *.css.gz file.

Trying to do this in Drupal 6.8

I made and double checked the changes in this article, checked to make sure the files were being created (they were), and cleared my cache. However, my pages are not loading my js and css files when I make these changes. Any ideas?

re: Trying to do this in Drupal 6.8

So, the usual aggregated files and gzipped versions of them are there? And you get unstyled content? That's super weird. :)

Also checked as anon user? Can you access the files over http?

JavaScript Aggregator

The JavaScript Aggregator module for both Drupal 5 and 6 are attempting to do this. So far it's only in the development release, but will make it into the official releases once it's perfect. Thanks a lot for the help on this.... Testing in GZip compression would be appreciated.

--
Rob Loach
http://www.robloach.net

not working?

I did what is said by the author of this blog.. and it does gzip my js/css files... (aggregate enabled in drupal) the only thing is.. it doesn't load them.. nor does it load the uncompressed but aggregated js/css..

so I get style-less and js-less output...

what can be the reason it doesn't load those files??

it doesnt work follow up

The strange thing is when I look at the source...

so it is trying to load the not gzipped... so you would think it would show at least my page with css... alas it does not...

the compressed and uncompressed are both there btw

Re: not working?

Ensure that you followed my instructions correctly.

And yes, the pages will keep referencing the plain files. If a browser requests a file from a server it will also tell the server what kind of content encodings it can deal with (if any). On the server side we take a look at the Accept-Encoding header and act accordingly (this behavior is controlled with that .htaccess file). If the browser accepts gzipped content, we'll send the compressed stuff along with a matching Content-Encoding header – otherwise the plain files are sent without that Content-Encoding header.

2 Dev Modules are now

2 Dev Modules are now available to do this
http://drupal.org/project/javascript_aggregator
http://drupal.org/project/css_gzip

-Mikeytown2

re: 2 Dev Modules are now

Awesome! :)

Won't Load .gz File

I've made sure I did everything like you said, but right now in my sites/default/files/css(js) folders I'm getting both the .css and then the .css.gz file. The site just loads the regular css but not the css.gz file. I know my server accepts gz extensions so I don't think thats the issue, but do you have any tips for things to try?

I read your comment about Content-Encoding headers but I'm not really sure if thats the issue, or even how to change it if it is.

Any tips on things I can try to make drupal pull the gz file?

Thanks a lot, I'm hoping I'm almost there! :)

re: Won't Load .gz File

Logout and clear your cache. If the .htaccess file is in place it should work as intended.

Post new comment

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options