Posts Tagged minify

How to Get a Directory Listing in C (POSIX)

Jun 7

Posted by Scott in Programming

For the minification scripts mentioned here, I had to write a program that would go through every file of a certain type in a directory and process (in this case, minify) that file. At the heart of this problem was the need to get a directory listing.

The obvious way to get a directory listing in C is to use the system call, as in system("ls");. But this is not only lame, it does not work on all systems, and the result cannot be used very easily. The better way to get a directory listing is to use system calls to the filesystem. A complete example is shown at the bottom of this post, as usual.

The functions used in getting a directory listing are opendir and readdir (note: these functions also exist in php, and the code to get a directory listing is in fact nearly the same in php and c). opendir() can be seen as kind of like a fopen() for directories. Following this analogy, readdir() is like a fgetc() for directories. To get a directory listing, we first open the directory with opendir("/dir/name/here"), and then read the name of each file in the directory with a loop of readdir(directory_handle)s. Read the rest of this entry »

Tags: , , , , , , , , ,

No Comments

Minifying CSS and JS

May 27

Posted by Scott in Programming, Website Development

Many web developers want to make use of large, flashy Javascript libraries that allow fancy effects. (Most such libraries also come with large CSS files). Team 449’s website uses Prototype (an AJAX library) and Scriptaculous (a JS effects library built on top of prototype). While many or most viewers of the website may enjoy the experience, others, who have slower connections or are farther away or have the bad luck to view the website during peak viewing time, get frustrated with the long load time.

The solution to this is twofold. First, install gzipping on the server – I’ll discuss this in a later entry. Second, use a minification program, such as the one Yahoo provides. These programs take the JS or CSS files and remove unnecessary whitespace and comments to decrease the file size. The end result – the load time is often cut in half or better.

Yahoo’s compressor must be called from the command line and told to actually compress the file. Naturally, most people will want to automate this process, so that they don’t have to remember to call the compressor everytime they install a new version of scriptaculous or prototype, or make a change.

One possibility is to set up a serving script that first calls the compressor and then serves the page. This can be done in the htaccess like so:

Options +FollowSymlinks
RewriteEngine on

RewriteRule ^js/(.*)$ jserve.php/$1 [L]
RewriteRule ^style/(.*)$ cserve.php/$1 [L]

This is grossly inefficient. For a medium or high-traffic website, it will completely kill your server. The better solution (and the one we used at the Blair Robot Project) is to have the compressor be called in your crontab, and then have the serving script serve the minified file. The crontab entry would then look like:

*/30 * * * * /var/www/robot/scripts/manage-roboweb.sh

The above entry runs manage-roboweb.sh every 30 minutes. In manage-roboweb.sh, we have a call to a c minification program that automatically finds every file in a specified directory (in this case /var/www/robot/js) that has a specified extension, and processes that file with a call to YUI (the Yahoo compression application).

This can be improved by having the serving script first check to see if the minified file is up-to-date, by checking the last modified times of each. You can also have the managing script do the same, so that the files are only re-minified if they were modified. I’ve placed the code we used for both serving scripts below.

cserve.php

  1. function minify_version($fn) {
  2.   $min_fn = str_replace(".css", ".min.css", $fn);
  3.   $min_stat = stat($min_fn);
  4.   $norm_stat = stat($fn);
  5.   return $min_stat && $min_stat['mtime'] >= $norm_stat['mtime']
  6.     ? $min_fn
  7.     : $fn
  8.     ;
  9. }
  10. if (substr_count($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip')) {
  11.      ob_start("ob_gzhandler");
  12. } else {
  13.   ob_start();
  14. }
  15. $offset=1000*3600*48;//48 hour cache
  16. header("Expires: ".gmdate("D, d M Y H:i:s",time()+$offset)." GMT");
  17. header("Cache-Control: max-age=$offset, must-revalidate");
  18. $gmdate_mod = gmdate('D, d M Y H:i:s', time()) . ' GMT';
  19. header("Last-Modified: $gmdate_mod");
  20. header("Pragma: public");
  21. header("Content-Type: text/css");
  22. $url=$_SERVER["REQUEST_URI"];
  23. $fn=minify_version("/var/www/robot/style/" .
  24.         str_replace("/style/","",$url));
  25. $file=file_get_contents($fn);
  26. echo $file;

jserve.php

  1. if (substr_count($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip')) {
  2.      ob_start("ob_gzhandler");
  3. } else {
  4.   ob_start();
  5. }
  6.  
  7. function minify_version($fn) {
  8.   $min_fn = str_replace(".js", ".min.js", $fn);
  9.   $min_stat = stat($min_fn);
  10.   $norm_stat = stat($fn);
  11.   return $min_stat && $min_stat['mtime'] >= $norm_stat['mtime']
  12.     ? $min_fn
  13.     : $fn;
  14. }
  15. $offset=1000*3600*48;//48 hour cache
  16. header("Expires: ".gmdate("D, d M Y H:i:s",time()+$offset)." GMT");
  17. header("Cache-Control: max-age=$offset, must-revalidate");
  18. $gmdate_mod = gmdate('D, d M Y H:i:s', time()) . ' GMT';
  19. header("Last-Modified: $gmdate_mod");
  20. header("Pragma: public");
  21. header("Content-Type: text/javascript");
  22. $url=$_SERVER["REQUEST_URI"];
  23.  
  24. $fn = minify_version("/var/www/robot/js/" .
  25.         str_replace("/js/", "", $url));
  26.  
  27. $contents=file_get_contents($fn);
  28.  
  29. echo $contents;

Tags: , , , , , , , , , ,

3 Comments