[200 OK]: A Port80 Software Blog

We're all 200 OK: Web, HTTP and IIS Insights
posts - 203, comments - 424, trackbacks - 100

Compression basics for software developers

All modern browsers can decompress gzipped or deflate-encoded content, what we all mean when we speak about HTTP compression.  But what if you are building the browser or a similar type of application yourself?  If you are building a custom client application to handle decompression where a browser would traditionally do the chore, this post helps to lay the ground for how compression works between a Web client and server.

If your client program is sending the Accept-Encoding HTTP header, then httpZip or ZipEnable managing IIS 6 native compression (or any other compression solution) should be responding with the first compression scheme specified by that header’s value.  So, in other words, if the client is sending:

Accept-Encoding: gzip

Or

Accept-Encoding: gzip, deflate

Then the server running httpZip should be responding with gzip-encoded data (which would be accompanied by the HTTP header “Content-Encoding: gzip”).  If, on the other hand, the client is sending:

Accept-Encoding: deflate

Or

Accept-Encoding: deflate, gzip

Then the response from the httpZip-enabled server should be deflate-encoded data (which would be signaled by the HTTP header “Content-Encoding: deflate”).

Both the “deflate” and the “gzip” encodings are done using the standard zlib library (http://www.zlib.net/ ).  There is often some confusion about these terms, due mostly to the loose usage of “deflate.”  What is called “deflate” in the HTTP headers is actually the raw deflate compression format specified in RFC1951 (http://www.faqs.org/rfcs/rfc1951.html ) *plus* the zlib headers and checksums specified in RFC1950 (http://www.faqs.org/rfcs/rfc1950.html ). 

Similarly, what is called “gzip” in the HTTP headers is the raw deflate of RFC1951 plus the gzip headers and checksums specified in RFC1952 (http://www.faqs.org/rfcs/rfc1952.html ).  So, in both cases, you are dealing with the basic deflated stream, wrapped in one or another type of header-and-checksum scheme.

How do you handle the decompression part in your futuristic, ultra-performing Web app?  These guys at Exceed have a nice decompression library that you can purchase: http://www.xceedsoft.com/.

Cheers,
Port80

posted on Monday, October 31, 2005 4:30 PM

Feedback

# re: Compression basics for software developers

Thanks for this wonderful blog. I was definitely informed and I would wish some other tips from you. I am a developer but still working on my basics. I wish to make some great sites when i am done with these learning process.
11/22/2005 11:44 PM | Online Wong PoKér Hu

# re: Compression basics for software developers

Hi
I've done some experiments with IIS 5.0 and a simple program that I wrote that imports the zlib library. It seems that data received from server in deflate encoding - is rejected by the zlib library - due to the fact the header of data is wrong. I found a similar comment on the web but no ansers at the mean time. got a clue?
Thanks
Amichai
5/7/2008 2:06 AM | Amichai

# re: Compression basics for software developers

You should really not try to code compression yourself -- there are 1000s of things that can go wrong, and it is best to rely on solidly tested, browser-compatible products like www.httpzip.com or www.zipenable.com.

Cheers,
Chris @ Port80
5/19/2008 2:53 PM | Chris @ Port80

Post Comment

Title:  
Name:  
Url:  
Comment:  
Verify:
(Enter the word as it appears in the box above.)