DJ’s Weblog

Well, a little evening hack while watching Inspector Morse has produced a minimalist script wget.pl - a tiny wrapper around the wget command so that you can be more polite when retrieving HTTP based information - in particular RSS feeds.

The idea was sparked by Simon’s post about using HTTP 1.1’s ETag and If-None-Match headers. I wanted to write as small and minimal a script as possible, and rely on as little as possible (hence the cramped code style), in honour of Blosxom, and of course Blagg, the RSS aggregator, for which the script was designed. You should be able to drop this script reference into Blagg by specifying the RSS retrieval program like this:

my $get_prog = '/path/to/wget.pl';

Don’t forget, the ETag advantage is only to be had from static files served by HTTP. Information generated on the fly, such as that from CGI scripts (such as Blosxom) aren’t given ETags.