[squid-users] Caching downloaded files from dynamic pages

From: Alexander K <alexanderk@dont-contact.us>
Date: Sun, 23 Sep 2007 23:27:39 +0300

I'm working in PC Repair service company. We are of course loading often
drivers, utilities, updates etc from manufacturer webpages (Acer, Lenovo,
Fujitsu-Siemens, etc).

Since we are on limited connection and we want to speed up downloads, we
want to use squid to cache those files. So you need to download each file
only once. We are building up a server for this so CPU/RAM/DISK won't be a
problem.

I have had a quick test with Squid and have found it working well.

However, I am having problem with manufacturers, which does not have
hard-coded http/ftp url, but dynamic pages which are "streaming" file to the
browser.

One problematic is Fujitsu-Siemens.

When I am downloading drivers from Lenovo, they work fine, since they are
using "normal" urls, for example:
http://download.boulder.ibm.com/ibmdl/pub/pc/pccbbs/mobiles/1rg807ww.exe

But I am unable to cache downloaded drivers and software from
Fujitsu-Siemens pages.

For example, when downloading this file:

http://support.fujitsu-siemens.com/Download/ShowDescription.asp?SoftwareGUID
=50AD6EEC-53F0-4B6E-9C13-53E2CB51D36B&OSID=DD13C337-8EFF-4CFB-A589-72971D7BC
BCE&Status=True&Component=Flash%20Bios%20for%20AMILO%20Pro%20V8210

It will show up in squid access.log like this:

--
1190578174.875   1422 192.168.11.5 TCP_MISS/200 22284 GET
http://support.fujitsu-siemens.com/Download/ShowDescription.asp? -
DIRECT/80.70.172.14 text/html
1190578176.921    765 192.168.11.5 TCP_MISS/302 646 POST
http://support.fujitsu-siemens.com/Download/Download.asp -
DIRECT/80.70.172.14 text/html
1190578177.093    172 192.168.11.5 TCP_MISS/302 1090 GET
http://support.fujitsu-siemens.com/Download/StreamFileToBrowser.asp? -
DIRECT/80.70.172.14 text/html
1190578184.078   6985 192.168.11.5 TCP_MISS/200 1263991 GET
http://support.fujitsu-siemens.com/download/FileDownload/fileDownload.aspx?
- DIRECT/80.70.172.14 application/octet-stream
--
I cannot figure out how to tell Squid to cache that content. :( Is that
possible at all?
Happy for assistance,
Alexander
Received on Sun Sep 23 2007 - 14:22:14 MDT

This archive was generated by hypermail pre-2.1.9 : Mon Oct 01 2007 - 12:00:02 MDT