[squid-users] Vedr.: Re: [squid-users] Compression

What about starting with what Tux 2.0 does..

It simply checks to see if the file requested exists as the same filename
with .gz at the end - and it checks the datestamps of the original and the
.gz version.

That way you can leave the compressing to a userspace program.

The way Squid would do it, would perhaps - that when it tries to retrieve
it from an origin-server - it tries to get the url.gz also..
or perhaps you should make it so you can .gz it's cache.. so when it looks
for a cache-element on it's disk-cache - if it finds a .gz version too it
will memory cache that and deliver it..

Just some thoughts..

-------------| This mail has been sent to you by: |------------
Klavs Klavsen, IT-coordinator and Systems Administrator at
Metropol Online - http://www.metropol.dk
Tlf. 33752700, Fax 33752720, Email ktk@metropol.dk

Private- Email klavs@klavsen.net - http://www.vsen.dk

--------------------[ I believe that... ]-----------------------
It is a myth that people resist change. People resist what other
people make them do, not what they themselves choose to do...
That's why companies that innovate successfully year after year
seek their peopl's ideas, let them initiate new projects and
encourage more experiments. -- Rosabeth Moss Kanter

I'm looking into this myself. I don't want to use mod_gzip
as I want to keep the load off the backend apache server,
which is already maxxed out.

Ideally I'd also like it in the squid proxy, but I was
seeing if there was another way before I start hacking squid.
The idea was to use a compression proxy as another proxy
layer in the content path [Robert Collins sent out some
good info on this a few weeks ago - see the archive]

As far as I can tell apache mod_gzip won't compress content
when apache is run as a proxy [I've tried apache 1.3.20,
mod_gzip isn't yet ready for the 2.X beta apaches yet as
far as I can tell]. Anyway, I doubt Apache in proxy mode
could perform, even if it did do the content compression
(which I can't get it to do). I posted something to
comp.infosystems.www.servers.unix about this, but nobody
has answered as yet.

So the alternatives:

1) squid : can be patched [some (very)beta patches for Content/Transfer
   encoding are available]. This is definitely the best solution.
2) Commercial solutions:
   www.vigos.com (I haven't looked at this)
   www.ehyperspace.com [software sold to them by
   remotecommunications inc who wrote mod_gzip]
   I've received a trial version of their software which I'm
   trying out this week. A bit of reverse engineering on the
   binary shows it's nothing more than a simple gzip compression
   proxy compiled with gcc - though you'd think it was something
   a lot more special if you look at their website ;-)
3) Rustle up my own gzip proxy. I've already done this, but
   don't have the time to 'harden' it for production as
   much as I would like [actually, it's just painfully
   tedious writing accurate/compliant parsing code for http].
   It's pretty simple - just using zlib and some basic proxy
   select/pthread code to make it shift.

I suspect to get this working I'll either have to invest some
serious hacking time in either my compression proxy or coming
up with a hardened squid patch (unless Robert/anyone else has
progressed any since the earlier thread). It does make a lot
more sense to hack it into squid.

It's not my primary project at the moment. But if you've got
more time than I have, It'd be good to hear what you come up with.

cheers,
Mike

>
> Hi there...
>
> With Apache web server, you can add the gzip module which compresses
> data on the fly to the visitor cutting down dramatically on bandwidth
> consumed. All current browsers support "on the fly" Gzip compression
> and it works extremely well.
>
> Is there a way that content served via the cache could be gzipped on the
> fly? Perhaps it's already possible and I am not aware of it? My
> opinion would be that this is a win/win scenario with the cache speeding
> up delivery of content to the browser, saving bandwidth on the backbone
> connections, and also delivering the content even faster to the browser
> (especially a dial-up user).
>
> Thanks,
>
> Paul Stewart

--
Michael Kiernan
Onet.pl S.A.
Krakow, Poland
http://www.onet.pl/