Re: Storing of information

<snip>
> * the parsing bottleneck gets crunched several times: on first
> arrival, in the ICAP server, and on return to Squid,
> * the ICAP server bypass optimization can't be used since quote needs
> to measure every byte,
> * tunneled data does not get sent to ICAP services,
>
> Not exactly perfect service, but it offers the most complete quota
> control without adding complexity to Squid.
>
> eCAP might be a slightly better. It sill runs inside Squid and has
> some processing overhead, but should reduce the parse problems and
> network delays involved with ICAP.
>
>>
>> Points to reading URL's are more than welcome, also, so is examples
>> of libicapapi :)
>
> Hopefully someone else knows some then, because I dont :(
>
> Amos
Hi Amos,

You said that you proposed some work a while ago, would you mind sharing
that? I gave the network thing some thoughts and I can see how the delay
would hurt squid. I kept on comparing it to milters, but these don't
mind a few ms delay, email is a lot less interactive.

The thought process I am going with is something along the lines of a
process that is "spoken" to, like ecap perhaps, via pipes or a lib or
some such. This process will be notified based on the following:

(* - Request, **-Reply)

* I would like to go to protocol://site
** Is there quota left to allow this, if the user has 0 quota left,
block the request, no use
* The server said the object is X bytes long, can I continue to download it
** Yes, there is quota. The problem comes in if the server didn't give a
length, if that is the case, perhaps only allow 1024 bytes until his
quota runs out. There is also the problem if the server said the object
is bigger than it really is...
* Can I sent the following 1024 bytes
** Yes, there is quota.

At any given step, if the quota runs out, the connect is aborted. This
will involve some tie in with the FD struct that you guys have already.
I do recall myself and Alex having a chat about this. I referred to it
as "hooks" into the FD struct. I *think* the talk about "hooks" in the
FD struct was aborted because it didn't add enough value at the time, or
real life caught up to me or or or :)

Based on this, I would like to re-float the idea of "hooks" in the FD
struct. From the top of my head, one would have modules that expose
certain function/procedures:

OnClientConnect (source_ip,source_port,target_ip,target_port);
OnClientRequest (URL);
OnClientRequestContent (content,size,offset);
OnClientResponse (URL,size);
OnClientResponseContent (content,size,offset);
OnClientDisconnect (<not sure>);

I will outright say, I have no clue how modules work (thinking about
apache etc) and these are shamelessly based on my Delphi XP with Objects.

Cheers,

Pieter

P.S. Might be worth starting a new thread perhaps ?
Received on Sun Feb 12 2012 - 04:05:51 MST