Re: Feature: quota control

On 03/30/2009 01:00 PM, Pieter De Wit wrote:
> On Mon, 30 Mar 2009 11:38:31 -0600, Alex Rousskov
> <rousskov_at_measurement-factory.com> wrote:
>> On 03/30/2009 03:16 AM, Pieter De Wit wrote:
>>
>>> I gave this some thought. Why don't we build a system close to
>>> sendmail's milter system. An API where by clients can plug in and offer
>>> services - one can be a traffic counter, traffic limiter (as what we
>>> proposing here) and maybe a URL block, a virus scanner etc.

>> We already have such a plugin API for message- and content-related
>> tasks. It is called eCAP, and it has been mentioned on this thread.
>>
>> Compared to eCAP, traffic shaping and quotas are different in scope as
>> they work on multiple messages and often do not care about the messages
>> at all (just raw bytes traffic). So, we have a few options:
>>
>> 1) Use eCAP nearly "as is" for traffic shaping and quotas, even though
>> it is not a perfect fit for the task.
>>
>> 2) Significantly enhance eCAP to offer traffic shaping-specific hooks
>> (as a standard addition or as a Squid extension), even though it may
>> lead to eCAP API bloat.
>>
>> 3) Develop a different plugin API that specializes in aggregate traffic
>> management and is unrelated to eCAP, even though it may lead to
>> duplicating a lot of eCAP-related code.

> Client/Bandwidth limiting registers with squid
> ...
> squid get a request to download object x from site y by user z at ip
> a.b.c.d etc
> squid sends that request to the client (just "text" not the object)
> client replies "yes" or "no" - if yes the client needs to track how much
> data is left for that user etc.
>
> (Client here is the limiting software - couldn't think of a better
> name...coffee....)
>
> I feel this should be a persistent connection that is matched by an acl for
> speed etc.

I doubt the traffic shaping API can be that simple and still cover most
useful cases: the size of the object is not always known a priori; some
traffic shaping decisions need more than user ID, client IP, and a URL;
there are several points at which traffic shaping may take place; etc.

I also suspect it would be better to avoid mandating a connection-based
interface for this API. A loadable module can use a connection-based
approach to communicate with the actual traffic shaper, but the reverse
is not possible: once you standardize on the connection-based API, you
cannot do embedded modules (which are usually much more efficient). We
probably would not invent eCAP if ICAP (which uses TCP connections) and
various helpers were good enough from that point of view.

> Now to the workings of this. I am leaning towards option 3. I am not sure
> how hard it is to maintain "two" API's. To be honest, I am not even sure
> how hard it is to maintain one API. I feel that this really only needs two
> or three commands, so is it really going to bloat the API ?

You need three commands. The person next to you needs another five. By
the end of the day, you end up with a few dozen commands OR several
distinct APIs. Just look at how many different "helpers" Squid already
has and how easy/difficult it is to extend them.

However, when I was talking about API bloat, I was not talking about the
number of "commands" in the new API. I was talking about bloating the
existing API (eCAP) with traffic shaping "commands".

> Broken down - what is the most "this API" would need ?

We probably should define the scope before counting the "commands":
Inbound, outbound, and peer traffic shaping. Shaping based on group
membership, user credentials, IP addresses, TCP connection IDs, MIME
types, HTTP headers. Shaping during message download. Reconfiguration of
quotas. Etc., etc.

Cheers,

Alex.
Received on Mon Mar 30 2009 - 20:44:25 MDT