Re: Feature: quota control

On 03/30/2009 03:16 AM, Pieter De Wit wrote:

> I gave this some thought. Why don't we build a system close to
> sendmail's milter system. An API where by clients can plug in and offer
> services - one can be a traffic counter, traffic limiter (as what we
> proposing here) and maybe a URL block, a virus scanner etc.

We already have such a plugin API for message- and content-related
tasks. It is called eCAP, and it has been mentioned on this thread.

Compared to eCAP, traffic shaping and quotas are different in scope as
they work on multiple messages and often do not care about the messages
at all (just raw bytes traffic). So, we have a few options:

1) Use eCAP nearly "as is" for traffic shaping and quotas, even though
it is not a perfect fit for the task.

2) Significantly enhance eCAP to offer traffic shaping-specific hooks
(as a standard addition or as a Squid extension), even though it may
lead to eCAP API bloat.

3) Develop a different plugin API that specializes in aggregate traffic
management and is unrelated to eCAP, even though it may lead to
duplicating a lot of eCAP-related code.

I am not sure, but I think option #2 is the worst. What do you think?

Cheers,

Alex.

> ----- Original Message ----- From: "Adrian Chadd" <adrian_at_squid-cache.org>
> To: "Amos Jeffries" <squid3_at_treenet.co.nz>
> Cc: "Robert Collins" <robertc_at_squid-cache.org>; "Mark Nottingham"
> <mnot_at_yahoo-inc.com>; "Pieter De Wit" <pieter_at_insync.za.net>;
> <squid-dev_at_squid-cache.org>
> Sent: Saturday, March 28, 2009 5:31 PM
> Subject: Re: Feature: quota control
>
>
> Just to add to this - implementing it as a delay pool inside Squid
> flattens traffic into one byte pool. Various places may not do this at
> all - there may be "free" versus "non-free" (which means one set of
> ACLs inside Squid); there may be "cheap" versus "expensive" (again,
> possibly requiring multiple delay pools and multiple ACLs to map it
> all together; again all inside Squid) - things get very messy, very
> quickly.
>
> This is why my proposal (which I hope -finally- gets approved so I can
> begin work on it ASAP! :) involves passing off the traffic assignment
> to an external daemon that implements -all- of the traffic assignment
> and accounting logic. Squid will then just send requests for traffic
> and interim updates like you've said.
>
> 2c,
>
>
> Adrian
>
> 2009/2/26 Amos Jeffries <squid3_at_treenet.co.nz>:
>> Robert Collins wrote:
>>>
>>> On Fri, 2009-02-27 at 10:00 +1100, Mark Nottingham wrote:
>>>>
>>>> Honestly, if I wanted to do byte-based quotas today, I'd have an
>>>> external ACL helper talking to an external logging helper; that way,
>>>> you
>>>> can just log the response sizes to a daemon and then another daemon
>>>> would
>>>> use that information to make a decision at access time. The only even
>>>> mildly hard part about this is sharing state between the daemons,
>>>> but if
>>>> you don't need the decisions to be real-time, it's not that bad
>>>> (especially
>>>> considering that in any serious deployment, you'll have state issues
>>>> between multiple boxes anyway).
>>>
>>> Sure; I think that would fit with 'ensuring enough hooks' :P
>>>
>>> -Rob
>>
>> The brief description of what I gave Pieter to start with was:
>>
>> A pool based on DelayPools in that Squid decrements live as traffic goes
>> through. With a helper/ACL hook to retrieve the initial pool size and to
>> call as needed to check for current quotas.
>>
>> How the helper operates is not relevant to Squid. Thats important.
>>
>> The key things being that; its always called for new visitors to
>> assign the
>> start quota, and when the quota is nearing empty its called again to
>> see if
>> they get more.
>>
>> Helper would need to send back "UNITS AMOUNT MINIMUM" where UNITS is the
>> unit of quota (seconds, bytes, requests, misses?, other?), AMOUNT being a
>> integer count of units the client is allowed to use, and MINIMUM is the
>> level of units where the helper is to be asked for an update.
>>
>> 0 remaining units results in an Error page 'quota exceeded' or somesuch.
>>
>> Amos
>> --
>> Please be using
>> Current Stable Squid 2.7.STABLE6 or 3.0.STABLE13
>> Current Beta Squid 3.1.0.5
>>
>>
Received on Mon Mar 30 2009 - 17:39:09 MDT