Re: [squid-users] squid2.7 Stable 6 passes in intervals where cpu usage hits 100%

Ragheb Rustom wrote:
> Hi Amos,
>
> Thanks for the info.
>
> According to what you have mentioned below what do you think should be the
> value of cache_swap_low and high values to lower the load on the system
> although the hardware is big enough I think to hold such a load (all system
> us SAS based with 15k rpm drives and has 16GB of pyshical memory.)

Squid still requires the % to be whole integers so the only choices you
have are 1 or 2 apart. Making it "94 95" may cut the overload time in half.
If it is the garbage collection then you will have to wait for the real fix.

PS. Alex is working on storage improvements for Squid-3 now if anyone is
able to throw any resources his way.

Amos

>
>> That indicates that when the cache dir gets more than 95% full it needs
>> to clear at least 2GB of objects (2% of 115,000MB).
>>
>> Depending on your disks and avg object sizes this may be a large part of
>> the load.
>
> As for the regex acls since like I have said these specified by the
> videocache python squid plugin redirector setup I can try turning all of
> these off and see if the problem persists or no.
> And these regex acls are used only with below access lines
>
> url_rewrite_access deny videocache_deny_url
> url_rewrite_access allow videocache_allow_url
> url_rewrite_access allow videocache_allow_dom
>
>
>> A lot of those can be reduced to dstdomain or at least dstdomain
>> followed by a regex.
>>
>> ie:
>> acl A dstdomain .googlevideo.com
>> acl R urlpath_regex -i /videoplayback
>>
>> http_access allow/deny A R
>>
>> Using dstdomain (very fast test) at the start of an access line to
>> determine if the slower regex later on in the line is even needed can
>> produce a dramatic speed boost in response times.
>>
>> Where you are testing for file types the rep_mime_type on reply can
>> check the mime faster than a regex can test the entire URL.
>>
>> Also matching all of that file type by mime from a whole dstdomain can
>> be more effective and faster than a specific regex testing for a range
>> of specific sub-domains and file endings.
>>
>> Also you do not indicate which if the *_access lines these are being
>> tested in. It's a very good idea to only to them in one of either
>> http_access or http_reply_access, not both.
>
>
> Ragheb Rustom
>
>
> -----Original Message-----
> From: Amos Jeffries [mailto:squid3_at_treenet.co.nz]
> Sent: Saturday, May 30, 2009 11:16 AM
> To: Ragheb Rustom
> Cc: squid-users_at_squid-cache.org; crobertson_at_gci.net
> Subject: Re: [squid-users] squid2.7 Stable 6 passes in intervals where cpu
> usage hits 100%
>
> Ragheb Rustom wrote:
>> Hi Chris,
>>
>> Thank you for the reply. I will use squid -k debug to check what is going
>> on.
>>
>> As for the additional info sorry for not clearing the squid setup more
>> anyway this is squid configuration values you have guessed as taken from
> my
>> squid.conf
>>
>> Wild guesses based on the information given:
>>
>> *) You are using UFS for your cache_dir instead of AUFS.
>>
>> cache_dir aufs /cache1 115000 16 256
>> cache_dir aufs /cache2 115000 16 256
>>
>> I am using AUFS not UFS.
>>
>> *) Your cache is pretty large and your cache_swap_high and
>> cache_swap_low are still at the defaults
>>
>> cache_swap_low 93
>> cache_swap_high 95
>>
>> Do u think these values are reasonable enough regarding the above values
> of
>> my Cacche dir or should they be changed.
>>
>
> That indicates that when the cache dir gets more than 95% full it needs
> to clear at least 2GB of objects (2% of 115,000MB).
>
> Depending on your disks and avg object sizes this may be a large part of
> the load.
>
>
>> *) You are using large lists of regex ACLs
>>
>> acl videocache_allow_url url_regex -i \.youtube\.com\/get_video\?
>> acl videocache_allow_url url_regex -i \.googlevideo\.com\/videoplayback
>> \.googlevideo\.com\/get_video\?
>> acl videocache_allow_url url_regex -i \.google\.com\/videoplayback
>> \.google\.com\/get_video\?
>> acl videocache_allow_url url_regex -i \.google\.[a-z][a-z]\/videoplayback
>> \.google\.[a-z][a-z]\/get_video\?
>> acl videocache_allow_url url_regex -i
>>
> (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]
> ?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][
>> 0-9]?)\/videoplayback\?
>> acl videocache_allow_url url_regex -i
>>
> (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]
> ?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][
>> 0-9]?)\/get_video\?
>> acl videocache_allow_url url_regex -i
>> proxy[a-z0-9\-][a-z0-9][a-z0-9][a-z0-9]?\.dailymotion\.com\/
>> acl videocache_allow_url url_regex -i vid\.akm\.dailymotion\.com\/
>> acl videocache_allow_url url_regex -i
>> [a-z0-9][0-9a-z][0-9a-z]?[0-9a-z]?[0-9a-z]?\.xtube\.com\/(.*)flv
>> acl videocache_allow_url url_regex -i bitcast\.vimeo\.com\/vimeo\/videos\/
>> acl videocache_allow_url url_regex -i
>> va\.wrzuta\.pl\/wa[0-9][0-9][0-9][0-9]?
>> acl videocache_allow_url url_regex -i \.files\.youporn\.com\/(.*)\/flv\/
>> acl videocache_allow_url url_regex -i
> \.msn\.com\.edgesuite\.net\/(.*)\.flv
>> acl videocache_allow_url url_regex -i
>> media[a-z0-9]?[a-z0-9]?[a-z0-9]?\.tube8\.com\/
>> mobile[a-z0-9]?[a-z0-9]?[a-z0-9]?\.tube8\.com\/
>> acl videocache_allow_url url_regex -i \.mais\.uol\.com\.br\/(.*)\.flv
>> acl videocache_allow_url url_regex -i
>>
> \.video[a-z0-9]?[a-z0-9]?\.blip\.tv\/(.*)\.(flv|avi|mov|mp3|m4v|mp4|wmv|rm|r
>> am)
>> acl videocache_allow_url url_regex -i video\.break\.com\/(.*)\.(flv|mp4)
>> acl videocache_allow_dom dstdomain v.mccont.com dl.redtube.com
>> .cdn.dailymotion.com
>> acl videocache_deny_url url_regex -i http:\/\/[a-z][a-z]\.youtube\.com
>> http:\/\/www\.youtube\.com
>>
>> acl bad_requests urlpath_regex -i cmd.exe \/bin\/sh default\.ida?XXXXXXX
>> omg.pif
>>
>> acl music url_regex -i .wav
>> acl video url_regex -i .avi .mpe
>> acl cddown url_regex -i .iso .raw
>>
>> These are all the acl regex entries I use in my squid.conf file mainly
> most
>> of them are those of videocache.
>
> A lot of those can be reduced to dstdomain or at least dstdomain
> followed by a regex.
>
> ie:
> acl A dstdomain .googlevideo.com
> acl R urlpath_regex -i /videoplayback
>
> http_access allow/deny A R
>
> Using dstdomain (very fast test) at the start of an access line to
> determine if the slower regex later on in the line is even needed can
> produce a dramatic speed boost in response times.
>
> Where you are testing for file types the rep_mime_type on reply can
> check the mime faster than a regex can test the entire URL.
>
> Also matching all of that file type by mime from a whole dstdomain can
> be more effective and faster than a specific regex testing for a range
> of specific sub-domains and file endings.
>
> Also you do not indicate which if the *_access lines these are being
> tested in. It's a very good idea to only to them in one of either
> http_access or http_reply_access, not both.
>
>
>> I will use the squid -k debug and try and get more info of what is going
> on
>> during the high cpu usage period and will share whatever I find on
>> squid-users mail list.
>>
>> Thanks Chris for your support.
>>
>> Ragheb
>>
>> -----Original Message-----
>> From: crobertson_at_gci.net [mailto:crobertson_at_gci.net]
>> Sent: Friday, May 29, 2009 10:18 PM
>> To: squid-users_at_squid-cache.org
>> Subject: Re: [squid-users] squid2.7 Stable 6 passes in intervals where cpu
>> usage hits 100%
>>
>> Ragheb Rustom wrote:
>>> Dear All,
>>>
>>> I have been noticing for some time now that my squid server passes in
>> nearly
>>> regular intervals during the day where the squid process CPU usage hits
>> 100%
>>> for around 20 - 30 seconds or so during which squid will not serve
>> requests.
>>> After these 20 -30sec the squid CPU usage drops to around 5 - 13% and
>>> website navigation is normal again. This happens like I said before at
>>> nearly regular intervals during each day several times and it happens
> only
>>> for around 20 -30sec. I have been observing the cache.log file but there
>> is
>>> nothing apparent there. Is there a way that I know what squid is doing
>> when
>>> this problem happens to try and resolve it as trying to solve it like it
>> is
>>> now with no info about what is going on is somewhat vague.
>> squid -k debug
>>
>>> I think this
>>> problem happened after I have changed memory and disk replacement
> policies
>>> from lru to heap but I am not so sure that this is the source of the
>>> problem. I am using squid2.7-Stable6 on a fedora9 server which is a dual
>>> xeon quad 2.8Ghz CPU with 16Gb of Ram installed and 3 SAS hard drives.
> One
>>> more thing this server is also running videocache 1.9.1 but it has been
>>> running videocache also before the problem started with no problems
>>> whatsoever. I appreciate any help you can offer me.
>>>
>> Wild guesses based on the information given:
>>
>> *) You are using UFS for your cache_dir instead of AUFS.
>> *) Your cache is pretty large and your cache_swap_high and
>> cache_swap_low are still at the defaults
>> *) You are using large lists of regex ACLs
>>
>>> Sincerely,
>>>
>>> Ragheb Rustom
>>> Smartelecom S.A.R.L
>>> Sin el fil - Mar Elias Street
>>> Absi Center - Ground Floor
>>> Beirut, Lebanon
>>> Email: ragheb_at_smartelecom.org
>>>
>> Chris
>>
>>
>
>

-- 
Please be using
   Current Stable Squid 2.7.STABLE6 or 3.0.STABLE15
   Current Beta Squid 3.1.0.8 or 3.0.STABLE16-RC1