Re: [squid-users] throughput limitation from cache

Also sprach Jason Healy <jhealy+squid@logn.net> (Thu, 12 Jan 2006
22:37:58 -0500 (EST)):
> At 1137142598s since epoch (01/12/06 21:56:38 -0500 UTC), Richard
> Mittendorfer wrote:
> > Well, can't reach this here. Cached ~260KkB/s. And I'm quite sure
> > the file was still in the linux disk cache. What does your cache_dir
> > looks like? aufs I assume.
>
> 27GB on our root filesystem:
>
> cache_dir aufs /var/spool/squid 27000 16 256

I did use aufs for a few tries, but as I saw memory consumption of their
threads going elsewhere, I switched back to diskd. It didn't show better
results. Since I've heard rumored it better fits linux systems, I'll
switch over to aufs when I've got back my memory*. ;)

> > Since you've got plenty of RAM - maybe this is the reason?
>
> Could be; I've thrown 1GB of RAM to cache_mem, so I should be holding
> a fair amount of stuff. Additionally, I've tuned the swap to a large
> cache size. Finally, the maximum_object_size is 160MB to make sure
> I'm keeping large OS updates on hand (saves us a lot of time for OS
> patches).

Besides it's usual role I use squid for caching downloaded updates
(.deb's) too. Very useful when updating a series of boxes. I don't
"feel" the 260Kb/s limit when I'm surfing (DSL ~100Kb/s download)
but when updating from cache I do.
  
> I've set the "swapiness" of the linux kernel to a low level, to
> prevent it from swapping too aggressively.

I don't think it's some limitation/bottleneck in this system. It's
fairly good balanced. Squid doesn't swap for sure. However, the system
currently only has 256M RAM (normaly 512, a 256M stick got broken* three
weeks ago and will _hopefully_ get replaced soon - hard to find one of
these sdram thingies).
  
> What are you using for your speed tests? I'm using wget, so I know
> there's no browser cache issue.

Originally I do a apt-get (it prints the downloadspeed), certainly wget
gives me same results.

100Mb/s FD switched Ethernet too. With quite good performance
NFS(~9.5MB/s) or FTP.

> Also, what network are you testing on? If I use our 802.11 wireless
> network, I can't get much over 280-300KB/s, because I start to bump up
> against the 11Mb/s limit of WIFI. The 2MB/s rate I quoted you earlier
> was over switch 100Mb/s ethernet.

It's the same 260Kb/s wirelessly or ethernet. Have seen it climbing to
290, but that may also be some burst or inexact measurement. 256Kb/s ist
"normal" behavior.

I've just reduced the /proc/net/core and /ipv4 tcp send- and
receivebuffers, which I use to have set quite high here. Also touched
some other settings. Nothing good for. I'm quite sure this has nothing
to do with networking problems/overall system configuration.

>
> > The IDE HD's that carries the storage are spun down most of the
> > time.
>
> You don't think you're waiting on spinup for some of these requests,
> do you? What's the largest file you've downloaded? If it's small
> enough, any spinup time might adversely affect your average bitrate.

erm no. There's just other data living on these disks. The squid caches
are on partitions on two similar scsi disks. They have good seek times
and work properly. They always spin. AFAICR the files were in system
cache. Cached reads/memory subsystem is about 250MB/s. It isn't a lot,
but good enough for this hardware and sure good enough for even a 10MB/s
stream of random reads.
 
> HTH,
>
> Jason

sl ritch
Received on Thu Jan 12 2006 - 22:13:36 MST