[squid-users] Squid does not use cache dir(WAS: Squid problem regarding memory usages)

Dear all,

     In my previous post, i stated that the total
space in arena issues. However, further inspection
revealed that the system is running out of file
descriptors ( as logged by cache.log). I would like to
sincerely apologizes for that issue. I go through the
FAQ and followed the steps on 11.4 Running out of file
descriptors. Instead of changing the
/usr/include/bits/types.h file and insert the
FD_SETSIZE, i used another method to accomplish it. I
edited the /etc/security/limits.conf file and set this
:-

root hard nofile 5120

Which means any processes run by root will be able to
open up to 5120 files. The reason is im not sure how
to define the __FD_SETSIZE in types.h. And im not sure
whether the above(limits.conf) method will bring any
performance issues.

Then, i recompiled squid and checked the startup logs.
Below is the startup log :-

2004/07/30 17:10:55| Starting Squid Cache version
2.5.STABLE6 for i686-pc-linux-gnu...
2004/07/30 17:10:55| Process ID 17421
2004/07/30 17:10:55| With 5120 file descriptors
available
2004/07/30 17:10:55| DNS Socket created at 0.0.0.0,
port 32768, FD 5
2004/07/30 17:10:55| Adding nameserver 202.188.0.133
from /etc/resolv.conf
2004/07/30 17:10:55| Adding nameserver 202.188.1.5
from /etc/resolv.conf
2004/07/30 17:10:55| User-Agent logging is disabled.
2004/07/30 17:10:55| Unlinkd pipe opened on FD 10
2004/07/30 17:10:55| Swap maxSize 6144000 KB,
estimated 472615 objects
2004/07/30 17:10:55| Target number of buckets: 23630
2004/07/30 17:10:55| Using 32768 Store buckets
2004/07/30 17:10:55| Max Mem size: 20480 KB
2004/07/30 17:10:55| Max Swap size: 6144000 KB
2004/07/30 17:10:55| Rebuilding storage in /var/squid
(CLEAN)
2004/07/30 17:10:55| Using Least Load store dir
selection
2004/07/30 17:10:55| chdir: /var/spool/squid: (2) No
such file or directory
2004/07/30 17:10:55| Current Directory is
/usr/local/squid/var
2004/07/30 17:10:55| Loaded Icons.
2004/07/30 17:10:55| Accepting HTTP connections at
0.0.0.0, port 3128, FD 11.
2004/07/30 17:10:55| Accepting ICP messages at
0.0.0.0, port 3130, FD 12.
2004/07/30 17:10:55| Accepting SNMP messages on port
3401, FD 13.
2004/07/30 17:10:55| WCCP Disabled.
2004/07/30 17:10:55| Ready to serve requests.
2004/07/30 17:10:56| Done scanning /var/squid swaplog
(0 entries)
2004/07/30 17:10:56| Finished rebuilding storage from
disk.
2004/07/30 17:10:56| 0 Entries scanned
2004/07/30 17:10:56| 0 Invalid entries.
2004/07/30 17:10:56| 0 With invalid flags.
2004/07/30 17:10:56| 0 Objects loaded.
2004/07/30 17:10:56| 0 Objects expired.
2004/07/30 17:10:56| 0 Objects cancelled.
2004/07/30 17:10:56| 0 Duplicate URLs purged.
2004/07/30 17:10:56| 0 Swapfile clashes
avoided.
2004/07/30 17:10:56| Took 0.3 seconds ( 0.0
objects/sec).
2004/07/30 17:10:56| Beginning Validation Procedure
2004/07/30 17:10:56| Completed Validation Procedure
2004/07/30 17:10:56| Validated 0 Entries
2004/07/30 17:10:56| store_swap_size = 0k
2004/07/30 17:10:56| storeLateRelease: released 0
objects

It successfully changed to use 5120 file descriptors.
Now, the question is squid did not use the assigned
6GB cache dir. Im not saying Squid didnt cache
anything. Here's the excerpt of access.log file
entries :-

......

1091463455.597 4 192.168.15.120 TCP_HIT/200 33086
GET
http://kupgrade.bullguard.com/update7cab/avx/Plugins/plugins.cab.cab
- NONE/- text/plain
1091463455.796 4 192.168.15.120 TCP_HIT/200 33086
GET
http://kupgrade.bullguard.com/update7cab/avx/Plugins/plugins.cab.cab
- NONE/- text/plain

And some others negative HIT and refresh HITS.

.....

However, the speed of browsing is fast and in good
shape. Just that it doesnt use the cache dir and that
bugs me for few days. Im curious to know what actually
happened. For more information, i attached the
Genereal Runtime Information file i got from cachemgr.
Hope anyone can guide me. Thanks a lot again!!

Brian

        
                
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!
http://promotions.yahoo.com/new_mail

Squid Object Cache: Version 2.5.STABLE6

Start Time: Fri, 30 Jul 2004 22:05:51 GMT
Current Time: Mon, 02 Aug 2004 16:53:57 GMT

Connection information for squid:
        Number of clients accessing cache: 201
        Number of HTTP requests received: 1941837
        Number of ICP messages received: 0
        Number of ICP messages sent: 0
        Number of queued ICP replies: 0
        Request failure ratio: 0.00
        Average HTTP requests per minute since start: 484.5
        Average ICP messages per minute since start: 0.0
        Select loop called: 51369996 times, 4.681 ms avg
Cache information for squid:
        Request Hit Ratios: 5min: 6.0%, 60min: 19.1%
        Byte Hit Ratios: 5min: 1.9%, 60min: 16.8%
        Request Memory Hit Ratios: 5min: 3.7%, 60min: 12.3%
        Request Disk Hit Ratios: 5min: 13.7%, 60min: 30.7%
        Storage Swap size: 0 KB
        Storage Mem size: 19288 KB
        Mean Object Size: 0.00 KB
        Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
        HTTP Requests (All): 0.76407 0.58309
        Cache Misses: 0.80651 0.72387
        Cache Hits: 0.01309 0.00919
        Near Hits: 0.94847 0.89858
        Not-Modified Replies: 0.00379 0.00286
        DNS Lookups: 0.08334 0.06963
        ICP Queries: 0.00000 0.00000
Resource usage for squid:
        UP Time: 240485.443 seconds
        CPU Time: 23643.270 seconds
        CPU Usage: 9.83%
        CPU Usage, 5 minute avg: 30.66%
        CPU Usage, 60 minute avg: 12.58%
        Process Data Segment Size via sbrk(): 68612 KB
        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 806
Memory usage for squid via mallinfo():
        Total space in arena: 68612 KB
        Ordinary blocks: 67721 KB 4377 blks
        Small blocks: 0 KB 0 blks
        Holding blocks: 1076 KB 2 blks
        Free Small blocks: 0 KB
        Free Ordinary blocks: 890 KB
        Total in use: 68797 KB 99%
        Total free: 890 KB 1%
        Total size: 69688 KB
Memory accounted for:
        Total accounted: 63162 KB
        memPoolAlloc calls: 329464198
        memPoolFree calls: 329403326
File descriptor usage for squid:
        Maximum number of file descriptors: 5120
        Largest file desc currently in use: 1790
        Number of file desc currently in use: 1123
        Files queued for open: 0
        Available number of file descriptors: 3997
        Reserved number of file descriptors: 100
        Store Disk files open: 0
Internal Data Structures:
          1694 StoreEntries
          1694 StoreEntries with MemObjects
          1195 Hot Object Cache Items
             0 on-disk objects
Received on Sun Aug 01 2004 - 22:59:03 MDT