Re: how to apply epoll-2_5 patch to squid2.5-stable9 (fwd)

From: Henrik Nordstrom <hno@dont-contact.us>
Date: Wed, 13 Apr 2005 15:29:08 +0200 (CEST)

---------- Forwarded message ----------
Date: Wed, 13 Apr 2005 12:15:40 +0200
From: "[Windows-1250] Paweł Staszewski" <pstaszewski@artcom.pl>
To: hno@squid-cache.org, squid-users@squid-cache.org
Subject: Re: Odp: Re: [squid-users] how to apply epoll-2_5 patch to
     squid2.5-stable9

Ok :)

Some comparisions:

##############################################################
Squid-2.5Stable9-20050330 with epoll

12:09:58 root@w3cache1 /home/paol/squid-2.5.STABLE9-20050330]$ mpstat -P ALL 1
Linux 2.6.12-rc2 (w3cache1) 04/13/05

12:12:36 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:12:37 all 1.50 0.00 0.50 3.50 0.00 0.50 94.00 2484.16
12:12:37 0 0.99 0.00 0.99 3.96 0.00 0.00 94.06 991.09
12:12:37 1 1.98 0.00 0.99 2.97 0.00 1.98 91.09 1493.07

12:12:37 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:12:38 all 1.00 0.00 1.00 2.49 0.00 0.50 95.02 2620.20
12:12:38 0 0.00 0.00 0.00 3.03 0.00 0.00 96.97 1011.11
12:12:38 1 2.02 0.00 2.02 1.01 0.00 0.00 95.96 1609.09

12:12:38 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:12:39 all 1.01 0.00 1.01 1.01 0.00 1.01 95.96 2809.09
12:12:39 0 0.00 0.00 0.00 1.01 0.00 0.00 98.99 1011.11
12:12:39 1 3.03 0.00 1.01 1.01 1.01 3.03 92.93 1797.98

12:12:39 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:12:40 all 1.49 0.00 0.99 5.45 0.50 1.49 90.10 2834.65
12:12:40 0 0.00 0.00 0.99 8.91 0.00 0.00 90.10 991.09
12:12:40 1 1.98 0.00 1.98 2.97 0.00 2.97 90.10 1843.56

12:12:40 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:12:41 all 2.00 0.00 2.00 23.00 0.00 1.50 71.50 2839.60
12:12:41 0 1.98 0.00 0.99 29.70 0.00 0.00 67.33 991.09
12:12:41 1 2.97 0.00 1.98 15.84 0.00 1.98 74.26 1848.51

[12:12:42 root@w3cache1 /home/paol/squid-2.5.STABLE9-20050330]$

[12:12:42 root@w3cache1 /home/paol/squid-2.5.STABLE9-20050330]$ iostat 1 -x
Linux 2.6.12-rc2 (w3cache1) 04/13/05

avg-cpu: %user %nice %sys %iowait %idle
            4.84 0.00 14.99 5.33 74.84

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 0.05 17.29 4.09 11.33 46.48 239.83 23.24 119.91 18.56 0.41 26.64 3.42 5.27
sdb 0.05 13.22 3.99 7.02 45.44 172.97 22.72 86.48 19.84 0.24 21.62 3.61 3.98
sdc 0.04 13.39 3.32 6.59 38.68 170.45 19.34 85.23 21.10 0.20 20.22 3.53 3.50
sdd 0.05 12.95 4.03 6.68 46.13 167.90 23.06 83.95 19.98 0.26 24.05 3.65 3.91

avg-cpu: %user %nice %sys %iowait %idle
            1.00 0.00 1.00 1.00 97.00

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 31.68 0.00 30.69 0.00 511.88 0.00 255.94 16.68 0.06 2.10 0.61 1.88
sdb 0.00 26.73 0.00 3.96 0.00 279.21 0.00 139.60 70.50 0.00 0.75 0.75 0.30
sdc 0.00 15.84 0.99 2.97 7.92 187.13 3.96 93.56 49.25 0.04 9.00 9.00 3.56
sdd 0.00 123.76 2.97 12.87 31.68 1165.35 15.84 582.67 75.56 0.20 12.75 3.94 6.24

avg-cpu: %user %nice %sys %iowait %idle
            1.49 0.00 2.49 0.50 95.52

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 0.00 2.02 18.18 24.24 141.41 12.12 70.71 8.20 0.13 6.35 4.15 8.38
sdb 0.00 0.00 4.04 7.07 48.48 42.42 24.24 21.21 8.18 0.04 3.91 3.27 3.64
sdc 0.00 0.00 6.06 1.01 64.65 64.65 32.32 32.32 18.29 0.03 4.00 4.00 2.83
sdd 0.00 0.00 3.03 3.03 24.24 17.17 12.12 8.59 6.83 0.01 1.50 1.17 0.71

avg-cpu: %user %nice %sys %iowait %idle
            1.00 0.00 1.49 1.49 96.02

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 2.00 0.00 24.00 0.00 12.00 0.00 12.00 0.01 3.00 3.00 0.60
sdc 0.00 0.00 7.00 3.00 88.00 17.00 44.00 8.50 10.50 0.03 3.30 3.10 3.10
sdd 0.00 0.00 6.00 0.00 64.00 0.00 32.00 0.00 10.67 0.04 6.00 6.00 3.60

[12:13:38 root@w3cache1 /home/paol/squid-2.5.STABLE9-20050330]$

Connection information for squid:
         Number of clients accessing cache: 0
         Number of HTTP requests received: 401769
         Number of ICP messages received: 0
         Number of ICP messages sent: 0
         Number of queued ICP replies: 0
         Request failure ratio: 0.00
         Average HTTP requests per minute since start: 2595.2
         Average ICP messages per minute since start: 0.0
         Select loop called: 11667085 times, 0.796 ms avg
Cache information for squid:
         Request Hit Ratios: 5min: 42.6%, 60min: 40.7%
         Byte Hit Ratios: 5min: 15.9%, 60min: 14.2%
         Request Memory Hit Ratios: 5min: 1.9%, 60min: 2.4%
         Request Disk Hit Ratios: 5min: 36.7%, 60min: 44.0%
         Storage Swap size: 28163112 KB
         Storage Mem size: 65360 KB
         Mean Object Size: 17.99 KB
         Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
         HTTP Requests (All): 0.07014 0.09219
         Cache Misses: 0.22004 0.24524
         Cache Hits: 0.00091 0.00179
         Near Hits: 0.16775 0.20843
         Not-Modified Replies: 0.00000 0.00000
         DNS Lookups: 0.00000 0.00000
         ICP Queries: 0.00000 0.00000
Resource usage for squid:
         UP Time: 9288.825 seconds
         CPU Time: 256.797 seconds
         CPU Usage: 2.76%
         CPU Usage, 5 minute avg: 2.85%
         CPU Usage, 60 minute avg: 2.70%
         Process Data Segment Size via sbrk(): 253996 KB
         Maximum Resident Size: 0 KB
         Page faults with physical i/o: 0
Memory usage for squid via mallinfo():
         Total space in arena: 253996 KB
         Ordinary blocks: 253604 KB 23824 blks
         Small blocks: 0 KB 0 blks
         Holding blocks: 14364 KB 7 blks
         Free Small blocks: 0 KB
         Free Ordinary blocks: 391 KB
         Total in use: 267968 KB 100%
         Total free: 391 KB 0%
         Total size: 268360 KB
Memory accounted for:
         Total accounted: 183849 KB
         memPoolAlloc calls: 59288434
         memPoolFree calls: 55796838
File descriptor usage for squid:
         Maximum number of file descriptors: 32768
         Largest file desc currently in use: 1055
         Number of file desc currently in use: 913
         Files queued for open: 0
         Available number of file descriptors: 31855
         Reserved number of file descriptors: 100
         Store Disk files open: 9
Internal Data Structures:
         1567568 StoreEntries
          15360 StoreEntries with MemObjects
          15284 Hot Object Cache Items
         1565477 on-disk objects

sockets: used 945
TCP: inuse 2060 orphan 1176 tw 630 alloc 2071 mem 18497
UDP: inuse 38
RAW: inuse 0
FRAG: inuse 0 memory 0

#########################################################

Squid-2.5Stable9-20050405 without epoll

[12:16:57 root@w3cache2 ~]$ mpstat -P ALL 1
Linux 2.6.12-rc2 (w3cache2) 04/13/05

12:17:01 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:17:02 all 9.60 0.00 23.74 7.07 0.00 1.01 58.59 2882.83
12:17:02 0 0.00 0.00 3.03 13.13 0.00 0.00 83.84 1011.11
12:17:02 1 18.18 0.00 44.44 1.01 0.00 3.03 34.34 1871.72

12:17:02 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:17:03 all 8.42 0.00 23.76 3.47 0.00 2.97 61.39 2756.86
12:17:03 0 0.98 0.00 2.94 5.88 0.00 0.00 90.20 981.37
12:17:03 1 15.69 0.00 44.12 1.96 0.98 4.90 30.39 1775.49

12:17:03 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:17:04 all 5.47 0.00 18.91 21.89 0.50 1.49 51.74 2737.76
12:17:04 0 0.00 0.00 2.04 24.49 0.00 0.00 73.47 1021.43
12:17:04 1 11.22 0.00 36.73 19.39 0.00 3.06 32.65 1716.33

12:17:04 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:17:05 all 6.50 0.00 18.50 3.00 0.00 0.50 71.50 2248.04
12:17:05 0 0.98 0.00 1.96 5.88 0.00 0.00 91.18 981.37
12:17:05 1 12.75 0.00 34.31 0.00 0.00 0.98 50.00 1266.67

12:17:05 CPU %user %nice %system %iowait %irq %soft %idle intr/s
12:17:06 all 4.52 0.00 14.07 5.03 0.00 0.50 75.88 2151.52
12:17:06 0 1.01 0.00 3.03 7.07 0.00 0.00 88.89 1011.11
12:17:06 1 8.08 0.00 25.25 3.03 0.00 2.02 63.64 1140.40

[12:17:06 root@w3cache2 ~]$

[12:17:06 root@w3cache2 ~]$ iostat 1 -x
Linux 2.6.12-rc2 (w3cache2) 04/13/05

avg-cpu: %user %nice %sys %iowait %idle
            5.42 0.00 17.04 8.29 69.25

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 0.07 18.23 4.92 14.80 57.31 264.33 28.66 132.17 16.32 0.47 23.90 3.42 6.74
sdb 0.07 15.86 4.94 11.33 55.57 217.64 27.78 108.82 16.79 0.36 21.84 3.36 5.46
sdc 0.07 15.48 4.93 10.88 56.06 211.01 28.03 105.50 16.89 0.38 24.19 3.39 5.36
sdd 0.07 15.87 4.97 10.63 56.65 212.13 28.32 106.07 17.23 0.41 26.24 3.40 5.30

avg-cpu: %user %nice %sys %iowait %idle
            2.50 0.00 10.50 3.00 84.00

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 0.00 1.00 0.00 8.00 0.00 4.00 0.00 8.00 0.01 11.00 11.00 1.10
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdc 0.00 0.00 2.00 0.00 16.00 0.00 8.00 0.00 8.00 0.02 8.00 8.00 1.60
sdd 0.00 0.00 3.00 0.00 24.00 0.00 12.00 0.00 8.00 0.04 12.00 9.67 2.90

avg-cpu: %user %nice %sys %iowait %idle
            4.52 0.00 16.58 6.03 72.86

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 0.00 7.07 0.00 56.57 0.00 28.28 0.00 8.00 0.04 5.29 5.29 3.74
sdb 0.00 0.00 4.04 0.00 32.32 0.00 16.16 0.00 8.00 0.01 3.00 3.00 1.21
sdc 0.00 0.00 10.10 0.00 80.81 0.00 40.40 0.00 8.00 0.05 4.90 4.90 4.95
sdd 0.00 0.00 8.08 0.00 72.73 0.00 36.36 0.00 9.00 0.10 12.50 7.75 6.26

avg-cpu: %user %nice %sys %iowait %idle
            5.97 0.00 13.93 4.98 75.12

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 0.00 1.00 0.00 8.00 0.00 4.00 0.00 8.00 0.01 8.00 8.00 0.80
sdb 0.00 0.00 3.00 0.00 24.00 0.00 12.00 0.00 8.00 0.02 6.00 6.00 1.80
sdc 0.00 0.00 2.00 0.00 16.00 0.00 8.00 0.00 8.00 0.02 10.00 10.00 2.00
sdd 0.00 0.00 5.00 0.00 40.00 0.00 20.00 0.00 8.00 0.07 13.20 7.80 3.90

sockets: used 945
TCP: inuse 2060 orphan 1176 tw 630 alloc 2071 mem 18497
UDP: inuse 38
RAW: inuse 0
FRAG: inuse 0 memory 0

Squid Object Cache: Version 2.5.STABLE9-20050405

Start Time: Fri, 08 Apr 2005 11:51:04 GMT
Current Time: Wed, 13 Apr 2005 10:17:58 GMT

Connection information for squid:
         Number of clients accessing cache: 0
         Number of HTTP requests received: 22141549
         Number of ICP messages received: 0
         Number of ICP messages sent: 0
         Number of queued ICP replies: 0
         Request failure ratio: 0.00
         Average HTTP requests per minute since start: 3115.5
         Average ICP messages per minute since start: 0.0
         Select loop called: 152771070 times, 2.791 ms avg
Cache information for squid:
         Request Hit Ratios: 5min: 43.1%, 60min: 46.2%
         Byte Hit Ratios: 5min: 18.2%, 60min: 20.3%
         Request Memory Hit Ratios: 5min: 1.4%, 60min: 1.7%
         Request Disk Hit Ratios: 5min: 43.6%, 60min: 41.2%
         Storage Swap size: 39326380 KB
         Storage Mem size: 65352 KB
         Mean Object Size: 15.67 KB
         Requests given to unlinkd: 0
Median Service Times (seconds) 5 min 60 min:
         HTTP Requests (All): 0.09736 0.09219
         Cache Misses: 0.25890 0.25890
         Cache Hits: 0.00919 0.00919
         Near Hits: 0.27332 0.24524
         Not-Modified Replies: 0.00179 0.00179
         DNS Lookups: 0.00278 0.00278
         ICP Queries: 0.00000 0.00000
Resource usage for squid:
         UP Time: 426413.551 seconds
         CPU Time: 195990.704 seconds
         CPU Usage: 45.96%
         CPU Usage, 5 minute avg: 52.81%
         CPU Usage, 60 minute avg: 43.51%
         Process Data Segment Size via sbrk(): 370404 KB
         Maximum Resident Size: 0 KB
         Page faults with physical i/o: 2
Memory usage for squid via mallinfo():
         Total space in arena: 370404 KB
         Ordinary blocks: 354889 KB 72219 blks
         Small blocks: 0 KB 0 blks
         Holding blocks: 23800 KB 14 blks
         Free Small blocks: 0 KB
         Free Ordinary blocks: 15514 KB
         Total in use: 378689 KB 96%
         Total free: 15514 KB 4%
         Total size: 394204 KB
Memory accounted for:
         Total accounted: 245383 KB
         memPoolAlloc calls: 2919539494
         memPoolFree calls: 2914209246
File descriptor usage for squid:
         Maximum number of file descriptors: 32768
         Largest file desc currently in use: 1055
         Number of file desc currently in use: 1021
         Files queued for open: 0
         Available number of file descriptors: 31747
         Reserved number of file descriptors: 100
         Store Disk files open: 12
Internal Data Structures:
         2523322 StoreEntries
          14260 StoreEntries with MemObjects
          14160 Hot Object Cache Items
         2510208 on-disk objects

[12:17:36 root@w3cache2 ~]$ cat /proc/net/sockstat
sockets: used 1000
TCP: inuse 2262 orphan 1290 tw 833 alloc 2263 mem 15274
UDP: inuse 15
RAW: inuse 0
FRAG: inuse 0 memory 0
[12:19:31 root@w3cache2 ~]$

Both squid's are balanced for 3500users with 18Mbit internet uplink (only www)
squid with epoll patch is performing much better :)

Best Regards
Paweł Staszewski
ART-COM
+48327522333
+480609183038

>>>Henrik Nordstrom <hno@squid-cache.org> 04/13/05 10:42 am >>>
On Wed, 13 Apr 2005, [Windows-1250] Paweł Staszewski wrote:

>When i apply epool-2_5.patch
>
>In my syslog i have this messages:
>
>Apr 13 09:42:25 w3cache1 squid[24895]: commSetSelect: epoll_ctl(EPOLL_CTL_DEL): failed on fd=197: (2) No such file or directory
>Apr 13 09:42:25 w3cache1 squid[24895]: commSetSelect: epoll_ctl(EPOLL_CTL_DEL): failed on fd=234: (2) No such file or directory

Looks like the epoll patch is not 100% perfect yet.

Regards
Henrik
Received on Wed Apr 13 2005 - 07:29:10 MDT

This archive was generated by hypermail pre-2.1.9 : Sun May 01 2005 - 12:00:06 MDT