Re: [squid-users] aufs problems in 2.5STABLE1

On 21 Oct 2002, at 21:03, Guido Serassio <serassio@libero.it> wrote:

> > > This is File descriptors cachemgr page from my Squid when idle, up from 11
> > > days:
> > >
> > > Active file descriptors:
> > > File Type Tout Nread * Nwrite * Remote Address Description
> > > 344 Socket 0 0* 0 .0 ICP Socket
> > > 974 Socket 0 4628 13759 127.0.0.1.65323 wb_ntlmauth #1
> > > 1069 Socket 0 824 2345 127.0.0.1.65325 wb_ntlmauth #2
> > > 1302 Socket 0 9 81 127.0.0.1.65327 wb_auth #1
> > > 1438 Socket 0 0 0 127.0.0.1.65329 wb_auth #2
> > > 1476 Socket 0 0 0 127.0.0.1.65331 wb_group #1
> > > 1632 Socket 0 0 0 127.0.0.1.65333 wb_group #2
> > >
> > > Please, look at the very high ordinal number of used FD. It seems that FD
> > > are not reused, why ?
> >
> > What does "pfiles squid-pid" show? smells like FD leak.
> > Solaris should always pick smallest free FD for next open.
>
> This is what cachemgr says:
>
> Active file descriptors:
> File Type Tout Nread * Nwrite * Remote Address Description
> ---- ------ ---- -------- -------- ---------------------
> 12 Socket 0 10755 44543 127.0.0.1.36728 wb_ntlmauth #1
> 13 Socket 0 1254 5144 127.0.0.1.36730 wb_ntlmauth #2
> 14 Pipe 0 0 0 unlinkd -> squid
> 15 Socket 0 12 108 127.0.0.1.36732 wb_auth #1
> 16 Socket 0 0 0 127.0.0.1.36734 wb_auth #2
> 17 Pipe 0 0 0 squid -> unlinkd
> 18 Socket 0 0 0 127.0.0.1.36736 wb_group #1
> 20 Socket 0 0* 0 .0 HTTP Socket
> 21 Socket 0 0* 0 .0 ICP Socket
> 128 Socket 0 0 0 127.0.0.1.36738 wb_group #2
> 226 Socket 1440 162* 0 127.0.0.1.38449 cache_object://
>
> and at the end there is the pfiles output, I'm think that there is some
> problem on Solaris ......

 I'm not so sure. If it was, there would be yelling all over the place.
 Notice that most of FD's you have open are regular files with size/offset 0.
 I would claim that squid thinks it has closed the file but infact hasn't
 closed it. Note that close() _may_ fail with:
    EINTR The close() function was interrupted by a signal.
 We never check for this. I wonder if this can happen with threaded IO..
 You can test for this with
   truss -f -l -t close -o /dev/stdout -p <squid_pid> | egrep -v =.0
 But I think this is unlikely.

 Haven't you compiled with --enable-truncate?
 Do you happen to have lsof to see what files are actually open? You could
 find name by inode but its a pain.

 It may have to do with aborted sessions when file has been opened for
 writing and then client aborts. How do we handle case when FD is in
 progress of open() and then decide to close it?

> # pfiles 785
> 785: (squid)
> Current rlimit: 4096 file descriptors
> 0: S_IFCHR mode:0666 dev:102,0 ino:111581 uid:0 gid:3 rdev:13,2
> O_RDWR
> 3: S_IFREG mode:0644 dev:102,0 ino:154394 uid:60001 gid:65534 size:41464
> O_RDWR|O_APPEND
> 4: S_IFDOOR mode:0444 dev:183,0 ino:12879 uid:0 gid:0 size:0
> O_RDONLY|O_LARGEFILE FD_CLOEXEC door to nscd[203]
> 5: S_IFREG mode:0444 dev:102,0 ino:175307 uid:0 gid:3 size:77
> O_RDONLY
> 6: S_IFSOCK mode:0666 dev:178,0 ino:43026 uid:0 gid:0 size:0
> O_RDWR|O_NONBLOCK FD_CLOEXEC
> sockname: AF_INET 0.0.0.0 port: 33457
> 7: S_IFREG mode:0644 dev:102,0 ino:154395 uid:60001 gid:65534 size:2345591
> O_WRONLY|O_NONBLOCK|O_APPEND FD_CLOEXEC
..
> 12: S_IFSOCK mode:0666 dev:178,0 ino:50130 uid:0 gid:0 size:0
> O_RDWR|O_NONBLOCK FD_CLOEXEC
> sockname: AF_INET 127.0.0.1 port: 36729
> peername: AF_INET 127.0.0.1 port: 36728
...
> 22: S_IFREG mode:0644 dev:102,71 ino:413973 uid:60001 gid:65534 size:0
> O_WRONLY
> 23: S_IFREG mode:0644 dev:102,0 ino:420279 uid:60001 gid:65534 size:0
> O_WRONLY
> 24: S_IFREG mode:0644 dev:102,0 ino:367380 uid:60001 gid:65534 size:0
> O_WRONLY
> 25: S_IFREG mode:0644 dev:102,71 ino:480530 uid:60001 gid:65534 size:0
> O_WRONLY
> 26: S_IFREG mode:0644 dev:102,0 ino:170602 uid:60001 gid:65534 size:0
> O_WRONLY
> 27: S_IFREG mode:0644 dev:102,71 ino:242688 uid:60001 gid:65534 size:16950
> O_RDONLY
> 28: S_IFREG mode:0644 dev:102,71 ino:418056 uid:60001 gid:65534 size:827
> O_RDONLY
> 29: S_IFREG mode:0644 dev:102,71 ino:476570 uid:60001 gid:65534 size:0
> O_WRONLY
> 30: S_IFREG mode:0644 dev:102,71 ino:480529 uid:60001 gid:65534 size:0
> O_WRONLY
> 31: S_IFREG mode:0644 dev:102,0 ino:11143 uid:60001 gid:65534 size:0
> O_WRONLY
> 32: S_IFREG mode:0644 dev:102,71 ino:480236 uid:60001 gid:65534 size:0
> O_WRONLY
> 33: S_IFREG mode:0644 dev:102,71 ino:472940 uid:60001 gid:65534 size:0
> O_WRONLY
> 34: S_IFREG mode:0644 dev:102,71 ino:480449 uid:60001 gid:65534 size:0
> O_WRONLY
> 35: S_IFREG mode:0644 dev:102,0 ino:292951 uid:60001 gid:65534 size:0
> O_WRONLY
> 36: S_IFREG mode:0644 dev:102,71 ino:472841 uid:60001 gid:65534 size:0
> O_WRONLY
> 37: S_IFREG mode:0644 dev:102,71 ino:70502 uid:60001 gid:65534 size:0
> O_WRONLY
> 38: S_IFREG mode:0644 dev:102,0 ino:410078 uid:60001 gid:65534 size:0
> O_WRONLY
> 39: S_IFREG mode:0644 dev:102,0 ino:415054 uid:60001 gid:65534 size:0
> O_WRONLY
> 40: S_IFREG mode:0644 dev:102,71 ino:41 uid:60001 gid:65534 size:0
> O_WRONLY
> 41: S_IFREG mode:0644 dev:102,0 ino:292983 uid:60001 gid:65534 size:0
> O_WRONLY
> 42: S_IFREG mode:0644 dev:102,71 ino:86145 uid:60001 gid:65534 size:0
> O_WRONLY
> 43: S_IFREG mode:0644 dev:102,0 ino:367370 uid:60001 gid:65534 size:0
> O_WRONLY
> 44: S_IFREG mode:0644 dev:102,71 ino:89793 uid:60001 gid:65534 size:0
> O_WRONLY
> 45: S_IFREG mode:0644 dev:102,71 ino:113267 uid:60001 gid:65534 size:0
> O_WRONLY

------------------------------------
 Andres Kroonmaa <andre@online.ee>
 CTO, Microlink Online
 Tel: 6501 731, Fax: 6501 725
 Pärnu mnt. 158, Tallinn,
 11317 Estonia
Received on Tue Oct 22 2002 - 05:23:24 MDT