Re: memory mapped store entries

On Thu, Aug 27, 1998 at 09:23:43AM +0300, Andres Kroonmaa wrote:

> Dirty pages have some max lifetime in any kernel. Kernel _must_
> flush them in some timeframe, otherwise it is a _broken_ kernel.

I agree it should, and I assumed it did - but my simple tests
indicate on an idle linux machine, this timeframe is more than 10
minutes.

> No. You don't want to explicitly msync anything. msync will block the
> whole process for the duration of a pageout.

No it won't - or shouldn't. See the man page for msync, it can be
done asynchronously, or at worst - in another thread (assuming were
using async-io).

If the kernel has a semaphore of lock around all those pages for the
duration the sync. is going on, then yes - that will block, but I
don't know of any kernel which does this.

> Again I disagree. mmaped data could get inconsistent _only_ in a case
> that squid proccess is wiped out of existence just while it is in the
> proccess of updating any single store entry AND has not yet updated
> structure consistently. In any other case, the mmapped store data will
> be consistent, as kernel _must_ flush all dirty pages when it has to
> unmap all pages related to the process.

Consider this:

  - swap is sync'd to disk (somehow)

  - lots of new files are added to store, perhaps some removed

  - file blocks and file-metadata blocks get written to disk

  - power goes off

Now, whats mmap'd is out of date and _wrong_.

> So, you can kill -9 squid any time, and the mmaped data will be flushed
> with the state it was at the time of receiving signal.

Yes. Assuming it dies this way.

> It is way much more reliable than catching signals and trying to write
> out swaplog as it is done currently.

Yes, I agree. mmap is a win - but its not 100% foolproof against
corruption because my test show the kernel does not flush dirty
buffers at all of very frequently, not my kernel anyhow.

Test code appended. Run it... wait for 10 minutes or so, then
HARD-REBOOT. Examine /tmp/mmap-test when the machine comes back up
and tell me if its not all zero's still.

Oh, and if this ruins you HD, I don't want to know.

-cw

    [ Part 2, Text/PLAIN 69 lines. ]
    [ Unable to print this part. ]
Received on Tue Jul 29 2003 - 13:15:53 MDT