Re: [RFC] Squid process model and -z

On 11/06/2013 02:49 AM, Amos Jeffries wrote:
> On 6/11/2013 11:00 a.m., Alex Rousskov wrote:
>> On 11/01/2013 10:09 AM, Alex Rousskov wrote:
>>> I would also like to propose an alternative:
>>>
>>> Make "squid -z" a no-op (Squid would just exit immediately after
>>> detecting that option). Teach Squid to always prime cache_dirs at
>>> startup. This will remove all problems discussed above. Moreover, it
>>> will make admins live easier because they do not need to think about
>>> writing correct scripts to detect cache_dir validity. It was probably a
>>> mistake to add "squid -z" in the first place. We can correct it now.
>> The "get rid of squid -z" idea is not new, of course. I now reviewed
>> several related threads on squid-users. I found only one class of use
>> cases where admins believe they need squid-z:
>>
>> * Squid should quit if there is no cache_dir instead of creating and
>> using that cache_dir.
>>
>> All the use cases in this class revolve around something that must
>> happen before Squid starts so that Squid disk caching works correctly
>> instead of destroying the server. The simplest example is when some
>> external force must mount the cache directory. If that external force
>> fails to do so, Squid must quit instead of, say, overflowing the root
>> partition. A more complex example is a broken filesystem that should be
>> repaired before Squid corrupts it further, making repairs more difficult
>> or impossible.
>>
>> In all these cases, the admins rely on Squid quitting if cache_dir is
>> missing. Since Squid does not really verify that subdirs are present,
>> not starting Squid if the top-level directory is missing ought to be
>> equivalent to the current Squid behavior. Thus, is we make squid-z a
>> no-op, the admins that do not want Squid to start when a cache_dir is
>> missing, can implement that directory check in their Squid startup
>> script.
>>
>> There were also suggestions to add a "create cache_dirs if missing"
>> option to squid.conf. I think that (together with making squid-z a
>> no-op) would be OK and would help in use cases described above. However,
>> the option should be ON by default for new Squid installations to work
>> seamlessly.
>>
>> To summarize, the proposal is
>>
>> 1) Teach Squid to create missing disk cache structures at runtime.
>> 2) Make squid-z a no-op and deprecate that option.
>
> You mean make command line -z halt squid with effectively a exit(0)
> "success" immediately on reading that option?

Yes, -z will cause Squid to exit ASAP, after printing a message that the
external cache_dir creation is no longer supported (just like -h prints
a help message and exists with no side effects).

>> 3) Add a squid.conf option to disable (1).
>
> Agreed. cache_dir ... no-auto-create

Yes, except we should use positive names and think about extendibility.
I would recommend something like

    cache_dir ... on-missing=create|quit|bypass ...

I am not sure whether "bypass", "skip", or "ignore" would be better but
the intent of that value would be to ignore the corresponding cache_dir
line as if it was not there.

I hope this check itself can be done by generic SwapDir code, without fs
specialization because all cache_dir types have the same concept of root
"directory" and we already have a type-specific creation API (which is
what squid-z is using now). However, the code needs to function
correctly in SMP cases, which may require some minor API changes.

> "create" disable rather that "repair"disable since the class of use case
> is for handling disk disappearances creating a new invalid FS structure.
> Repair of existing and linked disk structure is not something anyone has
> spoken against, so we have to reason to halt instead of fixing those.

Yes, repair is a different can of worms, outside this discussion scope,
but I recall using a similar approach in some long-forgotten Squid2 patch:

    cache_dir ... on-error=quit|bypass|repair ...

Again, this is a different problem so let's try to stay focused on -z.

> I have been and agree that we should do this.
>
> Now, who is keen to code it?

If nobody volunteers, Factory will do this.

Does anybody want to work on this?

Thank you,

Alex.
Received on Wed Nov 06 2013 - 15:58:47 MST