Re: Store_url_rewrite for squid 3+

On 9/5/2012 9:56 AM, Eliezer Croitoru wrote:
> any leads,?
Well there is a nice progress.
I reviewed the 2.7 store_url_rewrite and I divided the task into more
detailed smaller tasks.

1. Research the url_rewrite interface code and Introduce a modified
version of url_rewrite as url_store_rewrite_program.
- this task is kind of done(passed compiling and running on 3.2.1) by
now and I want to get some ideas on naming conventions for the code to
fit the project amazing code looks.

list of changed files and code:
create a mimic file of redirect.cc in ./store_rewrite.cc and change
all the methods and variables to fit store_rewrite.
strip all the url_rewrite data manipulating actions.
change the debugging info.
(after the store related planning tasks get back here to redo)

./structs.h
adding the proper variables for:
helper naming, bypass(on\off), acl_access namespace, child configs
the ??_rewrites_host of url_rewrite dosnt belong for store_rewrite
process at all.

./cache_cf.cc
state the default configuration for the helper

./cf.data.pre
stating all config directives for the the helper (copy and modify from
url_rewrite_program)

./ClientRequestContext.h
adding int for state
adding bool for done

./client_side_request.h
stating the start method as squidexternal something

./client_side_request.cc
adding calls and callouts

./protos.h
stating the start init and shutdown methods.

./main.cc:
calling init and shutdown methods at start/reconfigure etc..

./Makefile.am && ./Makefile.in
adding the source ?.cc file into the commands

2. Research the workflow of storing objects in memory and store and
introduce psudo for a new workflow of storing objects to avoid bad
effects on cache objects usage in any form that can be.
- I do know that squid uses some hash look-up and I have seen in the
things about it.
- as far I understood from the code:
client_request builds the request of the http object.
creates a mem-object and on the way creates a checksum.
a transfer from of the mem-object to a "store" happens.
if a store rebuild happens it takes all of the data from the file in the
store.

? question how cachemgr gets the list of urls in memory?

so probable points of failure:
using the wrong url to fetch the object.
wrong arguments for checksum.
storing with wrong arguments\url leading to faulty rebuild.

I do remember that when I looked at a stored old store_url_rewrite cahce
file long time ago there were two urls in the file what leads me (it's a
bit fogy) to think that the stored file was the memobject cache rather
then a set of arguments such as refresh time related
info,method,url,request,response.

I will look at it later but if someone have solid knowledge on how the
store routing was or implemented before i'm rushing into the code every
piece of info will help me when looking into it.

Eliezer

-- 
Eliezer Croitoru
https://www1.ngtech.co.il
IT consulting for Nonprofit organizations
eliezer <at> ngtech.co.il