Re: trying cache post request for some sites failed

On Mon, 22 Jun 2009 10:21:16 +0700, johan firdianto <johanfirdi_at_gmail.com>
wrote:
> dear guys,
>
> I'm using squid 2.7 stable 5. Tring to cache some sites that use POST
> method as download method.
> I modif the source code of client_side.c at line 1550 in function
> clientCachable
>
>
> if ((method == METHOD_POST) && (!
> strncmp(uri,"http://xxxxxxxxxxxxx/",40)))
> return 1;
>
> in store.c line 1225 in function storeCheckCachable
> if ((e->mem_obj->method == METHOD_POST) && (!
> strncmp(e->mem_obj->url,"http://xxxxxxxxx/",40)) ) {
> store_check_cachable_hist.yes.Default++;
> return 1;
> }
>
> Here the debug result, flag REQ_CACHABLE is SET
> The request POST http://xxxxx/3705584/SD-Mat07.pdf is ALLOWED, because
> it matched 'test'
> 2009/06/22 09:49:03| clientRedirectStart:
> 'http://xxxxxxxxx/3705584/SD-Mat07.pdf'
> 2009/06/22 09:49:03| clientRedirectDone:
> 'http://xxxxxxxx/3705584/SD-Mat07.pdf' result=NULL
> 2009/06/22 09:49:03| clientStoreURLRewriteStart:
> 'http://xxxxxxxxxxxxx/3705584/SD-Mat07.pdf'
> 2009/06/22 09:49:03| clientStoreURLRewriteDone:
> 'http://xxxxxxxxx/3705584/SD-Mat07.pdf' result=NULL
> 2009/06/22 09:49:03| 0xb9f8d8 lookup for 37
> 2009/06/22 09:49:03| 0xb9f8d8 lookup for 7
> 2009/06/22 09:49:03| 0xb9f8d8 lookup for 7
> 2009/06/22 09:49:03| 0xb9f8d8 lookup for 40
> 2009/06/22 09:49:03| 0xb9f8d8 lookup for 52
> 2009/06/22 09:49:03| check clientcacheble atau tidak:
> http://xxxxxxxxxxx/3705584/SD-Mat07.pdf
> 2009/06/22 09:49:03| clientInterpretRequestHeaders: REQ_NOCACHE = NOT SET
> 2009/06/22 09:49:03| clientInterpretRequestHeaders: REQ_CACHABLE = SET
> 2009/06/22 09:49:03| clientInterpretRequestHeaders: REQ_HIERARCHICAL =
NOT
> SET
>
> Looks like create privatekey of the object,
>
> new_MemObject: returning 0xb9bd10
> 2009/06/22 09:49:03| new_StoreEntry: returning 0xb7bac0
> 2009/06/22 09:49:03| storeKeyPrivate: POST
> http://xxxxxxxxxxxxxxxxxxxxxx/3705584/SD-Mat07.pdf
> 2009/06/22 09:49:03| storeHashInsert: Inserting Entry 0xb7bac0 key
> '137B66F60D79074DC0AD75776DA66457'
> 2009/06/22 09:49:03| storeLockObject: (store_client.c:122): key
> '137B66F60D79074DC0AD75776DA66457' count=2
> 2009/06/22 09:49:03| storeClientCopy:
> 137B66F60D79074DC0AD75776DA66457, seen 0, want 0, size 4096, cb
> 0x42a0e0, cbdata 0xb88258
> 2009/06/22 09:49:03| cbdataLock: 0xb88258
> 2009/06/22 09:49:03| cbdataLock: 0xb9a4b8
> 2009/06/22 09:49:03| storeClientCopy2: 137B66F60D79074DC0AD75776DA66457
> 2009/06/22 09:49:03| storeClientCopy3: Waiting for more
> 2009/06/22 09:49:03| cbdataUnlock: 0xb9a4b8
> 2009/06/22 09:49:03| aclCheckFast: list: (nil)
> 2009/06/22 09:49:03| aclCheckFast: no matches, returning: 1
> 2009/06/22 09:49:03| fwdStart:
> 'http://xxxxxxxxxxxxxxx/3705584/SD-Mat07.pdf'
> 2009/06/22 09:49:03| storeLockObject: (forward.c:964): key
> '137B66F60D79074DC0AD75776DA66457' count=3
> 2009/06/22 09:49:03| peerSelect:
> http://xxxxxxxxxxxxxxxxx/3705584/SD-Mat07.pdf
> 2009/06/22 09:49:03| storeLockObject: (peer_select.c:155): key
> '137B66F60D79074DC0AD75776DA66457' count=4
> 2009/06/22 09:49:03| cbdataLock: 0xb9a548
>
> Here response of POST request
> Date: Mon, 22 Jun 2009 02:52:42 GMT
> Server: Apache
> Expires: Thu, 19 Nov 1981 08:52:00 GMT
> Cache-Control: no-store, no-cache, must-revalidate, post-check=0,
> pre-check=0
> Pragma: no-cache
> Content-Disposition: attachment; filename="SD-Mat07.pdf";
> Last-Modified: Tue, 03 Mar 2009 01:21:22 GMT
> Content-Length: 85654
> ETag: "800023d7dcac-14e96-4642cc0c02880"
> Connection: close
> Content-Type: application/pdf
>
> at last minute of downloading, status store_status=OK
>
> 2009/06/22 09:49:19| storeSwapOut:
> http://xxxxxxxxxx/downloadfiles/3705584/SD-Mat07.pdf
> 2009/06/22 09:49:19| storeSwapOut: store_status = STORE_PENDING
> 2009/06/22 09:49:19| storeSwapOut: mem->inmem_lo = 81920
> 2009/06/22 09:49:19| storeSwapOut: mem->inmem_hi = 86083
> 2009/06/22 09:49:19| storeSwapOut: swapout.queue_offset = 0
> 2009/06/22 09:49:19| storeSwapOutMaintainMemObject: lowest_offset = 84945
> 2009/06/22 09:49:19| fwdComplete:
> http://xxxxxxxxxxxxx/downloadfiles/3705584/SD-Mat07.pdf
> status 200
> 2009/06/22 09:49:19| fwdReforward:
> http://xxxxxxxxxxxx/3705584/SD-Mat07.pdf?
> 2009/06/22 09:49:19| fwdReforward: No, ENTRY_FWD_HDR_WAIT isn't set
> 2009/06/22 09:49:19| fwdComplete: not re-forwarding status 200
> 2009/06/22 09:49:19| storeComplete: '137B66F60D79074DC0AD75776DA66457'
> 2009/06/22 09:49:19| storeEntryValidLength: Checking
> '137B66F60D79074DC0AD75776DA66457'
> 2009/06/22 09:49:19| storeEntryValidLength: object_len = 86083
> 2009/06/22 09:49:19| storeEntryValidLength: hdr_sz = 429
> 2009/06/22 09:49:19| storeEntryValidLength: content_length = 85654
> 2009/06/22 09:49:19| storeSwapOut:
> http://xxxxxxxxxxxxxx/downloadfiles/3705584/SD-Mat07.pdf
> 2009/06/22 09:49:19| storeSwapOut: store_status = STORE_OK
> 2009/06/22 09:49:19| storeSwapOut: mem->inmem_lo = 81920
> 2009/06/22 09:49:19| storeSwapOut: mem->inmem_hi = 86083
> 2009/06/22 09:49:19| storeSwapOut: swapout.queue_offset = 0
> 2009/06/22 09:49:19| storeSwapOutMaintainMemObject: lowest_offset = 84945
> 2009/06/22 09:49:19| InvokeHandlers: 137B66F60D79074DC0AD75776DA66457
> 2009/06/22 09:49:19| InvokeHandlers: checking client #0
> 2009/06/22 09:49:19| comm_close: FD 35
> 2009/06/22 09:49:19| commCallCloseHandlers: FD 35
> 2009/06/22 09:49:19| commCallCloseHandlers: ch->handler=0x4489a0
> 2009/06/22 09:49:19| cbdataValid: 0xb8f768
> 2009/06/22 09:49:19| storeUnlockObject: (http.c:75): key
> '137B66F60D79074DC0AD75776DA66457' count=3
> 2009/06/22 09:49:19| cbdataFree: 0xb8f768
> 2009/06/22 09:49:19| cbdataFree: 0xb8f768 has 1 locks, not freeing
> 2009/06/22 09:49:19| cbdataUnlock: 0xb8f768
> 2009/06/22 09:49:19| cbdataUnlock: Freeing 0xb8f768
> 2009/06/22 09:49:19| commCallCloseHandlers: ch->handler=0x438180
> 2009/06/22 09:49:19| cbdataValid: 0xb9a548
>
> But i check in var/log/store.log, status of object is RELEASE not SWAPOUT
> 1245638959.814 RELEASE -1 FFFFFFFF 137B66F60D79074DC0AD75776DA66457
> 200 sh1245639162 1236043282 375007920 application/pdf 85654/85654 POST
> http://xxxxxxxxxxxxxxxxxxx/3705584/SD-Mat07.pdf
>
> What should i do in order this object could be saved on disk ?
> Which part in source code should I modify ?

The web server is sending headers indicating the response MUST NOT be
cached. It is going to some length to ensure the object is not cached.

 Date: Mon, 22 Jun 2009 02:52:42 GMT
 Expires: Thu, 19 Nov 1981 08:52:00 GMT
 Cache-Control: no-store, no-cache, must-revalidate
 Pragma: no-cache

Any one of which will cause RELEASE.

Amos
Received on Mon Jun 22 2009 - 04:34:54 MDT