Re: StringNg review: MemBlob

On 10/19/2010 08:40 AM, Kinkie wrote:
> On Tue, Oct 19, 2010 at 4:35 PM, Alex Rousskov
> <rousskov_at_measurement-factory.com> wrote:
>> On 10/19/2010 08:08 AM, Kinkie wrote:
>>>
>>> On Mon, Oct 18, 2010 at 7:56 PM, Alex Rousskov wrote:
>>>
>>>> Once MemBlob is committed, please consider re-integrating it with memory
>>>> pools. This is the biggest remaining XXX, IMO.
>>>
>>> I've checked and confirmed why MemPools integration was disabled: it's
>>> due to an inherent asymmetry in memAllocString/memFreeString: if
>>> mempools are not initialized, memAllocString will happily allocate the
>>> string using xcalloc. When the time comes to deallocate such strings,
>>> it will fail to find a pool to dealloc from and fail an assertion.
>>>
>>> The only way I can think of to cleanly resolve this is quite
>>> disruptive, as it means making MemPools a proper singleton obect to
>>> make sure they are initialized before being used: it's a complex
>>> project on its own which I'd list as a dependency before integrating
>>> MemPools back in.
>>
>> What happens if you call Mem::Init() from memAllocString()? I am sure a few
>> things will need to be fixed (e.g., Mem::Init() should exit if called
>> twice), but it does not sound disruptive to me.
>
> Tried that, using a string-pool as singleton-like initialization
> indicator. There are some prerequisites to be called in turn. I have
> not investigated further; I can investigate further if you wish.

I think this must be fixed, but I do not think this has to be fixed
before MemBlob is committed.

>>> IMVHO body pipelining should go through SBuf, not MemBlob, but that's
>>> for body pipelining to work out.
>>
>> Currently, MemBlob's append-only design prevents efficient message body
>> pipelining. Since SBuf relies on MemBlob, SBuf inherits the same problem
>> (and adds more problems due to poor used area tracking). We will need to
>> decide whether to enhance MemBlob/SBuf to improve pipelining support or
>> switch to a pipeline that uses a "bunch of buffers" instead of a single one.
>> This is a topic for another thread though.
>
> Yes. Unfortunately I don't understand what is the use case you have in mind :(

See BodyPipe and BodyPipe::theBuf use specifically.

> My original thought was to use SBuf's append/consume pair for this.

Unlike MemBuf used by BodyPipe, SBuf::consume() does not free any space
and, hence, cannot be used for efficient body pipelining (i.e., using a
relatively small buffer to transfer a possibly large body using
asynchronous producer and consumer).

Hope this clarifies,

Alex.
Received on Tue Oct 19 2010 - 15:04:42 MDT