2014-11-13

The NoMemoryStream

The say there are two hard problems in computer science; cache invalidation, naming things and off by one errors. Hence I can proudly present the NoMemoryStream.

The NoMemoryStream solves a problem you may not have that often; to synchronize reads and writes within a process. For me this was useful when I had some data that I could read from one stream but then I needed to write it to another stream after some transformation. And I didn't want to read it all into memory since it was a lot of data.

I considered an option where I would create a stream that on the read (when the consumer read the data) would read from an underlying stream, transform the data and serve the original write. I could have done that with a variant of the PassThruStream but it would involve caching data in case my transformation produced more data than the consumer would read.

Hence it was just easier to create a stream that would never complete a read before some data was written and writes would never complete until all data was read. While the functionality was pretty straight forward the thing that haunted me for a long time was the name. What do I name this thing?

My first thought was SynchronizedStream but then that felt misleading since a lot of things are asynchronous in how it works. ReadWriteStream didn't feel right either since a lot of streams are read/writable. So in the end I settled for NoMemoryStream since it kind of is a MemoryStream except it really doesn't have an internal buffer. It just uses what ever buffer the read and write calls are providing.

However if you can come up with a better name; please let me know! Oh, and if you din't get it yet, the NoMemoryStream is part of my toolbox in the streams package.

4 comments:

  1. My first bet would be: ReadWriteCycleStream or WriteThenReadSingleBufferStream which would be quite explicit.

    ReplyDelete
    Replies
    1. But the stream also supports "ReadThenWriteSingleBufferStream"...

      Delete
  2. Sounds less of a stream but more like a producer consumer scenario. How about ProducerConsumerStream? There even exists an implementation for it: https://github.com/FrozenCow/sharpfilesystem/blob/master/SharpFileSystem/IO/ProducerConsumerStream.cs

    ReplyDelete
    Replies
    1. ProducerConsumerStream is not a bad name!
      But I do notice that the linked implementation does not work the same way my implementation does since it allows for data to be cached inside the stream. I.e. writes can complete without having a reader waiting for data. My implementation will block both reads and writes until there is a matching call.

      Delete