Hi
Actually, you ought to be able to pretty easily remove this tradeoff by introducing a strict read function as a new method in your class. So anyone who wants to strictly read lazy data can use that function instead of the lazy one.
Not quite, the library is written so that strict fields are at the current file pointer while deferred fields require a position jump. For example, two strict strings would be: "hello""world" But two lazy strings would be: [ptr 5][ptr 10]"hello""world"
:( Lazy reading seems to require strict writing, while lazy writing requires strict reading!
The library is all designed around making reading maximally efficient, and minimizing file seeks, but writing can be totally inefficient. As you mention, the lazy/strict IO trade off is very interesting, its just fortunate that for my application (Hoogle) the files are written once and read many many times.
I'm wondering if it would be an option to read the file backward. If so length annotations could be put at the end.
Reading backward seems to only fail with streamed data, which won't support the "seek"ing required by a lazy reading anyway.
You still want to minimize seeking, even if it is possible. In my case having length information at the front helps reading, and I don't care about writing, so its an easy win. I think I've got my design nailed down, and have most of it implemented and used in Hoogle. These discussions and points were very helpful! Thanks Neil