Hello Simon, Simon, are you remember problem with using hTell+hSeek on handles open in text (not binary) mode on Windows? afair, you was finished with the decision to use NoBuffering for text files on Windows? but this solution is very inefficient. i now thought about dealing with the same problem in my lib and found that there is another solution - prohibit using of hTell/hSeek on files open in text mode (on Unix, too?). i think this is better - one should either open file in binary mode and use random access or open file in text mode and read/write it sequentially. what you think about it? also, i will be glad to hear comments from other haskellers -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
On Thu, 2006-07-27 at 15:13 +0400, Bulat Ziganshin wrote:
Hello Simon,
Simon, are you remember problem with using hTell+hSeek on handles open in text (not binary) mode on Windows? afair, you was finished with the decision to use NoBuffering for text files on Windows?
but this solution is very inefficient. i now thought about dealing with the same problem in my lib and found that there is another solution - prohibit using of hTell/hSeek on files open in text mode (on Unix, too?). i think this is better - one should either open file in binary mode and use random access or open file in text mode and read/write it sequentially. what you think about it?
also, i will be glad to hear comments from other haskellers
According to MS documentation: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wceappdev5/... For streams opened in text mode, fseek has limited use, because carriage return — linefeed translations can cause fseek to produce unexpected results. The only fseek operations guaranteed to work on streams opened in text mode are as follows: * Seeking with an offset of 0 relative to any of the origin values. * Seeking from the beginning of the file with an offset value returned from a call to ftell It's a good point that you might want to seek to a point previously remembered with hTell, so we probably don't want to ban it entirely. I could not find any documentation in the Win32 API describing the behaviour of text files vs binary files. Is this translation behaviour only implemented at the MS C library layer? Duncan
Hello Duncan, Friday, July 28, 2006, 2:52:10 PM, you wrote:
Simon, are you remember problem with using hTell+hSeek on handles open in text (not binary) mode on Windows? afair, you was finished with the decision to use NoBuffering for text files on Windows?
but this solution is very inefficient. i now thought about dealing with the same problem in my lib and found that there is another solution - prohibit using of hTell/hSeek on files open in text mode (on Unix, too?). i think this is better - one should either open file in binary mode and use random access or open file in text mode and read/write it sequentially. what you think about it?
also, i will be glad to hear comments from other haskellers
According to MS documentation:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wceappdev5/...
For streams opened in text mode, fseek has limited use, because carriage return — linefeed translations can cause fseek to produce unexpected results.
The only fseek operations guaranteed to work on streams opened in text mode are as follows:
* Seeking with an offset of 0 relative to any of the origin values. * Seeking from the beginning of the file with an offset value returned from a call to ftell
It's a good point that you might want to seek to a point previously remembered with hTell, so we probably don't want to ban it entirely.
I could not find any documentation in the Win32 API describing the behaviour of text files vs binary files. Is this translation behaviour only implemented at the MS C library layer?
after writing this letter, i added one more comment to my internal documentation - "hRewind and seeking to EOF will be also useful". about fseek - i don't see an easy way to implement it. if your translated data are already in buffer, then fseek can't return just file position corresponding to buffer start + current offset in buffer. btw, the same problem arises when we want to use UTF-8 or some other Char<->Byte translation with buffered I/O (using iconv, f.e.) so, for translated text 1) we can implement hRewind/hSeekToEOF 2) we can implement some form of hSavePosition/hRestorePosition if this position will include tuple (file position of buffer start, offset in buffer) 3) we can implement tell/seek functionality but this will be slow -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
On Fri, 2006-07-28 at 15:10 +0400, Bulat Ziganshin wrote:
after writing this letter, i added one more comment to my internal documentation - "hRewind and seeking to EOF will be also useful". about fseek - i don't see an easy way to implement it. if your translated data are already in buffer, then fseek can't return just file position corresponding to buffer start + current offset in buffer. btw, the same problem arises when we want to use UTF-8 or some other Char<->Byte translation with buffered I/O (using iconv, f.e.)
so, for translated text
1) we can implement hRewind/hSeekToEOF 2) we can implement some form of hSavePosition/hRestorePosition if this position will include tuple (file position of buffer start, offset in buffer) 3) we can implement tell/seek functionality but this will be slow
How about an inefficient seek that just does a linear scan from the beginning of the file counting translated characters? Then the odd case of seek on a text file is penalised rather than doing no buffering on text files which penalises the normal case. Duncan
Bulat Ziganshin wrote:
Simon, are you remember problem with using hTell+hSeek on handles open in text (not binary) mode on Windows? afair, you was finished with the decision to use NoBuffering for text files on Windows?
Not quite - buffering is still used on text files, *unless* the file is opened in ReadWriteMode, in which case we force the buffering to NoBuffering. This is because when used with buffering, ReadWriteMode may need to seek internally. I believe hTell/hSeek will work properly on an output file (hTell forces a flush on Windows). They probably don't work properly with a ReadMode text file.
but this solution is very inefficient. i now thought about dealing with the same problem in my lib and found that there is another solution - prohibit using of hTell/hSeek on files open in text mode (on Unix, too?). i think this is better - one should either open file in binary mode and use random access or open file in text mode and read/write it sequentially. what you think about it?
also, i will be glad to hear comments from other haskellers
The single combination of (hTell + buffering + read mode + text translation) should be disallowed, but I think all other variants can be supported. Cheers, Simon
participants (3)
-
Bulat Ziganshin -
Duncan Coutts -
Simon Marlow