Re: [Haskell-beginners] R/W from/to partitions (on linux)

i'm trying to get for which purpose BS.strict and BS.lazy are adapted. am i right to think that: lazy bytestrings are not optimal when needing to read huge files of several gigabytes, bc they are only lazy to the extent that the very long bytestring is not read to the end. aka it's like haskell's natural laziness, if i compute [0..2^20] == [0..2^20] it won't just take a lot of time but will also (try to) keep in memory both huge lists, aka the operation is not intrinsically on minimal resources, comparing each item of each list and discarding the previous ones at the same time; more generally the lists or bytestrings, be they lazy, are still only deallocated when the whole variable (the whole list/array) is dereferenced. thus if i need say to read very huge files and compare them together block by block, it's better for me to use strict bytestrings for each pair of chunks and manually move the handle from one block to the other, looping over the whole file. correct? 2017-06-26 12:53 GMT+02:00 Silent Leaf :

yes, conduits really seem optimal! i'll probably use that in my final version, thanks! however i'll first try without using pre-made tools, to get the handle (ha) of manually using (binary) files with haskell, as after all my program isn that complicated.

2017-06-26 10:42 GMT+02:00 Stefan Risberg :

...

I would use some streaming library instead of lazy bytestring to keep memory at reasonably low levels. It will also help with reading chunks, and then composing actions on it.

For library need you got: conduits, iostreams and pipes

On 26 Jun. 2017 10:37, "Silent Leaf" wrote:

i'm reading on the doc of BS.Lazy.hGetContents: "Once EOF is encountered, the Handle is closed."

what does that imply if i'm using it inside of withFile? no risk of getting prematurely out of the function right? that doesn seem possible in a pure function but i'm asking either way. if say i do something that reads the whole file, say calculating its length, does it mean since EOF will be reached i'll have to open the file again? i think i'm a bit lost...

i'm trying to find how to read big chunks of two files, do stuff with each pair of chunk, and so on till the EOF, which may or may not happen at the same time for both... i don't really know how lazy bytestrings handle, for example, taking too much from a file. one way would be to calculate the length, ofc, but for files (partition) of several dozens of gigabytes, it's a bit delicate... the ideal would be to get the length from the system itself rather than calculate the whole string ...

2017-06-26 7:57 GMT+02:00 Silent Leaf :

...

Darn quick answer! Thanks Sylvain, that may be all i need to start!

2017-06-26 7:51 GMT+02:00 Sylvain Henry :

...

Hi,

It is not Haskell specific. You just have to read from the partition device special file (e.g., something like /dev/sdb2) as you would do with a normal file. You must have the permission to do so (e.g., be root). Be careful as you can destroy your system if you write something incorrect in your partitions.

Repositioning handles: https://www.stackage.org/haddo ck/lts-8.20/base-4.9.1.0/System-IO.html#g:13

Read/write: https://www.stackage.org/haddo ck/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf

Sylvain

On 26/06/2017 07:35, Silent Leaf wrote:

Hi,

I'd like to be able to read and write from/to partitions directly. I've had trouble with the documentation (honestly i can't find anything, and any mention of partitions leads to mathematical partitioning of lists or whatever).

I obviously would need to be able to write or read from a specific position in the partition. Mind you that would be good too for files (that is, being able to read/write from a specific position in it) since i plan on making disk images.

Thanks in advance!

_______________________________________________ Beginners mailing listBeginners@haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/beginners

_______________________________________________ Beginners mailing list Beginners@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners

_______________________________________________ Beginners mailing list Beginners@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners

_______________________________________________ Beginners mailing list Beginners@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners