July 2003 - Libraries

RE: monad library
by Simon Marlow 30 Jul '03

30 Jul '03

Iavor Diatchki writes: > i am not sure what should we put unedr Monad.* if not the > monad library. When you take a top-level name for a library, the programmer is prevented from using that top-level name for modules in a program. I'd prefer to keep the top-level Monad name for monads in the current program, rather than a generic Monad library. Also, I'm in favour of using the hierarchy for classification (within reason). It's not an exact science - there always seems to be several reasonable ways to organise things, but on the whole I believe libraries are easier to find when they're classified by functionality in a hierarchy. So my vote goes to keeping the current Control.Monad name for the library. Cheers, Simon

2 1

RE: Proposal for a new I/O library design
by Simon Marlow 30 Jul '03

30 Jul '03

> But the upshot of the discussion here is that there's no way > to implement > a well-behaved File abstraction on Win32 or Posix, so people are just > going to have to continue writing insecure programs on a > less-elegant API. > Stop me if I start to sound bitter. :-) I'm more familiar with Unix than Win32, but it seems to me that if you're willing to accept that the File abstraction includes access rights, then you can have Files with the identity properties you want. Equality can even ignore the access rights. You're right that dCreateFileEntry cannot be implemented on Unix, however. I think this is a killer for the Directory abstraction. I think there is merit in other parts of the proposal, though. Separating streams from the underlying File object seems worthwhile: - You can have several streams open on a single File - Streams make sense for other underlying objects (eg. Sockets) - You don't have to pretend that all streams are files (eg. stdout, stdin and stderr are just streams) And separating Input and Output streams also looks worthwhile - as the implemntor of GHC's IO library I know this would simplify things quite a bit, and to me it also looks nicer from the programmer's perspective. Cheers, Simon

1 0

RE: Parsing CSV files
by Simon Peyton-Jones 30 Jul '03

30 Jul '03

| > Hey, I wrote a CSV parser too. seeing as how this seems like a common | > thing (having 3 independent implementations at least), perhaps it | > belongs in libraries somewhere? some area dedicated to useful little | > grammers would be handy. csv, c header files, .x (rpcgen), various | > preference file formats, etc... | | What haskell needs is something along the lines of CPAN. Contributed modules | of various utility. Not part of the base libraries, but never-the-less useful | and maybe needed for your project. There's a lot of activity on this front on the libraries mailing list (libraries(a)haskell.org) http://haskell.org/hawiki/LibraryInfrastructure Simon

1 0

RE: Proposal for a new I/O library design
by Ben Rudiak-Gould 30 Jul '03

30 Jul '03

On Mon, 28 Jul 2003, Simon Marlow wrote: > I'm concerned about one implementation difficulty. Your File type is > independent of the filesystem. That is, on Unix it corresponds to an > inode. Creating a File must correspond to "opening" it (in Unix speak). > Creating a stream corresponds to duplicating the file descriptor (you > could probably avoid too many unnecessary dups by being clever). > There's a potential implementation difficulty, though: > lookupFileByPathname must open the file, without knowing whether the > file will be used for reading or writing in the future. I know; I'm hoping against hope that this isn't an insurmountable problem. If the OS provides a "reopen" function which is like open except that it takes a file handle instead of a pathname, then I think implementation is straightforward: a File contains a handle with minimal access permissions and maximal sharing permissions, and when a read or write operation is attempted we open a second handle based on the first with additional permissions. There's a Win32 function called "ReOpenFile" with this functionality, but it's only in Windows Server 2003. Sigh. If there's a way to open files by unique ID instead of pathname, that would also work. I think the NT API might provide something like this, but looking through the online documentation just now I can't find anything of the sort. If it's not possible to provide a guarantee of File identity then we should probably drop the whole idea of File values. See my comments under Directory below. > So I would suggest that operations which create a value of type File > take a read/write flag too. This would break the conceptual identity between a File value and a file, since read-only access is not a property of a file. (Well, it can be, but it isn't in this case.) Functions which allowed access rights to be specified would have to return a FileAccessPath instead of a File, and a FileAccessPath is basically a handle, so we're back where we started. All we need here is a way to change the access and sharing rights on an already-open handle. I find it hard to believe that after decades of use by millions of people, the UNIX file API provides no way to do this safely. Maybe there's an fcntl or something? > > > type FilePos = Word64 > > > type BlockLength = Int > > FilePos should be Integer. Seems reasonable. > > > fCheckRead :: File -> FilePos -> BlockLength -> IO Bool > > > fCheckWrite :: File -> FilePos -> BlockLength -> IO Bool > > What do these do? If they're supposed to return True if the required > data can be read/written without blocking, then I suspect that they are > not useful. They're supposed to return True if the data can be read/written successfully, the idea being that this is how you check whether you have read/write access to the file. Probably I should have omitted the second and third arguments. > I'd use the traditional 'isEOF' way of detecting end of file. Seems reasonable. (Should be "EOS" though, I think.) > On naming: it's probably not a good idea to use the 'is' prefix, since > it is already used for predicates (meaning literally 'is' rather than an > abbreviation for 'InputStream'). I agree completely. Come up with something better and I'll second it. :-) (How about renaming Streams to Channels? Then we could use "ic" and "oc".) > You will also want a way to get back from an InputStream to the > underlying object, eg. the (File,FilePos) pair if one exists. Agreed. > It's not pretty, but you certainly want a way to close a stream. > Finalizers aren't reliable enough. What are the practical problems with relying on finalizers? As far as I can see, the "no more filehandles available" problem is completely solved by forcing a major GC and trying again when it occurs. The only other issue I see is leaving other processes unable to access the file for an indeterminate period of time. The right solution to this, if it can be implemented, is something like withExclusiveWriteAccess :: File -> IO a -> IO a, with write access being non-exclusive (or even disallowed?) otherwise. > How did you intend text encodings to work? I see several possibilities: > > textDecode :: TextEncoding -> [Octet] -> [Char] > > or > > decodeInputStream :: TextEncoding -> InputStream -> TextInputStream > getChar :: TextInputStream -> IO Char > etc. > > or > > setInputStreamCoding :: InputStream -> TextEncoding -> IO () > getChar :: InputStream -> IO Char I was thinking of the second. It could easily be implemented as the third under the hood. But I was hoping someone else would worry about it. :-) > > > data Directory -- abstract > > I don't see a reason for changing the existing Directory support > (System.Directory). Could you give some motivation here? Is the idea > to abstract away from the syntax of pathnames on the platform (eg. > directory separator characters)? If so, I'm not sure it's worthwhile. > There are lots of differences between pathname conventions: case > sensitivity, arbitrary limits on the lengh of filenames, filename > extensions, and so on. Basically, the usual interface encourages programmers to treat pathnames as file/directory identifiers, even though they aren't. This is the root cause of a whole class of security vulnerabilities (not to mention some everyday annoyances). I want to avoid those vulnerabilities in the Haskell model by providing values that *really are* file and directory identifiers. Pathnames have one good property: they're human-readable and -writable. That's their only good property. Within an application, they should be converted immediately to a more secure internal representation. (And the conversion should be done exactly once -- any more and you're opening yourself to security exploits.) This is why I really don't want to use the File concept unless we can guarantee file-File identity. A system that appears to be secure but actually isn't is even worse than one which is obviously insecure. This idea isn't complete unless the model also supports persistence of File and Directory values, but I didn't even bother drawing up an API for this because I'm sure it's impossible to implement. Any sane OS would provide support for this, but I don't think any widespread OS does. There should also be a DirectoryEntry type, but, again, I'm pretty sure that this can't be implemented. On reflection I think the (Directory, Maybe String) return value is a mistake. The intent was to support creating a new file or directory by pathname, but that's probably better done by functions like those you propose below. (The return value was originally supposed to be Either DirectoryEntry (Directory,String), which made more sense.) > > > lookupFileByPathname :: String -> IO File > > Here, I suggest we need > > lookupFileByPathname :: FilePath -> IOMode -> IO File If so, it should be called something like "newFileAccessPath" or at least "lookupFileAccessPath". > > > lookupInputStreamByPathname :: String -> IO InputStream > > > -- at least as likely to succeed as lookupFileByPathname > > and similarly > > createFileOutputStream :: FilePath -> IO OutputStream > appendFile :: FilePath -> IO OutputStream Definitely. -- Ben

4 7

RE: Proposal for a new I/O library design
by Graham Klyne 29 Jul '03

29 Jul '03

At 17:50 29/07/03 +0100, Simon Marlow wrote: >John Meacham writes: > > please can we figure out portable binary IO before worrying > > about i18n? > >But I thought Ben's proposal included portable binary I/O? >Input/OutputStreams do I/O in terms of Octets, not Chars. It's >important to at least think about how text tranformations are layered on >top of the binary I/O layer, because making a wrong decision could be >costly in terms of performance later. The W3C "charmod" document has some useful background for this kind of thing: http://www.w3.org/TR/charmod/ (Which I think is in support of your point.) #g ------------------- Graham Klyne <GK(a)NineByNine.org> PGP: 0FAA 69FF C083 000B A2E9 A131 01B9 1C7A DBCA CB5E

1 0

RE: Proposal for a new I/O library design
by Simon Marlow 29 Jul '03

29 Jul '03

John Meacham writes: > please can we figure out portable binary IO before worrying > about i18n? But I thought Ben's proposal included portable binary I/O? Input/OutputStreams do I/O in terms of Octets, not Chars. It's important to at least think about how text tranformations are layered on top of the binary I/O layer, because making a wrong decision could be costly in terms of performance later. Cheers, Simon

2 1

System.Directory (was RE: Proposal for a new I/O library design)
by Hal Daume 29 Jul '03

29 Jul '03

Hi guys, I'm not replying to anything in the message, but... > Is the idea > to abstract away from the syntax of pathnames on the platform (eg. > directory separator characters)? If so, I'm not sure it's worthwhile. > There are lots of differences between pathname conventions: case > sensitivity, arbitrary limits on the lengh of filenames, filename > extensions, and so on. Would there be any way to get some of these differences into the System.Directory structure? At least the following would be nice: > pathSeparator :: Char > '\\' on Windows, '/' on unices, ':' (I believe) on macs, etc... > directorySeparator :: Char > ';' on Windows, ':' on unices, i have no idea on macs > isCaseSensitive :: Bool > False on Windows, True on (all?) unices, i have no idea on macs given just these, i think we'd all be a lot happier. I also don't particularly care whether these are IO operations or just values (so long as they are constant, they might as well be values with unsafePerformIO wrapped around them if necessary). My current approach to figuring this out is to create a directory, change to that directory, get the current path name and try to parse it. This is bad for so many reasons I won't enumerate them here. ...unless this stuff is hiding somewhere else, please let me know (but System.Directory would probably be a good place for it to end up)... - Hal

2 1

ENQUIRY
by MR. DAVID LARSON 25 Jul '03

25 Jul '03

FROM: MR. DAVID LARSON ANTHONY EZE & CO. SIR, SUBJECT: TRUST NEED YOUR KIND RESPONSE TO THE FOLLOWING REQUEST: I HAVE A MANDATE TO SOURCE FOR A FINANCIAL MANAGEMENT PERSONNEL ASIDE AFRICA, BE IT YOUR PROFFESSION OR NOT, ITS NATURE DOES NOT COUNT, BUT ONLY HONESTY IS NEEDED AT ITS PEAK AND WANTED AS MOST PRIORTY TO ACTUALIZE THIS EXPECTED GOAL. HAS CLOSE LINK WITH THE PERSONNELS IN THE CORRIDORS OF POWER IN MY COUNTRY HAS BEEN DELEGATED TO SOURCE FOR A COMPETENT ASSOCIATE OUTSIDE AFRICA IF YOU ARE WORTHY OF BEEN TRUSTED WITH HUGE CAPITAL FOR ESTABLISHMENT IN YOUR COUNTRY CONTACT ME FOR FULL PERSONAL SECRETIVE INFORMATION OR DETIALS FOR COMMENCEMENT AND TERMS OF REWARD FOR YOUR EXPERTED INTEREST AND CONTRIBUTION. THE VOLUME OF CAPITAL WILL BE DISCLOSED TO YOU ON MY RECEIPT OF YOUR RETURN CORRESPONDENCE INDICATING YOUR WILLINGNESS TO PER-TAKE. KINDLY RESPOND TO THIS MAIL INCLUDING YOUR PERSONAL CONTACT DATA'S FOR FULL DISCLOSURE OF PROCEEDURES. JUST WAITING FOR YOUR REVERT THANKS MR. DAVID LARSON

1 0

Packaging, and GUIs
by Simon Peyton-Jones 23 Jul '03

23 Jul '03

Dear Haskell library and GUI folk, Here's an interesting message from the Caml mailing list, largely focusing on a) packaging b) GUIs Interestingly, the very two topics on which we have active groups working! I wonder if it'd be a good time for a status report from both efforts? Both have been very quiet of late, but I see that Daan and Krasimir have both published new GUI libraries recently, so GUI stuff is still going on. Simon -----Original Message----- From: owner-caml-list(a)pauillac.inria.fr [mailto:owner-caml-list@pauillac.inria.fr] On Behalf Of Chris Clearwater Sent: 18 July 2003 22:21 To: caml-list(a)inria.fr Subject: [Caml-list] Roadplan for world domination (or constructive criticism of ocaml facilities) I have come across the Ocaml language several months ago and after much investigation and trial coding I have come to several conclusions. Most importantly I think Ocaml is a wonderful language to program in. It has all the features I would expect from a modern language and on top of that it's compiled native code appears to rival that of C in many areas. It truly is the language of the future. Now that I have expressed how much I am growing to love Ocaml I would like to share with you my initial experiences as a new user. First I would like to single out some issues that I believe threaten Ocaml from being taken seriously and then i would like to offer some solutions that would improve the usability of ocaml greatly. - Support on win32 could certainly be much better - Compiling and distributing Ocaml source is a very delicate process - There exists much fragmentation among usage of different GUI toolkits and they are limited to C/C++ conventions. (Is it possible to create your own custom GTK widgets within ocaml?) And without further ado I present to you 10 steps to world domination: 1) Support for Ocaml on win32 (both as a development and target enviroment) is very crucial for the adoption and practicality of Ocaml. For example, look at the trouble some developers must go through to get their application running under win32: http://groups.yahoo.com/group/unison-hackers/message/31. Forutnately this doesn't have to be the case. The mingw32 toolset allow compilation and linking to be done very similarily to how it is done on Unix. Also, it enables one to develop win32 apps without shelling out hundeds of dollars to Microsoft: http://msdn.microsoft.com/visualc/howtobuy/pricing.aspx I propose mingw32 be made the default compiler/linker for native win32 binaries (or even drop MSVC support entirely). 2) Take the idea of ocamlmklib further to generalize the compilation of both Ocaml programs and libraries into a module called "Ocamlmake". Also create a binary of ocamlmake which makes use of the module for command line compiling and include these in the standard distribution on all platforms. 3) Now that we have a easy cross-platform way to compile ocaml applications we can just distibute our code with Makefiles that call ocamlmake! WRONG. Makefiles suck. Now we standardize on the idea of an Ocaml "package". Each package would include in it's toplevel directory a file called setup.ml (This is starting to resemble python's distutils indeed). setup.ml would make use of the Ocamlmake module by building a record and passing it to Ocamlmake.setup. This record might be static or it might be created by self-configuration. For example, the ocamlsdl package would call sdl-config to retrieve some compilation flags. You _would_ rather configure in ocaml than "portable shell script", right? Then to build the application you would execute "ocamlrun setup.ml build" and "ocamlrun setup.ml install" to install it. Also the record would contain meta-information such as the author, copyright, etc. 4) Change the ocaml distribution to compile using Ocamlmake :) (except for bootstrapping if you dont already have a previous version of ocaml installed) 5) Create a module called Framebuffer which parallels the primtives found in OpenGL/DirectX. The Graphics module is close, but the design doesn't match well with these two APIs (We want hardware acceleration). Implement for each platform a Framebuffer module (DirectX or OpenGL where available, Xlib or other native graphics system otherwise). Include this in the standard distribution. 6) Create a cross-platform Event module. Include this in the standard distribution. 7) Create a cross-platform Font module (wrap freetype or create an Ocaml implementation). Include this in the standard distribution. 8) Create a GUI on top of the Framebuffer, Event, and Font modules, implemented in Ocaml :) Include this in the standard distribution. 9) Now the big payoff, we write a standard Ocaml IDE, to be included with the Ocaml distribution. It would be well integrated with the distribution. Ocamlmake module for compiling, the ocaml lexer for syntax highlighting, exporting packages (setup.ml). This would make it incredably easy to get started creating cross-platform libraries and modules. Realating to point #1, now a win32 user need only grab the mingw32 and Ocaml distributions and they are set. They can even easily export their code to a ocaml package for distribution. Obviously the benefit extends to all other platforms as well. 10) World domination. I would greatly appreciate any feedback. ------------------- To unsubscribe, mail caml-list-request(a)inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners

2 1

Packaging
by Simon Peyton-Jones 23 Jul '03

23 Jul '03

Here's another thread on packaging from the OCaml mailing list: http://caml.inria.fr/archives/200307/msg00193.html (go to the thread to see more). I don't know what it has to contribute to our packaging discussion, but someone informed might want to mine it for useful ideas. Simon

1 0