Hi all, Currently the unix package is a mishmash of different solutions to the portability problem. Here are all the ones I've found, along with what I think the best action would be. If this sounds reasonable then I'll start making patches and proposals as appropriate. (for POSIX/SUS, see http://www.opengroup.org/onlinepubs/009695399/) 1) Values outside of the scope of the package, conditionally available: * System.Posix.Signals.Exts.* Propose: Moving modules into a new package, unix-exts perhaps. 2) Things that fall back to probably-reasonable alternatives: * unsetEnv, setEnv * PATH_MAX Propose: Leave as they are. 3) Things falling back to functionality which can cause problems: * The Haskell usleep uses the C usleep (marked Obsolescent) if nanosleep (a Timer extension) isn't available. C's usleep interacts badly with GHC's threaded RTS. Propose: Leave as it is. 4) Things that raise errors at runtime if they are not available, but there is something we could fall back to: * getGroupEntryForID (getgrgid_r, thread-safe extension) * getGroupEntryForName (getgrnam_r, thread-safe extension) Propose: Write fallback code. 5) Things that raise errors at runtime if they are not available, but as far as I can see are required to be available anyway: * _PC_* in System/Posix/Files.hsc * getUserEntryForID (getpwuid) * getUserEntryForName (getpwnam) * grBufSize (sysconf, _SC_GETGR_R_SIZE_MAX) * pwBufSize (sysconf, _SC_GETPW_R_SIZE_MAX) * sysconfWithDefault (sysconf) Propose: Change to using them unconditionally and see if any platforms break. 6) Things depending on optional functionality, conditionally available: * setSymbolicLinkOwnerAndGroup depends on lchown, an XSI extension http://www.opengroup.org/onlinepubs/009695399/functions/lchown.html * RLIMIT_AS (ResourceTotalMemory), RLIM_SAVED_MAX, RLIM_SAVED_CUR (all of the rlimit stuff actually appears to be an XSI extension). Propose: Move to unix-xsi package. 7) Things that raise errors at runtime if they are not available: * getAllGroupEntries (getgrent, XSI extension) * getAllUserEntries (getpwent, setpwent, endpwent, XSI extensions) * Various in DynamicLinker.Prim Propose: Move to unix-xsi package. (for both 6 and 7, an alternative would be to just make them unconditionally used in the unix package; I'm not sure what platforms, if any, this would break on). 8) Things that are implemented only for certain compilers: * forkProcess (GHC-only) Propose: Write implementations for other compilers/interpreters. Thanks Ian
From: libraries-bounces@haskell.org [mailto:libraries-bounces@haskell.org] On Behalf Of Ian Lynagh
Currently the unix package is a mishmash of different solutions to the portability problem. Here are all the ones I've found, along with what I think the best action would be. If this sounds reasonable then I'll start making patches and proposals as appropriate.
This is outside the scope of Ian's proposal, but I was wondering... The unix package isn't available to Windows users. Unsurprising, maybe. But, given that it seems to be a layer over the Posix API, would it be reasonable to request that the unix package is made available on Windows? There is a Posix API in Windows (NT/2000/XP), although I understand it is next to useless. There is also this: http://sourceforge.net/projects/pw32/ and possibly some other projects which provide the necessary functionality on Windows (cygwin? MSYS?). Alistair ***************************************************************** Confidentiality Note: The information contained in this message, and any attachments, may contain confidential and/or privileged material. It is intended solely for the person(s) or entity to which it is addressed. Any review, retransmission, dissemination, or taking of any action in reliance upon this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer. *****************************************************************
Bayley, Alistair wrote:
The unix package isn't available to Windows users. Unsurprising, maybe. But, given that it seems to be a layer over the Posix API, would it be reasonable to request that the unix package is made available on Windows?
I've been thinking about just this, too.
The underlying problem is that Haskell's portable I/O-related libraries
are underpowered, particularly the stuff in System.Directory. Try doing
any significant filesystem-related work using only the portable APIs and
you'll see what I mean.
A portable library that put a POSIX-like wrapper atop filesystem
operations would be a big win, particularly if it were distributed as a
standard library. This is the approach that Python's standard libraries
take, and it works well.
If I had a vote on how Ian might spend his time, it would be on
something like this, which would have direct value to programmers by
addressing a significant deficit of the current standard libraries.
Splitting up the unix package seems more like an inconsequential
janitorial task. Given finite resources, I'd prefer to tackle the task
with greater benefits.
On Jun 28, 2007, at 18:25 , Bryan O'Sullivan wrote:
Bayley, Alistair wrote:
The unix package isn't available to Windows users. Unsurprising, maybe. But, given that it seems to be a layer over the Posix API, would it be reasonable to request that the unix package is made available on Windows?
I've been thinking about just this, too.
The underlying problem is that Haskell's portable I/O-related libraries are underpowered, particularly the stuff in System.Directory. Try doing any significant filesystem-related work using only the portable APIs and you'll see what I mean.
A portable library that put a POSIX-like wrapper atop filesystem operations would be a big win, particularly if it were distributed as a standard library. This is the approach that Python's standard libraries take, and it works well.
If I had a vote on how Ian might spend his time, it would be on something like this, which would have direct value to programmers by addressing a significant deficit of the current standard libraries. Splitting up the unix package seems more like an inconsequential janitorial task. Given finite resources, I'd prefer to tackle the task with greater benefits.
I started working on this for implementing the file operations needed by the tar package, but didn't take it further than that. Here's what I have so far: http://www.cs.chalmers.se/~bringert/darcs/unix-compat/ /Björn
On Thu, 2007-06-28 at 09:25 -0700, Bryan O'Sullivan wrote:
Bayley, Alistair wrote:
The unix package isn't available to Windows users. Unsurprising, maybe. But, given that it seems to be a layer over the Posix API, would it be reasonable to request that the unix package is made available on Windows?
I've been thinking about just this, too.
The underlying problem is that Haskell's portable I/O-related libraries are underpowered, particularly the stuff in System.Directory. Try doing any significant filesystem-related work using only the portable APIs and you'll see what I mean.
Yes. I've got a little Gtk2Hs demo that shows you a directory tree with file names and file sizes. With the portable Haskell directory stuff I can't even get the file size, I'd have to open every file and use hFileSize.
A portable library that put a POSIX-like wrapper atop filesystem operations would be a big win, particularly if it were distributed as a standard library. This is the approach that Python's standard libraries take, and it works well.
Yes, we know that both POSIX and Win32 have operations to 'stat' a file and get back some common attributes about it. Of course the intersection of those attributes is not huge, but it at least contains the file size!
If I had a vote on how Ian might spend his time, it would be on something like this, which would have direct value to programmers by addressing a significant deficit of the current standard libraries. Splitting up the unix package seems more like an inconsequential janitorial task. Given finite resources, I'd prefer to tackle the task with greater benefits.
Though it's also probably more work :-) It seems to me, one of our main problems is that the current base package cannot depend on the posix/unix or Win32 packages and so we cannot easily implement the portable Haskell APIs on top of the existing posix/win32 api bindings. All the portable stuff is currently implemented on top of C wrappers which interally use cppery to call posix or win32 functions. So fewer people feel able to contribute because most of the code is not Haskell. Perhaps with splitting the base package up we have the opportunity to have the directory & IO stuff live in a package that can depend on either the posix/unix or win32 packages, and implement the portable IO stuff on top of those (probably still using cpp). Duncan
On 6/30/07, Duncan Coutts
operations would be a big win, particularly if it were distributed as a standard library. This is the approach that Python's standard libraries take, and it works well.
Yes, we know that both POSIX and Win32 have operations to 'stat' a file and get back some common attributes about it. Of course the intersection of those attributes is not huge, but it at least contains the file size!
I can't help but to think this is exactly the way I don't want to see portable file IO api. I am fairly tired of Windows applications that misuse Windows file rights, and wierd software that got ported from unixy-type systems that interacts very badly because of misuse of file locking. I don't think difference in APIs is trivial. File locking and access rights work very diffrently.
If I had a vote on how Ian might spend his time, it would be on something like this, which would have direct value to programmers by addressing a significant deficit of the current standard libraries. Splitting up the unix package seems more like an inconsequential janitorial task. Given finite resources, I'd prefer to tackle the task with greater benefits.
Though it's also probably more work :-)
It seems to me, one of our main problems is that the current base package cannot depend on the posix/unix or Win32 packages and so we cannot easily implement the portable Haskell APIs on top of the existing posix/win32 api bindings. All the portable stuff is currently implemented on top of C wrappers which interally use cppery to call posix or win32 functions. So fewer people feel able to contribute because most of the code is not Haskell.
Indeed. Also, as Win32 maintainer, I am unsure how far we are supporting old Windowses. When I tentavily asked some people about it, they thought we should continue to support old Windows versions. Unfortunately file rights (in practice all of Security-API) is not available under Win 9x and Me. (But I might have broken somethin anyway, I recall ghc installer having some changes to Win32 to support old Windows versions.)
Perhaps with splitting the base package up we have the opportunity to have the directory & IO stuff live in a package that can depend on either the posix/unix or win32 packages, and implement the portable IO stuff on top of those (probably still using cpp).
I hope portable IO means something that still allows access to important details, and promotes good style. Best regards, Esa Ilari Vuokko
On Sat, Jun 30, 2007 at 08:01:09PM +0300, Esa Ilari Vuokko wrote:
Indeed. Also, as Win32 maintainer, I am unsure how far we are supporting old Windowses. When I tentavily asked some people about it, they thought we should continue to support old Windows versions.
Our (GHC) plan is not to support Win 9x (and Me?) from 6.8 onwards, as we don't have machines on which test it or debug problems. I don't think the installers that we make now install a GHC that works on Win 9x, but I could be wrong. Thanks Ian
Hello Duncan, Saturday, June 30, 2007, 8:17:05 PM, you wrote:
Yes. I've got a little Gtk2Hs demo that shows you a directory tree with file names and file sizes. With the portable Haskell directory stuff I can't even get the file size, I'd have to open every file and use hFileSize.
getFileSize isn't any harder to implement than getModificationTime, the only minoor problem is that function used (lstat) will truncate file size to low 32 bits, for full 64-bit value non-portable lstati64 should be used instead (and another type for C-level filesize too)
Perhaps with splitting the base package up we have the opportunity to have the directory & IO stuff live in a package that can depend on either the posix/unix or win32 packages, and implement the portable IO stuff on top of those (probably still using cpp).
anyway file operations best should be split away from base. the only problem is that Handle type used in definition of Exception which is used in definition of IO monad which is used in Prelude (and some Prelude functions need file i/o by itself) i once proposed plan of implementing stack of file system/io libraries http://haskell.org/haskellwiki/Library/IO , also portable base for file i/o and memory-mapped files i/o implemented in Streams library: see System.FD and System.MMFile modules in http://www.haskell.org/library/StreamsBeta.tar.gz -- Best regards, Bulat mailto:Bulat.Ziganshin@gmail.com
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bulat Ziganshin wrote:
anyway file operations best should be split away from base. the only problem is that Handle type used in definition of Exception which is used in definition of IO monad which is used in Prelude (and some Prelude functions need file i/o by itself)
Maybe we will end up having to move Prelude out of base (and all base modules import explicitly what they want; we could have a small prelude for them if there are some things that can only be gotten from the Prelude ( (.), ($), seq... ?), or make real modules that export those.) And there is a long-term plan for fixing Exception to be an extensible single-inheritance hierarchy ... maybe ghc 6.10 ... at which point low-level refactorings will be MUCH more possible (I tried once... Exception was one problem, avoiding orphan instances, another...) Isaac -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGh7TFHgcxvIWYTTURAomPAKCxLrNULfBLMWwcWmJRoVqVKiA9/ACaA8vz ZOwtpjK7Zasrvflfwwkq+aM= =H2Uc -----END PGP SIGNATURE-----
On Thu, Jun 28, 2007 at 09:25:44AM -0700, Bryan O'Sullivan wrote:
Splitting up the unix package seems more like an inconsequential janitorial task.
A lot of what I do (bug triaging, approving mailing list posts, ...) is janitorial, because it is those tasks that people don't tend to want to do in their evenings, can't do for SoC projects, and so on. And for what it's worth, in my opinion the task is important; there is quite a long road to having multiple (modern) Haskell implementations, with the same libraries, providing the same interfaces and coming with regression and performance testsuites, but we are making progress, and this represents one more step along the road. Writing a windows-posix package would also be useful, but would probably be better done by someone with a task they want it for, who is motivated to just sit down for a weekend or two and get it written. If I were to do it then it would keep getting put aside, as things like the ongoing janitorial tasks above need tending to, and bugs holding up the 6.8 release need fixing, and so on, so it would take much longer to materialise. Also, this is only one of dozens of libraries that it would be great to have. We need a similar number of people, each with their own itch that needs scratching! Thanks Ian
participants (8)
-
Bayley, Alistair -
Bjorn Bringert -
Bryan O'Sullivan -
Bulat Ziganshin -
Duncan Coutts -
Esa Ilari Vuokko -
Ian Lynagh -
Isaac Dupree