getSymbolicLinkStatus completely broken on some 6.8.2 systems

Hello, I get funny results when I call getSymbolicLinkStatus in ghc 6.8.2 on Ubuntu 7.10. This happens on several systems and has been confirmed by other people. Though, some people on almost the exact same system claimed to not see it at all. Here is an interactive example with ghci, note how the result changes from run to run. (Although not shown below, the values do sometimes repeat, so it is not completely random): Prelude> System.Posix.Files.getSymbolicLinkStatus "/etc/motd" >>= print . System.Posix.Files.fileID 13165085651657739420 Prelude> System.Posix.Files.getSymbolicLinkStatus "/etc/motd" >>= print . System.Posix.Files.fileID 13165312309966851228 Prelude> System.Posix.Files.getSymbolicLinkStatus "/etc/motd" >>= print . System.Posix.Files.fileID 623410571888290073 Prelude> System.Posix.Files.getSymbolicLinkStatus "/etc/motd" >>= print . System.Posix.Files.fileID 12892261005034 If I *compile* the following (with or without optimization): module Main where import System.Posix.Files main = do getSymbolicLinkStatus "/etc/motd" >>= print . fileID getSymbolicLinkStatus "/etc/motd" >>= print . fileID getSymbolicLinkStatus "/etc/motd" >>= print . fileID getSymbolicLinkStatus "/etc/motd" >>= print . fileID Then I get all 0: /tmp # ./test 0 0 0 0 I have tested the same code under GHC 6.6 and 6.4 and it works fine. Additionally, getFileStatus, seems to work fine everytime. (Unfortunately, I really need getSymbolicLinkStatus). If anyone has a work-around or fix that I can use in the next 15 hours, that would be extremely useful ;) Thanks in advance! j.

On Feb 12, 2008 8:38 PM, Jeremy Shaw
I get funny results when I call getSymbolicLinkStatus in ghc 6.8.2 on Ubuntu 7.10. This happens on several systems and has been confirmed by other people.
Currently I'm looking at hsc2hs for this bug. On a 32 bit box here: int main() { struct stat st; printf("size of structure is %d\n", sizeof(st)); printf("inode is at offset %d\n", __builtin_offsetof(struct stat, st_ino)); } Gives: size of structure is 88 inode is at offset 12 However, building Files.hsc gives: fileID (FileStatus stat) = unsafePerformIO $ withForeignPtr stat $ ((\hsc_ptr -> peekByteOff hsc_ptr 88)) {-# LINE 315 "Files.hsc" #-} e.g. it's reading random memory at the end of the structure. Looking at the code from the documentation: http://haskell.org/ghc/docs/latest/html/libraries/unix/src/System-Posix-File... fileID (FileStatus stat) = unsafePerformIO $ withForeignPtr stat $ ((\hsc_ptr -> peekByteOff hsc_ptr 8)) {-# LINE 315 "System/Posix/Files.hsc" #-} Seems like it might work, but that offset is too small. It appears that it might be a 64-bit build (because the size of the stat structure which when it gets malloced is 144), but a 64-bit build should have larger offsets, not smaller. You can't write your own fileID function, because FileStatus isn't exported from Posix.Files. You can, however, write getSymbolicLinkInode a c function (which does both the lstat and extraction from the structure) and FFI it. AGL -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org 650-283-9641

On Feb 12, 2008 10:44 PM, Adam Langley
Currently I'm looking at hsc2hs for this bug. On a 32 bit box here:
hsc2hs is forgiven; if you build with #define _FILE_OFFSET_BITS 32 then the structure is 96 bytes and the 64-bit offset is, indeed, at offset 88. However, if you end up calling the wrong libc lstat (and it's tough to tell, because both lstat and lstat64 in libc call the system call lstat64, so strace can't tell you) you'll only get an 88 byte structure filled in. HsUnix.h has a wrapper around lstat for exactly this reason, however ltrace shows it calling the wrong one. -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org 650-283-9641

On Feb 12, 2008 11:04 PM, Adam Langley
structure filled in. HsUnix.h has a wrapper around lstat for exactly this reason, however ltrace shows it calling the wrong one.
So (finally!) the real issue: hsc2hs has a C preprocessor prelude (utils/hsc2hs/template-hsc.h) which includes some GHC header files. As a result, hsc processes files in 64-bit mode: % cpp -I../../includes -dM template-hsc.h | grep FILE_OFFSET_BITS #define _FILE_OFFSET_BITS 64 However, when building HsUnix.c (or any c file with cabal), the environment is different and it's built in 32-bit mode. Thus the lstat wrapper called the 32-bit glibc function, but the structure sizes and offsets in Files.hs (from Files.hsc) are all for 64-bit mode. The fix is not quite obvious. Probably hsc2hs should try harder not to change the environment so much. I'm not quite sure why the hsc2hs template pulls in the HsFFI header - it doesn't seem to need it. Although you can remove the #include and rebuild the unix package, I can safely say that you'll be reinstalling ghc if you do. What might work is changing hsc2hs and then rebuilding all of GHC. However, that's an overnight build so I don't know yet. AGL -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org 650-283-9641

Hello, You rule! Here is where I am at now. It seems that when I build ghc from source, the top level configure script detects that my system has large file support, and enables it. As you discovered, this gets dumped into: /usr/lib/ghc-6.8.2/include/ghcautoconf.h which gets included when hsc2hs calculates the offsets. However, it seems that that header file does not get included when the unix library builds -- so it builds the library with out large file support. If I run ltrace on this program:
import System.Posix.Files
main = do getSymbolicLinkStatus "/etc/passwd" >>= print . fileID getFileStatus "/etc/passwd" >>= print . fileID
I see that __xstat64 is called, but plain old __xlstat is called. As a fix(??) I added the copied the following from ghc's configure.ac into unix's configure.ac and ran autoreconf: dnl ** Enable large file support. NB. do this before testing the type of dnl off_t, because it will affect the result of that test. AC_SYS_LARGEFILE This seemed to fix my problem -- ltrace shows that __lxstat64 is called and the results are normal. thanks a ton! j. At Tue, 12 Feb 2008 23:45:41 -0800, Adam Langley wrote:
On Feb 12, 2008 11:04 PM, Adam Langley
wrote: structure filled in. HsUnix.h has a wrapper around lstat for exactly this reason, however ltrace shows it calling the wrong one.
So (finally!) the real issue:
hsc2hs has a C preprocessor prelude (utils/hsc2hs/template-hsc.h) which includes some GHC header files. As a result, hsc processes files in 64-bit mode: % cpp -I../../includes -dM template-hsc.h | grep FILE_OFFSET_BITS #define _FILE_OFFSET_BITS 64
However, when building HsUnix.c (or any c file with cabal), the environment is different and it's built in 32-bit mode. Thus the lstat wrapper called the 32-bit glibc function, but the structure sizes and offsets in Files.hs (from Files.hsc) are all for 64-bit mode.
The fix is not quite obvious. Probably hsc2hs should try harder not to change the environment so much. I'm not quite sure why the hsc2hs template pulls in the HsFFI header - it doesn't seem to need it.
Although you can remove the #include and rebuild the unix package, I can safely say that you'll be reinstalling ghc if you do. What might work is changing hsc2hs and then rebuilding all of GHC. However, that's an overnight build so I don't know yet.
AGL
-- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org 650-283-9641

Filed as: http://hackage.haskell.org/trac/ghc/ticket/2096 -- Adam Langley agl@imperialviolet.org http://www.imperialviolet.org 650-283-9641
participants (2)
-
Adam Langley
-
Jeremy Shaw