Re: [commit: ghc] ghc-7.8: change deriveConstants to use nm in a POSIX way (fixes #8781) (e7563ec)

Hello, With the patch to deriveConstants, a build on an older Windows (32bit Vista, MinGW32 installation, which used to work) now fails with error Can't find "STD_HDR_SIZE" I ran the nm command manually on the generated o file (includes/dist-derivedconstants/header/tmp.o) and saw that the output follows pattern _derivedConstantBlaBlah_Blah C 000001b so it does not match the four-word pattern in the patch. Is this a problem with my setup, or does it fail for other people on Windows as well? Thanks, Jost On 03/23/2014 08:46 AM, ghc-commits-request@haskell.org wrote:
Message: 6 Date: Sun, 23 Mar 2014 02:01:50 +0000 (UTC) From: git@git.haskell.org To: ghc-commits@haskell.org Subject: [commit: ghc] ghc-7.8: change deriveConstants to use nm in a POSIX way (fixes #8781) (e7563ec) Message-ID: <20140323020150.1DCE82406B@ghc.haskell.org> Content-Type: text/plain; charset=utf-8
Repository : ssh://git@git.haskell.org/ghc
On branch : ghc-7.8 Link : http://ghc.haskell.org/trac/ghc/changeset/e7563ec2e03740074903036bf129fc972b...
---------------------------------------------------------------
commit e7563ec2e03740074903036bf129fc972b623c23 Author: Karel Gardas
Date: Sat Mar 22 22:33:05 2014 +0100 change deriveConstants to use nm in a POSIX way (fixes #8781)
The patch provided by Christian Maeder
Signed-off-by: Karel Gardas
Signed-off-by: Austin Seipp (cherry picked from commit 045b28033a33a48d31951240a8cb35f2b78345dc) ---------------------------------------------------------------
e7563ec2e03740074903036bf129fc972b623c23 utils/deriveConstants/DeriveConstants.hs | 30 ++++++++++-------------------- 1 file changed, 10 insertions(+), 20 deletions(-)
diff --git a/utils/deriveConstants/DeriveConstants.hs b/utils/deriveConstants/DeriveConstants.hs index 10df61c..54ee6a1 100644 --- a/utils/deriveConstants/DeriveConstants.hs +++ b/utils/deriveConstants/DeriveConstants.hs @@ -638,7 +638,7 @@ getWanted verbose tmpdir gccProgram gccFlags nmProgram oFile = tmpdir > "tmp.o" writeFile cFile cStuff execute verbose gccProgram (gccFlags ++ ["-c", cFile, "-o", oFile]) - xs <- readProcess nmProgram [oFile] "" + xs <- readProcess nmProgram ["-P", oFile] "" let ls = lines xs ms = map parseNmLine ls m = Map.fromList $ catMaybes ms @@ -707,27 +707,17 @@ getWanted verbose tmpdir gccProgram gccFlags nmProgram doWanted (ClosurePayloadMacro {}) = [] doWanted (FieldTypeGcptrMacro {}) = []
- -- parseNmLine parses nm output that looks like - -- "0000000b C derivedConstantMAX_Vanilla_REG" + -- parseNmLine parses "nm -P" output that looks like + -- "_derivedConstantMAX_Vanilla_REG C b 0" Mac OS X + -- "derivedConstantMAX_Vanilla_REG C 0000000b 0000000b" GNU + -- "derivedConstantMAX_Vanilla_REG D 1 b" Solaris -- and returns ("MAX_Vanilla_REG", 11) - parseNmLine xs0 = case break (' ' ==) xs0 of - (x1, ' ' : xs1) -> - case break (' ' ==) xs1 of - (x2, ' ' : x3) -> - case readHex x1 of - [(size, "")] -> - case x2 of - "C" -> - let x3' = case x3 of - '_' : rest -> rest - _ -> x3 - in case stripPrefix prefix x3' of - Just name -> - Just (name, size) - _ -> Nothing - _ -> Nothing - _ -> Nothing + parseNmLine xs0 = case words xs0 of + [x0, x1, x2, x3] -> case stripPrefix prefix $ dropWhile (== '_') x0 of + Just name -> case readHex $ if x1 == "C" then x2 else x3 of + [(size, "")] -> Just (name, size) _ -> Nothing + _ -> Nothing _ -> Nothing
-- If an Int value is larger than 2^28 or smaller

Jost, this is not good indeed, could you be so kind and let us know your nm --version? I guess mingw is using GNU binutils so I'm curious what old nm it is when it fails to implement POSIX output... Thanks! Karel On 03/23/14 05:51 PM, Jost Berthold wrote:
Hello,
With the patch to deriveConstants, a build on an older Windows (32bit Vista, MinGW32 installation, which used to work) now fails with error
Can't find "STD_HDR_SIZE"
I ran the nm command manually on the generated o file (includes/dist-derivedconstants/header/tmp.o) and saw that the output follows pattern
_derivedConstantBlaBlah_Blah C 000001b
so it does not match the four-word pattern in the patch.
Is this a problem with my setup, or does it fail for other people on Windows as well?
Thanks, Jost
On 03/23/2014 08:46 AM, ghc-commits-request@haskell.org wrote:
Message: 6 Date: Sun, 23 Mar 2014 02:01:50 +0000 (UTC) From: git@git.haskell.org To: ghc-commits@haskell.org Subject: [commit: ghc] ghc-7.8: change deriveConstants to use nm in a POSIX way (fixes #8781) (e7563ec) Message-ID: <20140323020150.1DCE82406B@ghc.haskell.org> Content-Type: text/plain; charset=utf-8
Repository : ssh://git@git.haskell.org/ghc
On branch : ghc-7.8 Link : http://ghc.haskell.org/trac/ghc/changeset/e7563ec2e03740074903036bf129fc972b...
---------------------------------------------------------------
commit e7563ec2e03740074903036bf129fc972b623c23 Author: Karel Gardas
Date: Sat Mar 22 22:33:05 2014 +0100 change deriveConstants to use nm in a POSIX way (fixes #8781)
The patch provided by Christian Maeder
Signed-off-by: Karel Gardas
Signed-off-by: Austin Seipp (cherry picked from commit 045b28033a33a48d31951240a8cb35f2b78345dc) ---------------------------------------------------------------
e7563ec2e03740074903036bf129fc972b623c23 utils/deriveConstants/DeriveConstants.hs | 30 ++++++++++-------------------- 1 file changed, 10 insertions(+), 20 deletions(-)
diff --git a/utils/deriveConstants/DeriveConstants.hs b/utils/deriveConstants/DeriveConstants.hs index 10df61c..54ee6a1 100644 --- a/utils/deriveConstants/DeriveConstants.hs +++ b/utils/deriveConstants/DeriveConstants.hs @@ -638,7 +638,7 @@ getWanted verbose tmpdir gccProgram gccFlags nmProgram oFile = tmpdir > "tmp.o" writeFile cFile cStuff execute verbose gccProgram (gccFlags ++ ["-c", cFile, "-o", oFile]) - xs <- readProcess nmProgram [oFile] "" + xs <- readProcess nmProgram ["-P", oFile] "" let ls = lines xs ms = map parseNmLine ls m = Map.fromList $ catMaybes ms @@ -707,27 +707,17 @@ getWanted verbose tmpdir gccProgram gccFlags nmProgram doWanted (ClosurePayloadMacro {}) = [] doWanted (FieldTypeGcptrMacro {}) = []
- -- parseNmLine parses nm output that looks like - -- "0000000b C derivedConstantMAX_Vanilla_REG" + -- parseNmLine parses "nm -P" output that looks like + -- "_derivedConstantMAX_Vanilla_REG C b 0" Mac OS X + -- "derivedConstantMAX_Vanilla_REG C 0000000b 0000000b" GNU + -- "derivedConstantMAX_Vanilla_REG D 1 b" Solaris -- and returns ("MAX_Vanilla_REG", 11) - parseNmLine xs0 = case break (' ' ==) xs0 of - (x1, ' ' : xs1) -> - case break (' ' ==) xs1 of - (x2, ' ' : x3) -> - case readHex x1 of - [(size, "")] -> - case x2 of - "C" -> - let x3' = case x3 of - '_' : rest -> rest - _ -> x3 - in case stripPrefix prefix x3' of - Just name -> - Just (name, size) - _ -> Nothing - _ -> Nothing - _ -> Nothing + parseNmLine xs0 = case words xs0 of + [x0, x1, x2, x3] -> case stripPrefix prefix $ dropWhile (== '_') x0 of + Just name -> case readHex $ if x1 == "C" then x2 else x3 of + [(size, "")] -> Just (name, size) _ -> Nothing + _ -> Nothing _ -> Nothing
-- If an Int value is larger than 2^28 or smaller
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 03/23/2014 06:32 PM, Karel Gardas wrote:
Jost,
this is not good indeed, could you be so kind and let us know your nm --version? I guess mingw is using GNU binutils so I'm curious what old nm it is when it fails to implement POSIX output...
I can see that the failing nm is one that came with ghc-7.6.3 (I installed a binary tarball from http://www.haskell.org/ghc/dist/7.6.3/) which is copied into the build tree. jost@ibm-jost ~/ghc $ C:/MinGW/msys/1.0/home/jost/ghc/inplace/mingw/bin/nm.exe --version GNU nm (GNU Binutils) 2.20.51.20100613 The nm version that came with MinGW itself is this: jost@ibm-jost ~/ghc $ which nm; nm --version /c/MinGW/bin/nm.exe GNU nm (GNU Binutils) 2.23.2 But it produces the same output jost@ibm-jost ~/ghc $ nm -P includes/dist-derivedconstants/header/tmp.o | head .bss b 00000000 .data d 00000000 .drectve i 00000000 .rdata$zzz r 00000000 .text t 00000000 _derivedConstantAP_STACK_SPLIM C 00000401 _derivedConstantBITMAP_BITS_SHIFT C 00000006 _derivedConstantBLOCK_SIZE C 00001001 _derivedConstantBLOCKS_PER_MBLOCK C 000000ff _derivedConstantCINT_SIZE C 00000005 This installation is not very old (last December, mostly following https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows but taking some shortcuts ). The MSYS2 solution described on a sub-page did not work on the 32-bit Vista (pacman did not work) jost@ibm-jost ~/ghc $ uname -a MINGW32_NT-6.0 IBM-JOST 1.0.18(0.48/3/2) 2012-11-21 22:34 i686 Msys I am not sure whether anything would actually break if deriveConstants.hs accepted and parsed this nm output as a fallback if it sees it. Will give it a try. / Jost
Thanks! Karel
On 03/23/14 05:51 PM, Jost Berthold wrote:
Hello,
With the patch to deriveConstants, a build on an older Windows (32bit Vista, MinGW32 installation, which used to work) now fails with error
Can't find "STD_HDR_SIZE"
I ran the nm command manually on the generated o file (includes/dist-derivedconstants/header/tmp.o) and saw that the output follows pattern
_derivedConstantBlaBlah_Blah C 000001b
so it does not match the four-word pattern in the patch.
Is this a problem with my setup, or does it fail for other people on Windows as well?
Thanks, Jost
On 03/23/2014 08:46 AM, ghc-commits-request@haskell.org wrote:
Message: 6 Date: Sun, 23 Mar 2014 02:01:50 +0000 (UTC) From: git@git.haskell.org To: ghc-commits@haskell.org Subject: [commit: ghc] ghc-7.8: change deriveConstants to use nm in a POSIX way (fixes #8781) (e7563ec) Message-ID: <20140323020150.1DCE82406B@ghc.haskell.org> Content-Type: text/plain; charset=utf-8
Repository : ssh://git@git.haskell.org/ghc
On branch : ghc-7.8 Link : http://ghc.haskell.org/trac/ghc/changeset/e7563ec2e03740074903036bf129fc972b...
---------------------------------------------------------------
commit e7563ec2e03740074903036bf129fc972b623c23 Author: Karel Gardas
Date: Sat Mar 22 22:33:05 2014 +0100 change deriveConstants to use nm in a POSIX way (fixes #8781)
The patch provided by Christian Maeder
Signed-off-by: Karel Gardas
Signed-off-by: Austin Seipp (cherry picked from commit 045b28033a33a48d31951240a8cb35f2b78345dc) ---------------------------------------------------------------
e7563ec2e03740074903036bf129fc972b623c23 utils/deriveConstants/DeriveConstants.hs | 30 ++++++++++-------------------- 1 file changed, 10 insertions(+), 20 deletions(-)
diff --git a/utils/deriveConstants/DeriveConstants.hs b/utils/deriveConstants/DeriveConstants.hs index 10df61c..54ee6a1 100644 --- a/utils/deriveConstants/DeriveConstants.hs +++ b/utils/deriveConstants/DeriveConstants.hs @@ -638,7 +638,7 @@ getWanted verbose tmpdir gccProgram gccFlags nmProgram oFile = tmpdir > "tmp.o" writeFile cFile cStuff execute verbose gccProgram (gccFlags ++ ["-c", cFile, "-o", oFile]) - xs <- readProcess nmProgram [oFile] "" + xs <- readProcess nmProgram ["-P", oFile] "" let ls = lines xs ms = map parseNmLine ls m = Map.fromList $ catMaybes ms @@ -707,27 +707,17 @@ getWanted verbose tmpdir gccProgram gccFlags nmProgram doWanted (ClosurePayloadMacro {}) = [] doWanted (FieldTypeGcptrMacro {}) = []
- -- parseNmLine parses nm output that looks like - -- "0000000b C derivedConstantMAX_Vanilla_REG" + -- parseNmLine parses "nm -P" output that looks like + -- "_derivedConstantMAX_Vanilla_REG C b 0" Mac OS X + -- "derivedConstantMAX_Vanilla_REG C 0000000b 0000000b" GNU + -- "derivedConstantMAX_Vanilla_REG D 1 b" Solaris -- and returns ("MAX_Vanilla_REG", 11) - parseNmLine xs0 = case break (' ' ==) xs0 of - (x1, ' ' : xs1) -> - case break (' ' ==) xs1 of - (x2, ' ' : x3) -> - case readHex x1 of - [(size, "")] -> - case x2 of - "C" -> - let x3' = case x3 of - '_' : rest -> rest - _ -> x3 - in case stripPrefix prefix x3' of - Just name -> - Just (name, size) - _ -> Nothing - _ -> Nothing - _ -> Nothing + parseNmLine xs0 = case words xs0 of + [x0, x1, x2, x3] -> case stripPrefix prefix $ dropWhile (== '_') x0 of + Just name -> case readHex $ if x1 == "C" then x2 else x3 of + [(size, "")] -> Just (name, size) _ -> Nothing + _ -> Nothing _ -> Nothing
-- If an Int value is larger than 2^28 or smaller
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 3/23/2014 20:51, Jost Berthold wrote:
Hello,
With the patch to deriveConstants, a build on an older Windows (32bit Vista, MinGW32 installation, which used to work) now fails with error
Can't find "STD_HDR_SIZE" Same here: Windows 64-bit, MSYS2, nm being called is from binutils 2.24.
Kyra

Guys, could you be so kind and test attached patch? Thanks! Karel On 03/24/14 07:49 AM, kyra wrote:
On 3/23/2014 20:51, Jost Berthold wrote:
Hello,
With the patch to deriveConstants, a build on an older Windows (32bit Vista, MinGW32 installation, which used to work) now fails with error
Can't find "STD_HDR_SIZE" Same here: Windows 64-bit, MSYS2, nm being called is from binutils 2.24.
Kyra _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 03/24/2014 08:39 AM, Karel Gardas wrote:
Guys,
could you be so kind and test attached patch?
The 32-bit Windows machine is at home; I will try your patch at night. Fairly certain that it works, though. A very similar patch (which I was just going to propose here) worked for me yesterday night: _ -> Nothing _ -> Nothing + [('_':x0), "C", x1] -> + do name <- stripPrefix prefix x0 + case readHex x1 of + [(size, "")] -> return (name,size) + _ -> fail "not a size" _ -> Nothing (using do-notation for Maybe, assuming exactly one "_") / Jost
Thanks! Karel
On 03/24/14 07:49 AM, kyra wrote:
On 3/23/2014 20:51, Jost Berthold wrote:
Hello,
With the patch to deriveConstants, a build on an older Windows (32bit Vista, MinGW32 installation, which used to work) now fails with error
Can't find "STD_HDR_SIZE" Same here: Windows 64-bit, MSYS2, nm being called is from binutils 2.24.
Kyra _______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 03/24/14 09:54 AM, Jost Berthold wrote:
On 03/24/2014 08:39 AM, Karel Gardas wrote:
Guys,
could you be so kind and test attached patch?
The 32-bit Windows machine is at home; I will try your patch at night.
Fairly certain that it works, though.
Indeed, it's working on your testing line you have provided. I just would like to know if it works on bootstrap too as I don't have win/ghc machine here. Thanks for testing! Karel

On 24/03/2014 06:49, kyra wrote:
On 3/23/2014 20:51, Jost Berthold wrote:
Hello,
With the patch to deriveConstants, a build on an older Windows (32bit Vista, MinGW32 installation, which used to work) now fails with error
Can't find "STD_HDR_SIZE" Same here: Windows 64-bit, MSYS2, nm being called is from binutils 2.24.
I also ran into this when I tried to do a Windows build today, so I've reverted the patch in my local tree. Karel: will you look into this? Note that on Windows, nm is part of the mingw tools that we provide in ghc-tarballs/, so if that needs upgrading we'll need to commit new versions of the tools (for both 32 and 64-bit mingw). Cheers, Simon

On 03/24/14 11:06 AM, Simon Marlow wrote:
On 24/03/2014 06:49, kyra wrote:
On 3/23/2014 20:51, Jost Berthold wrote:
Hello,
With the patch to deriveConstants, a build on an older Windows (32bit Vista, MinGW32 installation, which used to work) now fails with error
Can't find "STD_HDR_SIZE" Same here: Windows 64-bit, MSYS2, nm being called is from binutils 2.24.
I also ran into this when I tried to do a Windows build today, so I've reverted the patch in my local tree.
Karel: will you look into this?
I already provided a fix for testing on this mailing list. Jost (original reporter) promised to do that during the night on his home machine. If it's working, I'll prepare complete git patch immediately and attach to the #8781 for Austin. Perhaps you can test it too? Thanks! Karel

Hi, sorry, I've missed this thread. Did you just run "nm" or "nm -P" on includes/dist-derivedconstants/header/tmp.o? The comment should say how the output looks like. For GNU nm (without option -P) is was: "0000000b C derivedConstantMAX_Vanilla_REG" if it is now: "_derivedConstantBlaBlah_Blah C 000001b" for "nm -P" on Windows the patch could (and should) be adjusted, easily. (The 4th word only needs to be present and taken if the second word is not a "C", which is only the case under Solaris.) HTH Christian Am 23.03.2014 17:51, schrieb Jost Berthold:
Hello,
With the patch to deriveConstants, a build on an older Windows (32bit Vista, MinGW32 installation, which used to work) now fails with error
Can't find "STD_HDR_SIZE"
I ran the nm command manually on the generated o file (includes/dist-derivedconstants/header/tmp.o) and saw that the output follows pattern
_derivedConstantBlaBlah_Blah C 000001b
so it does not match the four-word pattern in the patch.
Is this a problem with my setup, or does it fail for other people on Windows as well?
Thanks, Jost
On 03/23/2014 08:46 AM, ghc-commits-request@haskell.org wrote:
Message: 6 Date: Sun, 23 Mar 2014 02:01:50 +0000 (UTC) From: git@git.haskell.org To: ghc-commits@haskell.org Subject: [commit: ghc] ghc-7.8: change deriveConstants to use nm in a POSIX way (fixes #8781) (e7563ec) Message-ID: <20140323020150.1DCE82406B@ghc.haskell.org> Content-Type: text/plain; charset=utf-8
Repository : ssh://git@git.haskell.org/ghc
On branch : ghc-7.8 Link : http://ghc.haskell.org/trac/ghc/changeset/e7563ec2e03740074903036bf129fc972b...
---------------------------------------------------------------
commit e7563ec2e03740074903036bf129fc972b623c23 Author: Karel Gardas
Date: Sat Mar 22 22:33:05 2014 +0100 change deriveConstants to use nm in a POSIX way (fixes #8781)
The patch provided by Christian Maeder
Signed-off-by: Karel Gardas
Signed-off-by: Austin Seipp (cherry picked from commit 045b28033a33a48d31951240a8cb35f2b78345dc) ---------------------------------------------------------------
e7563ec2e03740074903036bf129fc972b623c23 utils/deriveConstants/DeriveConstants.hs | 30 ++++++++++-------------------- 1 file changed, 10 insertions(+), 20 deletions(-)
diff --git a/utils/deriveConstants/DeriveConstants.hs b/utils/deriveConstants/DeriveConstants.hs index 10df61c..54ee6a1 100644 --- a/utils/deriveConstants/DeriveConstants.hs +++ b/utils/deriveConstants/DeriveConstants.hs @@ -638,7 +638,7 @@ getWanted verbose tmpdir gccProgram gccFlags nmProgram oFile = tmpdir > "tmp.o" writeFile cFile cStuff execute verbose gccProgram (gccFlags ++ ["-c", cFile, "-o", oFile]) - xs <- readProcess nmProgram [oFile] "" + xs <- readProcess nmProgram ["-P", oFile] "" let ls = lines xs ms = map parseNmLine ls m = Map.fromList $ catMaybes ms @@ -707,27 +707,17 @@ getWanted verbose tmpdir gccProgram gccFlags nmProgram doWanted (ClosurePayloadMacro {}) = [] doWanted (FieldTypeGcptrMacro {}) = []
- -- parseNmLine parses nm output that looks like - -- "0000000b C derivedConstantMAX_Vanilla_REG" + -- parseNmLine parses "nm -P" output that looks like + -- "_derivedConstantMAX_Vanilla_REG C b 0" Mac OS X + -- "derivedConstantMAX_Vanilla_REG C 0000000b 0000000b" GNU + -- "derivedConstantMAX_Vanilla_REG D 1 b" Solaris -- and returns ("MAX_Vanilla_REG", 11) - parseNmLine xs0 = case break (' ' ==) xs0 of - (x1, ' ' : xs1) -> - case break (' ' ==) xs1 of - (x2, ' ' : x3) -> - case readHex x1 of - [(size, "")] -> - case x2 of - "C" -> - let x3' = case x3 of - '_' : rest -> rest - _ -> x3 - in case stripPrefix prefix x3' of - Just name -> - Just (name, size) - _ -> Nothing - _ -> Nothing - _ -> Nothing + parseNmLine xs0 = case words xs0 of + [x0, x1, x2, x3] -> case stripPrefix prefix $ dropWhile (== '_') x0 of + Just name -> case readHex $ if x1 == "C" then x2 else x3 of + [(size, "")] -> Just (name, size) _ -> Nothing + _ -> Nothing _ -> Nothing
-- If an Int value is larger than 2^28 or smaller
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs

On 03/24/2014 11:25 AM, Christian Maeder wrote:
Hi,
sorry, I've missed this thread.
Did you just run "nm" or "nm -P" on includes/dist-derivedconstants/header/tmp.o?
I ran "nm -P".
The comment should say how the output looks like. For GNU nm (without option -P) is was:
"0000000b C derivedConstantMAX_Vanilla_REG"
if it is now:
"_derivedConstantBlaBlah_Blah C 000001b"
for "nm -P" on Windows the patch could (and should) be adjusted, easily. (The 4th word only needs to be present and taken if the second word is not a "C", which is only the case under Solaris.)
The output (with option "-P") was "_derivedConstantBlaBlah_Blah C 000001b" I agree the code of the previous patch is easy to adjust. While several ways are possible, my own suggestion was to add a new specialised case rather than trying to integrate into the existing code. / Jost
participants (5)
-
Christian Maeder
-
Jost Berthold
-
Karel Gardas
-
kyra
-
Simon Marlow