Building GHC 6.12 with GHC 6.12 + alex - Glasgow-haskell-users - Haskell.org

newer
Associativity of ViewPatterns

Building GHC 6.12 with GHC 6.12 + alex

older
Where did the GHC API go?

Antoine Latter

31 Dec 2009 31 Dec '09

10:19 a.m.

Folks, I've run into trouble building GHC 6.12 with alex built from GHC 6.12: "/Users/alatter/usr/bin/alex" -g compiler/parser/Lexer.x -o compiler/stage1/build/Lexer.hs alex: compiler/parser/Lexer.x: hGetContents: invalid argument (Illegal byte sequence) however things like: ghci> readFile "compiler/parser/Lexer.x" apear to work just fine. Has anyone else seen this? Has anyone else succeeded in this? This is on Mac OS X 10.6, running on Intel Core 2 Duo. I'm trying to compile a recent pull of the 6.12 branch. Thanks, Antoine

Reply

Sign in to reply online Use email software

Show replies by date

Antoine Latter

31 Dec 31 Dec

1:26 p.m.

On Thu, Dec 31, 2009 at 9:19 AM, Antoine Latter wrote:

Folks,

I've run into trouble building GHC 6.12 with alex built from GHC 6.12:

"/Users/alatter/usr/bin/alex" -g compiler/parser/Lexer.x -o compiler/stage1/build/Lexer.hs alex: compiler/parser/Lexer.x: hGetContents: invalid argument (Illegal byte sequence)

however things like:

ghci> readFile "compiler/parser/Lexer.x"

apear to work just fine. Has anyone else seen this? Has anyone else succeeded in this?

This is on Mac OS X 10.6, running on Intel Core 2 Duo. I'm trying to compile a recent pull of the 6.12 branch.

I've progressed to an advanced state of disfunction. I've patched alex src/Main.hs to use the following: #if __GLASGOW_HASKELL__ >= 612 readFile :: FilePath -> IO String readFile file = do h <- openFile file ReadMode hSetEncoding h utf8 hGetContents h #endif Now, I get the following error on the same file: $ ~/src/alex/dist/build/alex/alex -g compiler/parser/Lexer.x -o compiler/stage1/ build/Lexer.hs alex: compiler/stage1/build/Lexer.hs: commitAndReleaseBuffer: invalid argument (Illegal byte sequence)

Reply

Sign in to reply online Use email software

Antoine Latter

1:35 p.m.

On Thu, Dec 31, 2009 at 12:26 PM, Antoine Latter wrote:

On Thu, Dec 31, 2009 at 9:19 AM, Antoine Latter wrote:

...
Folks,

I've run into trouble building GHC 6.12 with alex built from GHC 6.12:

"/Users/alatter/usr/bin/alex" -g compiler/parser/Lexer.x -o compiler/stage1/build/Lexer.hs alex: compiler/parser/Lexer.x: hGetContents: invalid argument (Illegal byte sequence)

however things like:

ghci> readFile "compiler/parser/Lexer.x"

apear to work just fine. Has anyone else seen this? Has anyone else succeeded in this?

This is on Mac OS X 10.6, running on Intel Core 2 Duo. I'm trying to compile a recent pull of the 6.12 branch.

I've progressed to an advanced state of disfunction. I've patched alex src/Main.hs to use the following:

#if __GLASGOW_HASKELL__ >= 612 readFile :: FilePath -> IO String readFile file = do h <- openFile file ReadMode hSetEncoding h utf8 hGetContents h #endif

Now, I get the following error on the same file:

$ ~/src/alex/dist/build/alex/alex -g compiler/parser/Lexer.x -o compiler/stage1/ build/Lexer.hs alex: compiler/stage1/build/Lexer.hs: commitAndReleaseBuffer: invalid argument (Illegal byte sequence)

Further details: If I patch alex to explicitly set the encoding whenever it reads from or writes to a file, all of these problems go away. I'm not really sure why alex needs this and nothing else does, but so it goes. Is there a separate bug tracker for alex, or should this just be a GHC bug? Antoine

Reply

Sign in to reply online Use email software

Simon Marlow

2:47 p.m.

On 31/12/09 18:35, Antoine Latter wrote:

On Thu, Dec 31, 2009 at 12:26 PM, Antoine Latter wrote:

...
On Thu, Dec 31, 2009 at 9:19 AM, Antoine Latter wrote:

...
Folks,

I've run into trouble building GHC 6.12 with alex built from GHC 6.12:

"/Users/alatter/usr/bin/alex" -g compiler/parser/Lexer.x -o compiler/stage1/build/Lexer.hs alex: compiler/parser/Lexer.x: hGetContents: invalid argument (Illegal byte sequence)

however things like:

ghci> readFile "compiler/parser/Lexer.x"

apear to work just fine. Has anyone else seen this? Has anyone else succeeded in this?

This is on Mac OS X 10.6, running on Intel Core 2 Duo. I'm trying to compile a recent pull of the 6.12 branch.

I've progressed to an advanced state of disfunction. I've patched alex src/Main.hs to use the following:

#if __GLASGOW_HASKELL__>= 612 readFile :: FilePath -> IO String readFile file = do h<- openFile file ReadMode hSetEncoding h utf8 hGetContents h #endif

Now, I get the following error on the same file:

$ ~/src/alex/dist/build/alex/alex -g compiler/parser/Lexer.x -o compiler/stage1/ build/Lexer.hs alex: compiler/stage1/build/Lexer.hs: commitAndReleaseBuffer: invalid argument (Illegal byte sequence)

Further details:

If I patch alex to explicitly set the encoding whenever it reads from or writes to a file, all of these problems go away. I'm not really sure why alex needs this and nothing else does, but so it goes.

Is there a separate bug tracker for alex, or should this just be a GHC bug?

Thanks for looking into this - yes Alex needs to explicitly use utf8 for reading and writing files. Those of us using a utf-8 locale probably wouldn't have noticed the problem, what's your locale set to? If you can send me the patch, that will save me time figuring it out for myself. Alex doesn't have a bug tracker, but you're welcome to use GHC's if you want. Cheers, Simon

Reply

Sign in to reply online Use email software

Antoine Latter

3:08 p.m.

On Thu, Dec 31, 2009 at 1:47 PM, Simon Marlow wrote:

On 31/12/09 18:35, Antoine Latter wrote:

...
Further details:

If I patch alex to explicitly set the encoding whenever it reads from or writes to a file, all of these problems go away. I'm not really sure why alex needs this and nothing else does, but so it goes.

Is there a separate bug tracker for alex, or should this just be a GHC bug?

Thanks for looking into this - yes Alex needs to explicitly use utf8 for reading and writing files. Those of us using a utf-8 locale probably wouldn't have noticed the problem, what's your locale set to?

If you can send me the patch, that will save me time figuring it out for myself. Alex doesn't have a bug tracker, but you're welcome to use GHC's if you want.

Cheers, Simon

I have the following environment variable set: __CF_USER_TEXT_ENCODING=0x1F5:0:0 But I have no idea what that means. Presumably something other than utf8. The alex patch is attached, but I have not tested it on anything other than GHC 6.12. Antoine

Reply

Sign in to reply online Use email software

Simon Marlow

5 Jan 5 Jan

4:27 a.m.

On 31/12/2009 20:08, Antoine Latter wrote:

On Thu, Dec 31, 2009 at 1:47 PM, Simon Marlow wrote:

...
On 31/12/09 18:35, Antoine Latter wrote:

...
Further details:

If I patch alex to explicitly set the encoding whenever it reads from or writes to a file, all of these problems go away. I'm not really sure why alex needs this and nothing else does, but so it goes.

Is there a separate bug tracker for alex, or should this just be a GHC bug?

Thanks for looking into this - yes Alex needs to explicitly use utf8 for reading and writing files. Those of us using a utf-8 locale probably wouldn't have noticed the problem, what's your locale set to?

If you can send me the patch, that will save me time figuring it out for myself. Alex doesn't have a bug tracker, but you're welcome to use GHC's if you want.

Cheers, Simon

I have the following environment variable set:

__CF_USER_TEXT_ENCODING=0x1F5:0:0

But I have no idea what that means. Presumably something other than utf8.

The alex patch is attached, but I have not tested it on anything other than GHC 6.12.

Applied, and I've uploaded Alex 2.3.2 to Hackage. Cheers, Simon

Reply

Sign in to reply online Use email software

5614

Age (days ago)

5619

Last active (days ago)

Download

5 comments

2 participants

tags

participants (2)

Antoine Latter
Simon Marlow