May 2011 - Haskell-Cafe

Cabal: disabling -fvia-C
by Jacek Generowicz 24 May '11

24 May '11

I hope you will pardon another *ignorant* question. 'cabal install lambdabot' fails on Ubuntu Natty. According to the following exchange and bug report http://comments.gmane.org/gmane.comp.lang.haskell.cafe/88735 http://hackage.haskell.org/trac/ghc/ticket/5050 the workaround is not to use -fvia-C. I have found an {-# OPTION -fvia-C #-} deep inside the lambdabot code in Plugin/Pl/Common.hs. How can I tell cabal install to ignore this? Is there some command line option for doing so, or do I have to edit the source code and somehow persuade cabal to use my hacked version ?

3 4

GHC 7, Platform 2011.2 vs OS X 10.5, Ubuntu 11.04
by Jacek Generowicz 24 May '11

24 May '11

[TL;DR: there's a concrete question at the bottom.] Once again I find myself in the pleasant situation where the slings and arrows of life in general are slightly less intense, and I manage to find a few spare minutes and some free brain cycles to dedicate to Haskell. Why not try GHC 7.0.3? (after all, GHC 7 has been out for a good few months, so the teething troubles should have passed by now), and there's a lovely new Haskell Platform 2011.2.0.1 to go with it, so, maybe this time, I'll actually be able to use some of the libraries that I tried and failed to install during my last period of Haskell study. The two computers I have available for playing with Haskell run OS X Leopard (10.5), and Ubuntu Natty (11.04). First up OS X: It seems that GHC 7 (and consequently Platform 2011.2) require OS X 10.6. OK, let's forget that one. Next up: Ubuntu. There's a Linux Link on the Haskell Platform Page. Ooooh, there's an Ubuntu link. Ooooh! there's even a package for Natty, great! Version 2010.1.0.0.1 WTF? Where's the 2011.2 package? There isn't one. OOoooKaaaayyyy. Any workarounds. Well, we can install from source, apparently. But we need GHC 7.0.3 first. Fine. Done. Now let's install the platorm. [Does obvious thing, compilation fails.] Hmm, let's follow the "Complete instructions for installing from source" http://www.vex.net/~trebla/haskell/haskell-platform.xhtml just in case I did something stupid. [Follows instructions: failure.] OK, maybe something somewhere is causing some confusion with previously installed versions of GHC, Cabal, whatever. [Performs Haskell pogrom on system: removes all traces of anything Haskell related. Follows "Complete instructions [...]" WORD ... FOR ... WORD. Failure]. Aaaaaaaaaaaaaaaaaaaargh ! I can't take it any more. I just want to write some Haskell, and every time I want to venture beyond the safe confines of the Prelude I get bogged down in installation hell. Next time I have some spare time, I will be more reluctant to risk having that time stolen by the Haskell infrastructure installation process. So, after this tale of agony, here are two concrete questions: a) Am I right in concluding that GHC 7.0.3 will not run on OS X 10.5 (without unreasonable effort)? b) On Ubuntu Natty I installed the generic linux GHC 7.0.3 binary. Downloaded Haskell Platform 2011.2.0.1 source distribution. ./ configure worked happily. make fails as follows. Building transformers-0.2.2.0 "/usr/local/haskell-platform-2011.2.0.1/bin/ghc" "--make" "Setup" "-o" "Setup" "-package" "Cabal-1.10.1.0" <command line>: cannot satisfy -package Cabal-1.10.1.0: Cabal-1.10.1.0-1fb2094e19492373b1a39284193e7984 is unusable due to missing or recursive dependencies: process-1.0.1.5-55dfaccf3a91c4cb8f6284a0bafef198 Could anybody help me get past this problem?

6 17

SIGSEGV in yieldCapability ()
by Johannes Waldmann 24 May '11

24 May '11

Hi. What would be the reason for Program received signal SIGSEGV, Segmentation fault. 0x00000000006071de in yieldCapability () (gdb) where #0 0x00000000006071de in yieldCapability () #1 0x000000000060bc6b in schedule () #2 0x0000000000609294 in real_main () #3 0x000000000060938d in hs_main () #4 0x00007ffff6b07eff in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6 #5 0x000000000040a589 in _start () this happens in a program compiled with ghc-7.0.3 (and also, with ghc-6.12.3), running with +RTS -N on i7-920, ubuntu 11 (kernel 2.6.38-8) Could this simply mean "heap exhausted"? Thanks - J.W.

4 4

Bytestring package: Int and Int64
by Daniel Díaz 24 May '11

24 May '11

Hi, cafe, I just feel curiosity. In the bytestring package, Data.ByteString module, functions like length, index, and others with Int in its type signature, have Int64 in the analogous Data.ByteString.Lazy version. What is the reason? -- Daniel Díaz

2 2

Cabal: top level dependency base -any
by Jacek Generowicz 24 May '11

24 May '11

Hi, Could anyone shed light on the meaning of this error message? cabal: cannot configure xmonad-0.9.1 It requires base ==3.* For the dependency on base ==3.* there are these packages: base-3.0.3.1 and base-3.0.3.2. However none of them are available. base-3.0.3.1 was excluded because of the top level dependency base -any base-3.0.3.2 was excluded because of the top level dependency base -any My guess is that "-any" would mean either that any version is acceptable (in which case should always work, including this time) or that "-any" means that any version is unacceptable, in which case it would never work, which is daft. So clearly I don't understand what it is trying to say. Any hints? Is this documented anywhere? Thanks.

3 6

Imports in complex Setup.hs -- should we encourage/require PackageImports?
by John Millikin 24 May '11

24 May '11

Several libraries (notably Pandoc and Gtk2hs) have very complex Setup.hs scripts, which import several external libraries. In my experience, these imports are very fragile, because Cabal does not enforce package visibility in Setup.hs. For example, a Setup.hs that imports Control.Monad.Trans will break if "monads-tf" is installed, and one that imports System.FilePath will break if "system-filepath" is installed. My typical solution when this happens is to manually tweak the GHC package database before installing, but this is annoying and does not help other users. Based on a ticket in Cabal's Trac < http://hackage.haskell.org/trac/hackage/ticket/326 >, custom Setup.hs scripts are discouraged by the Cabal developers. I assume this means there will not be much development effort put towards an integrated solution (such as using -hide-all-packages and build-depends: when compiling Setup.hs). A possible solution is to ask developers with complex Setup.hs requirements to use the PackageImports language extension when importing external libraries. However, this places a burden on such developers, and I don't know if it's portable to non-GHC compilers. It would also need an analysis of currently published Hackage packages to see which have such scripts. Any ideas/comments? Has anyone by chance found a good solution to this?

2 1

Further installation adventures
by Jacek Generowicz 23 May '11

23 May '11

I'm trying to cabal install lambdabot. This depends on a squillion other libraries, which cabal tries to install, among which is readline, which fails to configure as follows. Configuring readline-1.0.1.0... checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for GNUreadline.framework... checking for readline... no checking for tputs in -lncurses... no checking for tputs in -ltermcap... no checking for tputs in -lcurses... no checking for rl_readline_version... no configure: error: readline not found, so this package cannot be built See `config.log' for more details. cabal: Error: some packages failed to install: readline-1.0.1.0 failed during the configure step. The exception was: ExitFailure 1 From which I conclude that it can't find the readline library. I have all sorts of flavours of readline on my system, so I'll need a bit more information about what exactly it's looking for. It tells me to look in config.log. I presume that this is the config log of the cabal package readline, which cabal automatically downloaded, unpacked ... and presumably automatically cleaned up at the end, because I can't find it anywhere. Where should I be looking for the relevant config.log?

2 2

The Lisp Curse
by Andrew Coppin 23 May '11

23 May '11

http://www.winestockwebdesign.com/Essays/Lisp_Curse.html Some of you might have seen this. Here's the short version: Lisp is so powerful that it discourages reuse. Why search for and reuse an existing implementation, when it's so trivially easy to reimplement exactly what you want yourself? The net result is a maze of incompatible libraries which each solve a different 80% of the same problem. To all the people who look at Hackage, see that there are 6 different libraries for processing Unicode text files, and claim that this is somehow a *good* thing, I offer the above essay as a counter-example.

27 37

blaze-builder and FlexibleInstances in code that aims to become part of the Haskell platform
by Simon Meier 23 May '11

23 May '11

Hello Haskell-Cafe, my main question is whether requiring FlexibleInstances is a problem for code that aims to become part of the Haskell platform. The following explanation gives the context for this question. As some of you may know the blaze-builder library is now used in quite a few places. That's nice, but it doesn't mean that blaze-builder is a finished solution to the problem of providing an API for high-performance buffered output (creation of chunked representations) of sequences of bytes. In fact, one of my current goals with this work is to polish it such that it can be integrated into the 'bytestring' library. This has the benefit that functions that create lazy bytestrings (e.g., pack, map, unfoldr, filter) can be implemented such that they create well-sized chunks even if the argument bytestring is hugely fragmented. Moreover, this integration also establishes a single builder type as the output representation. Therefore, other creators of bytestrings (e.g., 'text', 'base16-bytestring', 'zlib') can provide results of type Builder, which enables O(1) appends of their results and the preservation of well-sizedness of the created chunks. As part of this goal, I'm currently working on the first of the following three points that are paramount to achieving great encoding performance: 1. Ensure that individual Haskell values are encoded with minimal overhead. 2. Ensure that concatenation of sequences of bytes is efficient. 3. Ensure that the average chunk size is large. The core principle used to tackle (1) is avoiding intermediate data structures. The core abstraction used is the one of a Write (see [1] for the corresponding library.) data Write a = Write Int (a -> Ptr Word8 -> IO (Ptr Word8)) A value `Write bound io :: Write a` denotes an encoding scheme for values of type `a` that uses at most `bound` bytes space. Given a values `x :: a` and a pointer `po` to the next free byte `io x po` encodes `x` to memory starting from `po` and returns the pointer to the next free byte after the encoding of `x`. In most cases Writes are used as an abstract datatype. They serve as an interface between implementors of the low-level bit-twiddling required to efficiently implement encodings like UTF-8 or Base16 and the providers of efficient traversal functions through streams of Haskell values. Hence, typical users of Writes are functions like fromWrite :: Write a -> a -> Builder fromWriteList :: Write a -> [a] -> Builder fromWriteUnfoldr :: Write b -> (a -> Maybe (b, a)) -> a -> Builder mapWriteByteString :: Write Word8 -> S.ByteString -> Builder They consume the given datastructure efficiently and wrap the Writes in the bounds checking code to detect when a buffer is full and request a new one. There are many providers of Writes. Each bounded-length-encoding of a standard Haskell value is likely to have a corresponding Write. For example, encoding an Int32 as a big-endian, little-endian, and host-endian byte-sequence is currently achieved with the following three functions. writeInt32BE :: Write Int32 writeInt32LE :: Write Int32 writeInt32HE :: Write Int32 I would like to avoid naming all these encodings individually. Especially, as the situation becomes worse for more elaborate encodings like hexadecimal encodings. There, we encounter encodings like the utf8-encoding of the hexadecimal-encoding with lower-case letters of an Int32. writeInt32HexLowerUtf8 :: Write Int32 I really don't like that. Therefore, I'm thinking about the following solution based on type-classes. We introduce a single typeclass class Writable a where write :: Write a and use a bunch of newtypes to denote our encodings. newtype Ascii7 a = Ascii7 { unAscii7 :: a } newtype Utf8 a = Utf8 { unUtf8 :: a } newtype HexUpper a = HexUpper { unHexUpper :: a } newtype HexLower a = HexLower { unHexLower :: a } ... Assuming FlexibleInstnaces, we can write encodings like the above hex-encoding as instances instance Write (Utf8 (HexLower Int32)) where write = ... This composes rather nicely and allows the implementations to exploit special properties of the involved data. For example, if we also had a HTML escaping marker newtype Html a = Html { unHtml :: a } Then, the instance instance Write (Utf8 (HTML (HexLower Int32))) where write (Utf8 (HTML (HexLower i))) = write (Utf8 (HexLower i)) exploits that no HTML escaping is required for a hex-number. Assuming FlexibleContexts, the user can also build abbreviations for builders using fixed encodings. utf8 :: Writable (Utf8 a) => a -> Builder utf8 = fromWrite write . Utf8 Note that, on the Builder level, a probably better way would be to have an analogous 'ToBuilder' typeclass to abstract the various encodings. Part of these instances then reuse the corresponding instances from Writable. I think this type-class based interface to select the correct efficient implementation of an encoding is rather nice. However, I don't know if 'FlexibleInstances' and 'FlexibleContexts' are fine to use in code that aims to become part of the Haskell platform. Moreover, I might well overlook some drawbacks of this design. I'm looking forward to your feedback. best regards, Simon [1] https://github.com/meiersi/system-io-write

7 17

Handling a large database (of ngrams)
by Aleksandar Dimitrov 23 May '11

23 May '11

Hi Haskellers, I'm unaware of a "good method" or "default way" of handling large datasets to which I need non-sequential (i.e. random) access in Haskell. My use case is linguistic analysis of a ~30GB corpus — the most basic form of quantitative analysis here are ngram based HMMs, which aren't difficult to pull of in Haskell. However, I would need a *large* database of ngrams, possibly into the hundreds of gigabytes. Such a beast will obviously never fit into memory. Moreover, I'll need sort of random access both during training and testing/operation. My initial idea (that I'm currently following up on) is to use Haskell's Berkeley DB wrapper. In this case, I would just stuff all ngrams in one big table (with their counts in an associated column,) and let Berkeley DB do the heavy lifting. One simplifying assumption is, of course, using the flyweight pattern (i.e. having one table of unigrams (that would even fit into memory!) associated with indices, and then just reference the indices in the ngram table.) But this is only going to decrease the size of the db by some factor, not an order of magnitude. Tries come to mind, too. Not only for the unigram table (where they indeed do make a lot of sense,) but also for the ngram table, where they might or might not help. In this case, the cells of the trie would be word indices pointing to the unigram table. Is there a way to serialize tries over a finite (but possibly large) domain to disk and have a semi-speedy random access to them? I'm fully aware that I'll have to wait up to a couple of seconds per lookup, that's OK. I could hack together a primitive implementation of what I need here, but maybe there's something better out there already? Hackage didn't speak to me[1], and #haskell was busy discussing something else, so I hope -cafe can help me :-) Thanks for any hints and pointers! Aleks [1] There's the ngram package on Haskell, but that only queries Google. I actually want to build my own ngram database, because I'm training on a specific corpus, and will possibly have to adapt to different domains.

2 3