- Haskell-Cafe - Haskell.org

Exceeding OS limits for simultaneous socket connections
by Rob Stewart 07 Feb '12

07 Feb '12

Hi, I'm experiencing the "accept: resource exhausted (Too many open files)" exception when trying to use sockets in my Haskell program. The situation: - Around a dozen Linux machines running my Haskell program, transmitting thousands of messages to each other, sometimes within a small period of time. - I'm using the Network.Socket.ByteString.Lazy module to send and receive lazy bytestrings - The socket sent to the getMsg is bound and is listening, and is not closed until the program exits. getMsg is called in a loop to receive lazy bytestring from remote nodes. The socket is initialized with: sock <- Network.Socket.socket (addrFamily myAddr) Stream defaultProtocol bindSocket sock (addrAddress myAddr) listen sock 10 Here's the code: sendMsg :: Maybe HostName -> Int -> Lazy.ByteString -> IO () sendMsg dest sckt msg = do result <- try $ withSocketsDo $ do addrinfos <- getAddrInfo Nothing dest (Just (show sckt)) let serveraddr = head addrinfos sock <- socket (addrFamily serveraddr) Stream defaultProtocol connect sock (addrAddress serveraddr) sendAll sock msg sClose sock case result of Left (ex::IOException) -> return () -- permit send failure Right _ -> return () getMsg :: Socket -> IO Lazy.ByteString getMsg sock = do result <- try $ withSocketsDo $ do (conn, addr) <- accept sock getContents conn case result of Left (ex::IOException) -> putStrLn (show ex) >> getMsg sock Right msg -> return msg The current topology is a master/slave setup. For some programs that use these functions above, `sendMsg' is called thousands of times in quick succession on the remote nodes, where the destination of the `sendAll' function is the master node. Here's the maximum number of simultaneous sockets I am permitted to have open on my Linux machines: $ ulimit -n 1024 Indeed, when I experience the "accept: resource exhausted (Too many open files)" exception, I check the number of open sockets, which exceeds 1024, by looking at the contents of the directory: ls -lah /proc/<prod_id>/fd It is within the getContents function that, once the lazy bytestring is fully received, the socket is shutdown http://goo.gl/B6XcV : shutdown sock ShutdownReceive There seems to be no way of limiting the number of permitted connection requests from remote nodes. What I am perhaps looking for is a mailbox implementation on top of sockets, or another way to avoid this error. I am looking to scale up to 100's of nodes, where the possibility of more than 1024 simultaneous socket connections to one node is increased. Merely increasing the ulimit feels like a temporary measure. Part of the dilemma is that the `connect' call in `sendMsg' does not throw an error, despite the fact that it does indeed cause an error on the receiving node, by pushing the number of open connections to the same socket on the master node, beyond the 1024 limit permitted by the OS. Am I missing something? One would have thought such a problem occurs frequently with Haskell web servers and the like.. ? -- Rob Stewart

5 4

a variant of TDNR (was: Re: Some thoughts on Type-Directed Name Resolution)
by Evan Laforge 07 Feb '12

07 Feb '12

I brought up an idea on another thread, but it was a bit off subject and I think it got buried. Anyway, I'm curious to know if there are any obvious problems with this idea, or if it could be pursued further: It could be called Tagged Type Directed Name Resolution. The idea is a # prefix for identifiers. #f is a "type directed function". It requires the type of its argument to be monomorphic, and is desugared to M.f, where M is the module defining the type of 'f's argument. Everything else remains the same. Records wishing the same name must live in separate modules, but field access looks like: (#a . #b) record This works to set fields of a record as well. Let M.f be a function from a record to a lens focused on a certain field of that record. Now you can write: M.a :: M.Record -> Lens M.Record M.A set :: Lens record field -> record -> record val = set (#a 42) record which becomes: set (M.a record) 42 Or composed 'set ((#a.#b) record) 42' becomes 'set ((M.a . N.b) record) 42' An operator with convenient fixity should be able to remove the parens. As long as the compiler can figure out a monomorphic type expected by the input of #a then it shouldn't need type annotations. The thing I'm not sure about is if the ghc typechecker can do this. It would know that '#a' is a function 'alpha -> beta', then need to figure out what alpha is. Then once it has a monomorphic type for alpha, then it has to look up a symbol 'a' in the module defining that type, and only then does it know the type of beta. If #b.#a style composition is to work, then of course '#b' would have to hold off until the return type of '#a' has been resolved. I don't know how hard that would be, or it would even be possible. This idea is less powerful than other proposals because the #-decorated function must be able to statically determine a monomorphic type for its argument. So no structural types. And no records with the same field name in the same module. Personally that doesn't bother me much since I try to keep modules small, but perhaps that could be addressed in an orthogonal nested module feature. The '#' prefix would mess up people using that as an operator. I guess @ is free, right?

1 0

ghc-api Static Semantics?
by Christopher Brown 06 Feb '12

06 Feb '12

Hi, I was wondering if anyone could tell me if it's possible to get an AST from the ghc-api decorated with static-semantics? In particular, I am interested in use and bind locations for all names in the AST together with the module they are bound, etc. Looking through the online docs, there doesn't seem to be a way to do this. Even if I can tell from the AST where a variable is bound that would be enough, if this is by making all names unique and qualified that would be better than nothing. Hope someone can help, Chris.

4 10

Re: [Haskell-cafe] Rewrite this imperative in FP way
by Haisheng Wu 06 Feb '12

06 Feb '12

*d = [sum $ map (a !!) [i | i <- b, j <- c, i + j < 3, i + j == dIndex] | dIndex <- [0..3]] * This is cool. -Simon On Sun, Feb 5, 2012 at 5:07 PM, L Corbijn <aspergesoepje(a)gmail.com> wrote: > On Sun, Feb 5, 2012 at 7:28 AM, Haisheng Wu <freizl(a)gmail.com> wrote: > > a = [1,1,1,1] > > b = [0,1,2,3] > > d = [0,0,0,0] > > > > for i in b: > > for j in c: > > if (i+j)<3: > > d[i+j] += a[i] > > > > My just work implementation in Haskell > > http://hpaste.org/57452 > > > > Another people implementation in Haskell with Monad and it turns out > complex > > and very imperatively. > > http://hpaste.org/57358 > > > > Do you have any cool solution in FP way? > > > > Thanks. > > -Simon > > > > _______________________________________________ > > Haskell-Cafe mailing list > > Haskell-Cafe(a)haskell.org > > http://www.haskell.org/mailman/listinfo/haskell-cafe > > > > There are several ways to make it nicer. > Without assumption on what lists a, b and c contain it can be written as > d = [sum $ map (a !!) [i | i <- b, j <- c, i + j < 3, i + j == dIndex] > | dIndex <- [0..3]] > > With the assumption that b and c are both [0..3] this can be 'improved' to > d = (take 3 . map sum . tail $ inits a) ++ replicate (4 - 3) 0 > This generates the first three values by the complicated expression > and then adds the extra zero's by using the replicate expression. This > works as the value of the i-th element of d is the sum over the first > i elements in a if i < 3 and 0 otherwise. A list of lists with the > first i elements is generated with 'tail $ inits a' which is then > summed and restricted to length 3. > An alternative for this is > d = (take 3 . snd $ mapAccumL (\acc ai -> (acc + ai, acc + ai)) 0 a) > ++ replicate (4 - 3) 0 > Where the summation and tail generating is done in the mapAccumL function. > > Greetings, > Lars > > P.S. Yes the replicate (4-3) 0 can be replaced by [0], but I wanted to > explicitly include the length (4) of the expected list. >

2 1

ANN: Updated OpenGL libraries
by Jason Dagit 06 Feb '12

06 Feb '12

Hello, I'm pleased to anounce a minor bug fix release of the OpenGL libraries. This release was prompted by issues and warnings when compiling with ghc-7.4.1. The following packages have been updated: * OpenGLRaw 1.2.0.0 * OpenGL 2.5.0.0 * GLURaw 1.2.0.0 * GLUT 2.3.0.0 Thanks goes out to: * Corey O'Connor * mgajda (github id) * #haskell-game and #haskell on freenode The changes include: * Removing a data type context * Exposing the GLfoo constructors from OpenGLRaw * Deriving Typeable instead of using the now deprecated mkTyCon * Added Eq constraint, necessary now that Num doesn't have an Eq constraint * Tighten constraints in cabal files * Conservative package version bump (X.Y.Z -> X.(Y+1).0) As usual, please submit pull requests and bug reports on github: https://github.com/haskell-opengl Thank you! Jason

2 1

Rewrite this imperative in FP way
by Haisheng Wu 06 Feb '12

06 Feb '12

a = [1,1,1,1] b = [0,1,2,3] d = [0,0,0,0] for i in b: for j in c: if (i+j)<3: d[i+j] += a[i] My just work implementation in Haskell http://hpaste.org/57452 Another people implementation in Haskell with Monad and it turns out complex and very imperatively. http://hpaste.org/57358 Do you have any cool solution in FP way? Thanks. -Simon

7 10

Problem resolving: Hangman
by Haisheng Wu 06 Feb '12

06 Feb '12

Though the problem I'd ask is not quiet related to haskell, I hope talent people here could provide some suggestions. The problem is not just solve the Hangman problem but solve it in sort of AI way. General idea is : 1. Randomly pick words, e.g. 20 from a predefined word list file. ( http://www.word-list.com/) 2. The application play hangman with each word till either WON or FAIL. NO human being interactive. 3. The expect result is the application be able to win the game at most of time before a certain max wrong guess, e.g. 5 Please let me where is a better place to ask such question and thank you so much in advance. -Simon

1 0

Greetings and Maybe GSoC-2012
by Sergiu Ivanov 05 Feb '12

05 Feb '12

Hello everyone, (long mail ahead) My name is Sergiu Ivanov, I have been trying to wrap my mind around Haskell for 2 years already, but success still feels far away :-) This makes me more and more attached to Haskell though, so I can plainly say: I love this language :-) I guess I should have joined the online Haskell community waaaay earlier; too bad the idea's only struck me now. Therefore, I'd like to hereby greet everyone and wish a lot of good luck and good fun with whatever Haskell-related and non-Haskell-related tasks you may be on! :-) I have graduated Computer Science and I'm now pursuing my MSc in what is officially referred to "Mathematics and Informatics" (sic), but has turned out to be quite heavily biased towards algebra for me. The subject I currently seriously focus on is category theory; I have graduated a course in module theory and I'm now going through quasigroups, lattices, and Gröbner bases. I'm doing some research in computability theory, namely, in P systems (http://en.wikipedia.org/wiki/P_system) but also other computing devices, including some attempts at quantum computer science. My usual IRC nickname is scolobb, and I plan to be hanging out on #haskell quite regularly. I'd be glad to contribute to the community with whatever knowledge and skill assets I'd be able to provide :-) (End of Presentation) I'll turn to the Maybe part of the subject of this letter now. I've seen haskell.org has participated as an org in GSoC 2011. If the community plans to file an application this year as well and if haskell.org is accepted again, I'd be happy to submit an application for some fancy project. I'd enjoy working on something hard, requiring a lot of mental effort to understand, something which would make me feel cleverer when I have finished it :-) I have a fairly broad experience with programming languages and frameworks, so I don't mind committing to http://hackage.haskell.org/trac/summer-of-code/ticket/1592 , http://hackage.haskell.org/trac/summer-of-code/ticket/1547 , or http://hackage.haskell.org/trac/summer-of-code/ticket/1583 . However, among my favourites is this one: http://hackage.haskell.org/trac/summer-of-code/ticket/1582 . This one sounds complicated and mixing a lot of stuff. My experience with LLVM is zero, but I'm willing to learn, of course. You can apprehend my Haskell skills by throwing a look at https://gitorious.org/remoting and https://gitorious.org/psim/haskellengine . The first one is a simple actor-model based remoting framework; that was a project for one of the university projects. The second one is a part of my graduation project, in which I mixed Java, Jython, CUDA and, of course, Haskell. I'm not trying to boast, though: I can easily see the yet unwide horizon of my Haskell knowledge, which I would like to expand as far as possible. As far as I remember from my previous experience with GSoC, potential students are required to solve some simpler tasks to show what they're capable of. I understand that it's quite premature to talk about the summer of code as yet, but I'd be glad to hear of such possible test (or not-so-test) tasks which I could complete. Thank you for reading this much, Sergiu

1 0

Re: [Haskell-cafe] ANN: combinatorics
by Balazs Komuves 05 Feb '12

05 Feb '12

> -------------------------------------------- > -- combinatorics 0.1.0 > -------------------------------------------- > > The combinatorics package offers efficient *exact* computation of common > combinatorial functions like the binomial coefficients and factorial. > (For fast *approximations*, see the math-functions package instead.) > Shameless self-promotion: The combinat package (which deliberately does not try to own the valuable namespace Math.Combinatorics) is a more extensive combinatorics library: http://hackage.haskell.org/package/combinat While the main focus is the generation of combinatorial objects themselves, counting functions and common number sequences like the above are also offered. Even though the binomial and factorial definition in this package are the naive ones, a quick experiment imply that the differences start show themselves around 100,000 factorial, or choosing 50,000 elements out of 100,000, which is probably a rather specialized use case. The primes function in the combinat package is based on an old Cafe thread, and actually seems to be faster than the one in the combinatorics package. In any case, I may switch to the faster algorithms in the future :) Balazs

6 10

Increase GHC stack size?
by Michael Rice 05 Feb '12

05 Feb '12

Stack space overflow: current size 8388608 bytes. Use `+RTS -Ksize -RTS' to increase it. ====================== Couldn't find much on the man or info pages. Example please, say double it (16000000) for starters. Michael

3 4