May 2009 - Haskell-Cafe

Using Takusen is a real intellectual challenge
by Gü?nther Schmidt 04 May '09

04 May '09

Hi, well I'm *trying* to use Takusen to export from one database into the other. I'm using Takusen for this so that I can guarantee that during the export not a single huge result is assembled (possibly in memory) and then insert in one lump into the second database. Trying to figure out how to do that in Takusen has left me totally exhausted, my brain is pudding and I need to have a lay down now. I couldn't even figure out whether that's at all possible. help Günther

2 1

Odd error message from ghc --make
by Jeff Heard 04 May '09

04 May '09

I was just running a test that has compiled before just fine, and now I'm getting this rather odd message: EventBusTest.hs:20:9: Couldn't match expected type `Bus a' against inferred type `buster-1.1:App.EventBus.Bus [buster-1.1:App.EventBus.EData a1]' Expected type: MVar (Bus a) Inferred type: MVar (buster-1.1:App.EventBus.Bus [buster-1.1:App.EventBus.EData a1]) In the expression: mk In the first argument of `bus', namely `[mk]' Any ideas? I mean, it looks like it type checks to me.

1 0

Re: [Haskell-cafe] HDBC 2.1, UTF8 and Umlauts
by John Goerzen 04 May '09

04 May '09

Guenther Schmidt wrote: > Hi John, > > thanks for taking the time. It actually is \252 that turned into > something else because of my email client, damn the thing. OK, perhaps we have some confusion here. Are you saying that you entered the Unicode characters directly into your Haskell source as literals? In other words, you did not type: backslash two five two but instead just typed the umlaut on the keyboard? If so, that won't work directly -- I think. Maybe somebody can correct me on this, but my hunch is that would save the umlaut as UTF-8 when you save the .hs file. Then you will get a String which is supposed to have decoded Unicode data, instead having encoded UTF-8 data. You could wrap it with Codec.Binary.UTF8.String.decodeString from utf8-string and see if that helps. If it does, that'll be your problem. It's a complicated topic, I know. And the scary thing is that Unicode makes this all *easier*. > > I'll do some further investigating and give you some more details when I > have them, thanks in advance. > > Günther > > > John Goerzen schrieb: >> On Mon, May 04, 2009 at 04:44:04PM +0200, Guenther Schmidt wrote: >> >>> Hi John, >>> >>> I'm trying stuff like: >>> >>> dbc <- connectSqlite3 "somedatabase" >>> run dbc "insert into someTable values (?)" [toSql "Günni"]. >>> >> SO what do you get back after adding: >> >> commit >> r <- quickQuery' dbc "select * from someTable" >> print r >> >> Just knowing it's garbled doesn't help. Need to know *how* it's >> garbled. >> >> But the problem is that \374 isn't Unicode at all. It's ISO-8859-1. >> You're not actually giving it Unicode data to start with. I believe >> the proper sequence is \252. >> >> For all I know, \374 may not even be a valid Unicode encoding (haven't >> tested it). >> >> Try \252. >> >> >> >>> I also tried: >>> >>> dbc <- connectSqlite3 "somedatabase" >>> run dbc "insert into someTable values ('Günni')" []. >>> >>> So since this is Haskell code I presume it's in UTF-8, my emacs stores >>> all my *.hs files as UTF-8 >>> >>> In either case the "ü" becomes garbled. >>> >>> With the previous version of HDBC, 1.1.6, this worked just fine. >>> >>> >>> It also garbles any Umlauts coming *out*, the source is an UTF-8 sqlite3 >>> db file. >>> >>> Günther >>> >>> >>> >>> John Goerzen schrieb: >>> >>>> GüŸnther Schmidt wrote: >>>> >>>> >>>>> Hi guys, >>>>> >>>>> for some reason, any way I try, all the Umlauts get garbled with HDBC 2.1. >>>>> HDBC 1.16 worked fine with any backend (ODBC, Sqlite3, ... what have you). >>>>> >>>>> Anybody else had similar problems and knows how to solve this? >>>>> >>>>> >>>> You need to be more specific, but it is likely you are trying to send >>>> something to HDBC that isn't encoded in UTF-8. HDBC 2.x has a global >>>> preference for UTF-8 now, actually partly to resolve complaints like this. >>>> >>>> If you are feeding it ISO-8859-1 data or somesuch, try giving it UTF-8 >>>> instead. >>>> >>>> -- John >>>> >>>> >>> > > >

3 2

Darcs and binary files
by Gü?nther Schmidt 04 May '09

04 May '09

Hi, I've gotten used to darcs now, and use it for my project. There is one quirk though, I also added some rather large binary files, an sqlite database and an MS Access database to the repository. Now whenever I do a push from a branch into the main trunk the system runs out of memory. Guess I should have left the binary files out of there? Günther

2 1

Re: [Haskell-cafe] HDBC 2.1, UTF8 and Umlauts
by John Goerzen 04 May '09

04 May '09

Günther Schmidt wrote: > Am 04.05.2009 um 21:07 schrieb John Goerzen: > >> Günther Schmidt wrote: >>> Hi John, >>> >>> I'm afraid so. >>> >>> If it came back as an SqlString "G\252nni" then it propably wouldn't >>> be a problem. >> Can you boil this down to a few-line self-contained test program so I >> can try it myself? I do have test cases for Unicode stuff and they >> are >> all passing here. I would like to be able to eliminate your >> environment >> as a culprit. >> > I'll try, it will take me some time since I've never written a test in > haskell. You don't really have to make a special unit test. Just a simple client program that creates a table, inserts some data, reads it back, and demonstrates the corruption. > That'll be harder to do since I'm on XP here, but I'll try that too, No problem. It's just runghc setup configure -f buildtests runghc setup build and just run runtests >>> >>> >>> >>> Am 04.05.2009 um 20:47 schrieb John Goerzen: >>> >>>> Günther Schmidt wrote: >>>>> Hi John, >>>>> >>>>> what I just noticed is that *all* strings come back as >>>>> SqlByteStrings. >>>> That's normal, and pretty much irrelevant, since fromSql takes care >>>> of it. >>>> >>>> It's documented, even: the SqlByteString is assumed to be in UTF-8, >>>> and >>>> is decoded when converted to a String. >>>> >>>> It is not correct to have \252 in the SqlByteString. The proper >>>> sequence there is \xc3\xbc. When converted to String, *then* it >>>> should >>>> be \252. >>>> >>>> Are you positive you're seeing \252 in the SqlByteString? That >>>> doesn't >>>> make any sense to me. It's not a valid UTF-8 encoding. >>>> >>> How have Umlauts been behaving on your end? Were they as mean to you >>> as they were to me? >>> >>> >>> >>>>> ie. I get my "Günni", (G\252nni), back as an SqlByteString "G >>>>> \252nni" >>>>> instead of an SqlString "G\252nni". >>>>> >>>>> So when I cast, ie. fromSql x :: String, I get an "G\65533nni", >>>>> which >>>>> is where the garbling occurs. >>>>> >>>>> >>>>> BTW this is the 3rd time now that I'm writing the same bloody >>>>> email, >>>>> my email client "clipped" the previous 2. >>>>> >>>>> Günther >>>>> >>>>> >>> > >

1 0

Re: [Haskell-cafe] HDBC 2.1, UTF8 and Umlauts
by John Goerzen 04 May '09

04 May '09

Günther Schmidt wrote: > Hi John, > > I'm afraid so. > > If it came back as an SqlString "G\252nni" then it propably wouldn't > be a problem. Can you boil this down to a few-line self-contained test program so I can try it myself? I do have test cases for Unicode stuff and they are all passing here. I would like to be able to eliminate your environment as a culprit. Incidentally, could you run make test in the HDBC-sqlite3 source directory? > > > > > Am 04.05.2009 um 20:47 schrieb John Goerzen: > >> Günther Schmidt wrote: >>> Hi John, >>> >>> what I just noticed is that *all* strings come back as >>> SqlByteStrings. >> That's normal, and pretty much irrelevant, since fromSql takes care >> of it. >> >> It's documented, even: the SqlByteString is assumed to be in UTF-8, >> and >> is decoded when converted to a String. >> >> It is not correct to have \252 in the SqlByteString. The proper >> sequence there is \xc3\xbc. When converted to String, *then* it >> should >> be \252. >> >> Are you positive you're seeing \252 in the SqlByteString? That >> doesn't >> make any sense to me. It's not a valid UTF-8 encoding. >> > > How have Umlauts been behaving on your end? Were they as mean to you > as they were to me? > > > >>> ie. I get my "Günni", (G\252nni), back as an SqlByteString "G\252nni" >>> instead of an SqlString "G\252nni". >>> >>> So when I cast, ie. fromSql x :: String, I get an "G\65533nni", which >>> is where the garbling occurs. >>> >>> >>> BTW this is the 3rd time now that I'm writing the same bloody email, >>> my email client "clipped" the previous 2. >>> >>> Günther >>> >>> > >

1 0

Re: [Haskell-cafe] HDBC 2.1, UTF8 and Umlauts
by John Goerzen 04 May '09

04 May '09

Günther Schmidt wrote: > Hi John, > > what I just noticed is that *all* strings come back as SqlByteStrings. That's normal, and pretty much irrelevant, since fromSql takes care of it. It's documented, even: the SqlByteString is assumed to be in UTF-8, and is decoded when converted to a String. It is not correct to have \252 in the SqlByteString. The proper sequence there is \xc3\xbc. When converted to String, *then* it should be \252. Are you positive you're seeing \252 in the SqlByteString? That doesn't make any sense to me. It's not a valid UTF-8 encoding. > > ie. I get my "Günni", (G\252nni), back as an SqlByteString "G\252nni" > instead of an SqlString "G\252nni". > > So when I cast, ie. fromSql x :: String, I get an "G\65533nni", which > is where the garbling occurs. > > > BTW this is the 3rd time now that I'm writing the same bloody email, > my email client "clipped" the previous 2. > > Günther > >

1 0

When is it OK to create a new mailing list?
by Mads Lindstrøm 04 May '09

04 May '09

Hi I wanted a mailing list for my project WxGeneric and I am wondering when it is OK to do so? How big must the potential audience be? Is there any kind of etiquette or guidelines? Here http://haskell.org/mailman/admin it says that I must have "the proper authority" to create a mailing list. What is meant by "proper authority"? Can I just try to create one and see if I am successful? Or must I request someone to do it? Regards, Mads Lindstrøm

6 5

HDBC 2.1, UTF8 and Umlauts
by Günther Schmidt 04 May '09

04 May '09

Hi guys, for some reason, any way I try, all the Umlauts get garbled with HDBC 2.1. HDBC 1.16 worked fine with any backend (ODBC, Sqlite3, ... what have you). Anybody else had similar problems and knows how to solve this? Günther

2 1

Estimating the time to garbage collect
by Neil Davies 04 May '09

04 May '09

Hi With the discussion on threads and priority, and given that (in Stats.c) there are lots of useful pieces of information that the run time system is collecting, some of which is already visible (like the total amount of memory mutated) and it is easy to make other measures available - it has raised this question in my mind: Given that you have access to that information (the stuff that comes out at the end of a run if you use +RTS -S) is it possible to estimate the time a GC will take before asking for one? Ignoring, at least for the moment, all the issues of paging, processor cache occupancy etc, what are the complexity drivers for the time to GC? I realise that it is going to depend on things like, volume of data mutated, count of objects mutated, what fraction of them are live etc - and even if it turns out that these things are very program specific then I have a follow-on question - what properties do you need from your program to be able to construct a viable estimate of GC time from a past history of such garbage collections? Why am I interested? There are all manners of 'real time' in systems, there is a vast class where a statistical bound (ie some sort of 'time to complete' CDF) is more than adequate for production use. If this is possible then it opens up areas where all the lovely properties of haskell can be exploited if only you had confidence in the timing behaviour. Cheers Neil

4 6