Hi, I am using haskelldb and haskelldb-hdbc-sqlite3. Well, I finally got the source compiled and ran, I got this error: App: user error (SQL error: SqlError {seState = "", seNativeError = 21, seErrorMsg = "prepare 74: SELECT subject,\n timestamp\nFROM notes as T1\nORDER BY timestamp DESC: library routine called out of sequence"}) Any clue what I should check? Thanks. -- 竹密岂妨流水过 山高哪阻野云飞
Hi Cloud, this often occurs when the path to the database includes a non-ascii character. In my dev environment, the path to the database deliberately contains an umlaut and the original code base of hdbc.sqlite3 from John Goerzen, version 2.0 & version 2.1 thus does not work. John Goerzen, the author of HDBC has considerably rewritten some parts of his hdbc package to use utf8-string wrapping, which includes wrapping the connection string, and in my case caused considerable problems, it just wouldn't work. So my solution was to rollback all these changes where he used the utf8-wrapping, which was quite a lot of work. I did communicate this to John, but he insists that his current solution is correct, and I'm not going to argue with him. Anyway what you can do, for now, is to put your sqlite3 database file into a location where the path contains no non-ascii characters, that should fix the problem. You may experience other, utf8-wrapping related problems, for instance when you want to insert non-ascii strings into varchar columns. They may not come back as you put them in. HTH Günther Magicloud Magiclouds schrieb:
Hi, I am using haskelldb and haskelldb-hdbc-sqlite3. Well, I finally got the source compiled and ran, I got this error: App: user error (SQL error: SqlError {seState = "", seNativeError = 21, seErrorMsg = "prepare 74: SELECT subject,\n timestamp\nFROM notes as T1\nORDER BY timestamp DESC: library routine called out of sequence"}) Any clue what I should check? Thanks.
Günther Schmidt wrote:
Hi Cloud,
this often occurs when the path to the database includes a non-ascii character.
In my dev environment, the path to the database deliberately contains an umlaut and the original code base of hdbc.sqlite3 from John Goerzen, version 2.0 & version 2.1 thus does not work.
This is conflating many issues. I do recall some discussion about data within a database; I don't recall one about the filename of it, which would certainly be a separate discussion. I can see why a connectRaw or some such function could be useful if you want to pass a raw binary string as the file path to Sqlite3. (I don't think any other DB would have a use for such a thing).
John Goerzen, the author of HDBC has considerably rewritten some parts of his hdbc package to use utf8-string wrapping, which includes wrapping the connection string, and in my case caused considerable problems, it just wouldn't work. So my solution was to rollback all these changes where he used the utf8-wrapping, which was quite a lot of work. I did
And unnecessary work at that, if all you cared about was the filename. I see your point on the filename, and tweaking that would have been a one-line fix for you. The mess we had before was this huge cloud of **UNDEFINED BEHAVIOR** when dealing with anything other than 7-bit ASCII. Databases could have some encodings, systems could have encodings, and it was all a huge fiasco. So with HDBC 2, what we have is: * If you want to communicate with the database in a "raw" manner, use ByteStrings. If you want a String out of it, convert it yourself. * If you want to use Strings to communicate with the databases, these will automatically be converted to the appropriate Unicode representation by the library. For all current database backends, that means converting them to a UTF-8 CStringLen type of thing, and back.
Anyway what you can do, for now, is to put your sqlite3 database file into a location where the path contains no non-ascii characters, that should fix the problem.
His problem is not caused by non-ASCII characters.
You may experience other, utf8-wrapping related problems, for instance when you want to insert non-ascii strings into varchar columns. They may not come back as you put them in.
They will, unless you are doing something weird like putting Latin1 8-bit text into a String and passing it to HDBC as a String, when the documentation specifically states that Strings are expected to be in the Unicode space. As I recall, that is specifically what you were doing. That doesn't mean I haven't provided an outlet for you to do deal with things in the Latin1 space (see the ByteString discussion above.) But in truth, HDBC is not a character set conversion library, nor should it be. If you have more complex needs than Unicode Strings, use one of the many quality encoding libraries available for Haskell, and combine it with the ByteString features in HDBC. Every popular database that I am aware of can either speak UTF-8 directly, or convert transparently to and from it. So, to summarize: 1) This is not the original poster's problem. 2) HDBC 2 is simpler than HDBC 1, and actually defines behavior in terms of character sets rather than leaving it completely undefined. 3) HDBC 2 standardizes character sets around UTF-8, the most common global standard, and structures its API in a way that this is transparent when you want it to be, and available for manual processing when you want that. 4) Nothing requires you to use UTF-8, which is why the ByteString API is there. 5) A one-line patch would have fixed your filename connection issue. 6) If memory serves, your "not getting things back" is because you are storing non-Unicode data in your Strings, and then using an improper API to store it. -- John
John Goerzen wrote:
I do recall some discussion about data within a database; I don't recall one about the filename of it, which would certainly be a separate discussion. I can see why a connectRaw or some such function could be
I have just pushed a patch to my git repo that adds connectSqlite3Raw to help in these cases. You can grab the diff directly here: http://git.complete.org/hdbc-sqlite3?a=commitdiff_plain;h=0ef5df694d74cfaea7... -- John
Magicloud Magiclouds wrote:
Hi, I am using haskelldb and haskelldb-hdbc-sqlite3. Well, I finally got the source compiled and ran, I got this error: App: user error (SQL error: SqlError {seState = "", seNativeError = 21, seErrorMsg = "prepare 74: SELECT subject,\n timestamp\nFROM notes as T1\nORDER BY timestamp DESC: library routine called out of sequence"}) Any clue what I should check? Thanks.
At the HDBC level, I would say: I suspect that you have used a function that returns results lazily, but haven't completely read them before calling back into the database with something else. As an example, you should probably use quickQuery' instead of quickQuery, unless you are fully prepared to accept the consequences of reading data lazily from a database. I am not very familiar with HaskellDB, and can't really comment on what it's doing under the hood. If it is returning results to you lazily, make sure you have completely consumed them before sending more queries to the database. If you can post some example code, it would likely help. -- John
Hi John, let me first of all apologize, I didn't mean to criticize you, I'm sure you had good reasons for those changes, I'm merely mean to state how they did affect me after switching to HDBC 2.1. Since after the rollback they no longer occurred I surmise that there is a connection. The error that Magicloud describes I recall only occurring when an umlaut in the path to the database file, that may or may not be due to the utf8-wrapping or it may just be a problem of the sqlite3.dll. No harm in trying if it can be solved this way by just moving the database file to one without any non-ascii characters. While I don't think that umlauts actually cause Magiclouds problem I did notice that he signs his emails with Chinese symbols, quite possible thus, that he has paths on his machine with non-ascii characters. Günther John Goerzen schrieb:
Magicloud Magiclouds wrote:
Hi, I am using haskelldb and haskelldb-hdbc-sqlite3. Well, I finally got the source compiled and ran, I got this error: App: user error (SQL error: SqlError {seState = "", seNativeError = 21, seErrorMsg = "prepare 74: SELECT subject,\n timestamp\nFROM notes as T1\nORDER BY timestamp DESC: library routine called out of sequence"}) Any clue what I should check? Thanks.
At the HDBC level, I would say:
I suspect that you have used a function that returns results lazily, but haven't completely read them before calling back into the database with something else. As an example, you should probably use quickQuery' instead of quickQuery, unless you are fully prepared to accept the consequences of reading data lazily from a database.
I am not very familiar with HaskellDB, and can't really comment on what it's doing under the hood. If it is returning results to you lazily, make sure you have completely consumed them before sending more queries to the database.
If you can post some example code, it would likely help.
-- John
Günther Schmidt wrote:
Hi John,
let me first of all apologize, I didn't mean to criticize you, I'm sure you had good reasons for those changes, I'm merely mean to state how they did affect me after switching to HDBC 2.1.
No, I completely understand and I'm not offended; but I didn't want people reading this message to get the wrong idea about the state of Unicode support in HBDC. -- John
participants (3)
-
Günther Schmidt -
John Goerzen -
Magicloud Magiclouds