August 2022 - Haskell-Cafe

Re: [Haskell-cafe] Haskell SQL backends: How is data handed over?
by Olaf Klinke 12 Aug '22

12 Aug '22

Anthony Clayden wrote: > What is your evidence that number/String is a concern here? It's all > happening in-memory. > > Whereas sending whatever to SQL then needs putting it to disk, which is > orders-of-magnitude slower. (Or is this an in-memory database?) > > This thread has already devoted more cycles to speculation than you're > going to reclaim through number formatting, methinks. Very good point. I'm fairly new to SQL, have a solution that works and started to wonder how efficient it is. Since the way data is passed to and from the DB engine affects every database interaction, for every user, I thought it might be worthwile investigating. And as I said earlier, profiling other code has proved that number parsing can make up a significant portion of a data-heavy program's CPU time. Thus it's not speculation entirely. Olaf

1 0

Re: [Haskell-cafe] Haskell SQL backends: How is data handed over?
by Anthony Clayden 11 Aug '22

11 Aug '22

> the backend constructs a complete SQL query statement in form of a string (including the SqlValues), and hands that to the database driver to parse. Is that indeed so? > Hi Olaf, that's the usual format for any HLL to pass statements (queries or updates) to SQL. Most HLLs provide some sort of modulate/demodulate so your program doesn't see the raw numbers-as-strings, but the parsing is going on in the background. Going via CSV isn't really saving anything: the "bulk insert" is again parsing those commas and the bits in between. (A dedicated SQL engine is probably more efficient than Haskell's string handling.) AntC

3 2

Re: [Haskell-cafe] Haskell SQL backends: How is data handed over?
by Anthony Clayden 11 Aug '22

11 Aug '22

> @Olaf You mean that formatting a Scientific number as String is more efficient > than formatting an Integer/Double in its usual representation? > How about parsing? I'm not going to defend the 'efficiency' of Haskell's String formatting/parsing, but ... What is your evidence that number/String is a concern here? It's all happening in-memory. Whereas sending whatever to SQL then needs putting it to disk, which is orders-of-magnitude slower. (Or is this an in-memory database?) This thread has already devoted more cycles to speculation than you're going to reclaim through number formatting, methinks.

1 0

Just released: cabal 3.8.1.0
by Artem Pelenitsyn 11 Aug '22

11 Aug '22

---------- Forwarded message --------- From: Mikolaj Konarski <mikolaj(a)well-typed.com> Date: Thu, Aug 11, 2022 at 7:05 PM Subject: Just released: cabal 3.8.1.0 To: <cabal-devel(a)haskell.org> Here's the happy news. :) Let me copy below the contents of the announcement from https://discourse.haskell.org/t/just-released-cabal-3-8-1-0/4920 but, additionally, on this mailing list I'd like to personally thank Emily and Oleg who, among all the giants on whose shoulders we stand, have most directly affected 3.8.1.0 with their hearts and wisdom. Haskell community owes you big time. --- I’m proud to announce 3.8.1.0 version of Cabal the library and cabal-install the tool. It’s the fruit of concerted effort of innumerous collaborators, from users, through developers, to mentors, caretakers and administrators of our linked community and infrastructure sections. You, people, rock. This version works with the just released GHC 9.4.1 (in fact, for Windows it’s probably necessary) and is already available form GHCup and other channels. More technical details, APIs, changelogs and regressions can be found at Release cabal v3.8.1.0 · haskell/cabal · GitHub. The release is huge, so let me mention only a few random highlights: public/private sublibraries are fully supported and no longer experimental (6. Package Description — Cabal 3.8.1.0 User's Guide) you can now put in your cabal.project something like import: https://www.stackage.org/lts-18.5/cabal.config and it works (7. cabal.project Reference — Cabal 3.8.1.0 User's Guide) new Cabal-syntax and cabal-install-solver packages have been split off there’s no more confusion of compiler options for the local code and for the dependencies (https://github.com/haskell/cabal/pull/7973) one can now define code generators in test stanzas, which is one more principled step away from Custom Setups (https://github.com/haskell/cabal/pull/7688) Cabal badly needs your feedback and your contributions. Please keep them coming. _______________________________________________ cabal-devel mailing list cabal-devel(a)haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel

1 0

Re: [Haskell-cafe] Haskell SQL backends: How is data handed over?
by Olaf Klinke 11 Aug '22

11 Aug '22

> My observation about the "scientific" package is also relevant here: > most DBMSes have a different notion of "number" than most programming > languages, so unless there is specifically a binary number field type > it is probably necessary to send it as a string of some variety. The > "scientific" package abstracts over this for Haskell. You mean that formatting a Scientific number as String is more efficient than formatting an Integer/Double in its usual representation? How about parsing? In most parsing libraries I've seen [1,2,3] the following code transforms a string of digits to an integer. import Data.Char (ord) import Data.Foldable (foldl') -- The parser ensures only digit Chars -- are passed to this function. fromDigit :: Num a => Char -> a fromDigit = let z = ord '0' in \digit -> fromIntegral (ord digit - z) appendDigit :: Num a => a -> Char -> a appendDigit x d = 10 * x + fromDigit d digitsToNum :: Num a => [Char] -> a digitsToNum = foldl' appendDigit 0 If in addition to 'show' the function prependDigit (and its foldr cousin for digits behind the floating point) were particularly efficient for Scientific [*], that would indeed make a strong case for using Scientific in parsing and (SQL-)database applications. Olaf [1] https://hackage.haskell.org/package/parsec-3.1.15.1/docs/src/Text.Parsec.To… [2] https://hackage.haskell.org/package/megaparsec-9.2.1/docs/src/Text.Megapars… [3] https://hackage.haskell.org/package/attoparsec-0.14.4/docs/src/Data.Attopar… [*] This seems not to be the case currently: The Num instance is implemented as Scientific c1 e1 * Scientific c2 e2 = Scientific (c1 * c2) (e1 + e2) Would it be a bad idea to add special cases for numbers of the form Scientific 1 e? It penalizes all multiplications with a pattern match.

2 3

Re: [Haskell-cafe] Haskell SQL backends: How is data handed over?
by Olaf Klinke 10 Aug '22

10 Aug '22

Thanks all for the replies. It seems in my specific setup values are indeed passed as strings, see below. Oliver Charles wrote: > I can't speak for SQL in general, but if you're targeting PostgreSQL note that the hasql library uses a binary wire format. Good to know, thanks. It might be that for my specific case the database choice ship has sailed, unfortunately. Anthony Clayden wrote: > Going via CSV isn't really saving anything: the "bulk insert" is again > parsing those commas and the bits in between. (A dedicated SQL engine is > probably more efficient than Haskell's string handling.) That was what I was thinking, too: The bulk insert must also parse the numbers, so it would be kind of a waste to have Haskell parse the numbers, format them as Strings in the CSV just to have the SQL server parse them again. Brandon Allbery wrote: > More specifically, a query or DML statement is a string with ? > placeholders for values, which is "prepare"d to turn it into internal > format and then executed one or more times with values supplied. Sending the SQL statement as string is probably fine. I guess it depends on the specific HDBC backend whether run :: Connection -> String -> [SqlValue] -> IO Integer also sends the SqlValue arguments as String. Specifically, I chose persistent with HDBC-odbc backend because (1) We're talking to a Microsoft SQL server from a Linux client via unixODBC, (2) We chose Microsoft SQL to ensure trouble-free integration into the Microsoft office world, (3) I expect a web application will emerge from this, hence persistent because it integrates well with Yesod. In [HDBC-odbc] it is implemented in sqlBindValue, sqlBindParameter which contains CString -- ^ Parameter value pointer The actual value (if not a string) is converted to ByteString (using the Convertible class) and placed into the CString, it seems. The Convertible SqlValue Bytestring instance uses fromString and Convertible SqlValue String which in turn uses show [HDBC]. So yep, to ODBC the SqlValues are passed as Strings. HDBC-sqlite3 does the same. I opened [1]. Olaf [HDBC-odbc] https://hackage.haskell.org/package/HDBC-odbc-2.5.0.1/docs/src/Database-HDB… [HDBC] https://hackage.haskell.org/package/HDBC-2.4.0.4/docs/src/Database.HDBC.Sql… [1] https://github.com/hdbc/hdbc/issues/56

2 1

Haskell SQL backends: How is data handed over?
by Olaf Klinke 10 Aug '22

10 Aug '22

Dear Café, I was discussing with a friend the fastest way to shovel data into a SQL database after parsing the raw (textual) data with Haskell. I am concerned about numeric conversion: In my experience parsing the file structure is fast. The majority of the time is spent converting strings of digits to numbers. I hoped that once Haskell has constructed a Double, the backend driver can hand that binary value over to the database engine. But Database.HDBC.Statement has a field orginalQuery :: String and HDBC has functions prepare :: Connection -> String -> IO Statement run :: Connection -> String -> [SqlValue] -> IO Integer which makes me wonder whether the backend constructs a complete SQL query statement in form of a string (including the SqlValues), and hands that to the database driver to parse. Is that indeed so? If yes, the numbers should never be parsed in the first place: The DB will do that anyways. I might as well construct a new CSV file and let the database do a bulk insert on that. The above is specific to the HDBC backend. If there are other backends/frameworks that do the marshalling more efficiently, please let me know. Thanks Olaf

4 3

Re: [Haskell-cafe] ghc doesn't work after installing 9.4.1 on my Mac
by Shayne Fletcher 09 Aug '22

09 Aug '22

Known issue George. See e.g. https://gitlab.haskell.org/ghc/ghc/-/issues/21985 On Tue, Aug 9, 2022, 4:33 PM George Colpitts <george.colpitts(a)gmail.com> wrote: > Unfortunately ghc doesn't work after installing 9.4.1 on my Mac. Does it > work for others? > > After the install finishes when I do the following: > > $ ghc --version > bash: /usr/local/bin/ghc: Permission denied > $ sudo chmod +x /usr/local/bin/ghc > $ghc --version > /usr/local/bin/ghc: line 1: exec: : not found > $ cat /usr/local/bin/ghc > exec "$executablename" -B"$libdir" ${1+"$@"} > $ cat /usr/local/bin/ghc-9.4.1 > #!/bin/sh > exedir="/usr/local/lib/ghc-9.4.1/bin" > exeprog="ghc-9.4.1" > executablename="/usr/local/lib/ghc-9.4.1/bin/ghc-9.4.1" > bindir="/usr/local/bin" > libdir="/usr/local/lib/ghc-9.4.1/lib" > docdir="/usr/local/share/doc/ghc-9.4.1" > includedir="/usr/local/include" > > exec "$executablename" -B"$libdir" ${1+"$@"} > > > Thanks > George > > > > On Sun, Aug 7, 2022 at 6:30 PM Ben Gamari <ben(a)well-typed.com> wrote: > >> The GHC developers are very pleased to announce the availability of GHC >> 9.4.1. Binary distributions, source distributions, and documentation are >> available at downloads.haskell.org: >> >> https://downloads.haskell.org/ghc/9.4.1 >> >> This release includes: >> >> - A new profiling mode, `-fprof-late`, which adds automatic cost-center >> annotations to all top-level functions *after* Core optimisation has >> run. This provides informative profiles while interfering >> significantly less with GHC's aggressive optimisations, making it >> easier to understand the performance of programs which depend upon >> simplification.. >> >> - A variety of plugin improvements including the introduction of a new >> plugin type, *defaulting plugins*, and the ability for typechecking >> plugins to rewrite type-families. >> >> - An improved constructed product result analysis, allowing unboxing of >> nested structures, and a new boxity analysis, leading to less reboxing. >> >> - Introduction of a tag-check elision optimisation, bringing >> significant performance improvements in strict programs. >> >> - Generalisation of a variety of primitive types to be levity >> polymorphic. Consequently, the `ArrayArray#` type can at long last be >> retired, replaced by standard `Array#`. >> >> - Introduction of the `\cases` syntax from [GHC proposal 0302]. >> >> - A complete overhaul of GHC's Windows support. This includes a >> migration to a fully Clang-based C toolchain, a deep refactoring of >> the linker, and many fixes in WinIO. >> >> - Support for multiple home packages, significantly improving support >> in IDEs and other tools for multi-package projects. >> >> - A refactoring of GHC's error message infrastructure, allowing GHC to >> provide diagnostic information to downstream consumers as structured >> data, greatly easing IDE support. >> >> - Significant compile-time improvements to runtime and memory >> consumption. >> >> - On overhaul of our packaging infrastructure, allowing full >> traceability of release artifacts and more reliable binary >> distributions. >> >> - Reintroduction of deep subsumption (which was previously dropped with >> the >> *simplified subsumption* change) as a language extension. >> >> - ... and much more. See the [release notes] for a full accounting. >> >> Note that, as 9.4.1 is the first release for which the released artifacts >> will >> all be generated by our Hadrian build system, it is possible that there >> will be >> packaging issues. If you enounter trouble while using a binary >> distribution, >> please open a [ticket]. Likewise, if you are a downstream packager, do >> consider >> migrating to [Hadrian] to run your build; the Hadrian build system can be >> built >> using `cabal-install`, `stack`, or the in-tree [bootstrap script]. See >> the accompanying >> [blog post] for details on migrating packaging to Hadrian. >> >> We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, >> Well-Typed, Tweag I/O, Serokell, Equinix, SimSpace, Haskell Foundation, >> and >> other anonymous contributors whose on-going financial and in-kind support >> has >> facilitated GHC maintenance and release management over the years. >> Finally, >> this release would not have been possible without the hundreds of >> open-source >> contributors whose work comprise this release. >> >> As always, do give this release a try and open a [ticket] if you see >> anything amiss. >> >> Happy testing, >> >> - Ben >> >> >> [GHC proposal 0302]: >> https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-c… >> [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new >> [bootstrap script]: >> https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… >> [Hadrian]: >> https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… >> [release notes]: >> https://downloads.haskell.org/~ghc/9.4.1/docs/users_guide/9.4.1-notes.html >> [blog post]: >> https://www.haskell.org/ghc/blog/20220805-make-to-hadrian.html >> _______________________________________________ >> ghc-devs mailing list >> ghc-devs(a)haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> > _______________________________________________ > ghc-devs mailing list > ghc-devs(a)haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >

1 0

[ANNOUNCE] GHC 9.4.1-rc1 is now available
by Ben Gamari 08 Aug '22

08 Aug '22

The GHC developers are happy to announce the availability of the first (and likely last) release candidate of GHC 9.4.1. Binary distributions, source distributions, and documentation are available at [downloads.haskell.org] This major release will include: - A new profiling mode, `-fprof-late`, which adds automatic cost-center annotations to all top-level functions *after* Core optimisation has run. This provides informative profiles while interfering significantly less with GHC's aggressive optimisations, making it easier to understand the performance of programs which depend upon simplification.. - A variety of plugin improvements including the introduction of a new plugin type, *defaulting plugins*, and the ability for typechecking plugins to rewrite type-families. - An improved constructed product result analysis, allowing unboxing of nested structures, and a new boxity analysis, leading to less reboxing. - Introduction of a tag-check elision optimisation, bringing significant performance improvements in strict programs. - Generalisation of a variety of primitive types to be levity polymorphic. Consequently, the `ArrayArray#` type can at long last be retired, replaced by standard `Array#`. - Introduction of the `\cases` syntax from [GHC proposal 0302] - A complete overhaul of GHC's Windows support. This includes a migration to a fully Clang-based C toolchain, a deep refactoring of the linker, and many fixes in WinIO. - Support for multiple home packages, significantly improving support in IDEs and other tools for multi-package projects. - A refactoring of GHC's error message infrastructure, allowing GHC to provide diagnostic information to downstream consumers as structured data, greatly easing IDE support. - Significant compile-time improvements to runtime and memory consumption. - On overhaul of our packaging infrastructure, allowing full traceability of release artifacts and more reliable binary distributions. - Reintroduction of deep subsumption (which was previously dropped with the *simplified subsumption* change) as a language extension. - ... and much more. See the [release notes] for a full accounting. Note that, as 9.4.1 is the first release for which the released artifacts will all be generated by our Hadrian build system, it is possible that there will be packaging issues. If you enounter trouble while using a binary distribution, please open a [ticket]. Likewise, if you are a downstream packager, do consider migrating to [Hadrian] to run your build; the Hadrian build system can be built using `cabal-install`, `stack`, or the in-tree [bootstrap script]. We will be publishing a blog post describing the migration process to Hadrian in the coming weeks. We would like to thank Microsoft Azure, GitHub, IOG, the Zw3rk stake pool, Tweag I/O, Serokell, Equinix, SimSpace, the Haskell Foundation, and other anonymous contributors whose on-going financial and in-kind support has facilitated GHC maintenance and release management over the years. Finally, this release would not have been possible without the hundreds of open-source contributors whose work comprise this release. As always, do give this release a try and open a [ticket] if you see anything amiss. Happy testing, - Ben [downloads.haskell.org] https://downloads.haskell.org/ghc/9.4.1-rc1/ [GHC proposal 0302]: https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0302-c… [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new [bootstrap script]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… [Hadrian]: https://gitlab.haskell.org/ghc/ghc/-/blob/e2520df3fffa0cf22fb19c5fb872832d1… [release notes]: https://downloads.haskell.org/~ghc/9.4.1-rc1/docs/users_guide/9.4.1-notes.h…

6 19

Agda Implementors' Meeting XXXI - Edinburgh, 10-16 November 2022
by Orestis Melkonian 06 Aug '22

06 Aug '22

--------------------------------------------------------------- Agda Implementors' Meeting XXXI Call for participation http://wiki.portal.chalmers.se/agda/pmwiki.php?n=Main.AIMXXXI --------------------------------------------------------------- The 31st Agda Implementors' Meeting will take place in Edinburgh, Scotland from Thursday 10 November 2022 to Wednesday 16 November 2022. Note that AIMXXXI was initially planned for 2019 but got postponed due to COVID-19, whilst we had several online meetings XXXII-XXXV. The meeting will consist of: * Presentations concerning theory, implementation, and use cases of Agda and other Agda-like languages. * Discussions around issues related to the Agda language. * Plenty of time to work in, on, under or around Agda, in collaboration with other participants. * A full day dedicated to the EuroProofNet COST action, including talks and discussions on translating libraries and proofs between Agda and other proof assistants. Please direct enquiries about that to Jesper Cockx <Jesper(a)sikanda.be>. To register for AIM XXXI, please fill out the form below and send it to Orestis Melkonian by email <orestis.melkonian(a)ed.ac.uk>, or simply edit the AIMXXI wiki page yourself. > --------------------------------------------------------------- > Registration form for Agda Implementors' Meeting XXXI > > Name: > > Title and optionally abstract (if you want to give a talk or lead a discussion): > > Suggestions for code sprints (optional): > > Dietary restrictions: > > Additional comments: > > --------------------------------------------------------------- More information is available at http://wiki.portal.chalmers.se/agda/pmwiki.php?n=Main.AIMXXXI Please spread the word about this meeting. Best regards, Orestis

1 0