Re: [Haskell-cafe] Haskell SQL backends: How is data handed over?

My observation about the "scientific" package is also relevant here: most DBMSes have a different notion of "number" than most programming languages, so unless there is specifically a binary number field type it is probably necessary to send it as a string of some variety. The "scientific" package abstracts over this for Haskell.
You mean that formatting a Scientific number as String is more efficient than formatting an Integer/Double in its usual representation? How about parsing? In most parsing libraries I've seen [1,2,3] the following code transforms a string of digits to an integer. import Data.Char (ord) import Data.Foldable (foldl') -- The parser ensures only digit Chars -- are passed to this function. fromDigit :: Num a => Char -> a fromDigit = let z = ord '0' in \digit -> fromIntegral (ord digit - z) appendDigit :: Num a => a -> Char -> a appendDigit x d = 10 * x + fromDigit d digitsToNum :: Num a => [Char] -> a digitsToNum = foldl' appendDigit 0 If in addition to 'show' the function prependDigit (and its foldr cousin for digits behind the floating point) were particularly efficient for Scientific [*], that would indeed make a strong case for using Scientific in parsing and (SQL-)database applications. Olaf [1] https://hackage.haskell.org/package/parsec-3.1.15.1/docs/src/Text.Parsec.Tok... [2] https://hackage.haskell.org/package/megaparsec-9.2.1/docs/src/Text.Megaparse... [3] https://hackage.haskell.org/package/attoparsec-0.14.4/docs/src/Data.Attopars... [*] This seems not to be the case currently: The Num instance is implemented as Scientific c1 e1 * Scientific c2 e2 = Scientific (c1 * c2) (e1 + e2) Would it be a bad idea to add special cases for numbers of the form Scientific 1 e? It penalizes all multiplications with a pattern match.

Let me try this again.
Database backends generally use the scientific package to represent
numbers, because databases do not represent numbers as Haskell Int / C
(int) or even Haskell Integers (unless you mean something like
Data.Fixed). A database number typically has a number of decimal
digits and a precision again specified as decimal digits. (ex.
NUMERIC(9, 2)) It is not floating point.
The scientific package, to fit such applications (including that of
JSON), stores numbers as ASCII digits because they are so represented
(as decimal digits and decimal precision), not as machine numbers.
Accordingly, these numbers are stored and transmitted as decimal
digits, not as the sort of binary numbers you might expect.
On Thu, Aug 11, 2022 at 3:36 PM Olaf Klinke
My observation about the "scientific" package is also relevant here: most DBMSes have a different notion of "number" than most programming languages, so unless there is specifically a binary number field type it is probably necessary to send it as a string of some variety. The "scientific" package abstracts over this for Haskell.
You mean that formatting a Scientific number as String is more efficient than formatting an Integer/Double in its usual representation? How about parsing? In most parsing libraries I've seen [1,2,3] the following code transforms a string of digits to an integer.
import Data.Char (ord) import Data.Foldable (foldl')
-- The parser ensures only digit Chars -- are passed to this function. fromDigit :: Num a => Char -> a fromDigit = let z = ord '0' in \digit -> fromIntegral (ord digit - z)
appendDigit :: Num a => a -> Char -> a appendDigit x d = 10 * x + fromDigit d
digitsToNum :: Num a => [Char] -> a digitsToNum = foldl' appendDigit 0
If in addition to 'show' the function prependDigit (and its foldr cousin for digits behind the floating point) were particularly efficient for Scientific [*], that would indeed make a strong case for using Scientific in parsing and (SQL-)database applications.
Olaf
[1] https://hackage.haskell.org/package/parsec-3.1.15.1/docs/src/Text.Parsec.Tok... [2] https://hackage.haskell.org/package/megaparsec-9.2.1/docs/src/Text.Megaparse... [3] https://hackage.haskell.org/package/attoparsec-0.14.4/docs/src/Data.Attopars... [*] This seems not to be the case currently: The Num instance is implemented as
Scientific c1 e1 * Scientific c2 e2 = Scientific (c1 * c2) (e1 + e2)
Would it be a bad idea to add special cases for numbers of the form Scientific 1 e? It penalizes all multiplications with a pattern match.
-- brandon s allbery kf8nh allbery.b@gmail.com

Also let me say that databases *may* optimize this or allow use of native
types, with the risk of truncating or expecting a larger type than the
programmer expects. Scientific avoids this at the price of using the
textual representations the target expects.
On Thu, Aug 11, 2022, 15:45 Brandon Allbery
Let me try this again.
Database backends generally use the scientific package to represent numbers, because databases do not represent numbers as Haskell Int / C (int) or even Haskell Integers (unless you mean something like Data.Fixed). A database number typically has a number of decimal digits and a precision again specified as decimal digits. (ex. NUMERIC(9, 2)) It is not floating point.
The scientific package, to fit such applications (including that of JSON), stores numbers as ASCII digits because they are so represented (as decimal digits and decimal precision), not as machine numbers.
Accordingly, these numbers are stored and transmitted as decimal digits, not as the sort of binary numbers you might expect.
On Thu, Aug 11, 2022 at 3:36 PM Olaf Klinke
wrote: My observation about the "scientific" package is also relevant here: most DBMSes have a different notion of "number" than most programming languages, so unless there is specifically a binary number field type it is probably necessary to send it as a string of some variety. The "scientific" package abstracts over this for Haskell.
You mean that formatting a Scientific number as String is more efficient than formatting an Integer/Double in its usual representation? How about parsing? In most parsing libraries I've seen [1,2,3] the following code transforms a string of digits to an integer.
import Data.Char (ord) import Data.Foldable (foldl')
-- The parser ensures only digit Chars -- are passed to this function. fromDigit :: Num a => Char -> a fromDigit = let z = ord '0' in \digit -> fromIntegral (ord digit - z)
appendDigit :: Num a => a -> Char -> a appendDigit x d = 10 * x + fromDigit d
digitsToNum :: Num a => [Char] -> a digitsToNum = foldl' appendDigit 0
If in addition to 'show' the function prependDigit (and its foldr cousin for digits behind the floating point) were particularly efficient for Scientific [*], that would indeed make a strong case for using Scientific in parsing and (SQL-)database applications.
Olaf
[1]
https://hackage.haskell.org/package/parsec-3.1.15.1/docs/src/Text.Parsec.Tok...
[2] https://hackage.haskell.org/package/megaparsec-9.2.1/docs/src/Text.Megaparse... [3] https://hackage.haskell.org/package/attoparsec-0.14.4/docs/src/Data.Attopars... [*] This seems not to be the case currently: The Num instance is implemented as
Scientific c1 e1 * Scientific c2 e2 = Scientific (c1 * c2) (e1 + e2)
Would it be a bad idea to add special cases for numbers of the form Scientific 1 e? It penalizes all multiplications with a pattern match.
-- brandon s allbery kf8nh allbery.b@gmail.com

On Thu, 2022-08-11 at 15:45 -0400, Brandon Allbery wrote:
Let me try this again.
Database backends generally use the scientific package to represent numbers, because databases do not represent numbers as Haskell Int / C (int) or even Haskell Integers (unless you mean something like Data.Fixed). A database number typically has a number of decimal digits and a precision again specified as decimal digits. (ex. NUMERIC(9, 2)) It is not floating point.
Indeed, one could regard Scientific as the union of all types NUMERIC(x,y) for x,y natural numbers.
The scientific package, to fit such applications (including that of JSON), stores numbers as ASCII digits
The coefficient is a machine number, isn't it? So I'd not call it ASCII digits. Hence my proposal to investigate optimizing + and *. But probably this has been considered long ago.
because they are so represented (as decimal digits and decimal precision), not as machine numbers.
Accordingly, these numbers are stored and transmitted as decimal digits, not as the sort of binary numbers you might expect.
On Thu, Aug 11, 2022 at 3:36 PM Olaf Klinke
wrote: My observation about the "scientific" package is also relevant here: most DBMSes have a different notion of "number" than most programming languages, so unless there is specifically a binary number field type it is probably necessary to send it as a string of some variety. The "scientific" package abstracts over this for Haskell.
You mean that formatting a Scientific number as String is more efficient than formatting an Integer/Double in its usual representation? How about parsing? In most parsing libraries I've seen [1,2,3] the following code transforms a string of digits to an integer.
import Data.Char (ord) import Data.Foldable (foldl')
-- The parser ensures only digit Chars -- are passed to this function. fromDigit :: Num a => Char -> a fromDigit = let z = ord '0' in \digit -> fromIntegral (ord digit - z)
appendDigit :: Num a => a -> Char -> a appendDigit x d = 10 * x + fromDigit d
digitsToNum :: Num a => [Char] -> a digitsToNum = foldl' appendDigit 0
If in addition to 'show' the function prependDigit (and its foldr cousin for digits behind the floating point) were particularly efficient for Scientific [*], that would indeed make a strong case for using Scientific in parsing and (SQL-)database applications.
Olaf
[1] https://hackage.haskell.org/package/parsec-3.1.15.1/docs/src/Text.Parsec.Tok... [2] https://hackage.haskell.org/package/megaparsec-9.2.1/docs/src/Text.Megaparse... [3] https://hackage.haskell.org/package/attoparsec-0.14.4/docs/src/Data.Attopars... [*] This seems not to be the case currently: The Num instance is implemented as
Scientific c1 e1 * Scientific c2 e2 = Scientific (c1 * c2) (e1 + e2)
Would it be a bad idea to add special cases for numbers of the form Scientific 1 e? It penalizes all multiplications with a pattern match.
participants (2)
-
Brandon Allbery
-
Olaf Klinke