Non-deterministic behaviour of aeson's parser
I am observing a non-deterministic behaviour of aeson's parser. I'm writing here in addition to filing a bug report [1] to draw attention to this (pretty scary) problem. To try to reproduce this problem, do this: git clone https://gist.github.com/5604887.git aeson cd aeson ghc aeson.hs ./aeson | sort | uniq -c This is my result: 32 Left "key \"module\" not present" 55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present." 1913 Right () Can others reproduce this in their environments? Does anyone have ideas about where the bug may lie? Many aeson's dependencies do unsafe IO stuff that could lead to such consequences. Roman [1]: https://github.com/bos/aeson/issues/125
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 18/05/13 17:25, Roman Cheplyaka wrote:
I am observing a non-deterministic behaviour of aeson's parser.
I'm writing here in addition to filing a bug report [1] to draw attention to this (pretty scary) problem.
To try to reproduce this problem, do this:
git clone https://gist.github.com/5604887.git aeson cd aeson ghc aeson.hs ./aeson | sort | uniq -c
This is my result:
32 Left "key \"module\" not present" 55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present." 1913 Right ()
Can others reproduce this in their environments?
Does anyone have ideas about where the bug may lie? Many aeson's dependencies do unsafe IO stuff that could lead to such consequences.
Roman
[1]: https://github.com/bos/aeson/issues/125
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Here's my result: 3 Left "key \"module\" not present" 8 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present." 1989 Right () - -- Mateusz K. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (GNU/Linux) iQIcBAEBAgAGBQJRl7RYAAoJEM1mucMq2pqXt+AQALBsY3qgOawn4lHWK4myxXms ofwxyED2fJC+4BjjdfMrbHcWYpEM43K+QlAElx7sQcM/VCCjqRiJ61zHyLnicILE PwxF0/J0QQOkLF3/Q4ND7zgKqxbJWjOW2V6tf/WQulL4eKl1o40Xl4wuhoNqFrIA g5GKcY/MPdBzbcnOwep6DdRhm42SrO+WnmmWoNYPNWA43dZwatlBsCklS3Pj+SdS AYl10ZwNkMBT8Pj4N5A/z9C8LPKiZUDjuZxpMQT+PY4Ct318HtZVWDX3fM5QGvuE HVqBPlBZIQn+VWBDFVe3LqhfvAVpvmflJ99DVGeO73kBhEf4FMhVfnXcGDayYYn9 PGF4n7L7MP4hNVlQtQhWkLKgWz0Z+VND+6c8yGceEpak4WqpzSmBKXIpMa8fw7tz UXHUdJiVt6ZLGr1LjqDD3399+DCySj56XfyQk57xTn5gRjHtfz7QPSzgydDZzsJJ 8WHwQG1yHzNDfPATQwuZWss8FR53TXZbsn7zPfcJsro8bogNc2OH53HiVKoOm8l3 xH+A0D7169VCcKLvcVLmOb8gzAS2PiClm9/9H3/ho7iaU7gcYkZTaUO3JNz72mpn Z0dS2YIdgmUdEY6Ws5DSmySOmP0QdFUt7xMK2lfExugdPLatcT0D+CGvNRJpfNXB uL4I9+taoheV+3SLbM6M =iGjG -----END PGP SIGNATURE-----
My result: 2000 lines of "Right ()" ghc-pkg list aeson says "aeson-0.6.1.0" On May 18, 2013, at 8:25 PM, Roman Cheplyaka <roma@ro-che.info> wrote:
I am observing a non-deterministic behaviour of aeson's parser.
I'm writing here in addition to filing a bug report [1] to draw attention to this (pretty scary) problem.
To try to reproduce this problem, do this:
git clone https://gist.github.com/5604887.git aeson cd aeson ghc aeson.hs ./aeson | sort | uniq -c
This is my result:
32 Left "key \"module\" not present" 55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present." 1913 Right ()
Can others reproduce this in their environments?
Does anyone have ideas about where the bug may lie? Many aeson's dependencies do unsafe IO stuff that could lead to such consequences.
Roman
[1]: https://github.com/bos/aeson/issues/125
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Can't reproduce: % ./aeson | sort | uniq -c 2000 Right () % ./aeson | sort | uniq -c 2000 Right () % ./aeson | sort | uniq -c 2000 Right () % ./aeson | sort | uniq -c 2000 Right () % ./aeson | sort | uniq -c 2000 Right () Time 100: % ./aeson | sort | uniq -c 200000 Right () My packages: % ghc-pkg list /var/lib/ghc/package.conf.d Cabal-1.16.0 array-0.4.0.1 base-4.6.0.1 bin-package-db-0.0.0.0 binary-0.5.1.1 bytestring-0.10.0.2 containers-0.5.0.0 deepseq-1.3.0.1 directory-1.2.0.1 filepath-1.3.0.1 ghc-7.6.2 ghc-prim-0.3.0.0 haskell2010-1.1.1.0 haskell98-2.0.0.2 hoopl-3.9.0.0 hpc-0.6.0.0 integer-gmp-0.5.0.0 old-locale-1.0.0.5 old-time-1.1.0.1 pretty-1.1.1.0 process-1.1.0.2 rts-1.0 template-haskell-2.8.0.0 time-1.4.0.1 unix-2.6.0.1 /home/niklas/.ghc/x86_64-linux-7.6.2/package.conf.d HTTP-4000.2.8 HUnit-1.2.5.2 QuickCheck-2.6 Xauth-0.1 aeson-0.6.1.0 ansi-terminal-0.6 attoparsec-0.10.4.0 attoparsec-binary-0.2 base-unicode-symbols-0.2.2.4 blaze-builder-0.3.1.1 byteorder-1.0.4 cipher-aes-0.1.8 convertible-1.0.11.1 cpphs-1.16 dlist-0.5 ghc-paths-0.1.0.9 ghc-syb-utils-0.2.1.1 hashable-1.2.0.6 haskell-lexer-1.0 haskell-src-exts-1.13.5 hidapi-1.0 hlint-1.8.44 hscolour-1.20.3 hspec-1.5.4 hspec-expectations-0.3.2 io-choice-0.0.3 lifted-base-0.2.0.4 monad-control-0.3.2.1 mtl-2.1.2 network-2.4.1.2 parsec-3.1.3 pretty-show-1.5 primitive-0.5.0.1 quickcheck-io-0.1.0 random-1.0.1.1 robot-1.0.1.1 robot-1.1 setenv-0.1.0 stm-2.4.2 storable-record-0.0.2.5 syb-0.4.0 text-0.11.3.0 transformers-0.3.0.0 transformers-base-0.4.1 uniplate-1.6.10 unordered-containers-0.2.3.1 utility-ht-0.0.9 vector-0.10.0.1 vector-th-unbox-0.2.0.1 xhb-0.5.2012.11.23 zlib-0.5.4.1
CNR with aeson 0.6.1.0 and ghc 7.6.3. pkg-list output can be found at http://pastebin.com/Zuuujcaz On Saturday, May 18, 2013, Niklas Hambüchen wrote:
Can't reproduce:
% ./aeson | sort | uniq -c 2000 Right () % ./aeson | sort | uniq -c 2000 Right () % ./aeson | sort | uniq -c 2000 Right () % ./aeson | sort | uniq -c 2000 Right () % ./aeson | sort | uniq -c 2000 Right ()
Time 100:
% ./aeson | sort | uniq -c 200000 Right ()
My packages:
% ghc-pkg list /var/lib/ghc/package.conf.d Cabal-1.16.0 array-0.4.0.1 base-4.6.0.1 bin-package-db-0.0.0.0 binary-0.5.1.1 bytestring-0.10.0.2 containers-0.5.0.0 deepseq-1.3.0.1 directory-1.2.0.1 filepath-1.3.0.1 ghc-7.6.2 ghc-prim-0.3.0.0 haskell2010-1.1.1.0 haskell98-2.0.0.2 hoopl-3.9.0.0 hpc-0.6.0.0 integer-gmp-0.5.0.0 old-locale-1.0.0.5 old-time-1.1.0.1 pretty-1.1.1.0 process-1.1.0.2 rts-1.0 template-haskell-2.8.0.0 time-1.4.0.1 unix-2.6.0.1 /home/niklas/.ghc/x86_64-linux-7.6.2/package.conf.d HTTP-4000.2.8 HUnit-1.2.5.2 QuickCheck-2.6 Xauth-0.1 aeson-0.6.1.0 ansi-terminal-0.6 attoparsec-0.10.4.0 attoparsec-binary-0.2 base-unicode-symbols-0.2.2.4 blaze-builder-0.3.1.1 byteorder-1.0.4 cipher-aes-0.1.8 convertible-1.0.11.1 cpphs-1.16 dlist-0.5 ghc-paths-0.1.0.9 ghc-syb-utils-0.2.1.1 hashable-1.2.0.6 haskell-lexer-1.0 haskell-src-exts-1.13.5 hidapi-1.0 hlint-1.8.44 hscolour-1.20.3 hspec-1.5.4 hspec-expectations-0.3.2 io-choice-0.0.3 lifted-base-0.2.0.4 monad-control-0.3.2.1 mtl-2.1.2 network-2.4.1.2 parsec-3.1.3 pretty-show-1.5 primitive-0.5.0.1 quickcheck-io-0.1.0 random-1.0.1.1 robot-1.0.1.1 robot-1.1 setenv-0.1.0 stm-2.4.2 storable-record-0.0.2.5 syb-0.4.0 text-0.11.3.0 transformers-0.3.0.0 transformers-base-0.4.1 uniplate-1.6.10 unordered-containers-0.2.3.1 utility-ht-0.0.9 vector-0.10.0.1 vector-th-unbox-0.2.0.1 xhb-0.5.2012.11.23 zlib-0.5.4.1
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org <javascript:;> http://www.haskell.org/mailman/listinfo/haskell-cafe
First off, everyone reporting results to this thread: your bug report would be much more helpful if you included your OS/architecture/GHC version combo, as well as the results of re-running the tests if you build "hashable" with "cabal install -f-sse2". I have a funny feeling that this is a bug in hashable or unordered-containers. I'm guessing hashable, especially because of this: https://github.com/tibbe/hashable/issues/66 and because hashable has had subtle bugs in its C code before (see https://github.com/tibbe/hashable/issues/60). G On Sat, May 18, 2013 at 6:25 PM, Roman Cheplyaka <roma@ro-che.info> wrote:
I am observing a non-deterministic behaviour of aeson's parser.
I'm writing here in addition to filing a bug report [1] to draw attention to this (pretty scary) problem.
To try to reproduce this problem, do this:
git clone https://gist.github.com/5604887.git aeson cd aeson ghc aeson.hs ./aeson | sort | uniq -c
This is my result:
32 Left "key \"module\" not present" 55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present." 1913 Right ()
Can others reproduce this in their environments?
Does anyone have ideas about where the bug may lie? Many aeson's dependencies do unsafe IO stuff that could lead to such consequences.
Roman
[1]: https://github.com/bos/aeson/issues/125
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- Gregory Collins <greg@gregorycollins.net>
$ uname -a Linux clark-laptop 3.9.0-2-ARCH #1 SMP PREEMPT Tue Apr 30 09:48:29 CEST 2013 x86_64 GNU/Linux It'd take too long for my helpfulness to build with cabal install -fsse2 hashable and rebuild an environment. If someone writes a bash script to do it (using cabal-dev please!), I'd be more than happy to run it and post the results. - Clark On Saturday, May 18, 2013, Gregory Collins wrote:
First off, everyone reporting results to this thread: your bug report would be much more helpful if you included your OS/architecture/GHC version combo, as well as the results of re-running the tests if you build "hashable" with "cabal install -f-sse2".
I have a funny feeling that this is a bug in hashable or unordered-containers. I'm guessing hashable, especially because of this:
https://github.com/tibbe/hashable/issues/66
and because hashable has had subtle bugs in its C code before (see https://github.com/tibbe/hashable/issues/60).
G
On Sat, May 18, 2013 at 6:25 PM, Roman Cheplyaka <roma@ro-che.info<javascript:_e({}, 'cvml', 'roma@ro-che.info');>
wrote:
I am observing a non-deterministic behaviour of aeson's parser.
I'm writing here in addition to filing a bug report [1] to draw attention to this (pretty scary) problem.
To try to reproduce this problem, do this:
git clone https://gist.github.com/5604887.git aeson cd aeson ghc aeson.hs ./aeson | sort | uniq -c
This is my result:
32 Left "key \"module\" not present" 55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present." 1913 Right ()
Can others reproduce this in their environments?
Does anyone have ideas about where the bug may lie? Many aeson's dependencies do unsafe IO stuff that could lead to such consequences.
Roman
[1]: https://github.com/bos/aeson/issues/125
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org <javascript:_e({}, 'cvml', 'Haskell-Cafe@haskell.org');> http://www.haskell.org/mailman/listinfo/haskell-cafe
-- Gregory Collins <greg@gregorycollins.net <javascript:_e({}, 'cvml', 'greg@gregorycollins.net');>>
Indeed it looks like a bug in hashable — it goes away with hashable-1.1.2.5. Building with -f-sse2 results in a linker error Loading package hashable-1.2.0.7 ... linking ... ghc: /home/feuerbach/tmp/aeson/.cabal-sandbox/lib/i386-linux-ghc-7.6.3/hashable-1.2.0.7/libHShashable-1.2.0.7.a: unknown symbol `hashable_siphash24_sse2' ghc: unable to load package `hashable-1.2.0.7' Roman * Gregory Collins <greg@gregorycollins.net> [2013-05-18 22:19:38+0200]
First off, everyone reporting results to this thread: your bug report would be much more helpful if you included your OS/architecture/GHC version combo, as well as the results of re-running the tests if you build "hashable" with "cabal install -f-sse2".
I have a funny feeling that this is a bug in hashable or unordered-containers. I'm guessing hashable, especially because of this:
https://github.com/tibbe/hashable/issues/66
and because hashable has had subtle bugs in its C code before (see https://github.com/tibbe/hashable/issues/60).
G
On Sat, May 18, 2013 at 6:25 PM, Roman Cheplyaka <roma@ro-che.info> wrote:
I am observing a non-deterministic behaviour of aeson's parser.
I'm writing here in addition to filing a bug report [1] to draw attention to this (pretty scary) problem.
To try to reproduce this problem, do this:
git clone https://gist.github.com/5604887.git aeson cd aeson ghc aeson.hs ./aeson | sort | uniq -c
This is my result:
32 Left "key \"module\" not present" 55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present." 1913 Right ()
Can others reproduce this in their environments?
Does anyone have ideas about where the bug may lie? Many aeson's dependencies do unsafe IO stuff that could lead to such consequences.
Roman
[1]: https://github.com/bos/aeson/issues/125
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- Gregory Collins <greg@gregorycollins.net>
It's definitely hashable. Here's a minimal failing test case: https://gist.github.com/gregorycollins/5748445 On Sat, May 18, 2013 at 11:38 PM, Roman Cheplyaka <roma@ro-che.info> wrote:
Indeed it looks like a bug in hashable — it goes away with hashable-1.1.2.5.
Building with -f-sse2 results in a linker error
Loading package hashable-1.2.0.7 ... linking ... ghc: /home/feuerbach/tmp/aeson/.cabal-sandbox/lib/i386-linux-ghc-7.6.3/hashable-1.2.0.7/libHShashable-1.2.0.7.a: unknown symbol `hashable_siphash24_sse2' ghc: unable to load package `hashable-1.2.0.7'
Roman
* Gregory Collins <greg@gregorycollins.net> [2013-05-18 22:19:38+0200]
First off, everyone reporting results to this thread: your bug report would be much more helpful if you included your OS/architecture/GHC version combo, as well as the results of re-running the tests if you build "hashable" with "cabal install -f-sse2".
I have a funny feeling that this is a bug in hashable or unordered-containers. I'm guessing hashable, especially because of this:
https://github.com/tibbe/hashable/issues/66
and because hashable has had subtle bugs in its C code before (see https://github.com/tibbe/hashable/issues/60).
G
On Sat, May 18, 2013 at 6:25 PM, Roman Cheplyaka <roma@ro-che.info> wrote:
I am observing a non-deterministic behaviour of aeson's parser.
I'm writing here in addition to filing a bug report [1] to draw attention to this (pretty scary) problem.
To try to reproduce this problem, do this:
git clone https://gist.github.com/5604887.git aeson cd aeson ghc aeson.hs ./aeson | sort | uniq -c
This is my result:
32 Left "key \"module\" not present" 55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present." 1913 Right ()
Can others reproduce this in their environments?
Does anyone have ideas about where the bug may lie? Many aeson's dependencies do unsafe IO stuff that could lead to such consequences.
Roman
[1]: https://github.com/bos/aeson/issues/125
_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
-- Gregory Collins <greg@gregorycollins.net>
-- Gregory Collins <greg@gregorycollins.net>
participants (6)
-
Clark Gaebel -
Gregory Collins -
Mateusz Kowalczyk -
MigMit -
Niklas Hambüchen -
Roman Cheplyaka