Re: [Haskell-cafe] Package Takeover: `toml`

The TOML format is optimized for human-readability, not space efficiency. And it has some data redundancy, which makes it great for the application configuration use-case but not so great as a serialization format. If you are using TOML to serialise data or you need to parse 3-10 GB of application configuration, there's a chance that something can be improved without streaming TOML parsing. So streaming TOML parser looks like a very specific use-case that doesn't justify taking someone's package and making it the official TOML parser. Best regards, Dmitrii On Fri, 12 Mar 2021 at 12:40, Carter Schonwald wrote:

Hey Dmitrii, I believe emily has in mind fully incremental streaming support. Which requires a wildly different internal architecture than all the stuff in the aeson inspired design family

https://github.com/cartazio/streaming-machine-json Is an example of a constant space json parser with fully incremental consumption and emissions.

An predecessor was used in a work prject 5 years ago and it could handle multi gig json monsters like a champ. I never released it to hackage because I want to have a better / more reusable design. Parsing the same 3-10gb json with aeson was impossible on a machine that had hundreds of gigs of ram. :)

I believe emily has in mind similar levels of robustness

On Fri, Mar 12, 2021 at 7:08 AM Dmitrii Kovanikov wrote:

...

Hi everyone,

I feel extremely sad about this discussion for multiple reasons. But regarding the technical agenda:

...

I'm going to look at `toml-parser` in the meantime, but no toml library does what I have in mind (namely a full fledged implementation of the spec, streaming, deriving etc.), nor do many of them provide bidirectional serialization save `tomland`.

This does sound very disappointing to me and I don't fully understand the needs. Because:

* tomland is the official TOML parsing library in Haskell according to the TOML spec wiki https://github.com/toml-lang/toml/wiki * tomland fully supports the spec version 0.5.0, and the latest spec 1.0.0 was published relatively recently. And to my knowledge, it is the only Haskell library that supports the latest spec. * tomland is the most popular TOML parsing library according to reverse dependencies https://packdeps.haskellers.com/reverse on Hackage * tomland is based on explicit values, but nevertheless it provides deriving via Generics

I feel very confused about this situation. And again I feel like people the Haskell committees members are not willing to recognise other's people work and would rather rewrite everything from scratch instead of collaborating with existing projects created by people outside of committees. Even outside the Haskell community (the TOML org), tomland was acknowledged as the official TOML library, but not in the Haskell community itself.

At least, the following steps could be taken first:

* Why not open issues to tomland (or other libraries) and discuss the features you want? We maintain tomland for multiple years. The latest release was Feb 12 2021 (a month ago!). We constantly improve the implementation, fix parsing issues, improve interface and error-handling. Attempting to rewrite all this from scratch instead of collaborating with existing maintainers feels very unfriendly. * If you want to have the official TOML parsing library under the `toml` namespace on Hackage, again, why not ask the maintainers if they consider moving the library? And only after this discussion act accordingly. * If you are concerned about the lack of people working on the `tomland` library (which I don't fully understand, because in Kowainik we always have at least two people maintaining packages), then why not ask to add as a maintainer, instead of rewriting another library? Or even ask to move to the official `haskell` organization on GitHub, if you want to have more people maintaining the official package.

I mean, how am I supposed to feel motivated working on Haskell open-source projects, if my work can be just discarded at any time, the official library will be appointed without even communicating this desire? If I weren't subscribed to this thread, I probably wouldn't even know that something is going behind backs. We've put a lot of effort into tomland. We literally spent years of maintenance, UX improvements, bug fixes, writing tutorials and blog posts about the library and its implementation. And it is still not enough just to be respected and even give the chance to reply to the users needs?

That sounds very concerning to me. I don' feel like Haskell tech can move forward if people's (specifically if they are not associated with any Haskell leaders) work is disrespected.

Best regards, Dmitrii

On Fri, 12 Mar 2021 at 07:02, amindfv--- via Haskell-Cafe < haskell-cafe@haskell.org> wrote:

...

On Fri, Mar 12, 2021 at 06:28:44AM +0000, Emily Pillmore wrote:

...

Tom,

Look, I don't want to debate syntax and semantics here, but "burning need/desire/ambition" etc ( https://idioms.thefreedictionary.com/burning+desire ) is an extremely common colloquialism that doesn't imply an emergency, just a strongly felt urge. I can't apologize for my wording, but I'm sorry for the situation nonetheless.

I also don't want to debate semantics but I can tell you "I have a burning need" on a Hackage takeover has a different connotation of urgency than "I have a burning desire to write a toml parsing library and to have it be named 'toml'". I still feel duped and now condescended to as well. I do nonetheless appreciate your apology for the situation.

...

...

It's now seeming more just like a desire for the package name.

I'm going to look at `toml-parser` in the meantime, but no toml library does what I have in mind (namely a full fledged implementation of the spec, streaming, deriving etc.), nor do many of them provide bidirectional serialization save `tomland`. To reiterate Carter's point, hackage names are a community resource, and they deserve to be thought through carefully, so yes, the package name is part of the request. I do alot of community service to make sure things that take up precious Hackage real-estate are treated well, which is why `toml` posed an opportunity.

I'm actually open to the idea of using a simple name like "toml" for a best-in-class Haskell library, but I'd want to see proof that it is clearly the best in terms of implementation and adoption. I of course think my plans for toml parsing are the most wonderful, but if a consensus favorite package emerges and it's not mine I will step aside.

...

To that point, anything you put out is something I am interested in

investing time and effort into making a standard. Do you have any code currently, or is this a TODO on your list? Going through your hackage libraries, I see no source repository listings, issue trackers, or even an email to reach you by.

I do have code, yes. As mentioned earlier I'm in the middle of a rewrite. If there's more to discuss maybe we should move this conversation off-list as it's no longer about a package takeover?

Tom

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.

_______________________________________________ Haskell-Cafe mailing list To (un)subscribe, modify options or view archives go to: http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe Only members subscribed via the mailman list are allowed to post.