- Libraries - Haskell.org

Re: Proposal: Add the unordered-containers package and the hashable package to the Haskell Platform
by wren ng thornton 22 Mar '13

22 Mar '13

On 3/20/13 1:02 PM, Thomas Schilling wrote:> To make this more precise the next version of hashable (say, 1.3) > would include this: > > newtype SipHashed a = SipHashed a > > class SipHashable a where > sipHashWithSalt :: Int -> a -> Int > > instance SipHashable a => Hashable (SipHashed a) where > hashWithSalt salt (SipHashed x) = sipHashWithSalt salt x > > Then all Hashable instances are taken from hashable-1.1. All Hashable > instances from hashable-1.2 are renamed to become instances of > SipHashable. That seems like a reasonable solution. I only ever use hashmaps explicitly for speed reasons and in applications that are not susceptible to DoS. The arguments about not wanting to introduce security issues due to some transitive dependency also hold for not wanting to introduce performance issues due to some transitive dependency. I respect all the folks working on web frameworks in Haskell, but there are also those of us who work on programs where compute time really is the number one concern (once an acceptable level of correctness and precision has been obtained); and when experiments are measured in days or weeks of cluster time, those "little" 1.3x and 1.5x slowdowns really hurt, let alone the 2.0x people've mentioned with siphash. One concern with the above approach: is "siphash" a sufficiently generic name, or is it just one hashing method that happens to deflect this DoS issue? I haven't read the paper, so... -- Live well, ~wren

4 3

Re: Proposal: Add the unordered-containers package and the hashable package to the Haskell Platform
by wren ng thornton 22 Mar '13

22 Mar '13

On 3/21/13 4:30 PM, Roman Cheplyaka wrote: > * Thomas Moertel <tom(a)moertel.com> [2013-03-21 15:58:58-0400] >> On Wed, Mar 20, 2013 at 5:05 PM, Johan Tibell <johan.tibell(a)gmail.com>wrote: >>> The problem is that if the secure hashing makes HashMap as slow as Map, >>> there's no point in having the former in the first place, as it only exists >>> to provide speed. >> >> I understand that HashMap exists to provide speed, but are we sure that >> everyone who uses it for speed understands the security implications? >> >> Regardless of whether speed-first or security-first is the default, users >> of HashMap can be partitioned into two groups: those who knowingly choose >> between speed and security, and those who do not. For the first group, >> either default is safe because members of that group will have the >> opportunity to choose the other option. But for the second group, only >> security-first is safe: if its not the default, members of the second >> group wont have the option to choose it. > > Why not simply educate them by placing a prominent notice in the > haddock? +1. Whenever there's this split between knowing-choice and ignorance, my preferred solution is always to educate people. Sometimes there are cases where one side of the choice is obviously superior, or where it's too difficult to explain what is at stake for the choice. In these cases I'm a fan of preferring the "safe" choice, because (a) it's obvious, or (b) education is too costly. But that's not so clear for something like significant speed benefits (which are crucial for some, and nice for all) vs defense against DoS (which are crucial for some, but irrelevant to others). Given that both benefits are crucial for some, either we should have no default ---for fear of offending either side--- or we should ignore both of those parties (since they are of approximately equal prominence), in which case things fall back to nice-for-all vs irrelevant-to-others, which favors speed as the default. Thankfully, in Haskell we have enough polymorphism that it's feasible to eschew a default choice, and it's also feasible to override whatever the default is. That puts us in a much better place than most other languages faced with this choice. -- Live well, ~wren

1 0

ANNOUNCE: monoid-subclasses-0.1.1, incremental-parser-0.2.2, and SCC-0.8
by Mario Blažević 22 Mar '13

22 Mar '13

The new package monoid-subclasses [1] exports a number of classes that sit between monoids and groups: ReductiveMonoid, CancellativeMonoid, GCDMonoid, MonoidNull, and FactorialMonoid among others. The package also comes with class instances for all applicable data types from base, vector, containers, bytestring, and text packages. These classes were initially a part of the incremental-parser package [2] but they've moved, so incremental-parser-0.2.2 now depends on monoid-subclasses which has added more class methods and instances. The parsers constructed using the incremental-parser library generally work on any ReductiveMonoid, FactorialMonoid or TextualMonoid. You can thus construct a generic parser that operates on String, ByteString, or Text input, or even on Product Integer or a Map if you are so inclined. The version 0.8 of Streaming Component Combinators (a.k.a. SCC) has also been re-designed from the ground up. In the earlier versions, as in the pipes and conduit packages, the types of streams and stream-processing components were parametrized by the type of the individual items flowing through. All sources and sinks are now parametrized instead by the type of the monoid they produce or consume. As a consequence, SCC coroutines can now communicate in chunks of ByteString or Text instead of just lists, and these chunks can be of arbitrary size. The boundaries of producer's chunks are invisible to the consumer (excepting some low-level functions), which must specify the amount of data it wants to receive using a ReadRequest. This change at the heart of SCC has had a positive impact on outer layers, and especially on the Splitter component type which has become much simpler. [1] http://hackage.haskell.org/package/monoid-subclasses-0.1.1 [2] http://hackage.haskell.org/package/incremental-parser-0.2.2 [3] http://hackage.haskell.org/package/scc-0.8

1 0

Re: Proposal: Add the unordered-containers package and the hashable package to the Haskell Platform
by Roman Cheplyaka 21 Mar '13

21 Mar '13

* Thomas Moertel <tom(a)moertel.com> [2013-03-21 15:58:58-0400] > On Wed, Mar 20, 2013 at 5:05 PM, Johan Tibell <johan.tibell(a)gmail.com>wrote: > > > The problem is that if the secure hashing makes HashMap as slow as Map, > > there's no point in having the former in the first place, as it only exists > > to provide speed. > > > > I understand that HashMap exists to provide speed, but are we sure that > everyone who uses it for speed understands the security implications? > > Regardless of whether speed-first or security-first is the default, users > of HashMap can be partitioned into two groups: those who knowingly choose > between speed and security, and those who do not. For the first group, > either default is safe because members of that group will have the > opportunity to choose the other option. But for the second group, only > security-first is safe: if it’s not the default, members of the second > group won’t have the option to choose it. Why not simply educate them by placing a prominent notice in the haddock? > The problem for the Haskell Platform is that the second group is likely to > be large. The whole point, after all, is to make a curated set of tools and > libraries available to a large audience that doesn’t necessarily have the > time or knowledge to make its own informed choices. So if we decide, on > behalf of this audience, that the Haskell Platform is going to be less > secure to get more speed, aren’t we exposing a lot of Haskell users to > preventable security risks? > > I’m not just arguing the case of HashMap but of the whole Haskell Platform. > I’d like to think that a widely used platform should be safe by default and > offer danger-with-benefits as an option. The whole security issue is that, in some cases, the complexity of the code may be unexpectedly bad. Unfortunately, it is already the case with Haskell. Laziness makes it extremely hard to make guarantees about the worst-case complexity of a non-trivial code base. Roman

1 0

Re: Proposal: Add the unordered-containers package and the hashable package to the Haskell Platform
by Johan Tibell 20 Mar '13

20 Mar '13

On Wed, Mar 20, 2013 at 1:04 PM, Thomas Moertel <tom(a)moertel.com> wrote: > On the speed vs. security trade-off, there’s a good argument for the > Haskell Platform to choose security as its default policy: speed problems > are obvious and have understandable consequences and remedies, but security > problems are hidden and have poorly understood consequences and often lack > remedies altogether. You can’t go back in time to remedy being pwned. > > Therefore, if we make speed the default, we leave open to harm everybody > who fails to fully understand the security implications of their libraries’ > default choices, which is just about everybody. But if we make security the > default, nobody will be taken unaware, either by speed or preventable > security problems. > > Yes, those people who need the extra speed will have to suffer the cost of > opting-in to the faster implementations, but that cost is predictable and > relatively small. The point is that nobody is going to be taken unaware by > failing to make the right choice up front. The first time you run your code > and it’s too slow, you’ll know. > The problem is that if the secure hashing makes HashMap as slow as Map, there's no point in having the former in the first place, as it only exists to provide speed.

1 0

Re: Proposal: Add the unordered-containers package and the hashable package to the Haskell Platform
by Johan Tibell 20 Mar '13

20 Mar '13

On Wed, Mar 20, 2013 at 11:40 AM, Bardur Arantsson <spam(a)scientician.net>wrote: > On 03/20/2013 07:14 PM, Johan Tibell wrote:> Just as > > sandboxing is a better way to run untrusted software than trying to plug > > every whole in your code/RTS, > > (Citation needed). There have been lots of exploits of VMs and > sandboxes. This is also irrelevant to DoS prevention, AFAICT. > > > generic anti-DoS techniques are usually > > better than trying to figure out every possible way someone can DoS you. > > (Citation needed.) > > There's no excuse for not preventing obvious an relatively easily > preventable problems. Security in Depth and all that. My only citation is that this is how we do things, but I think you can make a theoretical argument for it as well. This is what I tried to do with the sandboxing analogy. The reason that sandboxing works well is that it's a whitelist approach (instead of a blacklist approach). I should note that when I talk about sandboxing I don't really mean a VM, but rather a chroot/process isolation approach, like AppEngine. If you only allow the e.g. AppEngine user code to modify the system via a few select RPCs and then block all file system access by default, you can avoid having to audit every single API that the platform offers. Now, for DoS protection we can do something similar. We have a detector for bad behavior (e.g. long running connections that time out, etc) and a generic mechanism for rejecting such users (IP blocking). That way we don't have to inspect everything the app itself does that could open it up for a DoS attack. You're of course right that current Haskell frameworks don't do this. There hasn't really been a need yet. -- Johan

1 0

Re: Proposal: Add the unordered-containers package and the hashable package to the Haskell Platform
by Bardur Arantsson 20 Mar '13

20 Mar '13

On 03/20/2013 07:40 PM, Bardur Arantsson wrote: > On 03/20/2013 07:14 PM, Johan Tibell wrote: > >Hash flooding is only one of many ways to DoS a server. You can try other > >things, like building a bot net and trying to flood the connection. > > Sure, but that's another step up in difficulty from using a single > client to DoS a server trivially. > I should also add: Whatever DDoS protections your ISP may (or may not) have in place does nothing to protect you if your host has a DoS vulnerability which is exploitable with a single connection.

1 0

Re: Stricter WriterT (Part II)
by Gabriel Gonzalez 20 Mar '13

20 Mar '13

> Presumably we'll also need > > writerT :: m (a, w) -> WriterT w m a Yes, we would need that, too: writerT m = WriterT $ \w -> do (a, w') <- m let wt = mappend w w' wt `seq` return (a, wt) > Is there any reason to keep Control.Monad.Trans.Writer.Strict, or should this replace it? If we replace the old one, it will break existing code that used the `WriterT` constructor. Same thing for the strict RWST constructor if we similarly modify that. However, I don't know exactly how many packages use those constructors. I will try to do a text search of Hackage this coming weekend to check and see if it is feasible to ask downstream packages that use `WriterT`/`RWST` constructors to set upper bounds on `transformers`.

2 1

Re: Data instance for Data.Version
by Administrator 18 Mar '13

18 Mar '13

> Hi, > > Can we derive Data for Data.Version.Version? It would benefit Cabal. > > Thanks, > David On a related note, what about also moving some (maybe all) of the nice Distribution.Version related functions from the Cabal package to Data.Version? I am sure other libraries would also be interested in using this functionality without having to include Cabal as a whole. One example is the Ghc runtime itself: this could be one small step in removing dependencies from Ghc to Cabal.

2 1

instance Applicative Data.Map
by Henning Thielemann 17 Mar '13

17 Mar '13

An ZipList-like Applicative instance for Data.Map would be nice. I have an application where I like to write liftA3 f amap bmap cmap meaning Map.intersectionWith ($) (Map.intersectionWith f amap bmap) cmap But I cannot complete the instance implementation because there is no sensible definition for 'pure' for Data.Map. :-(

14 31