April 2015 - Haskell-Cafe

Advice needed on how to improve some code
by Jeff 16 Apr '15

16 Apr '15

Hello, I am seeking some advice on how I might improve a bit of code. The function in question reads and parses part of a binary protocol, storing the parsed info as it proceeds. parseDeviceData is called by parseDevice (shown further down). It looks to me like there should be a more concise, less repetitive way to do what parseDeviceData does. Any advice on this would be greatly appreciated. parseDeviceData :: P.Payload -> Parser P.Payload parseDeviceData pl = let mdm = P.dataMask ( P.payloadData pl ) in ( let pld = P.payloadData pl in if testBit mdm ( fromEnum D.Sys ) then parseDeviceSysData >>= ( \s -> return ( pl { P.payloadData = pld { P.sysData = Just s } } ) ) else return pl ) >>= ( \pl' -> let pld = P.payloadData pl' in if testBit mdm ( fromEnum D.GPS ) then parseDeviceGPSData >>= ( \s -> return ( pl' { P.payloadData = pld { P.gpsData = Just s } } ) ) else return pl' ) >>= ( \pl' -> let pld = P.payloadData pl' in if testBit mdm ( fromEnum D.GSM ) then parseDeviceGSMData >>= ( \s -> return ( pl' { P.payloadData = pld { P.gsmData = Just s } } ) ) else return pl' ) >>= ( \pl' -> let pld = P.payloadData pl' in if testBit mdm ( fromEnum D.COT ) then parseDeviceCOTData >>= ( \s -> return ( pl' { P.payloadData = pld { P.cotData = Just s } } ) ) else return pl' ) >>= ( \pl' -> let pld = P.payloadData pl' in if testBit mdm ( fromEnum D.ADC ) then parseDeviceADCData >>= ( \s -> return ( pl' { P.payloadData = pld { P.adcData = Just s } } ) ) else return pl' ) >>= ( \pl' -> let pld = P.payloadData pl' in if testBit mdm ( fromEnum D.DTT ) then parseDeviceDTTData >>= ( \s -> return ( pl' { P.payloadData = pld { P.dttData = Just s } } ) ) else return pl' ) >>= ( \pl' -> let pld = P.payloadData pl' in if testBit mdm ( fromEnum D.OneWire ) then parseDeviceOneWireData >>= ( \s -> return ( pl' { P.payloadData = pld { P.iwdData = Just s } } ) ) else return pl' ) >>= ( \pl' -> if testBit mdm ( fromEnum D.ETD ) then parseDeviceEventData pl' else return pl' ) The Parser above is a Data.Binary.Strict.Get wrapped in a StateT, where the state is a top-level structure for holding the parsed packet. parseDevice :: Bool -> Parser () parseDevice _hasEvent = parseTimestamp >>= ( \ts -> if _hasEvent then lift getWord8 >>= ( \e -> lift getWord16be >>= ( \mdm -> return ( P.Payload "" ( Just ts ) $ P.blankDevicePayloadData { P.dataMask = mdm , P.eventID = toEnum ( fromIntegral e .&. 0x7f ) , P.deviceStatusFlag = testBit e 7 , P.hasEvent = True } ) ) ) else lift getWord16be >>= ( \mdm -> return ( P.Payload "" ( Just ts ) $ P.blankDevicePayloadData { P.dataMask = mdm } ) ) ) >>= parseDeviceData >>= ( \dpl -> get >>= ( \p -> put ( p { P.payloads = dpl : P.payloads p } ) ) ) Here are the data types for the Packet and Payload: data Payload = Payload { imei :: !BS.ByteString , timestamp :: Maybe Word64 , payloadData :: PayloadData } data PayloadData = HeartBeatPL | SMSFwdPL { smsMesg :: !BS.ByteString } | SerialPL { auxData :: !Word8 , fixFlag :: !Word8 , gpsCoord :: !GPSCoord , serialData :: !BS.ByteString } | DevicePL { hasEvent :: !Bool , deviceStatusFlag :: !Bool , eventID :: !E.EventID , dataMask :: !Word16 , sysData :: Maybe DS.SysData , gpsData :: Maybe DGP.GPSData , gsmData :: Maybe DGS.GSMData , cotData :: Maybe DC.COTData , adcData :: Maybe DA.ADCData , dttData :: Maybe DD.DTTData , iwdData :: Maybe DO.OneWireData , etdSpd :: Maybe ES.SpeedEvent , etdGeo :: Maybe EG.GeoEvent , etdHealth :: Maybe EH.HealthEvent , etdHarsh :: Maybe EHD.HarshEvent , etdOneWire :: Maybe EO.OneWireEvent , etdADC :: Maybe EA.ADCEvent } deriving ( Show ) data Packet = Packet { protocolVersion :: !Word8 , packetType :: !PT.PacketType , deviceID :: Maybe BS.ByteString , payloads :: ![ Payload ] , crc :: !Word16 } deriving ( Show ) Lastly, here is the Parser monad transformer: module G6S.Parser where import Control.Monad.State.Strict import Data.Binary.Strict.Get import qualified Data.ByteString as BS import qualified G6S.Packet as GP type Parser = StateT GP.Packet Get runParser :: Parser a -> BS.ByteString -> Maybe a runParser p bs = let ( result, _ ) = runGet ( runStateT p GP.initPacket ) bs in case result of Right tup -> Just $ fst tup Left _ -> Nothing I hope there is enough info here. Thanks, Jeff

6 10

Re: [Haskell-cafe] Ongoing IHG work to improve Hackage security
by Mikhail Glushenkov 16 Apr '15

16 Apr '15

Hi, On 16 April 2015 at 16:08, Herbert Valerio Riedel <hvriedel(a)gmail.com> wrote: > Being historically a tape-originating archive format, `tar` does support > such destructive updates as a standard-feature by appended content. Thanks, I understand this. My question was about whether cabal-install supports this feature of tar (turns out, it does). > Using this, we can actually have *all* historic .cabal revisions in the > 00-index.tar file as hidden entries. I don't think we do this right now, though.

1 0

Re: [Haskell-cafe] Ongoing IHG work to improve Hackage security
by Duncan Coutts 16 Apr '15

16 Apr '15

On Thu, 2015-04-16 at 16:08 +0200, Herbert Valerio Riedel wrote: > On 2015-04-16 at 15:14:47 +0200, Mikhail Glushenkov wrote: > > [...] > > > Thank you, this is very exciting. But won't the post-release .cabal > > update feature interfere with "package index as an append-only log" > > concept? IIUC, right now it is implemented as a destructive update of > > the corresponding package index entry, so making the package index > > immutable will break backwards compatibility. > > Being historically a tape-originating archive format, `tar` does support > such destructive updates as a standard-feature by appended content. See > also the '--update' operation in the tar(1) man-page: > > | -u, --update > | Append files which are newer than the corresponding copy in > | the archive. Arguments have the same meaning as with -c and > | -r options. > > Using this, we can actually have *all* historic .cabal revisions in the > 00-index.tar file as hidden entries. This would in theory give > `cabal-install` access to all cabal revisions (if we'd ever want to have > a feature in cabal that requires access to previous cabal revisions) Yes, we can choose to do that. That's sort-of what I was getting at when I mentioned that you could then view the tar file as a transaction log. > I actually find Duncan's approach rather elegant, as it simply builds on > existing standard features of both .tar files and the HTTP protocol > (resuming HTTP downloads). And they don't even require any custom HTTP > server implementation to work, and there's a graceful fallback. And we > don't need to introduce a completely new protocol/tool into the mix > (such as Git, which has dump-http(s), smart-http(s), git, as > transport-layers to chose from, and has various other moving parts such > as a ~/.gitconfig that can affect how Git operates, like redirecting > clone-urls transparently, also it avoids having to handle subtle > platform-specific issues with shelling out to an external command; from > this POV the HTTP/tar approach is the more KISS-compliant one, IMHO) > > Cheers, > hvr >

1 0

Re: [Haskell-cafe] Improvements to package hosting and security
by Michael Snoyman 16 Apr '15

16 Apr '15

That could work in theory. My concern with such an approach is that- AFAIK- the tooling around that kind of stuff is not very well developed, as opposed to an approach using Git, SHA512, and GPG, which should be easy to combine. But I could be completely mistaken on this point; if existing, well vetted technology exists for this, I'm not opposed to using it. On Mon, Apr 13, 2015 at 6:04 PM Arnaud Bailly | Capital Match < arnaud(a)capital-match.com> wrote: > Just thinking aloud but wouldn't it be possible to take advantage of > cryptographic ledgers a la Bitcoin for authenticating packages and tracking > the history of change ? This would provide redundancy as the transactions > log is distributed and "naturally" create a web of trust or at least > authenticate transactions. People uploading or modifying a package would > have to sign a transactions with someone having enough karma to allow this. > > Then packages themselves could be completely and rather safely distributed > through standard p2p file sharing. > > I am not a specialist of crypto money, though. > > My 50 cts > Arnaud > > Le lundi 13 avril 2015, Dennis J. McWherter, Jr. <dennis(a)deathbytape.com> > a écrit : > >> This proposal looks great. The one thing I am failing to understand (and >> I recognize the proposal is in early stages) is how to ensure redundancy in >> the system. As far as I can tell, much of this proposal discusses the >> centralized authority of the system (i.e. ensuring secure distribution) and >> only references (with little detail) the distributed store. For instance, >> say I host a package on a personal server and one day I decide to shut that >> server down; is this package now lost forever? I do see this line: "backup >> download links to S3" but this implies that the someone is willing to pay >> for S3 storage for all of the packages. >> >> Are there plans to adopt a P2P-like model or something similar to support >> any sort of replication? Public resources like this seem to come and go, so >> it would be nice to avoid some of the problems associated with high churn >> in the network. That said, there is an obvious cost to replication. >> Likewise, the central authority would have to be updated with new, relevant >> locations to find the file (as it is currently proposed). >> >> In any case, as I said before, the proposal looks great! I am looking >> forward to this. >> >> On Monday, April 13, 2015 at 5:02:46 AM UTC-5, Michael Snoyman wrote: >>> >>> Many of you saw the blog post Mathieu wrote[1] about having more >>> composable community infrastructure, which in particular focused on >>> improvements to Hackage. I've been discussing some of these ideas with both >>> Mathieu and others in the community working on some similar thoughts. I've >>> also separately spent some time speaking with Chris about package >>> signing[2]. Through those discussions, it's become apparent to me that >>> there are in fact two core pieces of functionality we're relying on Hackage >>> for today: >>> >>> * A centralized location for accessing package metadata (i.e., the cabal >>> files) and the package contents themselves (i.e., the sdist tarballs) >>> * A central authority for deciding who is allowed to make releases of >>> packages, and make revisions to cabal files >>> >>> In my opinion, fixing the first problem is in fact very straightforward >>> to do today using existing tools. FP Complete already hosts a full Hackage >>> mirror[3] backed by S3, for instance, and having the metadata mirrored to a >>> Git repository as well is not a difficult technical challenge. This is the >>> core of what Mathieu was proposing as far as composable infrastructure, >>> corresponding to next actions 1 and 3 at the end of his blog post (step 2, >>> modifying Hackage, is not a prerequesite). In my opinion, such a system >>> would far surpass in usability, reliability, and extensibility our current >>> infrastructure, and could be rolled out in a few days at most. >>> >>> However, that second point- the central authority- is the more >>> interesting one. As it stands, our entire package ecosystem is placing a >>> huge level of trust in Hackage, without any serious way to vet what's going >>> on there. Attack vectors abound, e.g.: >>> >>> * Man in the middle attacks: as we are all painfully aware, >>> cabal-install does not support HTTPS, so a MITM attack on downloads from >>> Hackage is trivial >>> * A breach of the Hackage Server codebase would allow anyone to upload >>> nefarious code[4] >>> * Any kind of system level vulnerability could allow an attacker to >>> compromise the server in the same way >>> >>> Chris's package signing work addresses most of these vulnerabilities, by >>> adding a layer of cryptographic signatures on top of Hackage as the central >>> authority. I'd like to propose taking this a step further: removing Hackage >>> as the central authority, and instead relying entirely on cryptographic >>> signatures to release new packages. >>> >>> I wrote up a strawman proposal last week[5] which clearly needs work to >>> be a realistic option. My question is: are people interested in moving >>> forward on this? If there's no interest, and everyone is satisfied with >>> continuing with the current Hackage-central-authority, then we can proceed >>> with having reliable and secure services built around Hackage. But if >>> others- like me- would like to see a more secure system built from the >>> ground up, please say so and let's continue that conversation. >>> >>> [1] https://www.fpcomplete.com/blog/2015/03/composable- >>> community-infrastructure >>> [2] https://github.com/commercialhaskell/commercialhaskell/wiki/ >>> Package-signing-detailed-propsal >>> [3] https://www.fpcomplete.com/blog/2015/03/hackage-mirror >>> [4] I don't think this is just a theoretical possibility for some point >>> in the future. I have reported an easily trigerrable DoS attack on the >>> current Hackage Server codebase, which has been unresolved for 1.5 months >>> now >>> [5] https://gist.github.com/snoyberg/732aa47a5dd3864051b9 >>> >> -- >> > You received this message because you are subscribed to the Google Groups >> "Commercial Haskell" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to commercialhaskell+unsubscribe(a)googlegroups.com. >> To post to this group, send email to commercialhaskell(a)googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/commercialhaskell/4487776e-b862-429c-adae… >> <https://groups.google.com/d/msgid/commercialhaskell/4487776e-b862-429c-adae…> >> . > > >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > *Arnaud Bailly* > > CTO | Capital Match > > CapitalMatch > > 71 Ayer Rajah Crescent | #06-16 | Singapore 139951 > > (FR) +33 617 121 978 / (SG) +65 8408 7973 | arnaud(a)capital-match.com | > www.capital-match.com > > Disclaimer: > > *Capital Match Platform Pte. Ltd. (the "Company") registered in Singapore > (Co. Reg. No. 201501788H), a subsidiary of Capital Match Holdings Pte. Ltd. > (Co. Reg. No. 201418682W), provides services that involve arranging for > multiple parties to enter into loan and invoice discounting agreements. The > Company does not provide any form of investment advice or recommendations > regarding any listings on its platform. In providing its services, the > Company's role is limited to an administrative function and the Company > does not and will not assume any advisory, fiduciary or other duties to > clients of its services.* > >

8 24

FunDeps and type inference
by Olaf Klinke 15 Apr '15

15 Apr '15

Dear cafe, I want to write an evaluation function that uncurries its function argument as necessary. Examples should include: eval :: (a -> b) -> a -> b eval :: (a -> b -> c) -> a -> (b -> c) eval :: (a -> b -> c) -> (a,b) -> c and hence have both eval (+) 4 5 eval (+) (4,5) typecheck. My approach is to use a type class: class Uncurry f a b where eval :: f -> a -> b instance Uncurry (a -> b) a b where eval = ($) instance (Uncurry f b c) => Uncurry ((->) a) f) (a,b) c where eval f (a,b) = eval (f a) b This works, but not for polymorphic arguments. One must annotate function and argument with concrete types when calling, otherwise the runtime does not know what instance to use. Type inference on ($) is able to infer the type of either of f, a or b in the expression b = f $ a if the type of two is known. Thus I am tempted to add functional dependencies class Uncurry f a b | f a -> b, a b -> f, f b -> a but I get scary errors: With only the first of the three dependencies, the coverage condition fails. Adding UndecidableInstances, the code compiles. Now type inference on the return type b works, but one can not use e.g. (+) as function argument. Adding the second dependency results in the compiler rejecting the code claiming "Functional dependencies conflict between instance declarations". I can not quite see where they would, and the compiler does not tell me its counterexample. I can see that eval max (True,False) :: Bool -- by second instance declaration, -- when max :: Bool -> Bool -> Bool eval max (True,False) :: (Bool,Bool) -> (Bool,Bool) -- by first instance declaration -- when max :: (Bool,Bool) -> (Bool,Bool) -> (Bool,Bool) but this ambiguity is precisely what the dependency a b -> f should help to avoid, isn't it? Judging by the number of coverage condition posts on this list this one is easy to get wrong and the compiler messages are not always helpful. Is this a kind problem? Would anyone care to elaborate? Thanks, Olaf

2 1

Why no Floating instance for Data.Fixed / Data.Fixed.Binary
by Douglas McClean 14 Apr '15

14 Apr '15

I'm wondering why the decision was made not to have a Floating instance for Data.Fixed. I understand that -- being a fixed point number type -- it doesn't line up with the name of the Floating typeclass. But it also seems that Floating is misnamed, because none of the functions there really have anything to do with floating point representation. This is why we still have Floating instances for, say, exact reals or symbolic numbers, neither of which has a floating point representation. It seems that the actual criterion for membership in Floating is something like "has a canonical choice for degree of approximation". Double and Float are in because we round to the nearest value. Exact reals are in because we don't have to approximate, symbolic numbers too. Rational is out because there's no clear choice of how to approximate. But by this criterion, Data.Fixed should be in. And indeed it would be useful for it to be in. If I choose to represent values from the real world by a fixed point approximation instead of a floating point approximation, shouldn't I still be able to take sines and cosines etc? Obviously I can fix this by using a newtype. I'm just curious if am I missing a reason why the current definition is more desirable? -Doug McClean

3 4

Do all "fromList" functions redirect to List operations?
by martin 14 Apr '15

14 Apr '15

Hello all, I have a datatype like this: data Time = DPast | T Integer deriving (Eq, Show) data Temporal a = Temporal { at :: Time -> (a, Maybe Time) } and I wrote a fromList function to create one. The fromList function implements it own "at" function and wraps it in a Temporal. IIUC, this means that whenever I invoke "at" on a Temporal created from a List I actually operate on that List. Alternatively I can directly create a Temporal as in exNat = Temporal f where f DPast = (0, Just $ T 1) f (T t) = (t, Just $ T $ t+1) whose "at" function is a lot faster. Do all "fromList" functions behave this way, i.e. they redirect operations on the new type to List operations? Is there a way to make a Temporal, created via fromList "forget" its List heritage?

1 0

Eta-reducing case branches
by Tom Ellis 14 Apr '15

14 Apr '15

Has anyone ever considered permitting case branches to be eta reduced? For example, it is often nice to rewrite foo x = bar baz x as foo = bar baz Likewise, I have often wanted to rewrite case m of Nothing -> n Just x -> quux x as case m of Nothing -> n Just -> quux Am I missing an obvious reason this wouldn't work? Tom

4 4

Re: [Haskell-cafe] Help wanted: working on the GHC webpage
by Sergey Bushnyak 14 Apr '15

14 Apr '15

> Why would it be easier? What's difficult about publishing on > https://ghc.haskell.org/trac/ghc/blog? I'm actually don't know how it's published on track. From my standpoint as newcomer it's better to see what's happening from one place, with one design, have some shared git repo where people contribute in markdown. > Moreover, the GHC weekly news are intimately linked to Trac, as they > reference Trac-tickets and Git commits, which Trac is able to annotate > with meta-data (ticket-type, -status, and -title for Ticket references, > as well as part of the Git commit msg for Git-commit refs). Ok, it was just a suggestion. Maybe it's a bad idea, doesn't know about annotation. Anyway, still can help on updating ghc home page.

2 1

Semantics of temporal data
by martin 14 Apr '15

14 Apr '15

Hello all, I wrote a Temporal datatype data Temporal a = Temporal { td :: a, -- "temporal default" tc :: [Change a] -- "temporal changes" } deriving (Eq, Show) data Change a = Chg { ct :: Time, -- "change time" cv :: a -- "change value" } deriving (Eq,Show) I got into semantic trouble when trying to write "join" for Temporal (Temporal a). If I have a Temporal Temporal, then each change's value is a Temporal. The question is: what significance does the temporal default have? In my implementation, when I look at this Chg t0 (Temporal default0 changes0) Chg t1 (Temporal default1 changes1) --<-- you are here Chg t2 (Temporal default2 changes2) I accept the changes1 which are >= t1 and less than t2. So far so good. But I also take the default1 and prepend it to the resuling list of changes with a change time of t1 (unless the first of the changes1 occurs also at t1). But by comparing results with a handwritten <*>, QuickCheck revealed that this is not in line with <*> = ap x >>= f = join $ fmap f x It appears to me that the defaults of the inner Temporal lose most of their meaning. The effective default is the change value of the last accepted change from the previous (t0) iteration. This would make some sense. If I have a summer schedule which lists the location of a single train over time, and it sais it is usually "nowhere", but at July 1st is shall be in Manchester. If I ask this schedule, where the train will be before July 1st it'll say "nowhere". Now if I prepend the summer schedule with a spring schedule, then there will be a last location of that train in the spring schedule (say Brighton), i.e. before the summer schedule becomes effective. Hence at the beginning of the summer, the train won't be "nowhere" but in Brighton. Does this make some sense? Please feel free to comment. BTW: I am really impressed that haskell pointed out this semantic issue to me. This could have gone unnoticed for months in other languages.

3 4