[GHC] #10364: Feature request: Add support for FMA
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature | Status: new request | Milestone: Priority: normal | Version: 7.11 Component: Core | Operating System: Unknown/Multiple Libraries | Type of failure: None/Unknown Keywords: | Blocked By: Architecture: | Related Tickets: Unknown/Multiple | Test Case: | Blocking: | Differential Revisions: | -------------------------------------+------------------------------------- FMA (fused-multiply-add) has been around for quite some-time, and is natively supported by many of the newer processors. It's an operation that is heavily used in HPC applications. I think it's time for Haskell to have native support for it as well. Ideally, it should be added to the `Num` class as a method. Assuming that is a possibility, we would have the following signature and default definition: {{{#!hs class Num a where ... fma :: a -> a -> a -> a fma x y z = x * y + z }}} Except of course the `Float` and `Double` instances would ensure that the rounding is done only once. If adding it to the `Num` class is not an option, then `RealFloat` class would be the next best place; and perhaps that's arguably also a better place because those types are the ones one usually have in mind when using FMA. I think either `Num` or `RealFloat` would be fine choices. Implementation: * If the underlying architecture supports it (which is very common), directly emit FMA instruction * Otherwise, FFI out to C and use `fma` and `fmaf` from the math library A direct software implementation might also be possible for platforms where neither choice above is an option, but that bridge can be crossed when we get there. As a final note; while supporting these functions might seem going-out-of- our-way; it is indeed a big selling point in HPC applications. Furthermore, hardware manufacturers are putting big resources to make these supported naively in the instruction sets. Supporting FMA right out of the box would be a very good step in wider adaptation of Haskell in the HPC community. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by lerkok): * cc: erkokl@… (added) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:1 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Description changed by lerkok: Old description:
FMA (fused-multiply-add) has been around for quite some-time, and is natively supported by many of the newer processors. It's an operation that is heavily used in HPC applications. I think it's time for Haskell to have native support for it as well.
Ideally, it should be added to the `Num` class as a method. Assuming that is a possibility, we would have the following signature and default definition:
{{{#!hs class Num a where ... fma :: a -> a -> a -> a fma x y z = x * y + z }}}
Except of course the `Float` and `Double` instances would ensure that the rounding is done only once.
If adding it to the `Num` class is not an option, then `RealFloat` class would be the next best place; and perhaps that's arguably also a better place because those types are the ones one usually have in mind when using FMA.
I think either `Num` or `RealFloat` would be fine choices.
Implementation:
* If the underlying architecture supports it (which is very common), directly emit FMA instruction * Otherwise, FFI out to C and use `fma` and `fmaf` from the math library
A direct software implementation might also be possible for platforms where neither choice above is an option, but that bridge can be crossed when we get there.
As a final note; while supporting these functions might seem going-out- of-our-way; it is indeed a big selling point in HPC applications. Furthermore, hardware manufacturers are putting big resources to make these supported naively in the instruction sets. Supporting FMA right out of the box would be a very good step in wider adaptation of Haskell in the HPC community.
New description: FMA (fused-multiply-add) has been around for quite some-time, and is natively supported by many of the newer processors. It's an operation that is heavily used in HPC applications. I think it's time for Haskell to have native support for it as well. Ideally, it should be added to the `Num` class as a method. Assuming that is a possibility, we would have the following signature and default definition: {{{#!hs class Num a where ... fma :: a -> a -> a -> a fma x y z = x * y + z }}} Except of course the `Float` and `Double` instances would ensure that the rounding is done only once. If adding it to the `Num` class is not an option, then `RealFloat` class would be the next best place; and perhaps that's arguably also a better place because those types are the ones one usually has in mind when using FMA. I think either `Num` or `RealFloat` would be fine choices. Implementation: * If the underlying architecture supports it (which is very common), directly emit FMA instruction * Otherwise, FFI out to C and use `fma` and `fmaf` from the math library A direct software implementation might also be possible for platforms where neither choice above is an option, but that bridge can be crossed when we get there. As a final note; while supporting these functions might seem going-out-of- our-way; it is indeed a big selling point in HPC applications. Furthermore, hardware manufacturers are putting big resources to make these supported naively in the instruction sets. Supporting FMA right out of the box would be a very good step in wider adaptation of Haskell in the HPC community. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:2 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Description changed by lerkok: Old description:
FMA (fused-multiply-add) has been around for quite some-time, and is natively supported by many of the newer processors. It's an operation that is heavily used in HPC applications. I think it's time for Haskell to have native support for it as well.
Ideally, it should be added to the `Num` class as a method. Assuming that is a possibility, we would have the following signature and default definition:
{{{#!hs class Num a where ... fma :: a -> a -> a -> a fma x y z = x * y + z }}}
Except of course the `Float` and `Double` instances would ensure that the rounding is done only once.
If adding it to the `Num` class is not an option, then `RealFloat` class would be the next best place; and perhaps that's arguably also a better place because those types are the ones one usually has in mind when using FMA.
I think either `Num` or `RealFloat` would be fine choices.
Implementation:
* If the underlying architecture supports it (which is very common), directly emit FMA instruction * Otherwise, FFI out to C and use `fma` and `fmaf` from the math library
A direct software implementation might also be possible for platforms where neither choice above is an option, but that bridge can be crossed when we get there.
As a final note; while supporting these functions might seem going-out- of-our-way; it is indeed a big selling point in HPC applications. Furthermore, hardware manufacturers are putting big resources to make these supported naively in the instruction sets. Supporting FMA right out of the box would be a very good step in wider adaptation of Haskell in the HPC community.
New description: FMA (fused-multiply-add) has been around for quite some-time, and is natively supported by many of the newer processors. It's an operation that is heavily used in HPC applications. I think it's time for Haskell to have native support for it as well. Ideally, it should be added to the `Num` class as a method. Assuming that is a possibility, we would have the following signature and default definition: {{{#!hs class Num a where ... fma :: a -> a -> a -> a fma x y z = x * y + z }}} Except of course the `Float` and `Double` instances would ensure that the rounding is done only once. If adding it to the `Num` class is not an option, then `RealFloat` class would be the next best place; and perhaps that's arguably also a better place because those types are the ones one usually has in mind when using FMA. I think either `Num` or `RealFloat` would be fine choices. Implementation: * If the underlying architecture supports it (which is very common), directly emit FMA instruction * Otherwise, FFI out to C and use `fma` and `fmaf` from the math library A direct software implementation might also be possible for platforms where neither choice above is an option, but that bridge can be crossed when we get there. As a final note; while supporting these functions might seem going-out-of- our-way; it is indeed a big selling point in HPC applications. Furthermore, hardware manufacturers are putting big resources to make these supported natively in the instruction sets. Supporting FMA right out of the box would be a very good step in wider adaptation of Haskell in the HPC community. -- -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:3 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by ekmett): This probably should go out through a standard libraries@ proposal process first, to get a sense of how willing the community would be to accept a change that far up the chain. You may get some horribly long `fusedMultiplyAdd` name though. =) I could definitely see putting it in RealFloat. We had a proposal last year for adding things like `expm1` and `log1p`, etc. which passed, which will involve adding members nearby. Warning though: when we lobbied for `expm1` the comunity pushed back against having sensible defaults like the one you mention here and preferred leaving it undefined, to prompt people to fill them in proactively. I'm not sure that is/was a good idea, but some folks fought pretty hard against defaults that might lose precision favoring bottoms instead. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:4 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by lerkok): Ah, excellent. Perhaps fma can be tacked on to that proposal. I was reading through https://wiki.haskell.org/Library_submissions and it actually suggests proposals should now be filed as tickets; unless I'm misreading it. There used to be an e-mail to the libraries@ mailing list I think; is that still the way to go? -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:5 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: Operating System: Unknown/Multiple | Architecture: Type of failure: None/Unknown | Unknown/Multiple Blocked By: | Test Case: Related Tickets: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by ekmett): The libraries@ list is still here. We typically escalate anything to it that seems remotely controversial to get a sense of community feedback, buy-in, counter-proposals, etc. That is especially true for things that affect the types and classes in the Haskell Report, like this. https://wiki.haskell.org/Library_submissions mentions it in section 7. If the CLC decides that the discussion must be discussed with the libraries@ mailing list, the original proposer may be asked to moderate the libraries@ mailing list discussion. The general format for this is: * Send your proposal by email to the libraries@haskell.org mailing list (which you need to subscribe to before posting). * Set a deadline for discussion (no less than two weeks), and act as chair/moderator for the discussion. * Following discussion, it's the responsibility of the core libraries committee to determine if the proposal should be accepted. Send an email to the core libraries committee requesting a decision, optionally including your own summary of the mailing list discussion. * From that point forward, the CLC and maintainer will again be responsible for making a decision on how to proceed, and will inform the proposer and the mailing list. Possible decisions include acceptance, rejection, and requesting a patch be provided. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:6 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: report- Operating System: Unknown/Multiple | impact Type of failure: None/Unknown | Architecture: Blocked By: | Unknown/Multiple Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by ekmett): * keywords: => report-impact -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:7 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: report- Operating System: Unknown/Multiple | impact Type of failure: None/Unknown | Architecture: Blocked By: | Unknown/Multiple Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by ekmett): That said, creating a ticket like this is likely very much the right place to start. It ensures it can't fall off our radar forever. =) -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:8 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: report- Operating System: Unknown/Multiple | impact Type of failure: None/Unknown | Architecture: Blocked By: | Unknown/Multiple Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by lerkok): Just sent a proposal to the mailing list: https://mail.haskell.org/pipermail/libraries/2015-April/025574.html Hopefully fma can make it in together with expm1 and friends.. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:9 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: report- Operating System: Unknown/Multiple | impact Type of failure: None/Unknown | Architecture: Blocked By: | Unknown/Multiple Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by Lemming): `RealFloat` would exclude an `instance Complex`. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:10 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: report- Operating System: Unknown/Multiple | impact Type of failure: None/Unknown | Architecture: Blocked By: | Unknown/Multiple Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by ekmett): Good point. That probably pushes it up to Floating or Num then. In Num you could use it for accelerated modular arithmetic as well, which has nothing to do with floating point. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:11 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: report- Operating System: Unknown/Multiple | impact Type of failure: None/Unknown | Architecture: Blocked By: | Unknown/Multiple Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by carter): sounds like it makes sense to 1) put fma in Num and 2) add ghc primitive support to applicable supported microarchitectures -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:12 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: new Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: | Keywords: report- Operating System: Unknown/Multiple | impact Type of failure: None/Unknown | Architecture: Blocked By: | Unknown/Multiple Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by lerkok): There was quite a bit of discussion and feedback on the mailing list, that I very much appreciated. See here: https://mail.haskell.org/pipermail/libraries/2015-May/025597.html Based on the analysis, I think this proposal should be shelved for the time being. And thus, I'm closing this feature-request. For the record, I'm quoting the final message from the above discussion. {{{ Thank you for all the feedback on this proposal. Based on the feedback, I came to conclude that the original idea did not really capture what I really was after, and hence I think this proposal needs to be shelved for the time being. I want to summarize the points made so far: * Almost everyone agrees that we should have this functionality available. (But see below for the direction I want to take it in.) * There's some disagreement on the name chosen, but I think this is less important for the time being. * The biggest gripe is where does "fma" really belong. Original suggestion was 'RealFloat', but people pointed 'Num' is just a good place as well. * Most folks want a default definition, and see "fma" as an optimization. It is these last two points actually that convinced me this proposal is not really what I want to have. I do not see "fma" as an optimization. In particular, I'd be very concerned if the compiler substituted "fma x y z" for "x*y+z". The entire reason why IEEE754 has an fma operation is because those two expressions have different values in general. By the same token, I'm also against providing a default implementation. I see this not as an increased-precision issue, but rather a semantic one; where "x*y+z" and "fma x y z" *should* produce two different values, per the IEEE754 spec. It's not really an optimization, but how floating-point values work. In that sense "fma" is a separate operation that's related to multiplication and addition, but is not definable in those terms alone. Having said that, it was also pointed out that for non-float values this can act as an optimization. (Modular arithmetic was given as an example.) I'd think that functionality is quite different than the original proposal, and perhaps should be tackled separately. My original proposal was not aiming for that particular use case. My original motivation was to give Haskell access to the floating-point circuitry that hardware-manufacturers are putting a lot of effort and energy into. It's a shame that modern processors provide a ton of instructions around floating-point operations, but such operations are simply very hard to use from many high-level languages, including Haskell. Two other points were raised, that also convinced me to seek an alternative solution: * Tikhon Jelvis suggested these functions should be put in a different class, which suggests that we're following IEEE754, and not some idealized model of numbers. I think this suggestion is spot on, and is very much in line with what I wanted to have. * Takebonu Tani kindly pointed that a discussion of floats in the absence of rounding-modes is a moot one, as the entire semantics is based on rounding. Haskell simply picks "RoundNearestTiesToEven," but there are 4 other rounding modes defined by IEEE754, and I think we need a way to access those from Haskell in a convenient way. Based on this analysis, I'm withdrawing the original proposal. I think fma and other floating-point arithmetic operations are very important to support properly, but it should not be done by tacking them on to Num or RealFloat; but rather in a new class that also considers rounding-mode properly. The advantage of the "separate" class approach is, of course, I (or someone else) can create such a class and push it on to hackage, using FFI to delegate the task of implementation to the land-of-C, by supporting rounding modes and other floating-point weirdness appropriately. Once that class stabilizes and its details are ironed out, then we can imagine cooperating with GHC folks to actually bypass the FFI and directly generate native code whenever possible. This is the direction I intend to move on. Please drop me a line if you'd like to help out and/or have any feedback. }}} -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:13 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: closed Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: wontfix | Keywords: report- Operating System: Unknown/Multiple | impact Type of failure: None/Unknown | Architecture: Blocked By: | Unknown/Multiple Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Changes (by lerkok): * status: new => closed * resolution: => wontfix -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:14 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
#10364: Feature request: Add support for FMA -------------------------------------+------------------------------------- Reporter: lerkok | Owner: ekmett Type: feature request | Status: closed Priority: normal | Milestone: Component: Core Libraries | Version: 7.11 Resolution: wontfix | Keywords: report- Operating System: Unknown/Multiple | impact Type of failure: None/Unknown | Architecture: Blocked By: | Unknown/Multiple Related Tickets: | Test Case: | Blocking: | Differential Revisions: -------------------------------------+------------------------------------- Comment (by rwbarton): Be aware (if you aren't already) that GHC does not do any management of floating-point control registers, so functions called through FFI should take care to clean up their floating-point state, otherwise the rounding mode can change unpredictably at the level of Haskell code. -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/10364#comment:15 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler
participants (1)
-
GHC