December 2017 - ghc-devs

Haskell Platform 8.2.2 - virus?
by Matthew Lamari 28 Dec '17

28 Dec '17

New Haskell install was tripping my Bitdefender like crazy and in weird ways - not new as that's how bitdefender rolls. However, I retested in a clean test, with (free) Hitman Pro I started from a base case with 2 clean windows 8 VMs. New 8.2.2 install - has virus Old 8.0.2 Jan 2017 - no virus According to Hitman Pro, touchy.exe, haddock-8.2.2, ghc-8.2.2.exe, and unlit.exe have some problem post-install. I went no further on the VMs. "Detection Names Kaspersky Trojan-Downloader.Win32.Paph.fsv " Bitdefender didn't get it on install but would lock the whole thing down on the first run of "Cabal".

3 4

How do I build basic GHC packages with -fPIC?
by Viacheslav Lotsmanov 28 Dec '17

28 Dec '17

Hi there. My actual main goal is to build my own shared library written in Haskell that would be compatible with application written in C even without knowing that is is written in Haskell. So for now I compiled my shared library but I only could dynamically link it to Haskell dependencies such as "base" and "ghc-prim" packages. But I want to statically link Haskell dependencies but I realized it isn't simple and straightforward task. On Freenode's #haskell I was advised I should build GHC from scratch with -fPIC, on the Linux (I'm using Fedora Workstation 25 on x86_64) I couldn't go forward without this step. So I wrote some Dockerfile based on Debian 9, skipping first part which is containing 'apt-get update' and installing 'build-essential' here is what I have: COPY my-build.mk /my-build.mk RUN mkdir /compile && cd /compile \ && wget https://downloads.haskell.org/~ghc/8.2.2/ghc-8.2.2-src.tar.xz \ && tar -xvf ghc-8.2.2-src.tar.xz \ && rm ghc-8.2.2-src.tar.xz \ && cd ghc-8.2.2/ \ && ./configure --prefix=/ghc-8.2.2-fpic --disable-library-profiling --enable-shared \ && cp /my-build.mk mk/build.mk \ && make install \ && cd /usr/local/bin \ && ls /ghc-8.2.2-fpic/bin/ | xargs -I{} ln -s /ghc-8.2.2-fpic/bin/{} And as you can see I just use my own prepared /my-build.mk/ file which is: SRC_HC_OPTS = -H64m -O EXTRA_HC_OPTS = -fPIC SRC_CC_OPTS = -fPIC -O GhcStage1HcOpts = -fasm -O0 GhcStage2HcOpts = -fasm -O0 GhcLibHcOpts = -fasm -O2 GhcLibWays = v dyn DYNAMIC_GHC_PROGRAMS = YES DYNAMIC_BY_DEFAULT = NO SplitObjs = NO HADDOCK_DOCS = NO BUILD_DOCBOOK_HTML = NO BUILD_DOCBOOK_PS = NO BUILD_DOCBOOK_PDF = NO V = 1 LATEX_DOCS = NO HSCOLOUR_SRCS = NO BeConservative = YES I just combined it from parts I found in the internet during searching answers to my questions. So I built this container, I also installed dependencies by this commands: cd /mnt cabal update cabal sandbox init cabal install --enable-shared --ghc-option=-fPIC happy alex cabal install --enable-shared --ghc-option=-fPIC base-unicode-symbols filepath process directory lens containers qm-interpolated-string And when I tried to build my app by following commands (first command compiles some C-code to automatically initialize Haskell runtime, see link posted below, not sure if /-static/, /-shared/ or /-fPIC/ means something here but it's work in progress): ghc -static -shared -fPIC -optc-DMODULE=Foo src/lib-autoinit.c -outputdir builddir ghc -package-db=SOME_CABALS_SANDBOX_PKGDB_DIR --make-static-shared-fPIC src/Foo.hs builddir/src/lib-autoinit.o -o builddir/libfoo.o -isrc -outputdir builddir -Wall -O2 I failed with a lot of similar errors like this one: /usr/bin/ld.gold: error: /ghc-8.2.2-fpic/lib/ghc-8.2.2/ghc-prim-0.5.1.1/libHSghc-prim-0.5.1.1.a(Classes.o): requires dynamic R_X86_64_PC32 reloc against 'stg_ap_0_fast' which may overflow at runtime; recompile with -fPIC What have I missed? What should I do to make this happen? Any progress could be found here (Dockerfile, sources of modules, build-scripts): https://github.com/unclechu/haskell-experiment-shared-library-for-c-applica… Related stack overflow issue: https://stackoverflow.com/questions/47978884/how-do-i-recompile-ghc-with-fp…

3 6

答复: ghc-devs Digest, Vol 172, Issue 40
by BalterNotz 27 Dec '17

27 Dec '17

发送自 Windows 10 版邮件应用发件人: ghc-devs-request(a)haskell.org 发送时间: 2017年12月27日 20:32 主题: ghc-devs Digest, Vol 172, Issue 40

1 0

CircleCI failed for me
by Richard Eisenberg 26 Dec '17

26 Dec '17

Hi devs, I've enabled CircleCI on my GitHub fork of GHC in order to get good validation results. This worked several months ago. But now, I see failures around performance tests: https://circleci.com/gh/goldfirere/ghc/12 <https://circleci.com/gh/goldfirere/ghc/12> This is a tiny patch on top of HEAD; I can't imagine my patch is at fault here. Does anyone know what settings I should look into to fix this? Thanks! Richard

1 0

RE: Can't push to haddock
by Simon Peyton Jones 20 Dec '17

20 Dec '17

| It's for technical reasons, and the strongest one being: GitHub doesn't | allow us to establish strong invariants regarding submodule gitlink | referential integrity for submodules (which I implemented a couple years ago | for git.haskell.org) Interesting. It'd be good to document what the technical reasons are. For example I don’t know what the strong invariants are. A good place to describe them might be the Repositories pages https://ghc.haskell.org/trac/ghc/wiki/Repositories Many thanks Simon | -----Original Message----- | From: Herbert Valerio Riedel [mailto:hvriedel@gmail.com] | Sent: 18 December 2017 11:13 | To: Simon Peyton Jones <simonpj(a)microsoft.com> | Subject: Re: Can't push to haddock | | On Mon, Dec 18, 2017 at 10:01 AM, Simon Peyton Jones via ghc-devs <ghc- | devs(a)haskell.org> wrote: | > But why don’t we just pull from github rather than mirroring on | > git.haskell.org? | | It's for technical reasons, and the strongest one being: GitHub doesn't | allow us to establish strong invariants regarding submodule gitlink | referential integrity for submodules (which I implemented a couple years ago | for git.haskell.org)

7 13

ghc-prim 0.5.1.1 not on Hackage
by Michael Snoyman 20 Dec '17

20 Dec '17

The Stackage issue tracker just received an issue about ghc-prim 0.5.1.1. https://github.com/fpco/stackage/issues/3115 This version of the package has not been uploaded to Hackage, so: * The Stackage snapshots refer to 0.5.1.1 * The stackage.org documentation shows version 0.5.1.1 * `stack unpack` and `cabal unpack` both fail on ghc-prim-0.5.1.1 Also, it appears that the GHC 8.2.2 release notes still refer to ghc-prim 0.5.1.0. I'm happy to file an issue for this, but last time this popped up it was unclear if the issue should be filed on GHC Trac or the Hackage Trustees issue tracker. Michael

3 2

Re: Can't push to haddock
by Gershom B 19 Dec '17

19 Dec '17

> You're also assuming github doesn't suddenly pull a SourceForge (or a > Gitorious for that matter). Business cares not what it steamrolls in the > name of profit. > > I fail to understand why, with multiple examples of the folly of this > belief out there, people are still willing to bet on *this* company being > *different* from all others and absolutely safe to trust. What the heck is everyone arguing over? The initial statement was that even if we moved everything to github "We'd need mirroring anyway". That's it! There's no either/or involved. Just a statement that when you have data in one location, no matter how trustworthy, you want a backup as well. And ideally you want the backup under your control. We can debate moving anything anywhere all we want, but the idea that stuff should also be backed up, regardless, in a source not under third-party control, seems inoffensive and besides the point. --Gershom

2 1

Re: [GHC DevOps Group] Release policies
by Boespflug, Mathieu 18 Dec '17

18 Dec '17

I think Mikhail's point is that if a package says build-type: Simple, then we know exactly what its Setup.hs says, and therefore also which part of the Cabal API it's using. Easy enough to keep that part stable even if others change. Case in point: Cabal-2.0 brought a number of changes to the overall API, but nothing that broke calling defaultMain from Distribution.Simple (which is what a build-type: Simple script does). At the end of the day the consumers of the wider Cabal API are pretty small. A substantial number of misc packages on Hackage do it but rarely heavily. Other than that it essentially comes down to Stack, cabal-install, Haskell For Mac and... any others? My takeaway from the discussion so far is that the number of heavy consumers looks small enough that a draconian BC policy for Cabal-the-library sounds overkill, provided, crucially, that everything is in place, by GHC feature freeze at the very latest, to allow a smooth migration. A "smooth transition" means having a migration guide available before start of feature freeze etc, but to Ben's concern stated earlier in this thread (about GHC/upstream coupling), ideally also a release. I should note that to the extent that GHC tracks upstream releases only (not git commits in unknown state), GHC can be released on a timely schedule without needing any coordination from upstream maintainers to await new releases on their part. So quite apart from the Cabal thing specifically, it's worth thinking about asking that the versions of all upstream packages only make it into GHC, at the behest of their respective maintainers, after a new release of upstream is made. This was already proposed earlier in the thread: > * [Proposal:] GHC does not track git commits of upstream dependencies > in an unknown state of quality, but tracks vetted and tested releases > instead. Potentially, this could even mean drastically cutting down on the number of git submodules carried in the GHC repo. Since these packages could as well be downloaded from Hackage. On 18 December 2017 at 05:04, Manuel M T Chakravarty <manuel.chakravarty(a)tweag.io> wrote: > Yes, you are right Haskell for Mac also links against Cabal-the-library and API changes have regularly required me to fix my code. I guess, I have never been particularly stressed about it, because I also link against GHC API and that doesn’t even know how to spell API stability — i.e., changes required by Cabal are usually drowned out by the chaos inflicted by GHC. > > In any case, you are making a good point. > > Mikhail, I don’t understand your response to Mathieu at all. What does the build-type have to do with this? > > Cheers, > Manuel > >> 15.12.2017, 19:41 Boespflug, Mathieu <m(a)tweag.io>: >> >> Thanks for the feedback, Michael. >> >> Manuel, I believe you are also a Cabal-the-library consumer in Haskell For Mac? >> >> Michael, you brought up another problem tangentially related to the >> original integer-gmp issue but that was not in my original list >> earlier in this thread: >> >> * Cabal-2.0.0 had breaking changes in the API. >> >> This means that by association GHC itself broke BC, because it shipped >> with Cabal-2.0, without the usual grace period. >> >> Now, there are far fewer users of Cabal than of base. All, Michael in >> his previous email seems to be okay with breaking changes in Cabal >> given the conditions he stated (2 months grace period, advance notice >> of when the 2 months start). And perhaps this points to the lack of a >> need for the regular grace period applying to Cabal. How many other >> users of Cabal-the-library are there? In principle, every single >> Hackage package out there, which all have a Setup.hs script. Most of >> them are trivial, but how many did break because of these API changes? >> I for one am pretty happy for Cabal to move fast, but I'm concerned >> that these breaking changes happened without any kind of advance >> notice. To Simon's original point - there does not to be a clear >> policy and a good process surrounding Cabal itself and other GHC >> dependencies. So far we discussed mostly metadata changes, not API >> changes. >> >> And to be clear, folks did get some (post facto) notice in September: >> http://coldwa.st/e/blog/2017-09-09-Cabal-2-0.html. That's helpful, but >> I submit that in the future this really should be part of the GHC >> release announcement (which happened over a month before that), and in >> fact a migration guide circulated before the feature freeze, so >> downstream tooling authors can adapt. If this is not possible, then >> perhaps it's premature for GHC to include that given Cabal release. >> Again, GHC should always have the option to stick to the old Cabal >> version until things get ironed out. >> >> >> On 15 December 2017 at 08:42, Michael Snoyman <michael(a)snoyman.com> wrote: >>> >>> >>> On Thu, Dec 14, 2017 at 12:27 PM, Boespflug, Mathieu <m(a)tweag.io> wrote: >>> >>> [snip] >>> >>>> * Or a middle ground: make feature freeze a thing. Meaning that for a >>>> couple of months before a major GHC release, the major new Cabal isn't >>>> technically released yet, but like GHC itself within this period, it's >>>> pretty staid, so not so much a moving target, and something downstream >>>> tooling authors can possibly adapt to even without any grace period on >>>> new metadata features. This assumes that the 2 months of feature >>>> freeze are enough time for downstream tooling. Thoughts from any of >>>> those maintainers? >>>> >>> >>> Short answer: if there's a clear idea in advance of when this feature freeze >>> is going to happen, I think we can coordinate releases of downstream tooling >>> (Stack being the most important, but stackage-curator playing in as well) so >>> that 2 months is sufficient. I'll talk with the rest of the Stack team to >>> see if there are any concerns. >>> >>> Longer answer: Stack intentionally avoids depending on the internals of >>> Cabal wherever possible. Instead of calling library functions directly from >>> within Haskell code to perform builds, for example, it interacts with the >>> Setup.hs files over their command line interface.[1] This has two results: >>> >>> * Stack can usually start using new GHC/Cabal versions without a new Stack >>> release, since it's just shelling out for the actual build >>> * There's not usually very much code churn needed in Stack to upgrade to a >>> newer Cabal release >>> >>> This past release was an exception because of all of the changes that >>> landed, both the new cabal grammar to support the ^>= operator (making the >>> old parser incapable of lossily parsing new files) and API changes (I think >>> mostly around Backpack, though there was some code cleanup as well). In >>> particular, the main interface we need from Cabal—the package description >>> data types and parser—changed significantly enough that it took significant >>> effort to upgrade. There were also new features added (like sub libraries >>> and foreign libraries) that weren't immediately supported by the old Stack >>> version, and had to be manually added in. >>> >>> Tying this up: generally upgrading to a new Cabal release should be fine, >>> and the only concern I'd have is fitting it into a release schedule with >>> Stack. The complications that could slow that down are: >>> >>> * Changes to the command line interface that Stack uses (hopefully those are >>> exceedingly rare) >>> * Major overhauls to the Stack-facing API >>> >>> Michael >>> >>> [1] This allows for more reproducible builds of older snapshots, insuring >>> that the exact same Cabal library is performing the builds >> _______________________________________________ >> Ghc-devops-group mailing list >> Ghc-devops-group(a)haskell.org >> https://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devops-group >

3 2

RE: Can't push to haddock
by Simon Peyton Jones 18 Dec '17

18 Dec '17

| Yes, the mirroring has a little bit of latency (assuming the mirroring | trigger event notification from github to git.haskell.org didn't get | lost). How much time did you wait between pushing to github and | ghc.git? I didn't allow any time -- I didn't know that time was needed. Perhaps we should add a note to https://ghc.haskell.org/trac/ghc/wiki/Repositories to explain? Under "Updating sub-repos" perhaps. I wonder if it'd be worth us articulating the reason why some submodules live in github, but some live in git.haskell.org -- with only mirroring github. I'm sure there's a rationale but I don't get it yet. Simon | -----Original Message----- | From: Herbert Valerio Riedel [mailto:hvriedel@gmail.com] | Sent: 07 December 2017 17:57 | To: Simon Peyton Jones <simonpj(a)microsoft.com> | Subject: Re: Can't push to haddock | | Hi Simon, | | Yes, the mirroring has a little bit of latency (assuming the mirroring | trigger event notification from github to git.haskell.org didn't get | lost). How much time did you wait between pushing to github and | ghc.git? | | On Thu, Dec 7, 2017 at 6:53 PM, Simon Peyton Jones via ghc-devs <ghc- | devs(a)haskell.org> wrote: | > But when I try to push the GHC patch, I get this message | > | > Ah… it worked after a while. Maybe a mirroring thing? | > | > But in pushing to GHC I saw: | > | > git push | > | > Counting objects: 45, done. | > | > Delta compression using up to 32 threads. | > | > Compressing objects: 100% (45/45), done. | > | > Writing objects: 100% (45/45), 27.56 KiB | 0 bytes/s, done. | > | > Total 45 (delta 43), reused 0 (delta 0) | > | > remote: performing commit message validations... | > | > remote: Commit message validation passed! | > | > remote: performing submodule-ref update validations... | > | > remote: Submodule update(s) detected in | > fa29df02a1b0b926afb2525a258172dcbf0ea460: | > | > remote: utils/haddock => 24841386cff6fdccc11accf9daa815c2c7444d65 | > | > remote: utils/hsc2hs => 9483ad10064fbbb97ab525280623826b1ef63959 | > | > remote: OK | > | > remote: performing whitespace validations... | > | > remote: whitespace validation passed! | > | > remote: mirroring ssh://git@git.haskell.org/ghc to | > ssh://git@github.com/ghc/ghc ... | > | > remote: To ssh://git@github.com/ghc/ghc | > | > remote: 5f332e1..fa29df0 master -> master | > | > remote: running notifier | > | > To ssh://git@git.haskell.org/ghc.git | > | > 5f332e1..fa29df0 HEAD -> master | > | > simonpj@cam-05-unx:~/code/HEAD$ | > | > I did not intend to monkey around with hsc2hs. I can’t think how | that | > happened, or whether it matter. | > | > With many apologies, would a wiser person that me like to see if | I’ve | > accidentally messed up hsc2hs. | > | > Thanks | > | > Simon | > | > | > | > From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | > Simon Peyton Jones via ghc-devs | > Sent: 07 December 2017 17:32 | > To: ghc-devs(a)haskell.org | > Subject: Can't push to haddock | > | > | > | > I’m trying to push a patch that needs a supporting change to | haddock. | > | > I’ve pushed the haddock change to the ghc-head branch of | > ssh://git@github.com/haskell/haddock.git, which is (according to | > ‘packages’) the relevant haddock upstream repo. | > | > But when I try to push the GHC patch, I get this message | > | > bash$ git push | > | > Counting objects: 45, done. | > | > Delta compression using up to 32 threads. | > | > Compressing objects: 100% (45/45), done. | > | > Writing objects: 100% (45/45), 27.56 KiB | 0 bytes/s, done. | > | > Total 45 (delta 43), reused 0 (delta 0) | > | > remote: performing commit message validations... | > | > remote: Commit message validation passed! | > | > remote: performing submodule-ref update validations... | > | > remote: Submodule update(s) detected in | > fa29df02a1b0b926afb2525a258172dcbf0ea460: | > | > remote: utils/haddock => 24841386cff6fdccc11accf9daa815c2c7444d65 | > | > remote: *FAIL* commit not found in submodule repo ('../haddock.git') | > | > remote: or not reachable from persistent branches | > | > remote: hooklet hooks/update.secondary.d/check-submodule-refs failed | > | > remote: hooks/update.secondary died | > | > remote: error: hook declined to update refs/heads/master | > | > To ssh://git@git.haskell.org/ghc.git | > | > ! [remote rejected] HEAD -> master (hook declined) | > | > error: failed to push some refs to | 'ssh://git@git.haskell.org/ghc.git' | > | > simonpj@cam-05-unx:~/code/HEAD$ | > | > | > | > What’s up? I have pushed the haddock commit! | > | > THanks | > | > Simon | > | > | > | > | > _______________________________________________ | > ghc-devs mailing list | > ghc-devs(a)haskell.org | > | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | > askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7C684b5c6cdac34213317708d5 | 3d9be387%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6364826622509770 | 90&sdata=kMhG2iTALLRxhwyDw%2BzTN8VvMMn%2FqfvnSn9cPm0AK4Q%3D&reserved=0 | >

2 2

Release policies
by Simon Peyton Jones 16 Dec '17

16 Dec '17

Dear GHC devops group The conversation on Trac #14558<https://ghc.haskell.org/trac/ghc/ticket/14558> suggests that we might want to consider reviewing GHC's release policies<https://ghc.haskell.org/trac/ghc/wiki/WorkingConventions/Releases>. This email is to invite your input. The broad questions is this. We want GHC to serve the needs of all its users, including downstream tooling that uses GHC. What release policies will best support that goal? For example, we already ensure that GHC 8.4 can be compiled with 8.2 and 8.0. This imposes a slight tax on GHC development, but it means that users don't need to upgrade quite as often. (If the tempo of releases increases, we might want to increase the window.) Trac #14558 suggests that we might want to ensure the metadata on GHC's built-in libraries is parsable with older Cabals. One possibility would be this: * Ensure that the Cabal metadata of non-reinstallable packages (e.g. integer-gmp) shipped with GHC be parsable by the Cabal versions shipped with the last two major GHC releases [i.e. have a sufficiently old cabal-version field]. That is, in general a new Cabal specification will need to be shipped with two GHC releases before GHC will use start using its features in non-reinstallable packages. * Upholding this policy won't always be possible. There may be cases (as is the case Hadrian for GHC 8.4) where the benefit of quickly introducing incompatible syntax outweighs the need for compatibility. In this (hopefully rare) case we would explicitly advertise the incompatibility in the release documentation, and give as much notice as possible to users to allow downstream tools to adapt. * For reinstallable packages, of which GHC is simply a client (like text or bytestring), we can't reasonably enforce such a policy, because GHC devs have no control over what the maintainers of external core libraries put in their Cabal files. This is just a proposal. The narrow questions are these: * Would this be sufficient to deal with the concerns raised in #14558? * Is it necessary, ow would anything simpler be sufficient? * What costs would the policy impose on GHC development? * There may be matters of detail: e.g. is two releases the right grace period. Would one do? Both the broad question and the narrow ones are appropriate for the Devops group. Thanks! Simon

4 8