#9647: allocation of 10790760 bytes too large -------------------------------------+------------------------------------- Reporter: mirko | Owner: Type: bug | Status: new Priority: normal | Milestone: Component: Compiler | Version: 7.8.3 Keywords: run time | Operating System: Windows insufficient memory | Type of failure: Incorrect Architecture: x86_64 (amd64) | warning at compile-time Difficulty: Unknown | Test Case: Blocked By: | Blocking: Related Tickets: | Differential Revisions: -------------------------------------+------------------------------------- I got an error message and invitation to submit a bug report. So here is the error message first, followed by my description of what happened. C:\Users\Owner\Documents\hs>ghc -o void void.hs -rtsopts [1 of 1] Compiling Main ( void.hs, void.o ) Linking void.exe ... C:\Users\Owner\Documents\hs>void +RTS -K400M -RTS > void.txt void: internal error: allocation of 10790760 bytes too large (GHC should have complained at compile-time) (GHC version 7.8.3 for x86_64_unknown_mingw32) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug C:\Users\Owner\Documents\hs>void +RTS -K40000M -RTS > void.txt void: internal error: allocation of 10790760 bytes too large (GHC should have complained at compile-time) (GHC version 7.8.3 for x86_64_unknown_mingw32) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Description of what happened. I gave my program really large numbers. It contains the following line: rr = (of40)*(r250^3041) In the above of40 is (if I remember correctly) the product of the first 40000 odd primes. I computed it with reduce (a computer algebra system) which saved the result (as an explicitly written out number in decimal notation), which I copied and inserted in my hs file, initializing of40 with its value. In the file it takes 2457 rows, 1024 cols, so it appears to be about 2 515 968 (that is two million+) decimal digits. As seen it is multiplied by r250^3041, here r250 is a 250-digit number so r250^3041 is about 760 250 (seven hundred sixty thousand+) decimal digits, so the product rr = (of40)*(r250^3041) seems to be more than 3 000 000 (three million+) decimal digits. I tried to compile it yesterday, I did get an error message from the compiler: I do not remember exactly what it was, I may try to reproduce it later and include it, but it was something along the lines of insufficient amount of memory. I run a Hewlett-Packard HPE 500y, upgraded to 16 Gb RAM (the max it would take), Windows 7, AMD Phenom(tm) II X6 1045T Processor 2.70 GHz (6 core). So yesterday after I got the first error message I increased the paging file size from automatic to 32768 Mb initial, and 65536 Mb max. Then tried to compile again, the compiler gradually took more and more memory (I was watching the performance using the Task Manager) so it gradually went up to using all 16 Gb of RAM, and stayed there for I think at least 40 min, could have been more than an hour (or even 2 hours, sorry can't quite remember). At some point it seemed the computer froze, the task manager stopped showing any activity, but I did not attempt to immediately restart it, the mouse would not respond anyway, and in a few minutes it fixed itself, it seemed to continue compiling, after which it eventually gave up and complained about insufficient amount of memory again. So this morning I increased the paging file to initial size 49152 Mb, max size 196608 Mb (I don't know if such big numbers make sense but I tried). I tried to compile again starting around 9am, left my apt at around 4pm and the compiler had not yet finished, and came back after midnight, the compiler had finished. It had produced a big exe file (I see now, around 9 pm, so it must have kept compiling for around 12 hours): 09/27/2014 09:38 PM 20,602,960 void.exe AVG (antivirus) claimed that this file was a threat and refused to let me run it, so I had to disable AVG, after which I got the error messages as already enclosed at the beginning of this description. The Paging file reports as: Recommended 24574 Mb, Currently allocated: 163765 Mb. Note I gave the options -rtsopts to the compiler, I do not remember anymore what they mean, and whether they are valid (or perhaps not used anymore), I got them from a webpage perhaps a couple of years ago and at that time they made sense, so I just kept using them. I also gave the option +RTS -K400M -RTS when I tried to execute the compiled program, and then +RTS -K40000M -RTS, both versions generated the error messages shown above. I tend to believe that giving Haskell such large numbers inside the .hs file explains all of the above problems, it might even be a bit surprising it went so far before it finally gave up. But I do not really know, so let me just report it. So I changed the paging file back to automatic, and I am compiling the same file (after renaming it) to try to reproduce the error message that I got yesterday (or rather the other day, now being after midnight). It has been compiling for about 4 min so far, nothing really visible, memory usage stays at 1.28 Gb (where it was before I started compiling). I don't quite remember what happened the first time when I tried to compile, I think I got the error message (about insufficient amount of memory) immediately, or almost immediately. This does not happen now, I wonder if the computer took into account that I changed the paging file back to automatic, I may need to restart to make sure. Well, ok, 13 min after it started compiling, the memory usage started going up gradually. It goes from e.g. 13.9 to 14 Gb in about a second or two. So in about 2 min it went from 1.28 to 15.9 Gb, and now stays there (I do not know how to interpret all that Task Manager shows me). I may not be able to reproduce the first error message: At that time I was running 5 copies of reduce, each busy multiplying numbers for some hours. Right now I am only running the GHC by itself, so no other load. It is running for about 23 min now, the last 6 of which the memory usage stays at 15.9 Gb. My computer may have learned to increase the paging file on its own. The computer refuses to give up this time, but I can't wait, going to bed is long overdue, if it generates an error message I'll copy it in the morning. Well, memory usage decreased a bit to 15.1, this happened after it stayed at 15.9 for about 15 min. It I remember correctly what had happened the other day, it will increase again, and then give an error message. It's at 15.3 Gb for about 3 min now. On the other hand the processor is doing almost nothing, going between 0% and 1%. Memory 15.4, 15.8, processor went up to 16% to 17% (which is = 1 out of 6 cores), memory 15.9, this is about 40 min after it started compiling and about 23 min after it went up to 15.9 Gb (15.8 presently). I should perhaps be patient and wait for the compiler to give up and give me an error message that I could copy, but I do not have the patience to do that: Again the original error message was something to the effect that the compiler refused to work because of insufficient amount of memory. I kept trying to compile, so apparently the compiler tried to be nice to me, and eventually compiled my program in 9 hours, after which when I tried to execute it, I was invited to write this report. It keeps compiling right now, at 16% (1 of 6 cores) processor, and 15.9 Gb memory usage, for about 47 min (30 min at 15.9). P.S. Thanks for Richard Eisenberg and Herbert Valerio Riedel for replying to my email. I initially had problems, trying to register and submit a new bug report, the BotScout stopped me, and it was too late at night to try to figure it, so I just emailed Richard in my frustration. Now, BotScout may not be the culprit: I did not read carefully what it asked me to do, and it was something like Evaluate `(!!6) ...' My knowledge of Haskell is rather superficial (I use it since it compiles and is relatively fast, and does not need an extra library to work with large numbers), so I have to often consult the tutorial for simple constructs. I have had some previous experience with Common Lisp (or Scheme, a course taken 12 years ago) and in Lisp there is a command "evaluate", and also single quotation marks are used within the language occasionally. So, I just copied the entire line Evaluate `(!!6) (whatever it was)' and pasted it in the interpreter which gave me some nonsense which I sent back to BotScout. BotScout then gave me a second chance, and I do not quite remember if I tried to be more careful, perhaps not, so BotScout wouldn't even let me try anymore after that (even if I started from scratch ... perhaps it remembered me). After that happened I copied only the (!!6), etc, and the my GHC interpreter did evaluate something meaningful (a single number, no error message). I just didn't want to spend the time to think that late at night, and at first it seemed to me that the word Evaluate might have been part of a Haskell expression. I question if the above (perhaps too lengthy) report is of any use, but again, I follow the instruction, if in doubt, submit it. Thank you again! -- Ticket URL: http://ghc.haskell.org/trac/ghc/ticket/9647 GHC http://www.haskell.org/ghc/ The Glasgow Haskell Compiler