On Wed, 2013-05-08 at 19:36 -0500, Austin Seipp wrote:
Hello all,
On IRC today, Nicolas Trangez brought up the idea of registering GHC for Coverity's SCAN project. SCAN is essentially a free service run by Coverity, which runs their Static Analyzer on open source projects ('open source' being defined by OSI) and gives the results back to developers. Coverity is widely regarded as having the absolute best tool in the business for C codebases as it stands. The idea is to try and rattle bugs out of the runtime system if possible.
I think this is a good idea, but it needs some public discussion. Namely, Coverity requires an official Point of Contact within the project (maintainer/author/governing body) to register the project in their name. We are left up to determine who this is and verify it. Bug reports which are found are not also made public, as they could be potentially serious security vulnerabilities.*
So, if done, this is a rather piece-by-piece project which requires a bit of commitment, because it will require the registree to not only assess the bugs, but sensibly move them into the issue tracker or fix it themselves.
It's also a question of whether it's worth it. Personally, I think it has the chance to uncover some real bugs. But the RTS is highly unusual C code (by some standards) and it will probably require tuning and patience to get tangible results. Bugs will also need to be assessed by someone. False positives happen etc. This is the double edged sword of static analysis.** Anyone who does it should be very aware of this.
So, what do you all think? I'd particularly like input from Simon M here since he's the primary author of the RTS, of course. :) And anyone who does this will undoubtedly have to be in contact with him to some degree, for all the aforementioned reasons.
If nobody else would step up for this, I would be interested in contacting them if it seems worth it and giving it a shot.
Although I believe this is a good idea (who'd have thought ;-)), I won't step up as a potential 'lead' for this due to a lack of knowledge, and I can't be sure I'll be able to spend enough time on the project. If someone does give it a shot, I'd love to help out triaging etc (a project 'owner' can provide access to the result to others as well if I'm not mistaken). As Simon M. mentioned, the analysis might be confused since the RTS is in several places not 'your average C code'. This makes sense (although adding some annotations in comments can help), but maybe some false positives caused by this could also point at 'too magic' code, which might need some simplification love? Lua, Mono & Racket are some projects already part of the program which might have similar code patterns etc. Nicolas