22 May
2007
22 May
'07
8:50 a.m.
On Tue, 22 May 2007, Vincent Kraeutler wrote:
as was pointed out on the programming reddit [1], crawling of the haskell wiki is forbidden, since http://www.haskell.org/robots.txt contains
User-agent: * Disallow: /haskellwiki/
and indeed, a google search gives the old wiki http://www.google.ch/search?q=haskell+wiki i.e. http://haskell.org/hawiki/FrontPage rather than http://haskell.org/haskellwiki/Haskell
This also applies to Haskell mailing lists as I mentioned recently: http://www.haskell.org/pipermail/haskell-cafe/2007-April/025006.html