Downloading the config archive

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hiya everyone. So once or twice people've asked for copies of my apparently rumored archive of xmonad.hss. Rather than check them into a repo on haskell.org or keep sending around tarballs, I decided to write a script using the mediawiki Haskell binding which would look up http://haskell.org/haskellwiki/Xmonad/Config_archive and download all the xmonad.hs files for one to peruse at leisure. (This had the advantage of keeping configs centralized at that one page.) Unfortunately it turns out haskell.org doesn't support the MediaWiki API. Oops. So I wrote it using Tagsoup: import Text.HTML.Download (openURL) import Text.HTML.TagSoup (parseTags, Tag(TagOpen)) import Data.List (isInfixOf, isPrefixOf) import Control.Monad (liftM, zipWithM_) main :: IO () main = do urls "http://haskell.org/haskellwiki/?title=" ++ drop 13 x ++ "&action=raw") $ filter (isInfixOf "xmonad.hs") urls let hs = map (liftM filterXmonad . openURL) raws zipWithM_ (\x y -> writeFile x = String filterXmonad = unlines . takeWhile (not . isPrefixOf "") . drop 1 . dropWhile (not . isPrefixOf "") . lines extractURLs :: String -> [String] extractURLs arg = [x | TagOpen "a" atts ls ./ 10.hs 13.hs 16.hs 19.hs 21.hs 24.hs 27.hs 2.hs 32.hs 35.hs 38.hs 40.hs 43.hs 46.hs 5.hs 8.hs xmonad.hi 11.hs 14.hs 17.hs 1.hs 22.hs 25.hs 28.hs 30.hs 33.hs 36.hs 39.hs 41.hs 44.hs 47.hs 6.hs 9.hs xmonad.hs 12.hs 15.hs 18.hs 20.hs 23.hs 26.hs 29.hs 31.hs 34.hs 37.hs 3.hs 42.hs 45.hs 4.hs 7.hs xmonad xmonad.o (I decided that a more elaborate or accurate naming scheme was too much of a bother.) Caveats: this runs in your current directory. this assumes all scripts are kept in a subpage of 'Xmonad/Config archive'; eg [[Xmonad/Config archive/Gwern's Config.hs]] and not [[Gwern's Config.hs]]. this assumes that 'xmonad.hs' appears in the page title. Else it'll be discarded as an irrelevant link, and not followed. this assumes that there is a newline after the first and before the first this also assumes that the xmonad.hs will be in the first clause; the rest will be dropped. That said, this downloads over 40 files for me, so it can't be missing too many, and many of those files compile. I hope this will be useful for people interested at looking at large numbers of xmonad.hs files. - -- gwern -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEAREKAAYFAkl6ngIACgkQvpDo5Pfl1oJLMwCfftZTZivjPjec0k6Zzq4d6zvc 4qcAoJJu4m7/z1CgclJk369G1DAgqMvj =RQr9 -----END PGP SIGNATURE-----
participants (1)
-
Gwern Branwen