Dear GHC Devs, I have some interesting information to share with you. You might have noticed that our gitlab was a bit more swamped the recent days again and today I investigated and discovered that Anubis was getting bypassed by requests with a new user agent that I haven't seen before, "Lightpanda/1.0". Normally, anubis checks for user agents that are known bot user agents and blocks them; Lightpanda was left unblocked because Anubis only filtered for LightPanda (capital p). This is normally not a problem because most crawlers try to use user agents that look like browsers; if you're using a browser user agent, you're getting a higher weight and you get presented as challenge; but you're also much harder to sift out because you look benign. While the amount of requests we were getting with that user agent had reached absurd dimensions, this was also a blessing. First of all, it was easy to ban it and reduce the gitlab load drastically. Second, now we could run an analysis on who is using these malicious clients. Xe, the creator of Anubis kindly provided me with a tool that qualitatively (based on unique IP addresses, disregarding hits per IP) puts IP addresses in buckets based on several known factors, e.g. location or the ASN (which entity the IP address block is delegated to). The results can be seen here: https://gitlab.haskell.org/-/snippets/6132 A few noteworthy things: - 1.4 Million distinct IPv4 addresses have been used with this bot user agent - the IP addresses stem from 225 different countries. I think we have ~ 30 more distinct ISO country codes. - we have some known faces! like spacex which got flagged 100% time on their 21000 distinct IP addresses they're using to DDoS us - we also have quite a lot of domestic use ASNs, which indicates that botnets are involved, too! - Xe noted, that we can proudly say this counts as a DDoS Mind that this does not include all the spammers that are actually trying to hide the fact that they're spammers, which still exist. Also mind that this was just the data from ~1 day of logs. I hope this gives a nice sense of what we're dealing with and maybe it also helps with your frustration to see my frustration and the fact that we're sometimes getting little wins. Like today. Best Magnus