Absolutely agree on this. I've needed to render a number in binary a couple of times, and each time I had to stop and think about how to use showIntAtBase. It's not hard per se, but it is really fiddly. The expression I end up using is hard to understand at a glance and uses a scary-looking (but safe-ish) partial function:
showIntAtBase 2 ("01" !!) x ""
A dedicated function is a clear improvement, although the ShowS bit is still unnecessarily fiddly for use outside implementing Show instances:
showBinary x ""
To be honest, I really don't like the Numeric module at all. I use hex a bit more often than binary, but I just do it with printf. I was going to suggest adding %b to printf alongside this change, but it looks like Text.Printf already supports it! So while I think adding showBinary absolutely makes sense, I'll probably just use printf myself going forwards.