A few points:
1. The Bits instance for Integer essentially already behaves as if they are the 2-adic numbers. For example, if you use testBit you can discover that (-2 :: Integer) is treated as if it were an infinite sequence of 1's followed by a single 0.
2. However, because testBit takes an Int index, instances of Bits actually can't have an infinite number of 1 bits---at least not in practice---since you cannot observe anything past the (maxBound :: Int)th bit.
3. Currently, popCount on negative Integer values satisfies popCount (-x) = -(popCount x) which does not seem very well motivated and does not match the way negative values are presented via testBit.
I am strongly -1 on moving popCount to FiniteBits. popCount on positive Integers is useful and well-defined. I am mildly +1 on Zemlya's proposal to change the behavior of popCount for negative Integer values, though I am not sure it is really worth the trouble.
-Brent