The caller of your function f, rather than the implementation of f, gets to decide which instance of A f should return. In other words, f's implementation must be polymorphic in its return type. Your example
f x = x does not satisfy that property. That implementation has type f :: A a => a -> a, not the required type f :: (A a, A b) => a -> b.
The error message is saying that, from the class definition, GHC has deduced that f must return a value of type A b => b, but your implementation is returning a String (a.k.a. [Char]) instead. The notation b ~ [Char] means "b is equivalent to [Char]".