In practice, you're better off letting idle connections stay that way, because then your DB server can close connections and free up resources. In other words, when you're done with a connection, put it at the front of the reuse queue, not the back.

You'll also want to handle the possibility that a connection that you grab from the pool has been closed by the server. Many connection pooling implementations I've seen get this wrong in subtle or expensive ways.