Definining a safe subset of IO is usually an application-specific decision, e.g. do you want to allow access to the filesystem but without allowing openFile "/dev/mem"? It's a minefield.
Ryan, if you want to give an operational semantics for memory in IO, why not start from the subset of IO that you can accurately model - the basic IO structure together with a set of primitives - and define the semantics for that? That's typically what we do when we're talking about something in the IO monad.