Totem offers this kind of functionality, and because both rhythmbox and totem uses gstreamer, I would think we already have a good solution for both of the playback scenarios with minimum hassle.
Yes, and I currently use totem that way. Hopefully in fc4 totem will be in default gnome desktop install, rather than needing to be selected.
But totem has a rather large interface when it starts up, it in a movie player.
Something like xmms but gtk2 and GStreamer is what I long for. Gamp is nice, and even works well for me - even with aac files (which Rhythmbox doesn't - yet ...) but Eina has the best interface I've seen (so far) and (well, it doesn't work for me) allows you to load a single song or load from a playlist.