There's a bit of a flap going on in the US at present regarding a silly bint (sorry, Senator) who got her knickers in a knot over being addressed as "ma'am" by a general. She wanted to be addressed as "Senator".
Blackfive gives a nice take on this hoo-hah from a US perspective. Here's my take, from an Australian perspective.
I actually know a couple of serving and former Senators personally. I've seen them interacting with people in all sorts of circumstances, and I've yet to see one get on their high horse about how they are addressed. Maybe that's just something about the Australian character - we don't get uptight about the whole title thing. Judges on the other hand... don't get me started on them. And idiots with PhD's - they are the worst, always wanting to be addressed as "doctor". But Senators - they're generally a pretty relaxed bunch.
As for the military angle, when not in a strictly formal setting, I always addressed my platoon commander as "boss" or "chief" or occasionally "bwana". You couldn't do that straight away - you had to evolve into an "old sweat" in order to reach that stage of informality. You certainly didn't do it if the RSM was in earshot. It was a private thing, for platoon members only. Other platoons had their own terms of endearment for their commanders. For the young platoon commanders, straight out of OCTU (officer training), they knew they'd made it when their form of address out bush changed to "boss", or something similar. I'm sure many remember when they "lost their virginity" in this respect - because it meant that the men had grown to respect them as a leader and commander. In modern parlance, they had "bonded".
However, it was interesting to note that everyone in the Regiment addressed one dickhead captain as "Sir" or "Captain" under all circumstances.
Now there may be areas of the Australian military where this sort of thing is frowned upon, and things might have changed since I hung up my giggle hat, but that's how I remember it.