I was in a public building. It felt like a student union, or something else of the sort with open areas and scattered seating.
I had been chatting with a man who then asked me to come with him.
I took off my coat and left it at the seat, and reached to do the same with my hat. He said, "You can leave your hat on."
He then led me to the men's restroom.
And that was when I realized why he'd phrased it that way.