You haven't said if this is a Black man; however, I'd think any Black man that has been raised by a Black woman or has been serious enough with a Black woman to have her sleeping over at his home regularly has seen women wear scarves to bed. It's not that serious to say your beauty is somehow diminished because you prepared your hair and slept in a scarf.
I sleep in a scarf every night and have at least 15 scarves/bonnets. If DH notices my scarf come off in the night, he will nudge me to fix it. DH sometimes sleeps in a loc soc (a recent addition) since his hair is so long that it could strangle me or him if not tied up.
His comment about being worried seems odd. Did he think you were taking out clip in weave?
Unrelated but it makes me wonder if he's one of those men who freaks out if you remove your makeup and look the slightest bit different without makeup. I've heard of women who won't remove their makeup around their guy. Sleeping in makeup is damaging to the skin but I guess some women do what they feel they need to do.