Look at it this way: Once you reach your goal length, you can do whatever you want.
I've never had an issue with wearing protective styles when I'm not working because they're easier for me, and I don't like my hair getting caught on my purse. So I found advantages to protective styling to mentally shift my perspective on it.
Now, I'm about 2 or 3 inches past brastrap, and I wear my hair out whenever I feel like it which is about once or twice a week. The other days it's in a pony or a bun because it's just easier for me. However, there are ladies here (Lauren450 & Sylver2 come to mind) who have BEAUTIFUL hair and wear theirs out almost daily. So, no protective styling doesn't have to equal damage.
The heat is a different story. When you survey the ladies here with amazing hair about their top hair care tips, low heat & low manipulation will certainly be on the top of the list.
Have you considered alternating the weeks you use heat? Sometimes I get in a mood for a month or two where I want semi-straight hair so if I decide to flat iron, I'll flat iron week 1, braidout week 2, flat iron week 3, bantu knot out week 4 or some variation of this. That way, I can enjoy a straight style for half the month and not be stressing about excessive heat. However, again, there are ladies here who use heat weekly and have beautiful manes. I think they're in the minority though.
Why do you lose so much hair from rollersetting and flat ironing? I lose almost no hair when I rollerset & flat iron the roots. Is there a moisture/protein issue?
Also, how often are you deep conditioning? I deep condition once or twice each week so I feel that that gives me a pass to wear my hair out when I want and/or use heat on occasion. However, if for some reason I slack on DCing, then I use NO heat and my hair stays in a protective style until my next wash.
Sorry to type you to death.
HTH
{DI}