Only when I wash and go. I think it depends on the conditioner if it leaves build up. Some do, but some don't. I think it's only the ones with Cones that do.
I've always got some sort of conditioner/leave-in in my hair. When I do a wash and go, I leave regular conditioner in my hair (for whatever reason it doesn't weigh my hair down) instead of a leave-in.
When I do a rollerset, I'll use a leave-in sometimes its NTM but nowadays its usually Aveda hang straight and/or Brilliant USC. I'll apply daily until I finally wash out the rollerset. There usually is some build up towards the end, so when I finally wash I'll use a clarifying shampoo like Aveda Detoxifier.
I apply a leave-in or a regular conditioner daily, usually Salerm or MNT (if I need some strength). I apply it at night so it soaks in. I don't get any build-up, though I don't use much, concentrating on my ends.
I only apply leave in condish on wash days. In between, I use water (to moisturize) and oil (to seal) as needed. I found if I use anything but water and oil on my dry hair, it's overload. But then, I have super fine, extra thin strands. . . .