I don't think it has anything to do with chemicals floating around in your system or anything, but rather the much simpler and more sensible explanation that the hair follicles have been damaged in some way by consistent application of the relaxer. It's not the chemicals themselves lingering and affecting the hair directly, it's the damage they did when they were first applied.
Some people will lose hair completely (hair falling out in patches) over a single relaxer application, and the hair doesn't just grow back regularly if the scalp has been damaged. Clearly relaxer can affect the follicles themselves if it gets on the scalp. And so it makes perfect sense that even though your relaxer may not cause that kind of visible damage, it can still be causing fairly minute damage to your scalp/hair follicles if you relax frequently. You just never notice the effects if you continue to relax as usual. It's when people decide to stop relaxing that they can tell their hair feels rough, doesn't act right, etc.
And once they've given their scalp enough time to do whatever healing it needs to do, then their hair starts growing more normally. This time period would obviously be different from person to person depending on their relaxing habits and how much damage there is. That's why some people have long periods of scab hair and some have only a little bit. And just as LynnieB mentioned, a lot of people who have scab hair tend to have it in the area they routinely applied the relaxer to first. It has been my general observation (only based on things that people have reported themselves in past threads on scab hair) that people like myself, who had/have no scab hair were not frequent relaxers (their scalps had more time to heal before the next touchup, so the cumulative effects were negligible), or usually rinsed the relaxer out quickly (not much damage was done to begin with). Or perhaps their scalps were just healthier in general (for diet and grooming reasons) and so they repaired themselves faster.
Anyway, my point is the idea of scab hair is not far-fetched at all. I think some people doubt it because of the way people suggest it's because of "chemicals lingering in your system" (which IMO is far-fetched).