The only true answer comes from prayer. If he's at fault, he needs prayer. If he's being misunderstood, he still needs prayer. Either way, don't stay where you don't feel comfortable. Ask God to show you where you should be in the place of worship. The Holy Spirit will never mislead you.
BTW: You are not under any obligation to remain at your co-workers church. If it's where you belong, God will secure your heart and give you His peace about it.
Take care.