Your refusal to help complicated matters.

I really can't take no for an answer.

I'm hardly in a position to disagree.

Her obstinate refusal frustrated the whole team.