Take whatever opportunity you see to disrupt the bandits' activities—destroy plans, rewrite ledgers, free prisoners. Just no direct attacks unless you can do so without raising suspicion.