You can reduce its effects—find gear strong enough, and you'll be able to put infusions into it. The more powerful your infusions, the more agony it will diffuse.