It started just after the charr pulled back their forces. The ogres came from the east and moved right in. It's a constant battle to keep them from getting closer.