It looks like all of the merchants have returned. You got lucky, but you've gone and stirred up the pirates now!