The defectors are weary from their journey, so keep their health up. They're too frightened to travel farther without an individual escort. Talk to one, and they'll follow when you beckon.