Look downward and swim forward to dive underwater.

Once under, your weapon set will automatically update for underwater combat.