There are three different gateways leading into the courtyard. Make sure to defend all of them.