Opening an instance for your squad requires an unstable rift motivation, which can be obtained from tier 2 and tier 3 rifts.