So I just rewatched the DBS:Broly Movie and I just realized Gogeta wasn't at full strength when he fought broly. Here's a picture of Goku and Vegeta before fusing. They were both
Fusion Dance of Stone Saiyan and Son Goku, Stoku! by UltimateSaiyanGod on DeviantArt