Galaxies like the Milky Way indeed could not have formed as a unit at such an early era after the Big Bang. The first galactic objects were smaller and denser than present-day galaxies, and thus could collapse and pull in gas on timescales of tens, rather than hundreds, of millions of years. Once gas compressed inside such a protogalaxy, stars could begin to form — an especially tricky step that is not fully understood — and then galactic evolution could proceed quickly.

Astronomy magazine subscribers can read the full answer for free. Just make sure you're registered with the website.