According to Herodotus, when Xerxes arrived, his million man army was initially opposed by a Greek force led by 300 Spartans at the mountain pass of Thermopylae.
But in reality, the Persian empire, Xerxes left because he'd done his year, which is what you're supposed to do, and then he's got a whole empire to run.
That made the Persian king Xerxes mad, so he led two major campaigns against the Athenians, and the Athenians enlisted the help of all the other Greek city-states.