Bryan Oviedo scored an own goal to equalise for Swansea just four minutes later.
However, Ross Barkley scored a late winner to move Roberto Martinez's side up to fouth in the table. .
Meanwhile, Tottenham Hotspur are up to seventh in the table after they beat Southampton 3-2 at St. Mary's.
Emmanuel Adebayor scored twice for Spurs on just his second league game of the season.
Tim Sherwood took charge of Spurs in his role as caretaker manager.
He has admitted, however, that he's interested in replacing Andre Villas Boas full time.
Also in the hunt for the job is former England boss Glenn Hoddle.
He says he'd be happy to return to Tottenham if he was approached to take over, even temporarily:
