Mario Balotelli drove in a shot to put the home side in front before Gareth Barry turned an Andreas Weimann cross into his own net for Villa’s equaliser.
Aleksandar Kolarov’s free-kick put City back ahead but Gabriel Agbonlahor swept in a shot as Villa levelled again.
In extra-time, Charles N’Zogbia slotted in after a parried shot by Agbonlahor, whose deflected strike sealed the win.
The League Cup may not have been high up on City manager Roberto Mancini’s list of priorities this season but the defeat and the manner of it will be a concern.
His side has already conceded 13 goals in seven games this season as City struggle to recapture the defensive qualities which helped them win the Premier League last season.
Villa’s win follows a 4-1 league defeat by Southampton on Saturday and will give their manager Paul Lambert plenty of encouragement.
City made 10 changes to their team which drew 1-1 with Arsenal on Sunday, although it was going to be 11 before centre-back Joleon Lescott replaced injured Jack Rodwell late on.
The alteration appeared to unsettle the home side as they made a sluggish start, while Villa were showing plenty of promise and purpose.
Weimann had a shot deflected wide and Christian Benteke blazed an effort over before the visitors lost their concentration and were clinically punished by City.
City midfielder James Milner, playing against his former club, found an unmarked Balotelli and he took a touch before driving in a shot for his sixth goal in four games against Villa.
The home side came out with more urgency after the break but, like the opener, the second goal came against the run of play as Villa equalised.
Weimann delivered a cross into the City area which was palmed out by keeper Costel Pantilimon and deflected off Barry for an own goal.
City regained the lead through a a superb Kolarov free-kick and appeared set to surge on to victory, with Carlos Tevez firing a free-kick on to the top of Villa crossbar.
But the visitors refused to be beaten as they showed fighting spirit and character, which was matched by Lambert on the touchline as he was involved in a touchline confrontation with Mancini following a tackle on the pitch.
And it was Agbonlahor who forced extra time when he used his pace to surge into the area before rifling home.
In the extra 30 minutes, City failed to find the guile to break down a resolute Villa side and the visitors capitalised on the counter-attack to knock City out.
Pantilimon could only parry Agbonlahor’s shot and N’Zogbia rolled in the rebound.
Agbonlahor then scored with a deflected strike, his brace being his first goals since January, to complete the fightback.