The Argentina international forward replaced Aleksandar Kolarov after 32 minutes, and rounded off a superb counterattacking goal on 83 minutes to seal the win after good work by Edin Dzeko and Adam Johnson.
Both teams played weakened sides, with key players rested, but neither took a step back in a tense and tight fixture in North London.
The result sees the Etihad Stadium outfit still undefeated domestically, and they will go into the hat for the semi finals draw.
Manager Roberto Mancini has big games in the league and Europe coming up in the coming weeks, but was delighted with the attitude of his players in getting the win against The Gunners.
“We had a chance seven minutes from the end and we scored. It was a great goal. A great counter-attack,” he told Sky Sports.
“I don’t think that Arsenal had a lot of chances, I think we played well.
“It will be better to win here in the Premier League but it is a good feeling because we are into the semi-finals of the competition.
“We have a strong squad – all of the top teams have a good squad and can change eight, nine or 10 players. It is normal for the Carling Cup to change.
“December and January will be crucial months because we play tough games and we play every three days. After January we can talk about whether we can win things or not,” he concluded.
City now host Norwich on Saturday in the Premier League, whilst Arsenal travel to take on Wigan at the DW Stadium.
By Gareth McKnight