Simon Vukcevic headed home the opener for the visitors in the first half, and it looked for all money that Rovers would claim an invaluable away win over their fellow relegation candidates.
However a David Vaughan strike and a Sebastian Larsson free kick in the last six minutes saw The Black Cats claim the three points in the Northern Irishman’s first game in charge.
O’Neill was delighted with the efforts of his men after the match, and hopes it will set the ball rolling for a recovery.
“I am not overly bothered about starting my era with a win,” O’Neill told Sky Sports.
“But it is important for the players to get that confidence up.
“I suppose it was a microcosm of our season where we played well enough in games not being able to score then concede a poor goal and succumbed.
“But I thought they were terrific today, played brilliant and kept at it. And even after falling a goal behind didn’t seem to affect them,” he stated.
Despite the last-gasp nature of the win, O’Neill feels that his side deserved to take the three points.
“I thought we deserved to win the game, but you don’t always get what you deserve.
“We had endless possession but you just want that breakthrough and now we have points on the board which is vital for confidence and the crowd.
“We have got some really decent players at the football club, and when you lose matches even the very, very best lose confidence. So that is so important to us, and the crowd when they roared for the equaliser and the winning goal was stunning. I am delighted,” he continued.
Sunderland face a difficult game against Spurs at White Hart Lane next weekend, but the win lifts them up to 16th in the standings.
By Gareth McKnight