The Reds fell behind in the opening minutes to a Matt Smith header, before Luis Suarez equalised.
Smith then restored the lead shortly before half-time, with on-loan defender Reece Wabara adding a third after the interval.
Joe Allen slammed home a deflected volley late on, but the Premier League side were unable to find an all-important equaliser.
Rodgers elected to rest a number of his senior players, pushing the likes of Jack Robinson, Sebastian Coates and Raheem Sterling into his starting XI.
He was disappointed by the way in which they responded and questioned their hunger during the 90 minutes:
“We are bitterly disappointed. We started the first half poorly and we started the second half and got punished, we’ve got a big week of games but our mentality in that opening period was poor and it ended up costing us.” He is quoted by Sky Sports.
“I was disappointed with the young players today I’ve got to be honest.
“We are trying to give them experience and let them see what it was like to play for Liverpool. It is not just about playing for Liverpool, you have to be competitive and be part of a group that is going to challenge for trophies.
“I thought how we reacted to that and how we performed, knowing that is what we are looking for, was disappointing.”
The Northern Irishman went on to suggest that his team did not show the required level of fighting spirit against the League One strugglers:
“We were just too much on the back foot early on, you really have to be aggressive.”
Oldham will host Everton in the next round.