Gervinho continued his recent strong goalscoring form with the opener on 42 minutes, but Kostas Mitroglou equalised on the stroke of half-time for the visitors.
Lukas Podolski put the Gunners back into the lead midway through the second 45, with Aaron Ramsey’s late strike securing all three points.
Steve Bould praised Olympiakos after the game, and was happy with the win.
“It was tough,” Bould told Sky Sports.
“They made it really tough for us and I thought they defended really well.
“We kicked the game off not too badly, but then we looked a little nervous. Maybe the result from the weekend drained us a little bit and we lost a bit of confidence, but overall it was just a fantastic result.
“They’re a good side, technically very sound and they made it really hard.
“We’ve watched tapes and we knew they were very good technically and they set the 4-1-4-1 with the big fella sat in front of the back four.
“Santi [Cazorla] couldn’t get on the ball in the first half and we tried to correct that at half-time, and I thought we improved,” he concluded.
Arsenal top Group B after Schalke and Montpellier drew 2-2 in the other pool match.