I don't rate Martin as high as Doiji does, but i think you're seriously overselling Olivo here.
His defense is not that great. He has a cannon arm, but he's not particularly mobile behind the plate, and he's a PB machine (he's lead the league in passed balls four times while not even being a regular catcher). If you ever want to call a season an outlier, then Olivo's 2010 was an outlier, and he still gave up 10 passed balls, not a good thing when you're handling a staff with so many guys with big, looping breaking balls.
His offense against lefties is also not that awesome. Sure, he has an .821 career OPS against lefties, but he has a .318 OBP to show for it, and an even worse .298 OBP against them last year.
All things considered, he's just not that good.