First off, when I talk about catcher defense, the most important component to me is how well the catcher handles the pitching staff, including game management and pitch framing. Blocking pitches in the dirt would be next. Caught stealing % is the area of a catcher's defense that I'm probably the least worried about. It is not worth as much in runs saved defensively as the ability to handle the staff. So yes, I think it's quite possible that a catcher could be 26, 27, or 28 years old before he has it all together defensively. Maybe his caught stealing won't improve a great deal, but his game management could and should.
Secondly, the scouting reports I've read have been favorable in terms of Swihart's defense. He is not supposed to be as good as Vazquez is, but he is supposed to be good enough defensively that when you combine that with his offense, he is supposed to be the better overall catcher between the two. He should be above average to good with some more development, not a defensive liability.
Thirdly, the guy is 23 and was forced into a situation well before he was ready. Seriously, are you going to write him off defensively based on such a small sample under less than ideal conditions? Please.