I don't think it was scorched, although I agree, it wasn't softly hit. But it's still just a ground ball, and, more importantly, it was on a pitch where Pettitte hit his spot.
Both of the hits were soft, and the walk was probably a pitch around to set up the DP. He got lucky with the line out, but Hardy should have been struck out on the 0-2 pitch.
My point is that Gardenhire was going to have him face the bottom five in the order no matter what, and that was the right move. If he had brought him in when you wanted him to, he would have to face nine batters.