We do that now, only not with our "closer".
I don't mind responding to a hypothetical question as long as you realize you're going to get a hypothetical answer.
To answer your question, obviously it would depend on the situation. Say the Sox are nursing a 2-0 lead in the 6th and the other team's #8 & 9 get on base, I'd get the best reliever up and tossing, just like I would with any other reliever. Remember, "tossing", starting to warm up, is different from the hard throwing when a pitcher is getting ready to come into the game.
But if the score were 14-12 with the #8 hitter coming up I wouldn't do it as quickly. I wouldn't because I have faith in our offense to score runs in the succeeding innings.
Hypothetically.