Um, my point was we'd acquire a closer like Hand. (We may also add a minor pen piece, but I only mentioned major additions.)
Kike Hernandez is a capable 2Bman and CF'er, so he could possibly upgrade both positions. While he has a minus UZR/150 at 2B, he's +18 DRS and plus in both metrics in CF. A career .738 OPS for a 2Bman is better than the norm for 2B, since he started playing in 2014 and pretty close to the league average for CF'ers, too. He may not be a plus, but he's about as close to average as one can be.
The question marks you raise about the Sox rotation, after adding Ordorrizi & Kluber can be equally raised with almost every other AL rotation, including your Yankees.