I think if the Sox trade Benintendi or Vazquez (or both) to get pitching, they will still acquire other players from outside the organization to fill both/either role(s). Chavis is really a bench player anyway, and is best used in the roles he excels at (such s hitting LHP). He's the new Steve Pearce, and if his career plays out like Pearce's, he'll be fine.
Now, will the Sox fill C and/or LF wth expensive players? Like Realmuto or Ozuna? Probably not. But just because pitching is the primary need does not make it the only need, and if the Sox deal one or both players for pitching, they very likely lessen the need to spend as much money on arms. As it stands now, they might only acquire one MLB starting pitcher, since they should have 4 returning if you count Pivetta (which we probably should, since Bloom clearly wanted him).
As for contributions from below, Dalbec is clearly in play and Duran might be. Beyond maybe a bullpen arm or two, I don't think the farm can supply enough players to turn this into a competitive team. As it stands, even relying on both Dalbec and Duran might be too much already.
But if the idea is to bulk up the pitching and depend on getting 600 PA's each from Dalbec, Munoz, Arroyo and Duran, there is no real point in improving all the pitching...