GMs are let go, all the time. It's nothing new. It comes with the territory.
No doubt, the Sox have had more turnover in the last 12 years than most other teams, and in some cases 3 more changes. I'm sure thinking you may be uprooting your family in 4 years might play a part in your choice, and does seem to be the "easy excuse" for saying no, but I'm not sure it's the number one factor for most of the "no's" we've gotten, so far.
For all we know, some who have backed away might have asked about the plan on spending over the next few years and not liked the answer. You can't exactly tell the media, "I didn't take the job/interview because they said they were not going to spend."
They also cannot say, "I wanted to hire my own manager, and JH wouldn't let me."