Offensive stats are subjective too. Perhaps they don't have as much subjectivity as defensive stats do, but they are subjective nonetheless. As Hugh said, every ball or strike called by an umpire is subjective. Hits versus errors are subjective, which makes BA, OBP, and OPS all subjective.
Calling defensive statistics "mostly voodoo" is a little out of line, IMO, though you are certainly entitled to view them that way. The key is, as always, to look at as many stats as possible in conjunction with the eye test. However, in a crunch, I'm going with UZR/150 and DRS and feeling fairly confident that I have a pretty good read on that player's defense.