Of course everyone digs for numbers to support their position, but to me some vary their criteria much more than others.
Guys like harmony catch a lot of grief, but he sticks to one criteria (WAR/projected WAR) for everybody. Yes, he may cherry pick the sample size, but he's much more consistent than others.
I think people that know me, knows I rarely use small sample sizes, and when I do I actually state "small sample size alert".
I have been pretty consistent with this criteria:
I use OPS for offense and UZR/150 for defense more than I use WAR.
I use 3 year sample sizes much more than 1 or 2 year sample sizes, although I do go to 4 years, if the player in question had a low PA season within the last 3 years.
I try to use a significant sample size of PAs or IP to create a group of players that makes sense-- like the top 30 everyday player at a given position or the top 150 starting pitchers (5 starters x 30 teams).
Occasionally, I will use a 2 year sample size, if I feel the 3rd year should not be counted due to an injury or a rookie season of futility. I guess, when I do this, one could argue "cherry-picking", but I do try to stay consistent with my criteria of judgement.
For pitchers, I use ERA- not ERA, WHIP, especially for RP'ers and OPS against. I do not count K% as much as others, and so I discount xFIP and related metrics to some extent.
I avoid Fldg%, BA, RBIs and W-Ls.
i always have.