The competition level might be the highest on a Sunday in April, but it's not the World Series. As for the 2004 ALCS, if the players had a let down against the Cardinals and lost, we'd still be eating s*** from the Yankee fans and no one would care about that ALCS, just like the exciting comeback in the 1986 ALCS became largely irrelevant when they lost the World Series to the Mets.
The World Series is it. As spectacular as Halladay's No-no was in last year's playoffs, even if it had been a perfect game, it would not rise to the same level as Don Larsen's game, simply because of the difference in the stages.