Well, if your goal is to get the best teams into the playoffs and you want to rank which teams are best, margin of victory absolutely matters. Of course, if you don't care whether the best teams get in or not, then it really doesn't matter. But if you don't care whether the best teams get in or not, why not just pull the names out of a hat?
As for why the Harbins are "unfair" - the biggest issue with the Harbins (as I have said multiple times in this thread) is you get points for playing teams that won a lot of games (whether you win or not or whether you get stomped or not). You get punished for playing teams who do not win a lot of games (whether you lose to them, beat them by 1, or beat them by 100). I understand this may be over your head, but that is what makes the Harbins "unfair."
I am not in favor of a poll. I am not in favor of letting everyone in. And I cheer for a team that is generally helped by the "unfairness" of the Harbins and pretty much always gets in anyway.
But I am definitely for the most deserving teams getting in and the current system doesn't always do that. It does largely get things right - there aren't a ton of instances of huge obvious mistakes - but Winton Woods (and probably a lot more that I don't know about) was one of them.