Shoutbox

that's the thing that comes in my mind immedialty... With the current system, your postcount doesn't rely on other people... Although mods should be objective (I'm not saying the currents ones aren't!) in their job modding the forums, the new point-system will take it a step further. This means the job of moderator will be more difficult, as it is a hell of a job to stay objective ALL the time (and we're only human)... That's the drawback of the new system...
Afcourse, to more people who can vote, the more accurate the point-system will be....

But then again, every system has his drawbacks...

Maybe to combine those two systems? (like some other forums do) Keep the post count, (loose the "special" title's though) and implement the point-system. IMHO, this is the best way of seeing who's who:
a respected good poster will have a big postcount and good points (green)
a spammer would have big postcount but bad negative points (red)
a new poster will have no points and no postcount
a respected member who doesn't post much will have low postcount but good points (in the current system he/she's considered a newbie)

Anyways, like WDZ said, every new implemented system will draw comments and you have to get used to it. It is always a hasle to change some long-running thing into something new....