The main problem with polygamy is how expensive alimony could get.

Okay, seriously, polygamy could be handled in the same way a business is run, you just have multiple parterners, no real problem with asset/tax management. The problems come into play when we're talking heterosexual polygamy and the potential for neglected kids or really ugly custody battles. I think the best way to keep things simple would be to just keep unions a two-person affair and be gender-blind.
As for incest and pedophilia, you're talking about either producing screwed up kids or screwing kids. Both of which have a direct effect on the overall welfare of society. Two men or two women of consentual age who are NOT RELATED BY BLOOD getting it on in the privacy of their own bedroom isn't going to produce screwed up kids, whether pre-existing or through conception.
It's like saying if you pass a law to allow people to ride Segways on the sidewalk that it will inevitably lead to people riding Harley Fat Boys down the sidewalk at 70 miles an hour. If there's an obvious gap in a law big enough to drive a motorcycle through, the law obviously needs to be fine-tuned to be more specific.