As UPnP is enabled, it shouldn't be a problem your end. However, since when you take the router out, the problem disappears, it is the router.
If you wish to fiddle with ports, read your router's manual. The ports MSN use are posted somewhere on this forum - a search should find them