I disagree on saying it's an invasion of privacy. There's nothing really to be kept private if you download an attachment.
Plus, if people do think that just put a simple check box (like said above) to opt-out and keep your username to 'anonymous' or 'private'.
I think it's a good idea