There is no cancel button in the "replace with" form. I think that's very useful if you change your mind and don't want to touch that.
When you hit "save settings", the form should close.
I got a little error, but can't do it again. My anti-virus auto-update lagged me for a bit, so I guess it was related.
It's a good plugin, it only changes your nick when the song changes. A timer would fill the events file/window with too much "you changed your nick".
Good! I like it.
Now you'll need to put separate tags, imo. Artist, title, etc. Like in MPWinamp