First of all, you forgot to set the Globel property to true. This is needed if you want to repeat the replacement for all finds and not just the first one:
regEx.Global = True
But both your pattern and replace strings are wrong though.
It should be something like this:
MsgBox (ReplaceTest("\b(\w)(\w+?)(\w)\b", "$3$2$1"))
See the description of the regular expression pattern property for VBScript:
http://www.msgpluslive.net/scripts/view/152-Windo...ipt-Documentation/
But note that it will not work properly if your words begin or end with other letters than the standard ones or with accents and stuff, eg: ö ô é, etc
-------
Also note that always replacing the first and last letter and adding "ay" isn't pig-latin.
Also the example shown by Jarrod isn't pig latin. Although it comes very close. There are more rules than just swapping the letters and adding "ay"
@Jarrod: Python works completely different and isn't going to help with VBScript in this case.
/ 
