Emulating DirectX in a virtual machine would be extremely hard to implement, also, VMWare is cross-platform so there would be no point in DirectX.
To allow DirectX usage, VMWare would have to emulate the exact hardware in your PC, or allow extremely low level access to it... which would be complicated. Like, immensely complicated. I think
.
I think that VMWare itself actually emulates an S3 Trio controller, although that might be Virtual PC. One of them uses a proprietary driver specific to the company which allows the window to be resized dynamically.
It would be much easier to try and work out why it's not installing on Windows XP SP2 for you, because it works fine for me.