You can't send a message without opening a window (that is: not without handling protocol messages yourself).
Your second suggestion will work, but the z-order of windows will also change when you put the previous on-top window back on top. And don't forget to also record the window state (minimize or not) and setting it again to this state afterwards. This is what happens when you implement your second suggestion:
Before: When sending: After:
Program1 Program1 Program1
Contact1 Program2 Program2
Program2 Program3 Program3
Program3 Program4 Contact1
Program4 Contact1 Program4
So, you see the Contact1 window is not exactly in the same z-position anymore like it was before. Normally it doesn't matter much, but still...
If you don't want that you could also record the current z-order of the open chat window and place it again in that position after closing the other one. Again, also recording the windows' state.
Of course, all this should only be done when tabbed chats are enabled and there is at least one chat window open. Otherwise it would be useless.
Also, make sure you first check to see if there isn't already a chat open with that contact you want to send a message too. Although you still must check the z-order and set it again like it was after sending the message, you don't want to close that chat window in that case.
At the moment I can't give you the exact code for all this because of lack of time, but at least you (or anyone wanting to code this) have some ideas now I hope. It involves some Windows APIs, which can be found on the MSDN library
under 'windows functions'.