I don't have much
(read: as good as no) experience with VB.NET, but here are my thoughts:
If "testingkey" is a registry key, you shouldn't be using DeleteValue but
DeleteSubKeyTree.
The HKEY_CURRENT_USER should also be left out, since you're already in the CurrentUser key.