Outlook Express is a basic email client and
Usenet news reader.
Outlook does not have news reading capability, but adds Calendar, Notes, Tasks and Journal. It is also a more advanced and configurable email client than OE.
One of the things I like about Outlook is the ease of backing up your data. Outlook stores everything in a single "personal folders" file, location of which you can choose. As long as you back up this one file, you will have everything (old email and other data) except your account settings. That is a huge improvement over OE, which stores data in multiple files in a hard-to-find location, below several levels of hidden folders.
If the only thing you want is basic email, then you will find both to be about the same, although Outlook will let you make more advanced rules, and give you additional options in some other areas. Outlook's "bonus" is the various productivity features mentioned above (which you may or may not care about), while OE offers email and newsgroups in the same client (again, you may or may not care about that).