it can still be the PSU, every PSU has more than one output voltage, so it could be that one of them died. I'd try one from another PC before running to the store to get new parts. Actually, I would recommend not to buy any new parts until you're sure it will solve your problem.
The fact that your CPU fan turns a little after turning the PC on does not necessarily point to a faulty CPU in any way. The CPU fan is connected to your mainboard, not to the CPU. In fact, if I cold-boot my pc, the CPU fan also spins only a little before stopping again (smart fan control by the bios, temperature controlled) so that could even be normal behaviour.
How about your PSU fan, is that one spinning? (although that one could be temperature controlled too)
Anyway, if a PSU from another PC does not solve your problem, try removing as much of the hardware as possible (eg. use only one memory bank(try all slots and banks), use onboard vga (if available) and remove the AGP/PCIe card, remove extra pci cards, hdds and cd/dvd/floppy drives, unplug usb devices (except mouse and keyboard ofcourse
), etc.) and see if it works.
If it does, put the hardware back one piece at a time until you identify the culprit.
If it still does not work, try borrowing parts until you find out what's causing the problem. If you can't borrow the parts it might be better to return it to the shop rather than buying new parts without knowing if they will solve your problem (unless you planned on buying them anyway)
Bit off-topic, my sisters pc wouldn't boot yesterday. No beeps, no errors, monitor displayed "signal cable not connected" message only when the pc was turned on, only the fans would start spinning (no smart fan control ), turned out my dad forgot to plug in the 4-pin atx 12V connector after switching her PSU 
/ 
