I mean no offence by this, but I have thrown away much better computers than that, and I keep everything...
I would concentrate on all the peripherals first as there is not much you can do with out a more serious upgrade, and could be transferred over to a new pc later.
=- DVD-RW drive -- $25
=- Case/PSU -- $50-$250...depends on psu etc
=- Bigger HDD -- $50-???... depends on size but your max choice will be limited if you stick with IDE
Add another gig of ram if you can find some at a decent price, older stuff like that tends to be more expensive if bought new, used or ebay etc might be a good place to look.
Finding out what your actual mobo supports as far as processors go, you may be able to pick up something a bit better ~2.8ghz in a 478.... again you pretty much will have to find that used.
Vid card... pretty much not worth worrying about as anything decent now will be pci-e, and you didnt mention whether the mobo even has an AGP slot. PCI vidcard....maybe if you can find an old Nvidia Riva TNT for $5-$10 used somewhere, otherwise dont worry about graphics too much.
Toss the floppy drive in the trash and just clean the whole thing up. If you have an XP disc, a fresh install should restore some pep to that relic....