The Church of Alvis

My box is a few years old now.

Its a Dual Core AMD athlon 64bit 3800+

I'm considering upgrading it with WIndow7 64bit so I can increase the ram to 6 or 8gb.
I also have an old Nvidia 7600Gt card that I'll probably upgrade to an AMD 6850.

Is it worth upgrading the CPU? Will I get a lot of bang for the buck? Right now I'm looking at about $350 for the video/ram/OS upgrade. How much more will I get out of a new CPU these days?

In my opinion, I think you need to upgrade that processor. It was new 5 years ago and has done well but it is deserving of retirement.
In your shoes I would look for a CPU/motherboard combo with 4GB RAM and a videocard. Stick with Win7 x86 for the time being. I think you will get the bigger improvement that way.

When you have the cash, I would then upgrade to Win7 x64 and add another 2 GB of RAM.

Of course you could contact Jimer. Win7 Home Premium is $30 at the MS company store.

The more important question is what socket the CPU goes in...

If it's Socket 939, you'll need a new motherboard and RAM as well as CPU. Socket AM2 you're a little better off, depending on chipset you can go up to some nicely priced upgrades without worrying about the board and then just add some RAM. A faster Dual core, or a triple core will help, quad cores tend to be more than most games need at this point, but the problem is actually finding them... supply has dried up recently and they're hard to find.

I'd look at something like this for $280 (not including OS or video card):

AMD Phenom II X4 840 3.2GHz = $110
Gigabyte GA-MA770T-UD3 770 ATX = $70
Kingston HyperX 8GB (4 x 2GB) DDR3 1333 = $100

I LOVE Gigabyte boards and my main system is running one and have built several high-spec systems with them. The board doesn't have USB3 or SATA-600, but for another $40 or so you can get a board with those options as well (I probably would at this point, especially if you're considering a SSD in the future).

I'm also in the market for a new video card sine my 8800GT is apparently ancient I'm looking at a 2GB card in the $250-$300 range (460GTX or 6850) and will probably upgrade my PSU at the same time (looking at an Antec 750w) since my warranty-claim OCZ 600w doesn't want to POST my board and the 500w I've got in there now probably won't cut it with the 4 HDs and Q9550/8GB DDR once I add the new video card.