I would generally say,
don't go lower than a GF6600
At least 1 GB of ram, but 2 is way better.
An ok cpu, don't skimp on this to mutch.(at least a 3.5GHz intel or a amd X2 4400+)
Make shure you have a good monitor, your gonna be looking at it a lot, so it better be good.

I'd echo the other two posters' comments. The graphics card is usually the most important part of hardware when it come to pure graphical programming and one of the best value for money cards at the moment is the GeForce 6600GT, it's a sweet little card and pushes along modern games fine. Go for a higher spec, such as the 7800, if you want to but if you're just learning then I would say that a 6600GT would be fine. RAM-wise, one gig is ample but more never hurts. ;) CPU I would go for a standard Athlon 64 with a Venice or San Diego core as these chips have SSE3.

As Erkokite says though, your poor performance could easily be a programming error or oversight, I wouldn't just go out and buy a new computer to make it run better unless you're absolutely sure that that is what the bottleneck is.

Best of luck!

Avoid getting anything lower than X1300 for ATi. They don't support PixelShader3 below that and it would be much less funny since there are no loops for you. HDR rendering seems to be much (proportionally) slower on ATi and I noticed perceived performance of blending on NV4x to be rather fast.

The problem is that X800 GTO2, although more expensive than 6600GT provides almost 2x the performance but without half-float blending and PS3.0, I don't find it very attractive, although I understand it's hard to say goodbye to this amount of performance.

Unluckly, everything over X1300 is rather expensive last time I checked and I find 6600GT to provide very good value. Don't even think at getting high-performance cards. They're too expensive for experiments, they're feature-wise aligned to 6600GT (unless you consider transparency AA to be a real bomb) and they'll be killed as always by next generation.

I suggest to take your money up to the next generation. Dynamic streams, geometry shaders, multiple render target to the next degree, unified memory resource management and such are something really smart every serious hobbist should really look for. It'll take some time before they ship, in the meanwhile take 6600GT and enjoy the rest of the money with your friends! Or keep it in your pocket or begin buying a pico-hydro system to power your PC if you have the water. ;)