The freeware program, Prio, has a feature that can save set Processor Affinities per executable file. Try it if you don't want to create. BAT files: http://www.prnwatch.com/prio.html
I have 30fps nearly everywhere, and 60 fps in the sky at day. This netbook.
All I do is set the terraria.exe process to high/realtime whenever I start it. You can automate that with a tool called Prio. Free and lightweight.
EDIT: The only person I've heard having trouble with it had a 700mhz processor, and I'm fairly certain he wasn't even on XP/win7/vista
He got around 5 fps.
Another thing that might possibly help, even a little, is running the game at above normal or high priority. If it does, install Prio which lets you set a programs priority and have it remembered and automatically set for you whenever it's run again.
This software can set affinity to one processor: Prio
I doubt this is the problem, though (but it won't hurt to try) as I am running Oblivion without issues on a multi-core processor (AMD Athlon II) and has even modified Oblvion's .ini to use more threads.
Already tried this. I actually have a cool program called prio which allows you to set the priority and it will save it so you wont have to constantly change it. I have the client set to below normal and the game itself set to high. Small FPS boost but I'm still getting these huge drops.