I'm not even kidding when I say; get a Gakken SX-150 Mark II. $38 bucks on Amazon, probably cheaper elsewhere:
It's sold as a toy in Japan, but it's a great way to learn the basics of subjective synthesis. By itself it sounds rather crappy, but put it through a couple of cheap guitar pedals or record to DAW and add plugins, and it's heaven. I'm not saying get it instead of the other excellent suggestions you're getting, but get it! I'm currently hooking it up to the audio-in of my Roland SP-404SX, utilizing the 404's effects, and it's blowing my little old mind!
How about that gekkokon synth or whatever. Has a ribbon band. I think it's the cheapest synth on amazon, less than 50.
Edit: searched for its called gakken http://www.amazon.com/Gakken-SX-150-MARK-Analog-Synthesizer/dp/B005Q22HV2
The Gakken SK-150 or Korg Monotron sound like just the things for you.
Instead of the PO , I recommend either the volca bass or keys for learning. They have parameters that will teach you important parts of a synthesizer. Especially if you love turning knobs, volca is the route to go. Of course you will have to up your budget to $150. PO and Volcas can be together if you eventually get both.
Now I know you were looking for something you will actually use. While Gakken's SX-150 Mark II synth provides little practicality, for $20 it is a nice little hands on learning tool. I did make mine practical though by getting a DIY kit by beatnic.jp so I can control it via a midi keyboard.
This was a busy week at work, so I only had a few hours to work on this song. I used a recently bought Gakken SX-150 Mark II synthesizer ($20 bucks at Amazon!), passing the sound through the Melda MAutoPitch VST.
Here's the song, hope you enjoy it: https://soundcloud.com/blowghoti/not-before-coffee