C++ is what you use. There's a library called Juce that takes care of a lot of the common stuff and is cross-platform: http://www.juce.com/about-juce
Here's a great site to get you started: http://www.redwoodaudio.net/Tutorials/juce_for_vst_development__intro.html
As far as the signal processing, take a look at this: https://ccrma.stanford.edu/software/stk/
Programming audio is not for the faint of heart.
There are tons of audio libraries out there. For example Juce or CoreAudio on OSX. Both well documented.
This is a good book for starting out with CoreAudio.
I worked for many years in DAW plugins and DSP. C++ is definitely going to be your language of choice: many of the vendor APIs for plugins (VST and Avid, at least) are in C++, so there'll be a minimum of C++ to write the plugin base.
That being said, I strongly recommend JUCE.
JUCE is a multi-platform C++ toolkit made for audio software. JUCE gives you all the classes, template projects and boilerplate code to start writing plugins and standalone audio apps right away. I understand learning a new toolkit might be a little off-putting, but I swear it'll be easier than learning each specific vendor API ; which tend to be extremely hard to understand and poorly documented. JUCE does all the menial work for you: write some DSP code, compile different project templates, and voilà.
Try JUCE!
I've been using it for all of my projects for a while now, its honestly the best app development framework I've ever come across (in the C/C++/C#/Objective-C world). Plus you can write the app once and deploy it on any device (including Android).
I think its a good idea especially if you want to get into games development. I believe you'll have to use the Eclipse ADT bundle, since Android Studio doesn't support the NDK (native development kit).
There's a C++ toolkit called Marmalade SDK which produces some great games. (n.b. their website is throwing errors for me right now).
There's also a toolkit called JUCE which supports Android - its meant to be a really clean ground-up cross platform C++ toolkit useful for all sorts of software.
If you write C++, you should try the JUCE framework which was written by a Brit and consequently has its class names spelled correctly. It's also amazing if you want to write cross-platform GUIs (better than Qt in my opinion) or apps dealing with audio.
I don't know anything about DSP hardware, either. But I found that many people have built Raspberry Pi and Arduino-based audio devices. For instance the Mutable Instrument Shruthi is built using the same chip as Arduino. The community around these devices will also lead to instructions for building nice boxes with knobs. On the Raspberry Pi side, I found that you can run the JUCE Audio Framework on it. You may already be familiar with JUCE, but I'd like to point out that it's a very well laid out C++ library for audio and MIDI programming (among other things).
The biggest issue with these devices vs. a DSP solution is probably going to be latency. But I never actually looked into it all that much.
Google are holding a workshop during the JUCE Summit in London on audio latency, details here:
The workshop slides should be posted online after the event, should be interesting reading.
Unfortunately all this comes far too late for me, I was heavily involved in the VST scene years ago but have since moved on to other projects.
However if any other developers are interested in this sort of thing I do thoroughly recommend checking out the JUCE framework as a starting point.
I usually use Xcode as my main IDE for programming audio tools now. Used to use vim text editor and the port audio library when I started out but not so much anymore. You'll find there's a lot of libraries/frameworks out there to help make it easier for you to build audio software, it's only a matter of choosing which one you like. Are you using mac or windows?
Like I said, if you want to get your foot in the door, go to http://www.juce.com and follow the directions to download and build the example projects. They have example projects and they are generally easy to analyze the code if you have had enough general coding experience. I'm sure it'll be generally the same concept when using Propellerhead's rack developer SDK if you are just looking to build rack extensions.
Here's the solution for hard-core music programmers. I wrote a commercial digital audio application in this, and many other famous ones also use this framework.
But as someone else here pointed out, this is not for the faint of heart.
JUCE's licensing is good example case here. They release it under the GPL so you can use it GPL programs but you can pay them to give you a different license to use in proprietary software.
> I've also heard of people essentially running real time SPICE simulations but I have no idea how that works.
One method uses Modified Nodal Analysis (the same as SPICE) but the coefficients are discretised using something like the trapezoid rule to have a constant time step (unlike SPICE). This time step will be the sample rate, most likely with some oversampling (otherwise hefty errors creep in). This simulates the linear components... resistors, capacitors, inductors, sources.
Non-linear components (diodes, transistors, valves) are calculated using something like Newton's method (or a variation) to find the zero point for the current-voltage relationship. Or possibly, precalculated lookup tables can be used, but this gets hairy as soon as soon as any component's value can be varied (eg. potentiometers) as each variable adds an extra dimension to each non-linear component's lookup table.
OP, I've heard RackAFX (willpirkle.com) mentioned a few times, all positive (in fact I've been meaning to check it out properly myself).
Other frameworks to look at for writing audio plugins are WDL-OL (with this great tutorial here) or JUCE (not free for non-GPL projects).
Of course you could stick to MaxMSP. Python or MATLAB/Octave are good for prototyping DSP algorithms as well.
/u/earslap posted a good link to KVR. I'd recommend hanging out on the 'DSP and Plug-in Development' forum there if you don't already.
Juce supports MIDI, among many other things.
It's actively under development, and the developer gave a presentation on it at CppCon2014.
Right here: http://www.juce.com/
Juce is amazing! Not for the faint of heart though, it's C++ coding. Still, fairly easy to use if you are a coder, and with good documentation and an active user community.
I develop commercial VST+AU plug-ins for a living, and I use JUCE for that (http://www.juce.com/). I would recommend either that or WDL-OL. WDL-OL is open-source and free for commercial licenses, whereas JUCE has a commercial license fee.
In general, JUCE has a lovely interface library and a lot of great C++ convenience classes (i.e. it has its own Atomic variable class built-in, and many useful data structures such as OwnedArrays, which are like a vector of pointers, but you don't need to manage each pointer before deleting the entire array).
WDL-OL, meanwhile, has better support for special per-format features. As an example, we ran into a bit of trouble with our newest plug-in, as we use an external sidechain. JUCE currently does not have a way to setup this external sidechain in the preferred AU way, which means that Logic does not see our sidechain (Same goes for VST3 at the moment).
If you are working on pushing your skills towards best C/C++ practices for DSP development, I would highly recommend either/both The Audio Programming Book (http://www.amazon.com/Audio-Programming-Book-Richard-Boulanger-ebook/dp/B00835SEWK/ref=sr_1_1?ie=UTF8&qid=1396297766&sr=8-1&keywords=the+audio+programming+book), and/or Will Pirkle's Designing Audio Effect Plug-ins in C++ (http://www.amazon.com/Designing-Audio-Effect-Plug-Ins-Processing-ebook/dp/B00ABL64Z8/ref=sr_1_1?ie=UTF8&qid=1396297711&sr=8-1&keywords=will+pirkle). Pirkle's book is extremely clear, but it only covers effect development and only on Windows. The APB is much more general, but moves a lot faster and expects you to have a bit of background (which, from reading above, I would say you're good on that one). The DVD on the APB is exceptional, as it has a couple dozen more chapters on more specialized topics.
I hope that helps!
The sequencer worked out good, but took me like 3 years of dedicated work.
On iOS you have interoperability between Objective-C, C and C++, so you can call functions from different languages directly. Audio callbacks (the piece of code that is called every 6ms or so that will fill the audio buffer up wth the next chunk of audio data) have to be written in C or C++ as they are very reliable and consistent in terms of speed. This is very important in audio processing as if you miss even 1 buffer deadline you are going to hear it immediately!
You certainly wont be wasting your time learning C++ if you want to do audio programming.
Another library you could look at is JUCE. Its a cross platform Audio framework that works on iOS, Android, Mac, Windows and Linux. So write it once and it will work on all of those. There are also some great tutorials on the site which will teach you how to use JUCE but also a lot about DSP. I've never used JUCE for a project but I did all the tutorials recently and for my next big audio project I will almost definitely be using it I think. JUCE is C++.
Really good question.
I'd think of a project you'd be really interested in doing first, then learn all the techniques and tools you need to do it. We make music software so my C++ experience is all centred around this: http://www.juce.com - but someone will know what to learn for building game engines if that's what takes your fancy.
I haven't used any audio apps outside of OSX in 10 years. That said, this project has caught my eye http://pd-pulp.net/ It marries PureData with VST via Juce framework. http://www.juce.com/
For those not familiar with Juce, it is a C++ framework that is behind Tracktion DAW and Max/MSP. It has some great features including Linux support and javascript engine. I don't know if PD-Pulp has official Linux VST support, but considering that both Puredata and Juce are Linux compatible and apparently Linux VSTs are a thing, it should be possible to roll your own VST plugins using Pd-Pulp.
Hey! Since you're in a good mood... I was planning on learning DSP programming, but Reason Developer Kit seems a little complicated to get.
I started doing the basics on JUCE, am I on the right track? I'm just happy that I can finally apply the stuff I learnt in college about it, but I wish I was working directly with Reason...
It very much depends on what your existing skill set is and the nature of the plug-in you would like to create. As has already been suggested you can use visual programming environments such as Max/MSP, PD and synthmaker which lend themselves to prototyping and for those who may not have an extensive technical background. The downsides with the visual programming route are limited output formats and are not always terribly efficient.
If you can program or would like to learn then your main options are the WDL-OL framework, RackAFX and JUCE library which I believe all use C++.
If you're building a synthesizer I might suggest JUCE (http://www.juce.com/). It's very popular for making e.g. VST plugins, which is one of the standards you'd need to adhere to if you want to be able to use your synth inside of DAWs.
If you're dead-set on UE4 or Unity and don't ever want your product to integrate with other programs, I'll suggest Unity. A quick search says UE4 doesn't have what you need in terms of low-level audio support: https://forums.unrealengine.com/showthread.php?10463-Audio-roadmap-and-APIs
You'd have to integrate something like FMOD in order to get what you need in UE4, which might be cost-prohibitive depending on your proposed licensing model.
Since you code in C++ and interested in graphics learning a cross-platform framework is handy. A few to investigate are:
JUCE (http://www.juce.com/)
Proton SDK (www.protonsdk.com)
Cocos2dx (http://www.cocos2d-x.org/)
Each have their strong points. JUCE is great for GL GUIs, Proton SDK features a component / entity system, etc.
If you are interested in modern GL check out the NVidia Gameworks demos and sample source code (desktop and Android via Tegra K1): https://developer.nvidia.com/gameworks
Re: >Artificial Intelligence demonstrating simple behaviours
>small 3D graphics rotating a drawn box with OpenGL's drawpixel call
Might as well learn GLSL and the proper way of drawing 2D / 3D shapes.
OpenGL in general will keep you busy for at least 3 months regarding the initial learning curve. If you are interested in graphics though that is the thing to get a handle on also in respect to using a cross-platform framework which will take a lot of the boiler plate heavy lifting away of getting things to run on desktop and mobile.
Good luck! The journey is just beginning after school. One remains a lifelong student! :D