Sedra and Smith is a great textbook if that is not what you are using already.
What part of Electronics I was the most troublesome? Large signal analysis? Small signal analysis?
You have to learn the concepts of digital signal processing. There are alot of really good books out there on amazon. You can choose to implement these algorithms in many ways, in C++ using various audio libraries, in CUDA/OCL on a GPU or even in hardware level via Verilog/VHDL on an FPGA.
If you want to deal with raw circuitry, you need to learn analog signal processing, you might have to learn the concept of analog electronics. BJT's/OpAmplifiers/ClassA/AB/D etc. I recommend this book, it is fantastic. http://www.amazon.com/gp/product/0195323033/
I recommend reading up the concepts of DSP first. Discrete Fourier Transforms, frequency manipulation such as low/high/band pass filtering. All the various concepts basically. This coursera site shows most important concepts. https://www.coursera.org/course/dsp
Also, unless you are working on an FPGA, there is not much point testing/starting out on an embedded system. Try it out on your PC/Mac first