I highly recommend Cryptography Decrypted. It presents crypto concepts in a very easy to read format. NOT a programming book, and not heavy on Maths but you'll learn a ton of core concepts especially public key crypto.
New to cryptography? Cryptography Decrypted by H. X. Mel was a great book that introduced me to the concepts and took me up to the basics of SSL/Public Key type stuff. I read it over a weekend because it was easy to read. IRC, it doesn't get into the math too much but instead explains the concepts in plain language and asks you to make a leap of faith that certain things work. This is a shortcut for getting a grasp on practical knowledge that will help you as a network dude.
The chapters are normally available as pdf at the authors site, but the links seem to be broken: http://docrypt.com/
You can get a used copy for about $20 via amazon prime: https://www.amazon.com/Cryptography-Decrypted-H-X-Mel/dp/0201616475/