Just an FYI to everyone downvoting, The Lazy Dungeon Master is actually a real book outlining a novel approach to planning a D&D campaign based on preparing a platform for improv rather than planning out every encounter and the whole story in advance, rather than just being a sarky comment.
All very good advice!
As I DM more and become more comfortable in the role with an ongoing campaign, I find that I use my "detailed" notes less and less. In the beginning, I did kind of a brain dump to build up the framework of the story but I found that I haven't open up my Google Docs link in a couple of sessions. I have been predominantly sticking with 1-2 pages of handwritten maps, plot points, and names and using those to guide the session. It's a whole lot less stressful than trying to refer back to my notes over and over again.
I highly recommend the book The Lazy Dungeon Master by Mike Shea (written for 4e but very little is edition specific...I know there is a new version out but I haven't gotten around to reading it yet). Following the advice in that book greatly helped me mellow out and cut down on my overprep.
If you are interested, check out the preview pages, for example on amazon
The main advice I got from it is spending preparation time on the most important/immediate parts of your adventure while staying flexible (consider 3 possible paths the adventure could continue on). There's some more advice on how to improve your game or make your life as a DM easier, often giving simple tools for it (at least simple steps you could follow to achieve it). The table of contents (shown in the preview on amazon) should give you a good idea on what's included.
Yes and Sly Flourish spends a great deal of time talking about this in his excellent book, The Lazy Dungeon Master.
The problem is if I spend 5 hours designing something that I love dearly and want to spring on my players, by the gods, they are going to go through it and ... that is railroading. Sly talks about how to spend those 5 hours smartly and prep for adventures that are flexible for you and the players and comfortable and enjoyable for you.
It doesn't have to be. Purchase Sly Flourish's book The Lazy Dungeon Master and you will see the benefits of being able to have a better game with less prep. Which is also the principles behind /r/DungeonWorld GMing as well.
Don't take my word for it. About 1/3 of the answers here to DM questions recommend this book.
Good advice here in this thread. Earlier this year I combined two separate campaigns / groups into one. I'll skip my experiences / philosophy and jump straight into what's working.
What's Working
>... we also wanted to craft our own completely unique story and make up characters rather than use premade ones.
Yea, that is fine and start working on them but you might find it less stressful and more helpful to at least play through the LMoP. That way you can learn the ins and outs of being a DM and they can make mistakes that will get A character killed, not THEIR character...yet
> My main disconnects with the rules and mechanics is all of the numbers and stats and when to roll dice and what dice to roll.
You need to do two things to learn this cold.
>How do I determine what weapons my adventurers will find and make sure they makes sense and the players know how to use them?
You have to have some notes on your PCs, passive check numbers primary weapons, etc. Basically all my players make sure I have a copy of their character sheet before we play and while I'm writing adventures.
>How do I populate my world with NPCs and enemies and determine their stats and weapons while keeping things balanced?
The DMG will help you with building balanced encounters and what level of magic items are appropriate but you won't advance in DM level until you screw this up some...we all have.
>What do you do when a PC says he wants to stab a random bartender and you weren't prepared for them to fight?
Have a statblock written down nearby for a Commoner - or just give them an hp:5, ac:12 or something like that. Most people in the "world" aren't anything special. The ones that are special, you will have found or written StatBlocks for and will be prepared.
>Also, In our story their may be more human enemies than monsters, but lots of resources only list monster stats.
In the MM and DM Basic Rules, there are more than just the Commoner, there is guard, soldier, mage, etc. You can start with those and then change them as necessary when you get a little more experience under your belt.
You can also use the great stuff by /u/beedrill330 on DriveThruRPG
The best help I've had are from Matthew Colville's Running the Game playlist, which has been incredibly helpful, and also Sly Flourish's Lazy Dungeon Master
But don't feel like you have to listen to all the videos or read/listen to the whole book. Just make sure you prepare enough for the first session which is usually 3 encounters or so and some RP scenes like introducing the player characters. Also, remember the phrase "What do you do?", it's a good prompt to show that you want player input. I like to print all of the monster stat blocks that I could need for that session into one or two convenient pages and have the notes or pages of the adventure I'm running in a binder. Once you have one session you'll have experience to build and reflect on. As long as you keep trying to get better, it's hard to be terrible.
Check out "The Lazy Dungeon Master". It's a quick read, but has a lot of good tips for DM'ing on the quick, with minimal prep. Also, if you haven't already, watch Chris Perkins do some Aquisitions Inc. DM'ing. He's awesome at a very free-flowing DM style, and essentially became my role-model for my games.
Yes, rules are important, but don't let that get in the way of everyone having fun (fun is the primary consideration for both the players and the DM). On the other hand, you need to know the rules of the game in order to break free from them, while still maintaining structure and consistency in the gaming world.
Railroading results from planning one outcome for your session. You can reduce railroading by planning multiple storylines for your adventurers to wander into. Another method is to plan less, since the time you invest into a plan makes you want to use that plan even if your players have other ideas. Or you could just be more willing to abandon plans for improvisation. Improvisation lets the story unfold in response to your player's choices, which is the opposite of railroading. You may be concerned that your improvised story won't be as fun as a prepared one. You just just need to build confidence in your ability to improvise by learning and practicing improv principles at D&D games. D&D is in large part improv theatre but the performers sit around a table with dice.
Three short but great books on this topic:
Unframed: The Art of Improvisation for Game Masters
Play Unsafe: How Improvisation Can Change the Way You Roleplay
I recommend you read the book The Lazy DM. It's relatively short, should only take you a couple hours to read but it'll cut your prep time so much
I'm working on my own campaign now and found this book to be helpful: http://www.amazon.com/dp/B00ADV2H8O/ref=pe_385040_118058080_TE_M1T1DP