I am doing exactly this. I decided I wanted to learn my heritage language, which I knew so very little of previously, to a very high degree and speak it only with my child. I decided this right before we found out we were pregnant, and have been not only studying, listening, practicing, finding people to speak with, watching, listening etc., but also hording interesting things for babies to do like shows and books to read to them. It will be VERY important for them to hear the language outside of just you, so find local playdates with other native speakers.
For me, it will be very important to not start it later, even at 3, but from birth. Even in the womb I am often speaking to her. I will ONLY speak the language with her, never reverting to English (well, I'm sure on occasion it will just make more sense). See OPOL (one parent, one language). This book has also been helpful: http://www.amazon.com/gp/product/B005MJC89K?psc=1&redirect=true&ref_=oh_aui_d_detailpage_o02_. I remember seeing a video of Richard Simcott at one of the latest polyglot gatherings where he answered a question about raising his daughter multilingual.
TLDR; don't be put off on trying this, but expect to put lots of time in getting it right. You'd be surprised how proficient you can become in 6-7 months, and with enough support, your child can be fully bilingual.
This book might be helpful. I enjoyed reading it