I know this might just be another tool you don't have access too, but These seemed to work the best for cleaning up the edges. Go slow, and use a drill press if possible so its completely vertical. Good Luck!
This is what you want, a deburring tool.