I've been talking to these guys about moving their crazy open source helmet personal assistant to being bone conduction. It's pretty ridiculous but the helmet project is pretty far along and open sourced over here.
To take a step back, this software project was designed to integrate into a motorcycle helmet, where isolation between the microphone and headphones should be guaranteed. In that environment, it'd be fine for it to always listen, since it can't hear itself. A "shut up" feature works fine there.
However, outside of that specialized hardware environment, it's a very bad idea to have the recognizer active while speech is being generated. The vast majority of home automation speech input systems do not require the user to have their microphone and speaker isolated through a headset.