If you want to comply with CC-BY-SA then at least the parts that are synced with the music should be CC-BY-SA, see https://creativecommons.org/licenses/by-sa/4.0/legalcode#s1a (previous versions from 2.0 have the same language).
You could still use CC0 to relinquish any privileges you have for parts not synced with the music, even the entire video footage so that others can use different audio with no restrictions.
You could also ask the music licensor to release that track you're interested in using under CC0. Or find a different track...