I suggest reading "From Babylom to Timbuktu" by Rudolph Windsor. He documents everything and lists sources. It is the best book on the topic.
Also read "The Thirteenth Tribe" by Arthur Koestler. Between these two books, your questions will be answered.
Their main source is the bible, specifically the curses outlined in Deut 28. The BHI claim that the chapter outlined a prophecy of what would be the trans-atlantic slave trade.
Another popular source is a book that came out in the 80s called: From Babylon to Timbuktu
https://www.amazon.com/Babylon-Timbuktu-History-Ancient-Including/dp/0962088110
A different explanation I've heard is that after the Babylonian exile, some of the tribes relocated to West Africa where their descendants became slaves in the New World.