Linux has a VM manager built directly into the kernel called kvm. I use QEMU with it and GPU passthrough to play videogames using a windows vm with passthrough on my Fedora computer.
It might be easier to just do that instead of ESXi.