This happens when FaceTime HD Camera is set as a camera by other apps or processes in your system. To fix this, we need to make sure the camera is not being used by other apps before starting VCam.
Restart your system and do not run VCam yet.
Check all the apps in your system that uses the FaceTime HD Camera, and make sure to select XSplit VCam as the camera in their settings instead. Close the apps once done.
Note: Unfortunately, Mac OS does not have the option to set a default camera for all your apps to start with. This means we need to individually check all the apps
Run XSplit VCam and reselect FaceTime HD Camera as your device.