'You guys weren't gonna fact-check, and since you're fact-checking me I think it's important to say what’s actually going on,' Vance said.