First, they're making this assumption:
"A political claim is false if it is inconsistent with the conclusions of people holding relevant expertise, including journalists, scientists, and eyewitnesses. "
If you're making the assumption that Journalists are the arbiters of truth, then yes truth is going to lean left. If you have EVER had journalists report on something you actually know a lot about, you may be more sceptical of this.
Second, they used pic related dataset:
Third, the questions often took the form
"Forbes magazine estimates that since his election, President Trump’s golf trips have cost taxpayers over $100 million, and would total over $340 million over two terms."
While this allows the authors to rate this as "true" since Forbes magazine estimates it as "true" the actual fact is disputable. In fact if we look at Forbes methodology they're just accounting for gross cost and not accounting for what would have been spent anyways. If the question was changed to:
"Forbes magazine estimates that since his election, President Trump’s golf trips have cost taxpayers over $110 million, and would total over $350 million over two terms." it would BTFO democrats extremely hard because they're more susceptible to being credulous if the esteemed Forbes Magazine was claimed to have said something.
Finally this question selection mechanism isn't actually neutral:
"Over a 6-month period, spanning January through June 2019, we used a social media monitoring service to identify 20 of the most viral political news stories, 10 true and 10 false, every 2 weeks. This provided a systematic method for selecting high-profile claims, balanced by veracity, and ensuring that ourbelief accuracy measures are not unduly swayed by idiosyncratic judgments about topic importance."
Ds are heavier followers of political news and more interested in it, and most journalists are Ds, so the idea this is going to generate a neutral dataset requires brain cancer.
