Who decides what AI can tell you? Campbell Brown, Meta’s former head of news, has some ideas

Campbell Brown has been working on his career in search of accurate information, first as a popular TV journalist, then the first on Facebook, and the only one, a dedicated news executive. Now, as he watches AI reshape the way people consume information, he sees history threatening to repeat itself. This time, he is not waiting for someone else to fix it.

his company, AI Forum – which he recently discussed with TechCrunch’s Tim Fernholz at the StrictlyVC evening in San Francisco – examines how basic models work on what he calls “big topics” – geopolitics, health, finance, hiring – topics where “there are no clear yes-or-no answers, where they are complex and difficult.”

The idea is to find the world’s leading experts, have them build benchmarks, and then train AI judges to evaluate models on a large scale. For the work of Forum AI on geopolitics, Mr. Brown recruited Niall Ferguson, Fareed Zakaria, former Secretary of State Tony Blinken, former Speaker of the House, Kevin McCarthy, and Anne Neuberger, who led cybersecurity in the Obama administration. The goal is to get AI judges to agree about 90% with human experts, which Forum AI says it has been able to achieve.

Brown traces the direction of Forum AI, which was founded 17 months ago in New York, to real time. He said: “I was in Meta when ChatGPT was released publicly for the first time, and I remember that it wasn’t long before I realized that this is the way that all messages go. The results of his children made that time important. “My children will be speechless if we don’t know how to fix this,” he remembers thinking.

What really disappointed him was that accuracy didn’t seem to be a priority for anyone. Foundation model companies, he said, “focus on writing and math,” while stories and information are difficult. But strength, he argued, does not mean choice.

Indeed, when Forum AI began evaluating advanced models, the findings were not exactly encouraging. He cited Gemini from the Chinese Communist Party’s website “on issues not related to China,” and attributed left-leaning prejudice to almost all ethnicities. Subtle failures also abound, he said, including missing content, missing ideas, grassroots arguments without consensus. “There’s a long way to go,” he said, “but I also think there are some simple changes that can lead to better results.”

Brown spent years at Facebook looking at what happens when the platform gets things wrong. “We failed at most things we tried,” Fernholz said. There is no program to evaluate their creations. The lesson, even though social media is still out there, is that dating optimization has become difficult for people and leaves many clueless.

His hope is that AI can solve this problem. “At present it may go either way,” said he; companies can give users what they want, or “they can give people content that is real and authentic and authentic.” He admitted that the ideal of this — AI optimization for reality — might sound like nonsense. But he thinks that business can be a partnership here. Businesses that use AI for credit decisions, lending, insurance, and lending services related to credit, and “need you to scale for success.”

Business demand is also where Forum AI is betting its business, although converting interest into fixed income remains a challenge, especially since the existing market is satisfied with box surveys and benchmarks that Brown considers insufficient.

The audio format, he said, “is a joke.” After New York City passed the first biased ordinance requiring AI inspections, the state inspector general found that more than half had violations that went undetected. A real investigation, he said, requires domain expertise to use not only known cases but cases that “can get you into trouble that people don’t think about.” And that work takes time. “Smart generalists won’t cut it.”

Brown – whose company last fall was promoted $3 million led by Lerer Hippeau – has a unique opportunity to explain the disconnect between the AI ​​industry’s view and reality for the general public. “You hear from the leaders of the big tech companies, ‘This technology is going to change the world,’ ‘it’s going to put you out of a job,’ it’s going to cure cancer,” he said. “But for the average person who’s just using a chatbot to ask basic questions, they’re still getting low-quality and inaccurate answers.”

Belief in AI is at an all-time low, and he thinks skepticism is often justified. “The conversation is happening in Silicon Valley around one product, and a different conversation is happening among consumers.”