Guide Labs releases a new version of the LLM definition

The problem with the deep learning method is often understanding why it does what it does: Whether it’s time to fight xAI over and over again to fix Grok’s odd politics, ChatGPT’s fight with sycophancy, or running the mill, dialing through a neural network with billions of parameters is not easy.

Guide Labs, a San Francisco startup founded by CEO Julius Adebayo and chief scientific officer Aya Abdelsalam Ismail, is offering a solution to this problem today. On Monday, the company opened an 8 billion parameter LLM, Control-8Btrained with new constructions designed to make its actions easier to interpret: Each symbol created by this model can be traced back to its origin in the LLM studies.

This can be as simple as knowing the things mentioned by the model, or as complex as understanding the model’s understanding of humor or gender.

“If I have a trillion ways to write gender, and I put it in 1 billion of the 1 trillion things I have, you have to make sure you get all 1 billion things I’ve written, then you have to turn it on, turn it off,” Adebayo told TechCrunch. “You can do it with modern models, but it’s fragile… It’s one of the sacred questions.”

Adebayo began this project while earning his PhD at MIT, co-authoring the widely cited publication 2020 paper which showed that existing methods for understanding deep learning models were not reliable. This work eventually led to a new way to create LLMs: Developers put a logic component into a model that classifies buckets into groups that can be tracked. This requires a lot of foreshadowing, but by using other types of AI to help, they were able to train this model as their biggest proof yet.

“The kind of interpretation that people do is … science about the model, and we turn it around,” Adebayo said. “What we do is develop a model from the ground up so you don’t have to do neuroscience.”

One concern with this approach is that it may eliminate some of the things that make LLMs so interesting: The ability to explain in new ways that have never been studied before. Adebayo says that this is still happening in his company’s model: His team is following what he calls “known ideas” that the model discovered on its own, like quantum computing.

Adebayo argues that this translation infrastructure will be what everyone needs. For consumer-facing LLMs, these methods should allow model builders to do things like restrict the use of copyrighted material, or better manage content on topics like violence or drug use. Regulated industries will require LLMs that can be regulated, for example in finance, where the screening model for lenders should consider things like financial history but not quality. There is also a need for interpretation in scientific applications, another area in which Guide Labs has developed technical expertise. Protein folding has been a great success in deep learning, but scientists need more information about why their programs are designed to integrate well.

“This model shows that interpretive studies is no longer a science; it is now a technical problem,” Adebayo said. “We figured out the science and we can scale it up, and there’s no reason why that kind of thing isn’t comparable to borderline functionality,” which has many other parameters.

Guide Labs claims that the Steerling-8B can achieve 90% of the capabilities of existing models, but uses more training, thanks to its new design. The next step for the company, which emerged from Y Combinator and raised $9 million in seed funding from Initialized Capital in November 2024, is to create a larger model and start providing APIs and access to users.

“The way we treat the education community is very old, so the definition of democracy will be a long-term good thing for our people,” Adebayo told TechCrunch. “When we’re going after these species that are going to be very intelligent, you don’t want something making decisions that don’t make sense to you.”