OpenAI’s long-awaited o1 model series made headlines for its advanced ability to reason, a significant step forward in artificial intelligence.
The reasoning abilities of o1 aren’t dissimilar from a feature familiar to Botpress users – the Autonomous Node. Launched in July 2024, the Autonomous Node was the first of its kind among AI agent platforms.
Sylvain Perron, CEO at Botpress and the chief architect behind the platform’s Autonomous Nodes, sat down to explain the similarities and the differences between the two reasoning entities.
Botpress CEO Sylvain Perron compares OpenAI o1 and the Autonomous Node
There are some major similarities between OpenAI’s new o1 model and the Botpress Autonomous Nodes, correct?
There are, definitely. It was very funny to see o1 come out. But a big thing to note is that we’re not quite comparing apples with apples here.
OpenAI o1 is, obviously, an LLM, while Autonomous Nodes are a platform feature. You can even plug o1 into Botpress and use it with your Autonomous Nodes.
But both of them use chain-of-thought reasoning, a bottom-up approach to reach conclusions about what to do, and what to say.
Could you explain chain-of-thought reasoning and how it differs from the way language models typically generate responses?
Of course – most LLMs generate responses by predicting the next word based on the previous one – they generate the first word, then use that to generate the next, and on and on. This works for answering straightforward questions, but it fails wildly in more complex tasks with multi-step reasoning.
Chain-of-thought reasoning, on the other hand, is when an AI breaks a task down into smaller steps and explicitly reasons through each one before proceeding to the next. It’s similar to how students are asked to build up to an answer in a STEM exam problem.
For example, if an AI is creating a lead in a CRM – where it needs to check for duplicates, assign the lead, and so on – it needs to use chain-of-thought reasoning to think about each step, and plan a full sequence of actions before it begins executing. It doesn’t start filling in a form before it’s realized it’s a duplicate, it knows before it begins.
So the system can handle more complex tasks with multiple dependencies, but it also slows it down because it has to pause and think, rather than just generating immediately.
Both o1 and Autonomous Nodes use chain-of-thought reasoning – is it the same process in both, or are there differences between the two?
There are some similarities and some differences. We still use chain-of-thought reasoning, but because we’re building for real-world clients, we optimize for speed and efficiency in a few different ways.
With Autonomous Nodes, we designed the system to handle multiple actions in a single LLM call. The idea is to think and act within the same process, so we don't lose time.
Autonomous Nodes think and give updates to the user so that the user is not simply waiting for its thought process to be over. This is more similar to natural conversations, where a person will not be left waiting in suspense for 30+ seconds. Developers can look into the hidden thoughts of the LLM and troubleshoot what went wrong.
On the other hand, o1 returns one answer, and there is limited visibility into its thought process. OpenAI provides a summary of the chain-of-thought without showing it, making it more tricky to troubleshoot.
Are there other differences between the two?
o1 requires a developer or a platform to host functions that call tool calls. Every step requires a separate API call, meaning the model has to request one action, wait for the result, and then call for the next. And you need to call this recursively again and again and again. This makes the barrier to use it higher, as you must provide the underlying function running architecture.
Autonomous Nodes do this automatically in the background.
How does o1 compare on cost and speed?
Let’s say you’re using an LLM or AI agent to assist with Hubspot tasks, like chatting with website visitors as a way to do AI lead generation.
If you asked o1 to fill in a lead form on Hubspot – I mean, assuming you could connect it without a third-party connector – it would take 12 seconds and cost 5x more than if you did it on Botpress.
Part of the speed comes from our platform’s schema-based design, which means the system knows upfront what data is needed and what the structure of that data looks like.
So when you’re working with a tool like HubSpot or Salesforce, the system already has a predefined JSON schema that tells it what inputs are required – things like first name, email address, company name, etc.
This allows Autonomous Nodes to reason through the entire task from the beginning, without having to stop and ask for more information at each step.
You can reduce the speed and the steps by providing context yourself, but at the cost of spending more time building an o1-based application. So that’s also a matter of increasing maintenance cost.
You mention heightened reliability. What makes Autonomous Nodes more reliable than o1?
The biggest way we made the output more reliable was by mixing chain of thought reasoning with the protocol tool calling we created.
What we did was essentially create an environment where it can think – and all in code, not plain text. Honestly it’s faster and more reliable than the OpenAI version, for a few reasons.
One is that it can talk simultaneously, throughout the sequence of actions. That’s something the OpenAI model doesn’t do.
Another reason is that the AI knows about the output schema. It knows what type of JSON it should be spitting out based on the query it receives.
And it’s something that developers can tune themselves. If they’re using this, and they’re getting a ton of similar questions from users, like most are, they can tune the AI so that it gives the best and most appropriate answer every time.
It uses JSON for how much of the process?
Plain English is just unreliable, right? It’s not formal enough. It’s not strict enough.
Programming language is a language, but it’s formal. It’s a domain-specific language; you can only use it within certain boundaries and syntax roles.
So our tool calls reply in JSON instead of English. It’s JSON output with a JSON schema instead of plain text.
When you make a request to create a lead form in Hubspot, it is written in JSON, and the schema – also in JSON – gives all the properties it needs to include, like name, company, etc.
What are the major differences between tool calling between Autonomous Nodes and the o1 model?
Our tool calling is highly context-aware. It understands all your systems, all the actions it can take, and how that data can be fed into the next tool. And we can generate one code block that gives all of this together, plus gives an answer, all in one LLM call.
In theory, you can use the o1 API to do tool calling, but there are constraints to what you can call the tool with. But Botpress is built for it. We have guardrails on top of other LLMs, including GPT.
Autonomous Nodes can also talk simultaneously while calling tools, which is not something that OpenAI currently supports. This saves a round-trip to the server and gives a better conversational UX, as users are informed before a long-running task is started.
Most organizations are warned against using ChatGPT with secure work data. Are there less concerns for Autonomous Nodes?
Our platform is built for high-volume, low-latency environments, and we’ve engineered it with the practical needs of businesses in mind.
The advantage of Autonomous Nodes isn’t that we’ve created a completely new type of AI – it’s that we’ve taken existing technology and applied smart engineering to make it work better for business needs.
We have secure sandboxes for AI generating communication. When you use an Autonomous Node, it runs these secret sandboxes for free. They’re secure, they’re scalable. And afterwards, the sandbox is destroyed.
This is a virtual isolation with two layers – checking in and checking out. It's pretty complicated. But it means we can run large scale LLM-generated code with minimal security risks.
If developers or AI enthusiasts want to try out the Autonomous Node, what do they have to do?
We have a generous free-tier. All of our users can try them out. We thought it was too cool of a feature to gate. So yeah, just make a free Botpress account and you can see for yourself.
Perfect. Thanks so much for sitting down to connect the dots between the two features. o1 is making a lot of headlines; it’s great to better understand how it links to what Botpress is developing.
