CASE FILE / Model & agent routing
Played around with @typesafeai Jev today, mostly to understand what it d…
Played around with @typesafeai Jev today, mostly to understand what it does for tool calling. Instead of an LLM deciding what to do, I substituted that part with Jev. My learning is that we will be able to make the agent act before the user finishes the sentence. So far, we've used different LLM combinations (non-thinking + thinking ) plus state machine setups to get the right experience for the end user. TLDR is to understand more about how voice agents can run decisions on partials instead of waiting for the end of the turn, with lower costs.
Original post
Played around with @typesafeai Jev today, mostly to understand what it does for tool calling. Instead of an LLM deciding what to do, I substituted that part with Jev. My learning is that we will be able to make the agent act before the user finishes the sentence. So far, we've used different LLM combinations (non-thinking + thinking ) plus state machine setups to get the right experience for the end user. TLDR is to understand more about how voice agents can run decisions on partials instead of waiting for the end of the turn, with lower costs.
Chinese translation
今天玩了一下@typesafeai Jev，主要是为了了解它对工具调用的作用。 我没有让法学硕士决定该做什么，而是用Jev代替了这部分。 我的了解是，我们将能够让代理在用户完成句子之前采取行动。到目前为止，我们已经使用不同的LLM组合（非思考+思考）加上状态机设置，为最终用户获得正确的体验。 TLDR是为了更多地了解语音代理如何对部分音进行决策，而不是等待回合结束，以更低的成本。
What this case shows
- Let JEV choose the model or tool first
- Auto-route on high confidence, escalate otherwise
- This is the most common production pattern