CASE FILE / Robotics
Built an SO-101 pick and place simulation using GPT-6 Astra and @typesaf…
Built an SO-101 pick and place simulation using GPT-6 Astra and @typesafeai Jev. Astra reads overhead and wrist camera images and produces structured observations and Jev returns typed decisions with probability scores. A separate local controller carries out the movements and checks that the arm stays within its limits and is ready for each step. The models never directly control the motors. A fixed pick and place sequence probably doesnt need this much decision making. I m using it as a testbed for situations where observations are uncertain and the system needs to decide whether to act wait or stop.
Original post
Built an SO-101 pick and place simulation using GPT-6 Astra and @typesafeai Jev. Astra reads overhead and wrist camera images and produces structured observations and Jev returns typed decisions with probability scores. A separate local controller carries out the movements and checks that the arm stays within its limits and is ready for each step. The models never directly control the motors. A fixed pick and place sequence probably doesnt need this much decision making. I m using it as a testbed for situations where observations are uncertain and the system needs to decide whether to act wait or stop.
Chinese translation
使用GPT-6 Astra和@typesafeai Jev构建SO-101拾取和放置模拟。 Astra读取头顶和手腕摄像机图像并产生结构化观察，Jev返回带有概率分数的输入决策。 单独的本地控制器执行运动并检查手臂是否保持在其极限内并为每一步做好准备。这些模型从不直接控制电机。 固定的拾取和放置顺序可能不需要这么多的决策。我将其用作观察不确定且系统需要决定是等待还是停止的情况的测试平台。
What this case shows
- Use JEV for fast discrete calls on robot state
- Threshold probability to act or halt
- See the original post for the sim or hardware demo