Johns Hopkins and Stanford researchers have achieved a major breakthrough in robotic surgery by teaching a surgical robot to perform basic tasks simply by watching videos of human surgeons at work.
Progress in Robotic Skills
The team successfully programmed the da Vinci Surgical System to execute three essential surgical functions: manipulating needles, lifting tissues, and stitching wounds. Impressively, it matches the skill level of seasoned human surgeons. This marks a significant advancement toward fully autonomous robotic surgery.
Dr. Axel Krieger, an assistant professor in JHU's Department of Mechanical Engineering and the study's senior author, remarked, "It's amazing to have this model where we just provide camera input, and it predicts the robotic movements needed for surgery." He added, "We think this is a big step forward in medical robotics."
Innovative Learning Techniques
To achieve this, the researchers combined imitation learning with a framework similar to the one employed in ChatGPT. However, their model focuses on breaking down surgical movements mathematically rather than on words, translating these movements into precise robotic actions.
For training the robot, they used hundreds of surgical videos filmed from wrist-mounted cameras on da Vinci robots during real operations. This substantial dataset was sourced from nearly 7,000 da Vinci systems globally, utilized by over 50,000 trained surgeons.
Enhancing Precision and Adaptability
Although the da Vinci system is widely recognized, it had some precision issues. The research team addressed this by programming the robot to perform actions relative to its environment instead of relying solely on fixed movements, leading to a remarkable improvement.
One of the most astonishing aspects is the system's adaptability. "The model is really great at picking up things we haven't explicitly taught it," said Ji Woong "Brian" Kim, the lead author and a postdoc at Johns Hopkins. "For instance, if it drops a needle, it automatically retrieves it and continues. That’s not something I explicitly trained it to do."
Future of Surgical Robotics
This progress means there is no longer a need to painstakingly program every single movement for a surgical task, a process that used to require years for just one procedure. With this new method, the robot can learn an entire procedure in just a few days by merely watching.
The researchers showcased their results at the Conference on Robot Learning in Munich, highlighting the convergence of AI and medical technology in the realm of surgical robotics.
Source: Link
