In A First, A Robot Listened To Spoken Instructions And Performed Surgery – Just Like A Human Would

0
27

In A First, A Robot Listened To Spoken Instructions And Performed Surgery – Just Like A Human Would

Robotic surgery has reached a new milestone after a robot successfully – and autonomously – performed gallbladder removal operations while listening and responding to voice commands. Just as a human trainee would be directed by a senior surgeon, the robot was able to act on verbal instructions after having been initially trained on videos of the procedure.

It was even able to respond to unexpected emergency scenarios, a vital skill that human surgeons must learn.

“This work represents a major leap from prior efforts because it tackles some of the fundamental barriers to deploying autonomous surgical robots in the real world,” said lead study author Ji Woong “Brian” Kim in a statement

Teaching a robot to execute a complex, multistep surgical procedure is a daunting prospect. You have to know every individual action the robot will need to take to perform the operation successfully, including things like where not to put the scalpel (pro tip: steer clear of big blood vessels), but then you also have to communicate all of that information to the robot.

“That’s what makes this so so humongously difficult, because how do you write this down? How do you write that in code?” Dr Mathias Unberath, John C. Malone Associate Professor at Johns Hopkins University, recently told IFLScience when we explored the topic in CURIOUS magazine.

“Can I watch a human learn what I’m supposed to be doing? If the answer is yes, things become, not trivial, but they’re considerably easier.”

That's called imitation learning, and it's what this latest study relied on. Supervised by Unberath’s colleague at Johns Hopkins, Professor Axel Krieger, Kim and a team of collaborators from across the United States began by training the system on videos of human surgeons performing gallbladder removal (cholecystectomy) on pig cadavers.

Last year, Krieger’s team revealed at a conference and in a preprint that they’d been able to train a robot to manipulate a surgical needle, lift sections of tissue, and suture, all with just a few seconds of learning. Gallbladder removal is a significant step up, requiring 17 individual stages of identifying tissues, making precise incisions, and placing clips. All of these come with individual challenges.

close up of surgical robot arms manipulating tissues during gallbladder removal ex vivo in a pig

During the intricate surgery, SRT-H was able to make decisions in real time and correct itself to avoid mistakes.

Image credit: Juo-Tung Chen/Johns Hopkins University

But unlike previous iterations of autonomous surgical robots, which Krieger likened to training a robot to drive a carefully charted route, Krieger explained that the latest system – Surgical Robot Transformer-Hierarchy (SRT-H) – “is like teaching a robot to navigate any road, in any condition, responding intelligently to whatever it encounters.”

Gallbladder removal, therefore, presented little issue. After watching the video tutorials, which included explanatory captions, the robot could perform the procedure with 100 percent accuracy. It may have taken longer than it would for a human surgeon, but the results were comparable. The 17 steps were executed successfully on eight different pig gallbladders ex vivo (that’s outside the body).

Because it has natural language capabilities – powered by the same kind of machine learning that underpins ChatGPT – the system can respond to verbal instructions given by a human surgeon in real time, such as “move the left arm a bit to the left”, learning from this feedback as it goes.

Even when the study team deliberately threw in a curveball, like moving the position of the robot or adding dyes to change the colors of some of the tissues, the robot wasn’t fazed.

“To me it really shows that it’s possible to perform complex surgical procedures autonomously. This is a proof of concept that it’s possible and this imitation learning framework can automate such complex procedure with such a high degree of robustness,” said Krieger.

AI-powered robots autonomously performing surgery? No longer the preserve of science fiction.

“Our work shows that AI models can be made reliable enough for surgical autonomy – something that once felt far-off but is now demonstrably viable,” said Kim.

The study is published in Science Robotics.

Search
Categories
Read More
History
15 Things Boomers Could Easily Afford That Millennials Can Only Dream Of
15 Things Boomers Could Easily Afford That Millennials Can Only Dream Of - History Collection...
By Test Blogger2 2025-07-07 06:00:09 0 123
Science
50,000-Year-Old Collagen Could Lead Us To Hippo-Sized Wombats In The Fossil Record
Earth Once Had Wombats The Size Of Hippos, And Fossilized Collagen Could Be The Key To Finding...
By test Blogger3 2025-06-03 17:00:18 0 990
Technology
The travel-ready DJI Mini drone bundle has dropped to a record-low price during Prime Day
Best Prime Day drone deal: Save 21% on the DJI Mini 4K bundle...
By Test Blogger7 2025-07-10 15:00:13 0 1
Technology
Seeking an upgrade? The Google Pixel 9 is at its lowest price this year.
Best smartphone deal: Save $250 on Google Pixel 9 Lowest...
By Test Blogger7 2025-06-02 10:00:18 0 989
Technology
Theres still time to shop the Dyson Memorial Day sale — vacuums start at $219
There's still time to shop Dyson's Memorial Day 2025 sale — if you hurry...
By Test Blogger7 2025-05-29 04:00:17 0 931