Episode 127
Humanoid Robots w/ Peter Corke
About this episode
Humanoid robots are rapidly moving from science fiction into reality, powered by breakthroughs in robotics, AI, and artificial intelligence.
In this episode of Untangling Web3, Professor Peter Corke, one of the world’s leading robotics experts, breaks down what robots really are, how they’ve evolved from factory arms to human-like machines, and how close we are to truly intelligent, autonomous humanoids.
From perception systems powered by AI to the economic and ethical implications of automation, this episode offers a grounded, technical, and deeply insightful look at the future of robotics.
Key highlights
- What Actually Defines a “Real” Robot. A robot is not just a machine that moves—it is a system built around three core components: perception, planning, and actuation. Perception enables robots to understand their environment using sensors like cameras and lidar, increasingly powered by AI and deep learning. Planning determines how a robot achieves a goal, breaking complex tasks into actionable steps in real time. Actuation executes those plans through motors and physical movement. These three components operate continuously in a loop, often hundreds or thousands of times per second, enabling autonomy.
- Why Humanoid Robots Exist in Human Form. Humanoid robots are designed to resemble humans not just for aesthetic reasons, but for practical and economic ones. The entire physical world, from tools, buildings, and infrastructure, has been designed for the human body. A robot with two arms, two legs, and human-like dexterity can immediately operate within this environment without requiring redesign.
- The Challenge of Trust, Safety, and Physical AI When AI operates in the digital world, errors are often harmless. In robotics, mistakes can have real-world consequences—damage, injury, or worse. This raises critical challenges around reliability, verification, and safety. Ensuring that robots behave predictably and safely in dynamic environments remains one of the biggest barriers to widespread adoption.
Humanoid robots represent the convergence of robotics, AI, and artificial intelligence into systems that can perceive, reason, and act in the physical world. While the technology is advancing rapidly, significant challenges remain in reliability, safety, and societal impact.
The future of robotics will not be defined by a single breakthrough, but by the integration of multiple disciplines, from deep learning to neuroscience. As machines become more capable, the key question is no longer just what robots can do, but how humanity chooses to integrate them into society.
Transcript
Auto-generated from the episode audio.
Welcome back to Entangling Web 3. We've got another great episode for you today, but before that, we'd love it if you could drop a like on the video and subscribe to the channel. >> It's the best way you can support the two of us and help the channel continue to grow. >> And with that said, on with the show. >> Welcome to the Entangling Web 3 podcast. >> So, robots are no longer science fiction. From warehouse automation to surgical systems, and now humanoid robots that can walk, grasp, and reason.
We're entering a new era where machines are starting to move and act in ways that look increasingly human. But, the big question is, how close are we to truly intelligent robots? >> To help us unpack that, we're joined by one of the world's leading robotics >> Professor Peter Cork is a distinguished professor at Queensland University of Technology, fellow of the IEEE and Australia's leading science and engineering academies, and author of the best-selling textbook Robotics, Vision, and Control. >> He's the creator of the Robot Academy, winner of the 2025 Engelberger Award for Education, and currently chief roboticist at Lyra >> Today, we'll explore robotics, autonomy, and whether general intelligence in robots is closer than we think.
>> Professor Peter Cork, welcome to Entangling Web 3. >> It's an absolute pleasure to have you on the show. We've been trying to book this in for a while now. I must say, I'm a big fan. Like, this is quite a big moment for me. I did my PhD in robotics and autonomous systems about 10 years ago now, and it was your book that I first picked up, Robotics, Vision, and Control, and it was your toolboxes that I first played around with. So, this is a real privilege.
>> That's great to great to hear somebody used it and got value from it. Uh, as an author, uh, it's it's it's an awful lot of work to write a book, and it's happy stories you bump into people who use the book, it helped them. Uh, that's to be honest what makes it worthwhile writing writing books. So, yeah, happy to be here. Happy we can make it work. >> But you've been in This is bonkers to say, but you've been in the robotic space since the 1980s, I think.
If You know, correct me if I'm wrong. And most people think of this like of as a like a as technology or an ecosystem that's only evolved in the last 5 years as they're seeing these humanoid robots on YouTube, on Facebook, on Twitter. Can you give us a little bit of an intro into the space? Like, what is the history of robotics? What the hell were you doing in the 1980s? And how come it's only starting to come onto the screen in the last 10 years for most >> So, I think where robotics actually originated from is probably in the minds of science fiction writers, right?
And and and authors from the creative types, not from engineers or geeks. So, the word robot itself probably well known, it was coined in a Czech play in 19 in 1920. Uh that's uh they coined the word robot. It's a Czech word. It's got some kind of patience of slavery about it, which is probably not not great. Uh and then people like Asimov wrote lots of books. And then there were all sorts of all sorts of movies, I guess from the from the '50s onwards. Many of them quite bad.
Uh but this idea of humanoid robots, if you like, mechanical men, metal men, uh who would do our bidding, uh that comes from that comes from that comes from fiction. And this original Czech play that I mentioned called Rossum's Universal Robots. I've read it only once. the storyline is basically the people create robots robotic machines to help them do their work. The robots are kind of sentient. Uh they resent their role. Uh the humans aren't going to change their role, so they rise up and kill all the humans.
Which common story, right? You've probably read that book or seen that movie lots and lots of times. But it it harks way, way back to the beginnings of robots. First robot that built mechanical arms, the sort of things you see in car factories, that dates back to about 1960. So, about the year I was born, right, was the first robot real robots went into into factories to start doing useful work, and there's millions of them around Uh they do they do assembly of circuit boards, they put things in boxes, they put boxes on pallets, uh all sorts of material handling is done by robots.
So, I think most things you buy today probably been touched by a robot somewhere along the way. But, these are very unglamorous, unsexy robots, right? These uh these orange arms or blue arms or whatever you see in factories. So, we keep sort of coming back to try to create robots in our own image. So, this humanoid robots, which have been the staple of science fiction for a long time. The first time I ever saw humanoid robot, I was a relatively young robotic researcher. It's probably Uh and the work that I was doing uh in the '80s was to do with traditional manufacturing robots.
So, those arms in factories doing useful useful tasks. That's where I started. Cuz that was all robots were at that point in time. I was in Japan, I went to a robotics lab and I saw my first ever humanoid robot. And I would found it quite unsettling. Uh I'd never seen one before. Maybe I'd seen pictures of them, but to see one in the flesh, so to speak, you know, walking around, not very well, but it's still it was uh yeah, quite eye-opening and confronting to me.
And now I see them everywhere, and you know, that I don't even give them give them second thought. the reason they came about in Japan in the 1980s was a couple of kind of prescient senior Japanese professors who were thinking about the demographic crisis in Japan, which even back in the 80s could have been predicted. And I think you probably know now that demographic crisis in Japan is probably most extreme of any country on on the planet. So, these guys in the 80s were well ahead of the curve.
And so, they thought that these robots, mechanical mechanical people, could help solve the labor crisis in in decades to come and even help with an age care crisis, which they saw coming. And I guess as senior robotics professors, they would have found it pretty cool to be in a retirement home and being looked after by by robots. But one one thought really really really stuck with me from talking to to these And it was that they thought that the people who would be best placed to build these these uh humanoid robots would have been car companies, because they're really complex electromechanical systems.
So, lots and lots of components, complex assembly, manufacturing process, and the need for after-sales service. So, they thought that you know, it's got the complexity sort of cost even of a of an automobile. Therefore, we usually go and talk to the car companies about about building them. And it's interesting that one of the uh humanoid robots that was outside of a laboratory, was made by Honda. It was a robot called Asimo. It was quite popular in the 1990s, early 2000s. By today's standard, it was woeful.
It would kind of shuffle, and it you'd ask it to jump, and it would jump about this Uh but anyway, it was it was a car company, and they they built humanoid And today, we have uh Hyundai, who own uh Boston Dynamics. and you have Tesla with their with their Optimus robot, and Musk is talking about converting the Fremont factory from building cars to to building robots. So, that kind of offhand observation of these Japanese professors 40 years ago sort of has come to come to pass.
So, yeah, it's absolutely fascinating. >> Yeah, it it definitely again, I um so, you look at Alec, he's got his robotics PhD. I'm someone who is very much a layman when it comes to the robot sphere and everything you guys have mentioned so far. So, I mean I'm I'm very intrigued to learn more about it. Um and there's a few things I think we can unpack even your kind of brief history there. You mentioned first real robots being those kind of mechanical arms on the production line.
Um what do you mean by real robots and then how do they distinguish themselves between I guess things that are pretending to be robots? And then also, where does the idea of humanoid come in? Like is it as simple as making them in our own image, giving them two arms, two legs, or is there anything more kind of fundamental to >> Okay, they're both they're both really really good questions. To to me, as a roboticist, I think that any robot system has got three important or software modules if you like, and they actually execute in in a loop.
So, the first module we generally refer to as perception. So, that is how does the robot understand the world around it? What are all the around it? Where are they? What are they? What can they do? That's really important for a robot to know that, "Hey, this thing's a table and I can put something on it. This thing here is a human. I don't want to put something on that cuz they'll be annoyed or hurt, right?" So, knowing what things are and what they can be used for, what we call their affordances, is really useful.
And that's perception problem. Now, that can be done with cameras, uh it can be done with lidar sensors, it can be done with ultrasonics. There's a whole bunch of sensors that roboticists have used over over decades to try and figure out what what's in the world, where am I in the in the world? I think where we where we've landed over the last decade is that cameras plus uh deep learning systems are absolutely tremendous for understanding this really complex of how do I map tens of millions of color pixels uh that I get from a camera looking at the world?
How do I turn that into into objects? You know, this group of pixels over here, they belong to a human. This group of pixels, they belong to a table. You know, we can do amazing things from vision now. We can find the flat just from a single view, uh we can find flat surfaces, how far away things are, what are the surface normals. So, you know, this is a horizontal This is a horizontal surface. This is a vertical surface. You know, we can get all that information from from a camera using deep learning algorithms.
And deep learning algorithms started to appear in the computer vision literature maybe 20 2013, and it's gone on in leaps and bounds since then. Data-driven approaches, so they used all the image labeled images on the internet. Uh GPUs were starting to become uh cheaper and cheaper and became the computational platform of choice. So, uh there are algorithms now that can process many, many times a second uh images and tell us what's in the world. So, that's the perception block. we see that on humanoid robots, but we also see it on autonomous cars, which are essentially robots with people inside.
Uh and they tend to use different sensors. They tend to use lidars, send out pulses of laser light, time how long it takes to go out and come back. Uh they also have radars do a similar thing. In automobiles, you've got this problem we call sensor fusion. That's how do you get information from a camera and from a lidar and from a radar and stitch it all together into one coherent model, if you like, a digital twin of the physical world. And each sensor's got imperfections.
So, you know, it might be that the camera doesn't see very well where there's a shiny reflection off a car, but the the lidar might pick that bit up. Each So, working together multiple sensors, sticks them together with some pretty amazing maths, and we can get a a pretty reliable model of the world. That's perception. We need to do that many times a second because the world is not constant. And the robot, if particularly it's a humanoid or autonomous vehicle moving at speed through the world, uh then we need to keep updating where our respect to the world.
Second module is planning. So, a robot's always got a goal, right? This is robot, this is what I want you to do. And so, the goal of the planning module is to say, "Okay, how do I satisfy the goal given I know where I am, I know what's in the world around me. What's What's the next thing that I should do?" Uh and often planners have got a long horizon and a short horizon. So, let's say your job is to clear a pallet of boxes and put them on a conveyor belt if you were a humanoid robot tasked with doing that.
So, you could say, "Well, okay, there's the pallet and there are some boxes." But, you know, you've got to take it down from the top. So, you find which is the top-most box. Okay, so I know where the top-most box is. Where should I place my hand so that I can grasp that top-most box in a way that's stable and I won't drop it. Right? And then how do I plan a a motion? How do I walk from the pallet to the conveyor belt without tripping over something on the floor?
So, that's And we've got to do that often as well, many times a second because the world is uh and the robot in this case is changing the world, right? It's moving It's changing the stack of boxes on a pallet to boxes on a conveyor belt. So, the robot is actively changing the world. We need to keep updating that. And the third component is actuation. That's the motors in the legs, or if it's a drone, it's the motors in the propellers. If it's a mechanical arm, it's the motors in the joints.
Right? They actually make the thing So, the robot moves. It changes its its position in the world. Maybe the world changes. So, we do the perception again. Things have changed. We do the planning again. All right, and then we send some more commands out to the motors. And we do this over and over again. Some robots that may be up to 1,000 times a second. Right? So, to me, that's a robot. That's a fully autonomous system. Give it a goal, and it will achieve the goal.
Now, there's a halfway house, a thing called teleoperation, right? And that's there isn't a perception system, and there isn't a planning system. There's a person behind the behind the black curtain with a joysticks. And unfortunately, we see this a lot at the moment in YouTube videos of humanoids. And it's impossible to tell, right? Because the the motion looks looks really good. Occasionally, you catch them out. There was a famous Optimus humanoid demo where you could actually see in the corner of the frame the guy's hands on the on the joystick, which is a bit of a bit of a giveaway.
The very first Optimus robot that was by Elon some years ago, the very first one, there was just one Optimus robot, and it was a dude in the suit. It wasn't even a robot being teleoperated. So, he's gone from a dude in the suit to a large amount of teleoperation through to the goal being fully autonomous, what I would call a proper robot, right? That It's got a goal like unload the dishwasher and YouTube's a wash now with videos of humanoids unloading dishwashers because it's considered to be a complex task.
If you look at the videos, mostly the dishwasher's got two plates and a glass in it. Uh but, you know, it we're starting to get there. But, for a lot of these videos, you cannot tell if it's being teleoperated or whether it's fully fully autonomous. This is a bit of a bit of a worry for the for the field. these days, too, the AI generation of of videos is is remarkable uh using using AI tools that exist at commodity tools. And there was a video I saw just yesterday of a whole bunch of humanoid robots uh wearing red shirts.
I looked like they're in Tiananmen Square. There were hundreds and hundreds of them. I'm pretty sure and some other people commenting on this post, pretty sure that this was this was a simulation. Uh or an AI-generated video. It wasn't actually robots. So, a few days before >> a profes- a professor in robotics is the one to actually verify whether these are real these days? >> And absolutely, we can't we can't tell. uh a few days ago, there was a big exhibition, I think, in China, and there were a lot of physical robots on stage, and they would do a synchronized backflip and and dancing and all that stuff.
I'm pretty sure they were real. But, the ones in Tiananmen Square, I don't think they were they were they were real. So, Jack, that to to me is my definition of what a real robot is. Uh and then this halfway house of And teleoperation is actually a really important technology. It it started in the Second World War when uh people were trying to build uh assemble the first nuclear weapons. So, not a not a great heritage, but they had to manipulate pieces of radioactive material and build the bomb.
And so, they had a remote mechanical arm miles of glass, and then a person on the other side of the glass could see what was going on, and they had some hand controllers, and they would manipulate control the robot on the other side of the glass. Uh deep underwater exploration is also done with teleoperation. So, the the pilot's on a ship uh and communicates their actions down to a robot way down down in the deep. And a lot of drones are teleoperated, of There are some Some drones are fully autonomous, but a lot of the drones you see, the ones we saw at the Winter Olympics, they were all piloted by remote controlled by people with some amazing flying skills.
Oh, Jack, I forgot your second question. I got carried away. >> Well, this was a I mean, that was such a fascinating answer, and I was just kind of I feel like I'm an electrical, which is great in in the best way possible. I feel like I No, no, no, in a really good way. Electrical that I'm fully engaged with, I have to add. Um yeah, my my next question on kind of following that was So, you you're you're talking about this humanoid concept, and you mentioned the beginning kind of the idea of, you know, naturally I mean, maybe it came first, I don't know the order of operations in the in the kind of sci-fi literature, like maybe it was humanoid first or not, is humanoid robotics just making them in our image, and maybe in like physical capabilities, or is there is there a more kind of fundamental scientific definition like the one you gave there for robotics generally for humanoid >> I think the definition is as simple as it being in
our in our image. And the reason that we that we do it, one is one is hubris, and I think that humans consider themselves you know the top of the pile when it comes to evolution and and capability. There are lots of things that uh that human beings are able to do quite extraordinary uh from sort of from tasks that are useful to all sorts of uh sporting achievements at at again Winter Olympics uh it's amazing to think the things that that human beings are actually able to do.
So I think it's hubris that we're kind of the top of the heap and we should build something like us cuz if it's like us then it can do it can do everything. It can certainly do everything that we can imagine. Uh and our imagination is limited by our form. Uh like if we're a cockroach we'd imagine different things that we might possibly be able to do but uh we can't know that. Uh the second argument uh and probably what drives the economics of humanoids is we've spent millennia building a world that suits us.
You know it's it's uh and you know it's the houses the roads the footpaths all the tools and utensils all of these things have been designed for for animals without with our form factor. So you know if I want to drill a hole in the wall I can just pick up an electric drill and drill a hole in the wall and I can pick up a spanner and I can pick up a screwdriver. I'm kind of a general a general purpose thing uh that can use tools to do an amazing variety of tasks.
And the thinking goes that if you build a a robot that looks just like us then it can use the full arsenal of tools that we've already built. You know I can ask the humanoid robot to pick up a drill make a hole in the wall and and and do whatever else that uh that I that I want it to do. The alternative is that you have a world that's full of special purpose robots. So in your house maybe you've already got a vacuum cleaning robot.
I've got Uh but then you'd need a you know, a table setting robot and a cooking robot uh and uh and you know, a a car robot, self-driving car, they're Uh but we don't make we'd have a world that was full of special purpose robots. Uh each one of them is complex, expensive to design and it probably just would be too too inconvenient. One person couldn't afford to have all of the possible robotic things in the house. But you could have a humanoid robot and it could use all the things you already have in your house.
Cook you a meal, just go and stand in the kitchen, use your knives, pots, pans, pantry, fridge uh and and create your and create your meal. So, I think that's the that's the uh and that was so that was the argument put to put to me by those senior Japanese professors all those decades ago and I think that's what the current co-order of humanoid robot companies are are chasing a general purpose assistant. So, in a manuf- in a manufacturing environment, which is where they are looking at first because uh there are a lot of a lot of facilities where they have labor shortages, particularly in Uh yeah, it's picking boxes off shelves and putting them in other boxes, moving them around.
Uh that's has it's not a particularly attractive job and they have a lot of labor churn, trouble attracting labor. So, they're the sort of jobs where they hope that a labor vacuum will act as a technology pull, if you like, for for humanoid robots and technology is always evolve much faster if there's a if there's a pull factor rather than just relying on on push factor and and I think if they can get established in those kind of manufacturing and logistics jobs, then the number of them being built uh will go up, the cost will come down, reliability will go up uh as you as you got the learning curve.
And then maybe they'll trickle down to rich households, and then later on perhaps they'll trickle down to uh to middle class and and poorer households. Uh that's kind of often the way the way it goes, right? High- high-tech cars uh bought by are expensive, bought by rich people, but those technologies, you know, like sort of lane lane keeping and intermittent even intermittent wipers, all those sort of things we take for granted in a mid-range car today used to only be found in in high-end cars. So, I expect there'll be a similar a similar trickle >> It seems like the customer stream for humanoid robots absolutely everywhere is kind of coming to fruition, maybe only a bit later than you predicted.
I think you said around the 2000s is when he was saying they were roughly coming out or something like this. I guess I Does This is a question I get asked often about the space period around like is it is it accelerating? Has there been something that's happened in the last 5 to 10 years that's really meant that the space is moving forward? You've broken down like robots into into three rough areas like perception, which is obviously hardware plus software to make it happen, planning, which I guess is mostly software, but obviously is based on the perception situation, and then actuation, which is much more physical, mechanical, mechatronics, that kind of stuff.
Is there something that's changed in the last 5 to 10 years, which is the reason why people are suddenly getting, you know, videos of humanoid robots on their YouTube, on their Twitter, all this kind of stuff? Has A lot of people attribute this to the breakthroughs in LLMs and all this kind of stuff. Do you think that's having a big influence? Or like what's your gut feeling on this? >> Look, it's a few factors coming together, but the point you make about a lot of those modules I mentioned earlier being software is exactly is exactly correct.
So, the perception is all software and planning is all software. And so, they're all riding the the deep learning data-driven uh wave that we've seen. So, perception started earliest and AI for for perception now is is is Uh so, we've had algorithms now, well, more than a decade to to get uh really robust perception systems. You can't do anything without good robust perception. So, we've had a a decade of of using deep learning for perception. It's got better and better. Uh the GPUs have gotten cheaper and cheaper.
Uh There There is still some still some concerns here around power consumption. Uh I keep sort of pointing out to people that this uses 18 W, right? Uh and I can do perception and planning. My perception part's only 6 W. So, uh you don't get much much in the way of GPU computation done at at 6 W. So, uh so, put the power issue aside for now. Uh let's let's consider then the next part, which is planning. And planning is a really gnarly problem. People have been working on planning problems for for decades.
the problem with planning is is the combinatorics, right? Uh it's a bit like playing chess, you know? I've got to achieve this goal. What should be my first move? And you know, if I make this first move, then there's a whole range of possibilities, which I can't know about until I've made the first move. And so, robust planning is computationally very very expensive. It used to take It used to take minutes uh to do planning. In fact, you look back at some old robot videos from Stanford, I don't know, probably '70s or 80s, the planning was maybe tens of minutes.
A robot would move move at 6 in and then it would plan for half an hour and it would move another another 6 in. Perception and planning, to be to be fair. So, planning was all done sort of analytically, algorithmically, with hand-crafted algorithms, and we had a few breakthroughs along along the way, but we really had plateaued. It's a bit like perception. People have been working on robot perception, artificial computer vision for decades until deep learning moment, right? But we'd had we had plateaued. Deep learning just kicked it up a huge step and it just then it was on another was on another growth curve.
I think with the same thing's happened with planning. Uh, and people started to apply deep learning, data-driven techniques to to planning. the LLM, uh, I was going to say the humble LLM, but it's not really that humble. They're actually remarkably good at planning. You can ask your LLM, right? Uh, if pick your favorite LLM, ask it how to do a task, how to make a cup of tea, how to fix a door. Uh, it's watched all the YouTube videos, right? It's transcribed them, it's learned from them, it's read books, it's read every Reddit uh that exists.
So, it knows how to do a lot of really practical thing, and it decomposes them for you into steps. And that's gold, right? So, you now you've got a perception system that looks at the world and says, "That's a screwdriver and that's a drill and that's a table and that's a wall." And then you've got an LLM that says, you know, "Pick up the pick up the drill and uh, you know, and make a hole in the wall and then pick up another tool." Right? So, the LLM can do reasoning at at a at a level of abstraction which matches the abstraction now we get from the perception system.
So, we're are longer about pixels, and we are no longer talking about uh really low-level motor commands. We can reason in a similar way to which we Doesn't mean it's the best way to reason ever, but it's a way of reasoning that we can we can check, we can understand what it's doing, what it's thinking, Uh and I think that's been a really big big step forward. The other thing is that people have now started to to demonstrate to robots tasks visually. So, you you train a robot and it does a Brazilian tasks.
The robot is is is maybe it's being guided by someone with joysticks to make a cup of tea or make a cup of coffee or to drill a hole in the wall or whatever. And through training on all of these videos of what what the camera images are and how the robot moves, is being moved by the human, these models are starting to themselves given a goal and a real-time view of the scene, how the robot should move in order to do the Beginning to be able to generalize.
So, you could ask it to make a cup of tea, and then you could change the shape and color of the teapot and the and the cup, and you could put them in different places on the table, and not always, but surprisingly often, it can generalize and do that task. And this is a really exciting moment. This is kind of where we are at now in learning from learning from it's far from perfect, and that's the problem we have we have at the moment. Maybe 50 to 60% accurate.
Uh so, we have this problem we with LLMs that we call hallucinations, and I'd argue they're not Uh People refer to it as which is maybe a bit more accurate. Hallucination is a perceptual problem, right? You think you see something that you're not seeing. LLMs are not seeing anything, so hallucination really is is the wrong word. And they're problematic. If you If you know you're If you're on your on your game, you can pick the the errors that the LLM makes and you can correct it and then apologize and then go and have a have another go.
If it's an office automation task or something like that, it's no big deal. If you're controlling a machine that can move at speed in an environment perhaps where there are people, then the consequences of these errors are much more profound. And this is what's holding us holding us back at the moment because the cost the cost of errors is is too high. So, this business is often referred to as embodied intelligence, embodied AI, or physical AI. It's where the AI is really tightly coupled to a machine.
The machine exists in the physical world. Most of the AIs we deal with exist in a virtual world, right? You type characters into a chat window, it sends stuff back, or it hacks your code for you. It's all really useful. But now we're putting the AI into the physical world where it can take actions, where it can change the world, which is quite profound. it's what we do. And there's a whole bunch of of theory from sort of psychology and and human development that says that's how we learn.
So, we are embodied in the world. We exist in the world. And really early when we're really young and we're lying on our backs, flailing our arms around, we learn the correlation between sensation in our skin and our fingertips and what we see with our actions. and initially we are just flailing but after not too long we start to become coordinated. So we got this massive coincidence detector in our heads and we correlate the perceptions that we get from all of our senses with our actions and we just get better and better and better but it takes us a long time to get there.
The advantage of robots and Sam Altman made a kind of controversial statement just a couple of days ago trying to compare the energy cost of training a network with the calorific food energy cost of training a person up to the year of year 20. And he's out by a factor of 3,000 I think I read somewhere. So it's not directly comparable but here's the thing that the that we learn we spend maybe 10 20 years learning and then we have a useful period where we apply the learning then we die.
which is kind of a waste. With these entities be you know either an AI or a it shouldn't it won't die. It's information its knowledge its weights are in a file. So the training should not ever have to be done from scratch. We should be on some sort of ratchet right where each one can start with the learning of the one The other thing that advantage of robots have which is kind of freaky is that they can communicate with each other in a way that we can't.
So you've got 10 robots doing a job one robot has a learning moment. It can instantly update all of the other robots with what it's just with what it's just learnt. And you scale it up to lots and lots of robots millions of robots I don't know where we're going to Uh you know the rate of learning will be phenomenal and because it's a ratchet it will never go backwards. Uh which is both fascinating and I think absolutely terrifying. >> I've got I've got so many things that I wanted to follow up on.
I don't know where to start. Like I guess there's a quick comment on that that learning stuff. I guess one of the scary things about that though is I heard you say this before I think in a podcast you did previously was around like you you create a system, you test it, you fine-tune it, all this stuff. It goes out into the world. If it has a learning moment, who's to say that learning moment doesn't then mess up the robot and all these tests you did across a million different units is valid anymore?
Like what happens if robots in out in the world have this ability to learn updates? Like how do we keep them safe and secure? Especially if they're across millions of different robots as well. But that's not what the question I want to ask you. The question I wanted to ask was you talked about generalized AI actually working to control robotic systems. So I didn't know that was a thing. I thought that it was going to be very much you have like the LLM which is good at high-level stuff, language, understanding the world, creating a rough high-level plan.
And then it would go to a control system which I know is expertise which is much more kinematics, actually, you know, how do I get from A to B, that kind of stuff. Are you saying that that's not the case? That it actually can have generalized AI that can do both those steps now? >> Correct. Uh, and this is where I begin to have problems with with the approach. So, yes, these AIs, they take images in from a camera many frames per second and they output the motor commands directly.
Right? So, it learns not only the task, but it also learns something about the robot's uh, what we call its kinematic structure. That is the way how you for a particular set of of motor commands, how the end of the robot gets to some some particular position in space. They learn And my problem with that is this business of kinematics, we've been studying it in robotics for 60 years, right? We're really good at it and it's really cheap uh, to do computationally. It's some multiplications and a bit of trigonometry.
Pretty cheap. Uh, yet we go to all the trouble of learning these things. So, to me a more sensible architecture would be a fusion of the classical stuff that we already know and the and these uh visual learning action models uh to do uh do the hard part, which is the is the planning, but to get them out of the control part. I don't think they've got any role being in the control part. But, this is a big debate at the moment about how much should we learn versus how much uh should we integrate existing uh approaches to to problems.
And I think there was some hype a couple of years ago that these LLMs were on the path to AGI. AGI is going to arrive in 2025 or 2026 or 2027. It's always a couple of years away. I think there's a growing number of voices now that say LLMs are not the path to AGI. They just won't scale anymore. Uh we are running running seriously running out of out of electricity and cooling water, but most importantly data on which to to train them. And the only way we can get more data is really to scrape the bottom of the barrel to get, you know, really low-quality data, which is more like opinions than facts uh you know, out of social media feeds.
So, we've maxed out and we haven't got to AGI, so we're we've hit a bit of a wall. So, I think smart money now is saying that to get to AGI, we have to fuse LLMs, which got amazing fluency with language, ability to understand language, and and and and uh and create language with with knowledge that's represented in a in a symbolic way. Uh so, you know, that that's algorithms that that can do things like you can ask an LLM to add up numbers and originally they were really bad at adding number up numbers.
Though, it's one of the simplest things to write a computer program to do. So, I think the future will be some mixture of analytic knowledge, symbolic knowledge, plus plus >> Do you think part of that is the the type of data that's being read now? And you talked about symbolism using symbolics and what what not. But, like the idea of you said, "Okay, right now we're just feeding them on what's on the internet, mostly text-based. They're great with language, all this kind of stuff." Do you think part of it, like from the human experience, I guess maybe us putting the human experience on it, is the idea of can they start to process, I know they can process images, but can we start to entrain these embodied AIs?
And do you think that will help them on their learning journey, or is that too far-fetched to think of right now? >> I, to be honest, Alec, I don't know the the answer to that. There's a lot of people who know far more about AI than than I do who who are working on this this goal of AGI has sort of been set. The gauntlet's been thrown down. So, there's a lot of people knocking themselves out to try and and achieve that. And I think using a whole bunch of bunch of different techniques.
So, it's really early days and I think can't yet say what what approach will will will prevail. >> Yeah, I I was I'm kind of really curious on the AGI question. Um because for me that's also the question that opens all of the the doors of the worries around the ethics and the the the future for humanity. I don't know if that's if that's a good place to go next in the conversation, Peter. Like if you have any thoughts broadly about the the implications cuz, you know, whether or not we get there, as you say, that is the stated goal now.
The gauntlet's been thrown down. People are trying they're throwing all their capital and and and kind of brain power with the human brain power we have left towards this problem. Uh do you think it's a problem we want to be trying to solve in terms of the impact it's going to have for for human the human race at large cuz obviously people cite, you know, the impact on employment and labor. There are good things that can come from that. There are also bad. Also, you know, things to do with basic economics, right?
What are we all going to do for money and are we going to need money when when the when the robots come? So, yeah, I'm I'm just curious just to open a huge can of worms, but like what what are you all kind of positions on these things? >> Look, these are a bunch of of really good questions and really important questions that I don't think get talked about talked about enough. Uh so, I can give you some opinions and some and some thoughts. I probably started actually as a bit of an LLM skeptic when they first came out and all my colleagues sort of jumped on the bandwagon and I sort of just kept a kept a bit of a distance.
I've been reading reading quite quite a bit. Uh I would say over the last 3-4 months I've become I have started using it LLMs in a in a very regular way in my in my workflow. So, I'm retired from the university, doing some consulting work, doing lots of open source code development. And the code coding tools are absolutely superb for, you know, refactoring and and and and and linting my code and all of these things. So, I'm really quite impressed at what it can do for code.
the implications of of AGI either it's an AGI in a chat window or it's an AGI in control of a physical agent in the world, so it's a humanoid with AGI. I think are really profound for our society and we've not thought them through and we are just going headlong into it because we think we can. Uh and that's a a bit of a bug in in in human nature. Uh we built we built nuclear weapons uh because we think we can, but we also because we thought the other guy is going to do that and we should be there.
We should be there first. We get do have actually some of that angle be when there's tension between the US and China. So, the US wants to get the AGI first, otherwise the Chinese will get there first and then we'll have uh have communist AGI and that won't that won't stand. So, yeah, we do have the the race. Uh humans are very are very AGI, the idea of AGI is out there. So, I don't think we can stop I don't think we can stop it. Uh I don't even think we could we could we could regulate Maybe we can regulate it.
Uh I don't think we can we can stop it. So, it will come to it will come to Things that I've been reading indicate that there are already implications in the job in the job sector, right? So, uh job growth is low, but industrial productivity is going up and people are saying, "Well, that's the whole that's the impact of AI in the workplace, right? People can establish people can get more done using AI tools. Uh therefore, we don't need to hire as many as many uh new Uh I think the jury is still out on this, but some people are saying this is the early hallmarks of the impact of of Uh and that then raises a whole lot of lot of questions uh that you that you mentioned.
Uh What happens to people if they can't if they can't work, if they are not needed? I think that for people who already established in a field, people who who know that who know their game, then AI is a massive um massive force multiplier. And I'd say that's the situation for for me and other other colleagues that I I think for younger people coming out of university or coming out of school, I think that's the the cohort to be most most worried about because they don't have the experience to be able to answer back to an LLM.
Uh how do we train them so that they've got the critical thinking skills to be able to push back against an LLM? Um I trained as an engineer, so many engineers uh I've got this kind of gut feel. You can sort of someone can tell you the answer to problem is this and you say, "No, that can't be right." Yeah, it doesn't it doesn't feel right in my gut. It doesn't feel right. Yeah, that engineering intuition I I don't know where it comes from. Uh you get a bit of training at university, but mostly it's just stuff you've picked up along the way.
Uh that means I can push back against an LLM when it tells me something that I don't believe it. I'll challenge it and it'll apologize and go around the loop again. How do we give it get young people that that skill? I think it's really hard. I think the only way we one of the ways we can do it, particularly at universities, is to have much more interactions with the with the lecturers and and the teachers, right? So, I think sort of sitting homework assignments is is a forlorn activity these days, right?
Cuz the many of the students will answer it using LLM. It'll be beautiful. Uh go back to the teachers. They'll say this is beautiful and they'll get a good mark or the teachers will put it through an AI which will grade it. It's the whole thing. It's completely pointless then. So, I think sort of more of an apprenticeship model. It's what we have for PhDs, but maybe that has to go down to undergraduate education. Uh more one-on-one with with teachers. But that's exactly the opposite direction that universities are going.
They're all so cash-strapped, not just in Australia, but I think in the UK as well, other parts of the world. Uh they're trying to do more students with less staff. And I I think this is this is going to end very very badly. Uh it's going to have to be some some major change in change in And there is the question that if AI and and robots are doing all of the work, who are they doing the work for? Uh who needs the work to be done?
Uh how do the people get the the essential essential comforts? How do they get food on the on the table? Uh a roof over their head? Uh and enough resource to do things that that keep them that keep them And that is a big challenge because I think that if we had a growing cohort of young people uh who have got no who've got no future and no jobs, uh they're going to get angry. And we've seen before how this plays out in history, right? It leads to revolutions and and overthrowing of the of the system.
So, yeah, I'm a bit bit worried, a lot worried about how this is all going to pan out as a society. And I don't know who's leading the thinking on on this. The problem gets mentioned a lot, but solutions don't get mentioned. UBI gets mentioned, universal basic income. but I don't know what you get paid in. Do you get paid in in Yeah, I don't know how an economy can can can work. Uh so maybe that's a good topic for a future for a future podcast.
Find some economists and thrash this and thrash this one out. The other And you're also Jack raised some you call moral issues. And certainly I unemployment, fundamental changing society is a deep moral issue. I think there is also a moral issue having probably the best and brightest minds working in this field. You could say that's a moral problem because there are other more important problems for humanity than solving the AGI problem. So, if you took a lot of those people and said, "Okay, how about you think about climate change for a few years?
Or how do you think about providing fresh water and and food for all the people on the planet?" That would be probably a better use of those minds than what they're currently employed in. The trouble is there's not enough money in it. There's investor money flowing in, torrents of it flowing into large language models and humanoids, not so much flowing into into into climate change. This is a problem. >> So many things to talk about on that. I think um it's a tricky one, right? The money goes with the capitalist at the end of the day.
And you kind of I I completely agree that we have so much focus on AGI and making money and like all of these LLM creators, makers are making so much money now and they will continue to make so much money even if the stock market collapses, which a lot of people predicting. It's inevitable, the rise of these LLMs that are so part of people's day-to-days now. We'll become completely dependent on them and so are corporations. What does that mean for humanity? I guess there's some that say that it could lead to a reduction of all costs when productivity goes through the roof.
Hopefully everything gets cheaper. Do I think that's true? Unsure. Like I guess it's more likely maybe it seems that capital will just concentrate on a few individuals that are running the show. I guess you can see some benefits for society in things like DeepMind being pointed at protein discovery to help with cancer and drug discovery and all this kind of stuff. There's obviously a lot of good that can come from this. It's just will it continue to be aligned with capital? Unsure. Just to take it a step back on this like this has been such a fascinating show.
I just want to ask you just to wrap things up. You know, obviously you've been in the space now for 40 years. What do you think the next 10 to 15 years look like? What does the robotics landscape look like? Obviously we see lots of hype on social media, some of it real some of it completely fake. What do you want to be the case in 10 to 15 years? What are your hopeful for and what do you think the reality is in 10 to 15 years in robotics specifically?
>> 10 to 15 years is a very long is a very long time horizon. I am pretty rubbish at predicting at predicting things. Uh I think we're in I think we will just continue along the groove that we are in. There's going to be a bunch of people working on on deep learning LLMs, AGI. Maybe we'll get there, maybe we won't. But all of the work that they do is going to help robots, right? Uh if perception systems improve, if planning systems improve, language interface improves, those are all things that roboticists take off the shelf.
And robotic roboticists are like magpies. We just take shiny things from wherever. You know, it it can be from gaming, it can be from some arcane field of research. You know, we'll just we'll just use it and use it to make our robots better. So I think there will be one aspect is we're just going along this groove and the technology will get Uh as the as the quantities go up and I don't think we're going to see widespread humanoid deployment in the next year or two, but I say 5 to 10 years out we will see widespread And those humanoids will get better and they'll get cheaper, right?
So we'll definitely see it's going up the learning curve with humanoid robots and we'll find many more applications for them. I don't think that that means that the old style old style traditional style mechanical arms that we that are dominant manufacturing they will go away. They are wickedly fast and very very very very cost effective efficient way of getting stuff done. So they're going to stay there just because they're mature and the cost is reasonable. We know we know how to use them. Humanoid robots I think will always be much slower than those arms and sometimes those arms can move you know faster than you can than you can see.
Humanoids won't certainly low cost humanoids won't be won't be doing that. what else will will come will be coming down the pike? I would hope that in parallel with all those developments in artificial intelligence and and humanoids that there'll be increased understanding of how our own brain works. So that's breakthroughs from neuroscience and already you know we can map all the neurons the connectivity in in brains. It's not quite enough just to know how everything's connected to everything else. I think we need people to try and interpret that and work out the different parts of the brain how do they actually work?
You know how does the visual cortex actually work? What tricks does it pull? Because I think that way we may get to be able to build intelligence that's way less power hungry than what we have at the moment. I don't think it's sustainable for the planet to keep trying to do AI the way we're doing it with with GPUs. Um some companies experimenting with their own chips Google Amazon for instance then more efficient but I don't think by even by a factor of two more efficient.
So to get to something like biological levels of power consumption for the sorts of things that we're able to do. Mimicking that, I think there'll be breakthroughs that come from neuroscience. There's a lot of work on people building neuromorphic computers. There's non-spiking architectures and things like that, mimicking at a really low level how our neurons work. Cuz what's often referred to as neural a very pale approximation of how neurons uh in our in our heads work. And we have lots and lots of different sorts of neurons that have evolved very specific Bit of a monoculture in the traditional way we do AI and deep neural networks.
So, I'm I'm hoping to be insights there cuz I think this energy frontier is a really important one for for us to break >> Yeah, that's a really interesting way I think to summarize um in as at least my limited knowledge of the robotic space, but it seems to me like in the same way that the human body and form and more importantly the brain has been formed over, you know, how many millions, billions of years, depending on how far back you want to go. It takes from so many different inputs, right?
And and and this process of evolution in a similar way, it seems like robotics is a is a science and a field that can take from different places to improve the uh the perception layer or the planning layer or the actuation layer. And it's very exciting cuz I guess that makes it hard to predict, you know, which will be the bottle that going forward, right? If you if if you get advancements in one area, then they might, you know, help one one one phase of that.
But yeah, I think Peter, it's been it's been such a fantastic conversation. I my brain is whirring with new questions and new things. So, I'm sure if you'd if you'd if you'd be willing, we'd love to have you back at some point. Um and yeah, I think I I really enjoyed your kind of very grounded view of what robotics is and putting it in plain English, but also I couldn't help but notice the parallel with Dr. Ian Malcolm. We were too busy thinking of whether we could do it to think about whether we should do it.
Uh Thanks. It's been a a great conversation. Thanks for thanks for having me on. >> That's been a real pleasure. And Peter, where can people go to find more about you and your work if they're interested? >> Um probably best place to go would be Uh it's kind of my homepage and links to the from there to all sorts of all sorts of things that I that I do and and and I'm interested in. You also I'm on on LinkedIn and post occasional things increasingly about AI.
Not what I'm doing in AI, but just interesting perspectives about AI because it's the biggest change that's I think we are collectively seeing. Uh we need to sort of read widely and share widely different opinions, I think, to unpack what the implications are for this stuff. >> Yeah, couldn't agree more. Peter, it's been it's been brilliant. I hope this again this isn't the last time we see you on the show. Uh and yeah, thanks to everyone listening at home from wherever you may be. And we can join us next time to untangle a little more Web3.
>> Thanks for tuning in to another episode of Untangling Web3. Don't forget to like and subscribe and to follow us on social media. It really does help us out. >> The views we express here are our own and do not reflect the views of our employers. And with that, we'll see you next time.
Topics: AI & Robotics