This is in teleoperated, not autonomous mode. Training data is collected while teleoperating. That robot can do some things autonomously, but not anything like the whole meal task.
The Github repository is more useful.[1] That separates autonomous from teleoperated mode. Also, there's video at 1x speed. This thing is actually very slow in autonomous mode.
It's real progress, but the hype exceeds the results.
Even when we reach a point where the hype is appropriate and the tech is actually ready and I can expect to finally buy a robot that will clean my house and do our dishes I still won't be getting one since you can bet that any robot sold to consumers will be constantly recording audio and video of the inside of our homes to send back to the manufacturer and all of that data will ultimately be used to enrich others at our expense, shared with law enforcement, and most likely end up in the hands of data brokers.
The days where new cool technology that would make our lives better was something to look forward to are over. Now it's "How is this new technology going to be used against me?" and "How soon will this technology be deliberately made worse in order to take more from my family?"
The best we can hope for is that cleaning robots will be something that people can build and program themselves from trusted parts, but considering how that hasn't worked out for most of the recent tech in our lives today I'm not encouraged. No one I know is using a cell phone, a smart TV, or a modern game console that they built themselves. Most people feel that they are incapable of even putting a simple camera on their own front door that doesn't also hand their data over to amazon and to the police.
Until new technology can be truly owned by the people who pay for it, and isn't used against the buyer for someone else's interests I'll be skeptical of every new device and gadget they try to sell us.
Subscriptions, telemetry, forced "upgrades," the need to create an account, and always-online requirements have ruined my love of technology products and made me skeptical by default. Gone are the days when you could just buy a product from a manufacturer and not have to have this weird ongoing relationship with that manufacturer. Everything is unnecessarily tethered to the manufacturer now. The Market is failing to solve this problem.
The problem isn’t about KGB style spying with half a million employees. The problem is technology enabling automated KGB style spying that feeds automated information operations on an international level.
If everything you say in your house is digitally stored in perpetuity, the things you say today can be used to manipulate you tomorrow by either a state or a corporation.
Historically, plenty have, and sometimes those governments arise unexpectedly and rapidly.
We have no idea what a fascist government with the ability to closely monitor every person in real-time looks like yet. Maybe it's the answer to the Fermi paradox.
The teleoperated thing was flagged by Rodney Brooks years ago as both a scary risk, but also a driver for autonomous robotics. The threat is that this is a mechanism to partition poor people from richer people comprehensively - they will not be permitted to share the same space but instead will operate proxy bodies that can be confiscated at any time. The poor will live 100's of KM from the rich and will be forced to accept harsh social conditions, and the rich will not have to confront the reality of the poor's humanity.
Oh, you mean a plumber working from home, just like a programmer? And not having to pay exorbitant real estate prices to live near his clients? What a horror!
We already have drone pilots working from "home", and it appears the alienation is real. Not that bombing military age children from an f-16 is a walk in the park.
And while I personally enjoy working from home I know a lot of people who would claw at the walls if forced to WFH.
Point being - it might seem like a small difference - doesn't mean that it is a small difference.
So, in our current reality rich people in rich countries are immensely worried about the fate of poor people in countries with terrible human rights abuses? But once avatar bodies are invented those fine folks will turn into blood sucking monsters?
Are they that concerned? Some people are, and they will take some action - but only on what is reported in the media they read, and its very sporadic and random.
Because the real world is infinitely more messy than a database. Do a few plumbing fixes and you will immediately understand the vast challenge of automating that type of work.
It's actually a fantastic example of probably one of the last jobs/trades to be automated.
It might work half the time in standardized catalog houses/apartments soon - but might need a few hours per job of human guidance in the majority of jobs for the forseeable future?
Why would they pay the per-job API charge for RoboMario when an actual sweatshop worker is cheaper than the first-world electricity to run the server farm?
Plumbers aren't really a good example as they are skilled and powerful. Therefore a teleoperated plumber robot could actually enhance the life of plumbers as they would be able to do more work more conveniently.
However, for cleaners or domestic servants, well expect a race to the bottom, think North Korean prison colony for a model.
How are plumbers so skilled and powerful that they can avoid being enslaved? If slavery is legal in your country, certainly being a plumber vs housemaid will not be the deciding factor in whether you become a slave. Clearly the key factor here is the legality of slavery, not the existence of remotely operated robots.
Edit: I also don't understand why we suddenly jumped from GP's scary dystopian warning to robots operated by foreign slaves. Casting aside how unrealistic the underlying technology is, applying it en masse is an entirely different discussion. You're essentially allowing the whole world's population to migrate to your country.
It's not literal slavery, rather what's often called 'a modern form of slavery' that is the unfortunate risk. People working for very low payment, they can barely live if it. Because of the tele operation, people will compete with a lot of peers across the world, hence the pressure on the salary. There is also a flip side, the even-poorer people get a chance to compete with the 'normal' poor and generate some income.
I believe there really is no way out of this direction of affairs. It's capatalism, free market at work.
Hopefully soon the machines become smart and cheap enough such that the poor can benefit as well. In the end anything can be automated, building and operating houses and infrastructure, and making food.
Also, some bad news, you've been outsourced and you're being evicted and the bank is selling it to a large corporation that owns half the property in the country.
Your skills and ability to add value compared to your neighbours.
Could the denizens of a North Korean prison colony do your job? If the answer is yes then shortly expect that to happen unless something like a functional democracy stands up and stops it. If teleoperated robots come in then expect that simple tasks like cleaning and laundry will get done by people living in 6' * 4' freezing cells and existing on cabbage soup. If they aren't good at it then they'll get the pigeon and another chance, or just get shot.
But my question is: why is that different to remote tech work? Has this (admittedly, probably joking, but still) scenario played out with remote jobs already?
Because you can't get untrained 15 year olds to deal with your complex IT issues. The same with skilled manual work, if you get an untrained 15 year old to do your plumbing or electrics in a building prepare for the building to get wreaked. Cleaning, labouring - not so much.
We do this already with things like textile manufacturing, IT support, call centre work, and we profit from the labour of those less fortunate than us. Teleoperation is an attempt to widen this pool at the cost of those folks who provide labour locally to you and me who do skilled work that can't be replicated (at the moment at least).
> It's real progress, but the hype exceeds the results.
It's to late, my girlfriend already caught me watching it. Telling her the real thing is slower had no effect. I then told her the final product looks like this. https://youtu.be/ACtozUE6Rzw
While i do believe that LLMs are much faster in becoming better due to their nature of just being software, i do think that GPT 3 broke the investment barriere due to people just understanding/getting it.
With Robots, i also think this is just another demonstration which shows more in what direction it is going than we had bevor. Teslas fake Robot demonstration helps here too.
I personally think we are at the start of the robot exponentail curve and i bet i will see a useful working robot in the next 10-20 years.
And the usefulness of a robot can be achieved a lot sooner now tx to the AI progress / investment. alone Wishper pushed boundaries a lot. Never seen something free with such a good audio to text quality.
If a robot is able to understand "bring this blocks to the top of the building" and will be able to act on it, boom.
We are also seeing a lot of progress in similiar fields like game character movement, training physical worlds in virtual simulation (a lot of progress of nvidia and digital twins).
I think it's in Heinlein's The Door Into Summer where the protagonist is a robotics expert. One part of the story that always bugged me was when the protagonist describes training household robots by showing them what to do. The line is something like, "Let it watch you wash the dishes once and you never have to wash the dishes again."
Teenage me read that years ago and scoffed, thinking "that's now how programming code works, like, at all." Now with this robot and the new Rabbit device from CES, it seems that Heinlein was exactly right.
Industrial robots have a training mode where the motors and clutches are disengaged so the operator can position this multi-ton robot through the movements it wants it to make. I thought that was really cool to watch them being programmed through physical input.
Thats why i think that AI/LLMs etc. are a gamechanger: Not because they can generate text (which is impressive by itself) but because they are very very good in translating human wishes into computer interfaces.
Hi robot, here is my kitchen. Here are dishes etc.
This is cool and interesting work but unfortunately extremely over-hyped. Their whole system is a low cost bimanual system which is interesting. However, beyond that there isn't any algorithmic improvements in here that would make robotics really work.
Basically, no real breakthrough has happened in robotics and everyone is hoping to be able to replicate LLMs/vision path for robotics (i.e. data). But no one really knows how to even collect so much high quality data and no one is even remotely close to executing it. The best we seem to have done is: https://robotics-transformer-x.github.io/
I don't believe there's any way to handle the open-ended complexity of the real world other than data. Whether that comes from human demonstrations or self-collected with reinforcement learning, there's just no way around it.
But I disagree there has been no progress. Check out this lecture from Sergey Levine about sample-efficient RL in the real world (no simulator): https://www.youtube.com/watch?v=17NrtKHdPDw
Problem is behaviour cloning doesnt generalize, and even on tasks where you have infinite optimal data (solved board games) it yields lower scores than self play. (and still doesnt generalize to unseen states)
Sim to real, self play, and curriculum learning have yielded superhuman performances when done correctly in situations where they can be done. Behaviour cloning doesnt.
> Basically, no real breakthrough has happened in robotics and everyone is hoping to be able to replicate LLMs/vision path for robotics (i.e. data). But no one really knows how to even collect so much high quality data and no one is even remotely close to executing it. The best we seem to have done is: https://robotics-transformer-x.github.io/
ALOHA + RT-X is a great combo to collect a lot of somewhat uniform data to enable such a breakthrough.
Historically, this has not been the case. Collecting data enabled speech recognition, image recognition (and subsequent breakthroughs), machine translation.
What is so special about Robotics that collecting data will not enable anything?
This is true. But Waymo is dealing with the same kind of challenges and it seems that they are already at a point, where their technology (enabled, among other things, by collecting tons of data) is useful.
I expect Robotics (in particular, solving manipulation) to follow the same route: super expensive R&D, multiple false starts, but eventually (tons of data => breakthrough to get smarter robots).
Wondering whether some combination of rfid, volume mapping, to map (learn) an apartment/home could help bridge the gap, on home automation. Could the stove, washers, driers, furniture... 'help' ? I guess it's been tried to death but I lack a good view of SOTA on home automation and robotics.
I don't think it's necessary for an open-ended system.
This is why I really hope some highly popular consumer product (smartphone, smart glasses etc.) starts having high quality LiDAR sensors so we have a lot of 3D training data.
> we are able to achieve over 80% success on these tasks with only 50 human demonstrations per task
The challenge is that 80% accuracy is a far cry from being actually useful in real-world situations to the point where humans can rely on them to perform the desired task correctly. They have to "just work". And closing the gap on that last 20% becomes exponentially difficult.
I've spend some time imaging about the day when general purpose cooking robots are inexpensive enough to be as ubiquitous as indoor plumbing.
How would it affect peoples' diets? Would we eat more healthily because our robot can easily cook delicious and nutritious meals? Or would we have our robots make us delicious unhealthy meals?
There are some genuinely delicious foods that are healthier than junk food and most people would
agree tastes better, but they don’t want to cook or don’t want to learn. A robot that excels at cooking can serve you a delicious, healthy meal from a different cuisine every day. Theoretically you can go for years without having the same meal twice. The robot can also take your target macros and nutrition into consideration when deciding on what to cook and how much of it.
I suspect it'd help people vary up their diets more at least, since they can ask for something new rather than rehashing the same 5-10 meal types over and over again. Would that make for healthier food? Maybe, because at least part of why at least some people eat unhealthily is because it's easy to cook the same few things over and over again.
One problem is that experts can’t even agree on what a “healthy” diet is. See Vegan vs. Low-Carb/Carnivore. For every documentary “proving” that one is better than the other, there is another one “proving” the opposite. There are doctors, professors of nutrition, corruption, paid for research results, and people who have dramatically improved their health on both sides of the battle line. Nutritional science nowadays reminds me more of politics or religion than science. With people screaming at each other from different camps.
I think this has a really good use case in homes of people who have physical limitations. The robot can augment their reach / mobility / dexterity, while they are still able to control it and improve the problem-solving abilities of the system.
Kind of like a spatially distributed cyborg. Imagine being confined in bed or in a wheelchair, and have this thing be your gofer / nimble hands. It could be a gamechanger for so many!
Do middle-aged (ie <45 years old) people realize that these robots will be their primary caregivers when they are old? People are having fewer children and populations are shrinking in most places. For increasingly large numbers of people, there really won't be anyone around to take care of you or help you out when you're old (ie, >70). You're either gonna have a humanoid to help you live independently, or you're gonna have a bad time.
I am pretty sure I've seen videos of commercial robots power washing public bathrooms. Probably not a solution for normal bathrooms with wood furniture and fabric items but it is definitely possible today.
In Scandinavia you get a lot of ceramics, such that a shower+toilet can effectively be the "water room". I'd think that ceramic tiles can be tackled by a 'bot. A flat, unyielding surface, usually with a light color that provides high contrast with gunk. Bring on the ScrubBots(TM).
As a Norwegian in the UK, I've spent 24 years despairing over - and annoying anyone within earshot by talking about - the lack of proper wetrooms in most UK houses. In fact, a lot of British builders seem to think making floors waterproof requires dark magic.
It's slowly getting better, but it's still confusing enough of them.
Reminds me, I read once about a guy who made a plastic-lined living room with plastic furniture and a drain in the floor. Post-party cleanup was just pointing a hose at it all.
Others mention autonomous mode thing being overhyped somewhat.
But besides, that starts to remind me of home automation in Ray Bradbury's "There Will Come Soft Rains"... There was also a Soviet animation film based on that short story https://www.youtube.com/watch?v=5LNHYz89sNc
Having a robot to solve that problem seems like overengineering - there are integrated washing-machine/dryer combos in one appliance, bit less efficient then separate devices, but solves the problem of transferring the clothes.
Perhaps I am not updated with the times but last I checked combo machines have no proper lint filters and are a lot more messy in terms of end result dust/fibers.
A reminder that so called engineers will bleed you dry before anything. As with many robotics projects, the people hired excel in one thing:doing what they already know. That is why you and I never see new robots.
You're looking in the wrong place. Give someone $500 million to build a pizza-making robot and they'll build a company designed to justify spending $500 million. Give a bunch of top tier engineering/CS students some budget for printer filament and pizza toppings over the summer holidays and they'll probably build you a functional pizza robot.
From https://www.trossenrobotics.com/aloha.aspx it looks like it's derived from "A Low-cost Open-source Hardware System for Bimanual Teleoperation" without the S, B or T.
The Github repository is more useful.[1] That separates autonomous from teleoperated mode. Also, there's video at 1x speed. This thing is actually very slow in autonomous mode.
It's real progress, but the hype exceeds the results.
[1] https://tonyzhaozh.github.io/aloha/