By Ahti Heinla, Co-Founder and CTO of Starship Technologies
I see robots every day. I see them sliding down the sidewalks at pedestrian speed, stopping to make sure it is safe to cross the road. Sometimes I even catch them talking to pedestrians. It’s a glimpse into the fantasies of the technological mind – an AI wonderland. But this is not a hallucination, not a dream, it is a reality that our team of dedicated visionaries have built over the past 5 years; we have brought the future to the present.
Until a few years ago, these robots needed a little human support and were accompanied in their movements, much like the format followed by the manufacturers of autonomous cars, which test their cars in public using ” safety conductors ”.
Starship became the first robotics team to start operating regularly in public spaces about 18 months ago, without the use of security drivers; we let our robots explore the world on their own. We now operate our network of robots on a daily basis in cities around the world, bringing people their meals, packages and groceries.
Shared knowledge is acquired knowledge
It’s exciting to be the first.
When I was a founding engineer at Skype, we were the first to make VoIP conveniently accessible; we are now working to do the same with robots in public spaces. For four years, our engineering teams have been working behind closed doors on what has been a significant breakthrough and an incredible experience.
I would like to share with you some details of our technical journey. Over the coming weeks and months, other members of the Starship engineering team will also share aspects of their journey.
Throughout this journey we have worked with computer vision, path planning, and obstacle detection – topics that are the subject of extensive research in academic robotics. Indeed, Starship started as a research project, but quickly grew into a functional and practical delivery operation.
This means that in addition to refining the Levenberg-Marquardt algorithm for nonlinear optimization, we had to develop software for:
- Automatically calibrate most of our sensors – after all, we don’t want to spend hours calibrating them by hand; we have manufactured hundreds of robots and we are now preparing for a larger scale operation.
- Predict how much power each trip will draw from a robot’s battery – so we can orchestrate which robot to send, based on its battery status.
- Predict how many minutes it takes for a restaurant to prepare food – so the robot will appear just in time!
Most of the autonomous robots that exist in the world today are expensive, they are built as technology demonstrators or research vehicles, and are not used for commercial operations. A set of sensors alone for a stand-alone device can cost up to $ 10,000. It just won’t work in the delivery space, it’s not a luxury industry where you can charge a premium.
Autonomous driving research vehicles often have 3 kilowatts of computing power in the trunk; impractical for a small, safe delivery robot. Therefore, part of our engineering journey has been to design lower units for the economy. Here are some topics we had to consider:
- Advanced image processing on a low-end computing platform.
- Troubleshoot hardware problems in the software.
- Track how often robots need maintenance and why.
- Develop advanced route planning systems, to ensure that we are using our robot network efficiently.
It was quite a journey in visual design too, involving hundreds of sketches, drawings and surveys before we made our robot’s first plastic body.
Back in the days when we were still in stealth mode, we didn’t want to reveal what our robots looked like. Regular public testing required the creative use of a trash bag, stuck to the robot’s body as a disguise!
Building practical robotics is a mixture of science, systematic engineering, and computer hacking. This mixture of various disciplines is the main characteristic of Starship. Nothing is ever easy in robotics. All of your situational awareness is probabilistic; all sensors have failure modes and faults, and even a seemingly simple task such as make the robot stop at obstacles can become its own little research project.
Starship is a quick start-up, and it’s important not to become just a research project. Engineers who get excited about Starship are often not pure scientists, nor pure hackers, nor pure engineers; they have many of these traits and can use them as appropriate for the task at hand. We need complex technical solutions that can be implemented quickly and within low cost hardware resources.
Ingenuity and resourcefulness are appreciated skills.
A week is a long time at Starship
At the start of the week our team will implement a new algorithm to detect point cloud borders and test it against a full test case database overnight, they will have it tested live in our field. private trial by the end of the week.
It will be on the streets next Monday, with the team already reporting on their progress at our engineering meeting on Monday. Most Mondays, some members of the engineering team report a gain of over 300% on at least one of the metrics that were achieved just the week before.
Data as a result and facilitator of scale
Metrics and data have become an important part of Starship’s engineering.
See, back when we were just starting, we didn’t have any data – we hadn’t driven much yet. Every day we modified our robot (yes, just the one back then), took it to the sidewalks and saw how it worked. We now have a lot of them, driving autonomously every day – too many for engineers to observe directly.
Thanks to the data, we can now see how our robots work, hundreds of them. We can run weekly “data diving” seminars, where engineers share their findings and observe random deliveries to keep in touch with their work in action.
When we work on making our robots more fluid, we analyze the data in the “acceleration events” table in our data warehouse; there are at least 1 billion rows in this table. Other tables include “road crossing events”, our maps, every order every robot has ever received from our servers and, of course, the data collected with every delivery they make.
Four years ago we had none of this. Back when we were just starting out – and we weren’t doing commercial deliveries yet – I often had to convince people that robotic delivery really worked. People found it hard to believe it and were quick to point out various reasons.
Do skepticism and fear always accompany new technologies?
Several years ago, I landed at JFK Airport in New York with a robot in my luggage. The customs officer obviously asked, “What is this thing?” I explained it was a sidewalk delivery robot, to which he replied, “Dude, this is New York! It will be stolen in minutes!
Indeed, at the time, almost everyone thought these robots would be stolen – I’m sure they probably will (postal delivery vans are stolen, though rarely). To date, our robots have traveled over 200,000 km (130,000 miles) and we have yet to see this problem.
There are of course safety devices in place. The robot has a siren and 10 cameras, it is constantly connected to the Internet and knows its precise location with an accuracy of 2 cm (thanks to the Levenberg-Marquardt algorithm mentioned above, and to the 66,000 lines of Automatically generated C ++ code that allow our robots to use it).
People also believed that pedestrians might be afraid of robots on the sidewalk or not accept their presence. Will people call the police? To be honest, we weren’t sure either! However, once we got one of the robots out onto the sidewalk, we had one hell of a surprise.
What happened next surprised us: people just ignored it. The vast majority of the public paid no attention to the robots, even those who saw them for the first time, and people certainly weren’t afraid. Others would pull out their phones and post on Instagram how they envisioned the future.
And that’s what we wanted.
We want people to pay as much attention to our robots as they do to their dishwashers. This pattern of silent acceptance of robots as if they had always been with us has been repeated in every city in the world where we have operated.
It gets better. Once people learn that these robots are providing a useful service to the neighborhood, they develop an affinity with them. Kids even write letters to thank the robots, we have a “thank you letter wall” to prove it!
Automating last mile delivery was never going to be easy, and we knew it would be a bold move. We also knew from the start that there would be more than one fundamental hurdle to resolve – there were hundreds of them! But we realized a long time ago that all of these problems can be solved – they only require ingenuity and persistence.
Some startups start out as a sprint, launching a minimum viable product in 3 months. For Starship, it feels more like a marathon – constant big effort is required, but the end result brings huge benefits to the world.
Last mile delivery is one of the global industries that has seen little technological disruption since the adoption of the automobile. The Starship team is looking to change that, and with over 20,000 deliveries to our credit, we’re on the right track.
If you’d like to learn more, check out our second engineering blog post on Neural Networks and How They Power Our Robots here – https://medium.com/starshiptechnologies/how-neural-networks-power-robots-at -starship -3262cd317ec0
Please feel free to contact us for more detail about us, visiting our Contact page.