Suddenly all this focus on world models by Deep mind starts to make sense. I've never really thought of Waymo as a robot in the same way as e.g. a Boston Dynamics humanoid, but of course it is a robot of sorts.
Google/Alphabet are so vertically integrated for AI when you think about it. Compare what they're doing - their own power generation , their own silicon, their own data centers, search Gmail YouTube Gemini workspace wallet, billions and billions of Android and Chromebook users, their ads everywhere, their browser everywhere, waymo, probably buy back Boston dynamics soon enough (they're recently partnered together), fusion research, drugs discovery.... and then look at ChatGPT's chatbot or grok's porn. Pales in comparison.
Google has been doing more R&D and internal deployment of AI and less trying to sell it as a product. IMHO that difference in focus makes a huge difference. I used to think their early work on self-driving cars was primarily to support Street View in thier maps.
There was a point in time when basically every well known AI researcher worked at Google. They have been at the forefront of AI research and investing heavily for longer than anybody.
It’s kind of crazy that they have been slow to create real products and competitive large scale models from their research.
But they are in full gear now that there is real competition, and it’ll be cool to see what they release over the next few years.
>It’s kind of crazy that they have been slow to create real products and competitive large scale models from their research.
Not really. If Google released all of this first instead of companies that have never made a profit and perhaps never will, the case law would simply be the copyright holders suing them for infringement and winning.
Ex-googler: I doubt it, but am curious for rationale (i know there was a round of PR re: him “coming back to help with AI.” but just between you and me, the word on him internally, over years and multiple projects, was having him around caused chaos b/c he was a tourist flitting between teams, just spitting out ideas, but now you have unclear direction and multiple teams hearing the same “you should” and doing it)
the rebuke is that lack of chaos makes people feel more orderly and as if things are going better, but it doesn't increase your luck surface area, it just maximizes cozy vibes and self interested comfort.
In Assistant having higher-ups spitting ideas and random thoughts ended up in people mistakenly assume that we really wanted to go/do that, meaning that chaos resulted in ill and cancelled projects.
The worst part was figuring what happened way too late. People were having trying to go for promo for a project that didn't launch. Many people got angry, some left, the product felt stale and leadership&management lost trust.
Isn’t that what the parent is describing? “Ill and cancelled projects” <==> “luck surface area”, and “trying to go for promotion” <==> “cozy vibes and self-interested comfort”?
My dynamic range of professional experience is high, dropout => waiter => found startup => acquirer => Google.
You're making an interesting point that I somewhat agree with from the perspective of someone was...clearly a little more feral than his surroundings in Google, and wildly succeeded and ultimately quietly failed because of it.
The important bit is "great man" theory doesn't solve lack of dynamism. It usually makes things worse. The people you read about in newspapers are pretty much as smart as you, for better or worse.
I actually disagreed with the Sergey thing along the same lines, it was being used as a parable for why it was okay to do ~nothing in year 3 and continue avoiding what we were supposed to ship in year 1, because only VPs outside my org and the design section in my org would care.
Not sure if all that rhymes or will make any sense to you at all. But I deeply respect the point you are communicating, and also mean to communicate that there's another just as strong lesson: one person isn't bright enough to pull that off, and the important bit there isn't "oh, he isn't special", it's that it makes you even more careful building organizations that maintain dynamism and creativity.
Yeah people seem to be pretty poor at judging the impact of 'key' people.
E.g. Steve Jobs was absolutely fundamental to the turn around of Apple. Will Brin have this level of incremental impact on the Goog/Alphabet of today? Nah.
The difference is: Apple had one "key person", Jobs, and yes the products he drove made the company successful. Now Jobs has gone I haven't seen anything new.
But if you look at Google, there isn't one key product. There are a whole pile of products that are best in class. Search (cringe, I know it's popular here to say Google search sucks and perhaps it does, but what search engine is far better?), YouTube, Maps, Android, Waymo, GMail, Deep Mind, the cloud infrastructure, translate, lens (OCR) and probably a lot of others I've forgotten. Don't forget Sheets and Docs, which while they have been replicated by Microsoft and others now were first done by Google. Some of them, like Maps, seem to have swapped entire teams - yet continued to be best in class. Predicting Google won't be at the forefront on the next advance seems perilous.
Maybe these products have key people as you call them, but the magic in Alphabet doesn't seem to be them. The magic seems to be Alphabet has some way to create / acquire these keep people. Or perhaps Alphabet just knows how to create top engineering teams that keep rolling along, even when the team members are replaced.
Apple produced one key person, Jobs. Alphabet seems to be a factory creating lots of key people moving products along. But as Google even manages to replace these key people (as they did for Maps) and still keep the product moving, I'm not sure they are the key to Googles success.
I'm in a similar position and generally agree with your take, but the plus side to his involvement is if he believed in your project or viewpoint he would act as the ultimate red tape cutter.
And there is absolutely nothing more valuable at G (no snark)
(cheers, don't read too much signal into my thoughts, it's more negative than I'd intend. Just was aware it was someone going off PR, and doing hero worship that I myself used to do, and was disabused over 7 years there, and would like other people outside to disabuse themselves of. It's a place, not the place)
Please, Google was terrible about using the tech the had long before Sundar, back when Brin was in charge.
Google Reader is a simple example: Googl had by far the most popular RSS reader, and they just threw it away. A single intern could have kept the whole thing running, and Google has literal billions, but they couldn't see the value in it.
I mean, it's not like being able to see what a good portion of America is reading every day could have any value for an AI company, right?
Google has always been terrible about turning tech into (viable, maintained) products.
What's striking is the sheer scale of Epstein's and Maxwell's scheduling and access. The source material makes it hard to even imagine how two people could sustain that many meetings/parties/dinners/victims, across so many places, with such high-profile figures. And, how those figures consistently found the time to meet them.
> It’s kind of crazy that they have been slow to create real products and competitive large scale models from their research.
It’s not that crazy. Sometimes the rational move is to wait for a market to fully materialize before going after it. This isn’t a Xerox PARC situation, nor really the innovator’s dilemma, it’s about timing: turning research into profits when market conditions finally make it viable. Even mammoths like Google are limited in their ability to create entirely new markets.
This take makes even more sense when you consider the costs of making a move to create the market. The organizational energy and its necessary loss in focus and resources limits their ability to experiment. Arguably the best strategy for Google: (1) build foundational depth in research and infrastructure that would be impossible for competition to quickly replicate (2) wait for the market to present a clear new opportunity for you (3) capture it decisively by focusing and exploiting every foundational advantage Google was able to build.
Their unreleased LaMDA[1] famously caused one of their own engineers to have a public crashout in 2022, before ChatGPT dropped. Pre-ChatGPT they also showed it off in their research blog[2] and showed it doing very ChatGPT-like things and they alluded to 'risks,' but those were primarily around it using naughty language or spreading misinformation.
I think they were worried that releasing a product like ChatGPT only had downside risks for them, because it might mess up their money printing operation over in advertising by doing slurs and swears. Those sweet summer children: little did they know they could run an operation with a seig-heiling CEO who uses LLMs to manufacture and distribute CSAM worldwide, and it wouldn't make above-the-fold news.
The front runner is not always the winner. If they were able to keep pace with openai while letting them take all the hits and miss steps, it could pay off.
Time will tell if LLM training becomes a race to the bottom or the release of the "open source" ones proves to be a spoiler. From the outside looking while ChatGPT has brand recognition for the average person who could not tell the difference between any two LLMs google offering Gemini in android phones could perhaps supplant them.
Indeed, none of the current AI boom would’ve happened without Google Brain and their failure to execute on their huge early lead. It’s basically a Xerox Parc do-over with ads instead of printers.
Not true at all. I interacted with Meena[1] while I was there, and the publication was almost three years before the release of ChatGPT. It was an unsettling experience, felt very science fiction.
The surprise was not that they existed: There were chatbots in Google way before ChatGPT. What surprised them was the demand, despite all the problems the chatbots have. The pig problem with LLMs was not that they could do nothing, but how to turn them into products that made good money. Even people in openAI were surprised about what happened.
In many ways, turning tech into products that are useful, good, and don't make life hell is a more interesting issue of our times than the core research itself. We probably want to avoid the valuing capturing platform problem, as otherwise we'll end up seeing governments using ham fisted tools to punish winners in ways that aren't helpful either
The uptake forced the bigger companies to act. With image diffusion models too - no corporate lawyer would let a big company release a product that allowed the customer to create any image...but when stable diffusion et al started to grow like they did...there was a specific price of not acting...and it was high enough to change boardroom decisions
Right. The problem was that people under appreciated ‘alignment’ even before the models were big. And as they get bigger and smarter it becomes more of an issue.
Well, I must say ChatGPT felt much more stable than Meena when I first tried it. But, as you said, it was a few years before ChatGPT was publicly announced :)
It was a surprise to OpenAI too. ChatGPT was essentially a demo app to showcase their API, it was not meant to be a mass consumer product. When you think about it, ChatGPT is a pretty awkward product name, but they had to stick with it.
Quibi would be if someone came in 10 years from now and said "if we put a lot more money behind spitting out content using characters and settings from Hollywood IP than we'll obviously be way more popular than a tech company can be!"
Tesla built something like this for FSD training, they presented many years ago. I never understood why they did productize it. It would have made a brilliant Maps alternative, which country automatically update from Tesla cars on the road. Could live update with speed cameras and road conditions. Like many things they've fallen behind
I love Volvo, am considering buying one in a couple weeks actually, but they're doing nothing interesting in terms of ADAS, as far as I can tell. It seems like they're limited to adaptive cruise control and lane keeping, both of which have been solved problems for more than a decade.
It sounds like they removed Lidar due to supplier issues and availability, not because they're trying to build self-driving cars and have determined they don't need it anymore.
Is lane keeping really a solved problem? Just last year one of my brand new rented cars tried to kill me a few times when I tried it again, and so far not even the simple lane leaving detection mechanism worked properly in any of the tried cars when it was raining.
Adaptive cruise control requires some degree of lane detection. It has to figure out what car it's actually following, not merely what car is in front of it. (The road is turning, the car in front of you can easily not be the car you are actually behind.)
Lane keep keeps your car in the lane so you can stop paying attention just like cruise control keeps you going the same speed so you can stop paying attention… they don’t.
They are just aids that ease fatigue on long trips.
Lane keep is absolutely not a solved problem. Go test drive any of the latest cars from Kia, Honda, Toyota, Hyundai, Ford, etc. They all will literally kill you.
I’d suggest doing some research on software quality. Two years back I was all for buying one (I was considering an EX40), but I got myself into some Facebook groups for owners and was shocked at the dreadful reports of quality of the software and it completely put me off. I got an ID4 instead. Reports about the EX90 have been dreadful. I was very interested, and I still admire their look and build when they drive by - but it killed my enthusiasm to buy one for a few years until they get it right.
Without Lidar + the terrible quality of tesla onboard cameras.. street view would look terrible. The biggest L of elon's career is the weird commitment to no-lidar. If you've ever driven a Tesla, it gives daily messages "the left side camera is blocked" etc.. cameras+weather don't mix either.
At first I gave him the benefit of the doubt, like that weird decision of Steve Jobs banning Adobe Flash, which ran most of the fun parts of the Internet back then, that ended up spreading HTML5. Now I just think he refused LIDAR on purely aesthetic reasons. The cost is not even that significant compared to the overall cost of a Tesla.
It's important to understand the timeline of the Steve Jobs open letter on Adobe Flash - at that point the iPhone had been out just shy of three years, and before the first public betas on Android. So for nearly three years, Apple had been investing in HTML5 technology because Flash wasn't in a form where it was deployable.
Additionally, Flash required android phones with 256MB ram as a minimum (which would have precluded two of the three shipped iPhone models at the time) and at least initially only supported software video decoding. Because of the difference in screen dimensions, resolutions and interaction models (plus the issues with embedding due to RAM limitations), the website was still basically broken whether your mobile phone had Flash or not.
My understanding (based on the timing) was always that when Adobe was finally ready to push its partners to bundle mobile Flash, Apple looked at it and decided against it. Adobe made public statements against their partner and so Jobs did so in kind.
That one was motivated by the need of controlling the app distribution channel, just like they keep the web as a second class citizen in their ecosystem nowadays.
People aren't setting them on fire during protests, and if an FSD Tesla plows into a farmers market, it might not even make the news.
People hate tech so much that self-driving companies with easy-to-spot cars have had to shut down after just a few mistakes.
Disguising Teslas as plain old regular human-driven cars is a great idea and I wouldn't be surprised if they win the market because of this. Even if they suck at driving.
he didn't refuse it. MobileEye or whoever cut Tesla off because they were using the lidar sensors in a way he didn't approve. From there he got mad and said "no more lidar!"
I think Elon announced Tesla was ditching LIDAR in 2019.[0] This was before Mobileye offered LIDAR. Mobileye has used LIDAR from Luminar Technologies around 2022-2025. [1][2] They were developing their own lidar, but cancelled it. [3] They chose Innoviz Technologies as their LIDAR partner going forward for future product lines. [4]
The original Mobileye EyeQ3 devices that Tesla began installing in their cars in 2013 had only a single forward facing camera. They were very simple devices, only intended to be used for lane keeping. Tesla hacked the devices and pushed them beyond their safe design constraints.
Then that guy got decapitated when his Model S drove under a semi-truck that was crossing the highway and Mobileye terminated the contract. Weirdly, the same fatal edge case occurred 2 more times at least on Tesla's newer hardware.
When Tesla debuted, the cost of batteries made electric cars more like an expensive novelty. The Tesla roadster certainly was fun, but it wasn't a practical car for day-to-day use.
Of course, things have changed.
Had Tesla gone all-in on Lidar, they could have turned the technology into a commodity, they are a trillion dollar company producing a million cars a year. Lidar is already present on cheap robot vacuum cleaners, and we have time-of-flight cameras in smartphones, I don't believe it would have been a problem to equip $50k cars with Lidar.
His stated reason was that he wanted the team focused on the driving problem, not sensor fusion "now you have two problems" problems. People assumed cost was the real reason, but it seems unfair to blame him for what people assumed. Don't get me wrong, I don't like him either, but that's not due to his autonomous driving leadership decisions, it's because of shitting up twitter, shitting up US elections with handouts, shitting up the US government with DOGE, seeking Epstein's "wildest party," DARVO every day, and so much more.
Sensor fusion is an issue, one that is solvable over time and investment in the driving model, but sensor-can't-see-anything is a show stopper.
Having a self-driving solution that can be totally turned off with a speck of mud, heavy rain, morning dew, bright sunlight at dawn and dusk.. you can't engineer your way out of sensor-blindness.
I don't want a solution that is available to use 98% of the time, I want a solution that is always-available and can't be blinded by a bad lighting condition.
I think he did it because his solution always used the crutch of "FSD Not Available, Right hand Camera is Blocked" messaging and "Driver Supervision" as the backstop to any failure anywhere in the stack. Waymo had no choice but to solve the expensive problem of "Always Available and Safe" and work backwards on price.
> Waymo had no choice but to solve the expensive problem of "Always Available and Safe"
And it's still not clear whether they are using a fallback driving stack for a situation where one of non-essential (i.e. non-camera (1)) sensors is degraded. I haven't seen Waymo clearly stating capabilities of their self-driving stack in this regard. On the other hand, there are such things as washer fluid and high dynamic range cameras.
(1) You can't drive in a city if you can't see the light emitted by traffic lights, which neither lidar nor radar can do.
Hence why both together make the solution waymo chose. The proof is in the pudding, Waymo's have been driving millions of miles without any intervention. Tesla requires safety drivers. I would never trust the FSD on my model 3 to be even nearly perfect all the time.
Lidar also gives you the ability to see through fog and as it scans, see the depth needed to nearly always understand what object is in front of them.
My Model 3 shows "degraded" or "unavailable" about 2% of the time i'm driving around populated areas. Zero chance it will ever be truly FSD capable, no matter the software improvements. It'll still be unavailable because the cameras are blinded/blocked/unable to process the scene because it can't see the scene.
While you're right, washer fluid works usually on the windshield, it doesn't on the side cameras, and yea hdr could improve things, it won't improve depth perception, and this will never be installed on my model 3..
Lidar contributes the data most needed to handle the millions of edge cases that exist. With both camera and lidar contributing the data they are both the best at collecting, the risk of the very worst type of accidents is greatly reduced.
but with occasional remote guidance (Waymo doesn't seem to disclose statistics of that). In some cases remote guidance includes placing waypoints[1].
> Lidar also gives you the ability to see through fog and as it scans
Nah. Lidar isn't much better in fog than cameras. If I'm not mistaken, fog, rain, smoke, snow scatter IR light approximately the same as visible light. The lidar beam needs to travel twice the distance and its power is limited by eye-safety concerns.
> FSD on my model 3 to be even nearly perfect all the time
It doesn't need to be perfect. It needs to not hit things, cars and pedestrians too hard and too often, while mostly obeying traffic rules. Waymo has quite a few complains about their cars' behavior[2], but they manage just fine.
Waymo had safety drivers for a long time. And still have safety drivers to this day when they roll out a new city. You wouldn't have known that because no one was paying attention to this stuff back then.
All you really need is "drive slower if you can't see (because rain, fog, or degraded cameras), or you're in an area where children might run out into the road"
If you have mud on a camera, you can't drive it either way. Lidar or not. The way to actually solve these issues is to have way more cameras for redundancy / self cleaning etc, not other sensors.
Yeah its absurd. As a Tesla driver, I have to say the autopilot model really does feel like what someone who's never driven a car before thinks it's like.
Using vision only is so ignorant of what driving is all about: sound, vibration, vision, heat, cold...these are all clues on road condition. If the car isn't feeling all these things as part of the model, you're handicapping it. In a brilliant way Lidar is the missing piece of information a car needs without relying on multiple sensors, it's probably superior to what a human can do, where as vision only is clearly inferior.
Tesla went nothing-but-nets (making fusion easy) and Chinese LIDAR became cheap around 2023, but monocular depth estimation was spectacularly good by 2021. By the time unit cost and integration effort came down, LIDAR had very little to offer a vision stack that no longer struggled to perceive the 3D world around it.
Also, integration effort went down but it never disappeared. Meanwhile, opportunity cost skyrocketed when vision started working. Which layers would you carve resources away from to make room? How far back would you be willing to send the training + validation schedule to accommodate the change? If you saw your vision-only stack take off and blow past human performance on the march of 9s, would you land the plane just because red paint became available and you wanted to paint it red?
I wouldn't completely discount ego either, but IMO there's more ego in the "LIDAR is necessary" case than the "LIDAR isn't necessary" at this point. FWIW, I used to be an outspoken LIDAR-head before 2021 when monocular depth estimation became a solved problem. It was funny watching everyone around me convert in the opposite direction at around the same time, probably driven by politics. I get it, I hate Elon's politics too, I just try very hard to keep his shitty behavior from influencing my opinions on machine learning.
> but monocular depth estimation was spectacularly good by 2021
It's still rather weak and true monocular depth estimation really wasn't spectacularly anything in 2021. It's fundamentally ill posed and any priors you use to get around that will come to bite you in the long tail of things some driver will encounter on the road.
The way it got good is by using camera overlap in space and over time while in motion to figure out metric depth over the entire image. Which is, humorously enough, sensor fusion.
It was spectacularly good before 2021, 2021 is just when I noticed that it had become spectacularly good. 7.5 billion miles later, this appears to have been the correct call.
What are the techniques (and the papers thereof) that you consider to be spectacularly good before 2021 for depth estimation, monocular or not?
I do some tangent work from this field for applications in robotics, and I would consider (metric) depth estimation (and 3D reconstruction) starting to be solved only by 2025 thanks to a few select labs.
Car vision has some domain specificity (high similarity images from adjacent timestamps, relatively simpler priors, etc) that helps, indeed.
depth estimation is but one part of the problem— atmospheric and other conditions which blind optical visible spectrum sensors, lack of ambient (sunlight) and more. lidar simply outperforms (performs at all?) in these conditions. and provides hardware back distance maps, not software calculated estimation
Lidar fails worse than cameras in nearly all those conditions. There are plenty of videos of Tesla's vision-only approach seeing obstacles far before a human possibly could in all those conditions on real customer cars. Many are on the old hardware with far worse cameras
There's a misconception that what people see and what the camera sees is similar. Not true at all. One day when it's raining or foggy, have some record the driving, through the windshield. You'll be very surprised. Even what the camera displays on the screen isn't what it's actually "seeing".
Always thought the case was for sensor redundancy and data variety - the stuff that throws off monocular depth estimation might not throw off a lidar or radar.
Monocular depth estimation can be fooled by adversarial images, or just scenes outside of its distribution. It's a validation nightmare and a joke for high reliability.
It isn't monocular though. A Tesla has 2 front-facing cameras, narrow and wide-angle. Beyond that, it is only neural nets at this point, so depth estimation isn't directly used; it is likely part of the neural net, but only the useful distilled elements.
How many of the 70 human accidents would be adequately explained by controlling for speed, alcohol, wanton inattention, etc? (The first two alone reduce it by 70%)
No customer would turn on FSD on an icy road, or on country lanes in the UK which are one lane but run in both directions; it's much harder to have a passenger fatality in stop-start traffic jams in downtown US cities.
Even if those numbers are genuine (2 vs 70) I wouldn't consider it apples-for-apples.
Public information campaigns and proper policing have a role to play in car safety, if that's the stated goal we don't necessarily need to sink billions into researching self driving
There are a sizeable number of deaths associated with the abuse of Tesla’s adaptive cruise control with lane cantering (publicly marketed as “autopilot”). Such features are commonplace on many new cars and it is unclear whether Tesla is an outlier, because no one is interested in obsessively researching cruise control abuse among other brands.
Good ole Autopilot vs FSD post. You would think people on Hacker News would be better informed. Autopilot is just lane keep and adaptive cruise control. Basically what every other car has at this point.
"MacOS Tahoe has these cool features". "Yea but what about this wikipedia article on System 1. Look it has these issues."
Isn't there a great deal of gaming going on with the car disengaging FSD milliseconds before crashing? Voila, no "full" "self" driving accident; just another human failing [*]!
[*] Failing to solve the impossible situation FSD dropped them into, that is.
Seeing how its by a lidar vendor, I don't think they're biased against it. It seems Lidar is not a panacea - it struggles with heavy rain, snow, much more than cameras do and is affected by cold weather or any contamination on the sensor.
So lidar will only get you so far. I'm far more interested in mmwave radar, which while much worse in spatial resolution, isn't affected by light conditions, weather, can directly measure stuff on the thing its illuminating, like material properties, the speed its moving, the thickness.
Fun fact: mmWave based presence sensors can measure your hearbeat, as the micro-movements show up as a frequency component. So I'd guess it would have a very good chance to detect a human.
I'm pretty sure even with much more rudimentary processing, it'll be able to tell if its looking at a living being.
By the way: what happened to the idea that self-driving cars will be able to talk to each other and combine each other's sensor data, so if there are multiple ones looking at the same spot, you'd get a much improved chance of not making a mistake.
Lidar is a moot point. You can't drive with just Lidar, no matter what. That's what people don't understand. The most common one I hear: "What if the camera gets mud on it", ok then you have to get out and clean it, or it needs an auto cleaning system.
Maybe vision-only can work with much better cameras, with a wider spectrum (so they can see thru fog, for example), and self-cleaning/zero upkeep (so you don't have to pull over to wipe a speck of mud from them). Nevertheless, LIDAR still seems like the best choice overall.
Yep, and won't activate until any morning dew is off the sensors.. or when it rains too hard.. or if it's blinded by a shiny building/window/vehicle.
I will never trust 2d camera-only, it can be covered or blocked physically and when it happens FSD fails.
As cheap as LIDAR has gotten, adding it to every new tesla seems to be the best way out of this idiotic position. Sadly I think Elon got bored with cars and moved on.
If the camera is covered or blocked, you can't drive plain and simple, as you can't drive a car (at least on Earth) with just Lidar. The roads are made for eyes. Maybe on Rocky's homeworld you can have a Lidar only system for traveling.
From the perspective of viewing FSD as an engineering problem that needs solving I tend to think Elon is on to something with the camera-only approach – although I would agree the current hardware has problems with weather, etc.
The issue with lidar is that many of the difficult edge-cases of FSD are all visible-light vision problems. Lidar might be able to tell you there's a car up front, but it can't tell you that the car has it's hazard lights on and a flat tire. Lidar might see a human shaped thing in the road, but it cannot tell whether it's a mannequin leaning against a bin or a human about to cross the road.
Lidar gets you most of the way there when it comes to spatial awareness on the road, but you need cameras for most of the edge-cases because cameras provide the color data needed to understand the world.
You could never have FSD with just lidar, but you could have FSD with just cameras if you can overcome all of the hardware and software challenges with accurate 3D perception.
Given Lidar adds cost and complexity, and most edge cases in FSD are camera problems, I think camera-only probably helps to force engineers to focus their efforts in the right place rather than hitting bottlenecks from over depending on Lidar data. This isn't an argument for camera-only FSD, but from Tesla's perspective it does down costs and allows them to continue to produce appealing cars – which is obviously important if you're coming at FSD from the perspective of an auto marker trying to sell cars.
Finally, adding lidar as a redundancy once you've "solved" FSD with cameras isn't impossible. I personally suspect Tesla will eventually do this with their robotaxis.
That said, I have no real experience with self-driving cars. I've only worked on vision problems and while lidar is great if you need to measure distances and not hit things, it's the wrong tool if you need to comprehend the world around you.
This is so wild to read when Waymo is currently doing like 500,000 paid rides every week, all over the country, with no one in the driver's seat. Meanwhile Tesla seems to have a handful of robotaxis in Austin, and it's unclear if any of them are actually driverless.
But the Tesla engineers are "in the right place rather than hitting bottlenecks from over depending on Lidar data"? What?
Tesla has driven 7.5B autonomous miles to Waymo's 0.2B, but yes, Waymo looks like they are ahead when you stratify the statistics according to the ass-in-driver-seat variable and neglect the stratum that makes Tesla look good.
The real question is whether doing so is smart or dumb. Is Tesla hiding big show-stopper problems that will prevent them from scaling without a safety driver? Or are the big safety problems solved and they are just finishing the Robotaxi assembly line that will crank out more vertically-integrated purpose-designed cars than Waymo's entire fleet every day before lunch?
waymo just hit it's first pedestrian, ever. It did it at a speed of 6mph and it was estimated a human would have hit the kid at 14mph (it was going 17mph when a small child jumped out in front of it from behind a black suv.
First pedestrian struck. That's crazy.
Tesla just disengages fsd anytime a sensor is slightly blocked/covered/blinded.. waymo out here doing fsd 100% of the time and basically never hurts anyone.
I don't get the tesla/elon love here, i like my model 3 but it's never going to get real fsd, and that sucks, elon also lies about the roadmap, timing, etc. I bet the roadster is canceled now. Why do people like inferior sensors and autistic hitler?
Not really. Waymos can’t be driven remotely, their remote operators can give the car directions, e.g. “use this lane”, and then the autonomous system controls the vehicle to execute those directions.
I’m sure latency and connectivity is too much of an risk to do it any other way.
The only Waymos driven by a human are the ones with human drivers physically in the car
There's more Tesla's on the road than Waymo's by several orders of magnitude. Additionally the types of roads and conditions Tesla's drive under is completely incomparable to Waymo.
I wasn't arguing Tesla is ahead of Waymo? Nor do I think they are. All I was arguing was that it makes sense from the perspective of a consumer automobile maker to not use lidar.
I don't think Tesla is that far behind Waymo though given Waymo has had a significant head start, the fact Waymo has always been a taxi-first product, and given they're using significantly more expensive tech than Tesla is.
Additionally, it's not like this is a lidar vs cameras debate. Waymo also uses and needs cameras for FSD for the reasons I mentioned, but they supplement their robotaxis with lidar for accuracy and redundancy.
My guess is that Tesla will experiment with lidar on their robotaxis this year because design decisions should differ from those of a consumer automobile. But I could be wrong because if Tesla wants FSD to work well on visually appealing and affordable consumer vehicles then they'll probably have to solve some of the additional challenges with with a camera-only FSD system. I think it will depend on how much Elon decides Tesla needs to pivot into robotaxis.
Either way, what is undebatable is that you can't drive with lidar only. If the weather is so bad that cameras are useless then Waymos are also useless.
What causes LiDAR to fail harder than normal cameras in bad weather conditions? I understand that normal LiDAR algorithms assume the direct paths from light source to object to camera pixel, while a mist will scatter part of the light, but it would seem like this can be addressed in the pixel depth estimation algorithm that combines the complex amplitudes at the different LiDAR frequencies.
I understand that small lens sizes mean that falling droplets can obstruct the view behind the droplet, while larger lens sizes can more easily see beyond the droplet.
I seldom see discussion of the exact failure modes for specific weather conditions. Even if larger lenses are selected the light source should use similar lens dimensions. Independent modulation of multiple light sources could also dramatically increase the gained information from each single LiDAR sensor.
Do self-driving camera systems (conventional and LiDAR) use variable or fixed tilt lenses? Normal camera systems have the focal plane perpendicular to the viewing direction, but for roads it might be more interesting to have a large swath of the horizontal road in focus. At least having 1 front facing camera with a horizontal road in focus may prove highly beneficial.
To a certain extend an FSD system predicts the best course of action. When different courses of action have similar logits of expected fitness for the next best course of action, we can speak of doubt. With RMAD we can figure out which features or what facets of input or which part of the view is causing the doubt.
A camera has motion blur (unless you can strobe the illumination source, but in daytime the sun is very hard to outshine), it would seem like an interesting experiment to:
1. identify in real time which doubts have the most significant influence on the determination of best course of action
2. have a camera that can track an object to eliminate motion blur but still enjoy optimal lighting (under the sun, or at night), just like our eyes can rotate
3. rerun the best course of action prediction and feed back this information to the company, so it can figure out the cost-benefit of adding a free tracking camera dedicated to eliminating doubts caused by motion blur.
Maybe they were focusing on a real world use that basically requires AI, but not LLMs.
Tesla claimed that all their "real world" recording would give them a moat on FSD.
Waymo is showing that a) you need to be able to incorporate stuff that isn't "real" when training, and b) you get a lot more information from alternate sensors to visible spectrum only.
I just listened to a fantastic multi-hour Acquired (https://www.acquired.fm/) podcast episode on Google and AI that talks about the history of Google and AI and all the ways they have been using it since 2012. It's really fascinating. You can forgive them for not focusing on Reader or any of their other properties when you realize they were pulling in hundreds of billions of dollars of value by making big bets in AI and incorporating it into their core business.
Pretty much. They banked on "if we can solve FSD, we can partially solve humanoid robot autonomy, because both are robots operating in poorly structured real world environments".
They started working on humanoid robots because Musk always has to have the next moonshot, trillion-dollar idea to promise "in 3 years" to keep the stock price high.
As soon as Waymo's massive robotaxi lead became undeniable, he pivoted to from robotaxis to humanoid robots.
Obviously both will exist and compete with each other on the margins. The thing to appreciate is that our physical world is already built like an API for adult humans. Swinging doors, stairs, cupboards, benchtops. If you want a robot to traverse the space and be useful for more than one task, the humanoid form makes sense.
The key question is whether general purpose robots can outcompete on sheer economies of scale alone.
I agree that each would be made slightly better with a more integrated system. But you could handle all of them in my hundred year old house with the form factor it was designed for: a humanoid. Probably pretty soon here for cheaper than each could be handled separately by more integrated systems.
The drop in demand for Tesla's clapped out model range would have meant embarrassing factory closures, so now they're being closed to start manufacturing a completely different product. Bait and switch for Tesla investors.
I wonder how long they'll be closed for "modifications" and whether the Optimus Prime robot factories will go into production before the "Trump Kennedy Center" is reopened after its "renovations".
>> Suddenly all this focus on world models by Deep mind starts to make sense.
The apparent applicability to Waymo is incidental, more likely because a few millions+ were spent on Genie and they have to do something with it. DeepMind started to train "world models" because that's the current overhyped buzzword in the industry. First it was "natural language understanding" and "question answering" back in the days of old BERT, then it was "agentic", then "reasoning", now it's "world models", next years it's going to be "emotions" or "social intelligence" or some other anthropomorphic, over-drawn neologism. If you follow a few AI accounts on social media you really can't miss when those things suddenly start trending, then pretty much die out and only a few stragglers still try to publish papers on them because they failed to get the memo that we're now all running behind the Next Big Thing™.
notice that all these buzzwords you give actually correspond to real advances in the field. All of these were improvements on something existing, not a big revolution for sure, but definitely measurable improvements.
The Next Big Thing™ is going to be "context learning", at least if Tencent have their way. And why do we need that?
>> Current language models do not handle context this way. They rely primarily on parametric knowledge—information compressed into their weights during massive pre-training runs. At inference time, they function largely by recalling this static, internal memory, rather than actively learning from new information provided in the moment.
>> This creates a structural mismatch. We have optimized models to excel at reasoning over what they already know yet users need them to solve tasks that depend on messy, constantly evolving context. We built models that rely on what they know from the past, but we need context learners that rely on what they can absorb from the environment in the moment.
I think you might be salty because the words become overused and overhyped, and often 90% of the people jumping on the bandwagon are indeed just parroting the new hot buzzword and don't really understand what they're talking about. But the terms you mentioned are all obviously very real and and very important in applications using LLMs today. Are you arguing that reasoning was vaporware? None of these things were meant to the be the final stop of the journey, just the next step.
So is this a model baked into the VLLM layer? Or a scaffold that the agent sits in for testing?
If the former then it’s relevant to the broader discourse on LLM generality. If the latter, then it seems less relevant to chatbots and business agents.
What do you think I said that you're contradicting?
IMO the presence of safety chase vehicles is just a sensible "as low as reasonably achievable" measure during the early rollout. I'm not sure that can (fairly) be used as a point against them.
I'm comfortably with Tesla sparing no expense for safety, since I think we all (including Tesla) understand that this isn't the ultimate implementation. In fact, I think it would be a scandal if Tesla failed to do exactly that.
Damned if you do and damned if you don't, apparently.
Setting aside the anti-Tesla bias, none of what I said relies on Tesla claims. The "chase vehicle" claims are all based on third-party accounts from actual rideshare customers.
> IMO the presence of safety chase vehicles is just a sensible "as low as reasonably achievable" measure during the early rollout. I'm not sure that can (fairly) be used as a point against them.
Only if you're comparing them to another company, which you seem to be. So yes, yes it can.
Seriously, the amount of sheer cope here is insane. Waymo is doing the thing. Tesla is not. If Tesla were capable of doing it, they would be. But they're not.
It really is as simple as that and no amount of random facts you may bring up will change the reality. Waymo is doing the thing.
Waymo has (very shrewdly, for prospective investors at least) executed a strategy that most quickly scales to 0.1% of the population. Unfortunately it doesn't scale further. The cars are too costly and the mapping is too costly. There is no workable plan for significant scale from Waymo.
Tesla is executing the strategy that most quickly scales to 100% of the population.
> Tesla is executing the strategy that most quickly scales to 100% of the population.
So, uh… where is this “scale” then? This “strategy” has been bandied about for better part of a decade. Why are they still in a tiny geofence in Austin with chase cars?
Waymo is doing it right now. Half a million rides every week, expansion to a dozen new cities. Tesla does a few hundred in a tiny area.
Scale is assessed by looking at concrete numbers, not by “strategies” that haven’t materialized for a decade.
Practically ALL course introductory materials that regard robotics and AI that I've seen began with "you might imagine a talking bipedal humanoid when you hear the word `robot`, but perhaps the most commonplace robot that you have seen is a vending machine", with the illustration of a typical 80s-90s outdoor soda vendor with no apparent moving parts.
So "maybe cars are a bit of robots too" is more like 30-50 years behind the time.
Erm, a dishwasher, washing machine, automated vacuum can be considered robots. Im confused as to this obsession of the term - there are many robots that already exist. Robotics have been involved in the production of cars for decades.
I think the (gray) line is the degree of autonomy. My washing machine makes very small, predictable decisions, while a Waymo has to manage uncertainty most of the time.
A robot is a robot, and a human is a creature that won't necessarily agree with another human on what the definition of a word is. Dictionaries are also written by humans and don't necessarily reflect the current consensus, especially on terms where people's understanding might evolve over time as technology changes.
Even if that definition were universally agreed on l upon though, that's not really enough to understand what the parent comment was saying. Being a robot "in the same way" as something else is even less objective. Humans are humans, but they're also mammals; is a human a mammal "in the same way" as a mouse? Most humans probably have a very different view of the world than most mice, and the parent comment was specifically addressing the question of whether it makes sense for an autonomous car to model the world the same way as other robots or not. I don't see how you can dismiss this as "irrelevant" because both humans and mice are mammals (or even animals; there's no shortage of classifications out there) unless you're completely having a different conversation than the person you responded to. You're not necessarily wrong because of that, but you're making a pretty significant misjudgment if you think that's helpful to them or to anyone else involved in the ongoing conversation.
No one is denying that robots existed already (but I would hardly call a dishwasher a robot FWIW)
But in my mind a waymo was always a "car with sensors", but more recently (especially having recently used them a bunch in California recently) I've come to think of them truly as robots.
In the same way people online have argued helicopters are flying cars, it doesn't capture what most people mean when they use the word "robot", anymore than helicopters are what people have in mind when they mention flying cars.
I know it’s gross, but I would not discount this. Remember why Blu-ray won over HDDVD? I know it won for many other technical reasons, but I think there are a few historical examples of sexual content being a big competitive advantage.
They couldn't even make burger flipping robots work and are paying fast food workers $20/hr in California.
If that doesn't make it obvious what they can and cannot do then I can't respect the tranche of "hackers" who blindly cheer on this unchecked corporate dystopian nightmare.
The vertical integration argument should apply to Grok. They have Tesla driving data (probably much more data than Waymo), Twitter data, plus Tesla/SpaceX manufacturing data. When/if Optimus starts on the production line, they'll have that data too. You could argue they haven't figured out how to take advantage of it, but the potential is definitely there.
Agreed. Should they achieve Google level integration, we will all make sure they are featured in our commentary. Their true potential is surely just around the corner...
"Tesla has more data than Waymo" is some of the lamest cope ever. Tesla does not have more video than Google! That's crazy! People who repeat this are crazy! If there was a massive flow of video from Tesla cars to Tesla HQ that would have observable side effects.
The key metric is more unusual situations. That scales with miles driven, not gigabytes. With onboard inference the car simply logs anything 'unusual' (low confidence) to selectively upload those needle-in-a-haystack rare events.
But somehow google fails to execute. Gemini is useless for programming and I don’t think even bother to use it as chat app. Claude code + gpt 5.2 xhigh for coding and gpt as chat app are really the only ones that are worth it(price and time wise)
I've recently switched to Claude for chat. GPT 5.2 feels very engagement-maxxed for me, like I'm reading a bad LinkedIn post. Claude does a tiny bit of this too, but an order of magnitude less in my experience. I never thought I'd switch from ChatGPT, but there is only so much "here's the brutal truth, it's not x it's y" I can take.
GPT likes to argue, and most of its arguments are straw man arguments, usually conflating priors. It's ... exhausting; akin to arguing on the internet. (What am I even saying, here!?) Claude's a lot less of that. I don't know if tracks discussion/conversation better; but, for damn sure, it's got way less verbal diarrhea than GPT.
Yes, GPT5-series thinking models are extremely pedantic and tedious. Any conversation with them is derailed because they start nitpicking something random.
But Codex/5.2 was substantially more effective than Claude at debugging complex C++ bugs until around Fall, when I was writing a lot more code.
I find Gemini 3 useless. It has regressed on hallucinations from Gemini 2.5, to the point where its output is no better than a random token stream despite all its benchmark outperformance. I would use Gemini 2.5 to help write papers and all, can't see to use Gemini 3 for anything. Gemini CLI also is very non-compliant and crazy.
To me ChatGPT seems smarter and knows more. That’s why I use it. Even Claude rates gpt better for knowledge answers. Not sure if that itself is any indication. Claude seems superficial unless you hammer it to generate a good answer.
Gemini is by far the best UI/UX designer model. Codex seems to the worst: it'll build something awkward and ugly, then Gemini will take 30-60 seconds to make it look like something that would have won a design award a couple years ago.
It is a bit mind boggling how behind they were considering they invented transformers and were also sitting on the best set of training data in the world, but they've caught up quite a bit. They still lag behind in coding, but I've found Gemini to be pretty good at more general knowledge tasks. Flash 3 in particular is much better than anything of comparable price and speed from OpenAI or Anthropic.
Yesterday GPT 5.2 wrote a python function for me that had the import in the middle of the code, for no reason. (It was a simple import of requests module in a REST client...)
Claude I agree is a lot better for backend,Gemini is very good for frontend
> The Waymo World Model can convert those kinds of videos, or any taken with a regular camera, into a multimodal simulation—showing how the Waymo Driver would see that exact scene.
Subtle brag that Waymo could drive in camera-only mode if they chose to. They've stated as much previously, but that doesn't seem widely known.
I think I'm misunderstanding - they're converting video into their representation which was bootstrapped with LIDAR, video and other sensors. I feel you're alluding to Tesla, but Tesla could never have this outcome since they never had a LIDAR phase.
(edit - I'm referring to deployed Tesla vehicles, I don't know what their research fleet comprises, but other commenters explain that this fleet does collect LIDAR)
I think what we are seeing is that they both converged on the correct approach, one of them decided to talk about it, and it triggered disclosure all around since nobody wants to be seen as lagging.
Exactly: they convert video into a world model representation suitable for 3D exploration and simulation without using LIDAR (except perhaps for scale calibration).
Tesla does collect LIDAR data (people have seen them doing it, it's just not on all of the cars) and they do generate depth maps from sensor data, but from the examples I've seen it is much lower resolution than these Waymo examples.
Human depth perception uses stereo out to only about 2 or 3 meters, after which the distance between your eyes is not a useful baseline. Beyond 3m we use context clues and depth from motion when available.
And I'll add that it in practice it is not even that much unless you're doing some serious training, like a professional athlete. For most tasks, the accurate depth perception from this fades around the length of the arms.
ok, but the point trying to be made is based on human's depth perception, but a car's basic limitation is the width of the vehicle, so there's missing information if you're trying to figure out if a car can use cameras to do what human eyes/brains do.
Humans are very good at processing the images that come into our brain. Each eye has a “blind spot” but we don’t notice. Our eyes adjust color (fluorescent lights are weird) and the amount of light coming in. When we look through a screen door or rain and just ignore it, or if you look outside a moving vehicle to the side you can ignore the foreground.
If you increase the distance of stereo cameras you probably can increase depth perception.
But a lidar or radar sensor is just sensing distance.
Radar has a cool property that it can sense the relative velocity of objects along the beam axis too, from Doppler frequency shifting. It’s one sense that cars have that humans don’t.
To this point, one of the coolest features Teslas _used_ to have was the ability for it to determine and integrate the speed of the car in front of you AND the speed of the car in front of THAT car, even if the second car was entirely visually occluded. They did this by bouncing the radar beam under the car in front and determining that there were multiple targets. It could even act on this: I had my car AEB when the second ahead car slammed on THEIR brakes before the car ahead even reacted. Absolutely wild. Completely gone in vision-only.
The company I used to work for was developing a self driving car with stereo depth on a wide baseline.
It's not all sunshine and roses to be honest - it was one of the weakest links in the perception system. The video had to run at way higher resolutions than it would otherwise and it was incredibly sensitive to calibration accuracy.
(Always worth noting, human depth perception is not just based on stereoscopic vision, but also with focal distance, which is why so many people get simulator sickness from stereoscopic 3d VR)
> Always worth noting, human depth perception is not just based on stereoscopic vision, but also with focal distance
Also subtle head and eye movements, which is something a lot of people like to ignore when discussing camera-based autonomy. Your eyes are always moving around which changes the perspective and gives a much better view of depth as we observe parallax effects. If you need a better view in a given direction you can turn or move your head. Fixed cameras mounted to a car's windshield can't do either of those things, so you need many more of them at higher resolutions to even come close to the amount of data the human eye can gather.
Easiest example I always give of this is pulling out of the alley behind my house: there is a large bush that occludes my view left to oncoming traffic, badly. I do what every human does:
1. Crane my neck forward, see if I can see around it.
2. Inch forward a bit more, keep craning my neck.
3. Recognize, no, I'm still occluded.
4. Count on the heuristic analysis of the light filtering through the bush and determine if the change in light is likely movement associated with an oncoming car.
My Tesla's perpendicular camera is... mounted behind my head on the B-pillar... fixed... and sure as hell can't read the tea leaves, so to speak, to determine if that slight shadow change increases the likelihood that a car is about to hit us.
I honestly don't trust it to pull out of the alley. I don't know how I can. I'd basically have to be nose-into-right-lane for it to be far enough ahead to see conclusively.
Waymo can beam the LIDAR above and around the bush, owing to its height and the distance it can receive from, and its camera coverage to the perpendicular is far better. Vision only misses so many weird edge cases, and I hate that Elon just keeps saying "well, humans have only TWO cameras and THEY drive fine every day! h'yuck!"
> owing to its height and the distance it can receive from,
And, importantly, the fender-mount LIDARs. It doesn't just have the one on the roof, it has one on each corner too.
I first took a Waymo as a curiosity on a recent SF trip, just a few blocks from my hotel east on Lombard to Hyde and over to the Buena Vista to try it out, and I was immediately impressed when we pulled up the hill to Larkin and it saw a pedestrian that was out of view behind a building from my perspective. Those real-time displays went a long way to allowing me to quickly trust that the vehicle's systems were aware of what's going on around it and the relevant traffic signals. Plenty of sensors plus a detailed map of a specific environment work well.
Compare that to my Ioniq5 which combines one camera with a radar and a few ultrasonic sensors and thinks a semi truck is a series of cars constantly merging in to each other. I trust it to hold a lane on the highway and not much else, which is basically what they sell it as being able to do. I haven't seen anything that would make me trust a Tesla any further than my own car and yet they sell it as if it is on the verge of being able to drive you anywhere you want on its own.
In fact there are even more depth perception clues. Maybe the most obvious is size (retinal versus assumed real world size). Further examples include motion parallax, linear perspective, occlusion, shadows, and light gradients
Actually the reason people experience vection in VR is not focal depth but the dissonance between what their eyes are telling them and what their inner ear and tactile senses are telling them.
It's possible they get headaches from the focal length issues but that's different.
I keep wondering about the focal depth problem. It feels potentially solvable, but I have no idea how. I keep wondering if it could be as simple as a Magic Eye Autostereogram sort of thing, but I don't think that's it.
There have been a few attempts at solving this, but I assume that for some optical reason actual lenses need to be adjusted and it can't just be a change in the image? Meta had "Varifocal HMDs" being shown off for a bit, which I think literally moved the screen back and forth. There were a couple of "Multifocal" attempts with multiple stacked displays, but that seemed crazy. Computer Generated Holography sounded very promising, but I don't know if a good one has ever been built. A startup called Creal claimed to be able to use "digital light fields", which basically project stuff right onto the retina, which sounds kinda hogwashy to me but maybe it works?
My understanding is that contextual clues are a big part of it too. We see a the pitcher wind up and throw a baseball as us more than we stereoscopically track its progress from the mound to the plate.
More subtly, a lot of depth information comes from how big we expect things to be, since everyday life is full of things we intuitively know the sizes of, frames of reference in the form of people, vehicles, furniture, etc
. This is why the forced perspective of theme park castles is so effective— our brains want to see those upper windows as full sized, so we see the thing as 2-3x bigger than it actually is. And in the other direction, a lot of buildings in Las Vegas are further away than they look because hotels like the Bellagio have large black boxes on them that group a 2x2 block of the actual room windows.
Hesai has driven the cost into the $200 to 400 range now. That said I don't know what they cost for the ones needed for driving. Either way we've gone from thousands or tens of thousands into the hundreds dollar range now.
Looking at prices, I think you are wrong and automotive Lidar is still in the 4 to 5 figure range. HESAI might ship Lidar units that cheap, but automotive grade still seems quite expensive: https://www.cratustech.com/shop/lidar/
Those are single unit prices. The AT128 for instance, which is listed at $6250 there and widely used by several Chinese car companies was around $900 per unit in high volume and over time they lowered that to around $400.
The next generation of that, the ATX, is the one they have said would be half that cost. According to regulator filings in China BYD will be using this on entry level $10k cars.
Hesai got the price down for their new generation by several optimizations. They are using their own designs for lasers, receivers, and driver chips which reduced component counts and material costs. They have stepped up production to 1.5 million units a year giving them mass production efficiencies.
That model only has a 120 degree field of view so you'd need 3-4 of them per car (plus others for blind spots, they sell units for that too). That puts the total system cost in the low thousands, not the 200 to 400 stated by GP. I'm not saying it hasn't gotten cheaper or won't keep getting cheaper, it just doesn't seem that cheap yet.
That was 2 generations of hardware ago (4th gen Chrysler Pacificas). They are about to introduce 6th gen hardware. It's a safe bet that it's much cheaper now, given how mass produced LiDARs cost ~$200.
Otto and Uber and the CEO of https://pronto.ai do though (tongue-in-cheek)
> Then, in December 2016, Waymo received evidence suggesting that Otto and Uber were actually using Waymo’s trade secrets and patented LiDAR designs. On December 13, Waymo received an email from one of its LiDAR-component vendors. The email, which a Waymo employee was copied on, was titled OTTO FILES and its recipients included an email alias indicating that the thread was a discussion among members of the vendor’s “Uber” team. Attached to the email was a machine drawing of what purported to be an Otto circuit board (the “Replicated Board”) that bore a striking resemblance to – and shared several unique characteristics with – Waymo’s highly confidential current-generation LiDAR circuit board, the design of which had been downloaded by Mr. Levandowski before his resignation.
The presiding judge, Alsup, said, "this is the biggest trade secret crime I have ever seen. This was not small. This was massive in scale."
(Pronto connection: Levandowski got pardoned by Trump and is CEO of Pronto autonomous vehicles.)
> Humans do this, just in the sense of depth perception with both eyes.
Humans do this with vibes and instincts, not just depth perception. When I can't see the lines on the road because there's too much slow, I can still interpret where they would be based on my familiarity with the roads and my implicit knowledge of how roads work, e.g. We do similar things for heavy rain or fog, although, sometimes those situations truly necessitate pulling over or slowing down and turning on your 4s - lidar might genuinely given an advantage there.
Yes and no - vibes and instincts isn't just thought, it's real senses. Humans have a lot of senses; dozens of them. Including balance, pain, sense of passage of time, and body orientation. Not all of these senses are represented in autonomous vehicles, and it's not really clear how the brain mashes together all these senses to make decisions.
I think there are two steps here: converting video to sensor data input, and using that sensor data to drive. Only the second step will be handled by cars on road, first one is purely for training.
It’s way easier to “jam” a camera with bright light than a lidar, which uses both narrow band optical filters and pulsed signals with filters to detect that temporal sequence. If I were an adversary, going after cameras is way way easier.
Autonomous cars need to be significantly better than humans to be fully accepted especially when an accident does happen. Hence limiting yourself to only cameras is futile.
Surely as soon as they're safer than humans they should be deployed as fast as possible to save some of the 3000 people who are killed by human drivers every day
Of course they should be, but that's not what will happen. Humans are not rational, so self-driving cars must be significantly safer than human drivers to avoid as much political pushback as possible.
I've always wondered... if Lidar + Cameras is always making the right decision, you should theoretically be able to take the output of the Lidar + Cameras model and use it as training data for a Camera only model.
That's exactly what Tesla is doing with their validation vehicles, the ones with Lidar towers on top. They establish the "ground truth" from Lidar and use that to train and/or test the vision model. Presumably more "test", since they've most often been seen in Robotaxi service expansion areas shortly before fleet deployment.
I don't have a specific source, no. I think it was mentioned in one of their presentation a few years back, that they use various techniques for "ground truth" for vision training, among those was time series (depth change over time should be continuous etc) and iirc also "external" sources for depth data, like LiDAR. And their validation cars equipped with LiDAR towers are definitely being seen everywhere they are rolling out their Robotaxi services.
"Exactly" is impossible: there are multiple Lidar samples that would map to the same camera sample. But what training would do is build a model that could infer the most likely Lidar representation from a camera representation. There would still be cases where the most likely Lidar for a camera input isn't a useful/good representation of reality, e.g. a scene with very high dynamic range.
No, I don't think that will be successful. Consider a day where the temperature and humidity is just right to make tail pipe exhaust form dense fog clouds. That will be opaque or nearly so to a camera, transparent to a radar, and I would assume something in between to a lidar. Multi-modal sensor fusion is always going to be more reliable at classifying some kinds of challenging scene segments. It doesn't take long to imagine many other scenarios where fusing the returns of multiple sensors is going to greatly increase classification accuracy.
The goal is not to drive in all conditions; it is to drive in all drivable conditions. Human eyeballs also cannot see through dense fog clouds. Operating in these environments is extra credit with marginal utility in real life.
But humans react to this extremely differently than a self driving car.
Humans take responsability, and the self-driving disengages and say : WELP.
Oh sorry were you "enjoying your travel time to do something useful" as we very explicitely marketed ? Well now your wife is dead and it's your fault (legally). Kisses, Elon.
No, but if you run a shadow or offline camera-only model in parallel with a camera + LIDAR model, you can (1) measure how much worse the camera-only model is so you can decide when (if ever) it's safe enough to stop installing LIDAR, and (2) look at the specific inputs for which the models diverge and focus on improving the camera-only model in those situations.
By leveraging Genie’s immense world knowledge, it can simulate exceedingly rare events—from a tornado to a casual encounter with an elephant—that are almost impossible to capture at scale in reality. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data.
How do you know the generated outputs are correct? Especially for unusual circumstances?
Say the scenario is a patch of road is densely covered with 5 mm ball bearings. I'm sure the model will happily spit out numbers, but are they reasonable? How do we know they are reasonable? Even if the prediction is ok, how do we fundamentally know that the prediction for 4 mm ball bearings won't be completely wrong?
There seems to be a lot of critical information missing.
The idea is that, over time, the quality and accuracy of world-model outputs will improve. That, in turn, lets autonomous driving systems train on a large amount of “realistic enough” synthetic data.
For example, we know from experience that Waymo is currently good enough to drive in San Francisco. We don’t yet trust it in more complex environments like dense European cities or Southeast Asian “hell roads.” Running the stack against world models can give a big head start in understanding what works, and which situations are harder, without putting any humans in harm’s way.
We don’t need perfect accuracy from the world model to get real value. And, as usual, the more we use and validate these models, the more we can improve them; creating a virtuous cycle.
I don't think you say "ok now the car is ball bearing proof."
Think of it more like unit tests. "In this synthetic scenario does the car stop as expected, does it continue as expected." You might hit some false negatives but there isn't a downside to that.
If it turns out your model has a blind spot for albino cows in a snow storm eating marshmallows, you might be able to catch that synthetically and spend some extra effort to prevent it.
The blackouts circumstance was because they escalate blinking/out of service traffic lights to a human confirmed decision, and they experienced a bottleneck spike in those requests for how little they were staffed. The Waymo itself was fine and was prepared to make the correct decision, it just needed a human in the loop.
In the video from the parade... there's just... people in the road. Like, a lot of small children and actual people on this tiny, super narrow bridge. I think that erring on the side of "don't think you can make it but accidentally drag a small child instead" is probably the right call, though admittedly, these cases are a bit wonky.
>The blackouts circumstance was because they escalate blinking/out of service traffic lights to a human confirmed decision
Which isn't really a scalable solution. In my city the majority of streetlights switch to blinking yellow at night, with priority/yield signs instead. I can't imagine a human having to approve 10 of these on any route.
Isn't that true for any scenario previously unencountered, whether it is a digital simulation or a human? We can't optimize for the best possible outcome in reality (since we can't predict the future), but we can optimize for making the best decisions given our knowledge of the world (even if it is imperfect).
In other words it is a gradient from "my current prediction" to "best prediction given my imperfect knowledge" to "best prediction with perfect knowledge", and you can improve the outcome by shrinking the gap between 1&2 or shrinking the gap between 2&3 (or both)
seems like the obvious answer to that is you cover a patch of road with 5mm ball bearings, and send a waymo to drive across it. if the ball bearings behave the way the simulation says they would, and the car behaves the way the simulation said it would, then you've validated your simulation.
do that for enough different scenarios, and if the model is consistently accurate across every scenario you validate, then you can start believing that it will also be accurate for the scenarios you haven't (and can't) validate.
>> How do you know the generated outputs are correct? Especially for unusual circumstances?
You know the outputs are correct because the models have many billions of parameters and were trained on many years of video on many hectares of server farms. Of course they'll generate correct outputs!
I mean that's literally the justification. There aren't even any benchmarks that you can beat with video generation, not even any bollocks ones like for LLMs.
I think because here there's no single correct answer that the model is allowed to be fuzzier. You still mix in real training data and maybe more physics based simulation of course but it does seem acceptable that you synthesize extremely tail evaluations since there isn't really a "better" way by definition and you can evaluate the end driving behavior after training.
You can also probably still use it for some kinds of evaluation as well since you can detect if two point clouds intersect presumably.
In much a similar way that LLMs are not perfect at translation but are widely used anyway for NMT.
As someone who lives in the Bay Area we already have trains, and they're literally past the point of bankruptcy because they (1) don't actually charge enough maintain the variable cost of operations, (2) don't actually make people pay at all, and (3) don't actually enforce any quality of life concerns short of breaking up literal fights. All of this creates negative synergies that pushes a huge, mostly silent segment of the potential ridership away from these systems.
So many people advocate for public transit, but are unwilling to deal with the current market tradeoffs and decisions people are making on the ground. As long as that keeps happening, expect modes of transit -- like Waymo -- that deliver the level of service that they promise to keep exceeding expectations.
I've spent my entire adult life advocating for transportation alternatives, and at every turn in America, the vast majority of other transit advocates just expect people to be okay with anti-social behavior going completely unenforced, and expecting "good citizens" to keep paying when the expected value for any rational person is to engage in freeloading. Then they point to "enforcing the fare box" as a tradeoff between money to collect vs cost of enforcement, when the actually tradeoff is the signalling to every anti-social actor in the system that they can do whatever they want without any consequences.
I currently only see a future in bike-share, because it's the only system that actually delivers on what it promises.
> they (1) don't actually charge enough maintain the variable cost of operations
Why do you expect them to make money? Roads don't make money and no one thinks to complain about that. One of the purposes of government is to make investment in things that have more nebulous returns. Moving more people to public transit makes better cities, healthier and happier citizens, stronger communities, and lets us save money on road infrastructure.
If a system doesn't generate enough revenue to cover the variable costs of operation, then every single new passenger drives the system closer to bankruptcy. The more "successful" the system is -- the more people depend on it -- the more likely it is to fail if anything happens to the underlying funding source, like a regular old local recession. This simple policy decision can create a downward economic spiral when a recession leads to service cuts, which leads to people unable to get to work reliably, which creates more economic pain, which leads to a bigger recession... rinse/repeat. This is why a public transit system should cover variable costs so that a successful system can grow -- and shrink -- sustainably.
When you aren't growing sustainably, you open yourself up to the whims of the business cycle literally destroying your transit system. It's literally happening right now with SF MUNI, where we've had so many funding problems, that they've consolidated bus lines. I use the 38R, and it's become extremely busy. These busses are getting so packed that people don't want to use them, but the point is they can't expand service because each expansion loses them more money, again, because the system doesn't actually cover those variable costs.
The public should be 100% completely covering the fixed capital costs of the system. Ideally, while there is a bit of wiggle room, the ridership should be 100% be covering the variable capital costs. That way the system can expand when it's successful, and contract when it's less popular. Right now in the Bay Area, you have the worst of both worlds, you have an underutilized system with absolutely spiraling costs, simply because there is zero connection between "people actually wanting to use the system" and "where the money comes from."
"The U.S. generates approximately $17.4 billion in annual toll revenue".
"The total annual cost for road maintenance in the U.S. is in the hundreds of billions of dollars, with estimates showing over $200 billion spent yearly".
The claim wasn't they pay for themselves but that they don't generate any income. If we want to look at externalities, we'd also have to figure out how much the Iraq war cost.
As a fellow public transit fan, you're on the money. Even the shining stars of transit in the US --- NYC MTA subway and CTA --- have huge qualuty of life issues. I can't fault someone for not wanting to ride trains ever again when someone who hasn't showered in 41 years pulls up with a cart full of whatever the fuck and decides to squat the corner seat closest to the car door and be a living biological weapon during rush hour. Or "showtime."
That's before you consider how it takes 2-4x as long to get somewhere by public transit outside of peak hours and/or well-covered areas. A 20 minute trip from a bar in Queens to Brooklyn by car takes an hour by train after 2300, not including walking time. I made that trip many, many times, and hated it each time.
It doesn't matter in this context. What matters is the hypothetical person in my post thinking "this is what will happen if my city proposes a train" and voting against any legislation trying to bring this forward where they live, even if they hate driving everywhere.
You're definitely right on (2) and (3). I've used many transit systems across the world (including TransMilenio in Bogota and other latam countries "renowned" for crime) and I have never felt as unsafe as I have using transit in the SFBA. Even standing at bus stops draws a lot of attention from people suffering with serious addiction/mental health problems.
1) is a bit simplistic though. I don't know of any European system that would cover even operating costs out of fare/commercial revenue. Potentially the London Underground - but not London buses. UK National Rail had higher success rates
The better way to look at it imo is looking at the economic loss as well of congestion/abandoned commutes. To do a ridiculous hypothetical, London would collapse entirely if it didn't have transit. Perhaps 30-40% of inner london could commute by car (or walk/bike), so the economic benefit of that variable transit cost is in the hundreds of billions a year (compared to a small subsidy).
It's not the same in SFBA so I guess it's far easier to just "write off" transit like that, it is theoretically possible (though you'd probably get some quite extreme additional congestion on the freeways as even that small % moving to cars would have an outsized impact on additional congestion).
>The better way to look at it imo is looking at the economic loss as well of congestion/abandoned commutes. To do a ridiculous hypothetical, London would collapse entirely if it didn't have transit.
You're making my argument for me. Again, my concern isn't the day-to-day conveniences of funding, my point is that building a fragile system (a system where the funding is unrelated to the usability of the service) is a system that can fail catastrophically... for systems where there are obviously alternatives (say, National Rail which can be substituted for automobile, bus, and airplane service) are less to worry about, because their failure will likely not cause cascading failures. When an entire local economy is dependent on that system -- when there are not viable substitutes -- then you're really looking at a sudden economic collapse if the funding source runs dry, or if the system is ever mismanaged.
This is a big deal. When funding really actually does run out and the system fails, then if the result is an economic cascade into a full blown depression, then you would have been much better off just building the robust system in the long term. I just really don't think people appreciate how systems can just fail. Whether it's Detroit or Caracas, when the economic tides turn in a fragile system people can lose everything in a matter of a few years.
But my point is that noone has a robust system according to you in Europe at least - the bar is so high to cover all operating costs with fares (or is that your point - if so I'm lost - I definitely would not recommend replacing European transit networks with nothing?).
And National Rail isn't replaceable at all with bus/cars/planes. You really underestimate the number of people which commute >1hr into London (100km+). There is just no way to do that journey by car or bus. It would turn a ~1hr commute into a 3hr _each way_ and that's not even considering the complete lack of parking OR the fact suddenly the roads would be at (even more) gridlock with many multiples of commuters.
That's not even getting into what you consider fixed vs variable costs. Are the trains themselves a fixed cost (they should last 30-40 years)? Is track maintenance a fixed cost (this has to be done more often than the trains themselves), etc etc. The 2nd point is very important - a lot of rail operators in the UK can be made profitable or not on your metric by how much the government subsidises track maintenance vs the operators paying for it in track access charges.
Equally, are signalling upgrades (for example) fixed costs? But really they are only required to run more frequent services. So you could argue they are a variable cost?
>Are the trains themselves a fixed cost (they should last 30-40 years)?
Yes
>Is track maintenance a fixed cost (this has to be done more often than the trains themselves)
Yes
>Equally, are signalling upgrades (for example) fixed costs?
Yes
Fixed costs are the costs that don't go away when the passengers go away. Variable cost, typically labor, go away when you don't actually need that additional marginal train. You still have to amortize that train even if it's not on the tracks. You still need to buy that marginal train when the service levels require it. You still have to do track maintenance even when you're not running trains (though, yes, at the very margin there could be some small rate adjustments). When you want to upgrade the signals, it's basically the definition of a fixed cost, because you do it once and it's done.
>And National Rail isn't replaceable at all with bus/cars/planes. You really underestimate the number of people which commute >1hr into London (100km+). There is just no way to do that journey by car or bus. It would turn a ~1hr commute into a 3hr _each way_ and that's not even considering the complete lack of parking OR the fact suddenly the roads would be at (even more) gridlock with many multiples of commuters.
I don't want to speak to National Rail or British Rail that preceded it. I want to stick to the transit system that I know well.
My point here isn't that money shouldn't be spent on "getting things back in shape" here is where I waffle on the "pay for fixed capital costs and mostly have the marginal variable costs covered by the marginal rider." If a system needs the occasional cash infusion, I'm fine with that, as long as it comes with new leadership.
My concern here is that, in the Bay Area, many, many people are eager to pay $25 for a Waymo to pick them up (they are NOT cheap) while Muni costs $3 (a near 10x increase in cost). When folks are willing to pay that much of a premium, then something is very wrong with the transit system. Muni has had zero enforcement of their code of conduct for decades. When you have a system that are large section of the populous actively avoid when it's perfectly convenient, then something is very wrong with the system.
When I see BART stations that look like abandoned parking lots surrounded by single family home sprawl, then it doesn't surprise me that the system is not sustainable. The stations that may get removed are all in areas that require people to drive, to then take the train, instead of the cities zoning density and retail around the train stations. When I yell at the occasional people smoking in BART stations and I go to tell the station attendant and get a shrug back -- even when we are paying for them to have their own police force -- that's why they are failing. These are political choices that BART has made in how they operate their service
These systems aren't even doing the bare minimum in providing a reliable pleasant service, so people stop using them, and that makes sense. The entire point is that these services should be relatively inexpensive to operate because of economies of scale, but when you don't actually make people pay, when you don't actually ask people to behave like responsible adults, when your running the service like a failing business then we should expect the service to fail, and when it does, when bailouts are needed, they should (and often do) come with strings attached. BART now has gates that stop most turnstile jumping... and they were forced to be installed by the state of California as part of their second bailout. The reason I'm harping on having variable costs attached to ridership is exactly because the systems needs to be forced to respond when a sizable amount of people no longer find the service valuable.
This is about sustainability, because the marginal tax dollar is better spend on something like providing people with the healthcare they need than it is providing people a bus service they're not even willing to actually use.
Lighting money on fire by funding an extremely expensive system that most people don't want to use is not an "investment." It's just a good way to make everyone much poorer and worse off than if we'd done nothing. The only way to change things is to convince the electorate that we actually do need rules and enforcement and a sustainable transportation system.
This isn't just happening in America. Train systems are in rough shape in the UK and Germany too.
Ebike shares are a much more sustainable system with a much lower cost, and achieve about 90% of the level of service in temperate regions of the country. Even the ski-lift guy in this thread has a much more reasonable approach to public transit, because they actually have extremely low cost for the level of service they provide. Their only real shortcoming is they they don't handle peak demand well, and are not flexible enough to handle their own success.
I'm not sure if this was intended or not, but this is a common NIMBY refrain. The argument of "This thing being advocated for that I'm fighting against isn't something people want anyway". And like walkable neighborhood architecture, extremely few Americans have access to light rail. Let alone light rail that doesn't have to yield to car traffic.
Regardless, the cost arguments fall apart once you take the total cost society pays for each system instead of only what the government pays. Because when you get the sum of road construction & maintenance, car acquisition, car maintenance, insurance, and parking, it dwarfs the cost of the local transit system. Break it down on a per-consumer basis and it gets even uglier. New York City is a good example to dive into, especially since it's the typical punching bag for "out-of-control" budgets.
Quick napkin math pins the annual MTA cost at $32-$33 billion and the total cost of the car system between $25 and $44 billion per year. Since the former serves somewhere around 5.5 million riders, and the latter only about 2 million, the MTA costs $5,300-6,600 per user annually where the car system costs $12,000–$22,500 per user annually.
People want transit as long as that transit reasonably meets their quality of life standards. The reason why automobiles have been so popular -- even while being wildly more expensive -- is exactly that they allow the user to adjust their travel to their optimal quality of life expectations.
Public transit advocates need to be honest with themselves that anti-social behavioral issues really matter to people. People are willing to pay more to have a more pleasant experience. When a transit system fails to meet that standard, then you'll suddenly find yourself with a transit system that people don't want to use.
> I AM saying "people don't want ride trains that allow 5% of the riders to smoke cigarettes on enclosed train platforms and in enclosed train cars."
Just don't allow that then?
> Public transit advocates need to be honest with themselves that anti-social behavioral issues really matter to people. People are willing to pay more to have a more pleasant experience. When a transit system fails to meet that standard, then you'll suddenly find yourself with a transit system that people don't want to use.
"we can't have good transit because a few people who call themselves transit advocates have bad opinions" is very defeatist. Weak-spined politicians find it much easier to just set money on fire than actually solving problems, so even though most transit advocacy groups in the US emphasize quality and being less wasteful with budgets, your politicians usually prefer the worse options.
>> I AM saying "people don't want ride trains that allow 5% of the riders to smoke cigarettes on enclosed train platforms and in enclosed train cars."
>Just don't allow that then?
>"we can't have good transit because a few people who call themselves transit advocates have bad opinions" is very defeatist.
My point here is only that this is a hard problem, not a trivial one. When the transit advocates in my area just say "transit should be free" in response to "transit pricing is a complex problem that affects system fragility" and they say "stop hating homeless people" in response to "quality of life concerns matter to keeping the system functional long term" then we're in bad place, because the non-transit advocates literally want to get rid of the system. The last TWO Muni funding bills in SF failed.
We've built a system that can fail catastrophically, in large part, because transit advocates don't want to deal with the realities of running a functional transit system. This is why I get grumpy when people say "all this work is impressive, but I'd rather have better trains" when it's very clear why Waymo is succeeding as Muni is failing, but it is exactly because Muni is mostly disconnected from market forces that we've got to this place, and the "solution" being proposed by most transit advocates is to just completely remove all market forces which will very obviously be worse is the long run.
over the long term, this is solved with a wealth tax, but undoing what rich ppl have done to society (i.e. making lots of poor people) will unfortunately take many, many years; so many years that it will never actually happen
My entire point is mostly not even about the money. It's about the system having to respond as a service to the fact that people don't want to use that service and are willing to pay a huge premium for alternatives like Waymo.
I’m not advocating that they do. Fixed costs should be fully subsidized. I’m only advocating that revenues are set so that during a median year, each additional rider on average, provides income that is proportional to the level of service needed to move that rider through the system.
It's worth noting that, at least for bart, the reason that it is facing bankruptcy is precisely because it was mostly rider supported and profitable, and not government supported.
When ridership plummeted by >50% during the pandemic, fixed costs stayed the same, but income dropped. Last time I checked, if Bart ridership returned to 2019 levels, with no other changes, it would be profitable again.
You can't say that BART "is facing bankruptcy is precisely because it was mostly rider supported and profitable, and not government supported" when it is very obvious that BART would be in a much worse situation if it had had more government support... because all those governments are facing massive budget deficits right now.
BART has already been bailed out by the state, twice. It has already failed, twice. It very much needs to reduce the level service it provides if it wants to be sustainable, or seek other forms of revenues while we wait to see if ridership returns. Many have suggested BART explore the SE Asian model of generating revenues by developing residential housing, which seems fairly straightforward.
If ridership never returns, then we ought not continue throwing good money after bad, and we ought to adjust the level of service to meet the level of revenues. Obviously the main problem here is that it's literally illegal to just build high density corridors directly adjacent to the transit stations... which is what we ultimately need to prioritize.
Where does the extra money come from in a deficit period?
> BART, Muni, Caltrain, AC Transit — which an independent analysis confirmed face annual deficits of more than $800 million annually starting in fiscal year 2027-28
Nearly a billion dollar shortfall per year going forward. That’s nontrivial, and the state has lost patience with the systems after providing two bailouts already.
Taxes? The same place tons of other stuff we buy as a society comes from. I expect the ballot measure this fall will pass, worst case they file bankruptcy and will probably need to reduce service
I mean, sure? I'd prefer to have a system that has a system built in that raises and lowers the level of service in accordance with the number of people using the system rather than having to have random elections that decide whether or not we're going to effectively scrape a large parts of the system.
1. You want to be forward looking, not backwards looking. Cutting services means less ridership means less revenue means cutting services means...etc. Bart is super useful for me during the week because headways from SF to West Oakland are often 5m. As I'm writing this (11 on a Friday) I missed a train and had to wait 20 minutes. Every seat on the car is also full, and while not packed, it's standing room only. If my choice is to wait 20 mins for the next train, other ways of getting places become a lot more appealing.
2. Government services should be good. This is good both because it makes people interested in using them (see 1) and because people who don't have other options deserve good services. The point of government is, at least in part, to serve those who can't serve themselves. I don't expect Bart to be revenue neutral for the same reason I don't expect CalFRESH to be.
> Cutting services means less ridership means less revenue means cutting services means...etc.
That's not true. If you have stations that are revenue positive and stations that are net negative, then cutting ridership at the net-negative stations can put the system in a much better financial position. E.g. If BART didn't end at Antioch, and instead continued to Rio Vista, it's entirely likely that the Rio Vista station would just cost more to operate than is worth operating. It takes time to go back and forth, nobody will ever want to be picked up there because it's car-dependent sprawl. Maybe have one or two stops there during rush hour, but you'll likely be better financially cutting most service.
>headways from SF to West Oakland are often 5m
Nobody is suggesting cutting service between SF and Oakland. I'm sure it's a wildly profitable route. Crossing the bay is the main benefit of BART.
>The point of government is, at least in part, to serve those who can't serve themselves. I don't expect Bart to be revenue neutral for the same reason I don't expect CalFRESH to be.
I also don't expect BART to be revenue neutral. I expect it to be funded -- in very large part -- by taxes. I'm only arguing it should be sustainable. It shouldn't get to the point of literal collapse during economic downturns (again, it's already been bailed out by the state and feds, twice, in the last six years).
I really don't think people realize what I'm getting at. I'm saying the system needs to be functional and needs to function long term. Yes, I think we should subsidize low-income users. Yes, I think people who can't afford it should still be able to use it. But that has to happen in a way that doesn't drive away significant numbers of other users. There's nothing about being low-income that means anti-social. I'm talking about anti-social behavior. I'm talking about people smoking cigarettes and using drugs on BART platforms and in BART cars. I'm talking about people who are actively bothering significant numbers of people around them by their behavior -- behavior that is against BART policy, but is tolerated.
You can't sit here and tell me the current system is working when BART is perpetually collapsing. I care about BART. That's why I'm articulating the systemic problems in the system.
> You can't say that BART "is facing bankruptcy is precisely because it was mostly rider supported and profitable, and not government supported" when it is very obvious that BART would be in a much worse situation if it had had more government support... because all those governments are facing massive budget deficits right now.
I don't think this follows. Government budgeting isn't zero based. A hypothetical Bart with 2x the government funding in 2019 would have faced cutbacks, but likely has more money today than what we have now!
> or seek other forms of revenues while we wait to see if ridership returns.
Yes, this is called "taxes".
> If ridership never returns, then we ought not continue throwing good money after bad
Agreed if it was stagnant, but ridership is up more than 10% y/y and that was also true last year. It's on track to be revenue neutral again in a few years. Gutting services today would be exactly opposite of what you'd do for something like a startup showing clear path toward profitability.
> Obviously the main problem here is that it's literally illegal to just build high density corridors directly adjacent to the transit stations... which is what we ultimately need to prioritize
While sure it's hard, there's lots of these that exist. There's new stuff in oakland basically constantly, and were even seeing midrise stuff along Bart in SF, but it's units being built now, so they won't be available until 2027, which is when your proposed service cuts would hit.
>Government budgeting isn't zero based. A hypothetical Bart with 2x the government funding in 2019 would have faced cutbacks, but likely has more money today than what we have now!
A hypothetical BART with 2x the government funding wouldn't have existed... because it didn't exist.
>Agreed if it was stagnant, but ridership is up more than 10% y/y and that was also true last year. It's on track to be revenue neutral again in a few years. Gutting services today would be exactly opposite of what you'd do for something like a startup showing clear path toward profitability.
You're mistaking what I'm saying. I want BART to flourish, but I want it to be sustainable. The choice isn't "keep it open" or "close it." How it is operated matters significantly. I'm very obviously going to vote to increase funding, my point is that it shouldn't have to come to a vote. If service is reduced to a more sustainable rate, the system could recover organically. The revenue jump that has happened at stations immediately after the gates were installed, for example, shouldn't surprise anyone. I'm a transit advocate, BART is mostly irrelevant to this discussion anyway, because we're talking about situations where Waymo is a viable alternative, which really doesn't apply to BART.
Trains need well behaved people, otherwise they are shit.
I don't want to hear tiktok or full volume soap operas blasting at some deaf mouth breather.
I don't want to be near loud chewing of smelly leftovers.
I don't want to be begged for money, or interact with high or psychotic people.
The current culture doesn't allow enforcement of social behaviour: so public transport will always be a miserable containment vessel for the least functional, and everyone with sense avoids the whole thing.
Or the majority of the residents of New York City on their daily commute? I like to think I have sense, and I happily use public transport most days. I prefer it to sitting in traffic, isolated in a car. At least I can read a book. If you work too hard to insulate yourself from the world, the spaces you'll feel comfortable in will get more and more narrow. I think that's a bad thing.
NYC people uses it because the alternatives are either slower or much more expensive. I'm sure they'd rather use a waymo if it was as fast and cheap as the subway.
I quite agree with the overall point but can we leave this kind of discourse on X, please? It doesn't add much, it just feels caustic for effect and engagement farming.
The vast majority of the anti-social behavior on public transit not relevant in automobiles because (1) you can't turnstile jump the gas tank, (2) an automobile is effectively very expensive set of headphones, and (3) you can inhale whatever you want in your vehicle and your neighbor doesn't have to breath it.
Automobiles are a wildly inefficient and expensive form of transportation in urban areas. At the same time, we ought to be willing to ask why a significant amount of our urban population still prefers to pay all that extra money to sit in traffic.
I think they have a point. But the anti-social behaviors in a car on the road are mostly a different set of anti social behaviors than you’d see on a bus or train. But they certainly exist.
No matter what, people are going to still use cars because they are an absolute advantage over public transportation for certain use cases. It is better that the existing status quo is improved to reduce death rates, than hope for a much larger scale change in infrastructure (when we have already seen that attempts at infrastructure overhaul in the US, like high-speed rail, is just an infinitely deep money pit)
Even though the train system in Japan is 10x better than the US as a whole, the per-capita vehicle ownership rate in Japan is not much lower than the US (779 per 1000 vs 670 per 1000). It would be a pipe dream for American trains/subways to be as good as Japan, but even a change that significant would lead to a vehicle ownership share reduced by only about 13%.
I don't think individual vehicles can ever achieve the same envirnmental economies of scale as trains. Certainly they're far more convenient (especially for short-haul journeys) but I also think they're somewhat alienating, in that they're engineering humans out of the loop completely which contributes to social atomization.
Trains only require subsidies in a world where human & robot cars are subsidized.
As soon as a mode of transport actually has to compete in a market for scarce & valuable land to operate on, trains and other forms of transit (publicly or privately owned) win every time.
Source? The biggest source of environmental issues from EVs, tire wear from a heavier vehicle, absolutely applies to AVs. VC subsidizing low prices only to hike them later isn't exactly "without subsidy" - we pay for it either way
Sure but most of the world has a density low enough that cars work and trains don't really. I like trains as much as the next nerd, but you're never going to be able to take a train from your house to your local farm shop or whatever.
Where trains work they are great. Where they don't, driverless electric cars seem like a great option.
AFAICT, the majority (60%) of funding for roads doesn't come from direct user charges...
Roads are subsidized, free parking (and generally a lot of paid parking) is subsidized, and the sprawl encouraged by car dependence combined with the resulting infrastructure costs has and will continue to bankrupt cities.
I don't think we should "just only have trains", but the current US transit landscape is absurdly stupid and inefficient.
Yeah I think the single biggest red flag that the US absolutely could support more public transit is the fact that many cities successfully had more public transit in the past.
Public transportation is the backbone of a functioning economy. It doesn't need to be fully paid by riders precisely because the rest of society benefits from it multiple times over.
NYC "congestion" pricing (actually cordon pricing) is a good idea. Would be great to see more road use fees proportional to use (distance, weight^3, etc.).
Me too but given our extensive car brain culture, Waymo is an amazing step to getting less drivers & cars off the road, and to further cement future generations not ever needing to drive or own cars
Pretty much this. Wild that you can traverse most of China in affordable high speed trains, yet the Amtrak from Seattle to Portland barely crawls along and has to regularly stop for long periods of time because the tracks get too hot in the Summer.
Enough with the trains. I’m all for trains but theyre good for in city or 1-3 hour journeys. Taking a train across the US would take a day even with high speed trains.
I’d much rather have my own vehicle than share my space with a bunch of people.
This is the real story buried under the simulation angle. If you can generate
reliable 3D LiDAR from 2D video, every dashcam on earth becomes training data.
Every YouTube driving video, every GoPro clip, every security camera feed.
Waymo's fleet is ~700 cars. The internet has millions of hours of driving
footage. This technique turns the entire internet into a sensor suite. That's a bigger deal than the simulation itself.
It's not unheard of, there are a handful [0] of metric monodepth methods that output data that's not unlike a really inaccurate 3D lidar, though theirs certainly looks SOTA.
The "world model" is a convenient fiction. Whether we’re talking about a carbon-based brain or a silicon-based transformer, there is no miniature, objective map of reality tucked away inside. What we mistake for a "model" is actually just the layered residue of experience.
From the perspective of enactivism and radical empiricism, intelligence doesn't "represent" the world; it simply navigates it. A biological organism doesn't need a 3D CAD file of a tree to survive; it only needs a history of sensory-motor contingencies—the "if I move this way, I see that" patterns. It’s a synthesis of interactions, not a library of blueprints.
AI operates on the same logic, albeit through a different medium. It isn't simulating the physical laws of the universe or "understanding" gravity. Instead, it navigates the high-dimensional geometry of human data. It’s a sophisticated engine of association, performing a high-speed synthesis of the patterns we've left behind.
In this view, "knowing" isn't about matching an internal image to an external truth. It is the seamless flow of past inputs into future predictions. There is no world model—only the habit of being.
It’s impressive to see simulation training for floods, tornadoes, and wildfires. But it’s also kind of baffling that a city full of Waymos all seemed to fail simultaneously in San Francisco when the power went out on Dec 22.
A power outage feels like a baseline scenario—orders of magnitude more common than the disasters in this demo. If the system can’t degrade gracefully when traffic lights go dark, what exactly is all that simulation buying us?
All this simulation buys a single vehicle that drives better. That failure was a fleet-wide event (overloading the remote assistance humans).
That is, both are true: this high-fidelity simulation is valuable and it won't catch all failure modes. Or in other words, it's still on Waymo for failing during the power outage, but it's not uniquely on Waymo's simulation team.
We started with physics-based simulators for training policies. Then put them in the real world using modular perception/prediction/planning systems. Once enough data was collected, we went back to making simulators. This time, they're physics "informed" deep learning models.
That's a very interesting way of looking at it. Yes, you start with simulating something simpler than the real world. Then you use the real world. Then you need to go back to simulations for real-world things that are too rare in the real world to train with.
Seems like there ought to be a name for this, like so-and-so's law.
Regardless of the corporate structure DeepMind is a lot more than just another Alphabet subsidiary at this point considering Demis Hassabis is leading all of Google AI.
Finally I understand the use case for Genie 3. All the talk about "you can make any videogame or movie" seems to have been pure distraction from real uses like this: limited, time-boxed simulated footage.
IIUC, there's a confusion of meaning for "World Model", between Waymo/Deepmind's which is something that can create a consistent world (for use to train Waymo's Driver), vs Yann LeCun/Advanced Machine Intelligence (AMI) which is something that can understand a world.
I'd like to see Waymo have a few of their Drivers do some sim racing training and then compete in some live events. It wouldn't matter much to me if they were fast at all, I'd like to see them go into the rookie classes in various games and see how they avoid crashes from inexperienced players. I believe that it would be the ultimate "shitty drivers vs. AI" test.
Racing and street driving are completely different. Racing involves detailed knowledge of vehicle dynamics and grip. Street driving is mainly obstacle recognition and avoidance. No waymo ever operates anywhere close to the limit of grip, which is where you are all the time when racing.
Interesting, but it feels like it's going to cope very poorly with actually safety-critical situations. Having a world model that's trained on successful driving data feels like it's going to "launder" a lot of implicit assumptions that would cause a car to get into a crash in real life (e.g. there's probably no examples in the training data where the car is behind a stopped car, and the driver pulls over to another lane and another car comes from behind and crashes into the driver because it didn't check its blindspot). These types of subtle biases are going to make AI-simulated world models a poor fit for training safety systems where failure cannot be represented in the training data, since they basically give models "free reign" to do anything that couldn't be represented in world model training.
You're forgetting that they are also training with real data from the 100+ million miles they've driven on real roads with riders, and using that data to train the world model AI.
> there's probably no examples in the training data where the car is behind a stopped car, and the driver pulls over to another lane and another car comes from behind and crashes into the driver because it didn't check its blindspot
While there most likely is going to be some bias in the training of those kinds of models, we can also hope that transfer learning from other non-driving videos will at least help generate something close enough to the very real but unusual situations you are mentioning. We could imagine an LLM serving as some kind of fuzzer to create a large variety of prompts for the world model, which as we can see in the article seems pretty capable at generating fictive scenarios when asked to.
As always tho the devil lies in the details: is an LLM based generation pipeline good enough? What even is the definition of "good enough"? Even with good prompts will the world model output something sufficiently close to reality so that it can be used as a good virtual driving environment for further training / testing of autonomous cars? Or do the kind of limitations you mentioned still mean subtle but dangerous imprecisions will slip through and cause too poor data distribution to be a truly viable approach?
My personal feeling is that this we will land somewhere in between: I think approaches like this one will be very useful, but I also don't think the current state of AI models mean we can have something 100% reliable with this.
The question is: is 100% reliability a realistic goal? Human drivers are definitely not 100% reliable. If we come up with a solution 10x more reliable than the best human drivers, that maybe has some also some hard proof that it cannot have certain classes of catastrophic failure modes (probably with verified code based approaches that for instance guarantees that even if the NN output is invalid the car doesn't try to make moves out of a verifiably safe envelope) then I feel like the public and regulators would be much more inclined to authorize full autonomy.
As a Londoner who used to have to ride up Abbey Road at least once per week there are people on that crossing pretty much all day every day reproducing that picture. So now Waymo are in Beta in London[1] they have only to drive up there and they'll get plenty of footage they could use for taht.
[1] I've seen a couple of them but they're not available to hire yet and are still very rare.
Yeah it’s interesting to imagine a London that had such a rebuild, like Napoleon’s rebuild of Paris. I personally love the weird narrow streets and little alleyways of the City, but that’s because when I’m there I’m pretty much exclusively on foot having taken the tube in.
The term "world model" seems almost meaningless. This is a world model in the same sense as ChatGPT is a world model. Both have some ability to model aspects of the real world.
Interesting, but I am very sceptical. I'd be interested in seeing actual verified results of how it handles a road with heavy snow, where the only lane references are the wheel tracks of other vehicles, and you can't tell where the road ends and the snow-filled ditch begins.
Very concerned with this direction of training
“counterfactual events such as whether the Waymo Driver could have safely driven more confidently instead of yielding in a particular situation.” Seems dicey. This could lead in the direction to a less safe Waymo. Since the counterfactual will be generated, I suspect that that the generations will be biased towards survivor situations where most video footage in its training data will be from environments where people reacted well not those that ended in tragedy. Emboldening Waymo on generated best case data. THIS IS DANGEROUS!!!
Not at all. It's not the counter-factual they're generating, it's the "too rare to capture often enough to train a response to" they're generating.
They're implying that without the model having knowledge, even approximate, of a scene to react to, it simply doesn't react at all; it simply "yields" to the situation until it passes. In my experience taking Waymo's almost daily this holds.
I would rather not have the Waymo yield to a tornado, rising flood-waters, or charging elephant...
Driving is always a balance between speed and safety. If you want ultimate safety you just sit in the driveway. But obviously that isn't useful. So functionally one of the most important things a self-driving system will decide is "how fast is it safe to drive right now". Slower is not always better and it has to balance safety with productivity.
Not entering a roundabout when it's clearly safe to do so is a mark against you at a driving exam. So would be always driving at 5mph. It's just not that simple.
Neat! What happens when the simulated data is hallucinated/incorrect?
In the example videos, the Golden Gate bridge with snow shows the bridge as 1 road, with total of 3 lanes. But in reality, it’s a split highway with divider, so 2 sides both have 3 lanes, 6 total lanes.
What happens when the car “learns” to drive on the simulated incorrect 3 lane example? For example will next time it goes on the real GG bridge hug to the rightmost lane?
Ideally it would learn a relationship between the sensor input and the correct actions, even if the sensor input is not realistic for the GG in reality.
Similar to Kiva Systems which was Amazon's best acquisition, Waymo is simply Google's best acquisition. (We live in San Francisco and it feels much safer around these Waymo cars than average "drivers".)
Another comment mentioned the Philippines as the manifest frontier. SF is not on the same plane of reality in terms of density or narrow streets as PH, I would argue in comparison it does not have both.
On that specific count, not really. There's a skate park north end of the Mission, and Stevenson St is a two way road that borders it, but it's narrow enough that you need to drive up on the curb to get two vehicles side by side on the street. Waymo's can't handle that on a regular basis. Being San Francisco and not London, you can just skip that road, but if you find yourself in a Waymo on that street and are unlucky to have other traffic on it, the Waymo will just have to back up the entire street. Hope there's no one behind you as well as in front of you!
Anyway, we'll see how the London rollout goes, but I get the impression London's got a lot more of those kinds of roads.
I live in London. Most residential streets are two-way but there is only space for one car, and driving on the curb is not really an option.
The trick to UK streets is that parking actually happens on the street itself, and when driving you must find a spot when people are not parking to make way for people coming the other way.
That is extremely narrow, I wonder why the city has not designated it as a one-way street? They've done that for other similarly narrow sections of the same street farther north.
This is an alley in Coimbra, Portugal. A couple years ago I stayed at a hotel in this very street and took a cab from the train station. The driver could have stopped in the praça below and told me to walk 15m up. Instead the guy went all the way up then curved through 5-10 alleys like that to drop me off right right in front of my place. At a significant speed as well. It was one of the craziest car rides I've ever experienced.
I live in such an area. The route to my house involves steep topography via small windy streets that are very narrow and effectively one-way due to parked cars.
Human drivers routinely do worse than Waymo, which I take 2 or 3 times a week. Is it perfect? No. Does it handle the situation better than most Lyft or Uber drivers? Yes.
As a bonus: unlike some of those drivers the Waymo doesn't get palpably angry at me for driving the route.
How large is the controlled area around the car? And how high do they look for objects? Like something is falling from a bridge, a falling pole or more extreme an falling plane.
as someone looking at this from a risk and liability perspective, the move toward world models is pretty wild. the edge cases where a model predicts a safe path that doesn't align with physics is basically the final boss of insurance underwriting for autonomous vehicles. curious how they actually validate for 'common sense' reality vs just predicting the next frame.
I doubt Waymo would publicly talk about this if it did happen.
I also doubt the IP is worth that much. Most of the secret sauce to starting a competitor probably isn't an end model tuned for a specific configuration of a car but the ability to produce end models, which wouldn't be stealable from the car.
It is great being able to generate a much larger universe of possibilities than what they can gather from real world data collection, but I'd be curious to learn how they check that the generated data is a superset of the possibility-space seen in the real world (e.g. confirm that their models closely match what is seen in the real world too)
1. Still hard not to think that this is a huge waste of time as opposed to something that's a little more like a public transport train-ish thing, i.e. integrate with established infrastructure.
2. No seriously, is the filipino driver thing confirmed? It really feels like they're trying to bury that.
"The Filipino driver thing" is simply that there's a manual override ability when this profoundly complex and marvelously novel technology gets trapped in edge cases.
I am very pro public transit. But there is still a place for cars (ideally mostly taxis). Going to more rural areas or when you need to carry more stuff. I think an ideal society would have both urban transit, inter-city transit and taxis for the other trips and going out into the country.
(2) I really don't understand why people are surprised that Waymo has fallbacks? The fact that they had a team ready to take over as necessary was well known. I've seen a bunch of comments about this and it seems like people are confused.
Yeah, that’s the root of a lot of problems in the US.
But pedestrian infrastructure is a very simple problem to solve, even in the US. Just look at how the market responds to public transit projects in the US —- developers trip over themselves to build projects in response to the induced demand.
Likewise they also responded to the reduced demand as a result of ripping out public transit infrastructure in the 20th century. Which is why doughnut-shaped cities are common in the US.
They are not trying to "bury" remote assistance at all. They wrote a white paper about it in 2020 and a blog post about it in 2024.
Anyway you can think it's a waste but they're wasting their money, not yours. If you want a train in your town, go get one. Waymo has only spent, cumulatively, about 4 months of the budgets of American transit agencies. If you had all that money it wouldn't amount to anything.
My view on Waymo and autonomous taxis in general is they will eventually make public transit obsolete. Once there is a robotaxi available to pick up and drop off every passenger directly from a to b, the whole system could be made to be super efficient. It will take time to get there though.
But eventually I think we will get there. Human drivers will be banned, the roads will be exclusively used by autonomous vehicles that are very efficient drivers (we could totally remove stoplights, for example. Only pedestrian crossing signs would be needed. Robo-vehicles could plug into a city-wide network that optimizes the routing of every vehicle.) At that point, public transit becomes subsidized robotaxi rides. Why take a subway when a car can take you door to door with an optimized route?
So in terms of why it isn’t a waste of time, it’s a step along the path towards this vision. We can’t flip a switch and make this tech exist, it will happen in gradual steps.
Automated taxis would still be stuck in traffic. Automation gets couple times in capacity, but the induced demand and extra cars looking for rides and parking will mean traffic.
Automation makes public transit better. There will be automated minibuses that are more flexible and frequent than today's buses. Automation also means that buses get a virtual bus lane. Taxis solve the last mile problem, by taking taxi to the station, riding train with thousands of people, and then taking more transit.
Also, we might discover the advantage of human powered transit. Ebikes are more efficient than cars and give health benefits. They will be much safer than automated cars. Could use the extra capacity for bike and bus lanes.
> There will be automated minibuses that are more flexible and frequent than today's buses.
In my sleepy metro area that has at least mid-tier respectable public transit (by US standards only), otherwise known as Portland, I think a lot of the routes would be better served by minibuses than full size. I wonder how the economics work out on that. Maybe dominated by labor? Tri-met drivers have a reputation of being paid handsomely as they gain seniority.
I'm also in Portland. In the US, bus costs are dominated by labor. It makes sense to use full size buses if paying for driver. For main routes, more automated buses would be best option. But there are cross town routes that should be served with minibuses. Especially ones feeding MAX stops.
> Human drivers will be banned, the roads will be exclusively used by autonomous vehicles
I basically agree with your premise that public transit as it exists today will be rendered obsolete, but I think this point here is where your prediction hits a wall. I would be stunned if we agreed to eliminate human drivers from the road in my lifetime, or the lifetime of anyone alive today. Waymo is amazing, but still just at the beginning of the long tail.
Did it? I did a cursory search and it seems like many places still permit horse-drawn carriages, just not on limited access highways. Sometimes with fairly onerous licensing and operational requirements (speed limits, poo management, etc), but still allowed.
A few years ago I would have (and did) considered the notion that manually programming was about to turn into a quaint relic and computers would be writing 90%+ of code preposterous. Once an alternative becomes obviously superior things can change very fast.
- I would be stunned if we agree to eliminate human drivers from 100% of roads in the lifetime of anyone alive today.
or
- I would be stunned if we agree to eliminate human drivers from 10% of roads...
...or is there some other percentage to qualify this? I guess I wouldn't expect there to be a decree that makes it happen all at once for a country. Especially a large country like the U.S.. More like, some really dense city will decide to make a tiny core autonomous vehicles only, and then some other cities also do years later. And then maybe it expands to something larger than just the core after 5 or 10 years. And so on...
That is a fair point, since it is fairly safe to make "this absolute claim will never happen." And the person I replied to did say 'exclusively', which implies 100% elimination of human drivers. But still, I appreciate your nuance.
And in the spirit of that nuance, I will revise my statement slightly. I think it is entirely possible we will eliminate drivers on 10% of roads. We have rules that are analogous to that already with limited access highways. Though I would rate this still as unlikely, since such roads only make up just over 1% of all the roads in the US as it is. Not sure what the % is for other countries, probably less.
> some really dense city will decide to make a tiny core autonomous vehicles only
Agree 100%, this kind of thing I do expect to see happen. We already have exclusions for cars altogether in favor of pedestrians, so the precedent is set.
In high density regions, vehicles on surface roads can’t meet the passenger demand required. Even if you banned human drivers, the other human users introduce too much variability and delay (passengers loading and unloading, errant objects, cyclists and pedestrians, etc). Roll a dumpster in the street, and have a couple of jaywalkers, and the entire system crawls to a stop.
Controlled access is required to get even medium-high throughput. But these systems already exist, they are called personal rapid transit systems.
> Once there is a robotaxi available to pick up and drop off every passenger directly from a to b, the whole system could be made to be super efficient.
Fundamentally impossible. You're moving some 2 tons of mass in a 2x5m box on polluting rubber tires to move a single 100kg human.
I can always take whatever efficiency gain you've thought up and simply make the vehicle bigger, decreasing the cost and space used per passenger, and maybe even put it on rails, making it less polluting, and more energy efficient.
You can't engineer your way out of the laws of physics.
I don't get how this solves the problem of edge cases with self driving
Even if you can generate simulated training data, don't you still have the problem where you don't even know what the edge cases you need to simulate are in the first place?
Well it certainly helps,doesn't it? This system is going to encounter more edge cases than a single human ever would. Hopefully the lessons from known unknowns generqlise to unknowns. And once they've been seen once they took can become part of the corpus.
It might be "never-ending", but you're going to encounter edge cases in approximate proportion to the rate at which they actually occur. Anyway, the hope would be to learn behaviors which generalize, not to respond to each edge case ad-hoc; the edge cases provide out-of-sample tests of generalizability.
Neither does the car — it won't drive into what LIDAR sees as a wall. But stopping is not good enough, it needs to be able to navigate the obstacle as well.
Also, even if the car behaved perfectly anyway, these scenarios are useful for testing — validating that the expected behavior happens.
The amount to which supposedly engineering based rationales are applied to Elon's decisions is cult like behavior. Here's what's really going on:
That thing I called full self driving just to troll government regulators? Just make it work and don't complain about what I called it.
That rocket that's way too big for orbital payloads but can't go beyond orbit without sending 20 more rockets full of fuel? Just make it work. Occupy Mars!
The two car models that work and have a halo effect on the rest of the product line? I'mma cancel that! Just make the stupid truck we're keeping work!
We've all had a boss like that. I'm sure they salute each other.
It seems inevitable that they'll soon be used as the starting points for developing almost all video game environments.
Not for the rendering (that's still way too expensive), but for the initial world generation that gets iteratively refined and then still ultimately gets converted into textured triangles.
Very impressive work from Waymo. The driving with a tornado in the horizon example kind of struck my imagination, many people actually panic in such scenarios. I wonder though the compute requirements to run these simulations and producing so many data points.
I would love to see more visibility into how this model’s simulation fidelity maps onto measurable safety improvements on public roads, especially in unusual edge conditions like partial sensor occlusion or atypical weather.
It's easier to build trust for such a safety-critical service when you're more open about how it works an performs. For the complete opposite approach, see Tesla.
Given the announcement from a few days ago of google trying to get external investment, this is their follow up, showing what that investment is good for. Also, it’s pretty light on details that are of much use to competitors. “We made an accurate simulation system to test our system in before deployment” would be pretty mundane if you were talking about any other field of engineering.
Under the same circumstances (kid suddenly emerging between two parked cars and running out onto the street), it could be debated that the outcome could have been worse if a human were driving.
I don't know about the remote driver conspiracy, but waymo slowing down and that kid surviving a crash after jumping on the road from behind a tall vehicle was the best PR waymo could have asked for.
been playing around with world models for sim-to-real transfer lately. the waymo approach looks solid, but curious how you're handling the distribution shift between generated scenes and real sensor data. any tricks for that besides the usual domain randomization?
This is cool, but they are still not going about it the right way.
Its much easier to build everything into the compressed latent space of physical objects and how they move, and operate from there.
Everyone jumped on the end-2-end bandwagon, which then locks you into the input to your driving model being vision, which means that you have to have things like genie to generate vision data, which is wasteful.
I posted this before, but Ill post again - this is one of the few things I feel confident enough to say that most people in the space are doing wrong. You can save my post and reference it when we actually get full self driving (i.e you can take a nap in the backseat while your car drives you), because its going to be implemented pretty much like this:
Humans don't drive well because we map vision policy to actions. We drive well (an in general, manipulate physical objects well), because we can do simulations inside our head to predict what the outcome will be. We aren't burdened by our inability to recognize certain things - when something is in the road, no matter what it is, we auto predict that we would likely collide with that thing because we understand the concept of 3d space and moving within it, and take appropriate action. Sure, there is some level of direct mapping as many people can drive while "spaced out", but attentive driving involves mostly the above.
The self driving system that can actually self drive needs to do the same. When you have this, you will no longer need to do things like simulate driving conditions in a computationally expensive sim. You aren't going to be concerned with training model on edge cases. All you would need to to ensure that your sensor processing results in a 3d representation of the driving conditions, and the model will then be able to do what humans do and explore a latent space of things it can do and predict outcomes then chose the best one.
You want proof? It exists in the form of Mu Zero, and it worked amazingly well. And driving can be easily reformated as a game that the engine plays in a simulator that doesn't involve vision, and learns both the available moves and also the optimal policy.
The reason everyone is doing end to end today is because they are basically trying to catch up to Tesla, and from a business perspective, nobody is willing to put money and pay smart enough people to research this, especially because there is also a legal bridge to cross when it comes to proving that the system can self drive while you napping. But nevertheless, if you ever want self driving, this is the right approach.
Meanwhile, Google who came up with Mu Zero, is now doing more advanced robotic stuff than anyone out there.
Whenever something like this comes out, it's a good moment to find people with no critical thinking skills who can safely be ignored. Driving a waymo like an RC car from the philippines? you can barely talk over zoom with someone in the philippines without bitrate and lag issues.
I haven't read anything about this but I would also suppose long distance human intervention cannot be done for truly critical situations where you need a very quick reaction, whereas it would be more appropriate in situations where the car has stopped and is stuck not knowing what to do. Probably just stating the obvious here but indeed this seems like something very different from an RC car kind of situation.
It’s not for that. It’s for things like the car drove into a protest area and people are surrounding the car. Or police blocked off an intersection and the car is stuck temporarily with people doing otherwise illegal u-turns or driving the wrong way on a one way road to get out of it.
The word "loop" here has multiple meanings. Only one is what you mean and the other person responding to you has understood another.
The first is the DDT control loop, what a human driver does. Waymo's remote assistants aren't involved in that. The computer always has responsibility for the safety of the vehicle and decisionmaking while operating, which is why Waymo's humans are remote assistants and not remote drivers. Their safety drivers do participate in the DDT loop, hence the name.
But there's also another "loop" of human involvement. Sometimes the vehicle doesn't understand the scene and asks humans for advice about the appropriate action to take. It's vaguely similar to captchas. The human will usually confirm the computer's proposed actions, but they can also suggest different actions. The computer the advice as a prior to continue operating instead of giving up the DDT responsibility. There's very likely a closely monitored SLA between a few seconds to a few minutes on how long it takes humans to start looking at the scene.
If something causes the computer to believe the advice isn't safe, it will ignore it. There have been cases where Waymos have erroneously detected collisions and remote assistants were unable to override that decisionmaking. When that happens, a vehicle recovery team is physically sent out to the location. The SLA here is likely between tens of minutes and a couple hours.
Interesting question. If the Waymo was driving aggressively to remove us from the situation but relatively safely I might stay in it.
This does bring up something, though: Waymo has a "pull over" feature, but it's hidden behind a couple of touch screen actions involving small virtual buttons and it does not pull over immediately. Instead, it "finds a spot to pull over". I would very much like a big red STOP IMMEDIATELY button in these vehicles.
>it's hidden behind a couple of touch screen actions involving small virtual buttons and it does not pull over immediately
It was on the home screen when I've taken it, and when I tested it, it seemed to pull to the first safe place. I don't trust the general pubic with a stop button.
Can you not just unlock and open the door? Wouldn't that cause it to immediately stop? Or can you not unlock the door manually? I'd be surprised if there was not an emergency door release.
The model generates camera and Lidar data. As if it was a Waymo car that drove through the simulated scenario with its cameras running. This synthetic training data can then be used to train the driving models.
Wonder how it'll do. The trees change shape (presumably the Lidar patterns do too). I get the premise/why but it seems odd to me (armchair) to use fake data. Real trees don't change shape (in real time) although it can be windy.
It probably doesn't matter though, "this general blob over there"
It's clear that lidar is superior to the Tesla cameras only approach, but I wonder if being able to take all that data and be able to better correlate what is being seen visually with what the lidar is showing.
That said, for autonomous driving I'd like to see as many sensor options as possible: lidar, radar, cameras, sonar. Belt and suspenders. I imagine as pricing continues to drop all will be embraced.
What if we put this mechanism of recording the world on people. We have mics listening to people talking to us and noises we hear.
Also we record body position actuation and self speech. As output then we put this on thousands of people to get as much data as Waymo gets.
I mean that’s what we need to imitate agi right? I guess the only thing missing is the memory mechanism. We train everything as if it’s an input and output function without accounting for memory.
No human needs to have seen an elephant standing in the road before to know that you should not drive through an elephant standing in the road. These are not "long tail" events as the waymo says. It's a big object in the road. You have seen that hundreds of thousands of times. Calling that a long tail event is an admission that your model has zero ability to generalize.
I'm a little sad that they talk about counterfactuals in the simulations, but then don't show any examples of even a single sharknado or giant loop-de-loop.
One interesting thing from this paper is how big of a LiDaR shadow there is around the waymo car which suggests they rely on cameras for anything close (maybe they have radar too?). Seems LiDaR is only useful for distant objects.
Seems interesting, but why is it broken. Waymo repeatedly directed multiple automated vehicles into the private alley off of 5th near Brannan in SF even after being told none of them have any business there ever, period. If they can sense the weather and stuff then maybe they could put out a virtual sign or fence that notes what appears to be a road is neither a through way nor open to the public? I'm really bullish on automated driving long term, but now that vehicles are present for real we need to start to think about potentially getting serious about finding some way to get them to comply with the same laws that limit what people can do.
Can you explain? I lived in PH, and my guess is that you mean navigating and modeling the unending and constantly changing chaos of the street systems (and lack thereof) is going to be a monumental task which I completely agree with. It would be an impressive feat if possible.
Software doesn’t get confused - it fails. Referring to your software as autonomous when you have to staff a 24/7 response center of humans to control it is not just misleading, it’s a lie.
It's worse than that for you, they never take control of the waymo, the waymo provides solutions but is unsure which one is correct. The people then tell the waymo which solution is the best one.
My understanding is that support is basically playing an RTS (point and click), not a 1P driving game. Which makes sense, if they were directly controlling the vehicles they'd put support in central America for better latency, like the food delivery bot drivers
This isn't news, they've always acknowledged that they have remote navigators that tell the cars what to do when they get stuck or confused. It's just that they don't directly drive the car.
Yeah I have some videos of these drivers in action, I think the sensors are as assistance and but not the whole story, so yeah there’s models lidars etc etc but human factor is there, unfortunately this means we should see soon many cobitics are teleopetated remotely from India and Philippines and the likes to satisfy the greed of these companies to pay peanuts to operate them.
This is not false, but gives the wrong idea that foreigners are driving them in real time.
> After being pressed for a breakdown on where these overseas operators operate, Peña said he didn’t have those stats, explaining that some operators live in the US, but others live much further away, including in the Philippines.
> “They provide guidance,” he argued. “They do not remotely drive the vehicles. Waymo asks for guidance in certain situations and gets an input, but the Waymo vehicle is always in charge of the dynamic driving tasks, so that is just one additional input.”
“When the Waymo vehicle encounters a particular situation on the road, the autonomous driver can reach out to a human fleet response agent for additional information to contextualize its environment,” the post reads. “The Waymo Driver [software] does not rely solely on the inputs it receives from the fleet response agent and it is in control of the vehicle at all times.” [from Waymo's own blog https://waymo.com/blog/2024/05/fleet-response/]
In my opinion there's nothing wrong with it per se, but (a) it's still worth mentioning, because most people have the impression that Waymo cars are completely unassisted, and (b) it makes me wonder how feasible Waymo's operations would be if it weren't for global income inequality.
Have you read the article ? The guys in the Philippines are providing high level executive indications, they don't drive remotely the car or have any low level control of the car.
What's going to happen to all the millions of drivers who will lose their job overnight? In a country with 100 million guns, are we really sure we've thought this through?
Autonomous private cars is not the technological progress you think it is. We’ve had autonomous trains for decades, and while it provides us with a more efficient and cost effective public transit system, it didn’t open the doors for the next revolutionary technology.
Self driving cars is a dead end technology, that will introduce a whole host of new problems which are already solved with public transit, better urban planning, etc.
Trains need tracks, cars - already have the infrastructure to drive on.
> Self driving cars is a dead end technology, that will introduce a whole host of new problems which are already solved with public transit, better urban planning, etc.
Self driving cars will literally become a part of public transit
Unfortunately, many of our urban areas have already been planned (for better or worse) for cars and not the density that makes public transit viable. Autonomous cars will solve a host of problems for the old, young, mobility limited, and just about everyone else.
It will prove disruptive to the driving industry, but I think we’ve been through worse disruptions and fared the better for it.
Nope. Humans are statistically fallible and their attention is too valuable to be obliged to a mundane task like executing navigation commands. Redesigning and rebuilding city transportation infrastructure isn't happening, look around. Also personal agency limits public transportation as a solution.
> Redesigning and rebuilding city transportation infrastructure isn't happening, look around.
The US already did it once (just in the wrong direction) by redesigning all cities to be unfriendly to humans and only navigable by cars. It should be technically possible to revert that mistake.
Unlike autonomous driving, public transit is a proven solution employed in thousands of cities around the world, on various scales, economies, etc.
> Redesigning and rebuilding city transportation infrastructure isn't happening, look around.
We have been redesigning and rebuilding city transportation infrastructure since we had cities. Where I live (Seattle) they are opening a new light rail bridge crossing just next month (first rail over a floting bridge; which is technologically very interesting), and two new rail lines are being planned. In the 1960s the Bay area completely revolutionized their transit sytem when they opened BART.
>> In the 1960s the Bay area completely revolutionized their transit sytem when they opened BART.
66 years later we see California struggling terribly with implementation of a high-speed rail system -- where the placement/location of the infrastructure largely is targeted for areas far less dense than the Bay Area.
I don't think there is any single reason why this is so much more difficult now then it was in 1960 -- but clearly things have changed quite a lot in that time.
> What's going to happen to all the millions of drivers who will lose their job overnight? In a country with 100 million guns, are we really sure we've thought this through?
People keep referencing history but this really is unprecedented. We are approaching singularity and many people will become obsolete in all areas. There are no new hypothetical jobs waiting on the horizon.
I don't think Uber goes out of business. There is probably a sweet spot for Waymo's steady state cars, and you STILL might want 'surge' capabilities for part time workers who can repurpose their cars to make a little extra money here and there.
As to the revolt, America doesn't do that any more. Years of education have removed both the vim and vigor of our souls. People will complain. They will do a TikTok dance as protest. Some will go into the streets. No meaningful uprising will occur.
The poor and the affected will be told to go to the trades. That's the new learn to program. Our tech overlords will have their media tell us that everything is ok (packaging it appropriately for the specific side of the aisle).
Ultimately the US will go down hill to become a Belgium. Not terrible, but not a world dominating, hand cutting entity it once was.
> Ultimately the US will go down hill to become a Belgium.
I'm curious why you say this given you start by highlighting several characteristics that are not like Belgium (to wit, poor education, political media capture, effective oligarchy). I feel there are several other nations that may be better comparators, just want to understand your selection.
Suddenly all this focus on world models by Deep mind starts to make sense. I've never really thought of Waymo as a robot in the same way as e.g. a Boston Dynamics humanoid, but of course it is a robot of sorts.
Google/Alphabet are so vertically integrated for AI when you think about it. Compare what they're doing - their own power generation , their own silicon, their own data centers, search Gmail YouTube Gemini workspace wallet, billions and billions of Android and Chromebook users, their ads everywhere, their browser everywhere, waymo, probably buy back Boston dynamics soon enough (they're recently partnered together), fusion research, drugs discovery.... and then look at ChatGPT's chatbot or grok's porn. Pales in comparison.
Google has been doing more R&D and internal deployment of AI and less trying to sell it as a product. IMHO that difference in focus makes a huge difference. I used to think their early work on self-driving cars was primarily to support Street View in thier maps.
There was a point in time when basically every well known AI researcher worked at Google. They have been at the forefront of AI research and investing heavily for longer than anybody.
It’s kind of crazy that they have been slow to create real products and competitive large scale models from their research.
But they are in full gear now that there is real competition, and it’ll be cool to see what they release over the next few years.
>It’s kind of crazy that they have been slow to create real products and competitive large scale models from their research.
Not really. If Google released all of this first instead of companies that have never made a profit and perhaps never will, the case law would simply be the copyright holders suing them for infringement and winning.
Also think of how LLMs are replacing web searches for most people - Google would have been cannibalising their Search profits for no good reason
It's not even that. It's way easier to do R&D when you don't have a customer base to support.
I also think the presence of Sergey Brin has been making a difference in this.
Ex-googler: I doubt it, but am curious for rationale (i know there was a round of PR re: him “coming back to help with AI.” but just between you and me, the word on him internally, over years and multiple projects, was having him around caused chaos b/c he was a tourist flitting between teams, just spitting out ideas, but now you have unclear direction and multiple teams hearing the same “you should” and doing it)
the rebuke is that lack of chaos makes people feel more orderly and as if things are going better, but it doesn't increase your luck surface area, it just maximizes cozy vibes and self interested comfort.
In Assistant having higher-ups spitting ideas and random thoughts ended up in people mistakenly assume that we really wanted to go/do that, meaning that chaos resulted in ill and cancelled projects.
The worst part was figuring what happened way too late. People were having trying to go for promo for a project that didn't launch. Many people got angry, some left, the product felt stale and leadership&management lost trust.
Isn’t that what the parent is describing? “Ill and cancelled projects” <==> “luck surface area”, and “trying to go for promotion” <==> “cozy vibes and self-interested comfort”?
My dynamic range of professional experience is high, dropout => waiter => found startup => acquirer => Google.
You're making an interesting point that I somewhat agree with from the perspective of someone was...clearly a little more feral than his surroundings in Google, and wildly succeeded and ultimately quietly failed because of it.
The important bit is "great man" theory doesn't solve lack of dynamism. It usually makes things worse. The people you read about in newspapers are pretty much as smart as you, for better or worse.
I actually disagreed with the Sergey thing along the same lines, it was being used as a parable for why it was okay to do ~nothing in year 3 and continue avoiding what we were supposed to ship in year 1, because only VPs outside my org and the design section in my org would care.
Not sure if all that rhymes or will make any sense to you at all. But I deeply respect the point you are communicating, and also mean to communicate that there's another just as strong lesson: one person isn't bright enough to pull that off, and the important bit there isn't "oh, he isn't special", it's that it makes you even more careful building organizations that maintain dynamism and creativity.
Yeah people seem to be pretty poor at judging the impact of 'key' people.
E.g. Steve Jobs was absolutely fundamental to the turn around of Apple. Will Brin have this level of incremental impact on the Goog/Alphabet of today? Nah.
The difference is: Apple had one "key person", Jobs, and yes the products he drove made the company successful. Now Jobs has gone I haven't seen anything new.
But if you look at Google, there isn't one key product. There are a whole pile of products that are best in class. Search (cringe, I know it's popular here to say Google search sucks and perhaps it does, but what search engine is far better?), YouTube, Maps, Android, Waymo, GMail, Deep Mind, the cloud infrastructure, translate, lens (OCR) and probably a lot of others I've forgotten. Don't forget Sheets and Docs, which while they have been replicated by Microsoft and others now were first done by Google. Some of them, like Maps, seem to have swapped entire teams - yet continued to be best in class. Predicting Google won't be at the forefront on the next advance seems perilous.
Maybe these products have key people as you call them, but the magic in Alphabet doesn't seem to be them. The magic seems to be Alphabet has some way to create / acquire these keep people. Or perhaps Alphabet just knows how to create top engineering teams that keep rolling along, even when the team members are replaced.
Apple produced one key person, Jobs. Alphabet seems to be a factory creating lots of key people moving products along. But as Google even manages to replace these key people (as they did for Maps) and still keep the product moving, I'm not sure they are the key to Googles success.
Docs was just an acquisition of Writely, an early „Web 2.0“ document editor service, so „first done by google“ is a bit imprecise
I'm in a similar position and generally agree with your take, but the plus side to his involvement is if he believed in your project or viewpoint he would act as the ultimate red tape cutter.
And there is absolutely nothing more valuable at G (no snark)
(cheers, don't read too much signal into my thoughts, it's more negative than I'd intend. Just was aware it was someone going off PR, and doing hero worship that I myself used to do, and was disabused over 7 years there, and would like other people outside to disabuse themselves of. It's a place, not the place)
That makes sense. A "secret shopper" might be a better way to avoid that but wouldn't give him the strokes of being the god in the room.
He was shopping for other strokes from Google employees: https://finance.yahoo.com/blogs/the-exchange/alleged-affair-...
Oh ffs, we have an external investor who behaves like that. Literally set us back a year on pet nonsense projects and ideas.
What'd he say
That the rocket company should buy an LLM
Please, Google was terrible about using the tech the had long before Sundar, back when Brin was in charge.
Google Reader is a simple example: Googl had by far the most popular RSS reader, and they just threw it away. A single intern could have kept the whole thing running, and Google has literal billions, but they couldn't see the value in it.
I mean, it's not like being able to see what a good portion of America is reading every day could have any value for an AI company, right?
Google has always been terrible about turning tech into (viable, maintained) products.
Is there an equivalent to Godwin's law wrt threads about Google and Google Reader?
See also: any programming thread and Rust.
I'm convinced my last groan will be reading a thread about Google paper clipping the world, and someone will be moaning about Google Reader.
“A more elegant weapon of a civilised age.”
Lol, it seems obvious in retrospect, there really, really, needs to be.
Therefore we now have “Vinkel’s Law”
It's far from the only example https://killedbygoogle.com/
I never get the moaning about killing Reader. It was never about popularity or user experience.
Reader had to be killed because it [was seen as] a suboptimal ad monetization engine. Page views were superior.
Was Google going to support minimizing ads in any way?
Right. Reader was not a case of apathy and failure to see the product’s value.
It was Google clearly seeing the product’s value, and killing it because that value was detrimental to their ads business.
How is this relevant? At best it’s tangentially related and low effort
Can you not vibe code it back into existence yet?
Took a while but I got to the google reader post. Self host tt-rss, it's much better
Because after the death of Epstein he suddenly had a lot of free time?
https://www.wsj.com/finance/jeffrey-epstein-advised-sergey-b...
https://x.com/MarioNawfal/status/2017428928814588323
If this is true, this is disappointing :/
On a similar topic, it is worth mentioning the entrepreneurs that are forced into sex (or let’s say, very pushed) by VCs.
For those who feel safe or taking it as a joke, this affects women AND men.
Some people are going to be disappointed about their heroes.
Dont mention the recent Eric Schmidt scandal.
Barely any of these jokers are clean. Makes MZ look seemingly normal in comparison.
>> If this is true, this is disappointing
Wait for the second set of files...
"...One of Mr. Epstein’s former boat captains told The New York Times earlier this year that he had seen Mr. Brin on the island more than once..."
https://dnyuz.com/2026/01/31/powerful-men-who-turn-up-in-the...
What's striking is the sheer scale of Epstein's and Maxwell's scheduling and access. The source material makes it hard to even imagine how two people could sustain that many meetings/parties/dinners/victims, across so many places, with such high-profile figures. And, how those figures consistently found the time to meet them.
Ghislaine making a speech at the UN... https://youtu.be/-h5K3hfaXx4?t=350
> It’s kind of crazy that they have been slow to create real products and competitive large scale models from their research.
It’s not that crazy. Sometimes the rational move is to wait for a market to fully materialize before going after it. This isn’t a Xerox PARC situation, nor really the innovator’s dilemma, it’s about timing: turning research into profits when market conditions finally make it viable. Even mammoths like Google are limited in their ability to create entirely new markets.
This take makes even more sense when you consider the costs of making a move to create the market. The organizational energy and its necessary loss in focus and resources limits their ability to experiment. Arguably the best strategy for Google: (1) build foundational depth in research and infrastructure that would be impossible for competition to quickly replicate (2) wait for the market to present a clear new opportunity for you (3) capture it decisively by focusing and exploiting every foundational advantage Google was able to build.
> It’s kind of crazy that they have been slow to create real products and competitive large scale models from their research.
I always thought they deliberately tried to contain the genie in the bottle as long as they could
Their unreleased LaMDA[1] famously caused one of their own engineers to have a public crashout in 2022, before ChatGPT dropped. Pre-ChatGPT they also showed it off in their research blog[2] and showed it doing very ChatGPT-like things and they alluded to 'risks,' but those were primarily around it using naughty language or spreading misinformation.
I think they were worried that releasing a product like ChatGPT only had downside risks for them, because it might mess up their money printing operation over in advertising by doing slurs and swears. Those sweet summer children: little did they know they could run an operation with a seig-heiling CEO who uses LLMs to manufacture and distribute CSAM worldwide, and it wouldn't make above-the-fold news.
[1] https://en.wikipedia.org/wiki/LaMDA#Sentience_claims
[2] https://research.google/blog/lamda-towards-safe-grounded-and...
The front runner is not always the winner. If they were able to keep pace with openai while letting them take all the hits and miss steps, it could pay off.
Time will tell if LLM training becomes a race to the bottom or the release of the "open source" ones proves to be a spoiler. From the outside looking while ChatGPT has brand recognition for the average person who could not tell the difference between any two LLMs google offering Gemini in android phones could perhaps supplant them.
I swear the Tay incident caused tech companies to be unnecessarily risk averse with chatbots for years.
Attention is all you need was written by Googlers IIRC.
Indeed, none of the current AI boom would’ve happened without Google Brain and their failure to execute on their huge early lead. It’s basically a Xerox Parc do-over with ads instead of printers.
It has always felt to me that the LLM chatbots were a surprise to Google, not LLMs, or machine learning in general.
Not true at all. I interacted with Meena[1] while I was there, and the publication was almost three years before the release of ChatGPT. It was an unsettling experience, felt very science fiction.
[1]: https://research.google/blog/towards-a-conversational-agent-...
The surprise was not that they existed: There were chatbots in Google way before ChatGPT. What surprised them was the demand, despite all the problems the chatbots have. The pig problem with LLMs was not that they could do nothing, but how to turn them into products that made good money. Even people in openAI were surprised about what happened.
In many ways, turning tech into products that are useful, good, and don't make life hell is a more interesting issue of our times than the core research itself. We probably want to avoid the valuing capturing platform problem, as otherwise we'll end up seeing governments using ham fisted tools to punish winners in ways that aren't helpful either
The uptake forced the bigger companies to act. With image diffusion models too - no corporate lawyer would let a big company release a product that allowed the customer to create any image...but when stable diffusion et al started to grow like they did...there was a specific price of not acting...and it was high enough to change boardroom decisions
ChatGPT really innovated on making the chat not say racist things that the press could report on. Other efforts before this failed for that reason.
Right. The problem was that people under appreciated ‘alignment’ even before the models were big. And as they get bigger and smarter it becomes more of an issue.
Well, I must say ChatGPT felt much more stable than Meena when I first tried it. But, as you said, it was a few years before ChatGPT was publicly announced :)
It was a surprise to OpenAI too. ChatGPT was essentially a demo app to showcase their API, it was not meant to be a mass consumer product. When you think about it, ChatGPT is a pretty awkward product name, but they had to stick with it.
Google and OpenAI are both taking very big gambles with AI, with an eye towards 2036 not 2026. As are many others, but them in particular.
It'll be interesting to see which pays off and which becomes Quibi
Quibi would be if someone came in 10 years from now and said "if we put a lot more money behind spitting out content using characters and settings from Hollywood IP than we'll obviously be way more popular than a tech company can be!"
Use your own sh*t is one of the best way to build excellent products.
Tesla built something like this for FSD training, they presented many years ago. I never understood why they did productize it. It would have made a brilliant Maps alternative, which country automatically update from Tesla cars on the road. Could live update with speed cameras and road conditions. Like many things they've fallen behind
No Lidar anymore on the 2026 Volvo models ES60 and EX60. See for example: https://www.jalopnik.com/2032555/volvo-ends-luminar-lidar-20...
I love Volvo, am considering buying one in a couple weeks actually, but they're doing nothing interesting in terms of ADAS, as far as I can tell. It seems like they're limited to adaptive cruise control and lane keeping, both of which have been solved problems for more than a decade.
It sounds like they removed Lidar due to supplier issues and availability, not because they're trying to build self-driving cars and have determined they don't need it anymore.
Is lane keeping really a solved problem? Just last year one of my brand new rented cars tried to kill me a few times when I tried it again, and so far not even the simple lane leaving detection mechanism worked properly in any of the tried cars when it was raining.
What problem is it even solving? Keeping my car straight so I can be less attentive on the road?
I get it in the context of driverless but find it nothing but annoying as a driver.
Adaptive cruise control requires some degree of lane detection. It has to figure out what car it's actually following, not merely what car is in front of it. (The road is turning, the car in front of you can easily not be the car you are actually behind.)
Lane keep keeps your car in the lane so you can stop paying attention just like cruise control keeps you going the same speed so you can stop paying attention… they don’t.
They are just aids that ease fatigue on long trips.
Lane keep is absolutely not a solved problem. Go test drive any of the latest cars from Kia, Honda, Toyota, Hyundai, Ford, etc. They all will literally kill you.
I’d suggest doing some research on software quality. Two years back I was all for buying one (I was considering an EX40), but I got myself into some Facebook groups for owners and was shocked at the dreadful reports of quality of the software and it completely put me off. I got an ID4 instead. Reports about the EX90 have been dreadful. I was very interested, and I still admire their look and build when they drive by - but it killed my enthusiasm to buy one for a few years until they get it right.
Software is pretty solid as of latest release. EX90 is a sleeper pick now because of the bad press being behind the latest software.
Without Lidar + the terrible quality of tesla onboard cameras.. street view would look terrible. The biggest L of elon's career is the weird commitment to no-lidar. If you've ever driven a Tesla, it gives daily messages "the left side camera is blocked" etc.. cameras+weather don't mix either.
At first I gave him the benefit of the doubt, like that weird decision of Steve Jobs banning Adobe Flash, which ran most of the fun parts of the Internet back then, that ended up spreading HTML5. Now I just think he refused LIDAR on purely aesthetic reasons. The cost is not even that significant compared to the overall cost of a Tesla.
It's important to understand the timeline of the Steve Jobs open letter on Adobe Flash - at that point the iPhone had been out just shy of three years, and before the first public betas on Android. So for nearly three years, Apple had been investing in HTML5 technology because Flash wasn't in a form where it was deployable.
Additionally, Flash required android phones with 256MB ram as a minimum (which would have precluded two of the three shipped iPhone models at the time) and at least initially only supported software video decoding. Because of the difference in screen dimensions, resolutions and interaction models (plus the issues with embedding due to RAM limitations), the website was still basically broken whether your mobile phone had Flash or not.
My understanding (based on the timing) was always that when Adobe was finally ready to push its partners to bundle mobile Flash, Apple looked at it and decided against it. Adobe made public statements against their partner and so Jobs did so in kind.
That one was motivated by the need of controlling the app distribution channel, just like they keep the web as a second class citizen in their ecosystem nowadays.
Years ago he called lidar a crutch...
And I agree, it is. Clearly it is theoretically possible without.
But when you can't walk at all, a crutch might be just what you need to get going before you can do it without the crutch!
> purely aesthetic reasons
This is huge though.
People aren't setting them on fire during protests, and if an FSD Tesla plows into a farmers market, it might not even make the news.
People hate tech so much that self-driving companies with easy-to-spot cars have had to shut down after just a few mistakes.
Disguising Teslas as plain old regular human-driven cars is a great idea and I wouldn't be surprised if they win the market because of this. Even if they suck at driving.
People aren't setting Teslas on fire? Where do you get that from?
https://www.forbes.com/sites/conormurray/2025/05/01/tesla-pr...
he didn't refuse it. MobileEye or whoever cut Tesla off because they were using the lidar sensors in a way he didn't approve. From there he got mad and said "no more lidar!"
Assuming what you say is true, are they the only LIDAR vendor?
False. Mobileye never used lidar. Lmao where do you all come up with this
I think Elon announced Tesla was ditching LIDAR in 2019.[0] This was before Mobileye offered LIDAR. Mobileye has used LIDAR from Luminar Technologies around 2022-2025. [1][2] They were developing their own lidar, but cancelled it. [3] They chose Innoviz Technologies as their LIDAR partner going forward for future product lines. [4]
0: https://techcrunch.com/2019/04/22/anyone-relying-on-lidar-is...
1: https://static.mobileye.com/website/corporate/media/radar-li...
2: https://www.luminartech.com/updates/luminar-accelerates-comm...
3: https://www.youtube.com/watch?v=Vvg9heQObyQ&t=48s
4: https://ir.innoviz.tech/news-events/press-releases/detail/13...
The original Mobileye EyeQ3 devices that Tesla began installing in their cars in 2013 had only a single forward facing camera. They were very simple devices, only intended to be used for lane keeping. Tesla hacked the devices and pushed them beyond their safe design constraints.
Then that guy got decapitated when his Model S drove under a semi-truck that was crossing the highway and Mobileye terminated the contract. Weirdly, the same fatal edge case occurred 2 more times at least on Tesla's newer hardware.
https://en.wikipedia.org/wiki/List_of_Tesla_Autopilot_crashe...
They had radar too. No such incidents since going camera only fyi, even on the old autopilot product
Thank you!
Never with the product used by Tesla early on.
https://www.mobileye.com/news/mobileye-to-end-internal-lidar...
Um, yes they did.
No idea if it had any relation to Tesla though.
Did not
> The cost is not even that significant compared to the overall cost of a Tesla.
That’s true now, but when they first debuted they would have doubled the cost of the car.
When Tesla debuted, the cost of batteries made electric cars more like an expensive novelty. The Tesla roadster certainly was fun, but it wasn't a practical car for day-to-day use.
Of course, things have changed.
Had Tesla gone all-in on Lidar, they could have turned the technology into a commodity, they are a trillion dollar company producing a million cars a year. Lidar is already present on cheap robot vacuum cleaners, and we have time-of-flight cameras in smartphones, I don't believe it would have been a problem to equip $50k cars with Lidar.
His stated reason was that he wanted the team focused on the driving problem, not sensor fusion "now you have two problems" problems. People assumed cost was the real reason, but it seems unfair to blame him for what people assumed. Don't get me wrong, I don't like him either, but that's not due to his autonomous driving leadership decisions, it's because of shitting up twitter, shitting up US elections with handouts, shitting up the US government with DOGE, seeking Epstein's "wildest party," DARVO every day, and so much more.
Sensor fusion is an issue, one that is solvable over time and investment in the driving model, but sensor-can't-see-anything is a show stopper.
Having a self-driving solution that can be totally turned off with a speck of mud, heavy rain, morning dew, bright sunlight at dawn and dusk.. you can't engineer your way out of sensor-blindness.
I don't want a solution that is available to use 98% of the time, I want a solution that is always-available and can't be blinded by a bad lighting condition.
I think he did it because his solution always used the crutch of "FSD Not Available, Right hand Camera is Blocked" messaging and "Driver Supervision" as the backstop to any failure anywhere in the stack. Waymo had no choice but to solve the expensive problem of "Always Available and Safe" and work backwards on price.
> Waymo had no choice but to solve the expensive problem of "Always Available and Safe"
And it's still not clear whether they are using a fallback driving stack for a situation where one of non-essential (i.e. non-camera (1)) sensors is degraded. I haven't seen Waymo clearly stating capabilities of their self-driving stack in this regard. On the other hand, there are such things as washer fluid and high dynamic range cameras.
(1) You can't drive in a city if you can't see the light emitted by traffic lights, which neither lidar nor radar can do.
Hence why both together make the solution waymo chose. The proof is in the pudding, Waymo's have been driving millions of miles without any intervention. Tesla requires safety drivers. I would never trust the FSD on my model 3 to be even nearly perfect all the time.
Lidar also gives you the ability to see through fog and as it scans, see the depth needed to nearly always understand what object is in front of them.
My Model 3 shows "degraded" or "unavailable" about 2% of the time i'm driving around populated areas. Zero chance it will ever be truly FSD capable, no matter the software improvements. It'll still be unavailable because the cameras are blinded/blocked/unable to process the scene because it can't see the scene.
While you're right, washer fluid works usually on the windshield, it doesn't on the side cameras, and yea hdr could improve things, it won't improve depth perception, and this will never be installed on my model 3..
Lidar contributes the data most needed to handle the millions of edge cases that exist. With both camera and lidar contributing the data they are both the best at collecting, the risk of the very worst type of accidents is greatly reduced.
I don't see these stats https://waymo.com/safety/impact/ happening for tesla anytime soon.
> without any intervention
but with occasional remote guidance (Waymo doesn't seem to disclose statistics of that). In some cases remote guidance includes placing waypoints[1].
> Lidar also gives you the ability to see through fog and as it scans
Nah. Lidar isn't much better in fog than cameras. If I'm not mistaken, fog, rain, smoke, snow scatter IR light approximately the same as visible light. The lidar beam needs to travel twice the distance and its power is limited by eye-safety concerns.
> FSD on my model 3 to be even nearly perfect all the time
It doesn't need to be perfect. It needs to not hit things, cars and pedestrians too hard and too often, while mostly obeying traffic rules. Waymo has quite a few complains about their cars' behavior[2], but they manage just fine.
[1] third video in https://waymo.com/blog/2024/05/fleet-response
[2] https://www.austintexas.gov/page/autonomous-vehicles
Waymo had safety drivers for a long time. And still have safety drivers to this day when they roll out a new city. You wouldn't have known that because no one was paying attention to this stuff back then.
Waymo also had safety drivers for years.
All you really need is "drive slower if you can't see (because rain, fog, or degraded cameras), or you're in an area where children might run out into the road"
If you have mud on a camera, you can't drive it either way. Lidar or not. The way to actually solve these issues is to have way more cameras for redundancy / self cleaning etc, not other sensors.
LIDAR is notoriously easy to blind, what are you on about? Bonus meme: LIDAR blinds you(r iPhone camera)!
Yeah its absurd. As a Tesla driver, I have to say the autopilot model really does feel like what someone who's never driven a car before thinks it's like.
Using vision only is so ignorant of what driving is all about: sound, vibration, vision, heat, cold...these are all clues on road condition. If the car isn't feeling all these things as part of the model, you're handicapping it. In a brilliant way Lidar is the missing piece of information a car needs without relying on multiple sensors, it's probably superior to what a human can do, where as vision only is clearly inferior.
The inputs to FSD are:
Source: https://youtu.be/LFh9GAzHg1c?t=571So if they’re already “fusioning” all these things, why would LIDAR be any different?
Tesla went nothing-but-nets (making fusion easy) and Chinese LIDAR became cheap around 2023, but monocular depth estimation was spectacularly good by 2021. By the time unit cost and integration effort came down, LIDAR had very little to offer a vision stack that no longer struggled to perceive the 3D world around it.
Also, integration effort went down but it never disappeared. Meanwhile, opportunity cost skyrocketed when vision started working. Which layers would you carve resources away from to make room? How far back would you be willing to send the training + validation schedule to accommodate the change? If you saw your vision-only stack take off and blow past human performance on the march of 9s, would you land the plane just because red paint became available and you wanted to paint it red?
I wouldn't completely discount ego either, but IMO there's more ego in the "LIDAR is necessary" case than the "LIDAR isn't necessary" at this point. FWIW, I used to be an outspoken LIDAR-head before 2021 when monocular depth estimation became a solved problem. It was funny watching everyone around me convert in the opposite direction at around the same time, probably driven by politics. I get it, I hate Elon's politics too, I just try very hard to keep his shitty behavior from influencing my opinions on machine learning.
> but monocular depth estimation was spectacularly good by 2021
It's still rather weak and true monocular depth estimation really wasn't spectacularly anything in 2021. It's fundamentally ill posed and any priors you use to get around that will come to bite you in the long tail of things some driver will encounter on the road.
The way it got good is by using camera overlap in space and over time while in motion to figure out metric depth over the entire image. Which is, humorously enough, sensor fusion.
It was spectacularly good before 2021, 2021 is just when I noticed that it had become spectacularly good. 7.5 billion miles later, this appears to have been the correct call.
What are the techniques (and the papers thereof) that you consider to be spectacularly good before 2021 for depth estimation, monocular or not?
I do some tangent work from this field for applications in robotics, and I would consider (metric) depth estimation (and 3D reconstruction) starting to be solved only by 2025 thanks to a few select labs.
Car vision has some domain specificity (high similarity images from adjacent timestamps, relatively simpler priors, etc) that helps, indeed.
depth estimation is but one part of the problem— atmospheric and other conditions which blind optical visible spectrum sensors, lack of ambient (sunlight) and more. lidar simply outperforms (performs at all?) in these conditions. and provides hardware back distance maps, not software calculated estimation
Lidar fails worse than cameras in nearly all those conditions. There are plenty of videos of Tesla's vision-only approach seeing obstacles far before a human possibly could in all those conditions on real customer cars. Many are on the old hardware with far worse cameras
Interesting, got any links? Sounds completely unbelievable, eyes are far superior to the shitty cameras Tesla has on their cars.
There's a misconception that what people see and what the camera sees is similar. Not true at all. One day when it's raining or foggy, have some record the driving, through the windshield. You'll be very surprised. Even what the camera displays on the screen isn't what it's actually "seeing".
Yea.. not holding my breath on links to superman tesla cameras performing better than eyes
Always thought the case was for sensor redundancy and data variety - the stuff that throws off monocular depth estimation might not throw off a lidar or radar.
It doesn't solve the "Coyote paints tunnel on rock" problem though.
IIRC, that was only ever a problem for the coyote, though.
Source: not a computer vision engineer, but a childhood consumer of looney toons cartoons.
Time for a car company to call itself "ACME" and the first model the "Road Runner".
Monocular depth estimation can be fooled by adversarial images, or just scenes outside of its distribution. It's a validation nightmare and a joke for high reliability.
It isn't monocular though. A Tesla has 2 front-facing cameras, narrow and wide-angle. Beyond that, it is only neural nets at this point, so depth estimation isn't directly used; it is likely part of the neural net, but only the useful distilled elements.
I never said it was. I was using it as a lower bound for what was possible.
Fog, heavy rain, heavy snow, people running between cars or from an obstructed view…
None of these technologies can ever be 100%, so we’re basically accepting a level of needless death.
Musk has even shrugged off FSD related deaths as, “progress”.
Humans: 70 deaths in 7 billion miles
FSD: 2 deaths in 7 billion miles
Looks like FSD saves lives by a margin so fat it can probably survive most statistical games.
How many of the 70 human accidents would be adequately explained by controlling for speed, alcohol, wanton inattention, etc? (The first two alone reduce it by 70%)
No customer would turn on FSD on an icy road, or on country lanes in the UK which are one lane but run in both directions; it's much harder to have a passenger fatality in stop-start traffic jams in downtown US cities.
Even if those numbers are genuine (2 vs 70) I wouldn't consider it apples-for-apples.
Public information campaigns and proper policing have a role to play in car safety, if that's the stated goal we don't necessarily need to sink billions into researching self driving
Is that the official Tesla stat? I've heard of way more Tesla fatalities than that..
There are a sizeable number of deaths associated with the abuse of Tesla’s adaptive cruise control with lane cantering (publicly marketed as “autopilot”). Such features are commonplace on many new cars and it is unclear whether Tesla is an outlier, because no one is interested in obsessively researching cruise control abuse among other brands.
There are two deaths associated with FSD.
I don't know what he's on about. Here's a better list:
https://en.wikipedia.org/wiki/List_of_Tesla_Autopilot_crashe...
Good ole Autopilot vs FSD post. You would think people on Hacker News would be better informed. Autopilot is just lane keep and adaptive cruise control. Basically what every other car has at this point.
"MacOS Tahoe has these cool features". "Yea but what about this wikipedia article on System 1. Look it has these issues."
That's how you come across
Autopilot is the shitty lane assist. FSD is the SOTA neural net.
Your link agrees with me:
> two that NHTSA's Office of Defect Investigations determined as happening during the engagement of Full Self-Driving (FSD) after 2022.
This is absolutely a Musk defender. FSD and Tesla related deaths are much higher.
https://www.tesladeaths.com/index-amp.html
Autopilot is the shitty lane assist. FSD is the SOTA neural net.
Your link agrees with me:
> 2 fatalities involving the use of FSD
Tesla sales are dead across the world. Cybertruck is a failure. Chinese EVs are demonstrably better.
No one wants these crappy cars anymore.
Isn't there a great deal of gaming going on with the car disengaging FSD milliseconds before crashing? Voila, no "full" "self" driving accident; just another human failing [*]!
[*] Failing to solve the impossible situation FSD dropped them into, that is.
Nope. NHTSA's criteria for reporting is active-within-30-seconds.
https://www.nhtsa.gov/laws-regulations/standing-general-orde...
If there's gamesmanship going on, I'd expect the antifan site linked below to have different numbers, but it agrees with the 2 deaths figure for FSD.
Better than I expected. So this was 3 days ago, is this for all previously models or is there a cut off date here?
I quickly googled Lidar limitations, and this article came up:
https://www.yellowscan.com/knowledge/how-weather-really-affe...
Seeing how its by a lidar vendor, I don't think they're biased against it. It seems Lidar is not a panacea - it struggles with heavy rain, snow, much more than cameras do and is affected by cold weather or any contamination on the sensor.
So lidar will only get you so far. I'm far more interested in mmwave radar, which while much worse in spatial resolution, isn't affected by light conditions, weather, can directly measure stuff on the thing its illuminating, like material properties, the speed its moving, the thickness.
Fun fact: mmWave based presence sensors can measure your hearbeat, as the micro-movements show up as a frequency component. So I'd guess it would have a very good chance to detect a human.
I'm pretty sure even with much more rudimentary processing, it'll be able to tell if its looking at a living being.
By the way: what happened to the idea that self-driving cars will be able to talk to each other and combine each other's sensor data, so if there are multiple ones looking at the same spot, you'd get a much improved chance of not making a mistake.
Lidar is a moot point. You can't drive with just Lidar, no matter what. That's what people don't understand. The most common one I hear: "What if the camera gets mud on it", ok then you have to get out and clean it, or it needs an auto cleaning system.
Maybe vision-only can work with much better cameras, with a wider spectrum (so they can see thru fog, for example), and self-cleaning/zero upkeep (so you don't have to pull over to wipe a speck of mud from them). Nevertheless, LIDAR still seems like the best choice overall.
Autopilot hasn’t been updated in years and is nothing like FSD. FSD does use all of those cues.
I misspoke, i'm using Hardware 3 FSD.
I have HW3, but FSD reliably disengages at this time of year with sunrise and sunset during commute hours.
Yep, and won't activate until any morning dew is off the sensors.. or when it rains too hard.. or if it's blinded by a shiny building/window/vehicle.
I will never trust 2d camera-only, it can be covered or blocked physically and when it happens FSD fails.
As cheap as LIDAR has gotten, adding it to every new tesla seems to be the best way out of this idiotic position. Sadly I think Elon got bored with cars and moved on.
If the camera is covered or blocked, you can't drive plain and simple, as you can't drive a car (at least on Earth) with just Lidar. The roads are made for eyes. Maybe on Rocky's homeworld you can have a Lidar only system for traveling.
This will considerably skew the statistics, a low sun dramatically increases accident rates on humans too.
FSD14 on hw4 does not. Its dynamic range is equivalent or better than human.
>>The biggest L of elon's career is the weird commitment to no-lidar.
I thought it was the Nazi salutes on stage and backing neo-nazi groups everywhere around the world, but you know, I guess the lidar thing too.
maybe it's better to say it was the biggest L of his engineering career instead of his political career
From the perspective of viewing FSD as an engineering problem that needs solving I tend to think Elon is on to something with the camera-only approach – although I would agree the current hardware has problems with weather, etc.
The issue with lidar is that many of the difficult edge-cases of FSD are all visible-light vision problems. Lidar might be able to tell you there's a car up front, but it can't tell you that the car has it's hazard lights on and a flat tire. Lidar might see a human shaped thing in the road, but it cannot tell whether it's a mannequin leaning against a bin or a human about to cross the road.
Lidar gets you most of the way there when it comes to spatial awareness on the road, but you need cameras for most of the edge-cases because cameras provide the color data needed to understand the world.
You could never have FSD with just lidar, but you could have FSD with just cameras if you can overcome all of the hardware and software challenges with accurate 3D perception.
Given Lidar adds cost and complexity, and most edge cases in FSD are camera problems, I think camera-only probably helps to force engineers to focus their efforts in the right place rather than hitting bottlenecks from over depending on Lidar data. This isn't an argument for camera-only FSD, but from Tesla's perspective it does down costs and allows them to continue to produce appealing cars – which is obviously important if you're coming at FSD from the perspective of an auto marker trying to sell cars.
Finally, adding lidar as a redundancy once you've "solved" FSD with cameras isn't impossible. I personally suspect Tesla will eventually do this with their robotaxis.
That said, I have no real experience with self-driving cars. I've only worked on vision problems and while lidar is great if you need to measure distances and not hit things, it's the wrong tool if you need to comprehend the world around you.
This is so wild to read when Waymo is currently doing like 500,000 paid rides every week, all over the country, with no one in the driver's seat. Meanwhile Tesla seems to have a handful of robotaxis in Austin, and it's unclear if any of them are actually driverless.
But the Tesla engineers are "in the right place rather than hitting bottlenecks from over depending on Lidar data"? What?
Tesla has driven 7.5B autonomous miles to Waymo's 0.2B, but yes, Waymo looks like they are ahead when you stratify the statistics according to the ass-in-driver-seat variable and neglect the stratum that makes Tesla look good.
The real question is whether doing so is smart or dumb. Is Tesla hiding big show-stopper problems that will prevent them from scaling without a safety driver? Or are the big safety problems solved and they are just finishing the Robotaxi assembly line that will crank out more vertically-integrated purpose-designed cars than Waymo's entire fleet every day before lunch?
Tesla's also been involved in WAY more accidents than Waymo - and has tried to silence those people, claim FSD wasn't active, etc.
What good is a huge fleet of Robotaxis if no one will trust them? I won't ever set foot in a Robotaxi, as long as Elon is involved.
waymo just hit it's first pedestrian, ever. It did it at a speed of 6mph and it was estimated a human would have hit the kid at 14mph (it was going 17mph when a small child jumped out in front of it from behind a black suv.
First pedestrian struck. That's crazy.
Tesla just disengages fsd anytime a sensor is slightly blocked/covered/blinded.. waymo out here doing fsd 100% of the time and basically never hurts anyone.
I don't get the tesla/elon love here, i like my model 3 but it's never going to get real fsd, and that sucks, elon also lies about the roadmap, timing, etc. I bet the roadster is canceled now. Why do people like inferior sensors and autistic hitler?
Waymos disengage and get tele operated too?
Not really. Waymos can’t be driven remotely, their remote operators can give the car directions, e.g. “use this lane”, and then the autonomous system controls the vehicle to execute those directions.
I’m sure latency and connectivity is too much of an risk to do it any other way.
The only Waymos driven by a human are the ones with human drivers physically in the car
There's more Tesla's on the road than Waymo's by several orders of magnitude. Additionally the types of roads and conditions Tesla's drive under is completely incomparable to Waymo.
Yes that was accounted for above, but this isn't autonomous apples to apples
semi autonomous
I wasn't arguing Tesla is ahead of Waymo? Nor do I think they are. All I was arguing was that it makes sense from the perspective of a consumer automobile maker to not use lidar.
I don't think Tesla is that far behind Waymo though given Waymo has had a significant head start, the fact Waymo has always been a taxi-first product, and given they're using significantly more expensive tech than Tesla is.
Additionally, it's not like this is a lidar vs cameras debate. Waymo also uses and needs cameras for FSD for the reasons I mentioned, but they supplement their robotaxis with lidar for accuracy and redundancy.
My guess is that Tesla will experiment with lidar on their robotaxis this year because design decisions should differ from those of a consumer automobile. But I could be wrong because if Tesla wants FSD to work well on visually appealing and affordable consumer vehicles then they'll probably have to solve some of the additional challenges with with a camera-only FSD system. I think it will depend on how much Elon decides Tesla needs to pivot into robotaxis.
Either way, what is undebatable is that you can't drive with lidar only. If the weather is so bad that cameras are useless then Waymos are also useless.
What causes LiDAR to fail harder than normal cameras in bad weather conditions? I understand that normal LiDAR algorithms assume the direct paths from light source to object to camera pixel, while a mist will scatter part of the light, but it would seem like this can be addressed in the pixel depth estimation algorithm that combines the complex amplitudes at the different LiDAR frequencies.
I understand that small lens sizes mean that falling droplets can obstruct the view behind the droplet, while larger lens sizes can more easily see beyond the droplet.
I seldom see discussion of the exact failure modes for specific weather conditions. Even if larger lenses are selected the light source should use similar lens dimensions. Independent modulation of multiple light sources could also dramatically increase the gained information from each single LiDAR sensor.
Do self-driving camera systems (conventional and LiDAR) use variable or fixed tilt lenses? Normal camera systems have the focal plane perpendicular to the viewing direction, but for roads it might be more interesting to have a large swath of the horizontal road in focus. At least having 1 front facing camera with a horizontal road in focus may prove highly beneficial.
To a certain extend an FSD system predicts the best course of action. When different courses of action have similar logits of expected fitness for the next best course of action, we can speak of doubt. With RMAD we can figure out which features or what facets of input or which part of the view is causing the doubt.
A camera has motion blur (unless you can strobe the illumination source, but in daytime the sun is very hard to outshine), it would seem like an interesting experiment to:
1. identify in real time which doubts have the most significant influence on the determination of best course of action
2. have a camera that can track an object to eliminate motion blur but still enjoy optimal lighting (under the sun, or at night), just like our eyes can rotate
3. rerun the best course of action prediction and feed back this information to the company, so it can figure out the cost-benefit of adding a free tracking camera dedicated to eliminating doubts caused by motion blur.
> Suddenly all this focus on world models by Deep mind starts to make sense
Google's been thinking about world models since at least 2018: https://arxiv.org/abs/1803.10122
FWIW I understood GP to mean that it suddenly makes sense to them, not that there’s been a sudden focus shift at google.
Maybe they were focusing on a real world use that basically requires AI, but not LLMs.
Tesla claimed that all their "real world" recording would give them a moat on FSD.
Waymo is showing that a) you need to be able to incorporate stuff that isn't "real" when training, and b) you get a lot more information from alternate sensors to visible spectrum only.
I just listened to a fantastic multi-hour Acquired (https://www.acquired.fm/) podcast episode on Google and AI that talks about the history of Google and AI and all the ways they have been using it since 2012. It's really fascinating. You can forgive them for not focusing on Reader or any of their other properties when you realize they were pulling in hundreds of billions of dollars of value by making big bets in AI and incorporating it into their core business.
I always understood this to be why Tesla started working on humanoid robots
Pretty much. They banked on "if we can solve FSD, we can partially solve humanoid robot autonomy, because both are robots operating in poorly structured real world environments".
They started working on humanoid robots because Musk always has to have the next moonshot, trillion-dollar idea to promise "in 3 years" to keep the stock price high.
As soon as Waymo's massive robotaxi lead became undeniable, he pivoted to from robotaxis to humanoid robots.
Yeah, that and running Grok on a trillion GPUs in space lol
I don't want a humanoid robot. I want a purpose built robot.
Obviously both will exist and compete with each other on the margins. The thing to appreciate is that our physical world is already built like an API for adult humans. Swinging doors, stairs, cupboards, benchtops. If you want a robot to traverse the space and be useful for more than one task, the humanoid form makes sense.
The key question is whether general purpose robots can outcompete on sheer economies of scale alone.
It's called a dishwasher, washing machine, and dryer. Plus like robomowers, vaccums etc.
I mean, I would take a robot to handle all of my housework.
Purpose built, that probably takes the form of a humanoid robot since all of tasks it needs to do were previously designed for humanoids.
Vacuuming and mopping are not inherently "designed" for humans.
Dusting with a single extensible and multiple degrees of freedom arm would be much more maneuverable than a human arm.
Loading and unloading washing machines or dryers or doign the same for dishes and cutlery in a dishwasher is not inherently designed for humans.
If anything, selling an integrated "housekeeping" system that fits into an existing laundry and combines features would be a much better approach.
I agree that each would be made slightly better with a more integrated system. But you could handle all of them in my hundred year old house with the form factor it was designed for: a humanoid. Probably pretty soon here for cheaper than each could be handled separately by more integrated systems.
The drop in demand for Tesla's clapped out model range would have meant embarrassing factory closures, so now they're being closed to start manufacturing a completely different product. Bait and switch for Tesla investors.
I wonder how long they'll be closed for "modifications" and whether the Optimus Prime robot factories will go into production before the "Trump Kennedy Center" is reopened after its "renovations".
It's so they can stick a Tesla logo on a bunch of chinese tech and call it innovation.
>> Suddenly all this focus on world models by Deep mind starts to make sense.
The apparent applicability to Waymo is incidental, more likely because a few millions+ were spent on Genie and they have to do something with it. DeepMind started to train "world models" because that's the current overhyped buzzword in the industry. First it was "natural language understanding" and "question answering" back in the days of old BERT, then it was "agentic", then "reasoning", now it's "world models", next years it's going to be "emotions" or "social intelligence" or some other anthropomorphic, over-drawn neologism. If you follow a few AI accounts on social media you really can't miss when those things suddenly start trending, then pretty much die out and only a few stragglers still try to publish papers on them because they failed to get the memo that we're now all running behind the Next Big Thing™.
notice that all these buzzwords you give actually correspond to real advances in the field. All of these were improvements on something existing, not a big revolution for sure, but definitely measurable improvements.
Those are not "real advances in the field", which is why they are constantly abandoned for the next new buzzword.
Edit:
This just in:
https://news.ycombinator.com/item?id=46870514#46929215
The Next Big Thing™ is going to be "context learning", at least if Tencent have their way. And why do we need that?
>> Current language models do not handle context this way. They rely primarily on parametric knowledge—information compressed into their weights during massive pre-training runs. At inference time, they function largely by recalling this static, internal memory, rather than actively learning from new information provided in the moment.
>> This creates a structural mismatch. We have optimized models to excel at reasoning over what they already know yet users need them to solve tasks that depend on messy, constantly evolving context. We built models that rely on what they know from the past, but we need context learners that rely on what they can absorb from the environment in the moment.
Yep. Reasoning is so 2025.
I think you might be salty because the words become overused and overhyped, and often 90% of the people jumping on the bandwagon are indeed just parroting the new hot buzzword and don't really understand what they're talking about. But the terms you mentioned are all obviously very real and and very important in applications using LLMs today. Are you arguing that reasoning was vaporware? None of these things were meant to the be the final stop of the journey, just the next step.
So is this a model baked into the VLLM layer? Or a scaffold that the agent sits in for testing?
If the former then it’s relevant to the broader discourse on LLM generality. If the latter, then it seems less relevant to chatbots and business agents.
Edit to add: this is not part of the model, it’s in a separate pillar (Simulator vs Driver). More at https://waymo.com/blog/2025/12/demonstrably-safe-ai-for-auto....
Also known as a monopoly. This should terrify us all.
No, it's known as vertical integration, which is legally permitted by default.
Monopolies are essentially 100% horizontal integration. Vertical integration is a completely different concept.
https://www.youtube.com/watch?v=ODSJsviD_SU&t=3594s
Aren't they still using safety drivers or safety follow cars and in fewer cities? Seems Tesla is pretty far behind.
What do you think I said that you're contradicting?
IMO the presence of safety chase vehicles is just a sensible "as low as reasonably achievable" measure during the early rollout. I'm not sure that can (fairly) be used as a point against them.
I'm comfortably with Tesla sparing no expense for safety, since I think we all (including Tesla) understand that this isn't the ultimate implementation. In fact, I think it would be a scandal if Tesla failed to do exactly that.
Damned if you do and damned if you don't, apparently.
I don't know if Tesla claiming they're doing something carries weight anymore.
Setting aside the anti-Tesla bias, none of what I said relies on Tesla claims. The "chase vehicle" claims are all based on third-party accounts from actual rideshare customers.
> IMO the presence of safety chase vehicles is just a sensible "as low as reasonably achievable" measure during the early rollout. I'm not sure that can (fairly) be used as a point against them.
Only if you're comparing them to another company, which you seem to be. So yes, yes it can.
Seriously, the amount of sheer cope here is insane. Waymo is doing the thing. Tesla is not. If Tesla were capable of doing it, they would be. But they're not.
It really is as simple as that and no amount of random facts you may bring up will change the reality. Waymo is doing the thing.
>Waymo is doing the thing.
This worldview is overly simplistic.
Waymo has (very shrewdly, for prospective investors at least) executed a strategy that most quickly scales to 0.1% of the population. Unfortunately it doesn't scale further. The cars are too costly and the mapping is too costly. There is no workable plan for significant scale from Waymo.
Tesla is executing the strategy that most quickly scales to 100% of the population.
> Tesla is executing the strategy that most quickly scales to 100% of the population.
So, uh… where is this “scale” then? This “strategy” has been bandied about for better part of a decade. Why are they still in a tiny geofence in Austin with chase cars?
Waymo is doing it right now. Half a million rides every week, expansion to a dozen new cities. Tesla does a few hundred in a tiny area.
Scale is assessed by looking at concrete numbers, not by “strategies” that haven’t materialized for a decade.
> most quickly scales to 0.1% of the population. Unfortunately it doesn't scale further
Data suggests that they’re already available to ~2% of the US population.
Practically ALL course introductory materials that regard robotics and AI that I've seen began with "you might imagine a talking bipedal humanoid when you hear the word `robot`, but perhaps the most commonplace robot that you have seen is a vending machine", with the illustration of a typical 80s-90s outdoor soda vendor with no apparent moving parts.
So "maybe cars are a bit of robots too" is more like 30-50 years behind the time.
Grok/xAI is a joke at this point. A true money pit without any hopes for a serious revenue stream.
They should be bought by a rocket company. Then they would stand a chance.
Which is why it's embarrassing how much worse Gemini is at searching the web for grounding information, and how incredibly bad gemini cli is.
Not my experience in either of those areas.
Internal firewalls and poor management means that the vast majority of integration opportunities are missed.
The flywheel is starting to spin......
> I've never really thought of Waymo as a robot in the same way as e.g. a Boston Dynamics humanoid, but of course it is a robot of sorts.
I view Tesla also more as a robot company than anything else.
What an upsetting comment. I'm glad you came around but what did you think was going to be effective before you came around to world models?
"Waymo as a robot in the same way"
Erm, a dishwasher, washing machine, automated vacuum can be considered robots. Im confused as to this obsession of the term - there are many robots that already exist. Robotics have been involved in the production of cars for decades.
......
I think the (gray) line is the degree of autonomy. My washing machine makes very small, predictable decisions, while a Waymo has to manage uncertainty most of the time.
Its irrelevant. A robot is a robot.
Dictionary def: "a machine controlled by a computer that is used to perform jobs automatically."
A robot is a robot, and a human is a creature that won't necessarily agree with another human on what the definition of a word is. Dictionaries are also written by humans and don't necessarily reflect the current consensus, especially on terms where people's understanding might evolve over time as technology changes.
Even if that definition were universally agreed on l upon though, that's not really enough to understand what the parent comment was saying. Being a robot "in the same way" as something else is even less objective. Humans are humans, but they're also mammals; is a human a mammal "in the same way" as a mouse? Most humans probably have a very different view of the world than most mice, and the parent comment was specifically addressing the question of whether it makes sense for an autonomous car to model the world the same way as other robots or not. I don't see how you can dismiss this as "irrelevant" because both humans and mice are mammals (or even animals; there's no shortage of classifications out there) unless you're completely having a different conversation than the person you responded to. You're not necessarily wrong because of that, but you're making a pretty significant misjudgment if you think that's helpful to them or to anyone else involved in the ongoing conversation.
No one is denying that robots existed already (but I would hardly call a dishwasher a robot FWIW)
But in my mind a waymo was always a "car with sensors", but more recently (especially having recently used them a bunch in California recently) I've come to think of them truly as robots.
TIL fuel injectors are robots. Probably my ceiling lights too.
Maybe we need to nitpick about what a job is exactly? Or we could agree to call Waymos (semi)autonomous robots?
In the same way people online have argued helicopters are flying cars, it doesn't capture what most people mean when they use the word "robot", anymore than helicopters are what people have in mind when they mention flying cars.
>or grok's porn
I know it’s gross, but I would not discount this. Remember why Blu-ray won over HDDVD? I know it won for many other technical reasons, but I think there are a few historical examples of sexual content being a big competitive advantage.
It's a 3500lb robot that can kill you.
Boston Robotics is working on a smaller robot that can kill you.
Anduril is working on even smaller robots that can kill you.
The future sucks.
and they're all controlled by (poorly compensated) humans anyway [1] [2]
[1] https://www.wsj.com/tech/personal-tech/i-tried-the-robot-tha...
[2] https://futurism.com/advanced-transport/waymos-controlled-wo...
They couldn't even make burger flipping robots work and are paying fast food workers $20/hr in California.
If that doesn't make it obvious what they can and cannot do then I can't respect the tranche of "hackers" who blindly cheer on this unchecked corporate dystopian nightmare.
They can totally make it work, it’s just currently cheaper to have humans do it.
Solving the technical challenges and using that solution profitably are two completely different things.
The vertical integration argument should apply to Grok. They have Tesla driving data (probably much more data than Waymo), Twitter data, plus Tesla/SpaceX manufacturing data. When/if Optimus starts on the production line, they'll have that data too. You could argue they haven't figured out how to take advantage of it, but the potential is definitely there.
Agreed. Should they achieve Google level integration, we will all make sure they are featured in our commentary. Their true potential is surely just around the corner...
"Tesla has more data than Waymo" is some of the lamest cope ever. Tesla does not have more video than Google! That's crazy! People who repeat this are crazy! If there was a massive flow of video from Tesla cars to Tesla HQ that would have observable side effects.
"More video" (gigabytes) is a straw man.
The key metric is more unusual situations. That scales with miles driven, not gigabytes. With onboard inference the car simply logs anything 'unusual' (low confidence) to selectively upload those needle-in-a-haystack rare events.
But somehow google fails to execute. Gemini is useless for programming and I don’t think even bother to use it as chat app. Claude code + gpt 5.2 xhigh for coding and gpt as chat app are really the only ones that are worth it(price and time wise)
I've recently switched to Claude for chat. GPT 5.2 feels very engagement-maxxed for me, like I'm reading a bad LinkedIn post. Claude does a tiny bit of this too, but an order of magnitude less in my experience. I never thought I'd switch from ChatGPT, but there is only so much "here's the brutal truth, it's not x it's y" I can take.
GPT likes to argue, and most of its arguments are straw man arguments, usually conflating priors. It's ... exhausting; akin to arguing on the internet. (What am I even saying, here!?) Claude's a lot less of that. I don't know if tracks discussion/conversation better; but, for damn sure, it's got way less verbal diarrhea than GPT.
Yes, GPT5-series thinking models are extremely pedantic and tedious. Any conversation with them is derailed because they start nitpicking something random.
But Codex/5.2 was substantially more effective than Claude at debugging complex C++ bugs until around Fall, when I was writing a lot more code.
I find Gemini 3 useless. It has regressed on hallucinations from Gemini 2.5, to the point where its output is no better than a random token stream despite all its benchmark outperformance. I would use Gemini 2.5 to help write papers and all, can't see to use Gemini 3 for anything. Gemini CLI also is very non-compliant and crazy.
Experiencing the same. It seems Anthropic’s human-focused design choices are becoming a differentiator.
To me ChatGPT seems smarter and knows more. That’s why I use it. Even Claude rates gpt better for knowledge answers. Not sure if that itself is any indication. Claude seems superficial unless you hammer it to generate a good answer.
Gemini is by far the best UI/UX designer model. Codex seems to the worst: it'll build something awkward and ugly, then Gemini will take 30-60 seconds to make it look like something that would have won a design award a couple years ago.
Gemini works well enough in Search and in Meet. And it's baked into the products so it's dead simple to use.
I don't think Google is targeting developers with their AI, they are targeting their product's users.
It is a bit mind boggling how behind they were considering they invented transformers and were also sitting on the best set of training data in the world, but they've caught up quite a bit. They still lag behind in coding, but I've found Gemini to be pretty good at more general knowledge tasks. Flash 3 in particular is much better than anything of comparable price and speed from OpenAI or Anthropic.
Yesterday GPT 5.2 wrote a python function for me that had the import in the middle of the code, for no reason. (It was a simple import of requests module in a REST client...) Claude I agree is a lot better for backend,Gemini is very good for frontend
> The Waymo World Model can convert those kinds of videos, or any taken with a regular camera, into a multimodal simulation—showing how the Waymo Driver would see that exact scene.
Subtle brag that Waymo could drive in camera-only mode if they chose to. They've stated as much previously, but that doesn't seem widely known.
I think I'm misunderstanding - they're converting video into their representation which was bootstrapped with LIDAR, video and other sensors. I feel you're alluding to Tesla, but Tesla could never have this outcome since they never had a LIDAR phase.
(edit - I'm referring to deployed Tesla vehicles, I don't know what their research fleet comprises, but other commenters explain that this fleet does collect LIDAR)
They can and they do.
https://youtu.be/LFh9GAzHg1c?t=872
They've also built it into a full neural simulator.
https://youtu.be/LFh9GAzHg1c?t=1063
I think what we are seeing is that they both converged on the correct approach, one of them decided to talk about it, and it triggered disclosure all around since nobody wants to be seen as lagging.
I watched that video around both timestamps and didn't see or hear any mention of LIDAR, only of video.
Exactly: they convert video into a world model representation suitable for 3D exploration and simulation without using LIDAR (except perhaps for scale calibration).
My mistake - I misinterpreted your comment, but after re-reading more carefully, it's clear that the video confirms exactly what you said.
tesla is not impressive, I would never put my child in one
Tesla does collect LIDAR data (people have seen them doing it, it's just not on all of the cars) and they do generate depth maps from sensor data, but from the examples I've seen it is much lower resolution than these Waymo examples.
Tesla does it to map the areas to come up with high def maps for areas where their cars try to operate.
Tesla uses lidar to train their models to generate depth data out of camera input. I don’t think they have any high definition maps.
The purpose of lidar is to prove error correction when you need it most in terms of camera accuracy loss.
Humans do this, just in the sense of depth perception with both eyes.
Human depth perception uses stereo out to only about 2 or 3 meters, after which the distance between your eyes is not a useful baseline. Beyond 3m we use context clues and depth from motion when available.
Thanks, saved some work.
And I'll add that it in practice it is not even that much unless you're doing some serious training, like a professional athlete. For most tasks, the accurate depth perception from this fades around the length of the arms.
ok, but a care is a few meters wide, isn't that enough for driving depth perception similar to humans
The depths you are trying to estimate are to the other cars, people, turnings, obstacles, etc. Could be 100m away or more on the highway.
ok, but the point trying to be made is based on human's depth perception, but a car's basic limitation is the width of the vehicle, so there's missing information if you're trying to figure out if a car can use cameras to do what human eyes/brains do.
Humans are very good at processing the images that come into our brain. Each eye has a “blind spot” but we don’t notice. Our eyes adjust color (fluorescent lights are weird) and the amount of light coming in. When we look through a screen door or rain and just ignore it, or if you look outside a moving vehicle to the side you can ignore the foreground.
If you increase the distance of stereo cameras you probably can increase depth perception.
But a lidar or radar sensor is just sensing distance.
Radar has a cool property that it can sense the relative velocity of objects along the beam axis too, from Doppler frequency shifting. It’s one sense that cars have that humans don’t.
To this point, one of the coolest features Teslas _used_ to have was the ability for it to determine and integrate the speed of the car in front of you AND the speed of the car in front of THAT car, even if the second car was entirely visually occluded. They did this by bouncing the radar beam under the car in front and determining that there were multiple targets. It could even act on this: I had my car AEB when the second ahead car slammed on THEIR brakes before the car ahead even reacted. Absolutely wild. Completely gone in vision-only.
The width of your own vehicle is (pretty much) a constant, and trivial to know. Ford F150 is ~79.9 inches. Done. No sensors needed.
All the shit out there in the world is another story.
You misundestood the assignment.
Write a sonnet about Elon musk.
The company I used to work for was developing a self driving car with stereo depth on a wide baseline.
It's not all sunshine and roses to be honest - it was one of the weakest links in the perception system. The video had to run at way higher resolutions than it would otherwise and it was incredibly sensitive to calibration accuracy.
(Always worth noting, human depth perception is not just based on stereoscopic vision, but also with focal distance, which is why so many people get simulator sickness from stereoscopic 3d VR)
> Always worth noting, human depth perception is not just based on stereoscopic vision, but also with focal distance
Also subtle head and eye movements, which is something a lot of people like to ignore when discussing camera-based autonomy. Your eyes are always moving around which changes the perspective and gives a much better view of depth as we observe parallax effects. If you need a better view in a given direction you can turn or move your head. Fixed cameras mounted to a car's windshield can't do either of those things, so you need many more of them at higher resolutions to even come close to the amount of data the human eye can gather.
Easiest example I always give of this is pulling out of the alley behind my house: there is a large bush that occludes my view left to oncoming traffic, badly. I do what every human does:
1. Crane my neck forward, see if I can see around it.
2. Inch forward a bit more, keep craning my neck.
3. Recognize, no, I'm still occluded.
4. Count on the heuristic analysis of the light filtering through the bush and determine if the change in light is likely movement associated with an oncoming car.
My Tesla's perpendicular camera is... mounted behind my head on the B-pillar... fixed... and sure as hell can't read the tea leaves, so to speak, to determine if that slight shadow change increases the likelihood that a car is about to hit us.
I honestly don't trust it to pull out of the alley. I don't know how I can. I'd basically have to be nose-into-right-lane for it to be far enough ahead to see conclusively.
Waymo can beam the LIDAR above and around the bush, owing to its height and the distance it can receive from, and its camera coverage to the perpendicular is far better. Vision only misses so many weird edge cases, and I hate that Elon just keeps saying "well, humans have only TWO cameras and THEY drive fine every day! h'yuck!"
> owing to its height and the distance it can receive from,
And, importantly, the fender-mount LIDARs. It doesn't just have the one on the roof, it has one on each corner too.
I first took a Waymo as a curiosity on a recent SF trip, just a few blocks from my hotel east on Lombard to Hyde and over to the Buena Vista to try it out, and I was immediately impressed when we pulled up the hill to Larkin and it saw a pedestrian that was out of view behind a building from my perspective. Those real-time displays went a long way to allowing me to quickly trust that the vehicle's systems were aware of what's going on around it and the relevant traffic signals. Plenty of sensors plus a detailed map of a specific environment work well.
Compare that to my Ioniq5 which combines one camera with a radar and a few ultrasonic sensors and thinks a semi truck is a series of cars constantly merging in to each other. I trust it to hold a lane on the highway and not much else, which is basically what they sell it as being able to do. I haven't seen anything that would make me trust a Tesla any further than my own car and yet they sell it as if it is on the verge of being able to drive you anywhere you want on its own.
In fact there are even more depth perception clues. Maybe the most obvious is size (retinal versus assumed real world size). Further examples include motion parallax, linear perspective, occlusion, shadows, and light gradients
Here is a study on how these effects rank when it’s comes to (hand) reaching tasks in VR: https://pubmed.ncbi.nlm.nih.gov/29293512/
Actually the reason people experience vection in VR is not focal depth but the dissonance between what their eyes are telling them and what their inner ear and tactile senses are telling them.
It's possible they get headaches from the focal length issues but that's different.
I keep wondering about the focal depth problem. It feels potentially solvable, but I have no idea how. I keep wondering if it could be as simple as a Magic Eye Autostereogram sort of thing, but I don't think that's it.
There have been a few attempts at solving this, but I assume that for some optical reason actual lenses need to be adjusted and it can't just be a change in the image? Meta had "Varifocal HMDs" being shown off for a bit, which I think literally moved the screen back and forth. There were a couple of "Multifocal" attempts with multiple stacked displays, but that seemed crazy. Computer Generated Holography sounded very promising, but I don't know if a good one has ever been built. A startup called Creal claimed to be able to use "digital light fields", which basically project stuff right onto the retina, which sounds kinda hogwashy to me but maybe it works?
My understanding is that contextual clues are a big part of it too. We see a the pitcher wind up and throw a baseball as us more than we stereoscopically track its progress from the mound to the plate.
More subtly, a lot of depth information comes from how big we expect things to be, since everyday life is full of things we intuitively know the sizes of, frames of reference in the form of people, vehicles, furniture, etc . This is why the forced perspective of theme park castles is so effective— our brains want to see those upper windows as full sized, so we see the thing as 2-3x bigger than it actually is. And in the other direction, a lot of buildings in Las Vegas are further away than they look because hotels like the Bellagio have large black boxes on them that group a 2x2 block of the actual room windows.
Another way humans perceive depth is by moving our heads and perceiving parallax.
How expensive is their lidar system?
Hesai has driven the cost into the $200 to 400 range now. That said I don't know what they cost for the ones needed for driving. Either way we've gone from thousands or tens of thousands into the hundreds dollar range now.
Looking at prices, I think you are wrong and automotive Lidar is still in the 4 to 5 figure range. HESAI might ship Lidar units that cheap, but automotive grade still seems quite expensive: https://www.cratustech.com/shop/lidar/
Those are single unit prices. The AT128 for instance, which is listed at $6250 there and widely used by several Chinese car companies was around $900 per unit in high volume and over time they lowered that to around $400.
The next generation of that, the ATX, is the one they have said would be half that cost. According to regulator filings in China BYD will be using this on entry level $10k cars.
Hesai got the price down for their new generation by several optimizations. They are using their own designs for lasers, receivers, and driver chips which reduced component counts and material costs. They have stepped up production to 1.5 million units a year giving them mass production efficiencies.
That model only has a 120 degree field of view so you'd need 3-4 of them per car (plus others for blind spots, they sell units for that too). That puts the total system cost in the low thousands, not the 200 to 400 stated by GP. I'm not saying it hasn't gotten cheaper or won't keep getting cheaper, it just doesn't seem that cheap yet.
Waymo does their LiDAR in-house, so unfortunately we don’t know the specs or the cost
We know Waymo reduced their LiDAR price from $75,000 to ~$7500 back in 2017 when they started designing them in-house: https://arstechnica.com/cars/2017/01/googles-waymo-invests-i...
That was 2 generations of hardware ago (4th gen Chrysler Pacificas). They are about to introduce 6th gen hardware. It's a safe bet that it's much cheaper now, given how mass produced LiDARs cost ~$200.
Otto and Uber and the CEO of https://pronto.ai do though (tongue-in-cheek)
> Then, in December 2016, Waymo received evidence suggesting that Otto and Uber were actually using Waymo’s trade secrets and patented LiDAR designs. On December 13, Waymo received an email from one of its LiDAR-component vendors. The email, which a Waymo employee was copied on, was titled OTTO FILES and its recipients included an email alias indicating that the thread was a discussion among members of the vendor’s “Uber” team. Attached to the email was a machine drawing of what purported to be an Otto circuit board (the “Replicated Board”) that bore a striking resemblance to – and shared several unique characteristics with – Waymo’s highly confidential current-generation LiDAR circuit board, the design of which had been downloaded by Mr. Levandowski before his resignation.
The presiding judge, Alsup, said, "this is the biggest trade secret crime I have ever seen. This was not small. This was massive in scale."
(Pronto connection: Levandowski got pardoned by Trump and is CEO of Pronto autonomous vehicles.)
https://arstechnica.com/tech-policy/2017/02/waymo-googles-se...
Less than the lives it saves.
Cheaper every year.
Exactly.
Tesla told us their strategy was vertical integration and scale to drive down all input costs in manufacturing these vehicles...
...oh, except lidar, that's going to be expensive forever, for some reason?
> Humans do this, just in the sense of depth perception with both eyes.
Humans do this with vibes and instincts, not just depth perception. When I can't see the lines on the road because there's too much slow, I can still interpret where they would be based on my familiarity with the roads and my implicit knowledge of how roads work, e.g. We do similar things for heavy rain or fog, although, sometimes those situations truly necessitate pulling over or slowing down and turning on your 4s - lidar might genuinely given an advantage there.
That’s the purpose of the neural networks
Yes and no - vibes and instincts isn't just thought, it's real senses. Humans have a lot of senses; dozens of them. Including balance, pain, sense of passage of time, and body orientation. Not all of these senses are represented in autonomous vehicles, and it's not really clear how the brain mashes together all these senses to make decisions.
I think there are two steps here: converting video to sensor data input, and using that sensor data to drive. Only the second step will be handled by cars on road, first one is purely for training.
That is still important for safety reasons in case someone uses a LiDAR jamming system to try to force you into an accident.
It’s way easier to “jam” a camera with bright light than a lidar, which uses both narrow band optical filters and pulsed signals with filters to detect that temporal sequence. If I were an adversary, going after cameras is way way easier.
Oh yeah, point a q-beam at a Tesla at night, lol. Blindness!
If somebody wants to hurt you while you are traveling in a car, there are simpler ways.
Autonomous cars need to be significantly better than humans to be fully accepted especially when an accident does happen. Hence limiting yourself to only cameras is futile.
Surely as soon as they're safer than humans they should be deployed as fast as possible to save some of the 3000 people who are killed by human drivers every day
Of course they should be, but that's not what will happen. Humans are not rational, so self-driving cars must be significantly safer than human drivers to avoid as much political pushback as possible.
They may be trying to suggest that, that claim does not follow from the quoted statement.
I've always wondered... if Lidar + Cameras is always making the right decision, you should theoretically be able to take the output of the Lidar + Cameras model and use it as training data for a Camera only model.
That's exactly what Tesla is doing with their validation vehicles, the ones with Lidar towers on top. They establish the "ground truth" from Lidar and use that to train and/or test the vision model. Presumably more "test", since they've most often been seen in Robotaxi service expansion areas shortly before fleet deployment.
Is that exactly true though? Can you give a reference for that?
I don't have a specific source, no. I think it was mentioned in one of their presentation a few years back, that they use various techniques for "ground truth" for vision training, among those was time series (depth change over time should be continuous etc) and iirc also "external" sources for depth data, like LiDAR. And their validation cars equipped with LiDAR towers are definitely being seen everywhere they are rolling out their Robotaxi services.
are definitely being seen everywhere they are rolling out their Robotaxi services
So...nowhere?
> you should theoretically be able to take the output of the Lidar + Cameras model and use it as training data for a Camera only model.
Why should you be able to do that exactly? Human vision is frequently tricked by it's lack of depth data.
"Exactly" is impossible: there are multiple Lidar samples that would map to the same camera sample. But what training would do is build a model that could infer the most likely Lidar representation from a camera representation. There would still be cases where the most likely Lidar for a camera input isn't a useful/good representation of reality, e.g. a scene with very high dynamic range.
No, I don't think that will be successful. Consider a day where the temperature and humidity is just right to make tail pipe exhaust form dense fog clouds. That will be opaque or nearly so to a camera, transparent to a radar, and I would assume something in between to a lidar. Multi-modal sensor fusion is always going to be more reliable at classifying some kinds of challenging scene segments. It doesn't take long to imagine many other scenarios where fusing the returns of multiple sensors is going to greatly increase classification accuracy.
The goal is not to drive in all conditions; it is to drive in all drivable conditions. Human eyeballs also cannot see through dense fog clouds. Operating in these environments is extra credit with marginal utility in real life.
But humans react to this extremely differently than a self driving car. Humans take responsability, and the self-driving disengages and say : WELP. Oh sorry were you "enjoying your travel time to do something useful" as we very explicitely marketed ? Well now your wife is dead and it's your fault (legally). Kisses, Elon.
Sure, but those models would never have online access to information only provided in lidar data…
No, but if you run a shadow or offline camera-only model in parallel with a camera + LIDAR model, you can (1) measure how much worse the camera-only model is so you can decide when (if ever) it's safe enough to stop installing LIDAR, and (2) look at the specific inputs for which the models diverge and focus on improving the camera-only model in those situations.
By leveraging Genie’s immense world knowledge, it can simulate exceedingly rare events—from a tornado to a casual encounter with an elephant—that are almost impossible to capture at scale in reality. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data.
How do you know the generated outputs are correct? Especially for unusual circumstances?
Say the scenario is a patch of road is densely covered with 5 mm ball bearings. I'm sure the model will happily spit out numbers, but are they reasonable? How do we know they are reasonable? Even if the prediction is ok, how do we fundamentally know that the prediction for 4 mm ball bearings won't be completely wrong?
There seems to be a lot of critical information missing.
The idea is that, over time, the quality and accuracy of world-model outputs will improve. That, in turn, lets autonomous driving systems train on a large amount of “realistic enough” synthetic data.
For example, we know from experience that Waymo is currently good enough to drive in San Francisco. We don’t yet trust it in more complex environments like dense European cities or Southeast Asian “hell roads.” Running the stack against world models can give a big head start in understanding what works, and which situations are harder, without putting any humans in harm’s way.
We don’t need perfect accuracy from the world model to get real value. And, as usual, the more we use and validate these models, the more we can improve them; creating a virtuous cycle.
It's a pareto principal.
You can get 80% of the way to "perfect" with 20% of the effort.
That’s just a platitude at this point. They for all intents and purposes solved the problem, atleast in the US.
I don't think you say "ok now the car is ball bearing proof."
Think of it more like unit tests. "In this synthetic scenario does the car stop as expected, does it continue as expected." You might hit some false negatives but there isn't a downside to that.
If it turns out your model has a blind spot for albino cows in a snow storm eating marshmallows, you might be able to catch that synthetically and spend some extra effort to prevent it.
Looks like they need to blackouts and parades to that simulator...
https://www.yahoo.com/news/articles/waymo-paralyzed-parade-b...
The blackouts circumstance was because they escalate blinking/out of service traffic lights to a human confirmed decision, and they experienced a bottleneck spike in those requests for how little they were staffed. The Waymo itself was fine and was prepared to make the correct decision, it just needed a human in the loop.
In the video from the parade... there's just... people in the road. Like, a lot of small children and actual people on this tiny, super narrow bridge. I think that erring on the side of "don't think you can make it but accidentally drag a small child instead" is probably the right call, though admittedly, these cases are a bit wonky.
>The blackouts circumstance was because they escalate blinking/out of service traffic lights to a human confirmed decision
Which isn't really a scalable solution. In my city the majority of streetlights switch to blinking yellow at night, with priority/yield signs instead. I can't imagine a human having to approve 10 of these on any route.
Isn't that true for any scenario previously unencountered, whether it is a digital simulation or a human? We can't optimize for the best possible outcome in reality (since we can't predict the future), but we can optimize for making the best decisions given our knowledge of the world (even if it is imperfect).
In other words it is a gradient from "my current prediction" to "best prediction given my imperfect knowledge" to "best prediction with perfect knowledge", and you can improve the outcome by shrinking the gap between 1&2 or shrinking the gap between 2&3 (or both)
seems like the obvious answer to that is you cover a patch of road with 5mm ball bearings, and send a waymo to drive across it. if the ball bearings behave the way the simulation says they would, and the car behaves the way the simulation said it would, then you've validated your simulation.
do that for enough different scenarios, and if the model is consistently accurate across every scenario you validate, then you can start believing that it will also be accurate for the scenarios you haven't (and can't) validate.
> from a tornado to a casual encounter with an elephant
A sims style game with this technology will be pretty nice!
>> How do you know the generated outputs are correct? Especially for unusual circumstances?
You know the outputs are correct because the models have many billions of parameters and were trained on many years of video on many hectares of server farms. Of course they'll generate correct outputs!
I mean that's literally the justification. There aren't even any benchmarks that you can beat with video generation, not even any bollocks ones like for LLMs.
You could train it in simulation and then test it in reality.
Would it actually be a good idea to operate a car near an active tornado?
It’s autonomous!
The tornado?
ML models doesn't have fight or flight, so we'll have to show them tornado and teach to run away.
They probably just look at the results of the generation.
I mean would I like a in-depth tour of this? Yes.
But it's a marketing blog article, what do you expect?
> just look at the results of the generation
And? The entire hallucination problem with text generators is "plausible sounding yet incorrect", so how does a human eyeballing it help at all?
I think because here there's no single correct answer that the model is allowed to be fuzzier. You still mix in real training data and maybe more physics based simulation of course but it does seem acceptable that you synthesize extremely tail evaluations since there isn't really a "better" way by definition and you can evaluate the end driving behavior after training.
You can also probably still use it for some kinds of evaluation as well since you can detect if two point clouds intersect presumably.
In much a similar way that LLMs are not perfect at translation but are widely used anyway for NMT.
You should be able to see if it is generated wrong after you see a car driving in it.
I can spot Halluzination in LLM too
All this work is impressive, but I'd rather have better trains
As someone who lives in the Bay Area we already have trains, and they're literally past the point of bankruptcy because they (1) don't actually charge enough maintain the variable cost of operations, (2) don't actually make people pay at all, and (3) don't actually enforce any quality of life concerns short of breaking up literal fights. All of this creates negative synergies that pushes a huge, mostly silent segment of the potential ridership away from these systems.
So many people advocate for public transit, but are unwilling to deal with the current market tradeoffs and decisions people are making on the ground. As long as that keeps happening, expect modes of transit -- like Waymo -- that deliver the level of service that they promise to keep exceeding expectations.
I've spent my entire adult life advocating for transportation alternatives, and at every turn in America, the vast majority of other transit advocates just expect people to be okay with anti-social behavior going completely unenforced, and expecting "good citizens" to keep paying when the expected value for any rational person is to engage in freeloading. Then they point to "enforcing the fare box" as a tradeoff between money to collect vs cost of enforcement, when the actually tradeoff is the signalling to every anti-social actor in the system that they can do whatever they want without any consequences.
I currently only see a future in bike-share, because it's the only system that actually delivers on what it promises.
> they (1) don't actually charge enough maintain the variable cost of operations
Why do you expect them to make money? Roads don't make money and no one thinks to complain about that. One of the purposes of government is to make investment in things that have more nebulous returns. Moving more people to public transit makes better cities, healthier and happier citizens, stronger communities, and lets us save money on road infrastructure.
>Why do you expect them to make money?
I don't.
That's why I said "variable cost of operations."
If a system doesn't generate enough revenue to cover the variable costs of operation, then every single new passenger drives the system closer to bankruptcy. The more "successful" the system is -- the more people depend on it -- the more likely it is to fail if anything happens to the underlying funding source, like a regular old local recession. This simple policy decision can create a downward economic spiral when a recession leads to service cuts, which leads to people unable to get to work reliably, which creates more economic pain, which leads to a bigger recession... rinse/repeat. This is why a public transit system should cover variable costs so that a successful system can grow -- and shrink -- sustainably.
When you aren't growing sustainably, you open yourself up to the whims of the business cycle literally destroying your transit system. It's literally happening right now with SF MUNI, where we've had so many funding problems, that they've consolidated bus lines. I use the 38R, and it's become extremely busy. These busses are getting so packed that people don't want to use them, but the point is they can't expand service because each expansion loses them more money, again, because the system doesn't actually cover those variable costs.
The public should be 100% completely covering the fixed capital costs of the system. Ideally, while there is a bit of wiggle room, the ridership should be 100% be covering the variable capital costs. That way the system can expand when it's successful, and contract when it's less popular. Right now in the Bay Area, you have the worst of both worlds, you have an underutilized system with absolutely spiraling costs, simply because there is zero connection between "people actually wanting to use the system" and "where the money comes from."
They do through taxes or tolls!
"The U.S. generates approximately $17.4 billion in annual toll revenue".
"The total annual cost for road maintenance in the U.S. is in the hundreds of billions of dollars, with estimates showing over $200 billion spent yearly".
> Roads don't make money and no one thinks to complain about that.
Between toll roads, and the toll lanes, they do?
If they paid for themselves then the DoT wouldn’t have a multibillion dollar highway budget, and that’s not even including all the state funding.
The claim wasn't they pay for themselves but that they don't generate any income. If we want to look at externalities, we'd also have to figure out how much the Iraq war cost.
That is for Capex. Govt can always easily spend Capex, but Opex has to be covered by the users, whether its roads or trains.
As a fellow public transit fan, you're on the money. Even the shining stars of transit in the US --- NYC MTA subway and CTA --- have huge qualuty of life issues. I can't fault someone for not wanting to ride trains ever again when someone who hasn't showered in 41 years pulls up with a cart full of whatever the fuck and decides to squat the corner seat closest to the car door and be a living biological weapon during rush hour. Or "showtime."
That's before you consider how it takes 2-4x as long to get somewhere by public transit outside of peak hours and/or well-covered areas. A 20 minute trip from a bar in Queens to Brooklyn by car takes an hour by train after 2300, not including walking time. I made that trip many, many times, and hated it each time.
Is that a public transit problem or a societal/homelessness problem?
It doesn't matter in this context. What matters is the hypothetical person in my post thinking "this is what will happen if my city proposes a train" and voting against any legislation trying to bring this forward where they live, even if they hate driving everywhere.
You're definitely right on (2) and (3). I've used many transit systems across the world (including TransMilenio in Bogota and other latam countries "renowned" for crime) and I have never felt as unsafe as I have using transit in the SFBA. Even standing at bus stops draws a lot of attention from people suffering with serious addiction/mental health problems.
1) is a bit simplistic though. I don't know of any European system that would cover even operating costs out of fare/commercial revenue. Potentially the London Underground - but not London buses. UK National Rail had higher success rates
The better way to look at it imo is looking at the economic loss as well of congestion/abandoned commutes. To do a ridiculous hypothetical, London would collapse entirely if it didn't have transit. Perhaps 30-40% of inner london could commute by car (or walk/bike), so the economic benefit of that variable transit cost is in the hundreds of billions a year (compared to a small subsidy).
It's not the same in SFBA so I guess it's far easier to just "write off" transit like that, it is theoretically possible (though you'd probably get some quite extreme additional congestion on the freeways as even that small % moving to cars would have an outsized impact on additional congestion).
>The better way to look at it imo is looking at the economic loss as well of congestion/abandoned commutes. To do a ridiculous hypothetical, London would collapse entirely if it didn't have transit.
You're making my argument for me. Again, my concern isn't the day-to-day conveniences of funding, my point is that building a fragile system (a system where the funding is unrelated to the usability of the service) is a system that can fail catastrophically... for systems where there are obviously alternatives (say, National Rail which can be substituted for automobile, bus, and airplane service) are less to worry about, because their failure will likely not cause cascading failures. When an entire local economy is dependent on that system -- when there are not viable substitutes -- then you're really looking at a sudden economic collapse if the funding source runs dry, or if the system is ever mismanaged.
This is a big deal. When funding really actually does run out and the system fails, then if the result is an economic cascade into a full blown depression, then you would have been much better off just building the robust system in the long term. I just really don't think people appreciate how systems can just fail. Whether it's Detroit or Caracas, when the economic tides turn in a fragile system people can lose everything in a matter of a few years.
But my point is that noone has a robust system according to you in Europe at least - the bar is so high to cover all operating costs with fares (or is that your point - if so I'm lost - I definitely would not recommend replacing European transit networks with nothing?).
And National Rail isn't replaceable at all with bus/cars/planes. You really underestimate the number of people which commute >1hr into London (100km+). There is just no way to do that journey by car or bus. It would turn a ~1hr commute into a 3hr _each way_ and that's not even considering the complete lack of parking OR the fact suddenly the roads would be at (even more) gridlock with many multiples of commuters.
That's not even getting into what you consider fixed vs variable costs. Are the trains themselves a fixed cost (they should last 30-40 years)? Is track maintenance a fixed cost (this has to be done more often than the trains themselves), etc etc. The 2nd point is very important - a lot of rail operators in the UK can be made profitable or not on your metric by how much the government subsidises track maintenance vs the operators paying for it in track access charges.
Equally, are signalling upgrades (for example) fixed costs? But really they are only required to run more frequent services. So you could argue they are a variable cost?
>Are the trains themselves a fixed cost (they should last 30-40 years)?
Yes
>Is track maintenance a fixed cost (this has to be done more often than the trains themselves)
Yes
>Equally, are signalling upgrades (for example) fixed costs?
Yes
Fixed costs are the costs that don't go away when the passengers go away. Variable cost, typically labor, go away when you don't actually need that additional marginal train. You still have to amortize that train even if it's not on the tracks. You still need to buy that marginal train when the service levels require it. You still have to do track maintenance even when you're not running trains (though, yes, at the very margin there could be some small rate adjustments). When you want to upgrade the signals, it's basically the definition of a fixed cost, because you do it once and it's done.
>And National Rail isn't replaceable at all with bus/cars/planes. You really underestimate the number of people which commute >1hr into London (100km+). There is just no way to do that journey by car or bus. It would turn a ~1hr commute into a 3hr _each way_ and that's not even considering the complete lack of parking OR the fact suddenly the roads would be at (even more) gridlock with many multiples of commuters.
I don't want to speak to National Rail or British Rail that preceded it. I want to stick to the transit system that I know well.
My point here isn't that money shouldn't be spent on "getting things back in shape" here is where I waffle on the "pay for fixed capital costs and mostly have the marginal variable costs covered by the marginal rider." If a system needs the occasional cash infusion, I'm fine with that, as long as it comes with new leadership.
My concern here is that, in the Bay Area, many, many people are eager to pay $25 for a Waymo to pick them up (they are NOT cheap) while Muni costs $3 (a near 10x increase in cost). When folks are willing to pay that much of a premium, then something is very wrong with the transit system. Muni has had zero enforcement of their code of conduct for decades. When you have a system that are large section of the populous actively avoid when it's perfectly convenient, then something is very wrong with the system.
When I see BART stations that look like abandoned parking lots surrounded by single family home sprawl, then it doesn't surprise me that the system is not sustainable. The stations that may get removed are all in areas that require people to drive, to then take the train, instead of the cities zoning density and retail around the train stations. When I yell at the occasional people smoking in BART stations and I go to tell the station attendant and get a shrug back -- even when we are paying for them to have their own police force -- that's why they are failing. These are political choices that BART has made in how they operate their service
These systems aren't even doing the bare minimum in providing a reliable pleasant service, so people stop using them, and that makes sense. The entire point is that these services should be relatively inexpensive to operate because of economies of scale, but when you don't actually make people pay, when you don't actually ask people to behave like responsible adults, when your running the service like a failing business then we should expect the service to fail, and when it does, when bailouts are needed, they should (and often do) come with strings attached. BART now has gates that stop most turnstile jumping... and they were forced to be installed by the state of California as part of their second bailout. The reason I'm harping on having variable costs attached to ridership is exactly because the systems needs to be forced to respond when a sizable amount of people no longer find the service valuable.
This is about sustainability, because the marginal tax dollar is better spend on something like providing people with the healthcare they need than it is providing people a bus service they're not even willing to actually use.
Well then invest in those things, then. It would probably cost less than the amount they're spending to make a Waymo World Model.
Lighting money on fire by funding an extremely expensive system that most people don't want to use is not an "investment." It's just a good way to make everyone much poorer and worse off than if we'd done nothing. The only way to change things is to convince the electorate that we actually do need rules and enforcement and a sustainable transportation system.
This isn't just happening in America. Train systems are in rough shape in the UK and Germany too.
Ebike shares are a much more sustainable system with a much lower cost, and achieve about 90% of the level of service in temperate regions of the country. Even the ski-lift guy in this thread has a much more reasonable approach to public transit, because they actually have extremely low cost for the level of service they provide. Their only real shortcoming is they they don't handle peak demand well, and are not flexible enough to handle their own success.
> most people don't want to use
I'm not sure if this was intended or not, but this is a common NIMBY refrain. The argument of "This thing being advocated for that I'm fighting against isn't something people want anyway". And like walkable neighborhood architecture, extremely few Americans have access to light rail. Let alone light rail that doesn't have to yield to car traffic.
Regardless, the cost arguments fall apart once you take the total cost society pays for each system instead of only what the government pays. Because when you get the sum of road construction & maintenance, car acquisition, car maintenance, insurance, and parking, it dwarfs the cost of the local transit system. Break it down on a per-consumer basis and it gets even uglier. New York City is a good example to dive into, especially since it's the typical punching bag for "out-of-control" budgets.
Quick napkin math pins the annual MTA cost at $32-$33 billion and the total cost of the car system between $25 and $44 billion per year. Since the former serves somewhere around 5.5 million riders, and the latter only about 2 million, the MTA costs $5,300-6,600 per user annually where the car system costs $12,000–$22,500 per user annually.
You seem to be misunderstanding my point. I am a transit alternatives advocate, and have been my entire adult life.
I'm NOT saying "people don't want to ride trains."
I AM saying "people don't want ride trains that allow 5% of the riders to smoke cigarettes on enclosed train platforms and in enclosed train cars."
You might says "what? but that's not happening."
In Chicago, yes it is: https://resphealth.org/snuff-out-smoking-on-cta/
People want transit as long as that transit reasonably meets their quality of life standards. The reason why automobiles have been so popular -- even while being wildly more expensive -- is exactly that they allow the user to adjust their travel to their optimal quality of life expectations.
Public transit advocates need to be honest with themselves that anti-social behavioral issues really matter to people. People are willing to pay more to have a more pleasant experience. When a transit system fails to meet that standard, then you'll suddenly find yourself with a transit system that people don't want to use.
> I AM saying "people don't want ride trains that allow 5% of the riders to smoke cigarettes on enclosed train platforms and in enclosed train cars."
Just don't allow that then?
> Public transit advocates need to be honest with themselves that anti-social behavioral issues really matter to people. People are willing to pay more to have a more pleasant experience. When a transit system fails to meet that standard, then you'll suddenly find yourself with a transit system that people don't want to use.
"we can't have good transit because a few people who call themselves transit advocates have bad opinions" is very defeatist. Weak-spined politicians find it much easier to just set money on fire than actually solving problems, so even though most transit advocacy groups in the US emphasize quality and being less wasteful with budgets, your politicians usually prefer the worse options.
>> I AM saying "people don't want ride trains that allow 5% of the riders to smoke cigarettes on enclosed train platforms and in enclosed train cars."
>Just don't allow that then?
>"we can't have good transit because a few people who call themselves transit advocates have bad opinions" is very defeatist.
My point here is only that this is a hard problem, not a trivial one. When the transit advocates in my area just say "transit should be free" in response to "transit pricing is a complex problem that affects system fragility" and they say "stop hating homeless people" in response to "quality of life concerns matter to keeping the system functional long term" then we're in bad place, because the non-transit advocates literally want to get rid of the system. The last TWO Muni funding bills in SF failed.
We've built a system that can fail catastrophically, in large part, because transit advocates don't want to deal with the realities of running a functional transit system. This is why I get grumpy when people say "all this work is impressive, but I'd rather have better trains" when it's very clear why Waymo is succeeding as Muni is failing, but it is exactly because Muni is mostly disconnected from market forces that we've got to this place, and the "solution" being proposed by most transit advocates is to just completely remove all market forces which will very obviously be worse is the long run.
By various definitions of most people, most people do want to use public transit.
It's just that Cars have rotted the American mind so much that to consider anything else is sacrilege.
People want to use it everywhere in the world
People want to have their cake and eat it too.
Maybe not BART but the new Caltrain electrification program seems to be a success and ridership and revenue are up
https://www.yahoo.com/news/articles/caltrain-says-could-clos...
over the long term, this is solved with a wealth tax, but undoing what rich ppl have done to society (i.e. making lots of poor people) will unfortunately take many, many years; so many years that it will never actually happen
My entire point is mostly not even about the money. It's about the system having to respond as a service to the fact that people don't want to use that service and are willing to pay a huge premium for alternatives like Waymo.
Very few transit agencies have fares that cover services. I know others said this, but I wanted to add my take as well
I’m not advocating that they do. Fixed costs should be fully subsidized. I’m only advocating that revenues are set so that during a median year, each additional rider on average, provides income that is proportional to the level of service needed to move that rider through the system.
Trains work in every city in Europe and Asia.
It's worth noting that, at least for bart, the reason that it is facing bankruptcy is precisely because it was mostly rider supported and profitable, and not government supported.
When ridership plummeted by >50% during the pandemic, fixed costs stayed the same, but income dropped. Last time I checked, if Bart ridership returned to 2019 levels, with no other changes, it would be profitable again.
You can't say that BART "is facing bankruptcy is precisely because it was mostly rider supported and profitable, and not government supported" when it is very obvious that BART would be in a much worse situation if it had had more government support... because all those governments are facing massive budget deficits right now.
BART has already been bailed out by the state, twice. It has already failed, twice. It very much needs to reduce the level service it provides if it wants to be sustainable, or seek other forms of revenues while we wait to see if ridership returns. Many have suggested BART explore the SE Asian model of generating revenues by developing residential housing, which seems fairly straightforward.
If ridership never returns, then we ought not continue throwing good money after bad, and we ought to adjust the level of service to meet the level of revenues. Obviously the main problem here is that it's literally illegal to just build high density corridors directly adjacent to the transit stations... which is what we ultimately need to prioritize.
The government facing a budget deficit doesn't mean BART would be worse off with more subsidies.
Where does the extra money come from in a deficit period?
> BART, Muni, Caltrain, AC Transit — which an independent analysis confirmed face annual deficits of more than $800 million annually starting in fiscal year 2027-28
https://www.usatoday.com/story/news/california/2026/01/06/ba...
Nearly a billion dollar shortfall per year going forward. That’s nontrivial, and the state has lost patience with the systems after providing two bailouts already.
Taxes? The same place tons of other stuff we buy as a society comes from. I expect the ballot measure this fall will pass, worst case they file bankruptcy and will probably need to reduce service
I mean, sure? I'd prefer to have a system that has a system built in that raises and lowers the level of service in accordance with the number of people using the system rather than having to have random elections that decide whether or not we're going to effectively scrape a large parts of the system.
I think there's two important things here:
1. You want to be forward looking, not backwards looking. Cutting services means less ridership means less revenue means cutting services means...etc. Bart is super useful for me during the week because headways from SF to West Oakland are often 5m. As I'm writing this (11 on a Friday) I missed a train and had to wait 20 minutes. Every seat on the car is also full, and while not packed, it's standing room only. If my choice is to wait 20 mins for the next train, other ways of getting places become a lot more appealing.
2. Government services should be good. This is good both because it makes people interested in using them (see 1) and because people who don't have other options deserve good services. The point of government is, at least in part, to serve those who can't serve themselves. I don't expect Bart to be revenue neutral for the same reason I don't expect CalFRESH to be.
> Cutting services means less ridership means less revenue means cutting services means...etc.
That's not true. If you have stations that are revenue positive and stations that are net negative, then cutting ridership at the net-negative stations can put the system in a much better financial position. E.g. If BART didn't end at Antioch, and instead continued to Rio Vista, it's entirely likely that the Rio Vista station would just cost more to operate than is worth operating. It takes time to go back and forth, nobody will ever want to be picked up there because it's car-dependent sprawl. Maybe have one or two stops there during rush hour, but you'll likely be better financially cutting most service.
>headways from SF to West Oakland are often 5m
Nobody is suggesting cutting service between SF and Oakland. I'm sure it's a wildly profitable route. Crossing the bay is the main benefit of BART.
>The point of government is, at least in part, to serve those who can't serve themselves. I don't expect Bart to be revenue neutral for the same reason I don't expect CalFRESH to be.
I also don't expect BART to be revenue neutral. I expect it to be funded -- in very large part -- by taxes. I'm only arguing it should be sustainable. It shouldn't get to the point of literal collapse during economic downturns (again, it's already been bailed out by the state and feds, twice, in the last six years).
I really don't think people realize what I'm getting at. I'm saying the system needs to be functional and needs to function long term. Yes, I think we should subsidize low-income users. Yes, I think people who can't afford it should still be able to use it. But that has to happen in a way that doesn't drive away significant numbers of other users. There's nothing about being low-income that means anti-social. I'm talking about anti-social behavior. I'm talking about people smoking cigarettes and using drugs on BART platforms and in BART cars. I'm talking about people who are actively bothering significant numbers of people around them by their behavior -- behavior that is against BART policy, but is tolerated.
You can't sit here and tell me the current system is working when BART is perpetually collapsing. I care about BART. That's why I'm articulating the systemic problems in the system.
> You can't say that BART "is facing bankruptcy is precisely because it was mostly rider supported and profitable, and not government supported" when it is very obvious that BART would be in a much worse situation if it had had more government support... because all those governments are facing massive budget deficits right now.
I don't think this follows. Government budgeting isn't zero based. A hypothetical Bart with 2x the government funding in 2019 would have faced cutbacks, but likely has more money today than what we have now!
> or seek other forms of revenues while we wait to see if ridership returns.
Yes, this is called "taxes".
> If ridership never returns, then we ought not continue throwing good money after bad
Agreed if it was stagnant, but ridership is up more than 10% y/y and that was also true last year. It's on track to be revenue neutral again in a few years. Gutting services today would be exactly opposite of what you'd do for something like a startup showing clear path toward profitability.
> Obviously the main problem here is that it's literally illegal to just build high density corridors directly adjacent to the transit stations... which is what we ultimately need to prioritize
While sure it's hard, there's lots of these that exist. There's new stuff in oakland basically constantly, and were even seeing midrise stuff along Bart in SF, but it's units being built now, so they won't be available until 2027, which is when your proposed service cuts would hit.
>Government budgeting isn't zero based. A hypothetical Bart with 2x the government funding in 2019 would have faced cutbacks, but likely has more money today than what we have now!
A hypothetical BART with 2x the government funding wouldn't have existed... because it didn't exist.
>Agreed if it was stagnant, but ridership is up more than 10% y/y and that was also true last year. It's on track to be revenue neutral again in a few years. Gutting services today would be exactly opposite of what you'd do for something like a startup showing clear path toward profitability.
You're mistaking what I'm saying. I want BART to flourish, but I want it to be sustainable. The choice isn't "keep it open" or "close it." How it is operated matters significantly. I'm very obviously going to vote to increase funding, my point is that it shouldn't have to come to a vote. If service is reduced to a more sustainable rate, the system could recover organically. The revenue jump that has happened at stations immediately after the gates were installed, for example, shouldn't surprise anyone. I'm a transit advocate, BART is mostly irrelevant to this discussion anyway, because we're talking about situations where Waymo is a viable alternative, which really doesn't apply to BART.
Trains need well behaved people, otherwise they are shit.
I don't want to hear tiktok or full volume soap operas blasting at some deaf mouth breather.
I don't want to be near loud chewing of smelly leftovers.
I don't want to be begged for money, or interact with high or psychotic people.
The current culture doesn't allow enforcement of social behaviour: so public transport will always be a miserable containment vessel for the least functional, and everyone with sense avoids the whole thing.
> everyone with sense avoids the whole thing
Or the majority of the residents of New York City on their daily commute? I like to think I have sense, and I happily use public transport most days. I prefer it to sitting in traffic, isolated in a car. At least I can read a book. If you work too hard to insulate yourself from the world, the spaces you'll feel comfortable in will get more and more narrow. I think that's a bad thing.
The cost of avoiding public transport in NYC is massive compared to most cities...
Living there, without the means to avoid public transport is something I would also consider insane.
NYC people uses it because the alternatives are either slower or much more expensive. I'm sure they'd rather use a waymo if it was as fast and cheap as the subway.
I think the point is doubting whether it is ever possible for Waymo to ever be as fast or cheap as public transport in NYC.
> some deaf mouth breather
I quite agree with the overall point but can we leave this kind of discourse on X, please? It doesn't add much, it just feels caustic for effect and engagement farming.
Roads (cars) need well behave people too. The only way cars filter some of the out is by the price.
Price helps a whole lot, I think more than you give it credit for. Driving is also an active thing, this also helps.
We also police driving behaviour, in a way that nobody does for public behaviour.
And no matter what I don't have to hear or smell other drivers.
The vast majority of the anti-social behavior on public transit not relevant in automobiles because (1) you can't turnstile jump the gas tank, (2) an automobile is effectively very expensive set of headphones, and (3) you can inhale whatever you want in your vehicle and your neighbor doesn't have to breath it.
Automobiles are a wildly inefficient and expensive form of transportation in urban areas. At the same time, we ought to be willing to ask why a significant amount of our urban population still prefers to pay all that extra money to sit in traffic.
I think they have a point. But the anti-social behaviors in a car on the road are mostly a different set of anti social behaviors than you’d see on a bus or train. But they certainly exist.
Oh, no doubt that people can be anti-social on the road. My only point is that anti-social behavior on the road is different in kind.
No matter what, people are going to still use cars because they are an absolute advantage over public transportation for certain use cases. It is better that the existing status quo is improved to reduce death rates, than hope for a much larger scale change in infrastructure (when we have already seen that attempts at infrastructure overhaul in the US, like high-speed rail, is just an infinitely deep money pit)
Even though the train system in Japan is 10x better than the US as a whole, the per-capita vehicle ownership rate in Japan is not much lower than the US (779 per 1000 vs 670 per 1000). It would be a pipe dream for American trains/subways to be as good as Japan, but even a change that significant would lead to a vehicle ownership share reduced by only about 13%.
Isn't a vehicle that goes from anywhere to anywhere on your own schedule, safely, privately, cleanly, and without billions in subsidies better?
I don't think individual vehicles can ever achieve the same envirnmental economies of scale as trains. Certainly they're far more convenient (especially for short-haul journeys) but I also think they're somewhat alienating, in that they're engineering humans out of the loop completely which contributes to social atomization.
> I don't think individual vehicles can ever achieve the same envirnmental economies of scale as trains.
I think you'd be surprised. Look at the difference in cost per passenger mile.
I'm looking. Comes out unfavorably to cars. Obviously.
I guess you're comparing the total cost of trains vs a subset of costs of cars, as is usual. Road use and pollution are free externalities after all.
Trains only require subsidies in a world where human & robot cars are subsidized.
As soon as a mode of transport actually has to compete in a market for scarce & valuable land to operate on, trains and other forms of transit (publicly or privately owned) win every time.
>cleanly >without subsidies
Source? The biggest source of environmental issues from EVs, tire wear from a heavier vehicle, absolutely applies to AVs. VC subsidizing low prices only to hike them later isn't exactly "without subsidy" - we pay for it either way
Cars don't work in dense places.
Sure but most of the world has a density low enough that cars work and trains don't really. I like trains as much as the next nerd, but you're never going to be able to take a train from your house to your local farm shop or whatever.
Where trains work they are great. Where they don't, driverless electric cars seem like a great option.
Most of the world's population lives in places where trains and public transit works far better than cars. Density doesn't move around, people do.
I don’t believe the data supports that claim.
https://csh.ac.at/news/over-half-of-global-commutes-are-by-c...
Not necessarily, and your premise is incorrect.
better for the person vs better for the people
sure, a private vehicle is better for me, but a train is better for the world
>without billions in subsidies
Is there a magic road wand?
No, but roads are paid for by road users (i.e. everyone).
AFAICT, the majority (60%) of funding for roads doesn't come from direct user charges...
Roads are subsidized, free parking (and generally a lot of paid parking) is subsidized, and the sprawl encouraged by car dependence combined with the resulting infrastructure costs has and will continue to bankrupt cities.
I don't think we should "just only have trains", but the current US transit landscape is absurdly stupid and inefficient.
Yeah I think the single biggest red flag that the US absolutely could support more public transit is the fact that many cities successfully had more public transit in the past.
So its subsidized? I thought that was the problem.
Huh? Last I looked, roads are paid for by the general public, not (car) road users?
Who pays for the war in Iraq?
Billions of subsidies? Im confused you talking about cars or trains.
No major US public transportation system is fully paid for by riders.
Yep. https://www.transitwiki.org/TransitWiki/index.php/Farebox_Re... is a sobering reminder that many cities’ public transportation would cost $20-50 per trip if paid entirely by riders and thus could not exist without subsidy.
Neither is any private transportation system?
That includes cars on public roads.
Public transportation is the backbone of a functioning economy. It doesn't need to be fully paid by riders precisely because the rest of society benefits from it multiple times over.
NYC congestion pricing seems to be working quite well though, and probably helps offset MTA costs.
NYC "congestion" pricing (actually cordon pricing) is a good idea. Would be great to see more road use fees proportional to use (distance, weight^3, etc.).
Me too but given our extensive car brain culture, Waymo is an amazing step to getting less drivers & cars off the road, and to further cement future generations not ever needing to drive or own cars
Ski lifts man, ski lifts all over the city
What a glorious utopia we could have
> Ski lifts man, ski lifts all over the city
Don't they have those somewhere in South America?
Quite a few places. Cable propelled transit (CPT) is the term to search for. https://en.wikipedia.org/wiki/Gondola_lift#Urban_transport
Pretty much this. Wild that you can traverse most of China in affordable high speed trains, yet the Amtrak from Seattle to Portland barely crawls along and has to regularly stop for long periods of time because the tracks get too hot in the Summer.
I think future generations will resent us for bureaucratizing our way out of the California HSR.
I'd rather be able to go wherever I want.
Enough with the trains. I’m all for trains but theyre good for in city or 1-3 hour journeys. Taking a train across the US would take a day even with high speed trains.
I’d much rather have my own vehicle than share my space with a bunch of people.
The novel aspect here seems to be 3D LiDAR output from 2D video using post-training. As far as I'm aware, no other video world models can do this.
IMO, access to DeepMind and Google infra is a hugely understated advantage Waymo has that no other competitor can replicate.
This is the real story buried under the simulation angle. If you can generate reliable 3D LiDAR from 2D video, every dashcam on earth becomes training data. Every YouTube driving video, every GoPro clip, every security camera feed.
Waymo's fleet is ~700 cars. The internet has millions of hours of driving footage. This technique turns the entire internet into a sensor suite. That's a bigger deal than the simulation itself.
3d from moving 2d images has been a thing for decades.
This is 3D LiDAR output (multimodal) from 2D images.
LiDAR is the technology used to do spatial capture. The output is just point clouds of surfaces. So they’re generating surface point clouds from video
It's not unheard of, there are a handful [0] of metric monodepth methods that output data that's not unlike a really inaccurate 3D lidar, though theirs certainly looks SOTA.
[0] https://github.com/YvanYin/Metric3D
The "world model" is a convenient fiction. Whether we’re talking about a carbon-based brain or a silicon-based transformer, there is no miniature, objective map of reality tucked away inside. What we mistake for a "model" is actually just the layered residue of experience.
From the perspective of enactivism and radical empiricism, intelligence doesn't "represent" the world; it simply navigates it. A biological organism doesn't need a 3D CAD file of a tree to survive; it only needs a history of sensory-motor contingencies—the "if I move this way, I see that" patterns. It’s a synthesis of interactions, not a library of blueprints.
AI operates on the same logic, albeit through a different medium. It isn't simulating the physical laws of the universe or "understanding" gravity. Instead, it navigates the high-dimensional geometry of human data. It’s a sophisticated engine of association, performing a high-speed synthesis of the patterns we've left behind.
In this view, "knowing" isn't about matching an internal image to an external truth. It is the seamless flow of past inputs into future predictions. There is no world model—only the habit of being.
It’s impressive to see simulation training for floods, tornadoes, and wildfires. But it’s also kind of baffling that a city full of Waymos all seemed to fail simultaneously in San Francisco when the power went out on Dec 22.
A power outage feels like a baseline scenario—orders of magnitude more common than the disasters in this demo. If the system can’t degrade gracefully when traffic lights go dark, what exactly is all that simulation buying us?
All this simulation buys a single vehicle that drives better. That failure was a fleet-wide event (overloading the remote assistance humans).
That is, both are true: this high-fidelity simulation is valuable and it won't catch all failure modes. Or in other words, it's still on Waymo for failing during the power outage, but it's not uniquely on Waymo's simulation team.
They've also been seen driving directly into flood waters, with one driving through the middle of a flooded parking lot.
https://www.reddit.com/r/SelfDrivingCars/comments/1pem9ep/hm...
curious what your take away from that is given the announcement.
cue the bell curve meme for learning autonomy:
Seems like it, no?We started with physics-based simulators for training policies. Then put them in the real world using modular perception/prediction/planning systems. Once enough data was collected, we went back to making simulators. This time, they're physics "informed" deep learning models.
That's a very interesting way of looking at it. Yes, you start with simulating something simpler than the real world. Then you use the real world. Then you need to go back to simulations for real-world things that are too rare in the real world to train with.
Seems like there ought to be a name for this, like so-and-so's law.
hazrmard's law
It is thus.
Deepmind's Project Genie under the hood (pun intended). Deepmind & Waymo both Alphabet(Google) subsidiaries obv.
https://deepmind.google/blog/genie-3-a-new-frontier-for-worl...
Discussed here,eg.
Genie 3: A new frontier for world models (1510 points, 497 comments)
https://news.ycombinator.com/item?id=44798166
Project Genie: Experimenting with infinite, interactive worlds (673 points, 371 comments)
https://news.ycombinator.com/item?id=46812933
Regardless of the corporate structure DeepMind is a lot more than just another Alphabet subsidiary at this point considering Demis Hassabis is leading all of Google AI.
Finally I understand the use case for Genie 3. All the talk about "you can make any videogame or movie" seems to have been pure distraction from real uses like this: limited, time-boxed simulated footage.
IIUC, there's a confusion of meaning for "World Model", between Waymo/Deepmind's which is something that can create a consistent world (for use to train Waymo's Driver), vs Yann LeCun/Advanced Machine Intelligence (AMI) which is something that can understand a world.
I don't think there's a conflict. If you can predict the world you understand it.
I'd like to see Waymo have a few of their Drivers do some sim racing training and then compete in some live events. It wouldn't matter much to me if they were fast at all, I'd like to see them go into the rookie classes in various games and see how they avoid crashes from inexperienced players. I believe that it would be the ultimate "shitty drivers vs. AI" test.
Racing and street driving are completely different. Racing involves detailed knowledge of vehicle dynamics and grip. Street driving is mainly obstacle recognition and avoidance. No waymo ever operates anywhere close to the limit of grip, which is where you are all the time when racing.
Sure but accident avoidance in sim racing is basically the ultimate test for any driver.
I also said it wouldn’t matter if they’re fast, I don’t care about driving at the limit of grip here, just avoiding accidents.
Interesting, but it feels like it's going to cope very poorly with actually safety-critical situations. Having a world model that's trained on successful driving data feels like it's going to "launder" a lot of implicit assumptions that would cause a car to get into a crash in real life (e.g. there's probably no examples in the training data where the car is behind a stopped car, and the driver pulls over to another lane and another car comes from behind and crashes into the driver because it didn't check its blindspot). These types of subtle biases are going to make AI-simulated world models a poor fit for training safety systems where failure cannot be represented in the training data, since they basically give models "free reign" to do anything that couldn't be represented in world model training.
You're forgetting that they are also training with real data from the 100+ million miles they've driven on real roads with riders, and using that data to train the world model AI.
> there's probably no examples in the training data where the car is behind a stopped car, and the driver pulls over to another lane and another car comes from behind and crashes into the driver because it didn't check its blindspot
This specific scenario is in the examples: https://videos.ctfassets.net/7ijaobx36mtm/3wK6IWWc8UmhFNUSyy...
It doesn't show the failure mode, it demonstrates the successful crash avoidance.
While there most likely is going to be some bias in the training of those kinds of models, we can also hope that transfer learning from other non-driving videos will at least help generate something close enough to the very real but unusual situations you are mentioning. We could imagine an LLM serving as some kind of fuzzer to create a large variety of prompts for the world model, which as we can see in the article seems pretty capable at generating fictive scenarios when asked to.
As always tho the devil lies in the details: is an LLM based generation pipeline good enough? What even is the definition of "good enough"? Even with good prompts will the world model output something sufficiently close to reality so that it can be used as a good virtual driving environment for further training / testing of autonomous cars? Or do the kind of limitations you mentioned still mean subtle but dangerous imprecisions will slip through and cause too poor data distribution to be a truly viable approach?
My personal feeling is that this we will land somewhere in between: I think approaches like this one will be very useful, but I also don't think the current state of AI models mean we can have something 100% reliable with this.
The question is: is 100% reliability a realistic goal? Human drivers are definitely not 100% reliable. If we come up with a solution 10x more reliable than the best human drivers, that maybe has some also some hard proof that it cannot have certain classes of catastrophic failure modes (probably with verified code based approaches that for instance guarantees that even if the NN output is invalid the car doesn't try to make moves out of a verifiably safe envelope) then I feel like the public and regulators would be much more inclined to authorize full autonomy.
I wonder if they can simulate the Beatles crossing the street at Abbey Road in the late '60s
As a Londoner who used to have to ride up Abbey Road at least once per week there are people on that crossing pretty much all day every day reproducing that picture. So now Waymo are in Beta in London[1] they have only to drive up there and they'll get plenty of footage they could use for taht.
[1] I've seen a couple of them but they're not available to hire yet and are still very rare.
Will Google finally fund Christopher Wren's post great fire "wide streets" rebuild of the City?
Yeah it’s interesting to imagine a London that had such a rebuild, like Napoleon’s rebuild of Paris. I personally love the weird narrow streets and little alleyways of the City, but that’s because when I’m there I’m pretty much exclusively on foot having taken the tube in.
i think we might need aother great fire to widen the streets at this point
The term "world model" seems almost meaningless. This is a world model in the same sense as ChatGPT is a world model. Both have some ability to model aspects of the real world.
It doesn't look like they're going to open sources or anything, but I could imagine this would be great for city planning.
Or the most realistic game of SimCity you could imagine.
Interesting, but I am very sceptical. I'd be interested in seeing actual verified results of how it handles a road with heavy snow, where the only lane references are the wheel tracks of other vehicles, and you can't tell where the road ends and the snow-filled ditch begins.
Very concerned with this direction of training “counterfactual events such as whether the Waymo Driver could have safely driven more confidently instead of yielding in a particular situation.” Seems dicey. This could lead in the direction to a less safe Waymo. Since the counterfactual will be generated, I suspect that that the generations will be biased towards survivor situations where most video footage in its training data will be from environments where people reacted well not those that ended in tragedy. Emboldening Waymo on generated best case data. THIS IS DANGEROUS!!!
Not at all. It's not the counter-factual they're generating, it's the "too rare to capture often enough to train a response to" they're generating.
They're implying that without the model having knowledge, even approximate, of a scene to react to, it simply doesn't react at all; it simply "yields" to the situation until it passes. In my experience taking Waymo's almost daily this holds.
I would rather not have the Waymo yield to a tornado, rising flood-waters, or charging elephant...
Driving is always a balance between speed and safety. If you want ultimate safety you just sit in the driveway. But obviously that isn't useful. So functionally one of the most important things a self-driving system will decide is "how fast is it safe to drive right now". Slower is not always better and it has to balance safety with productivity.
Not entering a roundabout when it's clearly safe to do so is a mark against you at a driving exam. So would be always driving at 5mph. It's just not that simple.
Neat! What happens when the simulated data is hallucinated/incorrect?
In the example videos, the Golden Gate bridge with snow shows the bridge as 1 road, with total of 3 lanes. But in reality, it’s a split highway with divider, so 2 sides both have 3 lanes, 6 total lanes.
What happens when the car “learns” to drive on the simulated incorrect 3 lane example? For example will next time it goes on the real GG bridge hug to the rightmost lane?
Ideally it would learn a relationship between the sensor input and the correct actions, even if the sensor input is not realistic for the GG in reality.
Similar to Kiva Systems which was Amazon's best acquisition, Waymo is simply Google's best acquisition. (We live in San Francisco and it feels much safer around these Waymo cars than average "drivers".)
those zoox cars though. watchout!
Still needs to be trained on the final boss: dense cities with narrow streets.
San Francisco isn't uniformly dense and narrow, but it does have both, and it's run remarkably well so far.
Another comment mentioned the Philippines as the manifest frontier. SF is not on the same plane of reality in terms of density or narrow streets as PH, I would argue in comparison it does not have both.
This is the craziest I've seen, but it was 10 months ago which is ~10 years in AI years
https://www.youtube.com/watch?v=3DWz1TD-VZg
On that specific count, not really. There's a skate park north end of the Mission, and Stevenson St is a two way road that borders it, but it's narrow enough that you need to drive up on the curb to get two vehicles side by side on the street. Waymo's can't handle that on a regular basis. Being San Francisco and not London, you can just skip that road, but if you find yourself in a Waymo on that street and are unlucky to have other traffic on it, the Waymo will just have to back up the entire street. Hope there's no one behind you as well as in front of you!
Anyway, we'll see how the London rollout goes, but I get the impression London's got a lot more of those kinds of roads.
I live in London. Most residential streets are two-way but there is only space for one car, and driving on the curb is not really an option.
The trick to UK streets is that parking actually happens on the street itself, and when driving you must find a spot when people are not parking to make way for people coming the other way.
> Stevenson St is a two way road
That is extremely narrow, I wonder why the city has not designated it as a one-way street? They've done that for other similarly narrow sections of the same street farther north.
What would be an example city? Waymo just announced they're ramping up in Boston: https://waymo.com/blog/?modal=short-back-to-boston
"we’re excited to continue effectively adapting to Boston’s cobblestones, narrow alleyways, roundabouts and turnpikes."
Any small city in Italy is going to be 10X more challenging than Boston
Depends, which is harder: a narrow street or a three lane one with no obvious lane markers with people double parking?
the absolute chaos of Paris would also be challenging.
and the failure mode for some of them are steep drops off of cliffs
Various European cities come to mind: Narrow streets are something of a trope in certain movies/genres.
To be fair, many of those films do not portray human drivers in the best light.
Not grandparent but I was rather thinking of medieval city centers in Italy or Spain.
edit: Case in point:
https://maps.app.goo.gl/xxYQWHrzSMES8HPL8
This is an alley in Coimbra, Portugal. A couple years ago I stayed at a hotel in this very street and took a cab from the train station. The driver could have stopped in the praça below and told me to walk 15m up. Instead the guy went all the way up then curved through 5-10 alleys like that to drop me off right right in front of my place. At a significant speed as well. It was one of the craziest car rides I've ever experienced.
Do we really need FSD cars (any cars, actually) in medieval city centers?
I live in such an area. The route to my house involves steep topography via small windy streets that are very narrow and effectively one-way due to parked cars.
Human drivers routinely do worse than Waymo, which I take 2 or 3 times a week. Is it perfect? No. Does it handle the situation better than most Lyft or Uber drivers? Yes.
As a bonus: unlike some of those drivers the Waymo doesn't get palpably angry at me for driving the route.
Yes, something like Ho Chi Minh or Mumbai in a peak hour! With lots of bike riders, pedestrians, and livestock at the same roundabout.
Like London? https://www.youtube.com/watch?v=KvctCbVEvwQ
Waymo cars are driving around London right now.
Not taking paying passengers yet though!
They're being trialled in London right now.
Napoli
Old Delhi is the the final boss.
Does it, though? Maybe Dhaka will never get Waymo. The same way you can’t get advanced gene therapy there.
How large is the controlled area around the car? And how high do they look for objects? Like something is falling from a bridge, a falling pole or more extreme an falling plane.
So in order to create a self driving car, you must first create the universe
So when will multiple Waymo cars communicate input data to one another to avoid the blind spots?
This would give the ability to see things other cars cannot see as well.
as someone looking at this from a risk and liability perspective, the move toward world models is pretty wild. the edge cases where a model predicts a safe path that doesn't align with physics is basically the final boss of insurance underwriting for autonomous vehicles. curious how they actually validate for 'common sense' reality vs just predicting the next frame.
Have there been any reported instances of Waymo cars being stolen?
Disabled and then loaded into a lead-lined trailer or something.
I imagine the IP running locally on the cars is worth billions.
I doubt Waymo would publicly talk about this if it did happen.
I also doubt the IP is worth that much. Most of the secret sauce to starting a competitor probably isn't an end model tuned for a specific configuration of a car but the ability to produce end models, which wouldn't be stealable from the car.
It is great being able to generate a much larger universe of possibilities than what they can gather from real world data collection, but I'd be curious to learn how they check that the generated data is a superset of the possibility-space seen in the real world (e.g. confirm that their models closely match what is seen in the real world too)
Does this model include things like how slippery the road is, or head wind/ cross wind. Or is it only visual / depth model?
1. Still hard not to think that this is a huge waste of time as opposed to something that's a little more like a public transport train-ish thing, i.e. integrate with established infrastructure.
2. No seriously, is the filipino driver thing confirmed? It really feels like they're trying to bury that.
"The Filipino driver thing" is simply that there's a manual override ability when this profoundly complex and marvelously novel technology gets trapped in edge cases.
Once it gets unstuck, it runs autonomously.
As someone who half-learned to drive in Manila, the idea that they would use Filipino drivers as backups is ironic.
For context, my "driver's test" was going to the back of the office, and driving some old car backwards and forwards a few meters.
2. Yes, a Waymo exec described it in a Congressional hearing.
https://news.ycombinator.com/item?id=46918043
I am very pro public transit. But there is still a place for cars (ideally mostly taxis). Going to more rural areas or when you need to carry more stuff. I think an ideal society would have both urban transit, inter-city transit and taxis for the other trips and going out into the country.
Filipino driver is false. Filipino guidance person is true.
The difference being?
They’re like a back-seat driver instead of a driver. They can say “take that turn”, or “go over there”, but they can’t operate the controls.
https://news.ycombinator.com/item?id=46918834
(2) I really don't understand why people are surprised that Waymo has fallbacks? The fact that they had a team ready to take over as necessary was well known. I've seen a bunch of comments about this and it seems like people are confused.
I think they're surprised to learn it's being done by a bunch of people on the other side of the world because they don't want to pay American wages.
I think you sort of fundamentally misunderstand the whole "steak vs sizzle" thing in capitalism?
The technology "feels" way less cool knowing that there are human backups, which would absolutely in turn make its percieved value go down.
America is not europe, how would public transport work for the last 1/2miles
Walking, bikes and scooters.
again, this is not europe. we don't ride bikes, and we certainly don't walk.
Yeah, that’s the root of a lot of problems in the US.
But pedestrian infrastructure is a very simple problem to solve, even in the US. Just look at how the market responds to public transit projects in the US —- developers trip over themselves to build projects in response to the induced demand.
Likewise they also responded to the reduced demand as a result of ripping out public transit infrastructure in the 20th century. Which is why doughnut-shaped cities are common in the US.
They are not trying to "bury" remote assistance at all. They wrote a white paper about it in 2020 and a blog post about it in 2024.
Anyway you can think it's a waste but they're wasting their money, not yours. If you want a train in your town, go get one. Waymo has only spent, cumulatively, about 4 months of the budgets of American transit agencies. If you had all that money it wouldn't amount to anything.
"At all?"
Oh come on -- of course they are. That's precisely why you put it in a "white paper" and not, you know, ads.
My view on Waymo and autonomous taxis in general is they will eventually make public transit obsolete. Once there is a robotaxi available to pick up and drop off every passenger directly from a to b, the whole system could be made to be super efficient. It will take time to get there though.
But eventually I think we will get there. Human drivers will be banned, the roads will be exclusively used by autonomous vehicles that are very efficient drivers (we could totally remove stoplights, for example. Only pedestrian crossing signs would be needed. Robo-vehicles could plug into a city-wide network that optimizes the routing of every vehicle.) At that point, public transit becomes subsidized robotaxi rides. Why take a subway when a car can take you door to door with an optimized route?
So in terms of why it isn’t a waste of time, it’s a step along the path towards this vision. We can’t flip a switch and make this tech exist, it will happen in gradual steps.
Automated taxis would still be stuck in traffic. Automation gets couple times in capacity, but the induced demand and extra cars looking for rides and parking will mean traffic.
Automation makes public transit better. There will be automated minibuses that are more flexible and frequent than today's buses. Automation also means that buses get a virtual bus lane. Taxis solve the last mile problem, by taking taxi to the station, riding train with thousands of people, and then taking more transit.
Also, we might discover the advantage of human powered transit. Ebikes are more efficient than cars and give health benefits. They will be much safer than automated cars. Could use the extra capacity for bike and bus lanes.
> There will be automated minibuses that are more flexible and frequent than today's buses.
In my sleepy metro area that has at least mid-tier respectable public transit (by US standards only), otherwise known as Portland, I think a lot of the routes would be better served by minibuses than full size. I wonder how the economics work out on that. Maybe dominated by labor? Tri-met drivers have a reputation of being paid handsomely as they gain seniority.
I'm also in Portland. In the US, bus costs are dominated by labor. It makes sense to use full size buses if paying for driver. For main routes, more automated buses would be best option. But there are cross town routes that should be served with minibuses. Especially ones feeding MAX stops.
If everyone in NYC tried to commute in a single-occupancy vehicle, there would be gridlock -- AVs or no.
> Human drivers will be banned, the roads will be exclusively used by autonomous vehicles
I basically agree with your premise that public transit as it exists today will be rendered obsolete, but I think this point here is where your prediction hits a wall. I would be stunned if we agreed to eliminate human drivers from the road in my lifetime, or the lifetime of anyone alive today. Waymo is amazing, but still just at the beginning of the long tail.
> I would be stunned if we agreed to eliminate human drivers from the road in my lifetime
It basically happened for horses.
Did it? I did a cursory search and it seems like many places still permit horse-drawn carriages, just not on limited access highways. Sometimes with fairly onerous licensing and operational requirements (speed limits, poo management, etc), but still allowed.
I think that will be how human-driven cars are in 30(?) years. Rare, but allowed with restrictions.
Horses don't vote.
Neither do cars?
Drivers, however, absolutely do. And I do not see enough drivers voting away their own ability to drive any time soon.
A few years ago I would have (and did) considered the notion that manually programming was about to turn into a quaint relic and computers would be writing 90%+ of code preposterous. Once an alternative becomes obviously superior things can change very fast.
Right, I was pointing out that at some point there was probably a horse-rider constituency as there is a driver constituency today.
Is that:
- I would be stunned if we agree to eliminate human drivers from 100% of roads in the lifetime of anyone alive today.
or
- I would be stunned if we agree to eliminate human drivers from 10% of roads...
...or is there some other percentage to qualify this? I guess I wouldn't expect there to be a decree that makes it happen all at once for a country. Especially a large country like the U.S.. More like, some really dense city will decide to make a tiny core autonomous vehicles only, and then some other cities also do years later. And then maybe it expands to something larger than just the core after 5 or 10 years. And so on...
That is a fair point, since it is fairly safe to make "this absolute claim will never happen." And the person I replied to did say 'exclusively', which implies 100% elimination of human drivers. But still, I appreciate your nuance.
And in the spirit of that nuance, I will revise my statement slightly. I think it is entirely possible we will eliminate drivers on 10% of roads. We have rules that are analogous to that already with limited access highways. Though I would rate this still as unlikely, since such roads only make up just over 1% of all the roads in the US as it is. Not sure what the % is for other countries, probably less.
> some really dense city will decide to make a tiny core autonomous vehicles only
Agree 100%, this kind of thing I do expect to see happen. We already have exclusions for cars altogether in favor of pedestrians, so the precedent is set.
Only in lower density areas.
In high density regions, vehicles on surface roads can’t meet the passenger demand required. Even if you banned human drivers, the other human users introduce too much variability and delay (passengers loading and unloading, errant objects, cyclists and pedestrians, etc). Roll a dumpster in the street, and have a couple of jaywalkers, and the entire system crawls to a stop.
Controlled access is required to get even medium-high throughput. But these systems already exist, they are called personal rapid transit systems.
> Once there is a robotaxi available to pick up and drop off every passenger directly from a to b, the whole system could be made to be super efficient.
Fundamentally impossible. You're moving some 2 tons of mass in a 2x5m box on polluting rubber tires to move a single 100kg human.
I can always take whatever efficiency gain you've thought up and simply make the vehicle bigger, decreasing the cost and space used per passenger, and maybe even put it on rails, making it less polluting, and more energy efficient.
You can't engineer your way out of the laws of physics.
And don't even get me started on e-bikes.
I don't get how this solves the problem of edge cases with self driving
Even if you can generate simulated training data, don't you still have the problem where you don't even know what the edge cases you need to simulate are in the first place?
Well it certainly helps,doesn't it? This system is going to encounter more edge cases than a single human ever would. Hopefully the lessons from known unknowns generqlise to unknowns. And once they've been seen once they took can become part of the corpus.
Right but does this just rely on some human "brainstorming" a bunch of edge cases?
It just strikes me as neverending edge case wack-a-mole
A human doesn't need to see tons of examples of tornados and elephants to know to stop the car
Doesn't that indicate some fundamental difference between the model and a human driver?
It might be "never-ending", but you're going to encounter edge cases in approximate proportion to the rate at which they actually occur. Anyway, the hope would be to learn behaviors which generalize, not to respond to each edge case ad-hoc; the edge cases provide out-of-sample tests of generalizability.
Neither does the car — it won't drive into what LIDAR sees as a wall. But stopping is not good enough, it needs to be able to navigate the obstacle as well.
Also, even if the car behaved perfectly anyway, these scenarios are useful for testing — validating that the expected behavior happens.
I don’t think coming up with novel situations is all that hard. LLMs already do it in text form all the time.
How big is the space of possible things a car can encounter?
It’s practically infinite, your domain is 3D space and time
You can’t just generate every possible scenario
That would be a combinatorially insane amount of data
The amount to which supposedly engineering based rationales are applied to Elon's decisions is cult like behavior. Here's what's really going on:
That thing I called full self driving just to troll government regulators? Just make it work and don't complain about what I called it.
That rocket that's way too big for orbital payloads but can't go beyond orbit without sending 20 more rockets full of fuel? Just make it work. Occupy Mars!
The two car models that work and have a halo effect on the rest of the product line? I'mma cancel that! Just make the stupid truck we're keeping work!
We've all had a boss like that. I'm sure they salute each other.
Could these world models be used to build some sort of endless GranTurismo type street racing game?
It seems inevitable that they'll soon be used as the starting points for developing almost all video game environments.
Not for the rendering (that's still way too expensive), but for the initial world generation that gets iteratively refined and then still ultimately gets converted into textured triangles.
> It seems inevitable that they'll soon be used as the starting points for developing almost all video game environments.
Almost all video game environments? No way. That statement really needs to be qualified with the genres of games you're considering.
Very impressive work from Waymo. The driving with a tornado in the horizon example kind of struck my imagination, many people actually panic in such scenarios. I wonder though the compute requirements to run these simulations and producing so many data points.
I would love to see more visibility into how this model’s simulation fidelity maps onto measurable safety improvements on public roads, especially in unusual edge conditions like partial sensor occlusion or atypical weather.
Dumb question - Why would Waymo disclose this much information to public and competitors?
It's easier to build trust for such a safety-critical service when you're more open about how it works an performs. For the complete opposite approach, see Tesla.
Given the announcement from a few days ago of google trying to get external investment, this is their follow up, showing what that investment is good for. Also, it’s pretty light on details that are of much use to competitors. “We made an accurate simulation system to test our system in before deployment” would be pretty mundane if you were talking about any other field of engineering.
Maybe to distract from the story that they use remote drivers after one of their cars hit a kid? [1]
[1] https://people.com/waymo-exec-reveals-company-uses-operators...
edit: fixed kill -> hit
The child did not die, and suffered only minor injuries: https://abc7.com/post/california-teamsters-call-suspension-w...
Under the same circumstances (kid suddenly emerging between two parked cars and running out onto the street), it could be debated that the outcome could have been worse if a human were driving.
It’s awful a child was hit, but they only suffered minor injuries [1]. Nowhere in your linked article does it say they were killed.
[1] https://people.com/waymo-car-hits-child-walking-to-school-du...
I don't know about the remote driver conspiracy, but waymo slowing down and that kid surviving a crash after jumping on the road from behind a tall vehicle was the best PR waymo could have asked for.
been playing around with world models for sim-to-real transfer lately. the waymo approach looks solid, but curious how you're handling the distribution shift between generated scenes and real sensor data. any tricks for that besides the usual domain randomization?
Every time I'm in town I use a waymo. It's still a little weird to be a passenger with no driver
Why don't they call it 'the Matrix' and should i prepare for the plugs?
So the waymo driver is dreaming, testing out different scenarios, basically…
I'm curious how they simulate equipment failures, like a flat tire or something
This is cool, but they are still not going about it the right way.
Its much easier to build everything into the compressed latent space of physical objects and how they move, and operate from there.
Everyone jumped on the end-2-end bandwagon, which then locks you into the input to your driving model being vision, which means that you have to have things like genie to generate vision data, which is wasteful.
The article is about using the world model to generate simulations, not for controlling the vehicle.
They form control policy from vision data directly, which is why they need to have a massive model generate simulation vision data.
This is cool, but they are still not going about it the right way.
This is legit hilarious to read from some random HN account.
I posted this before, but Ill post again - this is one of the few things I feel confident enough to say that most people in the space are doing wrong. You can save my post and reference it when we actually get full self driving (i.e you can take a nap in the backseat while your car drives you), because its going to be implemented pretty much like this:
Humans don't drive well because we map vision policy to actions. We drive well (an in general, manipulate physical objects well), because we can do simulations inside our head to predict what the outcome will be. We aren't burdened by our inability to recognize certain things - when something is in the road, no matter what it is, we auto predict that we would likely collide with that thing because we understand the concept of 3d space and moving within it, and take appropriate action. Sure, there is some level of direct mapping as many people can drive while "spaced out", but attentive driving involves mostly the above.
The self driving system that can actually self drive needs to do the same. When you have this, you will no longer need to do things like simulate driving conditions in a computationally expensive sim. You aren't going to be concerned with training model on edge cases. All you would need to to ensure that your sensor processing results in a 3d representation of the driving conditions, and the model will then be able to do what humans do and explore a latent space of things it can do and predict outcomes then chose the best one.
You want proof? It exists in the form of Mu Zero, and it worked amazingly well. And driving can be easily reformated as a game that the engine plays in a simulator that doesn't involve vision, and learns both the available moves and also the optimal policy.
The reason everyone is doing end to end today is because they are basically trying to catch up to Tesla, and from a business perspective, nobody is willing to put money and pay smart enough people to research this, especially because there is also a legal bridge to cross when it comes to proving that the system can self drive while you napping. But nevertheless, if you ever want self driving, this is the right approach.
Meanwhile, Google who came up with Mu Zero, is now doing more advanced robotic stuff than anyone out there.
Interesting that this should come out right as lawmakers are beginning to understand that Waymos have overseas operators making major decisions.
[*] https://futurism.com/advanced-transport/waymos-controlled-wo...
Completely false: https://x.com/i/status/2019213765506670738
Listen to the statement.
The operators help when the Waymo is in a "difficult situation".
Car drives itself 99% of the time, long tail of issues not yet fixed have a human intervene.
Everyone is making out like it's an RC car, completely false.
Whenever something like this comes out, it's a good moment to find people with no critical thinking skills who can safely be ignored. Driving a waymo like an RC car from the philippines? you can barely talk over zoom with someone in the philippines without bitrate and lag issues.
Hacker News has had some of the dumbest Tesla takes of all time. People should be embarrassed about some of the claims that were made here.
And apparently some people still haven't caught on.
Have a look if you don't believe me:
https://hn.algolia.com/?dateRange=custom&page=0&prefix=false...
Seems to be some ketamine on that boot.
Except that's not what the original posters said, rather 'operators making major decisions.' Don't strawman here, it wastes everyone's time.
I didn't mean to insult the original poster. But I have seen people who think that waymos are being driven remotely by filipinos
I haven't read anything about this but I would also suppose long distance human intervention cannot be done for truly critical situations where you need a very quick reaction, whereas it would be more appropriate in situations where the car has stopped and is stuck not knowing what to do. Probably just stating the obvious here but indeed this seems like something very different from an RC car kind of situation.
It’s not for that. It’s for things like the car drove into a protest area and people are surrounding the car. Or police blocked off an intersection and the car is stuck temporarily with people doing otherwise illegal u-turns or driving the wrong way on a one way road to get out of it.
Why is this relevant at all?
Having humans in the loop at some level is necessary for handling rare edge cases safely.
The word "loop" here has multiple meanings. Only one is what you mean and the other person responding to you has understood another.
The first is the DDT control loop, what a human driver does. Waymo's remote assistants aren't involved in that. The computer always has responsibility for the safety of the vehicle and decisionmaking while operating, which is why Waymo's humans are remote assistants and not remote drivers. Their safety drivers do participate in the DDT loop, hence the name.
But there's also another "loop" of human involvement. Sometimes the vehicle doesn't understand the scene and asks humans for advice about the appropriate action to take. It's vaguely similar to captchas. The human will usually confirm the computer's proposed actions, but they can also suggest different actions. The computer the advice as a prior to continue operating instead of giving up the DDT responsibility. There's very likely a closely monitored SLA between a few seconds to a few minutes on how long it takes humans to start looking at the scene.
If something causes the computer to believe the advice isn't safe, it will ignore it. There have been cases where Waymos have erroneously detected collisions and remote assistants were unable to override that decisionmaking. When that happens, a vehicle recovery team is physically sent out to the location. The SLA here is likely between tens of minutes and a couple hours.
If that’s true the system isn’t finished. That’s what reasoning is for.
Who ever said they were finished? You think the laid off the team since everything is “done”?
Do wayno models really use side cameras at only like 4 FPS?
Imagine driving in a Waymo 'out of a raging fire'.
Talk about edge cases.
But, what would you do? Trust the Waymo, or get out (or never get in) at the first sign of trouble?
Interesting question. If the Waymo was driving aggressively to remove us from the situation but relatively safely I might stay in it.
This does bring up something, though: Waymo has a "pull over" feature, but it's hidden behind a couple of touch screen actions involving small virtual buttons and it does not pull over immediately. Instead, it "finds a spot to pull over". I would very much like a big red STOP IMMEDIATELY button in these vehicles.
>it's hidden behind a couple of touch screen actions involving small virtual buttons and it does not pull over immediately
It was on the home screen when I've taken it, and when I tested it, it seemed to pull to the first safe place. I don't trust the general pubic with a stop button.
I feel like this ends with drunk morons accidentally creating Waymo barricades and totally ruining Mardi Gras
Can you not just unlock and open the door? Wouldn't that cause it to immediately stop? Or can you not unlock the door manually? I'd be surprised if there was not an emergency door release.
Imagine how many drunk/careless passengers might press it. Stopping in the middle of the street or highway could be a serious safety hazard.
I can! If the Waymo got you into one on the way home because Google didn’t integrate with watch duty yet, that’s plausible
Seems relevant: Waymo exec admits remote operators in Philippines help guide US Robotaxis
https://news.ycombinator.com/item?id=46918043
and I literally just saw the other headline "Waymo says its robotaxis get help from remote workers in the Philippines"
Instructions to load it on WAYMAX simulator?
so insane that this is the direction things are going, instead of just reducing our reliance on cars
What is the 5/3 tiles? Cameras?
The model generates camera and Lidar data. As if it was a Waymo car that drove through the simulated scenario with its cameras running. This synthetic training data can then be used to train the driving models.
Wonder how it'll do. The trees change shape (presumably the Lidar patterns do too). I get the premise/why but it seems odd to me (armchair) to use fake data. Real trees don't change shape (in real time) although it can be windy.
It probably doesn't matter though, "this general blob over there"
It's clear that lidar is superior to the Tesla cameras only approach, but I wonder if being able to take all that data and be able to better correlate what is being seen visually with what the lidar is showing.
That said, for autonomous driving I'd like to see as many sensor options as possible: lidar, radar, cameras, sonar. Belt and suspenders. I imagine as pricing continues to drop all will be embraced.
Have been seeing Waymo test vehicles regularly around central London recently, operating at speed.
For shits and giggles, I did stop randomly while crossing the road and acted like a jerk.
The Waymo did, in fact, stop.
Kudos, Waymo
What if we put this mechanism of recording the world on people. We have mics listening to people talking to us and noises we hear.
Also we record body position actuation and self speech. As output then we put this on thousands of people to get as much data as Waymo gets.
I mean that’s what we need to imitate agi right? I guess the only thing missing is the memory mechanism. We train everything as if it’s an input and output function without accounting for memory.
No human needs to have seen an elephant standing in the road before to know that you should not drive through an elephant standing in the road. These are not "long tail" events as the waymo says. It's a big object in the road. You have seen that hundreds of thousands of times. Calling that a long tail event is an admission that your model has zero ability to generalize.
I'm a little sad that they talk about counterfactuals in the simulations, but then don't show any examples of even a single sharknado or giant loop-de-loop.
> Simulation of the Waymo Driver evading a vehicle going in the wrong direction.
It really looks like waymo is the one going in the wrong direction and driving dangerously to evade traffic in this simulation.
Nvidia has had this for years. What am I missing?
For whatever it’s worth World models is going to be the dominant computing structure of the future
I started working heavily on realizing them in 2016 and it is unquestionably (finally) the future of AI
This page crashes my browser.
Vivaldi 7.8.3931.63 on iOS 26.2.1 iPhone 16 pro
One interesting thing from this paper is how big of a LiDaR shadow there is around the waymo car which suggests they rely on cameras for anything close (maybe they have radar too?). Seems LiDaR is only useful for distant objects.
At least 6 radar units: https://support.google.com/waymo/answer/9190838?hl=en
This might be relevant to the timing here: https://eletric-vehicles.com/waymo/waymo-exec-admits-remote-...
Meanwhile. https://eletric-vehicles.com/waymo/waymo-exec-admits-remote-...
Seems interesting, but why is it broken. Waymo repeatedly directed multiple automated vehicles into the private alley off of 5th near Brannan in SF even after being told none of them have any business there ever, period. If they can sense the weather and stuff then maybe they could put out a virtual sign or fence that notes what appears to be a road is neither a through way nor open to the public? I'm really bullish on automated driving long term, but now that vehicles are present for real we need to start to think about potentially getting serious about finding some way to get them to comply with the same laws that limit what people can do.
>> get them to comply with the same laws that limit what people can do
I think you meant, "Attempt" to limit what people can do.
Driving in SF (for example) provides many opportunities to see "free will" exerted in the most extreme ways -- laws be damned.
Have you ever seen a cop trying to pull over a Waymo? It isn't going well.
Ah the typical MDS npc comes out in the swarm, saying how FSD is no better than GM solutions.
Time to eat pop corn, while FSD drives me for the next hour while I scroll hackernews
The new frontier is manifestly the Phillipines.
Can you explain? I lived in PH, and my guess is that you mean navigating and modeling the unending and constantly changing chaos of the street systems (and lack thereof) is going to be a monumental task which I completely agree with. It would be an impressive feat if possible.
Edit: or are you talking about the allegations of workers in the Philippines controlling the Waymos: https://futurism.com/advanced-transport/waymos-controlled-wo... I guess both are valid.
Wow, interesting timing for this PR blast considering the admission in the Senate Commerce Committee hearing. Not transparent at all!
What was the admission? That they use cheap labor to provide the waymo clarity when it is confused? That has been known for a long time.
Software doesn’t get confused - it fails. Referring to your software as autonomous when you have to staff a 24/7 response center of humans to control it is not just misleading, it’s a lie.
It's worse than that for you, they never take control of the waymo, the waymo provides solutions but is unsure which one is correct. The people then tell the waymo which solution is the best one.
So maybe unsure is a better term than confused?
I don’t think anthropomorphizing an algorithm is the right approach to discussing this topic. It’s still failing software.
How many Filipinos, who do not have US drivers licenses, does it take to drive this new model?
"Autonomous"
https://cybernews.com/news/waymo-overseas-human-agents-robot...
My understanding is that support is basically playing an RTS (point and click), not a 1P driving game. Which makes sense, if they were directly controlling the vehicles they'd put support in central America for better latency, like the food delivery bot drivers
Yeah. Waymo described how this works a couple of years ago:
https://waymo.com/blog/2024/05/fleet-response/
Right, I totally believe Waymo, just like I totally believed Amazon's checkout-less stores.
What's your theory for how it works?
This isn't news, they've always acknowledged that they have remote navigators that tell the cars what to do when they get stuck or confused. It's just that they don't directly drive the car.
Yeah I have some videos of these drivers in action, I think the sensors are as assistance and but not the whole story, so yeah there’s models lidars etc etc but human factor is there, unfortunately this means we should see soon many cobitics are teleopetated remotely from India and Philippines and the likes to satisfy the greed of these companies to pay peanuts to operate them.
The Waymo driving model: hire some guys in Philippines: https://futurism.com/advanced-transport/waymos-controlled-wo...
This is not false, but gives the wrong idea that foreigners are driving them in real time.
> After being pressed for a breakdown on where these overseas operators operate, Peña said he didn’t have those stats, explaining that some operators live in the US, but others live much further away, including in the Philippines.
> “They provide guidance,” he argued. “They do not remotely drive the vehicles. Waymo asks for guidance in certain situations and gets an input, but the Waymo vehicle is always in charge of the dynamic driving tasks, so that is just one additional input.”
This is quite misleading... From the article:
“When the Waymo vehicle encounters a particular situation on the road, the autonomous driver can reach out to a human fleet response agent for additional information to contextualize its environment,” the post reads. “The Waymo Driver [software] does not rely solely on the inputs it receives from the fleet response agent and it is in control of the vehicle at all times.” [from Waymo's own blog https://waymo.com/blog/2024/05/fleet-response/]
What's the problem with this?
In my opinion there's nothing wrong with it per se, but (a) it's still worth mentioning, because most people have the impression that Waymo cars are completely unassisted, and (b) it makes me wonder how feasible Waymo's operations would be if it weren't for global income inequality.
Have you read the article ? The guys in the Philippines are providing high level executive indications, they don't drive remotely the car or have any low level control of the car.
Dig deep enough into any "AI" idea and you'll find the bottom end of the scam looks exactly like this.
We've simply relabeled the "Mechanical Turk" into "AI."
The rest is built on stolen copyrighted data.
The new corporate model: "just lie the government clearly doesn't give a shit anymore."
What's going to happen to all the millions of drivers who will lose their job overnight? In a country with 100 million guns, are we really sure we've thought this through?
Yes, let's stop all progress and roll-back all automation to keep hypothetical angry people with guns happy.
Seems like a good description on current events.
Autonomous private cars is not the technological progress you think it is. We’ve had autonomous trains for decades, and while it provides us with a more efficient and cost effective public transit system, it didn’t open the doors for the next revolutionary technology.
Self driving cars is a dead end technology, that will introduce a whole host of new problems which are already solved with public transit, better urban planning, etc.
> We’ve had autonomous trains for decades
Trains need tracks, cars - already have the infrastructure to drive on.
> Self driving cars is a dead end technology, that will introduce a whole host of new problems which are already solved with public transit, better urban planning, etc.
Self driving cars will literally become a part of public transit
> Self driving cars will literally become a part of public transit
I’ve been hearing people say that for almost 15 years now. I believe it when I see it.
>> I believe it when I see it.
I'm willing to wager that you might not actually believe it at that point either.
Unfortunately, many of our urban areas have already been planned (for better or worse) for cars and not the density that makes public transit viable. Autonomous cars will solve a host of problems for the old, young, mobility limited, and just about everyone else.
It will prove disruptive to the driving industry, but I think we’ve been through worse disruptions and fared the better for it.
> Self driving cars is a dead end technology
I would be happy to bet on some strict definition of your claim.
Nope. Humans are statistically fallible and their attention is too valuable to be obliged to a mundane task like executing navigation commands. Redesigning and rebuilding city transportation infrastructure isn't happening, look around. Also personal agency limits public transportation as a solution.
> Redesigning and rebuilding city transportation infrastructure isn't happening, look around.
The US already did it once (just in the wrong direction) by redesigning all cities to be unfriendly to humans and only navigable by cars. It should be technically possible to revert that mistake.
Unlike autonomous driving, public transit is a proven solution employed in thousands of cities around the world, on various scales, economies, etc.
> Redesigning and rebuilding city transportation infrastructure isn't happening, look around.
We have been redesigning and rebuilding city transportation infrastructure since we had cities. Where I live (Seattle) they are opening a new light rail bridge crossing just next month (first rail over a floting bridge; which is technologically very interesting), and two new rail lines are being planned. In the 1960s the Bay area completely revolutionized their transit sytem when they opened BART.
I think you are simply wrong here.
>> In the 1960s the Bay area completely revolutionized their transit sytem when they opened BART.
66 years later we see California struggling terribly with implementation of a high-speed rail system -- where the placement/location of the infrastructure largely is targeted for areas far less dense than the Bay Area.
I don't think there is any single reason why this is so much more difficult now then it was in 1960 -- but clearly things have changed quite a lot in that time.
Waymo has been operating since 2004 (22 years ago), and replacing drivers on the road will take many more decades. Nothing is happening "overnight".
If Waymo's history is any guide, it's not going to happen overnight. Even in San Francisco, their market share is only 20-30%.
> What's going to happen to all the millions of drivers who will lose their job overnight? In a country with 100 million guns, are we really sure we've thought this through?
Same was said about electricity, or the internet.
People keep referencing history but this really is unprecedented. We are approaching singularity and many people will become obsolete in all areas. There are no new hypothetical jobs waiting on the horizon.
this sounds like a major benefit.
i dont want my uber driver bragging anout how theyre going to shoot me before i get out of the car
Reminds me of the history or radio and the absolute uproar that someone played a record on the radio rather than live performances!!
same thing that happened during the industrial revolution, you pay enough of them to 'protect the law' vs the rest.
UBI or war, or both
what did all the farmers do when the first tractors rolled into the fields?
They invented offices because the massive increase in productivity required more organisation than "I'll take it all to the market on Saturday"
I don't think Uber goes out of business. There is probably a sweet spot for Waymo's steady state cars, and you STILL might want 'surge' capabilities for part time workers who can repurpose their cars to make a little extra money here and there.
Those are rookie numbers. The US has 400 million guns. https://www.theglobalstatistics.com/united-states-gun-owners...
As to the revolt, America doesn't do that any more. Years of education have removed both the vim and vigor of our souls. People will complain. They will do a TikTok dance as protest. Some will go into the streets. No meaningful uprising will occur.
The poor and the affected will be told to go to the trades. That's the new learn to program. Our tech overlords will have their media tell us that everything is ok (packaging it appropriately for the specific side of the aisle).
Ultimately the US will go down hill to become a Belgium. Not terrible, but not a world dominating, hand cutting entity it once was.
> Ultimately the US will go down hill to become a Belgium.
Sharing one's opinion in a respectful way is possible. Less spectacle, so less eyeballs, but worth it. Try it.
What's wrong with his comparison? He explained what he meant by "a Belgium".
The entire side topic of guns and revolt seems misplaced in this thread.
The original Luddite movement arose in response to automation in the textile industry.
They committed violence. Violence was committed against them. All tragic events when viewed from a certain perspective.
My rhetorical question is this: did any of this result in any meaningful impedance of the "march of technological progress"?
>in a respectful way
The irony
> Ultimately the US will go down hill to become a Belgium.
I'm curious why you say this given you start by highlighting several characteristics that are not like Belgium (to wit, poor education, political media capture, effective oligarchy). I feel there are several other nations that may be better comparators, just want to understand your selection.
I think it just means "once powerful and important, but now practically irrelevant." England is a better example though.