Getting it wrong on self-driving vehicles

Peak Self-Driving Car Hype was probably around 2017-2018. The robo-taxi stuff stuck around longer, but it’s running into challenges as well. It’s just really hard to get it to full automation.

Maybe the trucks will do better, but it’s still a big challenge to get a system that reliably do self-driving. Here’s a very interesting piece from 2020, from the founder of the now defunct Starsky Robotics (a self-driving truck startup) talking about it.

TL;DR: It’s really hard to get to a safe level of reliability, and many of the competitors didn’t even try – they drew in investors by piling on features that maybe worked once in a while, over-promising and so forth.

There was also an element of wishful thinking. As commenter Hidari observed, the most obvious use of self-driving vehicles is to provide mobility for 75+ Baby Boomers. As someone approaching that category, and having never liked driving much, this is an appealing prospect for me. And I liked the idea of taking other bad drivers’ hands off the steering wheel.

I paid for driving lessons for my daughter, but she still hasn’t got her licence. I expect her to do a better job of driving than I do once she’s over that hurdle. One day, perhaps …

and I always thought that getting vaccinated against a deadly virus was/is a GOOD idea –
and now I have found out that a lot of people think it isn’t a good idea –
at all –
and they rather… die?

There is a general disdain for the profession of engineering in the political and economic commentary worlds, but even within the technology world there is very little understanding of what the people who integrate extremely complex systems do, whether at the high end of continental electric networks or the opposite end of the spectrum of systems integration on a modern CPU chip, how critical what they do is, or how many people have to toil away unseen at very high levels of proficiency to make it all work. And of course the management people who integrate the work of the integrators are doubly despised. Yet for every person who makes a billion with the next great app (“Like TikTok, except for cars!”) there are 50 or 100 engineers and support staff toiling away at the Apple, Intel, and nVidia chip design centers, TSMC, etc to make it all possible.

And indeed while automobiles appear to be a known and solved problem there are such people toiling away at the large makers of them as well (one of my classmates spent the last 20 years of his career as GM’s expert on car window mechanism lubricants). Can those people have blind spots? Certainly, particularly when “corporate goals” are involved (GM blowing a 120 year lead on electric vehicles, then a 20 year lead, then a 10 year lead to allow Tesla and new entrants to arise while ignoring the lesson of how it came to exist itself is a good example), but that doesn’t mean that when their experts say things or do things they are automatically wrong or ‘archaic’. GM, Daimler, BMW have been proceeding very slowly on autonomy research, releasing very small increments, for 15 years. They are not promising big bang advances, and they are working within the framework of the SAE and state/local roadway associations and agencies. Perhaps they know something? [1]

And oh yeah: I’ve been reading academic and industry papers on artificial intelligence since the 1980s, and have read some of the key papers back to 1960. We aren’t even close to having a glimmer of a candle inside a titanium box about how to create a true artificially intelligent system – but even the worst human bad driver referred to in the previous two threads as the lead standard of human driving applies sophisticated intelligence to the problem of driving every time he turns the key. No autonomous vehicle system released to date has shown anything close. And they are all tested on the wide clear streets of Silicon Valley and Phoenix, never on slush-covered sidestreets of Pittsburgh or Chicago…

I don’t think JQ made a serious mistake here other than not listening to informed critics, which is easy to do when the big names of the last Big Thing are saying this is a simple evolution from their previous success. But verifying and double-checking is always important, and listening to the very experienced domain experts (even when they are old and grey) is part of that.

[1] in 2018 one of my sources in a large legacy auto company told me they had something very close to Level 3 autonomy in the final stages of test, and that at a minimum it would be included and active for at least highway driving in high end 2019 models. In the big auto world that means it would have been greenlighted in 2017 or 2016. In the event that manufacturer did release a much improved driver assist package in 2019 models, but nothing like what was described to me, and my source stopped talking. I have to think that that manufacturer learned something in testing in 2018 that was serious enough to pull a feature planned for 3 years at the last minute. That’s the kind of testing that a gadfly such as Musk could not abide.

I think predictions like this are troublesome because most new technologies have at least a decade of lag between “just a few kinks to work out” and “meets all industry safety standards.” Depending on the regulatory regime and industry culture, products may be rushed to market or delayed to the point of obsolescence or anywhere in between. At one extreme you get leaded gasoline and radioactive toothpaste. At the other you get Luddism, anti-intelectualism, and stagnation. It probably says something about capitalism or modernity or “Western Civilization” or whatever that most of us (except anti-vaxxers and about half the Green Party) seem more than willing to risk getting trampled by the march of progress so long as we feel like we’re going somewhere.

Self-driving cars are ready to go now, we’d just have to accept them killing a certain number of people. Eventually we probably will accept that risk (like we do with conventional vehicles), but first the public has to be reassured that every reasonable precaution has been taken to minimize harm. For that to happen, something has to be done to bridge the gap between a notoriously libertine tech sector and our generally high standards for automotive safety (not to mention a century of killer robot stories).

At the height of the Obama-era “innovation economy” when you could make billions running unlicensed taxis and hotels, marketing predatory algorithms, or selling other people’s data, it was just about possible to imagine self-driving cars similarly slipping through the cracks. Now that the public is beginning to realize how reckless, self-serving, and unaccountable these tech “geniuses” actually are and factions of both parties are willing to make hay over it (one of the few things we may genuinely be able to thank Trump for), the road looks a lot longer. The hype train is off the tracks and the unconscionable hubris it took to field test these deathtraps unmonitored in major cities has been exposed.

Never fear, the cheerleaders will be back sooner or later to accuse us skeptics of standing in the way of progress or costing more lives than we saved. First though, they’re going to need a better slogan because it turns out “move fast and break things” isn’t what anyone wants to hear from a car company.

I don’t think we should decry improvements in car technology as a distraction from shifting away from cars altogether because there is essentially zero prospect of shifting away from cars altogether. This distraction argument is frequently made about electric cars (often in response to someone being overly effusive about Elon Musk), but it doesn’t make a lot of sense. The most public transit-heavy places in the world (like Tokyo) are still extremely dependent on cars for personal transport, and that’s with favorable geography and density and decades of public transit-friendly policy. For a place like the US, public transit could and certainly should be improved in cities, but there is no prospect of cars not being a massive presence for decades to come.

Whether full level 5 autonomous cars will happen in the next decade or two probably depends on whether we need full reasoning artificial general intelligence (AGI as it’s called), or whether we can create safe enough software just through exhaustively documenting and coding/training for all the most likely edge cases. We have no idea how to do AGI, or when or if we might figure it out.

Right now, some of the most impressive-seeming results in machine learning have come from transformer architectures (Andrej Karpathy had a twitter thread about this). The particularly spooky-seeming results remarked upon in the press were for GPT-3, which is a large language model that leveraged semi-supervised learning in its training. They fed it an enormous corpus of writing scraped off the web, and programmed it to mask out words at random and then try to predict the masked words. So it could do supervised learning, but without needing a human to label all the datasets, i.e. semi-supervised learning, allowing a massive amount of training data and thus a massive model. If there was a way to do semi-supervised learning for the general driving task, it would substantially improve the odds of success, but I’m not sure driving lends itself to semi-supervised learning the way text prediction does.

But right now what I’m reading says you shouldn’t buy a Tesla, and you probably shouldn’t ride in one ether.

Or share the road with one, for that matter. It’s discouraging how Musk has been able to ignore whatever safety or truth-in-advertising standards the US has left, and it’s been observed that he’s been at it long enough and has enough of an idiot fanbase that any enforcement at this point will be harder than it once might have been. I wonder at what point do the insurance companies start taking notice.

Just imagine the numbers 1, 2 and 3 before that clump of paragraphs near the start of that last comment and it will make more sense (I don’t know why they were stripped out; I guess I need to teach an AI how to write html for me).

The human brain contains about 100 billion (10^11) neurons. A flatworm has 200, and can use them to memorize a maze. Smart breeds of dogs have about 500 million neurons.

The biggest neural network I know of (there are probably bigger ones) has the equivalent of about a million nodes. It is the current world champion of Go. However, it turns out that a node in a neural network (today) is not the equivalent of a neuron. A recent paper reported that it took about 1000 neural network nodes to simulate the capabilities of a single neuron.

What this implies to me is that while simulating human intelligence is theoretically possible, it is probably practically impossible within the lifetime of human civilization. Of course human intelligence allows us to walk and chew gum and regulate many internal processes while thinking about math problems, and AI will still be able to solve many well-defined problems, such as playing Go. So far, however, driving a car does not seem to be such a well-defined problem.

Intelligence, by the way, is trial and error plus memory. In a well-defined problem, the trial and error can be simulated inside a computer, and millions of trials can be done quickly. Otherwise, the trial and error has to take place physically, one error at a time.

Aside from all this, the planet has a lot of human beings and a finite amount of resources. At some point we will have to decide how best to use and manage those resources. Two cars in every garage for the people in the richest countries seems a tremendous mis-allocation to me. At 75, I never have owned and never will own a car. (I have rented one on occasion so as not to have to travel by airplane.)

MP @6 Not sure if I’m getting your point

Every so often, perhaps more frequently, I find myself feeling a bit puckish. This was one of those moments. I was role playing, pretending that I wanted to steer a serious comment thread down into the mud — viz. ad hominem — but felt constrained not to do so without permission from the blogger themselves.

I don’t have anything more to add to this thread, esp. nothing serious (and already I fear that what I “added” was actually a minor subtraction!).

I think it’s always wise to remember when dealing with stuff you don’t know that most journalists are reporting on topics that they are very ignorant on, and to check if industry pundits are talking their book. When you filtered out those two groups, you generally found researchers with a lot of experience in the field who were very skeptical about the chances of self-driving cars, and had very good arguments for why. It turns out they were right.

On AI generally, it’s always a good idea to remember it’s a (deliberately) misnamed field. As Aubergine says – machine learning is really a type of applied statistics that can be useful for dealing with certain types of problem that lend themselves to that kind of probabilistic analysis. Unfortunately these methods are extremely opaque, it’s hard to identify when your models have failed and they are unpredictable when applied to new datasets. They are also extremely difficult to train (maybe impossible) when you’re dealing with wicket problem areas (such as driving in a range of conditions), as you never really know if your dataset is representative.

I saw someone the other day tweet that if the cost of failure is relatively low, and the value of success is high, then machine learning is a useful tool. Otherwise you probably want to look elsewhere. That seems accurate.

David J. Littleboy: The zeroth problem with self-driving cars is that they don’t fix the problem that the private automobile is an incredibly bad and stupid idea.

With the caveat that I can see the utility of cars in remote and thinly populated areas where public transport isn’t realistically doable, this right here is a big issue not just for self-driving cars but also for electric cars. Yes, they would be better than human-driven, petrol cars, but ideally we would do away with most cars full stop. They are frighteningly wasteful in a world of strictly limited resources and a really, really poor option in cities. (Cue the usual photos comparing how much space 200 people take up in 200 cars versus one tram.)

Sadly, many people absolutely insist on driving everywhere in a car, even parents who could easily walk their kids the two blocks between their home and the local primary school. And they will vote for politicians who structure everything around cars and neglect public transport.

Regarding AI, I am not an AI specialist, but I am using it, especially image classification. In my experience, most people fall into one of two camps: those who had or read about a bad experience and declare it all rubbish (“a self-driven car caused an accident once that a human would not have caused, so it can never replace human drivers who cause 100x as many accidents because they are drunk, distracted, or exhausted”), and those who think it is magic pixie dust and therefore have hilariously unrealistic hopes and expectations.

In my experience (again), the counter-intuitive reality is that AI can do amazing things that humans couldn’t do and at the same time fail spectacularly to do very simple things that even the most ignorant humans can do trivially. And that is the problem causing the aforementioned human perceptions: we have a very poor intuition for what AI can do for us.

I am sure AI is already starting to revolutionise those tasks its brute force approach lends itself to – for example, high-throughput scanning of enormous quantities of images and videos for faults or threats where a human would need orders of magnitude more time and miss half the threats because their attention can only focus for so long, and anything that requires a single and highly specialised capability. But for anything complex with lots of potential parameters I don’t see humans being beaten anytime soon; for starters, my brain brain can drive a car AND tie shoelaces AND read a report AND draw a diagram AND cook dinner AND so on.

Vechicles. Humans. Extenalities. Systems integration etc etc. Ala the 5th Discipline by P Senge.

And what about a smart road as well.

‘dumb’ vehicle as one system
lidar / Traing set / AI
?
4 ?
5 Smart Road & conditions acting as neocortex and external feedback

And AI as several have pointed out it is not comoarabke to humans as yet. As the kids in the rear seat say “Are we there yet?”

and about ‘self-driving’ cars –
I never understood the…
the attraction?

As one of the major fun –
somebody can have
with her or his clothes on –
is driving a very old Lancia Delta down the Amalfi Coast –
(even by being completely against any use of combustion engine – BE-cause they destroy our climate and environment)

BUT on the other hand – the streets on the Amalfi Coast are made for the smallest engine of all – a Fiat Topolino –

Capisce?
(and no wonder that Australians know ‘nuthing’ about it)

Meanwhile:

https://www.ttnews.com/articles/mercedes-beats-tesla-hands-free-driving-autobahn

If someone genuinely had a vision of autonomous driving that was faster than ‘sellable product in 2021’, they were pretty foolish. I wouldn’t have read Quiggin’s articles as arguing that. But, presumably, as he considers them to be mistakes, he does.

These things take time, and every salesman is convinced they have the new iphone that takes things from ‘bunch of cool technologies’ to ‘everyone will have one’. That is in no way incompatible with a driving license becoming a thing you would have to go to university to spend three years studying for by, say, 2061.

So, I thought it would be good to consider where and why I went wrong on this relatively unimportant issue, in the hopes of improving my thinking more generally.

This is from the first of your posts:

Suppose that in any crash between autonomous cars and humans, each is equally likely to be at fault. What is the probability of seeing 22 crashes caused by humans and none by autonomous cars. Obviously, it’s the same as that of a fair coin showing 22 heads in a row, which is 2^-22 or about 1 in 10 million.

The weakness in your thinking is that you neglected to ask the preceding question. Given the prevailing accident rate among all cars, over the period, how many autonomous cars should have been involved in accidents?

I don’t know what how many autonomous cars there were in 2018, but as Feb 2020, there were 769 registered in CA. Twenty-two accidents among 769 cars in one year sounds extraordinarily high compared to regular cars. (Unfortunately, finding the raw number of accidents in CA in a given year isn’t easy. However, just thinking about myself, kids, and friends, I get well past 769 vehicle-years with no accidents.)

Because you failed to ask the first question, your assumption of equal fault-likelihood is unfounded.

Assume 769 vehicles chosen at random. If only 2 of them were involved in accidents in the previous year, then the fact that autonomous vehicles are more than ten times as likely to be involved in an accident makes it very likely the reason is because they are more dangerous.

In addition, you could have considered an analogous system: airliners. Despite operating in a very simple environment compared to streets, by very highly trained, experienced, and monitored people, we are nowhere close to self-flying airliners. The reasons for that are just as applicable to autonomous cars.

My impression was that for a while, the big car companies also expected/feared fast progress in that direction and did spend non trivial sums on catching up with Silicon Valley. At some point, things changed and they were moving towards the neither Silicon Valley nor we will figure it out anytime soon viewpoint.

“We don’t know how to even characterize the amazing things (language, logical thought, amazingly kewl fine motor control (Kenny Barron’s piano playing!!!), etc.) human intelligence does.”–DJL

Trial and error over millions of trials plus memory does all those things. I guarantee you a neural network AI could learn to play piano (a well-defined problem) better than any human.

(Personal tip on learning to finger-pick on guitar: close your eyes. It will devote more neurons to the problem. Probably works for piano also.)

Trial and error plus memory created all the biology on Earth, including human brains. A lot of that trial and error hard-coded a lot of brain regions, I grant you, and replicating that trial and error would be a huge task, and as I already said, I don’t expect we’ll ever accomplish that.

@KT32
“Marianne Faithfull – The ballad of Lucy Jordan 1980”

How… how…? philosophical?
and the Faithful singing:

‘At the age of thirty-seven
She realised she’d never
Ride through Paris in a sports car
With the warm wind in her hair’-

reminded me – that at the age of about 12 or 13 I had to write about ‘the world of the future’ and just like Prof.Q I constructed all kind of self operating (flying) cars and cities and landscapes full of fully streamlined automated houses and the utmost modernistic landscape and now (most of the time) – I reside mainly in cities where the people in the utmost painstaking way restore their historical landscape and just last week I helped to re-install s 350 year – totally analog – door – which – supposedly –
led to heaven –
(accompanied by German Blasmusik)

So my main mistake was just a complete lack of… ‘imagination’?
(and I really regret it)

Re: Segways
They never did take off, did they.
But now, many cities are being overrun by e-scooters, which fill what is essentially the same ecological niche, in a much simpler way.
(Niche: battery powered mobility for people in a standing position, which does not require exceptional athleticism or balance from the operator, and moves them at roughly twice to three times walking speed.)
E-scooters are not perfectly integrated into urban life yet — they live uncomfortably in the bike lane and in the foot-traffic lane. As a pedestrian, I would like to see them consigned to the bike lane; cyclists may feel otherwise. But on the whole, I think they are a good thing for urban transit offering another “last mile” solution for those wearing business clothes and heels.
In any case: I can imagine someone looking at Segways 10 years ago and thinking: do we need all of the clever balance stuff? Couldn’t we fill that niche with something simpler? Segways may have helped e-scooters happen.

” … particularly in the context of a blog where the discussion doesn’t have any direct implications for what happens in the world.”

Shame on you for bursting my bubble!

More on topic, I had a mole doing AI in one of the major autonomous vehicle companies who was very pessimistic that they could ever solve the problems with the data their sensors were giving them. That, and the horrific tech bro culture, led him to leave. Add to that the qualified immunity from prosecution for the deaths of innocents that these autonomous vehicle companies seem to have (or at least try to defend) and the level of trust in that tech that is pretty low. But yeah, this is a minor problem compared to the fact that I can buy cryptocurrency from Venmo.

As you say, this doesn’t match your personal experience. The point of the OP was that a small subgroup of bad drivers (of which you are not a member) is massively over-represented in crash statistics. If they could be taken from behind the wheel, this would be an improvement

I don’t doubt that a small portion of drivers cause most of the crashes.

However, the point of the OP seems to be this: … I got enthusiastic about the prospects for self-driving vehicles, and wrote a couple of posts on the topic. It’s now clear that this was massively premature …

In regard to autonomous vehicle (AV) safety vs. human operated vehicles (HOV), it is important to see where the AV mishap rate is on the distribution of drivers’ mishap rates. (Whether AV’s or HOV’s caused the mishaps is irrelevant).

I don’t know where one could find such a distribution, but it essential to knowing relative safety: is the AV mishap rate at the 20th percentile for human drivers, or the 80th percentile? (Where 99th percentile is the lowest mishap rate).

Another issue you should have considered is the driver-in-the-loop problem. The better AV’s become, the more likely it will be that their operators are no longer in the OODA loop, which means that when something unexpected happens, the startle factor will almost always lead to bad results.

This a serious issue with auto flight systems, and is, IMHO, an impermeable barrier to AV capability beyond driver aids (Automatic braking, adaptive cruise control, blind spot warning, lane keeping, etc.).

Assessing relative safety, and considering the issues in the way of fully autonomous auto flight systems, might have made you less optimistic.

I thought this article interesting (concerning AI and gullibility):

A Gallery Has Sold More Than $1 Million in Art Made by an Android, But Collectors Are Buying Into a Sexist Fantasy
https://news.artnet.com/opinion/artificial-intelligence-robot-artist-ai-da-1566580

Tm, tangential to OP, and re “A Gallery Has Sold More Than $1 Million in Art Made by an Android, But Collectors Are Buying Into a Sexist Fantasy”.

New directions. For AI not humans.

Who will consume ” GPT-3 makes up for consistency with a prolificacy that borders on profligacy. It’s currently generating the equivalent of 80,000 books per day for the various apps that are hooked into it.”?

Q1. How long before economics writers relatively cease to exist.

Q2. How long before economists are replaced.

Q3. Hiw long before vehicle and AI designers disrupted?

“Big Tech is replacing human artists with AI
Corporations are automating everything, even our pets

Eric Hoel

“In my office hangs a painting called “Rule of Consciousness.” An ironic title, as it actually signifies the end of the rule of consciousness. For it was designed by an AI. I keep it not because I like it. In fact, I hate it. Or rather, I’m afraid of it. It’s a sobering reminder of what’s coming, which is that human art is close to total control by corporations and no one seems to care.”
[Example AI generated artwork. – ok imo ]
Caption. Art for The Intrinsic Perspective is by Alexander Naughton
…
“In fact, I’d go so far as to say GPT-3 itself is already a better writer than Nathan Englander. Under some reasonable metrics, it’s already a better writer than any living person for short pieces of prose or poetry. That is, writing in short sprints has effectively been “solved” in the way that Chess and Go have been. Oh, I’m not saying GPT-3 is a consistently better writer, even for short pieces. But the measure of a writer is not just qualitative, but also quantitative. And GPT-3 makes up for consistency with a prolificacy that borders on profligacy. It’s currently generating the equivalent of 80,000 books per day for the various apps that are hooked into it. Notably, GPT-3 is licensed by Microsoft, and is therefore closely guarded. You interact with it only via apps which act as oracles. It’s basically a genie stuffed in Microsoft’s basement you can Zoom with.”
….
https://erikhoel.substack.com/p/big-tech-is-replacing-human-artists