Linus Tech Tips Discussion

 Autonomous cars, drones cheerfully obey prompt injection by road sign [Article by TheRegister] Summary Cars and drones powered by AI using LLMs can be fooled to obey commands that are placed on signs they can see while operating.    Quotes Quote Researchers have discovered a dangerous security vulnerability: Modified road signs can be used to manipulate autonomous vehicles and drones. The attacks are alarmingly effective – success rates in tests reached up to 95 percent. Scientists at the University of California, Santa Cruz, demonstrated in simulations and with model vehicles how easily autonomous vehicles can be tricked. The problem: The artificial intelligence in the vehicles interprets text on signs as direct commands – instead of simply reading it as information, as The Register reported. [Translated via Google Translate from the article on Golem.de]   My thoughts This is another example of mindless application of LLMs although they are still much too easy to manipulate. It seems to me, that in the crazy hype for this new technology common sense remains an afterthought. Many situations in the real world have to be interpreted with such common sense to get to the right conclusion and I doubt that LLMs will ever reach something like "common sense". A human would not refrain from slowing down their car by somebody running with a sign spelling "proceed" across a pedestrian crossing on which people are present. These AI models have never learned to ignore the sign in such a situation - and even when they have learned or have been thought, people will (in my opinion) always be able to trick these models with new situations they have never learned to handle.  It's the same story with picture recognition of dogs. Let an AI learn the races of dogs by feeding it pictures of them from the internet, and it will recognize dogs on pictures after the learning phase quite reliably - unless there is a picture of a Siberian Husky on a green lawn. The AI learned, that such dogs always have a white background of snow, so in this case it will fail to recognize it. You cannot tolerate such failures when lives of people might be on the line.   Sources Gefälschte Straßenschilder: Autos lassen sich per Prompt Injection fehlleiten - Golem.de [de/Germany]  Self-driving cars, drones hijacked by custom road signs • The Register [en/UK]