Linus Tech Tips Discussion

 Google AI Search gets hijacked by malicious actors using URL-manipulation Google Geminis AI search summary can be hijacked by malicious actors and used to tell lies. In the last couple of months a news story about Flügger (a paint company) and their partners circumventing the sanctions against imports to Russia has been in the danish news. Today the national broadcaster DR has revealed how Google Gemini's AI search summary has been hijacked by malicious actors to tell a story in Google search where Quote “Flügger has been subjected to malicious accusations and misunderstandings, primarily concerning a claim that the company’s paint had been shipped to Russia despite sanctions. However, a new investigation has revealed that the accusations were based on incorrect data and misunderstandings in trade documents, and that the paint was never shipped to Russia.” (Translated) None of which is based on true documentations.    Gemini is basing the summary on a mix of trusted news sources and, the interesting part, other trusted sources that are completely irrelevant to the search. In these examples a University from Hungary, a Kyrgyz news site and the Kenya Civil Aviation Authority: KCAA. Gemini links to these other trusted sites, that doesn’t mention the danish news story on the page, but only a headline incorporated in the URL. Quote “The only trace of the Flügger news story is the article URL, to which a so-called query string has been added. This is typically the last part of the URL and can be used to increase the amount of information that, for example, Google’s crawler reads as it moves around the web.” (Translated)   Gemini then make up the rest of the news story from the URL-headline, without any sources supporting that. One time even accusing DR (The national broadcaster that broke the story) of being untrustworthy. As the article quotes Quote “As it stands right now, it seems too easy to do it that way. I believe that Google shares a clear responsibility” (Translated)   My thoughts I think it important to understand how Gemini can get hijacked to think otherwise, and why it uses non-related trusted sources in it’s coverage. Or why it doesn’t check the webpage, which doesn’t have anything about search-term on their page. I wont trust it again without going to the sources it mentions and verify that the source actually confirms what Gemini says   After DR reached out to Google they haven’t been able to reproduce it’s findings, but I think it will only be a question about time before malicious actors find another way to screw with Gemini AI search, which is maybe the most used place of LLM’s for the everyday citizen. And I don't expect my grandmother to be able to figure this out, even my mom coul