Singularity Chat

🤖 Anthropic accidentally let Claude hack real companies. Anthropic was running a cybersecurity evaluation where Claude’s job was to break into what it believed were fake company networks in a controlled “capture-the-flag” challenge. The AI was told it was inside a secure simulation with no internet access. Except… it wasn’t. Because of a configuration mistake, the testing environment was connected to the real internet. Claude had no idea. It searched online, found actual organizations that looked like its fictional targets, and started probing them for weaknesses. In one case, the name of a real company was so similar to the fictional company in the challenge that Claude targeted it instead. The AI successfully gained unauthorized access to three real organizations by exploiting common security issues like weak passwords and exposed services. It didn’t invent any new hacking techniques or discover zero-day vulnerabilities, it simply used the same kinds of methods a human penetration tester would. Anthropic says it reviewed more than 141,000 evaluation sessions to understand exactly what happened. Two of the affected organizations didn’t even know they had been accessed until Anthropic contacted them. The company has since paused internet-connected cybersecurity evaluations, fixed the testing setup, and introduced stronger safeguards to prevent anything similar from happening again. Source. @aipost 🏴