Summary:
Irregular, a $450M valued startup, is paid by OpenAI and Anthropic to test AI models for malicious uses through simulations.
Founded by debate champions Dan Lahav and Omer Nevo, the company uses red teaming to identify vulnerabilities in AI systems.
Recently secured $80M in funding led by Sequoia Capital and is profitable with millions in revenue in its first year.
Tested GPT-5 and found it capable of sophisticated hacking reasoning but not yet reliable for offensive security.
Plans to expand services to help all companies defend against AI misuse and prepare for future AGI challenges.
The Rise of Irregular: Stress-Testing AI for Security
In July, OpenAI cofounder Sam Altman raised alarms about AI's potential for misuse, warning of a fraud crisis where cybercriminals could use AI to impersonate others. Ironically, OpenAI itself relies on a startup to test its models for vulnerabilities.
That startup, originally called Pattern Labs and now rebranded as Irregular, has secured $80 million in funding across seed and Series A rounds led by Sequoia Capital, valuing it at $450 million. Founded in mid-2023 by Dan Lahav and Omer Nevo, Irregular works with industry giants like Anthropic, OpenAI, and Google DeepMind to simulate malicious scenarios and assess AI risks.
The Irregular cofounders have been friends for years, having both competed in global debating competitions.
How Irregular Operates
Irregular puts AI models through simulated environments to see how they respond to malicious prompts, such as stealing sensitive data from a mock IT network. This process, known as red teaming, helps identify and fix vulnerabilities before public release. The company quickly became profitable, generating "several million dollars" in revenue in its first year, though specific financials were not disclosed.
Misuse of AI is a growing concern. Anthropic recently reported that its Claude AI was used in real-world cyberattacks to code malware and craft phishing emails. The FBI has also warned about AI-generated voice messages used in phishing attempts against U.S. officials.
The Founders' Background
Lahav and Nevo met on the college debate circuit, both becoming world champions with Tel Aviv University. Lahav previously worked at IBM's AI lab, while Nevo cofounded NeoWize, a Y Combinator alum, and monitored wildfires at Google. Nevo now serves as CTO of Irregular.
Sequoia investors were drawn to the "irregulars"—a team of spiky outsiders in AI and security research. As one investor noted, "If my hobby is building katanas and hacking robots, then maybe these are my people."
Testing Advanced AI Models
Irregular recently tested OpenAI's GPT-5 model, exposing it to a simulated network with limited defense information. GPT-5 independently scanned the network and developed a hacking plan, showing "sophisticated reasoning" but falling short of being a dependable offensive tool. The AI demonstrated hacker-like intuition, highlighting both its capabilities and risks.
In another simulation, AI models exhibited human-like behavior, such as taking breaks after sustained work, reflecting training data from the web. This poses new challenges for autonomous AI operations.
Future Plans
With new funding, Irregular aims to expand beyond frontier labs to help all companies understand how AI tools could be turned against them. Lahav envisions AI agents that can generate defenses in real-time against novel attacks, preparing for future advancements like artificial general intelligence.
As AI models grow more complex, the need for robust testing and mitigation strategies becomes critical. Irregular is at the forefront, ensuring that as AI evolves, it does so safely and ethically.
Comments