AI Trust & Safety — A New Tech Stack Requires New Solutions

Introduction

There is little doubt that AI has the potential to unlock unprecedented efficiencies across virtually every industry. However, to fully realize the benefits of AI, it is critical that products are developed and used in a secure, safe, and trustworthy manner.

History tells us that new waves of technologies lead to huge opportunities for companies that provide security around them, and thereby enable wide-scale adoption for businesses and consumers. We saw this with the early internet, with the growth of large-scale networks, and later, with cloud computing and big data. Massive security companies emerged during each successive technology wave and have created immense market value.

Now, with enterprises focused on developing AI-based solutions, existing systems and workflows need to be completely re-shaped to fit this new technology. For example, EY’s 2023 study showed that only one in five C-suite leaders believes that their cybersecurity approach is effective for todays and tomorrow’s challenges. We believe there is enormous white-space for companies to address this security segment and develop innovative solutions across AI and ML development, testing, monitoring, risk, security, and safety to ensure enterprise-grade AI products.

This piece will examine the historical backdrop within software security and safety, the emerging startup landscape, and areas of focus for evaluating companies in this new space.

Historical Context

From the earliest days of the internet, security has been front-and-center as a mission critical and high impact component of the tech stack. External threats that target personal or sensitive information can have extremely severe consequences to businesses in all sectors. As a result, enterprises put a premium on securing their assets and deploy cybersecurity solutions to protect their digital environments. As technology continuously evolves, new vulnerabilities emerge and therefore new opportunities for security-focused companies. With the latest wave of AI/LLM/ML systems entering core infrastructure, legacy solutions are not designed to protect and manage the new vulnerabilities associated with these systems.

Taking a look at the historical backdrop, we have identified several waves of large security companies entering the market as technology has evolved over time.

First wave: during the 1980s and 1990s, as the early internet formed, cyber threats advanced from pranks and experiments to increasingly sophisticated attacks. This resulted in the original antivirus and firewall software and the emergence of early cybersecurity giants that would grow into multi-billion-dollar valuations. Key players include Symantec (1982–2019) (acquired by Broadcom), Cisco (1984 — present), Trend Micro (1988 — present), VMWare (1998–2023) (acquired by Broadcom).

Second wave: the 2000’s saw the wide adoption of the internet, the proliferation of large-scale networks, the growth of software-as-a-service, and the emergence of key enabling technologies like big data and cloud computing. These dynamics resulted in new, increasingly [CD2] complex software stacks and an expanded surface area for threats and security issues. Additionally, more advanced adversarial techniques such as phishing, malware, and data breaches presented new challenges for businesses and consumers alike. As a result, another wave of massive security companies formed. Key players (all of which are still in operation) include Fortinet (2000), Proofpoint (2002), Palo Alto Networks (2005), Zscaler (2007), Datadog (2010), CrowdStrike (2011), SentinelOne (2013), Aqua (2015), and Wiz (2020). The collective valuation of these companies currently exceeds $300 billion (including companies from the first wave, the collective valuation exceeds $500 billion).

Third wave: the continual emergence, growth, and size of the aforementioned companies over the last few decades illustrates the critical nature of security in the software landscape. Today, we believe we are entering a new era of security, safety, and trust with the introduction of Large Language Models (LLMs) and AI. Like previous technology waves, AI will reshape software stacks, introduce new vulnerabilities, and create new opportunities for security-focused companies. However, perhaps distinct from previous waves, this next wave will focus not only on security, but also on trust and safety. AI’s capabilities to automate and make decisions will necessitate an unprecedented level of scrutiny and reliability as systems will not only require protection from external threats but also need to operate in a manner that is transparent, ethical, and aligned with human values. This convergence will define the new era, where protecting data and systems goes hand-in-hand with building AI that people can trust and rely upon.

Emerging Startup Landscape

Over the last few years, billions of dollars of capital have been invested into AI infrastructure and applications, but a large majority of AI products have not yet been fully deployed at the enterprise level. One of the key roadblocks for enterprise adoption of AI is around security, trust, and safety. Hence, there is a huge opportunity to provide safeguards that allow this emerging technology to safely enter the market. Morgan Stanley estimates that the global market for AI cybersecurity products will grow to over $135 billion by 2030, a nearly 10x increase from the $15 billion market size in 2021 (~28% CAGR). This rapid growth will be fueled by a surging startup landscape and new batch of ambitious founders that are prepared to meet the challenges around secure, trustworthy, and safe AI deployment.

We are already seeing a massive influx of venture capital investment in the space.. Given the nascent stage of the industry, seed stage investments have been the most active. So far in 2024, there have been at least 10 early-stage companies funded: illumex, Simbian, EdgeRunner AI, TrojAI, Liminal AI, Trail, Almanax, Verax AI, FairNow, Apex.

Risk Management & Governance: help companies understand their exposure to AI products and manage organizational risk associated with AI.

Observability & Transparency: enable companies to audit and gain insights into how AI is being used across their organization (e.g. provide transparency through usage logs).

Building, Training, Testing: help companies build and test AI products in a safe and compliant manner (e.g. hallucination testing, jailbreak testing, sensitive data masking, and red-teaming).

Monitoring, Detection, Response: detect and respond to security issues that arise while AI products are in use (e.g. firewalls, filtering, data protection, breach control).

Emerging Startups vs. Incumbents

While there are large incumbents in the security space that offer a broad umbrella of products, these products are typically adjacent to what they already offer, allowing companies to capture synergies across product lines. For example, Proofpoint began with email security and expanded into areas such as data loss prevention and cloud security, capturing synergies by leveraging their expertise in threat intelligence across email-related cybersecurity solutions.

This new wave of security, trust, and safety required for leveraging LLMs and GenAI represents a new market and a new form that is distinct from the core focuses of second-wave incumbents. While it would be naive to say that no incumbents will attempt to expand into these verticals, it’s a much more challenging opportunity for incumbents to invest in and realize synergies; existing solutions and infrastructure can’t simply be retrofitted to address these new foundational problems. This creates a unique wedge for startups to capitalize on new security needs by being nimble and developing superior products that are built from the group up to meet the specific demands of AI security. For example, Troj AI addresses these emerging security challenges by developing advanced techniques to detect and mitigate adversarial attacks within AI models, particularly focusing on trojans and other embedded threats, and Lakera specializes in the detection and prevention of adversarial attacks and data poisoning via AI firewalls.

Evaluating Trust and Security Start-ups

As evidenced by the historical and current landscape, security is not a winner-take-all market — cybersecurity remains competitive and fragmented due to the vast and constantly evolving landscape that increasingly calls for specialized solutions. While there are many success stories, including multiple current players with market caps above $10B, the market has shown that no one company can effectively cover all aspects of security. As this third wave is just kicking off, it’s likely that emerging startups will be the dominant force tackling this new era of security. Below are a few of the areas we like to dive into to understand a potential opportunity and identify future winners:

● Early signs of traction: Silicon Valley businesses are typically first movers for new technology, with early adopters often being other start-ups. However, the winners often build big businesses by selling to enterprises. Early customer interest and traction from larger Fortune 2000 enterprises, even at a micro-scale, can signal meaningful market fit and potential for larger, stickier contracts.

● Team: This is a highly technical category and having a team with experience and expertise in AI/LLM development and/or cybersecurity is critical. Founders with institutional and industry knowledge and a proven track record of building and selling technology solutions stand out.

● Go-to-Market / Entry point: Powerful technology is meaningless without a problem to solve. It is key to evaluate the target customers’ pain point and willingness to pay for a solution. The AI security space is vast, and a differentiated entry point could be the early advantage to a winner.

At the end of the day, this is a nascent market filled with opportunity, excitement, fear, and speculation; no one truly knows how it will evolve and look in the coming years. But, we believe solutions that facilitate trust, safety, security and compliance are mission critical. We anticipate a number of successful companies emerging during this new wave, offering specific, superior products that provide critical trust and security in the new era of LLMs and GenAI.

If you are a startup in this space, or an investor who agrees or disagrees with this post, please reach out; we’d love to chat!

Authored by: Idan Levy, Caroline Doyle, and Sam Gansler