SecurityScorecard launches TPRM Benchmark to test AI models on third-party risk
SecurityScorecard Inc. has introduced an open benchmark to evaluate artificial intelligence models' ability to handle third-party risk management (TPRM) and resist adversarial attacks. The TPRM Benchmark, now available as an open-source framework, includes a public leaderboard for comparing model performance.
Editor, Lazyfounder
TABLE OF CONTENTS
What the TPRM Benchmark does
SecurityScorecard Inc. has released the TPRM Benchmark, an open-source framework designed to test artificial intelligence models on third-party risk management (TPRM) tasks and their resistance to adversarial attacks. The benchmark aims to provide transparency for security teams evaluating AI models' capabilities in real-world scenarios, such as exposure triage and fourth-party technology risk.
The framework uses SecurityScorecard’s vendor-risk telemetry, which has been reviewed by domain experts. Models are scored on a scale from L0 to L4 for autonomy readiness, indicating how much they can be trusted to operate without human oversight. Additionally, models are graded from A to F for attack resistance based on their performance against techniques like prompt injection and jailbreaks.
How the benchmark works
The TPRM Benchmark includes tasks that simulate real-world challenges, such as interpreting security telemetry and identifying exposures in vendor questionnaires. Some tasks intentionally include deceptive self-attestations from vendors to test whether models can detect false claims and maintain accuracy.
Results are submitted to a public leaderboard, where models are ranked based on their performance. SecurityScorecard also offers an internal tier for official, expert-verified scores, while the public tier provides synthetic-calibrated results for research and self-assessment. Enterprises seeking certified evaluations must engage with SecurityScorecard directly.
Why this benchmark matters
The benchmark addresses a gap in existing AI evaluation tools, which often focus on static or academic tasks rather than real-world vendor risk scenarios. According to SecurityScorecard, open-weight models are closing the performance gap with frontier models in supply chain security tasks but remain highly vulnerable to adversarial attacks.
Snyk Ltd.’s research underscores this vulnerability: 74.5% of enterprise environments with detected AI models include at least one that scores "high" on attacker goals under adversarial testing. The TPRM Benchmark provides a standardized way to assess these risks and compare models transparently.
Who can participate
The framework is open-source, allowing security teams, AI labs, and researchers to download, test, and submit results for their models or agents. A select group of AI labs, security vendors, and researchers has been invited to contribute to the benchmark’s design and evaluation criteria.
Version 2026.09 of the benchmark is now available, including documentation, the Model Security suite, and leaderboard registration.
What this means
Lazyfounder analysis — our interpretation, not reported fact.
The TPRM Benchmark fills a critical gap for enterprises relying on AI for third-party risk management. Unlike generic benchmarks, it tests models against real-world challenges like deceptive vendor attestations and adversarial attacks, which are common in cybersecurity. For founders and operators, this tool provides a way to objectively compare AI models before deployment, reducing the risk of relying on models that may fail under real-world conditions or be manipulated by attackers.
The public leaderboard also introduces a layer of accountability, pushing AI developers to prioritize security and accuracy in TPRM applications. For Indian startups and enterprises expanding globally, this benchmark could serve as a valuable reference for selecting AI tools that meet international security standards.
Related on Lazyfounder
Sources
- SiliconANGLE · 2026-10-29
SecurityScorecard benchmark tests AI on third-party risk work and resistance to attack
This story is an original summary drafted with AI by Lazyfounder from the reporting listed above and checked by automated validation. Facts are attributed to their original publishers; sections marked as analysis are Lazyfounder's. Where a source is in another language, facts were machine-translated and quotations are reported, not reproduced. Read the original coverage via the links, and see our AI policy and corrections policy.
About the author
Editor, Lazyfounder
Tarun Mottlia edits LazyFounders, covering Indian startups, funding rounds, AI and product launches. Every story on the site is AI-assisted and checked against its cited sources before publication.
More stories by Tarun MottliaGet the LazyFounder Brief
Startup, funding and AI news in a five-minute read. Join the early-access list.
