Buyer´s Guide: Third Party Cyber Risk Ratings Tools

Buyer’s Guide for Third Party Cyber Risk Ratings Tools

You can download the buyer’s guide at the end of this article.

Supply chain risk management for large organizations or operations of critical infrastructure increasingly include cyber risk monitoring. From the suppliers’ perspective, having a shared repository that their customers can access, helps to avoid repetitively answering the same questions about their security controls and certifications (e.g., “Do you have IT emergency response plans in place and perform exercises at least yearly?”).

In general, there are two different approaches for security monitoring: The more common is to send out questionnaires in one way or the other. Automatic scanning solutions, on the other hand, are becoming more widely used – even if only a small part of actual cyber risk is covered.

But what is the best tool to support these efforts? Most solutions for third party / vendor risk management (VRM, TPRM) or governance, risk and compliance (GRC) as well as full procure-to-pay suites now offer at least some functions for vendor cyber risk ratings. External attack surface management (EASM) tools also market their ability to scan suppliers’ and partners’ exposed assets for vulnerabilities or misconfigurations.

Potential vendors (Examples):

VendorGartner: Market Guide for third party risk management systems, 2022Gartner Peer Insights ‘Voice of the Customer’: IT Vendor Risk Management Tools, 2022Forrester: Cybersecurity Risk Rating Platforms, 2021
RSA Archer (RiskRecon as cyber risk engine)xx
Diligent (former Galvanize)xx
Security Scorecardxx
Black Kitexx
RiskRecon (MasterCard)xx
Red Maple FractalScan
Rapid7 Intsights
ProcessBolt Threatscapex
CyberGRX / ProcessUnityxx
Palo Alto (Cortex Xpanse)
Darkscope CIQ360x
DGC Cyberscan

The list above excludes specialized offerings like e.g., Hellios who serve a community of defense and financial sector buyers in the UK. Some of the companies mentioned in analyst reports have merged in the meantime (e.g., Thirdpartytrust with Bitsight).

Scope and accuracy of external scans

As part of a recent RfP, we asked several vendors of third-party cyber risk scoring solutions to scan the same test domain and provide us their results for comparison purposes. As visible in the table below, the example findings differ significantly. The test domain was managed by CyberCompare – obviously, nobody is perfect, and the scan results helped us in closing some security gaps.

Of course, the relevance of each single finding in terms of security posture or data privacy is also very different and will vary in between customers and exposed assets. There could also be differences in findings because the timing of the scans was not identical. However, the categories shown might help in defining detailed requirements or evaluating vendor solutions.

Issues (examples)AreaBitsightBlack KiteCybervadisLocateRiskRed MapleRiskLedgerRiskReconSecurity ScorecardValid finding (at time of scan)
DNSSEC not supportedDNSxxxyes
No CAA Entry foundDNSxyes
No DANE entry foundDNSxyes
BIMI not supportedDNSxyes
SPF record missingDNS/Mailxxxxxxyes
DMARC mail record missingDNS/Mailxxxyes
Domain SquattingDomain Squattingxyes
Open SSH port (website)Networkxxxyes
Old/Weak TLS Versions accptedNetworkxxyes
Nginx CVEsPatchingxxxyes
OpenSSH CVEsPatching
CSP Not SetWebAppxxyes
X-Frame-Options header not setWebAppxxxxxyes
Referrer Policy header missingWebAppxyes
HSTS-Header not SetWebAppxxxxyes
Insecure HTTPS RedirectWebAppxno
X-XSS Protection not enabledWebAppxxxxxyes
Advertisment of Nginx versionWebAppxxyes
Weak CBC Ciphers offeredWebAppxyes
Inline CSSWebAppxno
HPKP not enabledWebAppxyes

Criteria for shortlisting and selection

Choosing the right tool should follow a thoughtful decision on the vendor cyber risk management process. No matter which solution is chosen, somebody in your own organization has to review the vendor feedback, work with findings and follow-up on actions (e.g., deal with missing vendor feedback, coordinate with the purchasing department), unless the whole process is outsourced as a managed service. This in itself will cause a substantial workload for your own team or cost for an external service provider. Also, while many vendors showcase their findings for AWS, Microsoft or Google (as they are some of the largest suppliers for most organizations today), how useful is this information actually?

For many companies, supply chain cyber risk is actually a lower risk than ESG or other compliance related risks. Therefore, a very pragmatic approach (e.g., only tracking very few vendors or only asking very few high-level questions) might be sensible.

For selection of the right tools, here are some typical requirements and questions:

  • Are either questionnaires or external scans sufficient, or are both required to manage third party cyber risks?
  • Can existing software (e.g., GRC) be integrated or a TPRM cyber risk module be added? For example, several external scan tools integrate natively with questionnaire-based compliance monitoring tools.
  • How many suppliers in which sectors are already onboarded on the platform? If there is a large overlap with your supplier base, you might save your suppliers and yourself lots of effort. Also, the benchmarking data might be much more useful if the regional industries covered are relevant for you.
  • Are assessment questionnaires (e.g., for ISO 27x, IEC 62443, NIST CSF, GDPR) available off the shelf and updated automatically whenever there are new/changes to these standards?
  • Can questionnaires be customized?
  • What other (non-cybersecurity) risks can be monitored?
  • Supported languages
  • How long does it take to get results for a new supplier?
  • Is there a managed service available for interactions with suppliers?
  • What is the standard frequency to update external scans (yearly, monthly, daily)?
  • What security controls are checked in the outside-in scans, and how reliable are the results?
  • Are cyber risk ratings and the false positive rate (both for discovered assets and for missed security controls) calculated/tracked in a transparent way, and what is the process to deal with false positives?
  • Are suppliers in control of the data they share? E.g., can they choose (or decline) to share the data also with other customers?
  • Is dark web threat intelligence used for ratings and recommendations?
  • Licensing costs for various scenarios (e.g., depending on number of scored vendors, number of scans)

CyberCompare is buying security tools and services every day on behalf of our customers. We are happy to advise you in your project to make sure you get the best security for your budget.

Download the Buyer’s Guide.pdf

CyberCompare Logo

Protect, Detect, Respond

20. June 2024 | 09:00 - 11:15 AM

Gain insights from security providers on incident response and forensic analysis of cyber attacks.