Marktkommentar #23: Incident Response

Hello everyone

am I actually the only one who lives in constantoverwhelm due to the deluge of new developments and articles around security ? While I’m crawling around in the rabbit hole of a newsletter (some are getting longer and longer, so I suspect OpenAI as a co-author), my colleagues have already posted 25 pieces of information from projects in Teams that I should urgently take a look at. The old iX is not yet halfway through, but the next one is already coming. On LinkedIn you are not only bombarded with megacringe self-portrayals and the most intrusive advertising, but every now and then you are delighted with a hidden gem => Don’t miss it! I now need more time to cancel well-intentioned personal invitations to events than I do to eat. The latter, of course, means virtual lunch with experts, of course. And when jogging, I don’t listen to the sounds of Shirin David, but security podcasts about SIEM and SOC, to stay ajour. With a playback speed of 1.25. Fortunately, the summer break is coming soon.

Keywordsecurity podcasts: Many of them (as well as security lectures) suffer from the fact that they are superficial and meaningless, avoiding any controversy or concreteness and thus annoyingly boring. Or I suffer from the fact that I simply do not understand 😉 them. On the other hand, I have often had good experiences, e.g. with theBlue Security Podcast (2 Microsoft employees, but not sponsored/edited by Microsoft) and the Arctic Wolf Podcast, which is moderated by Sebastian Schmerl with technical depth and at the same time humorously. Here is some information from thepractice of incident responders, which I had noted down when listening to it in the “Wolf’s Den” and from other interviews with practitioners:

  • In general, there are almost no on-site assignments anymore, all remotely
  • Network quarantine is regulated by firewall, not the famous power plug. There are also the exceptions for the IR team (Alternative Mobile Communications)
  • Example ECommerce: Emergency operation established in which only orders were forwarded through the firewall
  • At the same time, a jump host is installed that is not domain-joined
  • Early closing time is more important to attackers than remaining undetected (“Smash & Grab”) => Speaks for 24/7 monitoring and response capabilities
  • Initial access brokers are increasingly focusing on vulnerabilities in firewalls, VPNs and security gateways
  • As one of the first steps, the negotiators ask for a list of the stolen files. This can often be used to derive a list of servers that are affected
  • Data exfiltration: R-Sync or OneDrive are often used, in individual cases exotic ways such as limewire.com. A list can be found on LOTS Project – Living Off Trusted Sites
  • So far, no DLP tool has effectively prevented data exfiltration. In general, it is difficult to detect data leakage, visibility is usually lacking
  • 60-90% discounts on the blackmailers’ initial demand are common. Qualys has reported that e.g. Lockbit gives a 20% discount if you pay in Monero instead of Bitcoin (probably due to the better anonymization of payment flows).
  • Procedure for searching for Patient Zero:
    • Search for spikes in FW logs or flow data => Look for more indicators of exfiltration there
    • Search for newly created folders and newly installed tools
    • Search for executables in autostart (for machines that have not yet been rebooted)
  • Data from the customer (selection, typically only about 1-2% of the total log data) is loaded into quarantine storage and then automatically examined
  • If the customer has a reasonable EDR (no matter which one), it can be used => Thor scanner or Velociraptor or similar apparently not absolutely necessary. The core problem, of course, is that typically the rollout has not been completed
  • C2 channel: Remote access tools such as Teamviewer or Lolbins, e.g. PSExec or WMIC are still popular
  • So far, there is no known case in which OT production systems have actually been disrupted, attackers always focus on IT
  • In many cases, even Linux is too complicated for the attackers and Exfil+ encryption only breaks into Windows servers
  • The trend is to encrypt hypervisor => No EDR monitoring, configuration often poor
  • It has also happened that the ransomware gang has confused a company (and then wanted to blackmail one from which no data was stolen at all). Or it has already placed the payloads, but then accidentally locked itself out using a firewall rule. The bad guys are all human beings, very calming.

How is themarket for incident response services actually developing (thanks at this point especially to Sven and Ruben for the input)?

The BSI’s list of qualified APT service providers is continuously expanding, and there are now 58 providers listed there (of course there are more) – about twice as many as a few years ago. Or, to quote a colleague, “Everything that wasn’t on the trees at three”. In fact, in addition to the big players in the industry, the BSI list now also includes very small service providers where one wonders how the 24/7 readiness of sufficiently qualified forensic experts can actually be reconciled with the laws of physics. Many of the listed providers also have no legal advice, crisis communication, help with reconstruction or a technical laboratory in their portfolio. Certainly, an expansion of the criteria (e.g. number of missions actually carried out) would be appropriate.

Of course, the reason for the growth in list length and vendor landscape is thatincident response retainers have been among the most profitable services you can offer in the cybersecurity space. From the provider’s point of view, the contracts often have the following advantages:

  • Reservation of capacities => Similar predictability as with managed services. Even with 3-4 small retainer contracts, a full-time professional is fully financed with a toolset
  • The agreed hourly quotas (typically: 40 hours) are usually not enough for the assignments. Usual daily rates beyond the quota are around ~3000-4500 EUR, i.e. 2-3 times as high as for SOC analyses, pen tests or similar. And if the hut is on fire, there is no haggling with the fire brigade – the added value for the customer is quite tangible
  • The substitute services, when the contingent is not needed (which should statistically be the case in ~80-90% of the years), have a significantly lower value in many contracts (e.g. training)
  • Preparatory activities such as tabletop emergency exercises are an ideal basis for an intensive relationship of trust with the customer. And if you can actually help customers out of trouble with an assignment, a change of service provider is actually unthinkable
  • Customers find it difficult to assess the qualification of providers
  • Software sales in the context of IR use is a lucrative source of income

The number of deployments is probably also growing – at least I don’t have exact figures, but we can use the reported ransomware attacks in Germany by the police as an approximation.According to BKA situation reports, these have risen to 950 in 2024 (approx. 20% more than the 800 reported cases in 2023). On average, with about 70 IR service providers (not all of them are listed with the BSI), this results in an order of magnitude of about 13-14 assignments/year and IR team or 1-2/month. As an aside, if the BKA numbers are roughly correct, it means that about 3% of all organizations affected by NIS2 are successfully attacked per year.

However,the increasing competition is also noticeable in this category:

  • The price/performance ratio for customers is getting better
  • The first vendors sell retainers in which the entire application is covered, i.e., regardless of the effort, a result is delivered = > advantage for customers if the SoW is clearly defined
  • Customers occasionally contact IR service providers and want to reduce the agreed number of hours
  • Cyber insurance companies offer IR with their own departments or conclude wholesale contracts

It is obvious that MDR/MSOC and IR will continue to grow together. The synergies are clear, the boundaries between SOC Level 3 and post-breach IR are becoming increasingly blurred (the clearest definition for me so far: More than 1 host affected = post breach incident response). My best guess is that we see a consolidation in the MDR/MSOC market to ~80 providers in the DACH region by 2030 (i.e. about a halving), but then practically all of them also offer incident response retainers.

Other data points / experiences / additions? I’m looking forward to input.

By the way, when choosing a suitable IR service provider (and also security service providers in general), you can have AI support shown as a criterion. Only companies with sufficiently proprietary data / case studies from deployments should be able to teach their own tool with it at all.

Which brings us back to AI… Two original observations from Spotify CTO Gustav Söderström on this:

  • When spreadsheets came onto the market, there was a prediction that all accountants would lose their jobs. In fact, however, we have all become small accountants with the help of Excel, who use the spreadsheet for every rough calculation. And the number of permanent controllers who suspiciously question our amateurish spreadsheets has also increased. The cost of complex calculations has fallen to zero, and managers want more and more information and planning security. => Söderström believes, AI will therefore most likely not lead to job losses, but to the enrichment of jobs. The bar for expected quantity and quality of employee output will continue to rise.
  • Most software developers work at large established companies. However, large companies spend ~90% of their development capacity on adapting the existing code base (and only ~10% on programming new code). Currently, the AI is not yet good enough for refactoring or a real 4-eyes review. The published successes all refer to new developments (such as the recent Cloudflare OAuth2.1 implementation by Claude). However, this will change in the foreseeable future => The real efficiency gains on a broad basis are still to come, even in software development. This also fits the statement of the Google CEO last week that Google has achieved “10% engineering velocity increase using AI” so far .

Notable Cyber Investment Banking News:

  • Tenable buys Apex (Discovery of Shadow AI applications)
  • Netskope (CASB/SASE, approx. USD 500 million in revenue) prepares for IPO, target valuation USD 5 billion
  • ZScaler acquires Red Canary (US MDR provider, approx. 500 employees, ~100 million USD revenue) for ~700 million USD. Indicative of a general trend: service providers are becoming more attractive takeover targets
  • Cellebrite (“Accelerate Justice” – that’s the company that cracks mobile phones for the FBI) buys Corellium (digital twins of mobile devices and IoT / automotive ECUs)
  • Checkpoint acquires Veriti (Attack Surface / Exposure Management, including virtual patching for cloud workloads)
  • CyberArk (PAM/IAM, approx. 1 billion in sales, loss-making, but cash-flow positive) is borrowing 1 billion USD, presumably for further acquisitions. A look at the annual financial statements reveals that every 2nd dollar of sales at CyberArk is invested in marketing and sales. The resale is carried out by well-known resellers such as KPMG, Deloitte and PwC – and of course they are well paid for it.

Vendor Briefings:

FAST LTA:

  • German manufacturer of HW storage from Backup + Archivierung, i.e., competitor to IBM, NetApp, HPE, Pure Storage, partly
  • In-house development and production => Continuously short delivery times
  • Approx. 3500 corporate customers, including Aurubis, ZDF, Saint Gobain, many clinics, including Charité
  • Archiving: Focus on audit-proof hardware-based immutable (immutable / WORM) storage without data loss (WORM) Each memory has its own clock to prevent manipulation of the timestamp.).
  • Continuous automated integrity check (all data every 30 days)
  • Backup: Software-based immutability of snapshots. In addition, “Air Gap” option: Motor can be physically decoupled to prevent overwriting => Of course, this is at the expense of access times.
  • Integration with common SW solutions such as VEEAM, Cohesity, Commvault, etc.

Material Security:

  • M365 + Google Workspace Security – Mailbox, Sharepoint, Google Drive.In principle, a combination of phishing protection and DLP, clearly positioned against Proofpoint and Mimecast and as a complement to MS Defender
  • Customers include Starbucks, Mars, Datadog, Coinbase
  • Scan mailboxes (i.e., not just one-time filtering on receipt like standard mail gateways) regularly via the API for anomalies and new IoCs, also insider threat features such as unusual forwarding, and whether a user has clicked on the phishing link in the meantime
  • Automatic redacting of sensitive documents and email content
  • Posture Management: Check of settings against benchmarks + automated re-configuration (if you want to)
  • So far no resellers in the DACH market, so if you are interested, take a look. The introduction to the presentation was about the 20% margin that you can earn as a reseller

Appdome (Update):

  • Of course, the focus will continue to be on customers with payment and/or security-related apps – banking, gaming, e-commerce, healthcare and, most recently, hotels where rooms can be opened via app
  • I took a look at the new Anti Account Takeover feature: Deep fakes in conjunction with AI reengineering of facial recognition algorithms are now easily undermining Face ID and other biometric authentication methods in banking apps, for example. Attack methods include:
    • Spoofing the API or SDK calls between the endpoint and the central image recognition service to forge positive matches
    • “Face Swap” apps to replace the image of the real camera with one of a (virtual) camera
    • Modify the transmitted image data until a positive match is generated
  • The detection of such attacks works, for example, by checking which other applications are running on the mobile device, whether image data is buffered or the encoding algorithms have been changed
  • Compromised apps can now be closed – both on the mobile phone/tablet and by stopping mobile network traffic via WAF/Botnet Protection

Kertos:

  • German provider of a GRC solution with supplementary services (“consultation hours with DSB/ISB”), approx. 50 employees, VC funding
  • Approx. 180 customers, well-known names including Grohe, Personio, Flink, Enpal
  • In addition to tracking frameworks (GDPR, ISO27001, SOC2…), policy templates, automatic capture of technical controls via integrations, also standard training and workflow tool with progress tracking
  • Risk management: Catalog of risks that can be selected and linked to proposed controls for treatment
  • One of the founders comes from Hamburg. For this reason alone, all readers should take a look at it!😉

As always: Questions, suggestions, comments, experience reports and also opposing opinions or corrections are welcome by email. Ditto for unsubscribing from the mailing list.

For the people who have received the market commentary for the first time: Here you can register if you are interested or dust in the archive.

Regards

Jannis Stemmann

Scroll to Top