Published on: September 21, 2024
It’s hard for everyday people like me and you to remain on top of new cybersecurity threats and complex concepts while trying to protect our data and digital rights without giving up the convenience of our favorite apps and technologies.
In this interview series by Safety Detectives, I speak with cybersecurity experts who share actionable tips, insider knowledge, and predictions for the future, helping you understand what’s really happening with your data and how you can protect your digital life more effectively—without losing your sanity.
Mitja Kolsek is the CEO and co-founder of ACROS Security, an independent security research lab established in 1999 and recognized globally for its expertise in cybersecurity, particularly in the area of micropatching. Throughout his career, Mitja has been involved in various aspects of cybersecurity, and has co-founded 0patch, a company focused on providing micropatching solutions for software vulnerabilities.
You’ve been conducting penetration testing since ACROS Security was founded. What are the most overlooked vulnerabilities you still commonly find in organizations today?
When we started with penetration testing back in 1999, we found many of the usual suspects: SQL injections (which didn’t even have this name then), cross-site scripting issues, unauthorized access through direct object references, all sorts of other injections, and cross-site request forgeries. Of course, more “interesting”, unique flaws were made possible by specific features or their unique implementations.
Today, our customers for the penetration testing service are highly security-mature organizations, which means we’re typically not finding a lot of vulnerabilities in their products, especially not the “boring” kind.
The majority of issues we find today are in the “interesting” category, and each one is generally special so at least our team doesn’t find any vulnerabilities we could call “most overlooked”. This is not to say that the usual suspects have been globally eradicated, but software vendors with strong security teams today mostly know how to avoid them.
For our audience who might not be familiar, can you explain what micropatches are and how they work in practice? Can you share an example that shows the advantages?
Our experience in penetration testing and helping vendors fix the vulnerabilities we had found at some point produced an interesting idea:
Why don’t we provide our own patches for users who aren’t getting official vendor patches anymore? Like Windows versions that work perfectly well but stop receiving security patches from Microsoft?
A small research project proved this idea viable and grew into a more serious project; after two years we had an initial working product that was able to fix vulnerabilities in Windows applications and operating systems by applying tiny security patches (“micropatches”) directly to the original code in the memory of running processes.
Such a “hot patching” approach was nothing new in the industry but we employed it for security patching on a production scale that hadn’t been attempted before. Our micropatches are small, typically comprising just a couple of CPU instructions that need to be inserted at some location in the vulnerable code to remove the vulnerability. Let’s make an example…
A typical critical issue is a so-called “buffer overflow,” where the attacker supplies data that is too long for a fixed-size memory buffer that at some point receives it. When copied to this buffer, the attacker’s data overwrites memory locations beyond the end of the buffer, modifying internal values and addresses of objects in the memory. By carefully crafting malicious data, the attacker can often manipulate the attacked process into executing arbitrary code.
This type of vulnerability is very simple to fix: before the supplied data is copied to the buffer, its length should be checked, and if needed, the data shortened, or its further processing abandoned. Such a check can be implemented with a few instructions that should have been added by the original developer, but since they’re not there, we can add them with a micropatch.
How does 0patch compare to traditional patching methods? What are its main advantages?
Traditional patching is done by modifying the source code of a product and re-building it into executable components, then delivering these new executable components to users’ computers and replacing the old ones with the new ones – often requiring a computer reboot.
This may sound simple enough but many things can go wrong in the process: the source code has been modified in many places, not just with security fixes but also with various optimizations, functional fixes, and cosmetic modifications. Each of these has a potential to break something and this potential accumulates into a non-trivial risk, so the vendor has to perform lengthy and exhaustive testing to make sure they haven’t broken anything important.
Even so, software updates occasionally do break something important and when they do, that can cause a major incident for users. In such cases, uninstalling the update is often the only solution, and that typically means additional downtime and a reboot. Uninstalling an update of course also means that all vulnerabilities the update had fixed are back and available to attackers.
A micropatch can be applied (or un-applied) without restarting the computer or even relaunching the application we’re patching, no executable files are modified as we’re only modifying code in memory, and the risk of breaking the patched process or functionality is minimal due to the minimal change to the original code. Finally, each of our patches is written for a single vulnerability so even if one of them needs to be un-applied for some reason (which also happens instantly), other vulnerabilities we’re patching remain patched.
The benefits of Micropatching:
- Micropatches can be deployed within hours. Traditional patches can take weeks. Quick response is crucial, especially in environments where uptime is critical.
- By applying fixes directly in memory, 0patch corrects the vulnerable part of the code without the need for a reboot (so no system downtime).
- Extended usability for legacy software that is no longer supported by vendors, like older versions of Windows and Office. This is key for organizations that rely on outdated systems for critical operations.
- Micropatches are quite easy to apply and remove. This simplifies the testing and deployment process, which is something very important for IT administrators managing multiple systems efficiently
“0patch protects our POS devices across 300 stores worldwide, and it has proven to be an excellent alternative to an expensive and difficult upgrade program.”
Denise McConnell, IT Manager, Trespass
Many users don’t fully trust third-party patches. Why, and how do you overcome this objection?
By fixing vulnerabilities in someone else’s closed-source software, we’re breaking some long-established norms so doubt and distrust are normal reactions:
How could a third party understand and fix a vulnerability in a complex product when they don’t even have the source code for it?
How can we trust they won’t break something or even add risks to our computers?
After 8 years of patching (mostly) Microsoft’s security holes, our record speaks for itself:
- Most of our legacy Windows patches are logically identical to Microsoft’s patches that they had issued for still-supported Windows versions.
- Our 0day patches – issued before Microsoft’s – are sometimes followed by Microsoft’s patches that do effectively the same thing, and it also happens that our patch is better than Microsoft’s.
An example was the patch for CVE-2022-44698: while our 0day patch properly fixed the issue, Microsoft had to patch twice since their first patch was flawed.
Then there have been cases where the official update broke an important functionality, and we issued our own patch to fix a likely-to-be-exploited security issue while Microsoft was creating a new corrected update.
(For example, CVE-2018-8174: https://blog.0patch.com/2018/05/a-single-instruction-micropatch-for.html)
Clearly, the original vendor understands their product better and should theoretically be able to create the best fix for any flaw in it. In practice, product teams change and often even an official vendor fix involves someone correcting someone else’s code and wondering what that code was doing and why.
On the other hand, our advantages are:
- Experience with reproducing vulnerabilities
- State-of-the-art reverse engineering and patching toolset
- Expertise in vulnerabilities and exploitation from penetration testing services
- Ability to quickly see where an optimal patch should be so that it doesn’t break anything and also doesn’t leave any bypasses.
These, combined with the fact that minimizing code change, hourly deployments, and applying patches in real-time without a reboot make us sometimes better/faster, sometimes worse/slower, but certainly in the same category as official vendors.
As for testing our patches, these are all subject to two types of tests: security tests must confirm that the patch has removed the vulnerability, and functional tests must confirm that the patch has not changed the original legitimate behavior of the patched code. These tests are always performed by a different person than the person who wrote the patch.
We also publish the source code of many of our micropatches in blog posts, so anyone versed in assembly language can review them. Our plan is to automate this so that all our micropatches will have source code publicly available.
Are there cases where micropatching might be a better solution than waiting for a vendor patch?
Micropatching is perfect for “0day” vulnerabilities: due to the costly and complex vendor’s process of fixing a vulnerability in the source code, rebuilding the product, and thoroughly testing all the accumulated changes, vendors don’t like releasing “out of band” updates outside of their scheduled cycles.
Add to that the fact that people triaging vulnerability reports are typically not highly skilled security experts: while justifiably blocking invalid reports from overwhelming development teams, they sometimes also block valid vulnerability reports, which further delays the generation of an official vendor patch.
Our records show that our 0day patches become available 49 days before the original vendor updates for the same vulnerabilities. This is probably why about 40% of our customers are using 0patch on still-supported Windows systems.
Looking ahead, what innovations in vulnerability detection excite (or scare) you the most? What’s in your roadmap?
Ideally – even if this would mean the end of 0patch – vulnerabilities should be detected before a product is put in production. This is the pipe dream of vulnerability detection and has been so far been attempted with security training for developers, static and dynamic code analysis, secure code frameworks, security development lifecycle (SDL), penetration testing, formal methods, symbolic execution, and probably many other methods that I haven’t even heard of. But none of these have made a significant dent in the presence of critical vulnerabilities in software that supports all aspects of our lives.
These days, a lot of hope is being put into artificial intelligence as the solution that will finally eradicate vulnerabilities, but while it seems it may be capable of detecting some standard vulnerability patterns, it will be limited both by its cost per vulnerability (security is still part of the economic calculation) and by its ability to find issues that require a deeper understanding of unique processes and their context.
The bad news is that if AI’s capabilities in vulnerability detection are confirmed, the criminals will be willing to invest much more in it than software vendors because, unfortunately, they have much more to gain from finding vulnerabilities than vendors do.
As for our roadmap, we’re working on many improvements, a major one being extending our offering with “N-day” patches that will allow users of still-supported Windows versions to skip a couple of Windows updates or install them with a significant delay while enjoying our protection against the most-likely-to-be-exploited vulnerabilities.
How can our readers connect with you?
Website:
https://0patch.com
LinkedIn:
https://www.linkedin.com/in/mitjakolsek/
https://www.linkedin.com/company/0patch/
X: @0patch