In the industry’s first test to closely emulate real world conditions, ShiftLeft subjected its runtime protection capabilities to a 14-day penetration test from Cobalt.io and is now publishing the results.
The test deployed two identical applications. Both applications had the same vulnerabilities. One instance of the application was unprotected and the other was protected by ShiftLeft.
In order to accurately test ShiftLeft’s runtime protection capabilities, we created three teams that were isolated from each other: development, security, and penetration testing.
DevelopmentThe development team built a Java application with vulnerabilities in it representing the OWASP Top 10.
SecurityThe Security team relied solely on ShiftLeft’s ability to identify vulnerabilities in source code and protect them in runtime.
Penetration TestingCobalt.io performed the penetration testing.
The test application is a simple REST-based multi-tenant application emulating the functions of a retail-banking interface, including routes. The application was built with examples of six (6) of the relevant OWASP Top 10 vulnerabilities embedded into it. Hence, if the penetration testing team were able to breach runtime protection, the application would be exploitable. In order to make the application more vulnerable, it was built without support for any authorization or authentication schemes across tenants. All endpoints specified above can be exercised by any user. Hence, cross-site request forgery (CSRF) and the resulting cross-site scripting (XSS) were inherently out of scope.
To ensure that the pen test was unbiased and thorough, ShiftLeft sought the expertise of a third-party pen testing solution. After evaluating the top approaches and vendors in the market, ShiftLeft choose Cobalt.io— a Pen Testing as a Service Platform—for its superior expertise, technology-driven platform, and efficient workflows. Additional benefits included:
Cobalt.io tested both the protected and unprotected versions of the application over a 14-day period.