Abstract: Maintaining cyber resilience at a time when cyber threats constantly evolve is becoming increasingly complex. This blog post explores ways of conducting custom offensive security tests, highlighting the role of technologies like Breach and Attack Simulation (BAS) and automated red teaming. It addresses the inherent risks and challenges of custom-built offensive assessments in production environments and showcases how advanced technologies provide robust and user-friendly solutions.
Table of Contents
The Risks of Running Custom Offensive Testing in Production Environments
Complexity of Creating Attack Scenarios
Today’s threat landscape demands a constant state of cyber readiness. Cyber teams can no longer wait around for annual or semi-annual pen tests. Thanks to technologies like breach and attack simulation (BAS) and automated red teaming, enterprise security teams now have the tools to continuously validate their controls against new and emergent threats. While most controls can be tested with default, out-of-the-box templates, custom offensive testing is still required for many systems and deployments, but this does not have to mean manual assessments. Solutions like BAS and automated red teaming provide an arsenal of knowledge and automation to take custom offensive testing to the next level.
The Risks of Running Custom Offensive Testing in Production Environments
When run in production environments, customized offensive testing carries intrinsic risks. These risks should not prevent running assessments as long as they are meticulously planned and executed with a focus on safety. Testing within non-production environments to understand the potential impact is critical, allowing cybersecurity teams to plan how, where, and when custom offensive tests can be most effectively – and most safely – run in production counterparts.
The risks of custom offensive testing include:
- Unintentional Disruption of Critical Services: Running intrusive exploit testing directly on live production systems risks crashing critical business services and preventing access for real users.
For example, many known exploits can cause applications to crash or behave unexpectedly, leading to downtime and potential data loss. - Data Exposure and Potential Breaches: Attempting to extract or modify real production data for testing purposes may lead to sensitive data being accessed or modified unintentionally. For example, an assessment attempting to exfiltrate real customers’ data to test DLP controls could inadvertently cause an actual data breach if the target the data is sent to is outside of the control of the organization.
- Overloading of Production Network Infrastructure: Flooding production systems with high volumes of tests and attack traffic can degrade performance for real users by overutilizing shared compute resources.
For example, simulation of lateral movement without proper restrictions can result in massive amounts of traffic on the network – slowing services and blocking access to resources. - Triggering Intrusion Detection or Prevention Systems (IDS/IPS): Highly intrusive testing often generates excessive false positive alerts in IDS/IPS solutions, undermining SOC efficiency and creating alert fatigue.
For example, simulation of threat activity performed at scale across large numbers of endpoints can produce a flood of alert activity that is legitimate but doesn’t indicate an actual attack.
Offensive security testing tools like BAS and automated red teaming can help overcome these safety challenges. By allowing for targeted testing to limit system impact, providing known exfiltration targets, placing configurable limits on operations, and being clearly traceable for easier suppression of alerts, these tools reduce the potential for encountering many common concerns.
Overcoming the Challenges of Creating Custom Offensive Security Testing with Limited Red Teaming Resources
While providing immense value, custom offensive testing remains demanding for most organizations to achieve due to key challenges from resource constraints to attack complexity to repeatability.
Time and Resource Constraints
Creating new custom offensive security tests safely requires expertise across many specialized domains. Teams must invest in in-depth research and planning to first map out relevant threat activities and then design realistic attack scenarios with in-depth knowledge of each relevant attack technique and procedure. Moreover, coding each individual execution requires mastering the manual writing of scripts and compiling of binaries. At the same time, shortcuts involving reliance on third-party sources introduce additional risks, including compromised code.
Benefits of custom offensive testing solutions
Instead of relying on time-consuming and resource-heavy manual template creation, solutions such as BAS and automated red teaming can facilitate the automation and customization of offensive testing. Such validation tools should include:
- Pre-built individual executions – Libraries of offensive security test cases, payloads, and tools to provide an extensive starting point to customize assessments.
- Libraries of pre-built templates – A repository of ready-to-run attacks that can be quickly deployed or modified.
- Customizable executions and templates – A built-in system to facilitate customization of provided executions and templates with the ability to create net-new objects at will.
Complexity of Creating Attack Scenarios
Designing realistic composite attack scenarios is extremely challenging without deep knowledge across several complex domains including:
- Methodologies, tools, processes, and technologies used by attackers.
- Network topologies, configurations, and interconnections.
- Aggregation and application of the latest threat intel.
Benefits of custom offensive testing solutions
To overcome the complexity barriers of creating sophisticated attack scenarios, an optimal custom offensive testing solution provides diverse script and command libraries and custom code integration to develop fully customized payloads and attack package plugins tuned at will.
- Advanced offensive security testing automation with tools like BAS and automated red teaming.
- Extensive inventories of scripts, binaries, plugins, and tools to use as modular building blocks.
- Custom code integration enriches libraries with organization-specific code, SDKs, and APIs.
Repeatability and Consistency
Standardized execution ensures assessment fidelity pre and post-remediation by eliminating reliance on manual processes prone to degradation over time and provides reliable repeatability across environment.
- Reliable repeatability across environments.
- Up-to-date threat repository, including immediate threats.
- Consistency in applying post-remediation assessment.
Benefits of custom offensive testing solutions
To enable consistent, repeatable custom offensive testing, solutions should provide:
- Automation – The ability to create and save customized attack scenarios and rerun them at will through a single platform to enable frequent and consistent testing.
- Execution Chaining – The option to chain executions across pre-built and custom-made objects enables the easy combination of different techniques for comprehensive testing.
- API Integration – The ability to use existing scheduling and management tools to run assessments, report results, and schedule follow-up actions.
Making Findings Actionable
Turning findings into actions requires thorough documentation and clear remediation guidance. This means that security testers must spend hours documenting and supporting each finding with details, such as:
- Recommended remediation guidance.
- Alternative remediation options if something blocks the recommended methodology.
- Evidence and test logs that prove control failures or weaknesses along with successes.
- Mapping findings to frameworks like MITRE ATT&CK, ISO 27001, and NIST 800-53
Benefits of custom offensive testing solution
As manual documentation processes routinely fail to reliably capture key details, switching to automated report creation enables:
- Executive reporting: Easy-to-customize executive reports designed to match stakeholders’ priorities and provide easily accessible data.
- Technical reporting: Comprehensive reports that include a description of each action performed during the simulation, in-depth analysis of the findings, and actionable remediation guidance.
- API access to findings: The ability to ingest findings and other data into existing automation and management tools.
Technologies such as BAS and automated red teaming are instrumental in overcoming the challenges of custom offensive security testing. They facilitate the creation of complex, relevant, custom-built attack scenarios, enhancing attack simulations without the need for additional resources or manpower. These technologies mark a significant advancement in cybersecurity, offering an efficient, safe, and effective approach to maintaining cyber readiness in a constantly evolving threat environment.