diff --git a/docs/TTX/Tabletop-Exercise-Framework.md b/docs/TTX/Tabletop-Exercise-Framework.md new file mode 100644 index 0000000..d7c055b --- /dev/null +++ b/docs/TTX/Tabletop-Exercise-Framework.md @@ -0,0 +1,222 @@ +# Tabletop Exercise Framework # + +A Tabletop Exercise (TTX) is where an organization assembles subject matter experts to role-play through a mock incident that tests out the organization’s incident response plan (IRP) and gauges the ability of the org to react to plausible threats. The incident helps test the capabilities, processes, and policies of the organization and helps prepare all involved on how they should respond during a real event. The tabletop exercise should include subject matter experts from all critical organizations within the company (IT, operations, security, legal, HR, executives, etc.). +There are many benefits to conducting a TTX, including the following: +- Increasing organization preparedness for cybersecurity incidents +- Allowing the organization to perform a risk assessment of current processes and policies +- Improving communications and cross-team collaboration +- Training & skill development outside of an actual incident +- Policy testing & refinement +- Comply with Legal & Regulatory obligations +- Improve Incident documentation and processes + +## Tabletop Preparation ## +Tabletops simulate real threats that could manifest against the organization. To be successful, the TTX must be carefully thought-through, planned out, and include the critical stakeholders. The framework outlined below follows eight stages of the exercise from inception to after-actions. Each stage will be documented more in depth below. At a high level, the eight stages look like this: + +1. Define Objectives & Scenarios +2. Assemble the team +3. Develop the scenario & Establish ground rules +4. Create injects +5. Conduct exercise +6. Document the exercise +7. Debrief an evaluate +8. Post-Debrief Actions + +## Define Objectives & Scenarios ## +Before the TTX is announced and scheduled, it is important to set the objectives the organization seeks to achieve through the exercise’s activities. This can include such things as “improve our cyber incident preparedness”, “seek areas of automation”, or “demonstrate to our regulators we have an IR plan and regularly test it”. Key objectives, strategies and goals for the exercise should be documented so that as participants are engaged in the process, they all have a clear baseline understanding about the desired outcomes of the TTX. At this time the required subject matter experts are identified to ensure the appropriate people/teams/functions are participating in the exercise. Exercise logistics like length, systems in scope, and after-action steps to be taken are laid out. +The threat environment of the organization should be reviewed to understand the most likely threat actors and attack scenarios that could impact the organization so that a plausible attack scenario can later be developed with relevant incident injects that test out all areas of the organization’s preparedness. + +## Assemble the team ## +Next, the subject matter experts that need to be involved are contacted and briefed on the upcoming TTX. This initial meeting helps lay out all party’s Roles and Responsibilities within the organization and the system/application/process that is being tested. It is critical to have experienced and knowledgeable participants that have not only have the knowledge of operations and processes, but also can influence changes to those elements after the exercise. +As the team is assembled and briefed, documentation on the participant’s Roles & Responsibilities should be shared with all participants. The initial meeting allows the subject matter experts to ask questions about the logistics of the exercise (but not details about the scenario or injects). Everyone should come away with a clear understanding of their role in the exercise, the bounds of what is in or out of scope, and have clear expectations around their participation before, during, and after the exercise. +The Team’s participants will respond to the situations presented in the TTX based on their knowledge of the current Incident Response plan. They will draw on existing policies, procedures, and technology. Typical participants asked to be part of the TTX team include network/server administrators, help desk, IT support, CSIRT, PSIRT, managers, directors, legal, HR, business operations, corporate officers and executives. +There are five main types of participants: +- Subject Matter Experts (SME) - have an active role in discussing or performing their regular roles and responsibilities during the exercise. SMEs participate in the incident’s execution and discuss the scenario, and recommend best steps to take in response to the simulated emergency. +- Observers – these members will not directly participate in the exercise, but provide support to SMEs. They typically are actors outside the operations/incident response team that may be called in for expert opinions for relevant questions that arise during the mock incident. +- Facilitators – act as conductor of the incident, providing situation updates and moderating discussions. They also may provide additional information about the scenario, injects, possible external outcomes, and help resolve questions as required. +- Scribe – It is vital to have a 3rd party not participating in the event witness the TTX and take detailed notes. The Scribe helps track any Action Items that are assigned throughout the course of the exercise, capture discussions, and assist the Facilitator during the After-Action reviews. +- Sponsor – It is important that the TTX has a sponsor that helps set the organizational goals the TTX is seeking to explore and resolve. The Sponsor should assist in selecting a relevant scenario, participate as an Observer of the exercise, and ultimately is the party that receives the after-action and debriefing along with the TTX’s documentation. The Sponsor is the party that ultimately determines the effectiveness of the exercise and helps prioritize any improvement actions that follow. The TTX will typically begin with a kick-off call with the Sponsor to set the objectives and timelines for the event. + +## Develop the scenario & Establish ground rules ## +To have a productive exercise that resonates with the participants it is important to design the incident scenario so that they can see themselves in it. Having plausible scenarios and injects helps with the verisimilitude of the TTX and should as closely mimic real-world circumstances as possible. While it is an interesting thought exercise to “gamify” a meteor crashing into a building, the real-world benefits that could be derived from that are debatable. Walking through a scenario where the SMEs can envision as something that is part of their daily activities helps make the exercise more successful. +Thinking about historic events, or current events out of the news, allows the participants to make better connections (not having to suspend disbelief and “fight” the scenario) and ideally helps get a better map to how people would perform during an actual incident. The type of adversary/attacker and their capabilities should be determined to assist in planning out useful injects to deliver through out the event. +As the scenario is developed it is also important to establish ground rules of how the exercise will be conducted and how participants are asked to react during it. Documenting the communication protocols (especially to distinguish between “production” alerts and “exercise” ones) not only helps ensure both continued uninterrupted production environments, but makes the TTX flow more smoothly. Also properly time-boxing the event and the injects throughout help simulate the real-word pressures or “working on the clock” during an actual incident. +A good TTX scenario should have the following attributes: +- Be timely – be related to current threats that exist +- Be relevant – be related to threats the organization is concerned about or that have occurred to peers +- Be inclusive – the threat should span across multiple functions/organizations +- Be defined – have a clear beginning, middle, and end that all participants understand + +The scenario situation should be customized to support the exercise objectives. It should provide generic and qualitative descriptions of situations relevant to the overall exercise goal(s). The opening situation may be used as the context or starting point for Participants to identify major concerns and formulate their responses. Participants are welcome to share their internal concerns they hope to test out as part of the exercise, and the Facilitator and Scribe should pay careful attention to those as the TTX is documented and the after-action report is created. +Arrangements should be made for a shared physical/virtual space for all participants to join in. It is recommended that participants arrange to have their regular duties and meetings covered during the TTX so they can wholly be in the meeting and contribute to the session. As the scenario is being crafted and the meeting time arranged, now is a good time to create or collect any additional materials for the exercise. Artifacts like policies, procedures, presentation slides or a cheat sheet on terms or processes to help the participants understand their roles and the exercises as a whole. Additional context around the designated scenario may also be helpful and encourage a productive session together. +As the exercise is being created, the following areas should be created and/or considered: +- Create Pre-Exercise Briefing: +-- Provide participants with a briefing that outlines the objectives of the exercise, the scenario they will be facing, and any specific rules or constraints. This helps set the stage and provides context. +- Create Tabletop Exercise Facilitator's Guide: +-- Create a facilitator's guide to help guide the exercise. This should include an overview of the exercise flow, suggested discussion points, and guidance on how to handle specific scenarios. +- Create Communication Templates: +-- Provide templates for internal and external communication that would be used during a real incident. This includes notifications to employees, customers, and the public, as well as updates for the media. +- Determine Evaluation Criteria: +-- Define criteria for evaluating the participants' performance. This could include how well they follow the incident response plan, communication effectiveness, decision-making, and overall teamwork. + +## Create injects ## + +After the outline of the scenario is defined, just like a real incident, the situation will change as it runs its course. These alterations are an integral part of the inputs representing a new piece of information delivered at key times by the facilitator to expand the discussion. These changes to the scenario are referred to as “injects” that will periodically be released and shared with the TTX team that will provide new scenario parameters and information for them to respond to. The injects will describe some alteration of the parameters of the TTX that the participants need to analyze, discuss, react to, and collect additional evidence for. The Facilitator may inject different tactics or scenarios into a simulation to take the discussion in a different direction or unveil new simulated threats. Having access to internal processes, policies, and playbooks will structure how the team responds and what next steps they would take. +Ideally, the injects should be plausible and relevant to the event or are outcomes that logically would follow from the preceding stages. These injects attempt to simulate the “randomness”, stress, and pressures that will arise during an actual incident. Good injects will follow real world attacker’s Tactics, Tools, and Procedures (TTPs) and may not initially seem to have connection to the main event and be perceived as “red herrings” to some. Magical “McGuffins” or “deus ex machina” should be avoided. +During the preparation for the exercise, the complete attack should be mapped out to best present the injects to the team that provoke thought, conversation, and analysis of available capabilities and procedures. Defender’s reactions should be anticipated and countered/avoided, with alternate attack paths or tactics to be employed as the TTX progresses. + +### Good injects: ### +- Are clear and precise +- Build behind the scenes on actual attack scenarios/TTPs +- Require collaboration between multiple individuals/groups within the organization +- Exercise specific desired capabilities the organization should have + +The input presents a simulated, realistic cybersecurity situation that prompts participant discussion about the actions to be taken. There are many possible input types that could be considered when planning or executing a TTX. The scenario and corresponding inputs should depend on the types of incidents that might affect your organization. These injects simulate the evolving nature of a cyber incident. Below is a list of possible inject types that could be leveraged: +1. Incident Reports: +- Provide participants with simulated incident reports detailing the discovery of a potential cybersecurity threat or breach. Include information such as the type of attack, affected systems, and initial analysis. +2. Risk/Threat Assessments and contingency plan documentation: +- Can provide authentic data into threats/vulnerabilities that the organization has reviewed previously and their remediations/mitigations that could prove useful to test. +3. Internal Data, documentation, and systems output: +- Adding verisimilitude to the exercise, using internal data and reports are a way to potentially introduce conflicting or misleading information into the exercise to see how the participants cope. +4. Social Media/News Articles: +- Create mock news articles reporting on the cyber incident. These articles can include information that may or may not be accurate, reflecting the way news spreads during a real incident. +5. Emails and Alerts: +- Simulate the receipt of emails, alerts, or notifications that would be generated during an actual incident. These could come from internal monitoring systems, external security partners, or affected third parties. +6. Social Media Updates: +- Mimic the dissemination of information on social media platforms. Create posts or updates that reflect how the incident might be discussed externally and the potential impact on the organization's reputation. +7. Regulatory Notifications: +- Introduce mock notifications from regulatory bodies or authorities informing the organization of potential legal and compliance implications. This inject can prompt discussions about legal obligations and reporting requirements. +8. Customer Complaints: +- Simulate customer complaints related to service disruptions or data breaches. This inject can highlight the importance of customer communication and reputation management. +9. Technical Data and Logs: +- Provide participants with additional technical data, logs, or forensic analysis results. These injects can help simulate the evolving understanding of the incident and guide the technical response. +10. Third-Party Involvement: +- Introduce injects related to third-party involvement, such as reports from external security vendors, law enforcement, or incident response teams. This adds complexity to the scenario and requires collaboration with external entities. +11. Changes in the Threat Landscape: +- Describe changes in the threat landscape during the exercise. This could involve new tactics, techniques, or procedures used by the attackers, forcing participants to adapt their response strategies. +12. Supply Chain Impact: +- Simulate the impact on the supply chain, with injects related to disruptions in the delivery of goods or services due to the cyber incident. This inject can broaden the scope of the exercise. +13. Escalation of the Incident: +- Introduce injects that escalate the severity or complexity of the incident over time. This could include the compromise of additional systems or the emergence of new attack vectors. +14. Executive Communications: +- Provide participants with injects related to communication challenges with executives or board members. This could involve requests for specific information, updates, or decisions from leadership. +15. Legal and Privacy Concerns: +- Introduce injects related to legal and privacy considerations, such as the discovery of sensitive data being compromised or potential legal actions against the organization. +16. Resource Constraints: +- Simulate resource constraints, such as shortages of staff, tools, or budget, to test participants' ability to prioritize and make decisions under pressure. +17. Mistaken Identities and False Positives: +- Include injects that introduce confusion or uncertainty, such as mistaken identities of attackers or false positive alerts. This challenges participants to validate information and make informed decisions. + +These injects should be carefully designed to align with the exercise objectives and provide participants with realistic challenges that reflect the organization's specific threat landscape and vulnerabilities. The injects and scenario updates will be introduced at specific times during the exercise. These injects could include new information about the incident, changes in the environment, or additional challenges for participants to address. + +## Conduct the exercise ## +On the day of the exercise, the team should be assembled with introductions made for roles & responsibilities, so everyone understands who is present and the areas they represent. Make sure all materials are ready for the participants. Ensure all functions are included, such as legal, PR/comms, HR, operations, etc. since real-world incidents rarely contain themselves to one team or technology. +After introductions, the Facilitator will provide any background information to the group and will explain the scenario and its parameters and provide the initial injection to start the event. The team will be given a set amount of time to respond to each injection, assemble any required documentation or evidence, until the next scenario inject will be delivered. Typically, each stage in the TTX will have three core phases: +1. Inject is delivered +2. Team processes the inject and reacts to it +3. Team delivers outputs – documentation, evidence of activity, or clear statement of action taken + +This will be repeated multiple times throughout the TTX until the agreed-upon scenario is complete. Once the scenario is presented, the Facilitator will monitor and direct the discussion among the TTX participants. +As each inject is revealed to the participants, the following steps will be taken, and each team member will talk about actions they would take. Ideally through this dialog gaps in people, process, and technology are identified and captured by the Facilitator and Scribe. The Facilitator helps direct the team as these steps are worked through and discussed: +1. Status is reviewed. +2. Working assumptions discussed and confirmed/debunked based on current knowledge. +3. Organization and security concerns and implications are identified and discussed. +4. Next steps and actions outlined. +5. Required resources reviewed (people, process, technology). +6. Recommendations developed. +7. Implement agreed actions and changes reacting to the inject. +8. Repeat with each Inject until the exercise has resolved and is concluded. + +There will be times when participants will not have answers or solutions at each stage. This is perfectly acceptable, as it highlights gaps (in training, documentation, expertise) that can be addressed as improvement outcomes of the TTX. Having the discussions around the scenario should help bring unclear, undocumented, or areas where there is clear lack of ownership or experiences to the forefront so they can be developed before an actual incident occurs. +The Facilitator should keep the exercise on track by regularly checking in with each team/participant and making sure that everyone understands and is comfortable with their roles. During the exercise, ensure notes are being taken for key decisions that are being made so they can be discussed later. Ensure that the team is assessing the strengths and weaknesses of everyone’s responses throughout the TTX in order to learn from mistakes and identify processes that went well. + +## Outputs ## +Throughout the course of the TTX artifacts and other deliverables will be used as reference to guide how the exercises activities would progress. These along with any documentation created as part of the exercise forming should be collected as part of the exercise’s deliverables to the sponsors. These artifacts ultimately will help document any process improvements that are needed as a result of the findings of the TTX. Ideally these also help shape or improve playbooks the organization will use in the event of an actual cybersecurity incident. +There are many assorted types of artifacts that will become Output of the TTX: +- Detailed description of the Scenario and Injects and any contextual handouts. +- Detailed notes of the discussions and resolutions at each stage of the event (paying particular attention to any gaps in communication of process) including improvements to automated alerts/monitoring, knowledge sharing, and incident documentation requirements. +- Participant feedback/reflections/suggestions for improvement as well as their individual assessments of the organization readiness based on the TTX experience. +- Improvement suggestions to incident communications, documentation, leadership briefing, and any external reporting required artifacts. + +## Document the exercise ## +The TTX is designed to be a time-boxed time of collaboration and communication. It is important that the Scribe takes detailed notes of conversations and decisions, as these may well be informative to have captured for the time that an actual crisis occurs in the future. After the exercise has concluded, all participants should be polled and asked for feedback for process or future exercise improvements. The Scribe and Facilitator will be responsible for creating an After-Action Report (AAR) that summarizes the drill, documents the exercise, key decisions and discussions, key findings from the exercise, and a list of after-action tasks to improve the organization’s readiness for future cyber events. This detailed report should be created, along with an executive summery presentation that provides the key learnings of the exercise. This will be shared at the Debrief session, where all participants are invited back to review the findings and discuss their experiences and share feedback. + +The scope of the Debriefing meeting may include more individuals or teams than originally participated in the TTX (especially consider that now management and senior leaders will be curious about the results of the exercise). As the Debriefing session is being scheduled and planned, it is important that the materials reflect the Objectives of the Exercise, they provide details about the Scenario and injects, documentation about discussions and decisions made, and any notable observations the team made throughout the course of the exercise. + +## Debrief and evaluate ## +Conducting a thorough debriefing is a critical component of a cybersecurity tabletop exercise. The debriefing provides an opportunity to discuss the exercise, identify strengths and weaknesses, and gather insights for improvement. +After the AAR has been assembled and is ready to be shared with the Sponsor and TTX participants, a Debrief session should be scheduled. This helps ensure all stakeholders are on the same page about what happened during the TTX and allow the team to think about how best to address future cybersecurity threats. The AAR and Debrief also allow the organization the opportunity to track progress over time, and the Output documentation collected can serve as a resource for the future when respond to any incidents. +It is important for the team to review key decisions that were made and to discuss what improvements might be made to improve organizational readiness. The Debrief meeting also allows the opportunity to highlight and recognize team members, and their efforts and also collect feedback on how future tabletops can be improved. + +## Conducting the Debriefing: ## +1. Introduce the Debriefing: +- Begin the debriefing by outlining its purpose and emphasizing the importance of open and constructive communication. Set the tone for a collaborative and learning-oriented discussion. The Debrief should be a safe space where all feedback, both positive and negative, should be freely shared. +2. Review Exercise Objectives: +- Remind participants of the initial objectives set for the exercise. Discuss whether these objectives were met and if there were any unexpected outcomes. +3. Discuss Injects and Scenarios: +- Go through each inject or scenario introduced during the exercise. Discuss how well participants responded, whether they identified the key issues, and if there were any challenges in understanding or managing the simulated incidents. +4. Explore Decision-Making: +- Examine the decision-making process during the exercise. Discuss how participants communicated, collaborated, and made critical choices. Identify areas where decision-making could be improved or where there were notable successes. +5. Assess Communication Effectiveness: +6. Evaluate the effectiveness of communication, both within the organization and with external parties. +- Discuss how well information was shared, whether there were delays or misunderstandings, and how communication strategies could be enhanced. +7. Identify Strengths and Weaknesses: +8. Encourage participants to share their perspectives on what worked well and where there were challenges. +- Identify strengths and weaknesses in the incident response plan, team coordination, and individual roles. +9. Discuss Resource Utilization: +10. Evaluate how well resources, both human and technical, were utilized during the exercise. +- Discuss any resource constraints and identify opportunities for improvement in resource allocation. +11. Capture Lessons Learned: +12. Encourage participants to share their individual and collective lessons learned. +- Document these insights for future reference and improvement. +- Focus on actionable takeaways that can be applied to enhance cybersecurity readiness. +13. Review Documentation: +14. Refer to the documentation created during the exercise, such as incident reports, emails, and logs. +- Discuss the accuracy and completeness of the information gathered and recorded. +15. Generate Improvement Recommendations: +- Based on the discussion, collaboratively generate recommendations for improvement. +- These recommendations may include updates to the incident response plan, additional training, changes in communication protocols, or enhancements to technical capabilities. +16. Share the After-Action Report (AAR) with stakeholders: +17. Summarize the key findings, lessons learned, and improvement recommendations in an after-action report. +- Distribute the AAR to relevant stakeholders and leadership for further review and action. +- +If multiple improvements are discovered, asking the participants and Sponsor for feedback on how they might be prioritized and scheduled for completion helps keep the collaboration and teamwork going after the exercise and the Debriefing have concluded. + +## Post-Debrief Actions ## +The Debriefing should yield a list of observations and actions the organization desires to implement. It is critical that all the work of the TTX not be lost and that an action plan is developed, and follow-up on periodically to ensure that the organization learns from the lessons garnered from the exercise. The Facilitator and Scribe should assemble the desired improvements and work with leadership to schedule and prioritize any findings/learnings. The following four areas should be considered as the organization follow-through after the exercise has concluded: +1. Implement Changes: +- Act on the improvement recommendations by updating policies, procedures, and plans. Implement changes to address identified weaknesses and enhance the organization's cybersecurity posture. +2. Training and Awareness: +- Develop and deliver training sessions based on the lessons learned. Focus on building awareness, improving skills, and reinforcing best practices identified during the tabletop exercise. +3. Schedule Follow-Up Exercises: +- Plan for follow-up tabletop exercises to continue testing and refining the incident response plan. Use the insights gained from the debriefing to design more targeted and challenging scenarios. +4. Continuous Improvement: +- Establish a culture of continuous improvement by regularly reviewing and updating the incident response plan, conducting periodic exercises, and incorporating feedback from real incidents and simulations. +By following this debriefing process, organizations can maximize the value of cybersecurity tabletop exercises, turning them into valuable learning experiences and driving continuous improvement in cybersecurity readiness. + +## Tabletop Exercise Best Practices ## +- Set Clear Objectives +- Realistic Scenario +- Diverse Participants +- Skilled Facilitator +- Preparation +- Timed and realistic responses +- Documentation +- After-action review +- Iterative improvement +- Regular Candence +- External expertise +- Legal & ethical consideration +- Confidentiality + +## References ## +[CISA Exercise Overview template](https://www.cisa.gov/resources-tools/resources/cybersecurity-scenarios) +[REMS.ED Cybersecurity Tabletop](https://rems.ed.gov/docs/CybersecurityTabletop_508C.pdf) +[CSO Online How to conduct a Tabletop](https://www.csoonline.com/article/555131/how-to-conduct-a-tabletop-exercise.html ) + +## Playbook example ## +(forthcoming) +- Report issue +- Triage data available +- Understand risks involved with issue (financial, infrastructure (intel ops), reputation, market) +