Adversary simulation, emulation or purple teaming - How would you define it?

AI Generated Summary

This panel discussion from Adversary Village at DefCon USA 2021 explores the definitions, maturity requirements, business justification, and career paths for adversary simulation, emulation, and purple teaming.

Panelists

• Jean-Marie Bobon: Team leader, offensive security service, Post Luxembourg (formerly red teamer/penetration tester in France/Luxembourg)
• Joe Vest: 20 years in security/IT, last 10 years in offensive security testing, Technical Director for Cobalt Strike at Health Systems
• Vincent U: Red teamer for many years (UK → Asia), runs red team in Hong Kong, performs attack simulations
• Martin Lingus: Founder/Principal Consultant, Covert (Norway), penetration testing, red teaming, adversary simulation (5+ years)
• Samuel Kimmins: Red teamer at Cognizant, 10 years IT/security experience, US Air Force, Recon Infosec (100% adversary emulation focus)

Key Topics Discussed

Defining Adversary Simulation, Emulation, and Purple Teaming:

Jean-Marie’s Perspective:

• Emulation: Goal is to emulate TTPs, reproduce well-known campaigns
• Simulation: More dynamic - define flags/scenarios with customer, execute scenario, don’t provide TTPs upfront, adapt to what you encounter
• Red Teaming: More complex - social engineering, phishing campaigns, internal tests, multiple approaches
• Market reality: Many companies calling it “red teaming” are actually doing internal penetration tests (Active Directory focus)
• Depends on market/region - definitions vary

Joe’s Perspective:

• Key insight: “It doesn’t really matter what you call these” - definitions come with baggage
• Better approach: Work backwards from goals, not forward from definitions
• Goal: Measuring security operations ability to impact a threat
• Unified category: “Threat Testing” - different flavors are just definitions of the day
• Important: Come to answer before starting - what are you trying to measure?
• Red teaming misconception: Not about “covert testing” or “secret scroll things” - that’s penetration testing
• Penetration testing: Map out attacks, identify flaws - well defined
• If doing pen test in red team engagement: “You’re just doing a pen test, you just put a different name on there”
• Start with goals: What are we trying to measure? Security operations ability (people, processes, technology)
• Purple teaming: Hand-holding between red and blue through engagement

Vincent’s Perspective:

• Terminology evolution: Used to call it “red teaming”, then “adversary simulation” came along (better buzzwords/marketing terms)
• Adversary Simulation: Understanding the adversary, achieving same goals they’re trying to achieve, see how well organization can defend
• Focus: Understanding end goals (money, making something happen), not just tactics
• Shortest path: Don’t need to go through all hurdles of unnecessary attacks/tactics
• MITRE ATT&CK impact: Started mapping every tactic/attack, put attackers on heat map → pushed towards adversary emulation
• Adversary Emulation: Automated tooling running certain TTPs over and over, test defenses/detections
• Purple Teaming: Adversary simulation-ish hand-holding with blue team, or proper use case/detection case testing
• Future term: “Threat simulation or emulation” probably better description

Martin’s Perspective:

• Computer science definitions: Simulation = similar but not exactly same, Emulation = exactly the same
• Emulation: Replaying TTPs or IOCs
• Problem: Customers don’t know what any of this is
• Reality: “Everyone’s doing red teaming” but often just pen test
• Challenge: Defining it with customer - what are you after? What do you want to get out of this exercise?

Samuel’s Perspective:

• Adversary Simulation: Traditional red teaming - simulate goals of threat actor (not specific threat actor), adversary that might be targeting organization
• Adversary Emulation: Focus on emulating TTPs based on threat intelligence
• Key driver: Threat intelligence is key component to successful adversary emulation
• Client confusion: Customers don’t understand difference, see buzzwords, think they need this test
• Maturity requirement: Adversary emulation is high maturity scale - organization should be ready
• High end: Replicating malware from threat actors
• More needed: Covering standard detections regardless of what threat may be using
• Purple Teaming: Act between red and blue, offensive team + defensive team
• Flexibility: Emulation or simulation can fit into purple teaming bucket

Common Threads:

• All terms have place in the flow
• Clients not aware of what they want
• Everyone aware of buzzwords but don’t understand them
• Similar to bug bounty trend - everyone wants it without understanding

Maturity Requirements:

Samuel’s View:

• Need to be at higher end of SOC maturity for successful red team engagement
• Need SOC, detections, logs flowing to central location for threat hunting
• Problem: Low maturity + second-rate red team = “destroy that organization, thanks for the money, see you next time”
• Solution: If low maturity, hire consulting firm to build defenses first (if can afford)
• Alternative: Don’t go straight into red team, build up first

Joe’s View:

• Tabletop exercises: Work backwards - “I want red team engagement” → “What do you mean?” → Walk through threat scenario
• Three pillars: Prevention, Detection, Response (world focuses on prevention)
• Education piece: Walk through threat, decompose into main elements, find gaps immediately
• For everyone: Not just high maturity - up to consultant to pull information out, educate them
• Key: Bring them up to speed so decisions improve security

Jean-Marie’s View:

• Response phase critical: What happens if something happens in real life? If don’t know how to react, it’s too late
• Include response: Validate processes are efficient, team ready to react
• Underrated: Big focus on detection/use case development, but if see something and don’t know how to react = problem
• Ransomware example: Human-operated campaign with ransomware = few minutes to react correctly, otherwise lose everything
• Stress factor: If everyone aware of ongoing test, reaction totally different than real life attack
• Teach people: How to react, develop processes, be ready, know what to do even under stress

Martin’s View:

• Consultant responsibility: Make sure client can receive and integrate results
• Problem: If going to “steamroll this company” and they won’t understand how you did it, starting at wrong layer
• Important: Client understands what they want, we understand how they’re able to receive it

Vincent’s View:

• Two sides: Worked with large mature clients (centralized logging, blue teamers who know what they’re doing) AND small clients (few things in place, trying to piece together)
• Don’t wait: “Unless you start doing these assessments, how do you figure out exactly how much can you see?”
• Example: Incident response engagement against Lazarus group - if they did simulation instead of getting hacked, would have been better prepared
• Reality: “People saying you need certain maturity before doing anything → they got hacked → that’s how they learned where gaps are”
• Key insight: “Don’t wait until it’s too late - give it a shot now, try these engagements, see what you can learn, improve security posture”
• Not required: Don’t have to have all centralized logging in place first - performing engagements is where you identify gaps

Getting Budget/Investment:

Jean-Marie’s Approach:

• Argument: “In 2021, we cannot deal with security like 10 years ago” - threat evolves, we must evolve
• Value proposition: Teammates have dedicated R&D time, tools, bypasses ready
• Customer pays for mission, not preparation: Pay only for exercise, not setup/preparation
• Cost comparison: Adversary simulation exercise ≈ same cost as internal pentest, but very different results
• Coverage: More than basic IT security - validate process, detection, reaction from people

Joe’s Approach:

• Not technical problem, education problem: Must pull leadership into conversations to understand threat
• Security industry lacks threat perspective: Headlines boil down to “hacked” - attack steps lost
• Problem: “Find flaw, fix flaw” philosophy - can’t patch way out, but that’s what we focus on
• Three pillars: Protection, Detection, Response (world focuses on prevention)
• Shift mindset: “We are not preventing attacks - we will always fail if just prevent/prevent/prevent”
• Goal: “Impact a threat’s ability to be successful” - sometimes door gets kicked down, but detect and respond before ransomware deployed or IP stolen
• Language: Can’t use buzzwords or technical terms - leadership has their own playground, need right toys/message
• Key: “We don’t have our own place to play - we’re always invited to someone else’s playground”

Vincent’s Approach:

• Cost comparison: Percentage of revenue spent on security vs. ransomware payment
• Math: Can do 10 adversary simulation exercises over years, still better off than getting hit by ransomware once
• Cost: Only like 1.5-2 full-time employees worth of cost, but getting specialized resources doing this full-time
• Show impact: Show threat to CEO - “You’ve got access to inbox” or “Calling them on phone telling them you’ve been hacked”
• Attention: Find phone number from company database, call CEO → “If I get hacked today, someone could be calling me or blackmailing me”
• Simple fast red team exercise: Good way to get attention of leadership

Martin’s Approach:

• Fire drill analogy: Wait for real fire (hope you’re doing well) OR do drill, test if procedures are holding up
• Less technical: Testing procedures, preventing, detecting, responding to threat (like environmental threat, but technical threat)
• Reality: Usually 1.5 people or small team doing technical side, company created to earn money, not be secure
• Important: Bring higher-ups into conversation when defining what we’re delivering
• Follow-up costs: IT department will need more money/time after test - need to buy products, hire people
• Key: Higher-ups understand outcome of test, they probably need to follow up with more money/time

Samuel’s Approach:

• Initial assessment: Show weak spots (traditional pen testing manner)
• Correlate with threats: Correlate vulnerabilities with known threats - “These known threat actors target your industry, deploy ransomware, cost tens of millions”
• Bring threat to forefront: Into first initial conversation, show value is there
• Investment: In security (defense) or expanding security operations to include offensive security
• Continuous testing: Make testing continuous

Career Guidance - Getting Into Adversary Simulation/Emulation:

Samuel’s Recommendations:

• Threat intelligence analysis: Need to analyze threat intelligence reports, pick out key potential attacks from techniques (emulation bucket)
• Red teaming skills: Traditional red teaming skills (simulation)
• Combine: Red teaming skills + threat research skills = well on way to employment opportunities

Martin’s Recommendations:

• Don’t start directly: Probably don’t want to start directly doing adversarial simulation/emulation
• IT background: Good backup background in IT in general - good starting point
• Historical path: People used to start with network/server administration before security (security was natural part)
• Void: Being part of red team contains many different types of skills
• Foundation: Good understanding of IT, infrastructure, networking in general = number one
• Skills needed: Research, integrate threat intel, analyze threats, understand not just IOCs/TTPs but integrate them, do same things adversaries are doing

Vincent’s Recommendations:

• Backgrounds that work: CIS admin backgrounds with cybersecurity passion
• Pen testing: Can be easily picked up (few courses, dedication)
• Emulation levels: Varying levels of sophistication - don’t always need top-tier pen testing knowledge
• Example: Conti ransomware gang using AnyDesk - if can install AnyDesk in organization, see if allowed out, control machine without being detected = simulating specific TTP layer
• Blue team side: SOC doing detection engineering - worth having someone dedicate time looking for intelligence, picking out test cases to simulate, test detection cases
• Recent graduate: Can start looking at threat intel reports, using MITRE ATT&CK, start working on emulating TTPs
• Starting point: MITRE ATT&CK resources - fantastic place to start if in blue team, don’t know lot about red teaming

Joe’s Recommendations:

• Red teaming is subset of blue: Red cannot exist without blue, but blue can exist without red
• Understanding defenders: Only reason red exists is to understand edges defenders have
• Detection story: Need to understand what you’re doing, why detecting this, why spending money on tools
• Fundamental understanding: TTPs, IOCs, threat actor techniques
• Map attacks: Map any attacks (simple or complex) back to detection and response opportunity
• Core principle: Understanding technology as core - hacking is using technology already designed
• No magic: Use same fundamental principles, same APIs, same tools
• Difference: Maybe intent (between adversary and regular software developer)
• Foundation: Understanding those principles = foundation of everything

Jean-Marie’s Recommendations:

• No one path: Not one option, one career, one path that is true/wrong
• Defensive background necessary: Coming from defensive parties is necessary
• Blue mindset helpful: In simulation, provide use cases improvement, not only technical remediation but process enhancement
• Offensive mindset: Also need offensive mindset
• Pen tester perspective: Not talking about “monkeys that run nmap” - people who know how to exploit weakness, bypass mitigation, get initial access
• Example: Phishing campaign with MFA - should be able to get shell, if not, level up before switching to adversary simulation
• Prerequisite: Once know how to bypass mitigation and act like real threat actor, can pretend to simulate it
• Backgrounds: Can come from OSINT, networks
• Problem: Saw tons of pen testers that don’t know how infrastructure works - not possible
• Understand infrastructure: How financial company works, what internal infrastructure is, how it communicates
• Don’t copy-paste: Don’t just copy-paste from MITRE ATT&CK - makes no sense if don’t know what Pass the Hash is, how it works in reality
• Key: “You also have to be a threat actor to simulate threat actor”

Key Insights:

• Definitions don’t matter as much as goals - start with what you’re trying to measure
• Clients often don’t know what they want - education is key
• Maturity requirements debated - some say need high maturity, others say start early to identify gaps
• Budget justification requires threat perspective, not just technical terms
• Career paths vary - IT background helpful, threat intel important, understand both red and blue sides
• Response often underrated - detection is useless if don’t know how to respond
• Red teaming is subset of blue teaming - red cannot exist without blue

Important Concepts:

• Threat Testing: Unified category Joe suggests
• Three Pillars: Prevention, Detection, Response (world focuses on prevention)
• Tabletop Exercises: Good starting point for low maturity organizations
• Fire Drill Analogy: Test procedures before real incident
• MITRE ATT&CK: Key resource for understanding TTPs
• Cost Comparison: 10 exercises over years < one ransomware incident

Actionable Takeaways:

1. Start with goals, not definitions - what are you trying to measure?
2. Educate clients/leadership on threat perspective, not just technical terms
3. Include response phase in engagements - detection useless without response
4. Don’t wait for perfect maturity - start early to identify gaps
5. Use cost comparison (exercises vs. ransomware) for budget justification
6. For career: IT background + threat intel + understand both red and blue
7. Understand infrastructure before trying to attack it
8. Map attacks back to detection and response opportunities
9. Purple teaming can incorporate both simulation and emulation
10. Focus on impact threat’s ability to be successful, not preventing all attacks