Industries · Cybersecurity

Post-training and RL environments for defensive cybersecurity agents.

Defensive security work is sequential decision-making under partial observability: observe signals, investigate, act through tools, and absorb the consequences. That structure is exactly what agentic reinforcement learning trains, which makes defensive cybersecurity one of the strongest fits for bespoke post-training.

Capability targets

Programs train scoped defensive behaviors that can be exercised and scored inside controlled environments, not a general security oracle.

Defensive monitoring and alert triage
Incident investigation across logs, hosts, and network telemetry
Containment decisions with explicit action costs
Incident-response assistance under playbooks and policy
Vulnerability triage and prioritization
Tool selection and valid tool use across a security stack
Long-horizon recovery: multi-step remediation with delayed feedback

Environment architecture

The training environment simulates infrastructure under incident: hosts, services, identities, and telemetry, with a fault or attacker progression the agent cannot directly observe.

Network and system state as hidden world state
Partial observability: the agent sees telemetry, not ground truth
Tool interfaces mirroring real defensive stacks (queries, isolation, forensics)
Scripted incident progressions with controlled stochasticity
Action costs for disruptive interventions
Long episode horizons with reset and deterministic replay

Training signal

Defensive outcomes are unusually verifiable: the environment knows the ground-truth incident state, so rewards can be programmatic rather than judged.

Successful containment and eradication, verified against hidden state
Incident progression halted early versus late
Penalties for unnecessary intervention and business disruption
Penalties for invalid or unsafe tool use
Playbook and policy compliance checks
Time and cost pressure through step penalties
Hard penalties for catastrophic states (data loss, missed exfiltration)

Data and supervision

Supervised warm starts come from material most security organizations already produce, prepared and filtered into training form.

Simulator traces from scripted and scripted-adversary scenarios
Expert demonstrations of investigation and response
Incident-response procedures and playbooks as structured supervision
Redacted historical logs and case notes where governance allows
Synthetic scenarios with verified labels
Hard negatives mined from the policy's own failed episodes

A plausible training sequence

Stages follow the standard program logic: demonstrate, prefer, then optimize online where the environment can score outcomes.

SFT on valid defensive actions, tool syntax, and investigation structure
Preference optimization on better-versus-worse response comparisons
Online RL (GRPO or related methods) inside the incident environment
Held-out scenario evaluation against unseen incident types

Evaluation

Release evaluation reports distributions, not averages alone, because tail behavior is the operational risk in security.

Mean outcome and variance across held-out scenarios
Catastrophic failure rate as a first-class metric
Tool-call validity and policy compliance rates
Time-to-containment distributions
Transfer to scenario families never seen in training

Defensive agent environment blueprint

The blueprint below sketches one task family in the shape LLMix builds: hidden incident state, partial observations, costed actions, and a verifier on ground truth. Real programs define many such families plus held-out ones.

task_family: lateral-movement-containment
hidden_state:
  compromised_hosts: [ws-114, ws-207]        # ground truth, never observed
  attacker_stage: credential_harvesting
  progression: advances every k steps unless contained
observation:
  - alert_queue (noisy, delayed)
  - queryable logs: auth, dns, edr
  - host inventory and asset criticality
actions:
  - query(log_source, filter)                # cheap
  - isolate(host)                            # costly if wrong host
  - reset_credentials(identity)              # costly, sometimes required
  - escalate(report)                         # ends episode, judged on report
termination:
  success: all compromised hosts contained, no active persistence
  failure: exfiltration state reached, or step budget exhausted
reward_components:
  containment: +1.0 (verified against hidden_state)
  early_detection_bonus: +0.2 scaled by attacker_stage
  wrong_isolation: -0.3 per business-critical false positive
  invalid_tool_call: -0.05
  step_cost: -0.01
held_out:
  - unseen attacker progressions and network topologies

One task family from a defensive-agent environment: hidden incident state, costed actions, verifiable containment.

Safety and scope

Cybersecurity training work at LLMix is defensive by construction and bounded by explicit governance.

Defensive capabilities only: no offensive tooling or exploit development
Training runs in controlled, simulated environments
No interaction with real systems the client has not authorized
Client governance over data, access, and trajectory retention

What ships

The incident environment with task generators and held-out scenarios
Verifier and reward implementations with red-team notes
Trained model artifact with training assets and lineage
Evaluation package including catastrophic-failure analysis
Model card with scope, limits, and rollback criteria

RL environments for LLM agents

The environment engineering this vertical is built on: state, tools, verifiers, rollout integration.

How to build an RL environment

The step-by-step guide behind blueprints like the one on this page.

Why GRPO training fails

The diagnostic guide for the online-training stage of programs like this.

Scope a defensive-agent training program.

Scope a post-training program