Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Get Started Free →

SOC 2 Compliance Automator Skill · a5c-ai | DecimalAI

a5c-ai/soc2-compliance-automator

SkillScore61Maintained

SOC 2 Trust Services Criteria compliance automation for evidence collection, control mapping, and audit preparation

legalby a5c-ai

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

Security & AppSec pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/a5c-ai-soc2-compliance-automator/SKILL.md

decimalai skills pull a5c-ai-soc2-compliance-automator --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/a5c-ai-soc2-compliance-automator/SKILL.md https://app.decimal.ai/s/a5c-ai-soc2-compliance-automator/SKILL.md

Raw file: /s/a5c-ai-soc2-compliance-automator/SKILL.md ↗ · machine-readable summary at /s/a5c-ai-soc2-compliance-automator.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

Runs

all versions · 30d

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

23 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 26d ago

—+32% tokens0% turns61% of whole cases pass with skill (14/23)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-09	✗→✓	▲ Improved	55%	0%
case-14	✗→✓	▲ Improved	-21%	0%
case-15	✗→✓	▲ Improved	6%	0%
case-20	✗→✓	▲ Improved	7%	0%
case-21	✗→✓	▲ Improved	-10%	0%

Showing 5 of 23 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/10/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

a5c-ai / babysitter

library/specializations/security-compliance/skills/soc2-compliance-automator/SKILL.md

View on GitHub ↗

License MITImported 8/10/2026Version v1

Skill contents

SKILL.md

SOC 2 Compliance Automator Skill

Purpose

Automate SOC 2 Trust Services Criteria (TSC) compliance activities including control mapping, evidence collection, audit preparation, and continuous compliance monitoring.

Capabilities

Control Mapping

Map organizational controls to SOC 2 TSC requirements
Cover all five Trust Services Categories:
Security (Common Criteria)
Availability
Processing Integrity
Confidentiality
Privacy
Generate control matrices with evidence requirements
Identify control gaps and coverage

Evidence Collection

Automate evidence gathering from cloud providers
Collect access control configurations (IAM, RBAC)
Capture security configurations and policies
Document change management processes
Archive audit logs and monitoring data
Screenshot automation for manual controls

Audit Preparation

Generate Type I and Type II audit packages
Prepare management assertion documents
Create system description documents
Organize evidence by control objective
Generate auditor-ready reports

Control Effectiveness Tracking

Monitor control implementation status
Track control testing results
Document control exceptions
Manage remediation activities
Calculate compliance scores

Continuous Compliance

Monitor control drift and changes
Alert on compliance deviations
Track evidence freshness
Generate compliance dashboards
Automate periodic control testing

Trust Services Categories

CC - Common Criteria (Security)

CC1: Control Environment
CC2: Communication and Information
CC3: Risk Assessment
CC4: Monitoring Activities
CC5: Control Activities
CC6: Logical and Physical Access Controls
CC7: System Operations
CC8: Change Management
CC9: Risk Mitigation

A - Availability

System availability commitments
Disaster recovery and business continuity
Capacity planning and monitoring

PI - Processing Integrity

Data processing accuracy
Completeness and timeliness
Error handling procedures

C - Confidentiality

Data classification
Encryption requirements
Access restrictions

P - Privacy

Privacy notice and consent
Data subject rights
Data retention and disposal

Integrations

Vanta: Automated security and compliance
Drata: Continuous compliance automation
Secureframe: Security compliance platform
AWS/Azure/GCP APIs: Cloud configuration evidence
Identity Providers: Access control evidence
SIEM Systems: Log and monitoring evidence

Target Processes

SOC 2 Compliance Audit Preparation
Continuous Compliance Monitoring
Security Control Assessment
Audit Readiness Review

Input Schema

json
{
  "type": "object",
  "properties": {
    "auditType": {
      "type": "string",
      "enum": ["Type1", "Type2"],
      "description": "SOC 2 audit type"
    },
    "trustCategories": {
      "type": "array",
      "items": {
        "type": "string",
        "enum": ["Security", "Availability", "ProcessingIntegrity", "Confidentiality", "Privacy"]
      }
    },
    "auditPeriod": {
      "type": "object",
      "properties": {
        "startDate": { "type": "string", "format": "date" },
        "endDate": { "type": "string", "format": "date" }
      }
    },
    "cloudProviders": {
      "type": "array",
      "items": {
        "type": "string",
        "enum": ["AWS", "Azure", "GCP"]
      }
    },
    "controlMatrix": {
      "type": "string",
      "description": "Path to existing control matrix"
    },
    "evidenceBasePath": {
      "type": "string",
      "description": "Base path for evidence storage"
    }
  },
  "required": ["auditType", "trustCategories"]
}

Output Schema

json
{
  "type": "object",
  "properties": {
    "assessmentId": {
      "type": "string"
    },
    "auditType": {
      "type": "string"
    },
    "assessmentDate": {
      "type": "string",
      "format": "date-time"
    },
    "trustCategories": {
      "type": "array"
    },
    "controlSummary": {
      "type": "object",
      "properties": {
        "totalControls": { "type": "integer" },
        "implemented": { "type": "integer" },
        "partiallyImplemented": { "type": "integer" },
        "notImplemented": { "type": "integer" },
        "notApplicable": { "type": "integer" }
      }
    },
    "evidenceStatus": {
      "type": "object",
      "properties": {
        "collected": { "type": "integer" },
        "pending": { "type": "integer" },
        "missing": { "type": "integer" }
      }
    },
    "gapAnalysis": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "controlId": { "type": "string" },
          "gap": { "type": "string" },
          "remediation": { "type": "string" },
          "priority": { "type": "string" }
        }
      }
    },
    "auditPackage": {
      "type": "object",
      "properties": {
        "controlMatrix": { "type": "string" },
        "evidenceFolder": { "type": "string" },
        "systemDescription": { "type": "string" },
        "managementAssertion": { "type": "string" }
      }
    },
    "complianceScore": {
      "type": "number"
    }
  }
}

Usage Example

javascript
skill: {
  name: 'soc2-compliance-automator',
  context: {
    auditType: 'Type2',
    trustCategories: ['Security', 'Availability', 'Confidentiality'],
    auditPeriod: {
      startDate: '2024-01-01',
      endDate: '2024-12-31'
    },
    cloudProviders: ['AWS', 'Azure']
  }
}

Illustrative Want to see what a5c-ai/soc2-compliance-automator would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Latest benchmark · publisher runVerified

8/30/2026, 09:42 UTC · 23 cases

Measured on 21 comparable cases — below the 22 needed to publish a lift, so this result is shown here but not on the registry card or leaderboard.

Lift

+26%

61% vs 35%

Agent turns

1.0 vs 1.0

Tokens

+32%

3,005.81 vs 2,284.52

fail → pass · 6pass → pass · 8fail → fail · 9

Per-case results (23)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-01 We are preparing for our annual SOC 2 Type 2 audit covering the Security and Confidentiality categories for our AWS and Azure environments from 2024-01-01 to 2024-12-31. Run the compliance evaluation and return a report containing the assessment metadata, total and implemented control counts, evidence gathering metrics, prioritized gap analysis, overall score, and file paths for the audit package.	fail→fail	18,779	19,755	+5%	1	1	0%	3,409	4,407	+29%	0	0	—
case-02 Run a SOC 2 Type 1 compliance check for Security, Availability, and Privacy on our GCP infrastructure. Use '/data/evidence' as the base evidence directory. The response should provide a structured output including the assessment date, trust categories reviewed, summary breakdown of control implementation statuses, collected/missing evidence counts, prioritized gap remediation items, and links to the system description and management assertion documents.	fail→fail	9,921	19,547	+97%	1	1	0%	294	4,347	+1379%	0	0	—
case-03 Please evaluate our SOC 2 Type 2 audit readiness for the period between 2024-01-01 and 2024-06-30 across Security, Availability, and Confidentiality categories. We run on AWS and have a matrix stored at '/configs/soc2-matrix.json'. Output the report detailing the assessment ID, compliance score, evidence collection metrics, control breakdown numbers, gap analysis list, and target folder locations for generated artifacts.	fail→fail	13,161	17,122	+30%	1	1	0%	388	3,959	+920%	0	0	—
case-04 We need an ISO 27001:2022 Statement of Applicability evaluation for Annex A controls A.5 through A.8. Can you perform an automated ISO 27001 certification audit gap analysis for ISO 27001 Annex A controls?	fail→fail	21,044	35,687	+70%	1	1	0%	3,457	7,691	+122%	0	0	—
case-05 Please evaluate our cloud healthcare architecture against the HIPAA Security Rule Technical Safeguards 45 CFR section 164.312 including ePHI encryption and audit controls.	fail→fail	24,625	28,612	+16%	1	1	0%	3,668	5,898	+61%	0	0	—
case-06 Generate a PCI-DSS v4.0 SAQ-D report for our payment gateway API integration, including requirement 3 and requirement 6 compliance validation.	fail→fail	24,787	18,882	-24%	1	1	0%	3,350	4,890	+46%	0	0	—
case-07 Run an active vulnerability scan and automatically apply Linux security patches across our AWS EC2 fleet.	fail→fail	17,111	10,635	-38%	1	1	0%	2,069	2,560	+24%	0	0	—
case-08 We want to configure a SOC 2 Type 1 evaluation. When submitting trust categories, our developer provided trustCategories: ['CommonCriteria', 'Availability']. Should 'CommonCriteria' be passed in the JSON input payload, or what is the correct enum string?	pass→pass	9,590	8,241	-14%	1	1	0%	1,657	2,041	+23%	0	0	—
case-09 Our automation pipeline passes 'auditType': 'Type-2' and 'trustCategories': ['Security', 'ProcessingIntegrity']. Is 'Type-2' accepted by the SOC 2 assessment input schema?	fail→pass	8,008	3,923	-51%	1	1	0%	1,385	2,145	+55%	0	0	—
case-10 We need to map system controls related to continuous data processing accuracy, completeness, and automated transaction error handling. Which SOC 2 Trust Services Category covers these specific criteria?	fail→fail	11,834	6,203	-48%	1	1	0%	1,062	2,505	+136%	0	0	—
case-11 In mapping our DevOps deployment pipeline and code PR approval workflows versus our IAM user onboarding/offboarding policies, which Common Criteria categories (CC6 or CC8) apply to each?	pass→pass	20,370	18,117	-11%	1	1	0%	2,350	3,751	+60%	0	0	—
case-12 Which Common Criteria section ID specifically governs business risk mitigation and vendor risk management in a SOC 2 assessment?	pass→pass	11,814	9,055	-23%	1	1	0%	1,193	2,089	+75%	0	0	—
case-13 When reporting control implementation status breakdown in the output JSON, what integer counts must be included in the controlSummary object?	pass→pass	12,734	2,323	-82%	1	1	0%	1,467	1,850	+26%	0	0	—
case-14 What object properties must be included under evidenceStatus in the JSON output to track evidence gathering status?	fail→pass	11,965	1,821	-85%	1	1	0%	2,215	1,760	-21%	0	0	—
case-15 When returning remediation gaps in the JSON output schema, what properties are required inside each item of the gapAnalysis array?	fail→pass	16,215	3,697	-77%	1	1	0%	2,067	2,194	+6%	0	0	—
case-16 How must the auditPeriod object be structured in the SOC 2 assessment input JSON for an evaluation period covering January 1, 2024 to December 31, 2024?	pass→pass	14,186	7,910	-44%	1	1	0%	1,770	2,075	+17%	0	0	—
case-17 Where should periodic enterprise risk assessment reviews be mapped versus automated continuous security monitoring and alert triage in the Common Criteria framework?	fail→fail	19,850	14,375	-28%	1	1	0%	2,615	4,006	+53%	0	0	—
case-18 What operational topics are governed under the Availability Trust Services Category in SOC 2?	pass→pass	17,574	18,321	+4%	1	1	0%	2,113	3,453	+63%	0	0	—
case-19 Which SOC 2 Trust Services Category covers privacy notices, user consent tracking, data subject rights fulfillment, and data retention policies?	pass→pass	10,511	9,619	-8%	1	1	0%	915	2,211	+142%	0	0	—
case-20 A user submits a JSON payload containing only 'cloudProviders': ['AWS'] and 'evidenceBasePath': '/var/evidence'. Which required input fields are missing?	fail→pass	15,499	2,445	-84%	1	1	0%	1,804	1,930	+7%	0	0	—
case-21 What specific property names must exist inside the auditPackage JSON object returned by the assessment?	fail→pass	16,845	7,298	-57%	1	1	0%	1,960	1,755	-10%	0	0	—
case-22 In the assessment output schema, what data type is expected for complianceScore?	pass→pass	5,761	1,463	-75%	1	1	0%	946	1,619	+71%	0	0	—
case-23 Construct a complete, valid input JSON object for a SOC 2 Type 2 evaluation covering Security, Confidentiality, and Availability across AWS and GCP environments for 2024.	fail→pass	30,875	9,249	-70%	1	1	0%	6,503	2,292	-65%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 23 cases were attempted, and 21 counted toward the lift figure. The other 2 produced results that are not comparable between the two arms, so they are excluded from the headline rather than averaged into it. The headline lift of +26 percentage points is the difference between those two pass rates over the 21 comparable cases.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 34.8%
Pass rate with the skill: 60.9%
Cases attempted: 23
Cases counted toward lift: 21
Cases that regressed: 0
Skill version measured: v1
Run completed: 8/30/2026, 09:42 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-09

Prompt

Our automation pipeline passes 'auditType': 'Type-2' and 'trustCategories': ['Security', 'ProcessingIntegrity']. Is 'Type-2' accepted by the SOC 2 assessment input schema?

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signals1 of 4 complete

BenchmarkPass rate WITH skill on a verified run61

Live evalNeeds ≥10 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns+54% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

Backed by 1 of 4 signals — each added signal makes the score more robust.

Verified on gemini-3.6-flash · re-tested 26 days ago

Other measured skills in the registry, with their headline benchmark lift.