Producing a penetration test checklist and an evidence log that satisfy Essential Cybersecurity Controls (ECC – 2 : 2024) Control 2-11-1 requires disciplined scoping, clear artifact capture, and an auditor-focused presentation; this post shows practical steps, concrete checklist items, and a reproducible evidence-log format tailored for organizations following the Compliance Framework.
Understanding ECC 2-11-1 in the Compliance Framework context
ECC Control 2-11-1 expects organizations to validate security controls via periodic penetration testing and to retain demonstrable evidence that the tests were planned, executed, and remediated. Within the Compliance Framework this maps to three key objectives: (1) defined scope and rules of engagement, (2) verifiable test execution artifacts, and (3) documented remediation and retest evidence. Treat the control as both a technical and procedural requirement: auditors will want to see not only vulnerability findings, but also chain-of-custody, timestamps, and proof that fixes were verified against the original test.
Building an ECC‑compliant penetration testing checklist
Start with a checklist that an internal security lead or an external vendor can follow reproducibly. The checklist should be modular—pre-engagement, active testing, post-test—and explicitly reference Compliance Framework items. Use explicit pass/fail criteria and required evidence items for each step so auditors can confirm completeness.
Pre-engagement and planning items
Include checklist entries for: signed Rules of Engagement (RoE) and Authorization to Test with scope definitions (IP ranges, subdomains, cloud tenants), agreed start/end dates in ISO8601 format, contact and escalation information, and agreed risk/impact thresholds (e.g., do not exploit destructive exploits in production). Require the tester to submit credentials for authenticated testing (if used) via a secure vault (HashiCorp Vault, Azure Key Vault) and record the vault reference in the evidence log.
Active testing and technical checklist items
Practical checks should map to common threat areas and Compliance Framework expectations: discovery (Nmap with version detection - include command line used), vulnerability scanning (Nessus/Qualys report with plugin IDs and scan configuration), web app testing (OWASP Top 10 checks via Burp Suite—include request/response artifacts), authentication and session management tests, privilege escalation attempts on hosts (documented Metasploit or manual exploitation steps), and environment misconfiguration checks (S3 bucket permissions, IAM policies). For each check, require the tool name, exact version, command line or workspace/export, timestamps, and a concise proof-of-concept (PoC) such as sanitized HTTP request/response or shell session transcript.
Designing the auditor-ready evidence log
The evidence log is the single artifact auditors will review to confirm the test occurred and was effective. Use a structured format (CSV or JSONL) with mandatory fields: evidence_id, test_name, tester_name, tester_org, tester_contact, start_time_ISO8601, end_time_ISO8601, target_identifier (IP/hostname/URL), tool_name_and_version, command_or_playbook, artifact_type (screenshot/log/poC/report), artifact_location (secure URI), sha256_checksum, signature_reference (PGP or vendor signature), remediation_status, retest_reference, and auditor_notes. Require digital signatures or vendor-signed hashes for final reports to prevent tampering.
Example log row (CSV fields shown inline): evidence_id=EVID-2026-0001, test_name=External Web App - OWASP, tester_name=AcmeSec Ltd, start_time=2026-03-01T22:00:00Z, target=www.example-shop.com, tool=BurpSuite_3.5.1, artifact_type=poC_http_response, artifact_location=s3://ecc-evidence/2026/03/EVID-0001-poc.json, sha256=3b5d...f2a1, remediation_status=open. Store each artifact at the artifact_location and include the sha256 so auditors can verify integrity.
Retention, storage, and chain-of-custody
Store evidence in a secure, access-controlled repository: S3 buckets with server-side encryption (SSE-KMS) and object versioning, or an internal WORM (write-once-read-many) archive. Implement strict IAM roles for who can upload, view, and delete artifacts, and enable MFA-delete for retention-critical artifacts. Record every access and modification in an audit trail (CloudTrail, SIEM). For high-assurance needs, require that third-party vendors sign final reports with a PGP key and include the public key fingerprint in the evidence log.
Small-business scenarios and practical examples
Example 1 — Small e-commerce store: Scope the external web app, payment API endpoints, and admin console. Checklist items include automated authenticated scans against the checkout API, manual testing for CSRF on cart operations, and configuration checks for cloud storage used for product images. Evidence: screenshots of failed CSRF attempts, Burp request/response pairs, scan export (Nessus XML) with plugin IDs, and remediation tickets in the issue tracker (e.g., JIRA IDs) showing fixes and retest links.
Example 2 — Managed services SMB hosting client websites: Focus on multi-tenancy isolation, control plane access, and SSH hardening. Checklist items can include SSH cipher analysis (ssh-audit output), host-based checks (sudo permissions review), and container escape checks if using Kubernetes. Evidence artifacts: ssh-audit output file, kube-apiserver logs demonstrating RBAC enforcement, and SHA256-signed retest summary.
Compliance tips, best practices, and verification
Best practices to satisfy Compliance Framework reviewers: (1) map each checklist item to the specific paragraph of ECC 2-11-1 in a traceability matrix, (2) require independent third-party testing at least annually or after major changes, (3) capture signed remediation confirmation from system owners—include ticket references and timestamps, (4) maintain a retest policy (verify fixes within 30 days), and (5) perform scope reconciliation so auditors can see differences between planned and executed scope with justification for any deviations. Use CVSS scores in vulnerability findings and include CWE/CVE identifiers to make remediation prioritization auditable.
Risk of non‑implementation
Failing to implement ECC 2-11-1 observably increases the risk of undetected vulnerabilities, prolonged exposure after breaches, and audit findings that can lead to regulatory action or loss of customer trust. From a practical perspective, absent a robust evidence log, auditors cannot verify that remediation occurred or that tests were conducted to the required depth—this often results in repeat audit failures and demands for re-testing at the organization's cost.
In summary, create a reproducible penetration-testing checklist that covers planning, technical checks, and post-test verification; produce a structured evidence log with immutable artifact storage, checksums, and signatures; and map all items back to Compliance Framework ECC 2-11-1. For small businesses, focus on scoped, repeatable tests, clear remediation tickets, and affordable third-party validation to meet auditor expectations while minimizing operational disruption.
