compliance-scan/docs/detailed-guide.md

compliance-scan - Detailed Guide

Complete reference for developers and advanced users.

Core Entry Points

Component	Path	Purpose
CLI	src/sslysze_scan/__main__.py	Command-line interface entry
Scanner	src/sslysze_scan/scanner.py	SSLyze integration and scan execution
Database Writer	src/sslysze_scan/db/writer.py	Scan result persistence
Reporter	src/sslysze_scan/reporter/	Report generation (CSV/MD/reST)
Compliance	src/sslysze_scan/db/compliance.py	BSI/IANA validation logic
Query	src/sslysze_scan/reporter/query.py	Database queries using views

Installation

poetry install

Quick Reference

# Scan server on multiple ports
poetry run compliance-scan scan example.com:443,636

# Generate Markdown report
poetry run compliance-scan report -t md -o report.md

# Generate CSV reports
poetry run compliance-scan report -t csv --output-dir ./reports

# List all scans
poetry run compliance-scan report --list

CLI Commands

Scan Command

compliance-scan scan <hostname>:<port1>,<port2> [options]

Argument	Required	Description
<hostname>:<ports>	Yes	Target with comma-separated ports. IPv6: [2001:db8::1]:443,636
--print	No	Display summary in console
-db <path>	No	Database file path (default: compliance_status.db)

Examples:

compliance-scan scan example.com:443,636 --print
compliance-scan scan [2001:db8::1]:443,636 -db custom.db

Report Command

compliance-scan report [scan_id] -t <type> [options]

Argument	Required	Description
scan_id	No	Scan ID (default: latest scan)
-t <type>	Yes	Report type: csv, md, rest
-o <file>	No	Output file (md/rest only)
--output-dir <dir>	No	Output directory
--list	No	List all available scans

Examples:

compliance-scan report -t md -o report.md
compliance-scan report 5 -t csv --output-dir ./reports
compliance-scan report -t rest --output-dir ./docs

Report Formats

CSV

Generates granular files per port and category.

File Pattern	Content
summary.csv	Scan statistics and compliance summary
<port>_cipher_suites_<version>_accepted.csv	Accepted cipher suites per TLS version
<port>_cipher_suites_<version>_rejected.csv	Rejected cipher suites per TLS version
<port>_supported_groups.csv	Elliptic curves and DH groups
<port>_missing_groups_bsi.csv	BSI-approved groups not offered
<port>_missing_groups_iana.csv	IANA-recommended groups not offered
<port>_certificates.csv	Certificate chain with compliance
<port>_vulnerabilities.csv	Vulnerability scan results
<port>_protocol_features.csv	TLS protocol features
<port>_session_features.csv	Session handling features
<port>_http_headers.csv	HTTP security headers
<port>_compliance_status.csv	Aggregated compliance per check type

Behavior: Ports without TLS support generate no files. Empty sections are omitted.

Markdown

Single comprehensive report with:

1. Metadata: Scan ID, hostname, IPs, timestamp, duration, ports
2. Summary: Statistics table
3. Per-port sections (TLS-enabled ports only):
   • TLS configuration
   • Cipher suites (accepted/rejected by version)
   • Supported groups with compliance
   • Missing groups (collapsible details)
   • Certificates with key size and compliance
   • Vulnerabilities
   • Protocol features
   • Session features
   • HTTP security headers

reStructuredText

Identical structure to Markdown but uses .. csv-table:: directives for Sphinx integration.

Use case: Generate documentation that references CSV files for tabular data.

Database Structure

File: compliance_status.db (SQLite, Schema Version 5)

Template: src/sslysze_scan/data/crypto_standards.db

Full schema: schema.sql

Scan Result Tables

Table	Content
scans	Scan metadata: scan_id, hostname, ports, timestamp, duration
scanned_hosts	Resolved FQDN with IPv4/IPv6 addresses
scan_cipher_suites	Cipher suites per port and TLS version (accepted/rejected)
scan_supported_groups	Elliptic curves and DH groups per port
scan_certificates	Certificate chain with key type, size, validity
scan_vulnerabilities	Vulnerability test results per port
scan_protocol_features	TLS protocol features (compression, early data, etc.)
scan_session_features	Session renegotiation and resumption
scan_http_headers	HTTP security headers per port
scan_compliance_status	Compliance evaluation per item and port

Database Views (Schema v5)

Six optimized views eliminate complex JOINs and improve query performance:

View	Purpose
v_cipher_suites_with_compliance	Cipher suites with BSI/IANA compliance flags
v_supported_groups_with_compliance	Groups with compliance status
v_certificates_with_compliance	Certificates with key size compliance
v_port_compliance_summary	Aggregated compliance statistics per port
v_missing_bsi_groups	BSI-approved groups not offered by server
v_missing_iana_groups	IANA-recommended groups not offered by server

Reference Data Tables

IANA TLS:

• iana_tls_cipher_suites: Cipher suite registry with recommendations
• iana_tls_signature_schemes: Signature algorithm registry
• iana_tls_supported_groups: Named groups registry

BSI TR-02102-1 (Certificates):

• bsi_tr_02102_1_key_requirements: Key length requirements
• bsi_tr_02102_1_hash_requirements: Hash algorithm requirements

BSI TR-02102-2 (TLS):

• bsi_tr_02102_2_tls: TLS cipher suites and groups with validity periods

BSI TR-02102-3 (IPsec/IKEv2):

• Encryption, integrity, DH groups

BSI TR-02102-4 (SSH):

• Key exchange, encryption, MAC

CSV Export Metadata:

• csv_export_metadata: Stores CSV headers as JSON for all export types

Compliance Validation

BSI TR-02102-1 (Certificates)

Key length requirements:

Algorithm	Minimum Bits	Status
RSA	3000	Required
ECDSA	250	Required
DSA	3072	Deprecated (valid until 2029)

Hash algorithms:

• Allowed: SHA-256, SHA-384, SHA-512
• Deprecated: SHA-1, MD5

BSI TR-02102-2 (TLS)

Validates:

• Cipher suites against BSI-approved lists
• Supported groups against BSI requirements
• Validity periods (time-based expiration)

IANA

Validates:

• Cipher suite recommendations (Y/N/D flags)
• Supported group recommendations (Y/N/D flags)

Project Structure

src/sslysze_scan/
├── __main__.py              # CLI entry point
├── cli.py                   # Argument parsing
├── scanner.py               # SSLyze integration
├── protocol_loader.py       # Port-protocol mapping
├── output.py                # Console output
├── commands/
│   ├── scan.py              # Scan command handler
│   └── report.py            # Report command handler
├── db/
│   ├── schema.py            # Schema version management
│   ├── writer.py            # Scan result storage
│   ├── compliance.py        # Compliance validation
│   └── writers/             # Specialized writers
├── reporter/
│   ├── query.py             # Database queries (uses views)
│   ├── csv_export.py        # CSV generation
│   ├── markdown_export.py   # Markdown generation
│   ├── rst_export.py        # reST generation
│   └── template_utils.py    # Shared utilities
├── templates/
│   ├── report.md.j2         # Markdown template
│   └── report.reST.j2       # reST template
└── data/
    ├── crypto_standards.db  # Template DB (IANA/BSI + schema)
    └── protocols.csv        # Port-protocol mapping

Key Functions

CLI and Parsing

Function	Module	Purpose
parse_host_ports(target)	cli.py	Parse hostname:port1,port2 format
parse_arguments()	cli.py	Parse CLI arguments

Scanning

Function	Module	Purpose
perform_scan(hostname, port, start_time)	scanner.py	Execute SSLyze scan for one port
create_scan_request(hostname, port, use_opportunistic_tls)	scanner.py	Create SSLyze scan request

Database Writing

Function	Module	Purpose
save_scan_results(db_path, hostname, ports, results, start_time, duration)	db/writer.py	Store all scan results, returns scan_id
check_compliance(db_path, scan_id)	db/compliance.py	Validate compliance, returns statistics
check_schema_version(db_path)	db/schema.py	Verify schema compatibility
get_schema_version(db_path)	db/schema.py	Get current schema version

Database Querying

Function	Module	Purpose
get_scan_data(db_path, scan_id)	reporter/query.py	Get complete scan data using views
get_scan_metadata(db_path, scan_id)	reporter/query.py	Get scan metadata only
list_scans(db_path)	reporter/query.py	List all scans in database

Report Generation

Function	Module	Purpose
generate_csv_reports(db_path, scan_id, output_dir)	reporter/csv_export.py	Generate all CSV files
generate_markdown_report(db_path, scan_id, output)	reporter/markdown_export.py	Generate Markdown report
generate_rest_report(db_path, scan_id, output, output_dir)	reporter/rst_export.py	Generate reStructuredText report
_get_headers(db_path, export_type)	reporter/csv_export.py	Load CSV headers from database
build_template_context(data)	reporter/template_utils.py	Prepare Jinja2 template context
generate_report_id(metadata)	reporter/template_utils.py	Generate report ID (YYYYMMDD_scanid)

SQL Query Examples

All queries use optimized views for performance.

Cipher Suites with Compliance

SELECT cipher_suite_name, iana_recommended_final, bsi_approved_final, compliant
FROM v_cipher_suites_with_compliance
WHERE scan_id = ? AND port = ? AND accepted = 1;

Port Compliance Summary

SELECT check_type, total, passed, percentage
FROM v_port_compliance_summary
WHERE scan_id = ? AND port = ?;

Missing BSI Groups

SELECT group_name, tls_version, valid_until
FROM v_missing_bsi_groups
WHERE scan_id = ?;

Non-Compliant Certificates

SELECT port, key_type, key_bits, compliant, compliance_details
FROM v_certificates_with_compliance
WHERE scan_id = ? AND compliant = 0;

Vulnerabilities

SELECT port, vuln_type, vulnerable, details
FROM scan_vulnerabilities
WHERE scan_id = ? AND vulnerable = 1;

Supported Protocols

Opportunistic TLS (STARTTLS)

Protocol	Ports
SMTP	25, 587
LDAP	389
IMAP	143
POP3	110
FTP	21
XMPP	5222, 5269
RDP	3389
PostgreSQL	5432

Direct TLS

Protocol	Port
HTTPS	443
LDAPS	636
SMTPS	465
IMAPS	993
POP3S	995

Not Supported

MySQL (proprietary TLS protocol)

Fallback behavior: Automatic retry with direct TLS if STARTTLS fails.

Testing

poetry run pytest tests/ -v

Test structure:

• tests/conftest.py: Fixtures with test_db, test_db_path
• tests/fixtures/test_scan.db: Real scan data (Scan 1: dc.validation.lan:443,636)
• tests/test_csv_export.py: 11 CSV export tests
• tests/test_template_utils.py: 3 template utility tests
• tests/test_compliance.py: 2 compliance tests
• tests/test_cli.py: 3 CLI parsing tests

Total: 19 tests

Test database setup:

• Loads crypto_standards.db (reference data + schema)
• Loads test_scan.db (scan data only)
• Creates views dynamically
• In-memory for speed

Code Quality

Linter: Ruff

poetry run ruff check src/ tests/
poetry run ruff format src/ tests/

Configuration: pyproject.toml

• Line length: 90 characters
• Target: Python 3.13
• Rules: PEP 8, pyflakes, isort, naming, upgrades

Requirements

• Python 3.13+
• SSLyze 6.0.0+
• Poetry (dependency management)
• Jinja2 3.1+
• pytest 9.0+ (development)
• ruff (development)

Container Usage

./container-build.sh
podman run --rm compliance-scan:latest scan example.com:443

Database Workflow

1. First scan: Copies crypto_standards.db → compliance_status.db
2. Schema check: Validates schema version (must be 5)
3. Scan execution: SSLyze performs TLS analysis
4. Data storage: Results written to scan tables
5. Compliance check: Validation against BSI/IANA via views
6. Report generation: Queries use views for optimized performance

Architecture Notes

Design principles:

• Single database file contains everything (reference data + results)
• Views optimize complex queries (no N+1 queries)
• CSV headers in database (easy to modify)
• Template-based reports (Jinja2)
• Port-agnostic (one scan_id, multiple ports)

Key decisions:

• SQLite for simplicity and portability
• Views introduced in schema v5 for performance
• CSV export metadata centralized
• Test fixtures use real scan data
• Ruff for modern Python linting