# compliance-scan - Detailed Guide

Complete reference for developers and advanced users.

## Core Entry Points

| Component       | Path                                 | Purpose                               |
| --------------- | ------------------------------------ | ------------------------------------- |
| CLI             | `src/sslysze_scan/__main__.py`       | Command-line interface entry          |
| Scanner         | `src/sslysze_scan/scanner.py`        | SSLyze integration and scan execution |
| Database Writer | `src/sslysze_scan/db/writer.py`      | Scan result persistence               |
| Reporter        | `src/sslysze_scan/reporter/`         | Report generation (CSV/MD/reST)       |
| Compliance      | `src/sslysze_scan/db/compliance.py`  | BSI/IANA validation logic             |
| Query           | `src/sslysze_scan/reporter/query.py` | Database queries using views          |

## Installation

```bash
poetry install
```

## Quick Reference

```bash
# Scan server on multiple ports
poetry run compliance-scan scan example.com:443,636

# Generate Markdown report
poetry run compliance-scan report -t md -o report.md

# Generate CSV reports
poetry run compliance-scan report -t csv --output-dir ./reports

# List all scans
poetry run compliance-scan report --list
```

## CLI Commands

### Scan Command

```
compliance-scan scan <hostname>:<port1>,<port2> [options]
```

| Argument             | Required | Description                                                      |
| -------------------- | -------- | ---------------------------------------------------------------- |
| `<hostname>:<ports>` | Yes      | Target with comma-separated ports. IPv6: `[2001:db8::1]:443,636` |
| `--print`            | No       | Display summary in console                                       |
| `-db <path>`         | No       | Database file path (default: compliance_status.db)               |

Examples:

```bash
compliance-scan scan example.com:443,636 --print
compliance-scan scan [2001:db8::1]:443,636 -db custom.db
```

### Report Command

```
compliance-scan report [scan_id] -t <type> [options]
```

| Argument             | Required | Description                    |
| -------------------- | -------- | ------------------------------ |
| `scan_id`            | No       | Scan ID (default: latest scan) |
| `-t <type>`          | Yes      | Report type: csv, md, rest     |
| `-o <file>`          | No       | Output file (md/rest only)     |
| `--output-dir <dir>` | No       | Output directory               |
| `--list`             | No       | List all available scans       |

Examples:

```bash
compliance-scan report -t md -o report.md
compliance-scan report 5 -t csv --output-dir ./reports
compliance-scan report -t rest --output-dir ./docs
```

## Report Formats

### CSV

Generates granular files per port and category.

| File Pattern                                  | Content                                |
| --------------------------------------------- | -------------------------------------- |
| `summary.csv`                                 | Scan statistics and compliance summary |
| `<port>_cipher_suites_<version>_accepted.csv` | Accepted cipher suites per TLS version |
| `<port>_cipher_suites_<version>_rejected.csv` | Rejected cipher suites per TLS version |
| `<port>_supported_groups.csv`                 | Elliptic curves and DH groups          |
| `<port>_missing_groups_bsi.csv`               | BSI-approved groups not offered        |
| `<port>_missing_groups_iana.csv`              | IANA-recommended groups not offered    |
| `<port>_certificates.csv`                     | Certificate chain with compliance      |
| `<port>_vulnerabilities.csv`                  | Vulnerability scan results             |
| `<port>_protocol_features.csv`                | TLS protocol features                  |
| `<port>_session_features.csv`                 | Session handling features              |
| `<port>_http_headers.csv`                     | HTTP security headers                  |
| `<port>_compliance_status.csv`                | Aggregated compliance per check type   |

Behavior: Ports without TLS support generate no files. Empty sections are omitted.

### Markdown

Single comprehensive report with:

1. Metadata: Scan ID, hostname, IPs, timestamp, duration, ports
2. Summary: Statistics table
3. Per-port sections (TLS-enabled ports only):
   - TLS configuration
   - Cipher suites (accepted/rejected by version)
   - Supported groups with compliance
   - Missing groups (collapsible details)
   - Certificates with key size and compliance
   - Vulnerabilities
   - Protocol features
   - Session features
   - HTTP security headers

### reStructuredText

Identical structure to Markdown but uses `.. csv-table::` directives for Sphinx integration.

Use case: Generate documentation that references CSV files for tabular data.

## Database Structure

File: `compliance_status.db` (SQLite, Schema Version 5)

Template: `src/sslysze_scan/data/crypto_standards.db`

Full schema: [schema.sql](schema.sql)

### Scan Result Tables

| Table                    | Content                                                      |
| ------------------------ | ------------------------------------------------------------ |
| `scans`                  | Scan metadata: scan_id, hostname, ports, timestamp, duration |
| `scanned_hosts`          | Resolved FQDN with IPv4/IPv6 addresses                       |
| `scan_cipher_suites`     | Cipher suites per port and TLS version (accepted/rejected)   |
| `scan_supported_groups`  | Elliptic curves and DH groups per port                       |
| `scan_certificates`      | Certificate chain with key type, size, validity              |
| `scan_vulnerabilities`   | Vulnerability test results per port                          |
| `scan_protocol_features` | TLS protocol features (compression, early data, etc.)        |
| `scan_session_features`  | Session renegotiation and resumption                         |
| `scan_http_headers`      | HTTP security headers per port                               |
| `scan_compliance_status` | Compliance evaluation per item and port                      |

### Database Views (Schema v5)

Six optimized views eliminate complex JOINs and improve query performance:

| View                                 | Purpose                                       |
| ------------------------------------ | --------------------------------------------- |
| `v_cipher_suites_with_compliance`    | Cipher suites with BSI/IANA compliance flags  |
| `v_supported_groups_with_compliance` | Groups with compliance status                 |
| `v_certificates_with_compliance`     | Certificates with key size compliance         |
| `v_port_compliance_summary`          | Aggregated compliance statistics per port     |
| `v_missing_bsi_groups`               | BSI-approved groups not offered by server     |
| `v_missing_iana_groups`              | IANA-recommended groups not offered by server |

### Reference Data Tables

IANA TLS:

- `iana_tls_cipher_suites`: Cipher suite registry with recommendations
- `iana_tls_signature_schemes`: Signature algorithm registry
- `iana_tls_supported_groups`: Named groups registry

BSI TR-02102-1 (Certificates):

- `bsi_tr_02102_1_key_requirements`: Key length requirements
- `bsi_tr_02102_1_hash_requirements`: Hash algorithm requirements

BSI TR-02102-2 (TLS):

- `bsi_tr_02102_2_tls`: TLS cipher suites and groups with validity periods

BSI TR-02102-3 (IPsec/IKEv2):

- Encryption, integrity, DH groups

BSI TR-02102-4 (SSH):

- Key exchange, encryption, MAC

CSV Export Metadata:

- `csv_export_metadata`: Stores CSV headers as JSON for all export types

## Compliance Validation

### BSI TR-02102-1 (Certificates)

Key length requirements:

| Algorithm | Minimum Bits | Status                        |
| --------- | ------------ | ----------------------------- |
| RSA       | 3000         | Required                      |
| ECDSA     | 250          | Required                      |
| DSA       | 3072         | Deprecated (valid until 2029) |

Hash algorithms:

- Allowed: SHA-256, SHA-384, SHA-512
- Deprecated: SHA-1, MD5

### BSI TR-02102-2 (TLS)

Validates:

- Cipher suites against BSI-approved lists
- Supported groups against BSI requirements
- Validity periods (time-based expiration)

### IANA

Validates:

- Cipher suite recommendations (Y/N/D flags)
- Supported group recommendations (Y/N/D flags)

## Project Structure

```
src/sslysze_scan/
├── __main__.py              # CLI entry point
├── cli.py                   # Argument parsing
├── scanner.py               # SSLyze integration
├── protocol_loader.py       # Port-protocol mapping
├── output.py                # Console output
├── commands/
│   ├── scan.py              # Scan command handler
│   └── report.py            # Report command handler
├── db/
│   ├── schema.py            # Schema version management
│   ├── writer.py            # Scan result storage
│   ├── compliance.py        # Compliance validation
│   └── writers/             # Specialized writers
├── reporter/
│   ├── query.py             # Database queries (uses views)
│   ├── csv_export.py        # CSV generation
│   ├── markdown_export.py   # Markdown generation
│   ├── rst_export.py        # reST generation
│   └── template_utils.py    # Shared utilities
├── templates/
│   ├── report.md.j2         # Markdown template
│   └── report.reST.j2       # reST template
└── data/
    ├── crypto_standards.db  # Template DB (IANA/BSI + schema)
    └── protocols.csv        # Port-protocol mapping
```

## Key Functions

### CLI and Parsing

| Function                   | Module   | Purpose                             |
| -------------------------- | -------- | ----------------------------------- |
| `parse_host_ports(target)` | `cli.py` | Parse `hostname:port1,port2` format |
| `parse_arguments()`        | `cli.py` | Parse CLI arguments                 |

### Scanning

| Function                                                     | Module       | Purpose                          |
| ------------------------------------------------------------ | ------------ | -------------------------------- |
| `perform_scan(hostname, port, start_time)`                   | `scanner.py` | Execute SSLyze scan for one port |
| `create_scan_request(hostname, port, use_opportunistic_tls)` | `scanner.py` | Create SSLyze scan request       |

### Database Writing

| Function                                                                     | Module             | Purpose                                 |
| ---------------------------------------------------------------------------- | ------------------ | --------------------------------------- |
| `save_scan_results(db_path, hostname, ports, results, start_time, duration)` | `db/writer.py`     | Store all scan results, returns scan_id |
| `check_compliance(db_path, scan_id)`                                         | `db/compliance.py` | Validate compliance, returns statistics |
| `check_schema_version(db_path)`                                              | `db/schema.py`     | Verify schema compatibility             |
| `get_schema_version(db_path)`                                                | `db/schema.py`     | Get current schema version              |

### Database Querying

| Function                              | Module              | Purpose                            |
| ------------------------------------- | ------------------- | ---------------------------------- |
| `get_scan_data(db_path, scan_id)`     | `reporter/query.py` | Get complete scan data using views |
| `get_scan_metadata(db_path, scan_id)` | `reporter/query.py` | Get scan metadata only             |
| `list_scans(db_path)`                 | `reporter/query.py` | List all scans in database         |

### Report Generation

| Function                                                     | Module                        | Purpose                              |
| ------------------------------------------------------------ | ----------------------------- | ------------------------------------ |
| `generate_csv_reports(db_path, scan_id, output_dir)`         | `reporter/csv_export.py`      | Generate all CSV files               |
| `generate_markdown_report(db_path, scan_id, output)`         | `reporter/markdown_export.py` | Generate Markdown report             |
| `generate_rest_report(db_path, scan_id, output, output_dir)` | `reporter/rst_export.py`      | Generate reStructuredText report     |
| `_get_headers(db_path, export_type)`                         | `reporter/csv_export.py`      | Load CSV headers from database       |
| `build_template_context(data)`                               | `reporter/template_utils.py`  | Prepare Jinja2 template context      |
| `generate_report_id(metadata)`                               | `reporter/template_utils.py`  | Generate report ID (YYYYMMDD_scanid) |

## SQL Query Examples

All queries use optimized views for performance.

### Cipher Suites with Compliance

```sql
SELECT cipher_suite_name, iana_recommended_final, bsi_approved_final, compliant
FROM v_cipher_suites_with_compliance
WHERE scan_id = ? AND port = ? AND accepted = 1;
```

### Port Compliance Summary

```sql
SELECT check_type, total, passed, percentage
FROM v_port_compliance_summary
WHERE scan_id = ? AND port = ?;
```

### Missing BSI Groups

```sql
SELECT group_name, tls_version, valid_until
FROM v_missing_bsi_groups
WHERE scan_id = ?;
```

### Non-Compliant Certificates

```sql
SELECT port, key_type, key_bits, compliant, compliance_details
FROM v_certificates_with_compliance
WHERE scan_id = ? AND compliant = 0;
```

### Vulnerabilities

```sql
SELECT port, vuln_type, vulnerable, details
FROM scan_vulnerabilities
WHERE scan_id = ? AND vulnerable = 1;
```

## Supported Protocols

### Opportunistic TLS (STARTTLS)

| Protocol   | Ports      |
| ---------- | ---------- |
| SMTP       | 25, 587    |
| LDAP       | 389        |
| IMAP       | 143        |
| POP3       | 110        |
| FTP        | 21         |
| XMPP       | 5222, 5269 |
| RDP        | 3389       |
| PostgreSQL | 5432       |

### Direct TLS

| Protocol | Port |
| -------- | ---- |
| HTTPS    | 443  |
| LDAPS    | 636  |
| SMTPS    | 465  |
| IMAPS    | 993  |
| POP3S    | 995  |

### Not Supported

MySQL (proprietary TLS protocol)

Fallback behavior: Automatic retry with direct TLS if STARTTLS fails.

## Testing

```bash
poetry run pytest tests/ -v
```

**Test structure:**

- `tests/conftest.py`: Fixtures with test_db, test_db_path
- `tests/fixtures/test_scan.db`: Real scan data (Scan 1: dc.validation.lan:443,636)
- `tests/test_csv_export.py`: 11 CSV export tests
- `tests/test_template_utils.py`: 3 template utility tests
- `tests/test_compliance.py`: 2 compliance tests
- `tests/test_cli.py`: 3 CLI parsing tests

**Total:** 19 tests

**Test database setup:**

- Loads `crypto_standards.db` (reference data + schema)
- Loads `test_scan.db` (scan data only)
- Creates views dynamically
- In-memory for speed

## Code Quality

**Linter:** Ruff

```bash
poetry run ruff check src/ tests/
poetry run ruff format src/ tests/
```

**Configuration:** `pyproject.toml`

- Line length: 90 characters
- Target: Python 3.13
- Rules: PEP 8, pyflakes, isort, naming, upgrades

## Requirements

- Python 3.13+
- SSLyze 6.0.0+
- Poetry (dependency management)
- Jinja2 3.1+
- pytest 9.0+ (development)
- ruff (development)

## Container Usage

```bash
./container-build.sh
podman run --rm compliance-scan:latest scan example.com:443
```

## Database Workflow

1. **First scan:** Copies `crypto_standards.db` → `compliance_status.db`
2. **Schema check:** Validates schema version (must be 5)
3. **Scan execution:** SSLyze performs TLS analysis
4. **Data storage:** Results written to scan tables
5. **Compliance check:** Validation against BSI/IANA via views
6. **Report generation:** Queries use views for optimized performance

## Architecture Notes

**Design principles:**

- Single database file contains everything (reference data + results)
- Views optimize complex queries (no N+1 queries)
- CSV headers in database (easy to modify)
- Template-based reports (Jinja2)
- Port-agnostic (one scan_id, multiple ports)

**Key decisions:**

- SQLite for simplicity and portability
- Views introduced in schema v5 for performance
- CSV export metadata centralized
- Test fixtures use real scan data
- Ruff for modern Python linting