Developer Guide

Architecture Overview

System Components

Core Interfaces

Data Flow

1. Main Loop (every refresh_time minutes)
   ├─ Fetch battery SOC (battery_interface)
   ├─ Fetch PV forecast (pv_interface)
   ├─ Fetch price forecast (price_interface)
   ├─ Fetch load data (load_interface)
   ├─ Build optimization request
   ├─ Send to EOS/EVopt (optimization_interface)
   ├─ Process optimization response
   ├─ Update controls (base_control)
   ├─ Execute inverter commands (inverter_interface)
   ├─ Update EVCC (evcc_interface)
   ├─ Publish to MQTT (mqtt_interface)
   └─ Serve via web API

2. Config Web Module (parallel)
   ├─ Serve config REST API (/api/config/...)
   ├─ On PUT: validate → store in SQLite → rebuild merged config
   ├─ Hot-reloadable fields → apply instantly via HotReloadAdapter
   └─ Restart-required fields → flag in UI, apply on next restart

Interface Creation & Startup Error Handling

Two new modules work together to centralize interface creation and provide startup visibility:

InterfaceFactory (src/interface_factory.py)

Factory pattern for centralized interface creation with integrated startup validation.

Responsibilities:

• Instantiate all interface types (Load, Battery, Price, PV, MQTT, EVCC, Inverter, Optimization)
• Catch errors during instantiation
• Register errors with StartupValidator for visibility in startup panel
• Support critical vs non-critical interfaces (critical failures halt startup, non-critical fall back to defaults)
• Track created interfaces for lifecycle management

Usage in eos_connect.py:

from interface_factory import InterfaceFactory
from startup_validator import StartupValidator

# Initialize validator and factory at startup
validator = StartupValidator()
factory = InterfaceFactory(validator)

# Create interfaces with error handling
battery_interface = factory.create_battery_interface(
    config=config['battery'],
    time_zone=time_zone,
    critical=True,  # Failure halts startup
)

load_interface = factory.create_load_interface(
    config=config['load'],
    time_frame_base=config['refresh_time'],
    time_zone=time_zone,
    critical=False,  # Failure doesn't halt, uses default
)

Benefits:

• Reduces boilerplate in main app (no try/except for each interface)
• Consistent error categorization across all interface types
• Centralized startup error collection for web UI visibility
• Easy to extend with new interface types

StartupValidator (src/startup_validator.py)

Lightweight facade for startup error registration to the logging system.

Responsibilities:

• Register startup errors with structured metadata
• Write ERROR/WARNING logs captured by MemoryLogHandler
• Embed metadata markers in log messages for frontend parsing (e.g., | Config: #section | ACTION REQUIRED)
• Act as single source of truth for startup errors accessible via /logs/alerts endpoint

Method signature:

validator.add_error(
    category="connectivity",        # initialization, configuration, connectivity
    component="battery_interface",  # Component name for identification
    severity="error",              # error or warning
    title="Battery unavailable",   # User-friendly title
    message="Connection timeout",  # Detailed message
    action_required=True,          # Flag for startup panel badge
    config_link="#battery",        # Link to config section
)

Frontend Integration:

The web UI's startup panel fetches errors via:

GET /logs/alerts?startup_only=1&limit=20

Errors with metadata markers are parsed and rendered with:

• Timestamp and occurrence count
• ACTION REQUIRED badge (yellow) if flagged
• "Open Configuration" link pointing to the config section

Startup Error Flow

Startup:
  InterfaceFactory.create_*_interface()
    │
    ├─ Try to create interface
    │   │
    │   ├─ Success → return interface, user sees nothing
    │   │
    │   └─ Failure → catch exception
    │       │
    │       └─ validator.add_error(...)
    │           │
    │           └─ MemoryLogHandler captures ERROR/WARNING log
    │               │
    │               ├─ Metadata extracted by frontend parseAlertMeta()
    │               │   (Config link, ACTION REQUIRED flag)
    │               │
    │               └─ Stored in log buffer, visible via /logs/alerts

Runtime (user views dashboard):
  Fetch /logs/alerts?startup_only=1
    │
    └─ Frontend renderAlertSection()
       ├─ Deduplicates errors by title
       ├─ Sorts by severity (ACTION REQUIRED first)
       ├─ Shows: timestamp, count, config link button
       └─ User clicks → showConfigurationMenu(section)

Extending with New Interface Types

To add a new interface creation method to InterfaceFactory:

1. Add method to InterfaceFactory following the pattern of existing methods
2. Specify error category, component name, config link, and whether critical
3. Call from eos_connect.py during startup
4. If instantiation fails, StartupValidator.add_error() is called automatically
5. Errors appear in startup panel within 1-2 seconds

Example:

def create_my_new_interface(
    self,
    config: Dict[str, Any],
    critical: bool = True,
):
    return self._create_interface(
        component_name="my_new_interface",
        category="connectivity",
        critical=critical,
        title="My New Interface unavailable",
        error_message="Failed to initialize",
        config_link="#my_section",
        creator_func=lambda: self._import_and_create(
            "interfaces.my_new_interface",
            "MyNewInterface",
            config,
            request_timeout=self.validator.request_timeout,
        ),
    )

Startup Error Metadata in Logs

Error messages can embed metadata for frontend parsing:

Log message format:
"[component] Title: Message | Config: #section | ACTION REQUIRED"

Examples:
"[eos_backend] EOS Connection failed: Connection timeout | Config: #eos | ACTION REQUIRED"
"[battery_interface] Battery SOC error: Authentication failed | Config: #battery | ACTION REQUIRED"
"[load_interface] Load data unavailable: Request timeout | Config: #load"

The frontend's parseAlertMeta() function extracts:

• Config link: Matches Config: (#\w+)
• Action required flag: Presence of "ACTION REQUIRED"
• Component: Text in square brackets at start

Contributing

Branch Strategy

Contribution Workflow

# 1. Update local
git fetch origin
git switch develop
git pull --ff-only

# 2. Create feature branch
git switch -c feature/better-forecast

# 3. Code + test + document
# - Write code
# - Add/update tests
# - Update docs (README / CONFIG_README / MQTT if behavior changes)

# 4. Format and lint
black .
pylint src/

# 5. Run tests
pytest tests/

# 6. Rebase before PR
git fetch origin
git rebase origin/develop

# 7. Push
git push -u origin feature/better-forecast

# 8. Create PR targeting develop
# - Link relevant issues (Closes #123)
# - Provide clear description

Code Quality Requirements

• Formatting: Use Black for all Python files
• Linting: Pylint score ≥9.0
• Testing: Add/update tests for logic changes
• Documentation: Update relevant docs

Commit Message Format (Conventional Commits)

feat: add battery forecast smoothing
fix: correct negative PV handling
docs: update MQTT topic table
test: add tests for price calculation
refactor: simplify config loading
chore: update dependencies

Testing

Test Structure

Tests are organized to mirror the source structure:

tests/
├── test_control_states.py
├── config_web/
│   ├── test_api.py
│   ├── test_export_schema.py
│   ├── test_ha_addon.py
│   ├── test_hot_reload.py
│   ├── test_merger.py
│   ├── test_migration.py
│   ├── test_schema.py
│   └── test_store.py
└── interfaces/
    ├── test_base_control.py
    ├── test_base_inverter.py
    ├── test_battery_interface.py
    ├── test_battery_price_handler.py
    ├── test_dynamic_override.py
    ├── test_load_interface.py
    ├── test_optimization_interface.py
    ├── test_price_interface.py
    ├── test_pv_interface.py
    ├── test_update_checker.py
    ├── inverters/
    │   ├── test_base_inverter.py
    │   ├── test_fronius_legacy.py
    │   ├── test_fronius_v2.py
    │   ├── test_inverter_ha.py
    │   ├── test_null_inverter.py
    │   ├── test_victron.py
    │   └── test_victron_inverter.py
    └── optimization_backends/
        ├── test_optimization_backend_eos.py
        └── test_optimization_backend_evopt.py

Running Tests

# Run all tests
pytest tests/

# Run specific test file
pytest tests/interfaces/test_battery_interface.py

# Run with coverage
pytest --cov=src tests/

# Run with verbose output
pytest -v tests/

Writing Tests

Test File Naming

• Place in tests/ directory
• Mirror source structure
• Name as test_<module>.py

Example Test

import pytest
from src.interfaces.battery_interface import BatteryInterface

def test_battery_soc_calculation():
    """Test SOC calculation with known values"""
    battery = BatteryInterface(capacity_wh=10000)
    battery.set_soc(0.75)
    
    assert battery.get_remaining_wh() == 7500
    assert battery.get_soc() == 0.75

def test_battery_temperature_protection():
    """Test temperature-based power reduction"""
    battery = BatteryInterface(
        capacity_wh=10000,
        max_charge_power_w=5000,
        temperature_sensor="sensor.temp"
    )
    
    # Cold temperature should reduce power
    power = battery.calculate_max_charge_power(
        soc=0.50, 
        temperature=-5
    )
    assert power < 500  # Should be ~5-7.5% of max

Test Configuration

Configure pytest in pytest.ini:

[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*

Development Setup

Prerequisites

• Python 3.11 or higher
• Git
• pip
• (Optional) VS Code with Python extension

Setup Steps

# 1. Clone repository
git clone https://github.com/ohAnd/EOS_connect.git
cd EOS_connect

# 2. Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Install development dependencies
pip install black pylint pytest pytest-cov

# 5. Run EOS Connect
python src/eos_connect.py
# On first launch, a Setup Wizard in the web UI guides you through configuration.
# Only bootstrap settings (port, timezone, log level) can optionally be set in config.yaml.

VS Code Configuration

Recommended .vscode/settings.json:

{
  "python.defaultInterpreterPath": "./venv/bin/python",
  "python.linting.enabled": true,
  "python.linting.pylintEnabled": true,
  "python.linting.pylintArgs": ["--rcfile=.pylintrc"],
  "python.formatting.provider": "black",
  "[python]": {
    "editor.formatOnSave": true,
    "editor.codeActionsOnSave": {
      "source.organizeImports": true
    }
  },
  "python.testing.pytestEnabled": true,
  "python.testing.pytestArgs": ["tests"]
}

Environment Variables

For development, you can override config with environment variables:

# Linux/Mac
export EOS_CONNECT_LOG_LEVEL=debug
export EOS_CONNECT_PORT=8081

# Windows
set EOS_CONNECT_LOG_LEVEL=debug
set EOS_CONNECT_PORT=8081

Config Web Module

The web-based configuration system lives in src/config_web/ as a self-contained module. It replaces direct YAML editing with a browser UI, SQLite persistence, and a REST API — while keeping every existing interface working unchanged.

Module Files

Key Design Principles

• Zero interface changes: The merger produces the same dict shape interfaces always consumed. No code changes needed in existing interfaces.
• SPOT: schema.py defines all field metadata. The web UI, REST API validation, docs export, and merger all consume it — never duplicate field definitions.
• Bootstrap vs Store: ~5 bootstrap keys (web_port, time_zone, log_level, …) stay in config.yaml / ENV / HA options. Everything else lives in SQLite.
• 3-level progressive disclosure: Fields have a level (getting_started, standard, expert) controlling visibility in the setup wizard and config UI.
• Static asset caching: HTML is revalidated on each load, while local CSS/JS assets use versioned URLs (?v=<app_version>) so browsers can cache aggressively and still fetch new assets automatically after upgrades.

SPOT Pipeline

schema.py (Python FieldDefs)
   │
   ├──► export_config_schema.py ──► docs/assets/data/config_schema.json
   │                                     (consumed by GitHub Pages docs)
   │
   └──► /api/config/schema (live REST endpoint)
              │
              ├──► config.js  (ConfigurationManager reads fields + sections)
              └──► wizard.js  (SetupWizard reads fields + sections)

When you change a field definition in schema.py, the web UI, API validation, docs table, and merger all update automatically. Run python scripts/export_config_schema.py to regenerate the docs JSON.

Data Flow: Config Lifecycle

Startup:
  config.yaml ──► ConfigManager (bootstrap keys)
       │
       ▼
  migration.py: YAML values + HA options.json ──► SQLite store
       │
       ▼
  merger.py: bootstrap + SQLite + defaults ──► merged config dict
       │
       ▼
  Interfaces receive merged dict (same shape as before)

Runtime (user saves via web UI):
  Browser PUT /api/config/ ──► api.py validates against schema
       │
       ▼
  store.py writes to SQLite ──► merger rebuilds config dict
       │
       ├──► hot_reload.py applies hot-reloadable fields instantly
       └──► restart-required fields flagged in UI banner

REST API Endpoints

All endpoints are under /api/config/:

Hot Reload

HotReloadAdapter registers interface references at startup. When hot-reloadable fields are saved, changes are applied immediately without restart:

• Price fields — tibber token, awattar country, energycharts zone, etc.
• Battery SOC fields — min/max SOC percentages

Fields that require a restart (e.g., MQTT broker URL, web port) are flagged in the API response, and the UI shows a restart banner.

Frontend Architecture

Both config.js and wizard.js read the CONFIG_SECTIONS from the API response (schemaData.sections) — section icons and labels are never hardcoded in the frontend.

How to Add a New Config Field

1. Add a FieldDef(...) to _ALL_FIELDS in src/config_web/schema.py
2. Run python scripts/export_config_schema.py to update docs JSON
3. Done — Web UI, API validation, docs table, migration, and merger all pick it up automatically
4. If the field is hot-reloadable: add mapping to _PRICE_FIELD_MAP or _BATTERY_SOC_FIELDS in hot_reload.py

How to Add a New Config Section

1. Add fields with the new section name in schema.py
2. Add an entry to SECTION_META dict in schema.py (icon + label)
3. Run python scripts/export_config_schema.py
4. Done — Frontend falls back gracefully, but SECTION_META gives it the correct icon and label

Project Structure

EOS_connect/
├── src/
│   ├── eos_connect.py         # Main application
│   ├── config.py              # Bootstrap configuration
│   ├── config.yaml            # Bootstrap + fallback config
│   ├── constants.py           # Constants and enums
│   ├── log_handler.py         # Logging setup
│   ├── version.py             # Version information
│   ├── interface_factory.py   # Factory for centralized interface creation
│   ├── startup_validator.py   # Startup error registration facade
│   ├── config_web/            # Web-based config system
│   │   ├── __init__.py        # ConfigWebModule facade
│   │   ├── schema.py          # SPOT — all field definitions
│   │   ├── store.py           # SQLite persistence (WAL)
│   │   ├── migration.py       # YAML/HA → SQLite migration
│   │   ├── merger.py          # Builds merged config dict
│   │   ├── api.py             # Flask Blueprint REST API
│   │   └── hot_reload.py      # Live-apply changes
│   ├── interfaces/            # Data interfaces
│   │   ├── base_control.py
│   │   ├── base_inverter.py
│   │   ├── battery_interface.py
│   │   ├── battery_price_handler.py
│   │   ├── evcc_interface.py
│   │   ├── load_interface.py
│   │   ├── mqtt_interface.py
│   │   ├── optimization_interface.py
│   │   ├── port_interface.py
│   │   ├── price_interface.py
│   │   ├── pv_interface.py
│   │   ├── update_checker.py
│   │   ├── inverters/         # Inverter implementations
│   │   └── optimization_backends/
│   ├── web/                   # Web dashboard + config UI
│   │   ├── index.html
│   │   ├── css/
│   │   │   ├── style.css      # Main dashboard styles
│   │   │   ├── config.css     # Config UI styles
│   │   │   └── wizard.css     # Setup wizard styles
│   │   └── js/
│   │       ├── main.js        # Dashboard entry point
│   │       ├── config.js      # ConfigurationManager
│   │       ├── wizard.js      # SetupWizard
│   │       └── ...            # Other dashboard modules
│   └── json/                  # JSON schemas and examples
├── scripts/
│   └── export_config_schema.py  # SPOT → docs JSON export
├── tests/                     # Test suite (659+ tests)
├── docs/                      # Documentation (GitHub Pages)
│   └── assets/data/
│       └── config_schema.json # Exported schema for docs
├── docker-compose.yml         # Docker deployment
├── Dockerfile                 # Docker image
├── requirements.txt           # Python dependencies
├── pytest.ini                 # Pytest configuration
├── README.md                  # Project overview
├── CONTRIBUTING.md            # Contribution guidelines
└── LICENSE                    # MIT License

Resources

Documentation

• GitHub Repository
• README
• Configuration Reference
• Contributing Guidelines

Related Projects

• Akkudoktor EOS - Main optimization backend
• EVCC - EV charging controller
• Home Assistant - Smart home platform

Tools

• Black - Code formatter
• Pylint - Code analyzer
• Pytest - Testing framework

Get Involved!

We welcome contributions of all kinds:

HA Addon Integration

EOS Connect runs as a Home Assistant add-on via the ohAnd/ha_addons repository. The web-based config system changes how configuration is handled.

How it Works

Detection & Data Path

ConfigManager.is_ha_addon returns True when any of these are true:

• HASSIO environment variable is set
• HASSIO_TOKEN environment variable is set
• /data/options.json exists

When detected, data_dir resolves to /data and the SQLite DB is created at /data/eos_connect.db.

Bootstrap from options.json

Bootstrap values (web_port, time_zone, log_level) are read from /data/options.json and override config.yaml defaults. These are the only settings managed through the HA addon config panel.

Legacy Migration

Older HA addon versions provided ALL configuration options in options.json. On first run with the new system:

1. If SQLite store is empty AND options.json has non-bootstrap keys:
2. migrate_ha_options_to_store() imports all non-bootstrap values into SQLite
3. Bootstrap keys are skipped (they stay in options.json)
4. The migration is one-time and idempotent

Required Changes in ha_addons Repo

The following changes are needed in ohAnd/ha_addons:

# eos_connect/config.yaml (addon manifest)
options:
  web_port: 8081
  time_zone: "Europe/Berlin"
  log_level: "INFO"

schema:
  web_port: int
  time_zone: str
  log_level: list(DEBUG|INFO|WARNING|ERROR)

# Add persistent data mapping
map:
  - addon_config:rw   # persistent /data/ for SQLite DB

# Point to EOS Connect web UI
webui: http://[HOST]:[PORT:8081]

The same changes apply to both eos_connect/config.yaml and eos_connect_develop/config.yaml. Update translations/en.yaml to match the reduced option set.