Developer Guide

How a run is wired together, where the configuration schema lives, how to run the tests, and what a pull request needs before it can be merged.

Architecture

EOS Connect is a single Python process. src/eos_connect.py is both the entry point and the wiring: it builds the configuration, constructs every interface through a factory, starts three background threads, and serves the web UI. The chain is config → interfaces → optimizer → control.

Startup sequence

Everything below happens at module level in src/eos_connect.py, in this order, before the web server binds.

  1. Bootstrap config. ConfigManager (src/config.py) reads the three keys in src/config.yaml — eos_connect_web_port, time_zone, log_level. Running as a Home Assistant add-on, load_ha_bootstrap() layers /data/options.json over it. load_env_bootstrap() then layers the environment on top, and wins.
  2. Open the settings database. ConfigWebModule.start_db() opens the SQLite store and deep-merges every value the user set in the web UI into the config dict. Everything constructed after this point therefore receives authoritative values — there is no post-init re-sync step.
  3. Build the interfaces through InterfaceFactory, in dependency order.
  4. Start the scheduler. OptimizationScheduler spawns its three daemon threads.
  5. Serve. PortInterface.create_web_server_with_port_check() binds 0.0.0.0 on the configured port (default 8081) and hands Flask to waitress.

Only four environment variables are read, and all four are bootstrap-only (src/config.py:205-212). Everything else lives in the database.

Variable Config key Notes
EOS_WEB_PORT eos_connect_web_port Web UI port.
EOS_TIMEZONE time_zone Any IANA name, e.g. Europe/Berlin.
EOS_LOG_LEVEL log_level debug, info, warning, error.
EOS_DATA_PATH data_path Where eos_connect.db lives. Ignored in a Home Assistant add-on, where the path is always /data.

The interface factory

src/interface_factory.py exposes ten create_* methods, one per interface. Each delegates to the private _create_interface(), which catches whatever the constructor raises, registers it through StartupValidator.add_error(), and then either re-raises (when critical=True) or returns None so the caller can substitute a degraded stand-in.

StartupValidator (src/startup_validator.py, 58 lines) keeps no state of its own. It formats one log line onto the __main__ logger, with | Config: #section and | ACTION REQUIRED markers appended. MemoryLogHandler captures it, /logs/alerts serves it, and the web UI turns the marker into a deep link to the offending settings section. Logging is the single source of truth for startup errors.

Watch the constructor signatures. They are not uniform, and the example in CONTRIBUTING.md is out of date — it passes three arguments to create_battery_interface(), which needs four positional ones and would raise TypeError. The real call, as eos_connect.py makes it:

from interface_factory import InterfaceFactory
from startup_validator import StartupValidator

validator = StartupValidator()
factory = InterfaceFactory(validator)

battery_interface = factory.create_battery_interface(
    battery_config,     # config: Dict[str, Any]
    load_interface,     # load_interface — the battery reads through it
    time_zone,          # time_zone: tzinfo
    base_control,       # base_control: BaseControl
    request_timeout=10,
    critical=True,      # halt startup on failure
)

Note the import paths. _import_and_create() imports interfaces.load_interface, not src.interfaces.load_interface: at runtime src/ is the working directory and interfaces is a top-level package. Tests import the same modules as src.interfaces..., which is why pytest.ini puts the repository root on the path.

The three loops

OptimizationScheduler owns all periodic work. Each loop sleeps in one-second chunks so that shutdown and request_immediate_run() stay responsive.

Loop Interval Does
Optimization Computed by calculate_next_run_time() from the rolling average runtime Builds the request, calls the backend, extracts the control plan, publishes optimization/next_run over MQTT.
Control 1 s Picks the plan entry for the current hour and pushes the resulting battery mode to the inverter.
Data 15 s Polls the inverter for extended monitoring, refreshes live values for the dashboard.

Optimizer backends

OptimizationInterface is a thin abstraction over three backends, chosen by eos.source. All three speak the same EOS-shaped request and response; each backend does its own translation.

  • local_evopt — the default. A PuLP MILP model in src/interfaces/optimization_backends/local_evopt/optimizer.py. No external server.
  • evopt — an external EVopt instance.
  • eos_server — an external Akkudoktor EOS server.

Inverter support works the same way: inverters/inverter_factory.py maps inverter.type onto a class, and every class implements the abstract contract in src/interfaces/base_inverter.py. Adding hardware means adding a subclass plus an entry in INVERTER_TYPES — nothing outside the package changes.

The config schema

src/config_web/schema.py is the single point of truth for every configuration field. It holds 127 FieldDef entries across 12 sections. The web UI, the REST API, the migration logic and this documentation site all read from it. There is no second place to register a setting.

What a field carries

Attribute Meaning
key Dot notation, e.g. battery.capacity_wh.
field_type str, int, float, bool, select, password, sensor, array.
default, section The fallback value and the settings group it belongs to.
level getting_started, standard or expert. Cumulative: get_by_level() returns everything at or below the requested level. Currently 49 / 90 / 34 fields.
help_url config-battery.html#battery-price — a page under user-guide/ and a fragment. All 173 fields have one.
validation, depends_on Bounds, pattern or choices; and the conditional-visibility rule that hides the field until its parent is set.
hot_reload True for 87 fields. Requires matching logic in src/config_web/hot_reload.py, which pushes the new value into the running interface instead of asking for a restart.
labels, display_group Badges (restart_required, deprecated, experimental) and the sub-heading it sorts under.

SECTION_META, at the top of the same file, is what orders the settings menu and the setup wizard steps. Both frontends take the order from the API response rather than hard-coding it, so moving an entry in that dict moves the wizard step.

Adding a field

  1. Add a FieldDef(...) to src/config_web/schema.py. Give it a level — a field with no level is treated as expert.
  2. If it should apply without a restart, set hot_reload=True and add the handler in src/config_web/hot_reload.py.
  3. Point help_url at a heading on the configuration page that explains the field — config-battery.html#battery-price, not the section's catch-all anchor. It is a page under user-guide/ plus a fragment; config.js supplies the rest of the URL.
  4. Run the export and commit the result:
python scripts/export_config_schema.py
# Exported 173 fields and 6 managed-load presets to docs/assets/data/config_schema.json

Nothing regenerates that JSON automatically, so the export is a manual step — but it is a guarded one. tests/config_web/test_export_schema.py::test_the_committed_export_matches_the_schema compares the committed file field by field against a fresh ConfigSchema() and fails on any drift, naming the keys that moved.

How help_url is consumed, on both sides. In the application, src/web/js/config.js splits the value at the # and builds <docs-base>user-guide/<page>?level=<current>#<anchor> for the "Learn more" link next to each setting, where <docs-base> is whichever copy of this site matches the running build (below). On this site, docs/assets/js/config-reference.js reads the exported JSON and groups the reference tables by that same anchor, so every parameter table in the documentation is generated, never hand-written. An anchor that does not exist on the page the value names gives the in-app link nowhere to land.

Which page a setting is explained on

Configuration is one page per topic under docs/user-guide/, listed in NAV[].children in site.js and rendered as a second nav row. Each page carries its own sections' parameters and the explanation of them, so configuring one thing never means reading two pages. configuration.html is the way in: it holds the "I want to…" index and the complete A-Z list of every setting, for looking a name up when you do not know its topic.

A table is mounted where the prose explains it:

<h2 id="battery-price">What your stored energy cost</h2>
<p>…</p>
<div data-schema-section="battery" data-schema-anchor="battery-price"></div>

Without data-schema-anchor the whole section renders. The mount emits no heading of its own: the prose above it is the heading, and idAttr() makes the generated markup yield any anchor the page already owns — which is what keeps one id from being written twice.

Three tests hold this together. test_help_urls_reach_prose_not_the_generated_reference requires every help_url anchor to be a hand-written id on the page it names, so a setting cannot quietly point at a bare table heading; test_every_field_has_a_help_url requires every field to have one at all; and test_every_config_page_is_in_the_nav requires the nav and the directory to agree.

Generated tables for a list section

The Managed Loads page carries a parameter table per appliance profile rather than one table for the section, and those are generated too. Nothing lists which settings a sauna has: every managed_loads.* FieldDef already declares depends_on={"type": [...]}, because that is what the application's own config form filters on, so the same declaration answers the documentation's question. A page mounts one with an empty element:

<div data-managed-load-profile="sauna"></div>

One thing the fields alone cannot supply is the value a profile starts from. A FieldDef holds a single default, and managed_loads.target_temp is shared by four appliance types — its default is the pool's 28 °C, which would document a sauna as aiming twenty degrees below room temperature. The real starting values live in src/loads/presets.py, so the export script writes them out beside the fields as managed_load_presets, and tests/loads/test_schema_presets_agree.py fails if that copy drifts. A None there is meaningful and is rendered as no limit: a sauna has no allowed window, which is not the same as the field being unset.

Adding an appliance type is therefore three steps — the preset, the schema type list, and a <h3> with its mount on the configuration page. test_every_managed_load_type_has_a_profile_section is what stops the third being forgotten.

Development setup

Python 3.13 — that is what the Dockerfile builds on and what CI runs.

git clone https://github.com/ohAnd/EOS_connect.git
cd EOS_connect
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements.txt -r requirements-dev.txt
python src/eos_connect.py

The web UI comes up on http://localhost:8081. On a fresh checkout the setup wizard opens, because the store has no _wizard_completed marker yet.

Keeping a dev instance separate

Use the bootstrap environment variables so a development run cannot touch a real installation's database:

EOS_WEB_PORT=8082 \
EOS_LOG_LEVEL=debug \
EOS_DATA_PATH=/tmp/eos_connect_dev \
python src/eos_connect.py

What is and is not in requirements-dev.txt

It is four lines: pytest>=8.0, pylint>=3.0, black>=24.0, playwright>=1.40. Two of the comments around them matter more than the pins.

Install playwright, not pytest-playwright. The plugin drags in pytest-base-url, whose session-scoped base_url fixture collides with the function-scoped one in tests/interfaces/optimization_backends/test_optimization_backend_eos.py and errors out 44 unrelated tests. Nothing in the suite needs the plugin.

There is no coverage plugin. pytest-cov is not a project dependency, so pytest --cov=src fails on an unrecognised argument. Install it into your own virtualenv if you want a coverage report; do not add it to requirements-dev.txt as part of an unrelated PR.

Formatting and lint

Formatting is Black with its defaults: black .

Lint is pylint, also entirely on its defaults — the repository has no .pylintrc, no pyproject.toml and no setup.cfg. Nothing is disabled globally; suppressions are inline pylint: disable= comments where they are needed. .github/workflows/pylint.yaml runs it over every tracked Python file and fails the build below a score of 9.0:

pylint $(git ls-files '*.py') --output-format=parseable

CONTRIBUTING.md suggests pylint src/, which checks less than CI does — it misses tests/ and scripts/. Run the command above if you want the number CI will see.

Testing

Running the suite

python -m pytest                    # everything, 1772 tests
python -m pytest tests/config_web   # one directory
python -m pytest -k battery_price   # by name
python -m pytest -v tests/          # what CI runs

Both pytest and python -m pytest work, from the repository root, because pytest.ini is short and does exactly two things:

[pytest]
pythonpath = .
testpaths = tests

Tests import as src.config_web... and tests.config_web..., which only resolves with the repository root on sys.path. Bare pytest tests/ does not put it there on its own; pythonpath = . does.

Layout

tests/ mirrors src/. 73 test_*.py files, 1772 collected tests.

Directory Files Covers
tests/ 6 Cross-cutting behaviour: control states, fresh-install startup, degraded interface creation, SSL handling.
tests/config_web/ 17 Store, schema, REST API, migration, merger, hot reload, backup, the schema export guard.
tests/interfaces/ 23 Load, battery, PV, EVCC, MQTT, autoscaler, normalizer, update checker.
tests/interfaces/inverters/ 7 Every inverter implementation against the shared contract.
tests/interfaces/optimization_backends/ 4 The three backends plus the local MILP model.
tests/interfaces/price/ 10 Split further into sources/, battery/, feedin/ and scenarios/.
tests/persistence/ 1 PvYieldStore — the hourly yield history the autoscaler learns from.
tests/web/ 5 Browser tests, see below.

Name a test file after the unit under test: src/interfaces/mqtt_interface.py is tested by tests/interfaces/test_mqtt_interface.py. The one file that deliberately breaks the pattern is tests/interfaces/inverters/base_inverter_tests.py — it is a shared base class of 13 contract tests that each inverter's test class inherits, so it must not be collected on its own.

Browser tests

tests/web/ drives the real UI assets in Chromium through Playwright. It needs a browser build:

playwright install chromium
playwright install-deps chromium                  # needs root
scripts/install_playwright_deps_userspace.sh      # the alternative, without root

The userspace helper downloads libnspr4, libnss3 and libasound2t64 and unpacks them into ~/.cache/ms-playwright-deps/lib; tests/web/conftest.py adds that directory to LD_LIBRARY_PATH by itself. Nothing outside your home directory is touched.

Without Playwright installed, tests/web/ removes itself from collection via collect_ignore_glob rather than failing. A green run does not prove the browser tests ran — check the count. Excluding tests/web, the suite is 1628 tests.

Two harnesses live in that conftest. server/page open a configured install — the store has been migrated from a real config, so you get the dashboard a returning user sees. fresh_server/fresh_page open a first install, where the bootstrap config holds only the three config.yaml keys and the wizard is therefore pending.

What CI actually runs

Workflow Trigger Runs
pylint.yaml Every push; PRs to develop pylint over all tracked .py files, gate at 9.0.
docker_feature.yml Push to any branch but main/develop; PRs to develop Full pytest with Chromium, then a Docker image as a PR artifact.
docker_develop.yml Push to develop Full pytest with Chromium, then build and publish.
docker_main.yml Push to main Full pytest with Chromium, then the release image.
pages.yml Push to main or develop touching docs/ or src/version.py Publishes this site twice — see how the docs are published.

The three Docker workflows install Chromium with python -m playwright install --with-deps chromium before running python -m pytest -v tests/, so tests/web/ genuinely runs in CI. A fifth workflow, docker-image.yml, is manual-dispatch only.

How this site is published

This site is published twice, from two branches, into one GitHub Pages deployment:

URL Branch Who reads it
ohand.github.io/EOS_connect/ main Everyone on a release.
ohand.github.io/EOS_connect/develop/ develop Anyone running a -develop image, and anyone reviewing documentation before it ships.

Before this existed, only main was published, so documentation written on develop was invisible until release — which is why features used to ship with their docs already stale.

Nothing to remember at merge time

None of what marks the preview copy is committed to docs/. That is deliberate: anything committed would travel to main on the next merge and mark the released site instead. A docs PR from develop to main therefore needs no cleanup of any kind.

What marks the preview Where it comes from
The banner across the top docs/assets/js/site.js raises it when the URL contains a develop/ segment. The file is byte-identical on both branches.
noindex, so the preview never competes with the real docs in search results Injected into the preview copy by pages.yml. A robots.txt cannot do this: a project page shares ohand.github.io, whose root is not ours.
The version and commit shown in the banner pages.yml writes develop/build-info.json from the develop checkout's src/version.py. The banner renders without it, which is what happens from a checkout.

A Pages deployment replaces the whole site. pages.yml therefore checks out both branches on every run, whichever one triggered it. A run that published only develop/ would delete the production documentation.

Which version each copy shows

Two different numbers, from two different places, and they answer two different questions:

Shown where Comes from Shape
The badge beside the logo, on both copies VERSION in site.js, one constant the release bump rewrites. Pinned to VERSION_PREFIX in docker_develop.yml by test_version_badge_matches_release_prefix. <major>.<minor>.<release>
The preview banner, on the develop copy only develop/build-info.json, written from the develop checkout's src/version.py. <release>.<build>-develop

The badge names the release line, which is what a reader wants to know: it says which version of the software these pages describe. The banner names the exact build, because on a preview that is the thing you are checking against. The released copy deliberately shows no build number — main builds 0.3.<run number>, a counter that moves on every merge and tells a reader nothing about which features are documented here.

The shapes are written out rather than shown as real numbers on purpose: test_current_version_is_not_hand_written fails on any page that contains the current version, because a copy of it here would be one more place to forget at release time.

Why the publish runs twice. The build number reaches src/version.py only after the push that started the build, in the [AUTO] commit the image workflows push with GITHUB_TOKEN — and GitHub starts no workflow run from an event that token created. So the run that the docs push started reads the previous build number. pages.yml therefore also listens on workflow_run for those two workflows finishing, and publishes again with the version current. That trigger only fires for the copy of the file on the default branch, so it is dormant until pages.yml reaches main.

The app links into whichever copy matches the binary that is running, so a develop build never sends a user to documentation for a release that does not describe the feature they are looking at.

src/docs_links.py decides which base applies from the -develop suffix that docker_develop.yml writes into src/version.py. eos_connect.py resolves it once per page load and writes it onto <body data-docs-base>; every link is then built from it by docsUrl() in src/web/js/constants.js. It is resolved server-side because the version only reaches the browser through /json/current_controls.json, which lands after the menu and the configuration panel have already rendered their links.

The same helper builds the documentation URL that src/interfaces/timeseries_normalizer.py puts into its error messages.

The one exception is README.md. GitHub renders it per branch and gives no injection point, so its documentation links always point at the released site — on both branches.

Contributing

Branches

main holds tagged releases and receives merges from develop only. develop is the integration branch and the target of every normal pull request. Branch off develop, name the branch feature/<description>, and rebase onto origin/develop before opening the PR.

git fetch origin && git switch develop && git pull --ff-only
git switch -c feature/better-forecast
# ... code, tests, docs ...
git fetch origin && git rebase origin/develop
git push -u origin feature/better-forecast

The other roles follow the same shape: fix/ or bugfix/ for something already on develop, and hotfix/ branched from main for an urgent production fix — that one is merged to main, tagged, and then fast-forwarded back into develop. Branches created from a GitHub issue keep their generated issue-<number>-<description> name, which is fine and good for traceability.

CONTRIBUTING.md contradicts itself on the separator. Its branch-roles list writes feature_<short-desc> with an underscore; its own step-by-step flow, six lines later, writes git switch -c feature/better-forecast with a slash. The slash is what the repository actually uses — every merged feature branch in the history is feature/..., and there is not one feature_ among them. Use the slash.

Commit messages

Conventional Commits, subject line 50 characters or fewer. The body explains why; the diff already shows what. Reference issues in the footer as Fixes: #123.

feat: add battery forecast smoothing
fix: correct negative PV handling
docs: update MQTT topic table

Do not edit src/version.py. Versions are bumped by automation and land as [AUTO] Update version to ... commits.

What a pull request needs

  • One logical change. Split anything that needs the word "and" in its subject line.
  • Tests for logic changes, placed to mirror src/ and named test_<module>.py.
  • A clean black . and a pylint score of 9.0 or better.
  • Documentation. The pages under docs/ are the primary documentation and must match actual behaviour — config names, types and defaults included. README.md is an entry point: one to three sentences for a user-visible feature, linking here. Implementation detail does not belong in it.
  • A regenerated docs/assets/data/config_schema.json if you touched schema.py.
  • Base branch develop, rebased, no merge commits.

Some paths have owners recorded in .github/CODEOWNERS; the Victron inverter implementation is one, and a PR touching it requests its maintainer automatically. And an old pointer worth correcting: CONTRIBUTING.md still asks you to update "CONFIG_README". That file is retired — it survives as src/CONFIG_README_OLD.md. Configuration is documented by the schema and by the configuration page instead.

Project structure

EOS_connect/
├── src/
│   ├── eos_connect.py             entry point: wiring, scheduler, Flask routes
│   ├── config.py                  ConfigManager — yaml + options.json + env bootstrap
│   ├── interface_factory.py       InterfaceFactory — 10 create_* methods
│   ├── startup_validator.py       startup errors → the __main__ logger → /logs/alerts
│   ├── log_handler.py             MemoryLogHandler — in-memory log and alert buffers
│   ├── timeseries_grid.py         shared slot-grid placement (neutral layer: loads + interfaces)
│   ├── constants.py
│   ├── version.py                 generated, do not edit
│   ├── config.yaml                bootstrap keys only
│   ├── CONFIG_README_OLD.md       retired
│   │
│   ├── config_web/                settings: store, schema, API, migration, hot reload
│   │   ├── __init__.py            ConfigWebModule — registers both blueprints
│   │   ├── api.py                 config_bp — /api/config, 12 routes
│   │   ├── backup.py              backup_bp — /api/backup, 3 routes
│   │   ├── entity_probe.py        "test this entity" probe for the UI
│   │   ├── timeseries_probe.py    "test this URL" probe for the UI
│   │   ├── hot_reload.py          applies live changes to running interfaces
│   │   ├── merger.py              merges stored values into the config dict
│   │   ├── migration.py           config.yaml / options.json → SQLite, one-time
│   │   ├── schema.py              127 FieldDef entries + SECTION_META
│   │   └── store.py               SQLite key/value store, WAL mode
│   │
│   ├── interfaces/                16 modules
│   │   ├── base_control.py            shared control state
│   │   ├── base_inverter.py           abstract inverter contract
│   │   ├── battery_interface.py
│   │   ├── battery_price_handler.py   per-kWh cost of stored energy
│   │   ├── evcc_interface.py
│   │   ├── feed_in_price_interface.py
│   │   ├── http_fetch.py              shared retry loop + bearer-request kwargs
│   │   ├── load_interface.py
│   │   ├── mqtt_interface.py
│   │   ├── optimization_interface.py  backend selection
│   │   ├── port_interface.py          web server bind and port check
│   │   ├── price_interface.py
│   │   ├── pv_autoscaler.py           corrects forecasts from measured yield
│   │   ├── pv_interface.py
│   │   ├── state_source.py
│   │   ├── timeseries_normalizer.py
│   │   ├── update_checker.py
│   │   ├── config/inverter.settings
│   │   ├── inverters/             11 modules + README.md
│   │   │   ├── inverter_factory.py    INVERTER_TYPES / LEGACY_INVERTER_TYPES
│   │   │   ├── fronius_v2.py          fronius_gen24
│   │   │   ├── fronius_legacy.py      deprecated
│   │   │   ├── victron.py             + victron_old.py, ccgx_registers*.py
│   │   │   ├── inverter_ha.py         via Home Assistant
│   │   │   ├── evcc_inverter.py       EVCC controls the battery
│   │   │   └── null_inverter.py       display-only mode
│   │   └── optimization_backends/
│   │       ├── optimization_backend_eos.py
│   │       ├── optimization_backend_evopt.py
│   │       ├── optimization_backend_local_evopt.py
│   │       └── local_evopt/optimizer.py    the built-in PuLP MILP model
│   │
│   ├── persistence/
│   │   └── pv_yield_store.py      hourly PV yield history for the autoscaler
│   ├── web/                       the application UI (not this site)
│   │   ├── index.html
│   │   ├── css/                   config.css, style.css, wizard.css
│   │   └── js/                    16 modules — main, config, wizard, chart, ...
│   ├── json/                      request/response samples + test/ fixtures
│   └── data/eos_connect.db        the settings and yield database
│
├── tests/                         73 test_*.py files, 1772 tests
├── docs/                          this documentation site
│   ├── assets/data/config_schema.json    generated by the export script
│   └── assets/js/                 site.js (nav, TOC, footer),
│                                  config-reference.js (renders the schema)
├── scripts/
│   ├── export_config_schema.py
│   ├── insert_data_to_db.py
│   ├── install_playwright_deps_userspace.sh
│   └── timeseries_doctor.py       offline payload diagnosis, probe verdicts
├── .github/workflows/             pylint.yaml + 4 Docker image workflows
├── requirements.txt               14 runtime dependencies
├── requirements-dev.txt           pytest, pylint, black, playwright
├── pytest.ini
├── Dockerfile                     python:3.13-slim
├── docker-compose.yml
└── CONTRIBUTING.md

Two directories are easy to confuse. src/web/ is the application's own dashboard and settings UI, served by Flask from the running process. docs/ is this GitHub Pages site. They share no assets and no CSS.

Where to ask

Design questions and "would you accept a PR that..." belong in Discussions; reproducible defects belong in Issues. If a change is large, open the discussion before writing it — it is a smaller conversation than a rejected branch.