modules.services

All pyobs process and filesystem logic – module discovery, start/stop/restart, log reading, config read/write, ACL resolution and editing, and ejabberd identity resolution. See How modules are managed and ACL matrix and per-module ACL editing for the user-facing behaviour this implements.

build_acl_matrix() → dict

Builds the fleet-wide (target x caller) ACL matrix.

Rows are every module list_modules() returns; columns are that same full module list plus every caller name mentioned in any module’s resolved acl: block (“allow” keys or “deny” entries) that isn’t itself a managed module (e.g. a human/external caller like “scheduler” if it has no config of its own) – every module is always a column, whether or not it’s ever actually referenced as a caller anywhere, so “could A reach B” is answerable for any pair, not just pairs where B happens to already appear in some acl: block. A module whose config/acl can’t be resolved (bad YAML, broken {include}, …) is still included as a row, with its “error” set, rather than aborting the whole scan.

build_comm_user_map() → dict[str, list[str]]

Maps every local module’s resolved comm.user to the list of module names using it – the reverse direction of find_modules_sharing_comm_user, built once across all of list_modules() rather than queried one identity at a time.

Feeds the fleet-wide Users page (DEVELOPMENT.md’s Ideas -> promoted here): unlike the module page’s own XMPP row, that page needs “for every registered ejabberd account, which module(s) if any use it” – the reverse lookup, not “for this one identity, which modules share it.”

build_package_version_matrix(per_host: list[tuple[str, list[dict]]]) → dict

Turns get_package_overview()-shaped per-host package lists into a package x host matrix for the fleet Overview page – one row per pyobs-* package name (the union across every host), one cell per host in the same order as hosts, each either that host’s get_package_overview() entry for the package or None if that host doesn’t have it installed at all. Mirrors merge_acl_matrices’ row[“cells”]-is-a-dict-keyed-by-column shape turned into a positional list instead (row[“cells”][c] there vs. cells[i] here) – Django templates can’t do a dict lookup keyed by a {% for %} loop variable, only a literal attribute/key, so the per-host values need to already be in column order by the time they reach the template (see fleet_overview.html’s parallel {% for host in package_hosts %} / {% for cell in pkg.cells %} loops).

latest_version is taken from whichever host happened to report one – PyPI has no notion of “per host”, so any host’s non-None reading is as good as any other’s; a package only installed on an unreachable host reports None here, same as get_package_overview()’s own “latest lookup failed” case.

create_module(name: str) → None

Creates a brand-new module config with minimal starter YAML – unlike save_config, which refuses to write a file that doesn’t exist yet, this is the one path that’s allowed to. Refuses if a config with this name already exists, same as it would if someone tried to hand-create a file that’s already there.

find_modules_sharing_comm_user(user: str) → list[str]

Every locally-configured module whose resolved comm.user equals user.

Needed because DEV_EJABBERD_USER_MANAGEMENT.md’s write actions (register/change_password/ ban_account/unban_account/unregister) affect every module sharing an XMPP identity, not just whichever module’s page an action was triggered from – DEV_EJABBERD_INTEGRATION.md’s own “third bug” documents _test and camera sharing one comm.user for real, in this exact fleet, not a hypothetical edge case.

get_comm_user(name: str) → str | None

Resolves a module’s own XMPP identity – its comm.user, e.g. “camera” in comm: {user: camera, …}. Display-only convenience wrapper around get_resolved_comm, dropping the password/source – most callers (dashboard, module page) only ever show this value, they don’t edit it or need its credential. See get_resolved_comm for the fuller resolution DEV_EJABBERD_USER_MANAGEMENT.md’s write actions need.

get_module_status(name: str) → str

Returns ‘running’, ‘stopped’, or ‘unknown’.

get_package_overview() → list[dict]

list_pyobs_packages() plus each package’s latest PyPI release, fetched in parallel since PyPI’s JSON API is one HTTP round-trip per package and this page’s whole point is showing every pyobs-* package at once.

get_resolved_acl(name: str) → tuple[dict | None, str | None]

Returns (acl_block, source) for a module’s effective acl: config, resolving any {include} the same way pyobs-core does.

acl_block is the raw “acl:” dict (with “allow”/”deny”/”mode” keys) or None if the module has no acl: key at all (fully open access). source is None if the block is defined directly in the module’s own config file, or the shared fragment’s name (as used by list_shared_configs()/get_shared_config()) if pulled in via {include}.

get_resolved_comm(name: str) → tuple[str | None, str | None, str | None]

Returns (comm_user, comm_password, source) for a module’s effective comm: block – the same resolution get_resolved_acl uses for acl:, via pre_process_yaml + yaml.safe_load, since comm: can equally arrive via {include} or a YAML anchor/merge key (a real config uses comm: {<<: *comm, user: camera, password: pyobs}).

comm_user/comm_password are None if the module has no comm: block at all (confirmed real example: HttpFileCache) or the respective sub-key is missing – not an error, just “this module was never expected to have an XMPP identity” (see DEV_EJABBERD_INTEGRATION.md, “Where it surfaces”). source is None if comm: is defined directly in the module’s own file, or the shared fragment’s name if pulled in via {include}.

The password is needed (not just user) for DEV_EJABBERD_USER_MANAGEMENT.md’s register action: it registers a new XMPP account using whatever password the module’s config already declares, rather than prompting for a new one – the whole point is making an existing comm.user/comm.password config actually work, not choosing a fresh credential. source is needed for that same doc’s config write-back (change_password), which must refuse to edit comm.password: when it resolves to a shared fragment, exactly the guard save_local_acl already applies to acl:.

list_pyobs_packages() → list[dict]

Installed pyobs-* packages (name + version), via pip list –format=json rather than importlib.metadata – pyobs-web-admin itself may run in a different environment than pyobs (see _pip_exec), so introspecting its own imports wouldn’t reflect what pyobs actually has installed. Also includes any package – pyobs-prefixed or not – listed in settings.PYOBS_MANAGED_PACKAGES, but only if it’s actually installed: that setting can extend which installed packages are shown/managed, never invent an entry for one that isn’t really there.

merge_acl_matrices(per_host: list[tuple[str, dict]]) → dict

Combines each host’s build_acl_matrix()-shaped result into one fleet-wide matrix – see DEV_ACL_MATRIX.md, “Hub mode interaction”. per_host is a list of (host_name, matrix) pairs, e.g. [(“localhost”, build_acl_matrix()), (“MONETS”, <that host’s own matrix, fetched via the hub proxy>), …].

Each host only knows about the callers its own modules’ acl: blocks reference, so a row fetched from one host is missing cells for callers that only appear on some other host. Cells are therefore recomputed here against the union of every host’s callers, reusing _acl_cell (a pure function of a target’s acl: dict + a caller name – safe to call again outside the host that originally resolved that acl:) rather than trusting each host’s own host-local cells.

merge_log_lines(line_lists: list[list[str]], lines: int) → list[str]

Merges several already-formatted, already-oldest-first-ordered log line lists into one list ordered by each line’s own leading timestamp, trimmed to the overall last lines.

Used both for the file backend’s per-module tail merge (_get_all_logs_file) and, in views.py, for combining each hub host’s own already-merged fleet-wide result into one cross-host view – same “no shared time index, so merge-and-trim after the fact” shape either way, just one level up in the second case.

resolve_and_validate_acl(name: str) → tuple[dict | None, str | None, str | None]

Like get_resolved_acl, but also validates the acl:’s shape (allow must be a mapping, deny must be a list) and catches any resolution error (bad YAML, broken {include}, …) into a returned message instead of raising. Returns (acl, source, error) – acl and source are None whenever error is set. Shared by build_acl_matrix (one row’s error shouldn’t abort the whole fleet-wide scan) and the single-module ACL tab endpoint (api_acl’s GET), which need the identical error-handling contract.

save_comm_password(user: str, new_password: str) → list[str]

Writes new_password into comm.password: for every local module whose comm.user resolves to user, splicing just that sub-key (_replace_comm_password). Returns the list of module names updated.

All-or-nothing: if any matching module’s comm: resolves to a shared fragment, raises before writing to any of them – a partial write (some modules updated, others left with a now-stale password) would be a worse outcome than not writing at all, exactly the risk DEV_EJABBERD_USER_MANAGEMENT.md’s Design section calls out for a shared comm.user. If verification fails partway through (some files written, a later one doesn’t check out), rolls back every file this call itself wrote, mirroring save_local_acl’s safety net but extended across the whole matching set.

save_local_acl(name: str, acl: dict | None) → None

Writes a structured acl: edit (from the matrix’s per-target edit form) into a module’s own raw config file.

Splices just the acl: block into the raw text (_replace_local_acl_block) rather than doing a full YAML round-trip of the whole file, since the raw file can contain bare {include …} lines that aren’t valid standalone YAML on their own (see pyobs_config.pre_process_yaml) – a generic YAML parser can’t load it directly.

Refuses to write if the module’s acl: currently comes from a shared fragment; callers must route that edit to the fragment’s own file instead (get_resolved_acl’s source). After writing, re-resolves the module’s acl: and rolls back to the original content if it doesn’t match what was requested – seeing DEV_ACL_MATRIX.md’s note on the splice’s simplifying assumption, this is the safety net against a silent bad write rather than trying to make the splice logic exhaustively correct up front.

update_package(name: str, installed_version: str) → tuple[bool, str]

Runs pip install –upgrade <spec> in pyobs’s own environment (_pip_exec), where <spec> is name itself unless PYOBS_MANAGED_PACKAGES configures a fuller spec for it (see _install_spec_for). Callers (api_package_update) must already have checked name against list_pyobs_packages() – the name check here is just defense in depth, not the primary access control, so that this function alone can never be used to pip-install something arbitrary even if a caller forgot that check. Mirrors list_pyobs_packages’ own “pyobs- prefixed, or explicitly allow-listed via PYOBS_MANAGED_PACKAGES” rule – a name only reachable here if it could also have shown up on the Packages page in the first place.

Adds –pre when installed_version is itself a pre/dev release, mirroring the exact same is_prerelease check _select_latest_version uses to decide what “latest” even means for this package – without it, pip’s own resolver leaves an already-installed pre-release alone entirely rather than upgrading it, even to a newer pre-release (verified live, see _select_latest_version’s docstring), so Update would silently do nothing for exactly the packages this policy exists to handle. –upgrade-strategy=only-if-needed (pip’s own default, made explicit here rather than trusted to stay that way under any local pip.conf) keeps –pre’s effect scoped to resolving this package – already-satisfied dependencies aren’t re-examined for a newer prerelease of their own just because this install allows prereleases in general.