Conceptual Overview

1. What PropFlow Solves

PropFlow is a belief propagation (BP) experimentation platform. It builds factor graphs, runs synchronous BP with configurable policies, and records the evolution of messages, assignments, and costs. The toolkit targets researchers and engineers who need a reproducible environment for comparing BP variants such as pure Min-Sum, damped updates, splitting, or custom convergence strategies.

2. Core Runtime Concepts

Concept	Description
Factor Graph	Bipartite graph of VariableAgent and FactorAgent nodes. Variables carry domains; factors encode local cost tables. Defined in src/propflow/bp/factor_graph.py and constructed via helpers in src/propflow/utils/fg_utils.py.
Message	Directed data structure (src/propflow/core/components.py) representing variable→factor (Q) or factor→variable (R) updates. Messages contain NumPy vectors aligned to variable domains.
Engine	Implementation of the BP update loop (src/propflow/bp/engine_base.py and concrete classes in src/propflow/bp/engines.py). Engines orchestrate the synchronous schedule: variable compute/send, factor compute/send, cost and snapshot capture, and convergence evaluation.
Policy	Optional behaviours applied during message computation: damping, splitting, cost reduction, pruning. Policies live under src/propflow/policies.
Simulator	High-level runner (src/propflow/simulator.py) that executes batches of engine configurations over a set of factor graphs, collects cost trajectories, and handles multiprocessing.
History & Snapshots	engine.history is a compatibility view over automatic per-step engine.snapshots; snapshots can be serialized to JSON for later analysis.
Analyzer Utilities	Use propflow.snapshots.SnapshotAnalyzer and propflow.snapshots.SnapshotVisualizer to interpret minimisers over time.

3. Creation Pipeline (Top-Down)

PropFlow’s runtime mirrors the documentation flow:

1. Agents — VariableAgent and FactorAgent live in src/propflow/core/agents.py. They encapsulate message mailboxes and call into computators.

2. Factor graphs — FactorGraph (src/propflow/bp/factor_graph.py) stitches agents together, assigns dimension indices, and triggers cost-table creation. Use FGBuilder helpers (src/propflow/utils/fg_utils.py) wherever possible.

3. Engines — BPEngine and variants (src/propflow/bp/engine_base.py, src/propflow/bp/engines.py) orchestrate the synchronous message schedule, apply policies, and record history.

4. Simulator — Simulator (src/propflow/simulator.py) executes batches of engine configurations over one or many graphs, typically for benchmarks or parameter sweeps.

5. Analyzer — Modules under src/propflow/snapshots/ capture snapshots and visualise results, letting you inspect or report on specific runs.

The quickest path for an end user is thus: FGBuilder → BPEngine (or variant) → Simulator (optional) → Analyzer tooling. Drop down to manual factor graph construction only when you need a topology that FGBuilder does not support, taking care to supply ordered factor–variable edge lists so cost-table dimensions remain aligned.

4. Directory Layout

src/
  propflow/
    bp/            # Engines, engine components, factor graph primitives
    core/          # Shared agent/message abstractions
    policies/      # Damping, splitting, convergence controls
    configs/       # Global defaults, logging, registries
    utils/         # Graph builders, tooling (FGBuilder, etc.)
    simulator.py   # High-level simulation orchestrator
    cli.py         # Placeholder CLI entrypoint
  propflow/snapshots/   # Snapshot capture, analysis, visualisation
    builder.py          # Construct EngineSnapshot from engine state
    manager.py          # Minimal SnapshotManager used by engines
    analyzer.py         # Jacobian, cycle analysis, reporting tools
    visualizer.py       # Standalone snapshot visualiser

examples/          # Demonstrations (e.g. min-sum walkthrough)
tests/             # Pytest suite covering engines, policies, utilities
docs/handbook/     # Deployment & operations handbook (this document)
configs/           # Logs and generated artefacts (excluded from version control)

5. Data Flow Through an Engine Step

1. Variable Phase: Variable agents read inbox messages (Mailer component), compute outgoing Q-messages via the assigned computator (default Min-Sum), and stage them.

2. Variable Send & Reset: Messages are dispatched; inboxes are cleared in preparation for the next iteration.

3. Factor Phase: Factor agents assemble incoming Q-messages and cost tables to compute R-messages back to neighbours.

4. Factor Send & Reset: R-messages are sent, and mailboxes are cleared.

5. Bookkeeping: Engine updates the global cost and captures an EngineSnapshot through SnapshotManager.

6. Convergence Checks: Using ConvergenceMonitor and the configured thresholds, engines decide whether to halt.

This loop repeats for each iteration until convergence or the configured maximum is reached.

6. Extensibility Points

• Custom Engines: Subclass BPEngine and override hooks (post_var_compute, pre_factor_compute, etc.). Register the class in simulation configs for reuse.

• Policies: Implement new policies under src/propflow/policies and wire them into engines or simulator configs.

• Cost Table Factories: Extend CT_FACTORIES and CTFactories in src/propflow/configs/global_config_mapping.py to generate domain-specific cost tables.

• Analysis: Use propflow.snapshots utilities to capture structured outputs suitable for dashboards, machine learning pipelines, or audits.

7. Operational Roles

• Simulation Engineer: Configures factor graphs, chooses engine variants, evaluates convergence metrics.

• Algorithm Researcher: Modifies computators, policies, or engines to test new BP strategies.

• Data Analyst: Uses snapshot recordings and visualisers to interpret message dynamics and assignment stability.

• DevOps / Platform Team: Deploys the simulator into production or research clusters, automates pipelines, and monitors resource consumption.

Understanding these components provides the foundation for the remaining sections, which focus on getting the project running, deploying it reliably, and maintaining it over time.