:description: High-level map of the system and where to find things. :group: Overview Architecture ============ What PySymGym is ---------------- PySymGym couples a Python reinforcement-learning agent with symbolic execution engines (symbolic virtual machines, SVMs). Path selection is modeled as a game: the current state of the symbolic execution process, represented as an interprocedural control flow graph equipped with execution metadata, is the game map; symbolic states are the pieces the agent moves. For each step the agent observes the map, selects a state, and sends it to the game server; the server performs the step and returns the updated map. Depending on the scoring function, the agent can aim for full coverage in a minimal number of moves, or with a minimal number of generated tests, or something else. All data, including the map, crosses the boundary as JSON, so the gym is not tied to any single symbolic engine. Components ---------- The repository separates the engine, the agent, the inputs, and the tooling. ``AIAgent/`` The Python side: the game protocol, machine-learning models, training, and inference. Its subpackages are: ``common/`` Shared domain model (``GameMap``/``GameState``), typed config, and general utilities. ``connection/`` Transport to game servers (a websocket broker) and the protocol message types. ``ml/`` Datasets, graph-neural-network models, training (including Optuna hyper-parameter tuning), validation, and inference. Main entrypoints: ``run_training.py`` (training and tuning), ``derive_dataset_improvement_config.py`` (derives the dataset-improvement config from a tuning run's MLflow artifacts), ``launch_servers.py`` (server manager), and ``onyx.py`` (PyTorch-to-ONNX conversion and verification). ``GameServers/`` Symbolic execution engines extended to speak the game protocol, kept as git submodules. V# (.NET) is the primary engine; usvm covers the JVM. ``maps/`` Target programs used as symbolic-execution inputs and as training data, mostly git submodules. ``tools/`` Standalone command-line tools: ``runstrat`` (benchmark a strategy), ``compstrat`` (compare two strategies), ``dataset_tools`` (expand and clean datasets), and ``util``. Each tool directory is a package named after the tool (``tools/runstrat/runstrat/``, ``tools/compstrat/ compstrat/``) with the CLI in its ``cli`` module (``python -m runstrat``); ``dataset_tools`` and ``util`` are flat script directories. The tools→AIAgent dependency direction is documented in ``tools/README.md``. ``configs/`` Ready-to-run training and validation YAML configurations. ``workflow/`` Test-workflow configurations and dataset templates referenced by the README and CI. Data and control flow --------------------- A training run starts from an initial dataset produced by a baseline path strategy, then alternates two stages: hyper-parameter tuning to get the best model for the current dataset, and dataset improvement in which that model guides SVM runs and contributes better episodes. Tuning and dataset improvement are repeated until the desired result is reached, after which the trained model can guide symbolic execution directly. The sequence is illustrated by ``resources/game_process_illustration.png`` in the repository root. Where to look for X ------------------- .. list-table:: :header-rows: 1 :widths: 45 55 * - Need - Look at * - Add or change an ML model - ``AIAgent/ml/models/`` * - Change training or tuning - ``AIAgent/run_training.py``, ``AIAgent/ml/training/`` * - Change validation or statistics - ``AIAgent/ml/validation/`` * - Change the game protocol - ``AIAgent/connection/``, ``AIAgent/common/game.py`` * - Integrate a new symbolic engine - ``GameServers/``, ``AIAgent/connection/`` * - Add a target program - ``maps/`` * - Benchmark or compare strategies - ``tools/runstrat/``, ``tools/compstrat/`` * - Maintain the dataset - ``tools/dataset_tools/`` * - Run or tune a configuration - ``configs/``, ``workflow/`` * - Understand or change CI - ``.github/workflows/``