From: Jérôme Benoit Date: Fri, 2 Oct 2026 16:37:42 +0000 (+0200) Subject: fix(reforcexy): enforce PBRS contracts and 70% coverage X-Git-Url: https://git.piment-noir.org/?a=commitdiff_plain;h=b2c4f57a095b9e0763735cd53f85980165a1c687;p=freqai-strategies.git fix(reforcexy): enforce PBRS contracts and 70% coverage --- diff --git a/.github/workflows/quality.yml b/.github/workflows/quality.yml index ed7f54d..d9f8c80 100644 --- a/.github/workflows/quality.yml +++ b/.github/workflows/quality.yml @@ -43,12 +43,10 @@ jobs: context: quickadapter project: quickadapter image: freqai-strategies-quickadapter-qa:ci - coverage: true - name: ReforceXY context: ReforceXY project: reforcexy image: freqai-strategies-reforcexy-qa:ci - coverage: true steps: - name: Check out repository uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 @@ -65,18 +63,7 @@ jobs: "${{ matrix.image }}" \ /workspace/scripts/check_basedpyright.py \ --project "${{ matrix.project }}" - - name: Run runtime regressions - if: ${{ !matrix.coverage }} - run: | - docker run --rm \ - --mount "type=bind,src=${GITHUB_WORKSPACE},dst=/workspace,readonly" \ - --workdir /workspace \ - --env PYTHONPATH=/workspace/${{ matrix.context }}/user_data/strategies \ - --entrypoint python \ - "${{ matrix.image }}" \ - -m unittest discover -s "${{ matrix.context }}/tests" -v - name: Run runtime regressions with coverage - if: matrix.coverage run: | docker run --rm \ --mount "type=bind,src=${GITHUB_WORKSPACE},dst=/workspace,readonly" \ @@ -87,17 +74,16 @@ jobs: --entrypoint sh \ "${{ matrix.image }}" \ /workspace/scripts/run-coverage.sh -s "${{ matrix.context }}/tests" -v - - name: Run runtime regressions under three method orderings + - name: Run runtime regressions with an alternate method ordering if: matrix.context == 'ReforceXY' run: | docker run --rm \ --mount "type=bind,src=${GITHUB_WORKSPACE},dst=/workspace,readonly" \ --workdir /workspace \ - --entrypoint sh \ + --env FREQAI_QA_SHUFFLE_SEED=1 \ + --entrypoint python \ "${{ matrix.image }}" \ - -c 'for seed in 1 2 3; do - FREQAI_QA_SHUFFLE_SEED=$seed python -m unittest discover -s "${{ matrix.context }}/tests" || exit 1 - done' + -m unittest discover -s "${{ matrix.context }}/tests" -v reward-space-analysis: name: Reward space analysis diff --git a/README.md b/README.md index 88c4c46..fd20675 100644 --- a/README.md +++ b/README.md @@ -475,124 +475,77 @@ candle whose price filled the event. Action fills use `execution_tick`; terminal liquidations use the returned post-increment tick. `terminal_liquidation` and `exit_pnl` remain on the transition history row. -## Development - -### Runtime regressions - -Run each suite in its matching Freqtrade QA image, with the repository mounted -at `/workspace` and `/workspace` as the working directory: - -```shell -# ReforceXY -python -m unittest discover -s ReforceXY/tests -v - -# QuickAdapter -PYTHONPATH=/workspace/quickadapter/user_data/strategies \ - python -m unittest discover -s quickadapter/tests -v -``` - -Both commands must be run from the repository root, which is what the -container's `--workdir /workspace` provides. To select one concern, pass a -pattern that matches the whole `module.Class.method` name; a bare substring -selects more than you want, and a pattern that matches nothing runs zero tests -and fails with exit code 5: - -```shell -PYTHONPATH=quickadapter/user_data/strategies \ - python -m unittest discover -s quickadapter/tests -k 'test_utils_zigzag.*' -v -``` - -CI runs type checks and runtime regressions in one QA matrix entry per strategy. -The shared runtime step sets each strategy's `PYTHONPATH`. QuickAdapter needs -this for direct `unittest` discovery because its model imports `LabelTransformer` -and `Utils` by bare names; ReforceXY resolves its imports without it, so the -setting is optional there. Both strategies' regressions run under -`coverage.py`; the reward-space analysis suite runs separately with `uv`, -without a Freqtrade image. +Every enabled PBRS transition, including neutral steps with retained potential, +uses `gamma * next_potential - prev_potential`. Both `canonical` and +`non_canonical` have zero exit potential; the latter permits optional additives. +The `pbrs_invariant` diagnostic recognizes either zero-exit mode when both +effective additives are disabled. It is a conservative configuration check, not +a guarantee about fitted policies: theoretical invariance also requires an +observable state potential, the learner's discount, and appropriate boundaries. + +An absent learner `gamma` uses the canonical discount. An effective `gamma: null` +is rejected before constructing or replacing environments. Validation follows +HPO overrides and the resumed learner's gamma, so a valid higher-priority value +can supersede a null base value. Learner and environment receive the same +effective discount. +The corrected reward objective invalidates persisted HPO studies and best +parameters from the previous objective identity; incompatible studies are reset +and stale best-parameter payloads are ignored rather than warm-started. -### Coverage gate - -Within the strategy QA matrix, QuickAdapter and ReforceXY both enforce a -coverage gate. The standalone reward-space analysis suite has its own gate; see -[its testing documentation](ReforceXY/reward_space_analysis/tests/README.md). -Each strategy's configuration is that strategy's `.coveragerc`, selected -explicitly because coverage.py looks for `.coveragerc` in the directory it is run -from and does not search parents: - -```shell -export PYTHONPATH=quickadapter/user_data/strategies -export COVERAGE_RCFILE=quickadapter/.coveragerc COVERAGE_FILE=/tmp/.coverage - -sh scripts/run-coverage.sh -s quickadapter/tests -v -``` +## Development -The shared runner gives both coverage commands the same environment. When running -`coverage run` and `coverage report` separately, export both variables for both -commands: prefixing only the run command does not configure the later report, -which then looks in its default data path and fails with `No data to report.` +### Runtime and coverage -Branch coverage is required. Every guard in this codebase is an early return -or a raise, and statement coverage marks a guard covered the instant its `if` -is evaluated. `include_namespace_packages` is required for the opposite -reason: without it coverage skips directories that have no `__init__.py`, so -the `user_data` trees would be absent from the report and the gate would -measure whatever happened to be imported. +Run a strategy's suite inside its matching Freqtrade QA image, with the checkout +mounted at `/workspace` and that directory as the working directory. Use the +shared runner for canonical discovery and the coverage gate: ```shell -# ReforceXY -export COVERAGE_RCFILE=ReforceXY/.coveragerc COVERAGE_FILE=/tmp/.coverage - -sh scripts/run-coverage.sh -s ReforceXY/tests -v +# Inside the ReforceXY QA image; select quickadapter inside its own QA image. +strategy=ReforceXY +export PYTHONPATH=/workspace/$strategy/user_data/strategies +export COVERAGE_RCFILE=$strategy/.coveragerc COVERAGE_FILE=/tmp/.coverage +sh scripts/run-coverage.sh -s "$strategy/tests" -v ``` -Some of the measured tree cannot be reached by a test, and the total therefore -has a ceiling below 100 %. Six sites are recorded here rather than excluded with -`omit`, which the floor guard bans — an exclusion key would shrink the -denominator without the trade-off being visible: - -- **Unreachable while the runner is CPU-only.** `_configure_gpu_memory:757-767` - (`:751-756` returns before the out-of-range arm is evaluated) and - `MyRLEnv.close:4616-4617` (`th.cuda.empty_cache()`). A GPU runner would make - both testable. -- **Version-dependent.** `ReforceXY.py:307`'s `_repair_historic_predictions is - None` branch, arc `307→316`; `:309-315` itself is covered. A Freqtrade - revision adding the hook would make it reachable. -- **Dead at any revision.** The `except OverflowError` arm of - `_potential_transform` (`:3399-3400`); `:249`, the `raise RuntimeError` for an - asynchronous drawer method, guarded by `iscoroutinefunction` rather than by the - version probe; and `create_sampler`'s `case _: assert_never(sampler)` - (`:2244-2245`), since `_Samplers` has exactly two members and `:2218` raises - for anything else. - -CI additionally runs the ReforceXY suite three times with -`FREQAI_QA_SHUFFLE_SEED=1|2|3`. The seed reorders test METHODS within each -class — `unittest` sorts both module and class order alphabetically and -consults no hook for either, so cross-module ordering is not permuted. A leaked -global is still caught: `QaTestCase` restores the process globals in `setUp` -and again in `addCleanup`, so every test starts from the import-time state -whatever ran before it. - -`fail_under` is an absolute floor, never per-module. To change it, re-measure -with the shipped configuration already in place — a measurement taken without -`branch` and `include_namespace_packages` reports a different denominator and -is not a valid input — then update the value and the `# measured` annotation -above it in the same commit. Raise the floor only; a drop needs the reason in -the pull request. Both floor guards refuse a -placeholder, a missing or undated measurement annotation, a measurement below -the floor it justifies, a floor below the current minimum, a floor that is not a -percentage, a `precision` coarse enough to round the total past the floor, a -disabled branch trace, a disabled namespace walk, a source tree that is not the -measured one, a `fail_under` that has drifted into the inert `[run]` section, the -source-level `no cover` or `no branch` pragmas (with or without a colon), -ellipsis-only bodies, additional `TYPE_CHECKING` blocks beyond the existing -import-only block, and any `omit`, `include`, `exclude_lines`, `exclude_also` or -`partial_*` of production code. `ReforceXY`'s guard additionally rejects `patch`, -misspelled options, and a second measurement root (`source_pkgs`, `source_dirs`, -`plugins`). It detects default statement and branch exclusions by parsing each -measured file twice, with and without coverage's defaults. The shared -`scripts/run-coverage.sh` runner is -exercised with real passing and failing coverage reports and a failing test suite; -the checks assert process exit status, not workflow token spelling. +`COVERAGE_RCFILE` explicitly selects the strategy configuration; coverage.py does +not search parent directories. Both coverage commands need the same configuration +and data path. The runner propagates test and coverage-report failures. +QuickAdapter also needs the strategy path for bare-name imports; it is optional +for ReforceXY. + +For a focused debug run without coverage, use `python -m unittest discover` with +the same suite path and an exact `module.Class.method` pattern, for example +`-k 'test_model_pbrs_transitions.*' -v` for ReforceXY. A pattern matching no tests +fails with exit code 5. This debug run does not establish the coverage gate. + +Each strategy QA matrix entry runs its type check and canonical coverage suite. +ReforceXY additionally runs once with `FREQAI_QA_SHUFFLE_SEED=1`. This permutes +methods within test classes, not module or class order. `QaTestCase` restores the +action-mask cache and the Python, NumPy and CPU Torch RNG states before and after +each test. Tests must isolate other mutable state themselves; the alternate order +does not prove the absence of all global-state leaks. + +The ReforceXY gate measures **only `ReforceXY/user_data`**, with branch coverage +and non-imported namespace files included. Tests, the analytical package and +QuickAdapter do not contribute to that percentage. Its minimum is **70%**, with +an approximately 75% coverage objective. Each strategy's current measurement and +configured floor are authoritative in its `.coveragerc`. The reward-space analysis +suite runs separately with `uv` and has +[its own gate](ReforceXY/reward_space_analysis/tests/README.md). + +CPU QA does not execute CUDA device paths. Compatibility guards, including the +rejection of asynchronous drawer methods, are testable; a coverage gap is not +evidence that code is unreachable. Gaps remain in the measured denominator. + +`fail_under` is an absolute aggregate floor, not a per-module floor. Re-measure +using the shipped branch and namespace configuration before raising it; update +the dated measurement annotation and floor together. Do not lower the ReforceXY +minimum or narrow its measurement scope. Its guards reject alternate roots, +plugins, exclusions, source pragmas, placeholder bodies, invalid options and +default exclusions that would remove executable statements or branches. The +shared runner is exercised against passing and failing coverage reports and +failing suites by process exit status. ### Quality checks diff --git a/ReforceXY/.basedpyright/diagnostics.json b/ReforceXY/.basedpyright/diagnostics.json index 1bacced..e0c2a4f 100644 --- a/ReforceXY/.basedpyright/diagnostics.json +++ b/ReforceXY/.basedpyright/diagnostics.json @@ -33,1093 +33,1093 @@ }, { "endCharacter": 5, - "endLine": 2136, + "endLine": 2140, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "No overloads for \"cut\" match the provided arguments", "rule": "reportCallIssue", "severity": "error", "startCharacter": 17, - "startLine": 2131 + "startLine": 2135 }, { "endCharacter": 21, - "endLine": 2133, + "endLine": 2137, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Argument of type \"_Array1D[Any]\" cannot be assigned to parameter \"bins\" of type \"int | Sequence[float] | Index[int] | Index[float] | IntervalIndex[Interval[Any]] | Series[Any]\" in function \"cut\"\n  Type \"_Array1D[Any]\" is not assignable to type \"int | Sequence[float] | Index[int] | Index[float] | IntervalIndex[Interval[Any]] | Series[Any]\"\n    \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"int\"\n    \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"Sequence[float]\"\n    \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"Index[int]\"\n    \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"Index[float]\"\n    \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"IntervalIndex[Interval[Any]]\"\n    \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"Series[Any]\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 13, - "startLine": 2133 + "startLine": 2137 }, { "endCharacter": 5, - "endLine": 2158, + "endLine": 2162, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to declared type \"RewardParams\"\n  Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to type \"RewardParams\"\n    \"dict[bytes, bytes]\" is not assignable to \"dict[str, RewardParamValue]\"\n      Type parameter \"_KT@dict\" is invariant, but \"bytes\" is not the same as \"str\"\n      Type parameter \"_VT@dict\" is invariant, but \"bytes\" is not the same as \"RewardParamValue\"", "rule": "reportAssignmentType", "severity": "error", "startCharacter": 34, - "startLine": 2154 + "startLine": 2158 }, { "endCharacter": 43, - "endLine": 2155, + "endLine": 2159, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "No overloads for \"__init__\" match the provided arguments", "rule": "reportCallIssue", "severity": "error", "startCharacter": 8, - "startLine": 2155 + "startLine": 2159 }, { "endCharacter": 42, - "endLine": 2155, + "endLine": 2159, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"iterable\" of type \"Iterable[list[bytes]]\" in function \"__init__\"\n  Type \"Any | None\" is not assignable to type \"Iterable[list[bytes]]\"\n    \"None\" is incompatible with protocol \"Iterable[list[bytes]]\"\n      \"__iter__\" is not present", "rule": "reportArgumentType", "severity": "error", "startCharacter": 13, - "startLine": 2155 + "startLine": 2159 }, { "endCharacter": 60, - "endLine": 2415, + "endLine": 2419, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Cannot access attribute \"importances_mean\" for class \"dict[Unknown, Bunch]\"\n  Attribute \"importances_mean\" is unknown", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 44, - "startLine": 2415 + "startLine": 2419 }, { "endCharacter": 58, - "endLine": 2416, + "endLine": 2420, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Cannot access attribute \"importances_std\" for class \"dict[Unknown, Bunch]\"\n  Attribute \"importances_std\" is unknown", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 43, - "startLine": 2416 + "startLine": 2420 }, { "endCharacter": 88, - "endLine": 2435, + "endLine": 2439, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Cannot access attribute \"columns\" for class \"NDArray[Unknown]\"\n  Attribute \"columns\" is unknown", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 81, - "startLine": 2435 + "startLine": 2439 }, { "endCharacter": 88, - "endLine": 2435, + "endLine": 2439, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Cannot access attribute \"columns\" for class \"list[Unknown]\"\n  Attribute \"columns\" is unknown", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 81, - "startLine": 2435 + "startLine": 2439 }, { "endCharacter": 17, - "endLine": 2444, + "endLine": 2448, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Object of type \"None\" cannot be called", "rule": "reportOptionalCall", "severity": "error", "startCharacter": 28, - "startLine": 2438 + "startLine": 2442 }, { "endCharacter": 29, - "endLine": 3043, + "endLine": 3047, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "No overloads for \"ptp\" match the provided arguments", "rule": "reportCallIssue", "severity": "error", "startCharacter": 11, - "startLine": 3043 + "startLine": 3047 }, { "endCharacter": 28, - "endLine": 3043, + "endLine": 3047, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"a\" of type \"_ArrayLikeNumeric_co\" in function \"ptp\"\n  Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"_ArrayLikeNumeric_co\"\n    Type \"ExtensionArray\" is not assignable to type \"_ArrayLikeNumeric_co\"\n      \"ExtensionArray\" is incompatible with protocol \"_SupportsArray[dtype[number[Any, Any] | numpy.bool[builtins.bool] | object_ | timedelta64[Any]]]\"\n        \"__array__\" is not present\n      \"ExtensionArray\" is incompatible with protocol \"_NestedSequence[_SupportsArray[dtype[number[Any, Any] | numpy.bool[builtins.bool] | object_ | timedelta64[Any]]]]\"\n        \"__reversed__\" is not present\n        \"count\" is not present\n        \"index\" is not present\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 18, - "startLine": 3043 + "startLine": 3047 }, { "endCharacter": 65, - "endLine": 3052, + "endLine": 3056, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Cannot access attribute \"mean\" for class \"Categorical[object]\"\n  Attribute \"mean\" is unknown", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 61, - "startLine": 3052 + "startLine": 3056 }, { "endCharacter": 65, - "endLine": 3052, + "endLine": 3056, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Cannot access attribute \"mean\" for class \"ExtensionArray\"\n  Attribute \"mean\" is unknown", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 61, - "startLine": 3052 + "startLine": 3056 }, { "endCharacter": 5, - "endLine": 3952, + "endLine": 3956, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to declared type \"RewardParams\"\n  Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to type \"RewardParams\"\n    \"dict[bytes, bytes]\" is not assignable to \"dict[str, RewardParamValue]\"\n      Type parameter \"_KT@dict\" is invariant, but \"bytes\" is not the same as \"str\"\n      Type parameter \"_VT@dict\" is invariant, but \"bytes\" is not the same as \"RewardParamValue\"", "rule": "reportAssignmentType", "severity": "error", "startCharacter": 34, - "startLine": 3948 + "startLine": 3952 }, { "endCharacter": 43, - "endLine": 3949, + "endLine": 3953, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "No overloads for \"__init__\" match the provided arguments", "rule": "reportCallIssue", "severity": "error", "startCharacter": 8, - "startLine": 3949 + "startLine": 3953 }, { "endCharacter": 42, - "endLine": 3949, + "endLine": 3953, "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"iterable\" of type \"Iterable[list[bytes]]\" in function \"__init__\"\n  Type \"Any | None\" is not assignable to type \"Iterable[list[bytes]]\"\n    \"None\" is incompatible with protocol \"Iterable[list[bytes]]\"\n      \"__iter__\" is not present", "rule": "reportArgumentType", "severity": "error", "startCharacter": 13, - "startLine": 3949 + "startLine": 3953 }, { "endCharacter": 22, - "endLine": 1011, + "endLine": 1017, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Cannot assign to attribute \"train_env\" for class \"ReforceXY*\"\n  Expression of type \"VecEnv\" cannot be assigned to attribute \"train_env\" of class \"ReforceXY\"\n    Type \"VecEnv\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n      \"VecEnv\" is not assignable to \"VecMonitor\"\n      \"VecEnv\" is not assignable to \"SubprocVecEnv\"\n      \"VecEnv\" is not assignable to \"Env[Unknown, Unknown]\"", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 13, - "startLine": 1011 + "startLine": 1017 }, { "endCharacter": 37, - "endLine": 1011, + "endLine": 1017, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Cannot assign to attribute \"eval_env\" for class \"ReforceXY*\"\n  Expression of type \"VecEnv | None\" cannot be assigned to attribute \"eval_env\" of class \"ReforceXY\"\n    Type \"VecEnv | None\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n      Type \"VecEnv\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n        \"VecEnv\" is not assignable to \"VecMonitor\"\n        \"VecEnv\" is not assignable to \"SubprocVecEnv\"\n        \"VecEnv\" is not assignable to \"Env[Unknown, Unknown]\"", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 29, - "startLine": 1011 + "startLine": 1017 }, { "endCharacter": 24, - "endLine": 1270, + "endLine": 1276, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"VecEnv\" cannot be assigned to parameter \"eval_env\" of type \"BaseEnvironment\" in function \"__init__\"\n  \"VecEnv\" is not assignable to \"BaseEnvironment\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 16, - "startLine": 1270 + "startLine": 1276 }, { "endCharacter": 38, - "endLine": 1498, + "endLine": 1504, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"obj\" of type \"Sized\" in function \"len\"\n  Type \"Any | None\" is not assignable to type \"Sized\"\n    \"None\" is incompatible with protocol \"Sized\"\n      \"__len__\" is not present", "rule": "reportArgumentType", "severity": "error", "startCharacter": 30, - "startLine": 1498 + "startLine": 1504 }, { "endCharacter": 36, - "endLine": 1502, + "endLine": 1508, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"obj\" of type \"Sized\" in function \"len\"\n  Type \"Any | None\" is not assignable to type \"Sized\"\n    \"None\" is incompatible with protocol \"Sized\"\n      \"__len__\" is not present", "rule": "reportArgumentType", "severity": "error", "startCharacter": 29, - "startLine": 1502 + "startLine": 1508 }, { "endCharacter": 80, - "endLine": 1505, + "endLine": 1511, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"timeframe\" of type \"str\" in function \"steps_to_days\"\n  Type \"Any | None\" is not assignable to type \"str\"\n    \"None\" is not assignable to \"str\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 52, - "startLine": 1505 + "startLine": 1511 }, { "endCharacter": 78, - "endLine": 1506, + "endLine": 1512, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"timeframe\" of type \"str\" in function \"steps_to_days\"\n  Type \"Any | None\" is not assignable to type \"str\"\n    \"None\" is not assignable to \"str\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 50, - "startLine": 1506 + "startLine": 1512 }, { "endCharacter": 80, - "endLine": 1507, + "endLine": 1513, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"timeframe\" of type \"str\" in function \"steps_to_days\"\n  Type \"Any | None\" is not assignable to type \"str\"\n    \"None\" is not assignable to \"str\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 52, - "startLine": 1507 + "startLine": 1513 }, { "endCharacter": 56, - "endLine": 1592, + "endLine": 1599, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\" cannot be assigned to parameter \"eval_env\" of type \"VecEnv | None\" in function \"get_callbacks\"\n  Type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\" is not assignable to type \"VecEnv | None\"\n    Type \"Env[Unknown, Unknown]\" is not assignable to type \"VecEnv | None\"\n      \"Env[Unknown, Unknown]\" is not assignable to \"VecEnv\"\n      \"Env[Unknown, Unknown]\" is not assignable to \"None\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 43, - "startLine": 1592 + "startLine": 1599 }, { "endCharacter": 84, - "endLine": 1612, + "endLine": 1619, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | list[float]\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | list[float]\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"list[float]\" is not assignable to type \"ConvertibleToFloat\"\n      \"list[float]\" is not assignable to \"str\"\n      \"list[float]\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"list[float]\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"list[float]\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 67, - "startLine": 1612 + "startLine": 1619 }, { "endCharacter": 84, - "endLine": 1612, + "endLine": 1619, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | list[float]\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | list[float]\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"list[float]\" is not assignable to type \"ConvertibleToFloat\"\n      \"list[float]\" is not assignable to \"str\"\n      \"list[float]\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"list[float]\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"list[float]\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 67, - "startLine": 1612 + "startLine": 1619 }, { "endCharacter": 44, - "endLine": 2686, + "endLine": 2693, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "\"best_trial_params\" is possibly unbound", "rule": "reportPossiblyUnboundVariable", "severity": "error", "startCharacter": 27, - "startLine": 2686 + "startLine": 2693 }, { "endCharacter": 12, - "endLine": 2706, + "endLine": 2713, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Parameter declaration \"seed\" is obscured by a declaration of the same name", "rule": "reportRedeclaration", "severity": "error", "startCharacter": 8, - "startLine": 2706 + "startLine": 2713 }, { "endCharacter": 16, - "endLine": 2707, + "endLine": 2714, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Parameter declaration \"env_info\" is obscured by a declaration of the same name", "rule": "reportRedeclaration", "severity": "error", "startCharacter": 8, - "startLine": 2707 + "startLine": 2714 }, { "endCharacter": 51, - "endLine": 2750, + "endLine": 2757, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"list[() -> BaseEnvironment]\" cannot be assigned to parameter \"env_fns\" of type \"list[() -> Env[Unknown, Unknown]]\" in function \"__init__\"\n  \"list[() -> BaseEnvironment]\" is not assignable to \"list[() -> Env[Unknown, Unknown]]\"\n    Type parameter \"_T@list\" is invariant, but \"() -> BaseEnvironment\" is not the same as \"() -> Env[Unknown, Unknown]\"\n    Consider switching from \"list\" to \"Sequence\" which is covariant", "rule": "reportArgumentType", "severity": "error", "startCharacter": 42, - "startLine": 2750 + "startLine": 2757 }, { "endCharacter": 53, - "endLine": 2756, + "endLine": 2763, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"list[() -> BaseEnvironment]\" cannot be assigned to parameter \"env_fns\" of type \"list[() -> Env[Unknown, Unknown]]\" in function \"__init__\"\n  \"list[() -> BaseEnvironment]\" is not assignable to \"list[() -> Env[Unknown, Unknown]]\"\n    Type parameter \"_T@list\" is invariant, but \"() -> BaseEnvironment\" is not the same as \"() -> Env[Unknown, Unknown]\"\n    Consider switching from \"list\" to \"Sequence\" which is covariant", "rule": "reportArgumentType", "severity": "error", "startCharacter": 45, - "startLine": 2756 + "startLine": 2763 }, { "endCharacter": 22, - "endLine": 2819, + "endLine": 2826, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Operator \"*\" not supported for \"None\"", "rule": "reportOptionalOperand", "severity": "error", "startCharacter": 15, - "startLine": 2819 + "startLine": 2826 }, { "endCharacter": 96, - "endLine": 2821, + "endLine": 2828, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Operator \"*\" not supported for \"None\"", "rule": "reportOptionalOperand", "severity": "error", "startCharacter": 89, - "startLine": 2821 + "startLine": 2828 }, { "endCharacter": 23, - "endLine": 2824, + "endLine": 2831, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Operator \"*\" not supported for \"None\"", "rule": "reportOptionalOperand", "severity": "error", "startCharacter": 16, - "startLine": 2824 + "startLine": 2831 }, { "endCharacter": 98, - "endLine": 2826, + "endLine": 2833, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Operator \"*\" not supported for \"None\"", "rule": "reportOptionalOperand", "severity": "error", "startCharacter": 91, - "startLine": 2826 + "startLine": 2833 }, { "endCharacter": 44, - "endLine": 2838, + "endLine": 2845, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Operator \"*\" not supported for types \"Any | None\" and \"Any | int | None\"\n  Operator \"*\" not supported for types \"None\" and \"None\"\n  Operator \"*\" not supported for types \"None\" and \"int\"", "rule": "reportOperatorIssue", "severity": "error", "startCharacter": 15, - "startLine": 2838 + "startLine": 2845 }, { "endCharacter": 133, - "endLine": 2840, + "endLine": 2847, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Operator \"*\" not supported for types \"Any | None\" and \"Any | int | None\"\n  Operator \"*\" not supported for types \"None\" and \"None\"\n  Operator \"*\" not supported for types \"None\" and \"int\"", "rule": "reportOperatorIssue", "severity": "error", "startCharacter": 106, - "startLine": 2840 + "startLine": 2847 }, { "endCharacter": 30, - "endLine": 2843, + "endLine": 2850, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Operator \">\" not supported for \"None\"", "rule": "reportOptionalOperand", "severity": "error", "startCharacter": 15, - "startLine": 2843 + "startLine": 2850 }, { "endCharacter": 54, - "endLine": 2908, + "endLine": 2916, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "\"is_pruned\" is not a known attribute of \"None\"", "rule": "reportOptionalMemberAccess", "severity": "error", "startCharacter": 45, - "startLine": 2908 + "startLine": 2916 }, { "endCharacter": 67, - "endLine": 2911, + "endLine": 2919, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "\"use_masking\" is not a known attribute of \"None\"", "rule": "reportOptionalMemberAccess", "severity": "error", "startCharacter": 56, - "startLine": 2911 + "startLine": 2919 }, { "endCharacter": 60, - "endLine": 2919, + "endLine": 2927, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "\"update_best_reward\" is not a known attribute of \"None\"", "rule": "reportOptionalMemberAccess", "severity": "error", "startCharacter": 42, - "startLine": 2919 + "startLine": 2927 }, { "endCharacter": 84, - "endLine": 2919, + "endLine": 2927, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | list[float]\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | list[float]\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"list[float]\" is not assignable to type \"ConvertibleToFloat\"\n      \"list[float]\" is not assignable to \"str\"\n      \"list[float]\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"list[float]\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"list[float]\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 67, - "startLine": 2919 + "startLine": 2927 }, { "endCharacter": 84, - "endLine": 2919, + "endLine": 2927, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | list[float]\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | list[float]\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"list[float]\" is not assignable to type \"ConvertibleToFloat\"\n      \"list[float]\" is not assignable to \"str\"\n      \"list[float]\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"list[float]\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"list[float]\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 67, - "startLine": 2919 + "startLine": 2927 }, { "endCharacter": 46, - "endLine": 2973, + "endLine": 2981, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "\"is_pruned\" is not a known attribute of \"None\"", "rule": "reportOptionalMemberAccess", "severity": "error", "startCharacter": 37, - "startLine": 2973 + "startLine": 2981 }, { "endCharacter": 57, - "endLine": 2976, + "endLine": 2984, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "\"best_mean_reward\" is not a known attribute of \"None\"", "rule": "reportOptionalMemberAccess", "severity": "error", "startCharacter": 41, - "startLine": 2976 + "startLine": 2984 }, { "endCharacter": 37, - "endLine": 2986, + "endLine": 2994, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Cannot assign to attribute \"train_env\" for class \"ReforceXY*\"\n  Type \"None\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n    \"None\" is not assignable to \"VecMonitor\"\n    \"None\" is not assignable to \"SubprocVecEnv\"\n    \"None\" is not assignable to \"Env[Unknown, Unknown]\"", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 33, - "startLine": 2986 + "startLine": 2994 }, { "endCharacter": 36, - "endLine": 2991, + "endLine": 2999, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Cannot assign to attribute \"eval_env\" for class \"ReforceXY*\"\n  Type \"None\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n    \"None\" is not assignable to \"VecMonitor\"\n    \"None\" is not assignable to \"SubprocVecEnv\"\n    \"None\" is not assignable to \"Env[Unknown, Unknown]\"", "rule": "reportAttributeAccessIssue", "severity": "error", "startCharacter": 32, - "startLine": 2991 + "startLine": 2999 }, { "endCharacter": 7, - "endLine": 3036, + "endLine": 3044, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Declaration \"MyRLEnv\" is obscured by a declaration of the same name", "rule": "reportRedeclaration", "severity": "error", "startCharacter": 0, - "startLine": 3036 + "startLine": 3044 }, { "endCharacter": 36, - "endLine": 3048, + "endLine": 3056, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Declaration \"_last_closed_trade_tick\" is obscured by a declaration of the same name", "rule": "reportRedeclaration", "severity": "error", "startCharacter": 13, - "startLine": 3048 + "startLine": 3056 }, { "endCharacter": 13, - "endLine": 3640, + "endLine": 3644, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Method \"reset\" overrides class \"BaseEnvironment\" in an incompatible manner\n  Return type mismatch: base method returns type \"tuple[DataFrame, dict[Unknown, Unknown]]\", override returns type \"tuple[NDArray[float32], dict[str, Any]]\"\n    \"tuple[NDArray[float32], dict[str, Any]]\" is not assignable to \"tuple[DataFrame, dict[Unknown, Unknown]]\"\n      Tuple entry 1 is incorrect type\n        \"ndarray[_AnyShape, dtype[float32]]\" is not assignable to \"DataFrame\"", "rule": "reportIncompatibleMethodOverride", "severity": "error", "startCharacter": 8, - "startLine": 3640 + "startLine": 3644 }, { "endCharacter": 26, - "endLine": 3661, + "endLine": 3665, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Type \"tuple[DataFrame, dict[Unknown, Unknown]]\" is not assignable to return type \"tuple[NDArray[float32], dict[str, Any]]\"\n  \"DataFrame\" is not assignable to \"ndarray[_AnyShape, dtype[float32]]\"", "rule": "reportReturnType", "severity": "error", "startCharacter": 15, - "startLine": 3661 + "startLine": 3665 }, { "endCharacter": 55, - "endLine": 3969, + "endLine": 3973, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 44, - "startLine": 3969 + "startLine": 3973 }, { "endCharacter": 55, - "endLine": 3969, + "endLine": 3973, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 44, - "startLine": 3969 + "startLine": 3973 }, { "endCharacter": 59, - "endLine": 3993, + "endLine": 3997, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 48, - "startLine": 3993 + "startLine": 3997 }, { "endCharacter": 59, - "endLine": 3993, + "endLine": 3997, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 48, - "startLine": 3993 + "startLine": 3997 }, { "endCharacter": 24, - "endLine": 4022, + "endLine": 4026, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Method \"_get_observation\" overrides class \"BaseEnvironment\" in an incompatible manner\n  Return type mismatch: base method returns type \"DataFrame\", override returns type \"NDArray[float32]\"\n    \"ndarray[_AnyShape, dtype[float32]]\" is not assignable to \"DataFrame\"", "rule": "reportIncompatibleMethodOverride", "severity": "error", "startCharacter": 8, - "startLine": 4022 + "startLine": 4026 }, { "endCharacter": 12, - "endLine": 4124, + "endLine": 4128, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Method \"step\" overrides class \"Base5ActionRLEnv\" in an incompatible manner\n  Return type mismatch: base method returns type \"tuple[DataFrame, float, bool, Literal[False], dict[str, Unknown]]\", override returns type \"tuple[NDArray[float32], float, bool, bool, dict[str, Any]]\"\n    \"tuple[NDArray[float32], float, bool, bool, dict[str, Any]]\" is not assignable to \"tuple[DataFrame, float, bool, Literal[False], dict[str, Unknown]]\"\n      Tuple entry 1 is incorrect type\n        \"ndarray[_AnyShape, dtype[float32]]\" is not assignable to \"DataFrame\"", "rule": "reportIncompatibleMethodOverride", "severity": "error", "startCharacter": 8, - "startLine": 4124 + "startLine": 4128 }, { "endCharacter": 20, - "endLine": 4314, + "endLine": 4318, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Method \"action_masks\" overrides class \"BaseEnvironment\" in an incompatible manner\n  Return type mismatch: base method returns type \"list[bool]\", override returns type \"NDArray[bool_]\"\n    \"ndarray[_AnyShape, dtype[bool_]]\" is not assignable to \"list[bool]\"", "rule": "reportIncompatibleMethodOverride", "severity": "error", "startCharacter": 8, - "startLine": 4314 + "startLine": 4318 }, { "endCharacter": 40, - "endLine": 4463, + "endLine": 4467, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "\"Figure\" is not exported from module \"matplotlib.pyplot\"", "rule": "reportPrivateImportUsage", "severity": "error", "startCharacter": 34, - "startLine": 4463 + "startLine": 4467 }, { "endCharacter": 54, - "endLine": 4966, + "endLine": 4970, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 45, - "startLine": 4966 + "startLine": 4970 }, { "endCharacter": 54, - "endLine": 4966, + "endLine": 4970, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 45, - "startLine": 4966 + "startLine": 4970 }, { "endCharacter": 50, - "endLine": 4978, + "endLine": 4982, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"object\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"object\" is not assignable to type \"ConvertibleToFloat\"\n    \"object\" is not assignable to \"str\"\n    \"object\" is incompatible with protocol \"Buffer\"\n      \"__buffer__\" is not present\n    \"object\" is incompatible with protocol \"SupportsFloat\"\n      \"__float__\" is not present\n    \"object\" is incompatible with protocol \"SupportsIndex\"\n      \"__index__\" is not present", "rule": "reportArgumentType", "severity": "error", "startCharacter": 37, - "startLine": 4978 + "startLine": 4982 }, { "endCharacter": 72, - "endLine": 4989, + "endLine": 4993, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 70, - "startLine": 4989 + "startLine": 4993 }, { "endCharacter": 72, - "endLine": 4989, + "endLine": 4993, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 70, - "startLine": 4989 + "startLine": 4993 }, { "endCharacter": 73, - "endLine": 4998, + "endLine": 5002, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 71, - "startLine": 4998 + "startLine": 5002 }, { "endCharacter": 73, - "endLine": 4998, + "endLine": 5002, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 71, - "startLine": 4998 + "startLine": 5002 }, { "endCharacter": 58, - "endLine": 5007, + "endLine": 5011, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 56, - "startLine": 5007 + "startLine": 5011 }, { "endCharacter": 58, - "endLine": 5007, + "endLine": 5011, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 56, - "startLine": 5007 + "startLine": 5011 }, { "endCharacter": 50, - "endLine": 5238, + "endLine": 5242, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Mapping[Unknown, Unknown]\" cannot be assigned to parameter \"src\" of type \"dict[str, Any]\" in function \"deepmerge\"\n  \"Mapping[Unknown, Unknown]\" is not assignable to \"dict[str, Any]\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 49, - "startLine": 5238 + "startLine": 5242 }, { "endCharacter": 20, - "endLine": 5415, + "endLine": 5419, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 18, - "startLine": 5415 + "startLine": 5419 }, { "endCharacter": 20, - "endLine": 5415, + "endLine": 5419, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 18, - "startLine": 5415 + "startLine": 5419 }, { "endCharacter": 59, - "endLine": 5420, + "endLine": 5424, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 31, - "startLine": 5420 + "startLine": 5424 }, { "endCharacter": 59, - "endLine": 5420, + "endLine": 5424, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 31, - "startLine": 5420 + "startLine": 5424 }, { "endCharacter": 65, - "endLine": 5421, + "endLine": 5425, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 34, - "startLine": 5421 + "startLine": 5425 }, { "endCharacter": 65, - "endLine": 5421, + "endLine": 5425, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 34, - "startLine": 5421 + "startLine": 5425 }, { "endCharacter": 57, - "endLine": 5422, + "endLine": 5426, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 31, - "startLine": 5422 + "startLine": 5426 }, { "endCharacter": 57, - "endLine": 5422, + "endLine": 5426, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 31, - "startLine": 5422 + "startLine": 5426 }, { "endCharacter": 63, - "endLine": 5424, + "endLine": 5428, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 34, - "startLine": 5424 + "startLine": 5428 }, { "endCharacter": 63, - "endLine": 5424, + "endLine": 5428, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 34, - "startLine": 5424 + "startLine": 5428 }, { "endCharacter": 61, - "endLine": 5426, + "endLine": 5430, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 32, - "startLine": 5426 + "startLine": 5430 }, { "endCharacter": 61, - "endLine": 5426, + "endLine": 5430, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 32, - "startLine": 5426 + "startLine": 5430 }, { "endCharacter": 67, - "endLine": 5427, + "endLine": 5431, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 36, - "startLine": 5427 + "startLine": 5431 }, { "endCharacter": 67, - "endLine": 5427, + "endLine": 5431, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 36, - "startLine": 5427 + "startLine": 5431 }, { "endCharacter": 73, - "endLine": 5428, + "endLine": 5432, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 39, - "startLine": 5428 + "startLine": 5432 }, { "endCharacter": 73, - "endLine": 5428, + "endLine": 5432, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 39, - "startLine": 5428 + "startLine": 5432 }, { "endCharacter": 61, - "endLine": 5429, + "endLine": 5433, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 33, - "startLine": 5429 + "startLine": 5433 }, { "endCharacter": 61, - "endLine": 5429, + "endLine": 5433, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 33, - "startLine": 5429 + "startLine": 5433 }, { "endCharacter": 89, - "endLine": 5436, + "endLine": 5440, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 52, - "startLine": 5436 + "startLine": 5440 }, { "endCharacter": 89, - "endLine": 5436, + "endLine": 5440, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 52, - "startLine": 5436 + "startLine": 5440 }, { "endCharacter": 83, - "endLine": 5437, + "endLine": 5441, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 49, - "startLine": 5437 + "startLine": 5441 }, { "endCharacter": 83, - "endLine": 5437, + "endLine": 5441, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 49, - "startLine": 5437 + "startLine": 5441 }, { "endCharacter": 57, - "endLine": 5462, + "endLine": 5466, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 31, - "startLine": 5462 + "startLine": 5466 }, { "endCharacter": 57, - "endLine": 5462, + "endLine": 5466, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 31, - "startLine": 5462 + "startLine": 5466 }, { "endCharacter": 65, - "endLine": 5463, + "endLine": 5467, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 34, - "startLine": 5463 + "startLine": 5467 }, { "endCharacter": 65, - "endLine": 5463, + "endLine": 5467, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 34, - "startLine": 5463 + "startLine": 5467 }, { "endCharacter": 67, - "endLine": 5465, + "endLine": 5469, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 35, - "startLine": 5465 + "startLine": 5469 }, { "endCharacter": 67, - "endLine": 5465, + "endLine": 5469, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 35, - "startLine": 5465 + "startLine": 5469 }, { "endCharacter": 87, - "endLine": 5468, + "endLine": 5472, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 46, - "startLine": 5468 + "startLine": 5472 }, { "endCharacter": 87, - "endLine": 5468, + "endLine": 5472, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 46, - "startLine": 5468 + "startLine": 5472 }, { "endCharacter": 93, - "endLine": 5469, + "endLine": 5473, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 49, - "startLine": 5469 + "startLine": 5473 }, { "endCharacter": 93, - "endLine": 5469, + "endLine": 5473, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 49, - "startLine": 5469 + "startLine": 5473 }, { "endCharacter": 89, - "endLine": 5470, + "endLine": 5474, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 47, - "startLine": 5470 + "startLine": 5474 }, { "endCharacter": 89, - "endLine": 5470, + "endLine": 5474, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n    Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsFloat\"\n        \"__float__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 47, - "startLine": 5470 + "startLine": 5474 }, { "endCharacter": 89, - "endLine": 5471, + "endLine": 5475, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 46, - "startLine": 5471 + "startLine": 5475 }, { "endCharacter": 89, - "endLine": 5471, + "endLine": 5475, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 46, - "startLine": 5471 + "startLine": 5475 }, { "endCharacter": 75, - "endLine": 5472, + "endLine": 5476, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"", "rule": "reportArgumentType", "severity": "error", "startCharacter": 39, - "startLine": 5472 + "startLine": 5476 }, { "endCharacter": 75, - "endLine": 5472, + "endLine": 5476, "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py", "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n  Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n    Type \"None\" is not assignable to type \"ConvertibleToInt\"\n      \"None\" is not assignable to \"str\"\n      \"None\" is incompatible with protocol \"Buffer\"\n        \"__buffer__\" is not present\n      \"None\" is incompatible with protocol \"SupportsInt\"\n        \"__int__\" is not present\n      \"None\" is incompatible with protocol \"SupportsIndex\"\n ...", "rule": "reportArgumentType", "severity": "error", "startCharacter": 39, - "startLine": 5472 + "startLine": 5476 }, { "endCharacter": 33, diff --git a/ReforceXY/.coveragerc b/ReforceXY/.coveragerc index e9e85fd..a922b88 100644 --- a/ReforceXY/.coveragerc +++ b/ReforceXY/.coveragerc @@ -8,5 +8,5 @@ relative_files = True show_missing = True precision = 1 include_namespace_packages = True -# measured 2026-10-02 at 59.1% — raise only, never lower -fail_under = 59 \ No newline at end of file +# measured 2026-10-02 at 75.3% — raise only, never lower +fail_under = 70 \ No newline at end of file diff --git a/ReforceXY/reward_space_analysis/README.md b/ReforceXY/reward_space_analysis/README.md index 5000b4e..2220b58 100644 --- a/ReforceXY/reward_space_analysis/README.md +++ b/ReforceXY/reward_space_analysis/README.md @@ -619,11 +619,15 @@ configuration. ### PBRS Configuration -Canonical mode enforces terminal release (Φ terminal ≈ 0) and suppresses -entry/exit additive terms. - -Non-canonical exit modes can introduce non-zero terminal shaping; enable -additives only when you want those extra terms to contribute. +Every enabled PBRS transition uses `gamma * next_potential - prev_potential`, +including neutral self-loops that retain potential. Termination clears the +potential in every exit mode. `canonical` suppresses entry/exit additives; +`non_canonical` also has zero exit potential but permits those optional terms. +Both zero-exit modes without effective additives are eligible for observed PBRS +verification. The report's canonical classification describes the shaping form, +not just the literal exit-mode name, and does not guarantee fitted-policy +equivalence. Residual exit modes retain history-dependent potential and are not +classified as generally policy-invariant. ### Real Data Comparison diff --git a/ReforceXY/reward_space_analysis/reward_space_analysis.py b/ReforceXY/reward_space_analysis/reward_space_analysis.py index c4f6854..563570e 100644 --- a/ReforceXY/reward_space_analysis/reward_space_analysis.py +++ b/ReforceXY/reward_space_analysis/reward_space_analysis.py @@ -1434,7 +1434,7 @@ def calculate_reward( breakdown.exit_component += liquidation_reward breakdown.exit_pnl = terminal_context.current_pnl - # Apply PBRS only if enabled and not neutral self-loop + # Resolve effective PBRS and additive enablement for every transition. exit_mode = _resolve_exit_potential_mode( params.get("exit_potential_mode", DEFAULT_MODEL_REWARD_PARAMETERS["exit_potential_mode"]), warn_invalid=True, @@ -1459,7 +1459,9 @@ def calculate_reward( # Neutral self-loops retain potential except at the terminal boundary. breakdown.prev_potential = prev_potential breakdown.next_potential = 0.0 if terminated else prev_potential - breakdown.reward_shaping = -prev_potential if terminated else 0.0 + breakdown.reward_shaping = ( + _get_potential_gamma(params) * breakdown.next_potential - prev_potential + ) breakdown.pbrs_delta = breakdown.reward_shaping breakdown.total = base_reward + breakdown.reward_shaping return breakdown @@ -1734,7 +1736,9 @@ def simulate_samples( entry_enabled_raw, exit_enabled_raw, ) - pbrs_invariant = bool(exit_mode == "canonical" and not (entry_enabled or exit_enabled)) + pbrs_invariant = bool( + exit_mode in ("canonical", "non_canonical") and not (entry_enabled or exit_enabled) + ) max_idle_duration_candles = get_max_idle_duration_candles( params, max_trade_duration_candles=max_trade_duration_candles @@ -4441,7 +4445,7 @@ def write_complete_statistical_analysis( } ) canonical_configuration = classification_metadata_valid and ( - exit_potential_mode == "canonical" + exit_potential_mode in ("canonical", "non_canonical") and not (entry_additive_effective or exit_additive_effective) ) observed_additive_issues = [] @@ -4475,7 +4479,7 @@ def write_complete_statistical_analysis( ) elif not canonical_configuration: reasons = [] - if exit_potential_mode != "canonical": + if exit_potential_mode not in ("canonical", "non_canonical"): reasons.append(f"exit_potential_mode='{exit_potential_mode}'") if entry_additive_effective or exit_additive_effective: additive_types = [] diff --git a/ReforceXY/reward_space_analysis/tests/pbrs/test_pbrs.py b/ReforceXY/reward_space_analysis/tests/pbrs/test_pbrs.py index 7f2eb92..05af8aa 100644 --- a/ReforceXY/reward_space_analysis/tests/pbrs/test_pbrs.py +++ b/ReforceXY/reward_space_analysis/tests/pbrs/test_pbrs.py @@ -66,6 +66,7 @@ class TestSimulationParity(RewardSpaceTestBase): for enabled in (False, True): params = self.base_params( hold_potential_enabled=enabled, + potential_gamma=0.8, exit_potential_mode="retain_previous", entry_additive_enabled=False, exit_additive_enabled=False, @@ -101,12 +102,20 @@ class TestSimulationParity(RewardSpaceTestBase): self.assertAlmostEqualFloat( terminal.reward_shaping, -0.25 if enabled else 0.0, tolerance=TOLERANCE.GENERIC_EQ ) + self.assertAlmostEqualFloat( + ongoing.reward_shaping, -0.05 if enabled else 0.0, tolerance=TOLERANCE.GENERIC_EQ + ) + self.assertAlmostEqualFloat( + ongoing.total - ongoing.base_reward, + -0.05 if enabled else 0.0, + tolerance=TOLERANCE.GENERIC_EQ, + ) self.assertAlmostEqualFloat( terminal.pbrs_delta, terminal.reward_shaping, tolerance=TOLERANCE.GENERIC_EQ ) self.assertAlmostEqualFloat( terminal.total - ongoing.total, - -0.25 if enabled else 0.0, + -0.20 if enabled else 0.0, tolerance=TOLERANCE.GENERIC_EQ, ) self.assertEqual(terminal.entry_additive + terminal.exit_additive, 0.0) @@ -872,37 +881,33 @@ class TestPBRS(RewardSpaceTestBase): # ---------------- Invariance flags (simulate_samples) ---------------- # - def test_canonical_invariance_flag(self): - """Canonical mode + no additives -> invariant flag True per-sample. - - Note: `simulate_samples()` generates synthetic trajectories (coherent episodes). - This test only verifies the per-sample invariance flag and numeric stability; it does not - assert any telescoping/zero-sum property for the shaping term. - """ - - params = self.base_params( - exit_potential_mode="canonical", - entry_additive_enabled=False, - exit_additive_enabled=False, - hold_potential_enabled=True, - ) - df = simulate_samples( - params={**params, "max_trade_duration_candles": 100}, - num_samples=SCENARIOS.SAMPLE_SIZE_MEDIUM, - seed=SEEDS.BASE, - base_factor=PARAMS.BASE_FACTOR, - profit_aim=PARAMS.PROFIT_AIM, - risk_reward_ratio=PARAMS.RISK_REWARD_RATIO, - max_duration_ratio=2.0, - trading_mode="margin", - pnl_base_std=PARAMS.PNL_STD, - pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE, - ) - unique_flags = set(df["pbrs_invariant"].unique().tolist()) - self.assertEqual(unique_flags, {True}, f"Unexpected invariant flags: {unique_flags}") - for v in df["reward_shaping"].tolist(): - self.assertFinite(float(v), name="reward_shaping") - self.assertLessEqual(float(df["reward_shaping"].abs().max()), PBRS.MAX_ABS_SHAPING) + def test_zero_exit_modes_without_additives_match_verified_episode_flags(self): + for mode in ("canonical", "non_canonical"): + with self.subTest(mode=mode): + params = self.base_params( + exit_potential_mode=mode, + entry_additive_enabled=False, + exit_additive_enabled=False, + hold_potential_enabled=True, + max_trade_duration_candles=100, + ) + df = simulate_samples( + params=params, + num_samples=SCENARIOS.SAMPLE_SIZE_MEDIUM, + seed=SEEDS.BASE, + base_factor=PARAMS.BASE_FACTOR, + profit_aim=PARAMS.PROFIT_AIM, + risk_reward_ratio=PARAMS.RISK_REWARD_RATIO, + max_duration_ratio=2.0, + trading_mode="margin", + pnl_base_std=PARAMS.PNL_STD, + pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE, + ) + self.assertEqual(set(df["pbrs_invariant"]), {True}) + evidence = reward_space_analysis.verify_pbrs_trajectory( + df, _get_potential_gamma(params) + ) + self.assertTrue(evidence["verified"], evidence["reason"]) def test_non_canonical_flag_false_and_sum_nonzero(self): """Non-canonical mode -> invariant flags False and Σ shaping non-zero.""" diff --git a/ReforceXY/tests/test_coverage_floor.py b/ReforceXY/tests/test_coverage_floor.py index 3409ca6..431d4ba 100644 --- a/ReforceXY/tests/test_coverage_floor.py +++ b/ReforceXY/tests/test_coverage_floor.py @@ -20,11 +20,8 @@ BRANCH = "branch" NAMESPACE_PACKAGES = "include_namespace_packages" OMIT = "omit" INCLUDE = "include" -# This is the floor ReforceXY has actually reached, not the target. The gate is only as -# strong as the measurement behind it, so `MINIMUM_FLOOR` starts at the calibrated value -# and may only rise. Reaching the agreed 70% target is a separate delivery; until then a -# lower value here would be a control the project believes it has and does not. -MINIMUM_FLOOR = 59.0 +# Minimum accepted branch-inclusive coverage over the complete measured tree. +MINIMUM_FLOOR = 70.0 EXCLUSION_KEYS = ( OMIT, INCLUDE, diff --git a/ReforceXY/tests/test_model_optuna_lifecycle.py b/ReforceXY/tests/test_model_optuna_lifecycle.py new file mode 100644 index 0000000..930917d --- /dev/null +++ b/ReforceXY/tests/test_model_optuna_lifecycle.py @@ -0,0 +1,275 @@ +"""Study reuse, purge, warm-start and fallback with real local Optuna storage.""" + +import tempfile +import unittest +from types import SimpleNamespace +from unittest import mock + +import optuna +import pandas as pd +from optuna.distributions import FloatDistribution +from optuna.exceptions import DuplicatedStudyError +from optuna.storages import RDBStorage +from optuna.trial import TrialState +from qa_support import PAIR, QaTestCase, model_config + +from ReforceXY.user_data.freqaimodels.ReforceXY import ReforceXY + +PPO_PARAMS = { + "learning_rate": 0.001, + "clip_range": 0.2, + "n_steps": 512, + "batch_size": 64, + "gamma": 0.83, + "ent_coef": 0.01, + "n_epochs": 1, + "gae_lambda": 0.9, + "max_grad_norm": 0.5, + "vf_coef": 0.5, +} + + +class OptunaLifecycleTest(QaTestCase): + def model(self, backend="file", **options): + temporary = tempfile.TemporaryDirectory() + self.addCleanup(temporary.cleanup) + config = model_config( + temporary.name, + freqai={ + "model_training_parameters": {"gamma": 0.41, "learning_rate": 0.03}, + "rl_config_optuna": { + "enabled": True, + "storage": backend, + "sampler": "tpe", + "seed": 42, + "n_trials": 1, + "n_startup_trials": 2, + **options, + }, + }, + ) + model = ReforceXY(config=config) + self.addCleanup(model.close_envs) + return model + + def storage(self, model, pair=PAIR): + storage = model.create_storage(pair) + if isinstance(storage, RDBStorage): + self.addCleanup(storage.engine.dispose) + return storage + + def reopened(self, model, pair=PAIR): + return optuna.load_study( + study_name=f"{model.freqai_info['identifier']}-{pair}", + storage=self.storage(model, pair), + ) + + def objective(self, trial, *_): + # Isolate only the expensive training boundary. Suggestions, study state, + # winner selection and all on-disk persistence remain real Optuna code. + gamma = trial.suggest_float("gamma", 0.5, 1.0) + for name, value in PPO_PARAMS.items(): + if name != "gamma": + trial.suggest_categorical(name, [value]) + return gamma + + def optimize(self, model, *, pair=PAIR, objective=None): + with mock.patch.object(model, "objective", side_effect=objective or self.objective): + return model.optimize(SimpleNamespace(pair=pair), 512, pd.DataFrame(), pd.DataFrame()) + + def test_reuse_keeps_previous_winner_and_sampled_values_override_user_configuration(self): + for backend in ("sqlite", "file"): + with self.subTest(backend=backend): + model = self.model(backend) + first = self.optimize(model) + first_gamma = first["gamma"] + model.rl_config_optuna["seed"] = 43 + second = self.optimize(model) + study = self.reopened(model) + self.assertEqual([trial.number for trial in study.trials], [0, 1]) + self.assertLess(study.trials[1].value, study.trials[0].value) + self.assertEqual(second["gamma"], first_gamma) + self.assertEqual(second["learning_rate"](0.5), 0.001) + self.assertEqual(second["device"], "cpu") + self.assertEqual(second["policy_kwargs"]["net_arch"], {"pi": [8], "vf": [8]}) + self.assertEqual( + study.user_attrs["objective_identity"], ReforceXY._OPTUNA_OBJECTIVE_IDENTITY + ) + self.assertEqual(model.load_best_trial_params(PAIR)["gamma"], first_gamma) + self.assertEqual(model._load_optuna_retrain_counters(PAIR), {"BTC_USDT": 2}) + + def test_continuous_mode_resets_trials_instead_of_reusing_a_previous_winner(self): + model = self.model(continuous=True) + first = self.optimize(model) + model.rl_config_optuna["seed"] = 43 + second = self.optimize(model) + study = self.reopened(model) + self.assertEqual([trial.number for trial in study.trials], [0]) + self.assertNotEqual(second["gamma"], first["gamma"]) + self.assertEqual(study.best_value, second["gamma"]) + + def test_periodic_purge_warm_starts_even_when_warm_start_is_disabled(self): + model = self.model(purge_period=2, warm_start=False) + first = self.optimize(model) + self.optimize(model, pair="ETH/USDT") + model.rl_config_optuna["seed"] = 43 + second = self.optimize(model) + study = self.reopened(model) + self.assertEqual([trial.number for trial in study.trials], [0]) + self.assertEqual(second["gamma"], first["gamma"]) + self.assertEqual(study.trials[0].params["gamma"], first["gamma"]) + self.optimize(model) + self.assertEqual([trial.number for trial in self.reopened(model).trials], [0, 1]) + self.assertEqual(len(self.reopened(model, "ETH/USDT").trials), 1) + self.assertEqual(model._load_optuna_retrain_counters(PAIR), {"BTC_USDT": 3, "ETH_USDT": 1}) + + def test_warm_start_uses_saved_coordinates_and_missing_coordinates_do_not_block_trials(self): + for saved in (False, True): + with self.subTest(saved=saved): + model = self.model(warm_start=True) + if saved: + model.save_best_trial_params({**PPO_PARAMS, "gamma": 0.91}, PAIR) + result = self.optimize(model) + study = self.reopened(model) + self.assertEqual(study.trials[0].state, TrialState.COMPLETE) + if saved: + self.assertEqual(result["gamma"], 0.91) + self.assertEqual(study.best_value, 0.91) + else: + self.assertNotEqual(result["gamma"], 0.91) + + def test_incompatible_or_missing_study_identity_discards_old_trials(self): + for identity in (None, "terminal-liquidation-risk-normalized-trained-policy-v3"): + with self.subTest(identity=identity): + model = self.model() + study = optuna.create_study( + study_name=f"{model.freqai_info['identifier']}-{PAIR}", + storage=self.storage(model), + direction="maximize", + ) + if identity is not None: + study.set_user_attr("objective_identity", identity) + study.add_trial( + optuna.create_trial( + value=100.0, + params={"gamma": 0.99}, + distributions={"gamma": FloatDistribution(0.5, 1.0)}, + ) + ) + result = self.optimize(model) + reopened = self.reopened(model) + self.assertEqual(len(reopened.trials), 1) + self.assertLess(reopened.best_value, 1.0) + self.assertEqual( + reopened.user_attrs["objective_identity"], ReforceXY._OPTUNA_OBJECTIVE_IDENTITY + ) + self.assertNotEqual(result["gamma"], 0.99) + + def test_failed_identity_reset_fails_closed_without_reusing_incompatible_trials(self): + model = self.model() + storage = self.storage(model) + study = optuna.create_study( + study_name=f"{model.freqai_info['identifier']}-{PAIR}", + storage=storage, + ) + study.set_user_attr("objective_identity", "legacy-objective") + with ( + mock.patch.object( + type(storage), "delete_study", side_effect=OSError("read-only storage") + ), + self.assertRaises(DuplicatedStudyError), + ): + self.optimize(model) + self.assertEqual(self.reopened(model).user_attrs["objective_identity"], "legacy-objective") + + def test_pruned_trials_fall_back_to_saved_parameters_or_return_no_candidate(self): + def pruned(trial, *_): + trial.report(0.0, 1) + raise optuna.TrialPruned() + + model = self.model() + self.assertIsNone(self.optimize(model, objective=pruned)) + model.save_best_trial_params(PPO_PARAMS, PAIR) + result = self.optimize(model, objective=pruned) + self.assertEqual(result["gamma"], 0.83) + self.assertEqual( + [trial.state for trial in self.reopened(model).trials], [TrialState.PRUNED] * 2 + ) + self.assertEqual(model.load_best_trial_params(PAIR), PPO_PARAMS) + + def test_failed_trial_uses_saved_fallback_even_when_study_has_a_previous_winner(self): + def failed(*_): + raise RuntimeError("training failure") + + model = self.model() + first = self.optimize(model) + model.save_best_trial_params({**PPO_PARAMS, "gamma": 0.93}, PAIR) + result = self.optimize(model, objective=failed) + study = self.reopened(model) + self.assertEqual( + [trial.state for trial in study.trials], [TrialState.COMPLETE, TrialState.FAIL] + ) + self.assertEqual(study.best_params["gamma"], first["gamma"]) + self.assertEqual(result["gamma"], 0.93) + self.assertEqual(model.load_best_trial_params(PAIR)["gamma"], 0.93) + + def test_keyboard_interrupt_reuses_only_completed_candidates(self): + def interrupted(*_): + raise KeyboardInterrupt() + + model = self.model() + self.assertIsNone(self.optimize(model, objective=interrupted)) + first = self.optimize(model) + result = self.optimize(model, objective=interrupted) + self.assertEqual(result["gamma"], first["gamma"]) + self.assertEqual( + [trial.state for trial in self.reopened(model).trials], + [TrialState.FAIL, TrialState.COMPLETE, TrialState.FAIL], + ) + + def test_shared_lstm_disables_a_separate_critic_in_the_selected_parameters(self): + def recurrent(trial, *_): + value = self.objective(trial) + trial.suggest_categorical("lstm_hidden_size", [8]) + trial.suggest_categorical("n_lstm_layers", [1]) + trial.suggest_categorical("enable_critic_lstm", [True]) + return value + + model = self.model() + model.model_type = "RecurrentPPO" + model.model_training_parameters["policy_kwargs"]["shared_lstm"] = True + model._model_params_cache = None + result = self.optimize(model, objective=recurrent) + self.assertFalse(result["policy_kwargs"]["enable_critic_lstm"]) + self.assertFalse(model.load_best_trial_params(PAIR)["enable_critic_lstm"]) + + def test_value_based_lifecycle_derives_gradient_steps_from_sampled_subsampling(self): + def value_based(trial, *_): + gamma = trial.suggest_float("gamma", 0.5, 1.0) + for name, value in { + "learning_rate": 0.001, + "batch_size": 8, + "buffer_size": 64, + "train_freq": 8, + "subsample_steps": 3, + "exploration_fraction": 0.2, + "exploration_initial_eps": 0.8, + "exploration_final_eps": 0.1, + "target_update_interval": 8, + "learning_starts": 0, + }.items(): + trial.suggest_categorical(name, [value]) + return gamma + + model = self.model() + model.model_type = "DQN" + model.model_training_parameters = {"device": "cpu", "policy_kwargs": {"net_arch": [8]}} + model._model_params_cache = None + result = self.optimize(model, objective=value_based) + self.assertEqual(result["gradient_steps"], 3) + self.assertNotIn("subsample_steps", result) + self.assertEqual(self.reopened(model).trials[0].state, TrialState.COMPLETE) + + +if __name__ == "__main__": + unittest.main() diff --git a/ReforceXY/tests/test_model_optuna_storage.py b/ReforceXY/tests/test_model_optuna_storage.py new file mode 100644 index 0000000..45e97f6 --- /dev/null +++ b/ReforceXY/tests/test_model_optuna_storage.py @@ -0,0 +1,346 @@ +"""Durable Optuna studies, recovery and atomic best-parameter persistence.""" + +import json +import os +import subprocess +import sys +import tempfile +import unittest +from contextlib import contextmanager +from datetime import datetime, timezone +from pathlib import Path +from unittest import mock + +import optuna +from optuna.distributions import FloatDistribution +from optuna.storages import RDBStorage +from optuna.trial import TrialState +from qa_support import PAIR, REPO_ROOT, QaTestCase, model_config + +from ReforceXY.user_data.freqaimodels.ReforceXY import ReforceXY + +MODULE = "ReforceXY.user_data.freqaimodels.ReforceXY" + + +class OptunaStorageTest(QaTestCase): + def setUp(self): + super().setUp() + temporary = tempfile.TemporaryDirectory() + self.addCleanup(temporary.cleanup) + self.root = Path(temporary.name) + self.model = ReforceXY(config=model_config(temporary.name)) + self.addCleanup(self.model.close_envs) + + def storage(self, backend="file", pair=PAIR): + self.model.rl_config_optuna["storage"] = backend + storage = self.model.create_storage(pair) + if isinstance(storage, RDBStorage): + self.addCleanup(storage.engine.dispose) + return storage + + def study(self, storage, name="winner"): + study = optuna.create_study(storage=storage, study_name=name, direction="maximize") + study.add_trial( + optuna.create_trial( + value=2.0, + params={"gamma": 0.83}, + distributions={"gamma": FloatDistribution(0.0, 1.0)}, + ) + ) + return study + + def test_backends_reopen_winners_and_keep_studies_sharing_a_base_asset_isolated(self): + for backend in ("sqlite", "file"): + with self.subTest(backend=backend): + first = self.study(self.storage(backend), "BTC/USDT") + other = self.study(self.storage(backend, "BTC/EUR"), "BTC/EUR") + first.add_trial(optuna.create_trial(value=4.0)) + reopened = optuna.load_study(study_name="BTC/USDT", storage=self.storage(backend)) + self.assertEqual(reopened.best_value, 4.0) + self.assertEqual(len(reopened.trials), 2) + self.assertEqual(other.best_value, 2.0) + self.assertEqual(len(other.trials), 1) + ReforceXY.delete_study("BTC/USDT", self.storage(backend)) + with self.assertRaises(KeyError): + optuna.load_study(study_name="BTC/USDT", storage=self.storage(backend)) + ReforceXY.delete_study("BTC/USDT", self.storage(backend)) + self.assertEqual(other.best_params, {"gamma": 0.83}) + + def test_only_complete_trials_make_a_best_trial_available(self): + storage = self.storage() + study = optuna.create_study(storage=storage, study_name="availability") + self.assertFalse(ReforceXY.study_has_best_trial(None)) + self.assertFalse(ReforceXY.study_has_best_trial(study)) + study.add_trial(optuna.create_trial(state=TrialState.FAIL)) + study.add_trial(optuna.create_trial(state=TrialState.PRUNED)) + self.assertFalse(ReforceXY.study_has_best_trial(study)) + study.add_trial(optuna.create_trial(value=3.0)) + self.assertTrue(ReforceXY.study_has_best_trial(study)) + self.assertEqual(study.best_value, 3.0) + + def test_retrain_counters_survive_restart_and_recover_invalid_payloads(self): + model = self.model + self.assertEqual(model._increment_optuna_retrain_counter("BTC/USDT:USDT"), 1) + self.assertEqual(model._increment_optuna_retrain_counter("ETH/USDT"), 1) + restarted = ReforceXY.__new__(ReforceXY) + restarted.full_path = model.full_path + self.assertEqual(restarted._increment_optuna_retrain_counter("BTC/USDT:USDT"), 2) + self.assertEqual( + restarted._load_optuna_retrain_counters(PAIR), {"BTC_USDT_USDT": 2, "ETH_USDT": 1} + ) + path = model._optuna_retrain_counters_path() + path.write_text(json.dumps({"BTC_USDT": 4, "bad": "4", "other": []}), encoding="utf-8") + self.assertEqual(model._increment_optuna_retrain_counter(PAIR), 5) + self.assertEqual(model._load_optuna_retrain_counters(PAIR), {"BTC_USDT": 5}) + for payload in ("[]", "{broken"): + path.write_text(payload, encoding="utf-8") + self.assertEqual(model._increment_optuna_retrain_counter(PAIR), 1) + path.unlink() + path.mkdir() + self.assertEqual(model._increment_optuna_retrain_counter(PAIR), 1) + self.assertTrue(path.is_dir()) + + def test_corrupt_journal_tails_are_preserved_and_new_trials_survive_restart(self): + path = self.model.full_path / "optuna-BTC.log" + for index, tail in enumerate( + ( + b"partial", + b"{invalid}\n", + b"\n", + b"[]\n", + b'{"op_code":true}\n', + b'{"op_code":99}\n', + b"{}\n", + ) + ): + with self.subTest(tail=tail): + path.unlink(missing_ok=True) + self.study(self.storage(), f"old-{index}") + with path.open("ab") as stream: + stream.write(tail) + corrupt = path.read_bytes() + recovered = self.storage() + quarantines = list(path.parent.glob(path.name + ".corrupt-*")) + self.assertIn(corrupt, [item.read_bytes() for item in quarantines]) + with self.assertRaises(KeyError): + optuna.load_study(study_name=f"old-{index}", storage=recovered) + self.study(recovered, f"fresh-{index}") + reopened = optuna.load_study(study_name=f"fresh-{index}", storage=self.storage()) + self.assertEqual(reopened.best_params, {"gamma": 0.83}) + self.assertEqual(reopened.best_value, 2.0) + + def test_interior_corruption_is_detected_by_real_journal_replay(self): + self.study(self.storage()) + path = self.model.full_path / "optuna-BTC.log" + records = path.read_bytes().splitlines(keepends=True) + corrupt = records[0] + b"{invalid}\n" + b"".join(records[1:]) + path.write_bytes(corrupt) + self.assertFalse(ReforceXY._journal_has_corrupt_tail(path)) + recovered = self.storage() + with self.assertRaises(KeyError): + optuna.load_study(study_name="winner", storage=recovered) + self.assertEqual(next(path.parent.glob(path.name + ".corrupt-*")).read_bytes(), corrupt) + + def test_large_journal_records_replay_without_false_quarantine(self): + study = self.study(self.storage()) + padding = "x" * (ReforceXY._JOURNAL_TAIL_PROBE_BYTES + 1024) + study.set_user_attr("padding", padding) + path = self.model.full_path / "optuna-BTC.log" + reopened = optuna.load_study(study_name="winner", storage=self.storage()) + self.assertEqual(reopened.user_attrs["padding"], padding) + self.assertEqual(reopened.best_value, 2.0) + self.assertEqual(list(path.parent.glob(path.name + ".corrupt-*")), []) + # A single oversized malformed operation passes the bounded tail probe, + # but must still be rejected by full replay and preserved for diagnosis. + path.write_text(json.dumps({"padding": padding}) + "\n", encoding="utf-8") + corrupt = path.read_bytes() + recovered = self.storage() + self.study(recovered, "replacement") + self.assertEqual(next(path.parent.glob(path.name + ".corrupt-*")).read_bytes(), corrupt) + self.assertEqual( + optuna.load_study(study_name="replacement", storage=self.storage()).best_value, 2.0 + ) + + def test_quarantine_collision_never_overwrites_existing_evidence(self): + path = self.model.full_path / "optuna-BTC.log" + now = datetime(2026, 1, 1, tzinfo=timezone.utc) + with mock.patch(MODULE + ".datetime") as clock: + clock.now.return_value = now + path.write_bytes(b"first truncated operation") + self.storage() + first = next(path.parent.glob(path.name + ".corrupt-*")) + path.write_bytes(b"second truncated operation") + self.storage() + self.assertEqual(first.read_bytes(), b"first truncated operation") + self.assertEqual( + {item.read_bytes() for item in path.parent.glob(path.name + ".corrupt-*")}, + {b"first truncated operation", b"second truncated operation"}, + ) + for index in range(2, ReforceXY._QUARANTINE_TIE_BREAK_LIMIT + 1): + first.with_name(first.name + f"-{index}").write_bytes(b"evidence") + path.write_bytes(b"third truncated operation") + with self.assertRaises(FileExistsError): + self.storage() + self.assertEqual(path.read_bytes(), b"third truncated operation") + self.assertEqual(first.read_bytes(), b"first truncated operation") + + def test_best_params_are_atomic_pair_specific_and_preserve_existing_permissions(self): + model = self.model + first = {"gamma": 0.83, "target_kl": None, "nested": {"width": [8, 16]}} + model.save_best_trial_params(first, PAIR) + model.save_best_trial_params({"gamma": 0.91}, "ETH/USDT:USDT") + path = model._best_trial_params_path(PAIR) + path.chmod(0o640) + restarted = ReforceXY.__new__(ReforceXY) + restarted.full_path = model.full_path + self.assertEqual(restarted.load_best_trial_params(PAIR), first) + model.save_best_trial_params({"gamma": 0.97}, PAIR) + self.assertEqual(restarted.load_best_trial_params(PAIR), {"gamma": 0.97}) + self.assertEqual(path.stat().st_mode & 0o777, 0o640) + self.assertEqual(restarted.load_best_trial_params("ETH/USDT:USDT"), {"gamma": 0.91}) + + def test_invalid_identity_and_payloads_are_ignored_without_destroying_the_file(self): + path = self.model._best_trial_params_path(PAIR) + self.assertIsNone(self.model.load_best_trial_params(PAIR)) + for payload in ( + [], + {}, + {"params": {"gamma": 0.1}}, + {"objective_identity": "legacy", "params": {"gamma": 0.1}}, + {"objective_identity": ReforceXY._OPTUNA_OBJECTIVE_IDENTITY, "params": []}, + ): + with self.subTest(payload=payload): + encoded = json.dumps(payload).encode() + path.write_bytes(encoded) + self.assertIsNone(self.model.load_best_trial_params(PAIR)) + self.assertEqual(path.read_bytes(), encoded) + self.assertEqual(list(path.parent.glob(path.name + ".corrupt-*")), []) + model = ReforceXY.__new__(ReforceXY) + model.full_path = self.root / "absent" + self.assertIsNone(model.load_best_trial_params(PAIR)) + + def test_malformed_best_params_are_quarantined_and_can_be_replaced(self): + path = self.model._best_trial_params_path(PAIR) + for corrupt in (b"{broken", b"\xff"): + with self.subTest(corrupt=corrupt): + path.write_bytes(corrupt) + self.assertIsNone(self.model.load_best_trial_params(PAIR)) + self.assertFalse(path.exists()) + self.assertIn( + corrupt, + [item.read_bytes() for item in path.parent.glob(path.name + ".corrupt-*")], + ) + self.model.save_best_trial_params({"gamma": 0.83}, PAIR) + self.assertEqual(self.model.load_best_trial_params(PAIR), {"gamma": 0.83}) + + def test_failed_or_interrupted_save_keeps_previous_version_and_cleans_temporary_file(self): + self.model.save_best_trial_params({"gamma": 0.83}, PAIR) + with self.assertRaises(TypeError): + self.model.save_best_trial_params({"bad": object()}, PAIR) + for error in (OSError("disk unavailable"), KeyboardInterrupt()): + with ( + self.subTest(error=type(error).__name__), + mock.patch(MODULE + ".os.fsync", side_effect=error), + self.assertRaises(type(error)), + ): + self.model.save_best_trial_params({"gamma": 0.97}, PAIR) + self.assertEqual(self.model.load_best_trial_params(PAIR), {"gamma": 0.83}) + self.assertEqual(list(self.model.full_path.glob(".*.tmp")), []) + + def test_symlink_targets_and_nonregular_locks_are_rejected_without_modification(self): + target = self.root / "witness.json" + target.write_bytes(b"witness") + path = self.model._best_trial_params_path(PAIR) + path.symlink_to(target) + with self.assertRaises(OSError): + self.model.load_best_trial_params(PAIR) + with self.assertRaises(OSError): + self.model.save_best_trial_params({"gamma": 0.83}, PAIR) + self.assertEqual(target.read_bytes(), b"witness") + path.unlink() + lock = self.model.full_path / ReforceXY._BEST_PARAMS_LOCK_FILENAME + lock.unlink(missing_ok=True) + lock.symlink_to(target) + with self.assertRaises(OSError): + self.model.save_best_trial_params({"gamma": 0.83}, PAIR) + self.assertEqual(target.read_bytes(), b"witness") + lock.unlink() + os.mkfifo(lock) + # A subprocess timeout also protects against a regression to blocking open(). + script = ( + "from pathlib import Path; import sys; " + "from ReforceXY.user_data.freqaimodels.ReforceXY import ReforceXY; " + "m=ReforceXY.__new__(ReforceXY); m.full_path=Path(sys.argv[1]); " + "m.load_best_trial_params('BTC/USDT')" + ) + result = subprocess.run( + [sys.executable, "-c", script, str(self.model.full_path)], + cwd=REPO_ROOT, + capture_output=True, + text=True, + timeout=30, + check=False, + ) + self.assertEqual(result.returncode, 1) + self.assertIn("OSError", result.stderr) + self.assertTrue(lock.is_fifo()) + + def test_reader_without_existing_lock_does_not_create_one(self): + self.model.save_best_trial_params({"gamma": 0.83}, PAIR) + lock = self.model.full_path / ReforceXY._BEST_PARAMS_LOCK_FILENAME + lock.unlink() + self.assertEqual(self.model.load_best_trial_params(PAIR), {"gamma": 0.83}) + self.assertFalse(lock.exists()) + + def test_writer_repair_or_removal_between_reads_is_not_quarantined(self): + path = self.model._best_trial_params_path(PAIR) + original_lock = ReforceXY._locked_best_trial_params + for repair in (True, False): + path.write_bytes(b"{broken") + + @contextmanager + def intervening_writer(path, *, exclusive, repair=repair): + with original_lock(path, exclusive=exclusive): + if exclusive: + if repair: + path.write_text( + json.dumps( + { + "objective_identity": ReforceXY._OPTUNA_OBJECTIVE_IDENTITY, + "params": {"gamma": 0.97}, + } + ), + encoding="utf-8", + ) + else: + path.unlink() + yield + + with ( + self.subTest(repair=repair), + mock.patch.object( + ReforceXY, "_locked_best_trial_params", staticmethod(intervening_writer) + ), + ): + self.assertEqual( + self.model.load_best_trial_params(PAIR), {"gamma": 0.97} if repair else None + ) + self.assertEqual(list(path.parent.glob(path.name + ".corrupt-*")), []) + + def test_quarantine_failure_is_visible_and_leaves_corrupt_evidence_in_place(self): + path = self.model._best_trial_params_path(PAIR) + path.write_bytes(b"{broken") + with ( + mock.patch.object(Path, "rename", side_effect=PermissionError("read-only evidence")), + self.assertRaises(PermissionError), + ): + self.model.load_best_trial_params(PAIR) + self.assertEqual(path.read_bytes(), b"{broken") + + def test_unsupported_storage_is_rejected_instead_of_falling_back(self): + with self.assertRaises(ValueError): + self.storage("unsupported") + + +if __name__ == "__main__": + unittest.main() diff --git a/ReforceXY/tests/test_model_parameters.py b/ReforceXY/tests/test_model_parameters.py new file mode 100644 index 0000000..f603013 --- /dev/null +++ b/ReforceXY/tests/test_model_parameters.py @@ -0,0 +1,195 @@ +"""Configuration and sampled parameters exercised by their real SB3 consumers.""" + +import tempfile +import unittest + +import pandas as pd +import torch as th +from optuna.trial import FixedTrial +from qa_support import PAIR, QaTestCase, model_config + +from ReforceXY.user_data.freqaimodels.ReforceXY import ( + MyRLEnv, + ReforceXY, + convert_optuna_params_to_model_params, +) + + +class ModelParametersTest(QaTestCase): + def model(self, model_type="MaskablePPO", *, parameters=None, **rl_options): + temporary = tempfile.TemporaryDirectory() + self.addCleanup(temporary.cleanup) + config = model_config(temporary.name) + config["freqai"]["rl_config"].update( + { + "model_type": model_type, + "policy_type": "MlpLstmPolicy" if model_type == "RecurrentPPO" else "MlpPolicy", + **rl_options, + } + ) + if parameters is not None: + config["freqai"]["model_training_parameters"] = parameters + model = ReforceXY(config=config) + model.live = True + self.addCleanup(model.close_envs) + return model + + def learner(self, model, parameters): + prices = pd.DataFrame({"open": [100.0] * 16}) + env = MyRLEnv(df=prices.copy(), prices=prices, **model.pack_env_dict(PAIR, parameters)) + self.addCleanup(env.close) + parameters = {**parameters, "device": "cpu"} + learner = model.MODELCLASS(model.policy_type, env, **parameters) + self.assertEqual(learner.gamma, env._potential_gamma) + return learner + + def test_ppo_architecture_resolution_and_schedules_reach_the_actual_policy(self): + for architecture, pi_width, vf_width in ( + ([8], 8, 8), + ("small", 128, 128), + ("unknown", 128, 128), + ({"pi": [8], "vf": [16]}, 8, 16), + ({"pi": None, "vf": [16]}, 128, 16), + (17, 128, 128), + ): + with self.subTest(architecture=architecture): + model = self.model( + parameters={ + "n_steps": 8, + "batch_size": 8, + "n_epochs": 1, + "learning_rate": 0.002, + "clip_range": 0.4, + "policy_kwargs": { + "net_arch": architecture, + "activation_fn": "tanh", + "optimizer_class": "rmsprop", + }, + }, + lr_schedule=True, + cr_schedule=True, + ) + learner = self.learner(model, model.get_model_params()) + self.assertEqual(learner.policy.mlp_extractor.policy_net[0].out_features, pi_width) + self.assertEqual(learner.policy.mlp_extractor.value_net[0].out_features, vf_width) + self.assertIsInstance(learner.policy.mlp_extractor.policy_net[1], th.nn.Tanh) + self.assertIsInstance(learner.policy.optimizer, th.optim.RMSprop) + self.assertAlmostEqual(learner.lr_schedule(0.5), 0.001) + self.assertAlmostEqual(learner.clip_range(0.5), 0.2) + + def test_dqn_architecture_and_explicit_gradient_override_survive_normalization(self): + for architecture, explicit_steps, width, expected_steps in ( + ("small", None, 128, 3), + ("unknown", 7, 128, 7), + ({"pi": [8]}, None, 128, 3), + ([8], None, 8, 3), + ): + with self.subTest(architecture=architecture, explicit_steps=explicit_steps): + model = self.model( + "DQN", + parameters={ + "train_freq": 8, + "subsample_steps": 3, + "gradient_steps": explicit_steps, + "learning_starts": 0, + "buffer_size": 64, + "batch_size": 8, + "policy_kwargs": {"net_arch": architecture}, + }, + ) + learner = self.learner(model, model.get_model_params()) + self.assertEqual(learner.policy.q_net.q_net[0].out_features, width) + self.assertEqual(learner.gradient_steps, expected_steps) + self.assertEqual(learner.train_freq.frequency, 8) + + def test_samplers_build_all_algorithm_families_and_keep_optional_policy_controls(self): + ppo = { + "n_steps": 512, + "batch_size": 64, + "gamma": 0.95, + "learning_rate": 0.001, + "ent_coef": 0.01, + "clip_range": 0.2, + "n_epochs": 1, + "gae_lambda": 0.9, + "max_grad_norm": 0.5, + "vf_coef": 0.5, + "lr_schedule": "linear", + "cr_schedule": "linear", + "target_kl": None, + "ortho_init": False, + "net_arch": "small", + "activation_fn": "elu", + "optimizer_class": "rmsprop", + "n_lstm_layers": 1, + "lstm_hidden_size": 64, + "enable_critic_lstm": True, + } + for family, shared in ( + ("PPO", False), + ("MaskablePPO", False), + ("RecurrentPPO", False), + ("RecurrentPPO", True), + ): + with self.subTest(family=family, shared=shared): + policy_kwargs = {"net_arch": [8]} + if family == "RecurrentPPO": + policy_kwargs["shared_lstm"] = shared + model = self.model(family, parameters={"policy_kwargs": policy_kwargs}) + sampled = model.get_optuna_params(FixedTrial(ppo)) + sampled["policy_kwargs"].update(policy_kwargs if shared else {}) + learner = self.learner(model, sampled) + self.assertAlmostEqual(learner.lr_schedule(0.5), 0.0005) + self.assertAlmostEqual(learner.clip_range(0.5), 0.1) + self.assertIsInstance(learner.policy.optimizer, th.optim.RMSprop) + if family == "RecurrentPPO": + self.assertEqual(learner.policy.lstm_actor.hidden_size, 64) + self.assertEqual(learner.policy.lstm_critic is None, shared) + + def test_value_based_exploration_boundaries_change_the_consumed_epsilon_schedule(self): + for family, initial, final, fraction in ( + ("DQN", 0.9, 0.2, 0.2), + ("DQN", 0.8, 0.1, 0.15), + ("QRDQN", 0.6, 0.2, 0.05), + ): + with self.subTest(family=family, initial=initial): + raw = { + "train_freq": 8, + "subsample_steps": 2, + "gamma": 0.95, + "batch_size": 64, + "learning_rate": 0.001, + "lr_schedule": "linear", + "buffer_size": 10000, + "exploration_initial_eps": initial, + "exploration_final_eps": final, + "exploration_fraction": fraction, + "target_update_interval": 1000, + "learning_starts": 500, + "net_arch": "small", + "activation_fn": "elu", + "optimizer_class": "rmsprop", + "n_quantiles": 10, + } + model = self.model(family, parameters={"policy_kwargs": {"net_arch": [8]}}) + learner = self.learner(model, model.get_optuna_params(FixedTrial(raw))) + self.assertAlmostEqual( + learner.exploration_schedule(1 - fraction / 2), (initial + final) / 2 + ) + self.assertEqual(learner.gradient_steps, 4) + if family == "QRDQN": + self.assertEqual(learner.policy.quantile_net.n_quantiles, 10) + + def test_incomplete_hyperparameters_fail_before_a_partial_learner_is_returned(self): + for family, raw in ( + ("PPO", {}), + ("PPO", {"learning_rate": 0.001}), + ("DQN", {"learning_rate": 0.001}), + ("unknown", {"learning_rate": 0.001}), + ): + with self.subTest(family=family), self.assertRaises(ValueError): + convert_optuna_params_to_model_params(family, raw) + + +if __name__ == "__main__": + unittest.main() diff --git a/ReforceXY/tests/test_model_pbrs_transitions.py b/ReforceXY/tests/test_model_pbrs_transitions.py new file mode 100644 index 0000000..db4265d --- /dev/null +++ b/ReforceXY/tests/test_model_pbrs_transitions.py @@ -0,0 +1,277 @@ +"""PBRS transitions, boundary identities and discount resolution in the real RL runtime.""" + +import math +import tempfile +import unittest +from types import SimpleNamespace + +import numpy as np +import pandas as pd +from qa_support import QaTestCase, model_config +from sb3_contrib import MaskablePPO + +from ReforceXY.user_data.freqaimodels.ReforceXY import Actions, MyRLEnv, Positions, ReforceXY + + +class PbrsTransitionsTest(QaTestCase): + def env(self, *, short=False, fee=0.0, **parameters): + prices = pd.DataFrame({"open": [100.0, 100.0] + [90.0 if short else 110.0] * 10}) + env = MyRLEnv( + df=prices.copy(), + prices=prices, + df_raw=prices.copy(), + window_size=1, + reward_kwargs={"rr": 2.0, "profit_aim": 0.03}, + fee=fee, + can_short=True, + config={ + "stake_amount": "unlimited", + "freqai": { + "rl_config": { + "add_state_info": True, + "max_training_drawdown_pct": 0.99, + "model_reward_parameters": { + "potential_gamma": 0.8, + "hold_potential_enabled": True, + "entry_additive_enabled": False, + "exit_additive_enabled": False, + **parameters, + }, + } + }, + }, + live=True, + ) + self.addCleanup(env.close) + env.reset() + return env + + def test_retained_potential_shapes_every_nonterminal_neutral_step(self): + for mode in ("retain_previous", "progressive_release", "spike_cancel"): + with self.subTest(mode=mode): + env = self.env(exit_potential_mode=mode, exit_potential_decay=0.25) + for action in (Actions.Long_enter, Actions.Neutral, Actions.Long_exit): + _, _, done, truncated, _ = env.step(action.value) + self.assertFalse(done or truncated) + potential = env._last_potential + self.assertNotEqual(potential, 0.0) + for _ in range(2): + _, _, done, truncated, _ = env.step(Actions.Neutral.value) + self.assertFalse(done or truncated) + self.assertEqual(env._last_prev_potential, potential) + self.assertEqual(env._last_next_potential, potential) + self.assertAlmostEqual(env._last_reward_shaping, -0.2 * potential, places=12) + + def test_complete_episodes_telescope_and_change_the_actual_reward_by_the_shaping(self): + for mode in ReforceXY._EXIT_POTENTIAL_MODES: + for short in (False, True): + with self.subTest(mode=mode, short=short): + shaped = self.env(short=short, exit_potential_mode=mode) + base = self.env( + short=short, exit_potential_mode=mode, hold_potential_enabled=False + ) + actions = [ + Actions.Short_enter if short else Actions.Long_enter, + Actions.Neutral, + Actions.Short_exit if short else Actions.Long_exit, + ] + [Actions.Neutral] * 8 + previous = 0.0 + discounted = 0.0 + for tick, action in enumerate(actions): + _, reward, done, truncated, _ = shaped.step(action.value) + _, base_reward, base_done, base_truncated, _ = base.step(action.value) + self.assertEqual((done, truncated), (base_done, base_truncated)) + self.assertAlmostEqual(shaped._last_prev_potential, previous, places=12) + expected = 0.8 * shaped._last_next_potential - previous + self.assertAlmostEqual(shaped._last_reward_shaping, expected, places=12) + self.assertAlmostEqual(reward - base_reward, expected, places=12) + discounted += 0.8**tick * shaped._last_reward_shaping + previous = shaped._last_next_potential + if done or truncated: + break + self.assertTrue(done or truncated) + self.assertEqual(previous, 0.0) + self.assertAlmostEqual(discounted, 0.0, places=12) + history = shaped.get_env_history() + np.testing.assert_allclose( + history["reward_shaping"], + 0.8 * history["next_potential"] - history["prev_potential"], + atol=1e-12, + ) + + def test_zero_exit_modes_without_additives_have_identical_observations_and_rewards(self): + canonical = self.env(exit_potential_mode="canonical") + optional = self.env(exit_potential_mode="non_canonical") + for action in [Actions.Long_enter, Actions.Neutral, Actions.Long_exit] + [ + Actions.Neutral + ] * 8: + observation, reward, done, truncated, info = canonical.step(action.value) + other_observation, other_reward, other_done, other_truncated, other_info = ( + optional.step(action.value) + ) + np.testing.assert_array_equal(observation, other_observation) + self.assertEqual(reward, other_reward) + self.assertEqual((done, truncated), (other_done, other_truncated)) + self.assertTrue(info["pbrs_invariant"]) + self.assertTrue(other_info["pbrs_invariant"]) + if done or truncated: + break + + def test_canonical_disables_requested_additives_but_noncanonical_applies_them_at_fills(self): + for mode in ("canonical", "non_canonical"): + with self.subTest(mode=mode): + env = self.env( + fee=0.0015, + exit_potential_mode=mode, + entry_additive_enabled=True, + exit_additive_enabled=True, + ) + base = self.env(fee=0.0015, exit_potential_mode=mode) + for action in ( + Actions.Long_enter, + Actions.Neutral, + Actions.Long_exit, + Actions.Neutral, + ): + _, reward, _, _, info = env.step(action.value) + _, base_reward, _, _, _ = base.step(action.value) + additive = env._last_entry_additive + env._last_exit_additive + self.assertAlmostEqual(reward - base_reward, additive, places=12) + if mode == "canonical" or action == Actions.Neutral: + self.assertEqual(additive, 0.0) + else: + self.assertNotEqual(additive, 0.0) + self.assertEqual(info["pbrs_invariant"], mode == "canonical") + + def test_reset_discards_the_previous_episode_potential_and_additive_totals(self): + env = self.env( + exit_potential_mode="retain_previous", entry_additive_enabled=True, fee=0.0015 + ) + env.step(Actions.Long_enter.value) + self.assertNotEqual(env._last_potential, 0.0) + env.reset() + _, _, _, _, _ = env.step(Actions.Neutral.value) + self.assertEqual(env._last_prev_potential, 0.0) + self.assertEqual(env._last_reward_shaping, 0.0) + self.assertEqual(env._total_entry_additive, 0.0) + self.assertEqual(env._total_exit_additive, 0.0) + + def test_bounded_transforms_preserve_signed_signal_and_extreme_stability(self): + env = self.env() + transforms = { + "tanh": math.tanh, + "softsign": lambda x: x / (1 + abs(x)), + "arctan": lambda x: 2 * math.atan(x) / math.pi, + "sigmoid": lambda x: math.tanh(x / 2), + "softsign_sqrt": lambda x: x / math.hypot(1, x), + "clip": lambda x: min(1, max(-1, x)), + } + for name, oracle in transforms.items(): + for x in (-2.0, -0.5, 0.0, 0.5, 2.0): + with self.subTest(transform=name, x=x): + self.assertAlmostEqual(env._potential_transform(name, x), oracle(x), places=12) + for x in (-1000.0, 1000.0): + self.assertEqual(env._potential_transform("sigmoid", x), math.copysign(1, x)) + for x in (-1e200, 1e200): + self.assertEqual(env._potential_transform("softsign_sqrt", x), math.copysign(1, x)) + self.assertAlmostEqual(env._potential_transform("unknown", 0.5), math.tanh(0.5)) + + def test_hold_signal_amplifies_loss_duration_only_and_clamps_negative_duration(self): + env = self.env(hold_potential_gain=1.0) + for position in (Positions.Long, Positions.Short): + for pnl in (-0.06, 0.0, 0.06): + for duration in (-0.5, 0.5): + with self.subTest(position=position, pnl=pnl, duration=duration): + multiplier = 2.0 if pnl < 0 else 1.0 + expected = 0.2 * ( + math.tanh(pnl / 0.06) + + np.sign(pnl) * multiplier * math.tanh(max(0.0, duration)) + ) + self.assertAlmostEqual( + env._compute_hold_potential(position, pnl, 0.06, duration, 0.4), + expected, + places=12, + ) + self.assertEqual(env._compute_hold_potential(Positions.Neutral, 0.06, 0.06, 1, 0.4), 0.0) + self.assertEqual(env._compute_hold_potential(Positions.Long, 0.06, 0.0, 1, 0.4), 0.0) + self.assertEqual(env._compute_hold_potential(Positions.Long, math.nan, 0.06, 1, 0.4), 0.0) + for ratio in (0.0, math.nan): + env.rr = ratio + self.assertAlmostEqual( + env._compute_hold_potential(Positions.Long, -0.06, 0.06, 0.5, 0.4), + -0.2 * (math.tanh(1) + math.tanh(0.5)), + places=12, + ) + + def test_exit_decay_clamps_to_its_probability_bounds(self): + for decay, expected in ( + (-0.5, 0.4), + (0.0, 0.4), + (0.25, 0.3), + (1.0, 0.0), + (2.0, 0.0), + (math.nan, 0.4), + ): + with self.subTest(decay=decay): + env = self.env( + exit_potential_mode="progressive_release", exit_potential_decay=decay + ) + self.assertAlmostEqual(env._compute_exit_potential(0.4, 0.8), expected) + env = self.env(exit_potential_mode="spike_cancel") + self.assertEqual(env._compute_exit_potential(0.4, 0.0), 0.4) + self.assertAlmostEqual(env._compute_exit_potential(0.4, 0.8), 0.5) + + +class DiscountResolutionTest(QaTestCase): + def model(self, **parameters): + temp = tempfile.TemporaryDirectory() + self.addCleanup(temp.cleanup) + model = ReforceXY( + config=model_config(temp.name, freqai={"model_training_parameters": parameters}) + ) + model.live = True + self.addCleanup(model.close_envs) + return model + + def test_effective_null_gamma_is_rejected_before_environment_construction(self): + model = self.model(gamma=None) + with self.assertRaises(ValueError): + model.pack_env_dict("BTC/USDT") + model = self.model(gamma=0.83) + with self.assertRaises(ValueError): + model.pack_env_dict("BTC/USDT", {"gamma": None}) + + def test_invalid_replacement_discount_does_not_close_running_environments(self): + model = self.model(gamma=0.83) + features = pd.DataFrame({"f": np.arange(16, dtype=float)}) + prices = pd.DataFrame({"open": [100.0] * 16}) + data = {"train_features": features.copy(), "test_features": features.copy()} + kitchen = SimpleNamespace(pair="BTC/USDT") + model.set_train_and_eval_environments(data, prices, prices, kitchen) + running_train, running_eval = model.train_env, model.eval_env + first = running_train.reset() + with self.assertRaises(ValueError): + model.set_train_and_eval_environments(data, prices, prices, kitchen, {"gamma": None}) + self.assertIs(model.train_env, running_train) + self.assertIs(model.eval_env, running_eval) + second, _, done, _ = running_train.step([Actions.Neutral.value]) + self.assertFalse(done[0]) + np.testing.assert_array_equal(second[:, :, 0], first[:, :, 0] + 1) + + def test_valid_override_supersedes_null_base_and_configured_potential_discount(self): + model = self.model(gamma=None) + model.rl_config["model_reward_parameters"]["potential_gamma"] = 0.41 + kwargs = model.pack_env_dict("BTC/USDT", {"gamma": 0.83}) + prices = pd.DataFrame({"open": [100.0] * 16}) + env = MyRLEnv(df=prices.copy(), prices=prices, **kwargs) + self.addCleanup(env.close) + learner = MaskablePPO( + "MlpPolicy", env, gamma=0.83, n_steps=8, batch_size=8, n_epochs=1, device="cpu" + ) + learner.learn(8) + self.assertEqual(learner.gamma, env._potential_gamma) + self.assertGreater(learner._n_updates, 0) + + +if __name__ == "__main__": + unittest.main() diff --git a/ReforceXY/tests/test_model_review_contracts.py b/ReforceXY/tests/test_model_review_contracts.py index 95fc7f9..93fed14 100644 --- a/ReforceXY/tests/test_model_review_contracts.py +++ b/ReforceXY/tests/test_model_review_contracts.py @@ -23,7 +23,6 @@ from ReforceXY.user_data.freqaimodels.ReforceXY import ( convert_optuna_params_to_model_params, deepmerge, ) -from ReforceXY.user_data.strategies.RLAgentStrategy import RLAgentStrategy class ReviewContractsTest(QaTestCase): @@ -41,64 +40,6 @@ class ReviewContractsTest(QaTestCase): self.addCleanup(model.close_envs) return model - def test_strategy_leverage_respects_pair_bounds_and_invalid_config(self): - """A strategy callback always returns a finite leverage inside pair limits.""" - strategy = RLAgentStrategy.__new__(RLAgentStrategy) - arguments = { - "pair": "BTC/USDT", - "current_time": dt(2026, 1, 1, tzinfo=timezone.utc), - "current_rate": 100.0, - "proposed_leverage": 2.0, - "max_leverage": 5.0, - "entry_tag": None, - "side": "long", - } - for configured, expected in ( - (None, 2.0), - (0.5, 1.0), - (10.0, 5.0), - (3.0, 3.0), - (float("nan"), 2.0), - (float("inf"), 2.0), - (10**500, 2.0), - (-5.0, 1.0), - ("invalid", 2.0), - (True, 2.0), - ): - with self.subTest(configured=configured): - strategy.config = {} if configured is None else {"leverage": configured} - self.assertEqual(strategy.leverage(**arguments), expected) - - def test_strategy_leverage_warnings_follow_invalid_config_transitions(self): - """Warn once per invalid setting, including values below the leverage floor.""" - strategy = RLAgentStrategy.__new__(RLAgentStrategy) - arguments = { - "pair": "BTC/USDT", - "current_time": dt(2026, 1, 1, tzinfo=timezone.utc), - "current_rate": 100.0, - "proposed_leverage": 2.0, - "max_leverage": 5.0, - "entry_tag": None, - "side": "long", - } - strategy.config = {"leverage": float("nan")} - with self.assertLogs(RLAgentStrategy.__module__, level="WARNING") as logs: - for _ in range(3): - self.assertEqual(strategy.leverage(**arguments), 2.0) - self.assertEqual(len(logs.records), 1) - strategy.config["leverage"] = float("inf") - self.assertEqual(strategy.leverage(**arguments), 2.0) - self.assertEqual(len(logs.records), 2) - strategy.config["leverage"] = 0.5 - for _ in range(2): - self.assertEqual(strategy.leverage(**arguments), 1.0) - self.assertEqual(len(logs.records), 3) - strategy.config["leverage"] = 2.5 - self.assertEqual(strategy.leverage(**arguments), 2.5) - strategy.config["leverage"] = float("nan") - self.assertEqual(strategy.leverage(**arguments), 2.0) - self.assertEqual(len(logs.records), 4) - def test_training_preserves_raw_prices_and_returns_best_checkpoint(self): for drop in (False, True): with self.subTest(drop_ohlc_from_features=drop), tempfile.TemporaryDirectory() as temp: @@ -678,24 +619,6 @@ class ReviewContractsTest(QaTestCase): self.assertEqual(env._current_tick, 6) self.assertEqual(info["next_potential"], 0.0) - def test_pbrs_history_preserves_verifier_precision(self): - model = self.model(hold=True) - model.CONV_WIDTH = 2 - features = pd.DataFrame({"f": np.arange(5, dtype=float)}) - prices = pd.DataFrame({"open": np.full(5, 100.0)}) - env = MyRLEnv(df=features, prices=prices, **model.pack_env_dict("BTC/USDT")) - env.fee = 0.0 - self.addCleanup(env.close) - env.reset() - - with mock.patch.object(env, "_compute_hold_potential", return_value=0.123456): - env.step(Actions.Long_enter.value) - - row = env.get_env_history().iloc[-1] - expected_shaping = env._potential_gamma * row["next_potential"] - row["prev_potential"] - self.assertLessEqual(abs(row["reward_shaping"] - expected_shaping), 1e-6) - self.assertEqual(row["next_potential"], 0.123456) - def test_state_info_normalizes_leveraged_profit_ratio(self): trade = SimpleNamespace( pair="BTC/USDT", diff --git a/ReforceXY/tests/test_model_schedules_params_and_net.py b/ReforceXY/tests/test_model_schedules_params_and_net.py index d6c23c2..e4bfba3 100644 --- a/ReforceXY/tests/test_model_schedules_params_and_net.py +++ b/ReforceXY/tests/test_model_schedules_params_and_net.py @@ -28,6 +28,9 @@ class LinearScheduleTest(QaTestCase): schedule = SimpleLinearSchedule(0.25) self.assertEqual(0.25, schedule(1.0)) self.assertEqual(0.0, schedule(0.0)) + for progress, expected in ((0.25, 0.0625), (0.5, 0.125), (0.75, 0.1875)): + with self.subTest(progress=progress): + self.assertEqual(expected, schedule(progress)) def test_a_string_initial_value_is_read_as_a_number(self): self.assertEqual(0.5, SimpleLinearSchedule("0.5")(1.0)) @@ -64,6 +67,8 @@ class ScheduleClassificationTest(QaTestCase): self.assertIsInstance(schedule, SimpleLinearSchedule) self.assertEqual(0.1, schedule(1.0)) self.assertEqual(0.0, schedule(0.0)) + self.assertAlmostEqual(0.025, schedule(0.25)) + self.assertAlmostEqual(0.05, schedule(0.5)) def test_every_other_request_builds_a_constant_schedule(self): for kind in (CONSTANT, UNKNOWN): @@ -93,6 +98,8 @@ class GradientStepTest(QaTestCase): def test_it_is_the_ceiling_of_the_frequency_quotient_capped_by_the_frequency(self): self.assertEqual(4, compute_gradient_steps(8, 2)) self.assertEqual(1, compute_gradient_steps(3, 8)) + self.assertEqual(3, compute_gradient_steps(8, 3)) + self.assertEqual(3, compute_gradient_steps(5, 2)) def test_a_non_positive_frequency_reports_minus_one_rather_than_a_step_count(self): self.assertEqual(-1, compute_gradient_steps(0, 2)) @@ -175,6 +182,17 @@ class EvalFrequencyTest(QaTestCase): self.assertLess(model.get_eval_freq(16_000), 16_000) self.assertEqual(16, model.get_eval_freq(16)) + def test_configured_ppo_rollout_takes_precedence_and_respects_the_callback_budget(self): + model = self._model(n_envs=4) + self.assertEqual(8, model.get_eval_freq(400, model_params={"n_steps": 8})) + self.assertEqual(5, model.get_eval_freq(20, model_params={"n_steps": 8})) + self.assertEqual(2, model.get_eval_freq(400, model_params={"n_steps": 8}, hyperopt=True)) + + def test_value_based_frequency_rounds_partial_vector_calls_up_before_the_cap(self): + model = self._model(model_type="DQN", n_envs=4, n_eval_steps=17) + self.assertEqual(5, model.get_eval_freq(400)) + self.assertEqual(4, model.get_eval_freq(16)) + def test_more_environments_mean_fewer_callback_calls_for_the_same_budget(self): self.assertLess( self._model(n_envs=4).get_eval_freq(16_000), diff --git a/ReforceXY/tests/test_strategy_semantics.py b/ReforceXY/tests/test_strategy_semantics.py index 2d5a964..292623c 100644 --- a/ReforceXY/tests/test_strategy_semantics.py +++ b/ReforceXY/tests/test_strategy_semantics.py @@ -41,17 +41,6 @@ def _frame(actions, do_predict=1, count=None): ) -class ActionLiteralTest(QaTestCase): - def test_the_strategy_literals_are_the_production_action_codes(self): - # Asserted as a cross-object equality, not as two literals: the strategy's constants - # are correct only because they equal the enum's values, and two independent - # literals would both still pass after a freqtrade renumbering. - self.assertEqual(Actions.Long_enter.value, RLAgentStrategy._ACTION_ENTER_LONG) - self.assertEqual(Actions.Long_exit.value, RLAgentStrategy._ACTION_EXIT_LONG) - self.assertEqual(Actions.Short_enter.value, RLAgentStrategy._ACTION_ENTER_SHORT) - self.assertEqual(Actions.Short_exit.value, RLAgentStrategy._ACTION_EXIT_SHORT) - - class DateColumnTest(QaTestCase): def test_a_missing_date_column_is_refused(self): with self.assertRaises(ValueError): @@ -162,11 +151,14 @@ class LeverageTest(QaTestCase): arguments.update(overrides) return arguments - def test_an_absent_configuration_yields_the_proposed_leverage(self): - self.assertEqual(2.0, self._strategy(None).leverage(**self._arguments())) + def test_absent_and_null_configuration_use_the_proposed_leverage(self): + strategy = self._strategy(None) + self.assertEqual(2.0, strategy.leverage(**self._arguments())) + strategy.config.clear() + self.assertEqual(2.0, strategy.leverage(**self._arguments())) def test_a_configured_value_is_clamped_into_the_pair_bounds(self): - for configured, expected in ((1.5, 1.5), (0.5, 1.0), (9.0, 5.0)): + for configured, expected in ((1.5, 1.5), (0.5, 1.0), (-5.0, 1.0), (9.0, 5.0)): with self.subTest(configured=configured): self.assertEqual(expected, self._strategy(configured).leverage(**self._arguments())) @@ -188,6 +180,39 @@ class LeverageTest(QaTestCase): def test_the_result_is_always_at_least_one(self): self.assertEqual(1.0, self._strategy(0.5).leverage(**self._arguments(max_leverage=0.2))) + def test_integer_conversion_overflow_falls_back_instead_of_escaping_the_callback(self): + self.assertEqual(2.0, self._strategy(10**500).leverage(**self._arguments())) + + def test_warnings_follow_invalid_configuration_transitions(self): + strategy = self._strategy(float("nan")) + with self.assertLogs(RLAgentStrategy.__module__, level="WARNING") as logs: + for _ in range(3): + self.assertEqual(strategy.leverage(**self._arguments()), 2.0) + self.assertEqual(len(logs.records), 1) + strategy.config["leverage"] = float("inf") + self.assertEqual(strategy.leverage(**self._arguments()), 2.0) + self.assertEqual(len(logs.records), 2) + strategy.config["leverage"] = 0.5 + for _ in range(2): + self.assertEqual(strategy.leverage(**self._arguments()), 1.0) + self.assertEqual(len(logs.records), 3) + strategy.config["leverage"] = 2.5 + self.assertEqual(strategy.leverage(**self._arguments()), 2.5) + strategy.config["leverage"] = float("nan") + self.assertEqual(strategy.leverage(**self._arguments()), 2.0) + self.assertEqual(len(logs.records), 4) + + def test_missing_configuration_resets_warning_deduplication(self): + strategy = self._strategy("invalid") + with self.assertLogs(RLAgentStrategy.__module__, level="WARNING") as logs: + self.assertEqual(strategy.leverage(**self._arguments()), 2.0) + strategy.config.clear() + self.assertEqual(strategy.leverage(**self._arguments()), 2.0) + self.assertEqual(len(logs.records), 1) + strategy.config["leverage"] = "invalid" + self.assertEqual(strategy.leverage(**self._arguments()), 2.0) + self.assertEqual(len(logs.records), 2) + class EntrySignalTest(QaTestCase): def _strategy(self): @@ -309,30 +334,29 @@ class FeatureEngineeringTest(QaTestCase): def _strategy(self): return RLAgentStrategy.__new__(RLAgentStrategy) - def test_the_standard_features_are_exact_copies_of_the_ohlc(self): + def test_time_features_encode_week_and_day_boundaries_with_the_original_index(self): frame = _frame([0, 0, 0]) + frame.index = [7, 13, 29] + frame["date"] = pd.to_datetime( + ["2024-01-01T00:00:00Z", "2024-01-07T23:00:00Z", "2024-01-08T00:00:00Z"] + ) result = self._strategy().feature_engineering_standard(frame, {}) - for source, produced in (("close", "%-raw_close"), ("open", "%-raw_open")): - with self.subTest(feature=produced): - self.assertEqual(list(frame[source]), list(result[produced])) - - def test_the_time_features_are_normalised_into_the_unit_interval(self): - result = self._strategy().feature_engineering_standard(_frame([0] * 48), {}) - for feature in ("%-day_of_week", "%-hour_of_day"): - with self.subTest(feature=feature): - self.assertTrue(((result[feature] > 0.0) & (result[feature] <= 1.0)).all()) - - def test_the_day_of_week_never_reports_a_zero_weekday(self): - frame = pd.DataFrame({"date": pd.Series(pd.to_datetime(["2024-01-01"], utc=True))}) - result = self._strategy().feature_engineering_standard(frame, {}) - self.assertGreater(result["%-day_of_week"].iloc[0], 0.0) - - def test_the_expanding_feature_is_the_log_difference_of_the_close(self): - frame = _frame([0, 0, 0]) - frame["close"] = [100.0, 110.0, 121.0] + self.assertEqual([7, 13, 29], list(result.index)) + np.testing.assert_allclose(result["%-day_of_week"], [1 / 7, 1, 1 / 7], atol=1e-12) + np.testing.assert_allclose(result["%-hour_of_day"], [1 / 25, 24 / 25, 1 / 25], atol=1e-12) + + def test_log_returns_keep_positive_negative_and_flat_moves_in_index_order(self): + frame = _frame([0, 0, 0, 0]) + frame.index = [7, 13, 29, 41] + frame["close"] = [100.0, 110.0, 99.0, 99.0] result = self._strategy().feature_engineering_expand_basic(frame, {}) - self.assertTrue(pd.isna(result["%-close_log_return"].iloc[0])) - self.assertAlmostEqual(np.log(1.1), result["%-close_log_return"].iloc[1], places=12) + self.assertEqual([7, 13, 29, 41], list(result.index)) + np.testing.assert_allclose( + result["%-close_log_return"], + [np.nan, np.log(1.1), np.log(0.9), 0.0], + atol=1e-12, + equal_nan=True, + ) def test_the_target_column_is_the_neutral_action(self): result = self._strategy().set_freqai_targets( diff --git a/ReforceXY/user_data/freqaimodels/ReforceXY.py b/ReforceXY/user_data/freqaimodels/ReforceXY.py index ce26369..ffdcfcb 100644 --- a/ReforceXY/user_data/freqaimodels/ReforceXY.py +++ b/ReforceXY/user_data/freqaimodels/ReforceXY.py @@ -436,6 +436,7 @@ class ReforceXY(BaseReinforcementLearningModel): _DEPLOYMENT_COORDINATE_GENERATION: Final[str] = "chronological-frozen-pipelines-v2" DEFAULT_BASE_FACTOR: Final[float] = 100.0 + DEFAULT_DISCOUNT_GAMMA: Final[float] = 0.95 DEFAULT_MAX_TRADE_DURATION_CANDLES: Final[int] = 128 DEFAULT_IDLE_DURATION_MULTIPLIER: Final[int] = 4 @@ -548,7 +549,7 @@ class ReforceXY(BaseReinforcementLearningModel): _BEST_PARAMS_LOCK_FILENAME: Final[str] = ".hyperopt-best-params.lock" # Bump on objective changes that make persisted trials or best params incompatible. _OPTUNA_OBJECTIVE_IDENTITY: Final[str] = ( - "terminal-liquidation-risk-normalized-trained-policy-v3" + "terminal-liquidation-risk-normalized-trained-policy-v4" ) _PPO_N_STEPS: Final[tuple[int, ...]] = (512, 1024, 2048, 4096) _PPO_N_STEPS_MIN: Final[int] = min(_PPO_N_STEPS) @@ -937,6 +938,8 @@ class ReforceXY(BaseReinforcementLearningModel): f"Config [{pair}]: backtesting does not support hold_potential_enabled=True " "because add_state_info is unavailable" ) + effective_params = self.get_model_params() if model_params is None else model_params + gamma = ReforceXY._resolve_discount_gamma(effective_params) env_info = super().pack_env_dict(pair) # Each environment owns its effective parameters; do not mutate global config. config = copy.deepcopy(env_info["config"]) @@ -944,16 +947,19 @@ class ReforceXY(BaseReinforcementLearningModel): model_reward_parameters = config["freqai"]["rl_config"].setdefault( "model_reward_parameters", {} ) - effective_params = self.get_model_params() if model_params is None else model_params - gamma = effective_params.get("gamma") - - if gamma is not None: - model_reward_parameters["potential_gamma"] = gamma - else: - logger.warning("Env [%s]: no valid discount gamma resolved for environment", pair) - + model_reward_parameters["potential_gamma"] = gamma return env_info + @staticmethod + def _resolve_discount_gamma(model_params: Mapping[str, Any]) -> float: + """Resolve discount after precedence, rejecting explicit null instead of defaulting.""" + gamma = model_params.get("gamma", ReforceXY.DEFAULT_DISCOUNT_GAMMA) + if gamma is None: + raise ValueError( + "Config [global]: effective model_training_parameters.gamma must not be null" + ) + return cast("float", gamma) + def set_train_and_eval_environments( self, data_dictionary: dict[str, DataFrame], @@ -965,6 +971,7 @@ class ReforceXY(BaseReinforcementLearningModel): """ Set training and evaluation environments """ + env_dict = self.pack_env_dict(dk.pair, model_params) data_dictionary["train_prices"] = prices_train data_dictionary["test_prices"] = prices_test if self.train_env is not None or self.eval_env is not None: @@ -973,7 +980,6 @@ class ReforceXY(BaseReinforcementLearningModel): train_df = data_dictionary["train_features"] test_df = data_dictionary["test_features"] - env_dict = self.pack_env_dict(dk.pair, model_params) effective_params = self.get_model_params() if model_params is None else model_params seed = effective_params.get("seed", 42) @@ -1029,7 +1035,7 @@ class ReforceXY(BaseReinforcementLearningModel): model_params: dict[str, Any] = copy.deepcopy(self.model_training_parameters) model_params.setdefault("seed", 42) - model_params.setdefault("gamma", 0.95) + model_params.setdefault("gamma", ReforceXY.DEFAULT_DISCOUNT_GAMMA) if not self.hyperopt and self.lr_schedule: lr = model_params.get("learning_rate", 0.0003) @@ -1567,6 +1573,7 @@ class ReforceXY(BaseReinforcementLearningModel): learner_gamma = getattr(model, "gamma", None) if isinstance(learner_gamma, (int, float)) and np.isfinite(learner_gamma): effective_params["gamma"] = float(learner_gamma) + effective_params["gamma"] = ReforceXY._resolve_discount_gamma(effective_params) # Preserve raw prices and the resumed learner's discount in its environments. try: self.set_train_and_eval_environments( @@ -1585,11 +1592,11 @@ class ReforceXY(BaseReinforcementLearningModel): self.policy_type, self.train_env, tensorboard_log=tensorboard_log_path, - **model_params, + **effective_params, ) total_timesteps = self._align_model_budget(model, total_timesteps) - eval_freq = self.get_eval_freq(total_timesteps, model_params=model_params) + eval_freq = self.get_eval_freq(total_timesteps, model_params=effective_params) callbacks = self.get_callbacks(self.eval_env, eval_freq, str(dk.data_path)) logger.debug( "Training [%s]: starting model.learn with total_timesteps=%d, eval_freq=%d", @@ -2848,6 +2855,7 @@ class ReforceXY(BaseReinforcementLearningModel): # Ensure that the sampled parameters take precedence params = deepmerge(self.get_model_params(), params) + params["gamma"] = ReforceXY._resolve_discount_gamma(params) params["seed"] = params.get("seed", 42) + trial.number logger.info("Hyperopt [%s]: trial #%d params: %s", study_name, trial.number, params) @@ -3079,7 +3087,9 @@ class MyRLEnv(Base5ActionRLEnv): ) ) # === PBRS COMMON PARAMETERS === - self._potential_gamma = float(model_reward_parameters.get("potential_gamma", 0.95)) + self._potential_gamma = float( + model_reward_parameters.get("potential_gamma", ReforceXY.DEFAULT_DISCOUNT_GAMMA) + ) if np.isclose(self._potential_gamma, 0.0): logger.warning( "PBRS [%s]: potential_gamma=0 detected; PBRS delta will be -Φ(s) " @@ -3090,7 +3100,7 @@ class MyRLEnv(Base5ActionRLEnv): # === EXIT POTENTIAL MODE === # exit_potential_mode options: # 'canonical' -> Φ(s')=0 (preserves invariance, disables additives) - # 'non_canonical' -> Φ(s')=0 (allows additives, breaks invariance) + # 'non_canonical' -> Φ(s')=0 (allows optional non-PBRS additives) # 'progressive_release' -> Φ(s')=Φ(s)*(1-decay_factor) # 'spike_cancel' -> Φ(s')=Φ(s)/γ (Δ ≈ 0, cancels shaping) # 'retain_previous' -> Φ(s')=Φ(s) @@ -3447,23 +3457,17 @@ class MyRLEnv(Base5ActionRLEnv): return next_potential def is_pbrs_invariant_mode(self) -> bool: - """Return True if current configuration preserves PBRS policy invariance. - - PBRS policy invariance (Ng et al. 1999) requires: - 1. Canonical exit mode: Φ(terminal) = 0 - 2. No path-dependent additives: entry_additive = exit_additive = 0 - - When True, the shaped policy π'(s) is guaranteed to be equivalent to - the policy π(s) learned with base rewards only. + """Conservatively identify zero-at-exit PBRS without effective additives. - Returns - ------- - bool - True if configuration preserves theoretical PBRS invariance + Policy invariance additionally assumes an observable state potential, + the learner's discount, and zero boundary potentials. This configuration + diagnostic does not guarantee equivalence of policies fitted by SB3. + Residual exit modes can retain history-dependent potential while neutral. """ - return self._exit_potential_mode == ReforceXY._EXIT_POTENTIAL_MODES[0] and not ( - self._entry_additive_enabled or self._exit_additive_enabled - ) # "canonical" + return ( + self._exit_potential_mode == ReforceXY._EXIT_POTENTIAL_MODES[0] + or self._exit_potential_mode == ReforceXY._EXIT_POTENTIAL_MODES[1] + ) and not (self._entry_additive_enabled or self._exit_additive_enabled) @staticmethod def is_unsupported_pbrs_config(hold_potential_enabled: bool, add_state_info: bool) -> bool: @@ -3593,7 +3597,7 @@ class MyRLEnv(Base5ActionRLEnv): else: # Neutral self-loop next_potential = prev_potential - reward_shaping = 0.0 + reward_shaping = gamma * next_potential - prev_potential self._last_potential = float(next_potential) self._last_prev_potential = float(prev_potential)