PPO, MaskablePPO, RecurrentPPO, DQN, QRDQN
### Configuration tunables
-
The documented list of model tunables is at the top of the
[ReforceXY.py](./ReforceXY/user_data/freqaimodels/ReforceXY.py) file.
-Continual learning reuses the deployed policy and fitted feature pipeline.
-Reset trained models or use a new `freqai.identifier` to change feature
-coordinates or migrate incompatible artifacts. HPO studies and saved best
-parameters are reused only when their objective identity matches.
+### Continual learning
+
+Continual learning trains an independent copy of the deployed policy with its
+fitted feature pipeline. DQN/QRDQN deployments each persist their replay buffer;
+missing or incompatible replay data prevents continuation. Reset trained models
+or use a new `freqai.identifier` to migrate incompatible artifacts, including
+deployments without the chronological training marker. Training disables
+`shuffle_after_split`. HPO studies and saved best parameters are reused only
+when their objective identity matches.
+
+### Live inference
Optional `fit_live_predictions_candles` statistics count produced observations
per pair after session startup; restarts reset the warmup. See the model
docstrings for continuation, HPO and statistics details.
-The rewarding logic and tunables are documented in the
+With `hold_potential_enabled=true`, ReforceXY enables `add_state_info` before
+constructing environments so training and inference use the same observations.
+Freqtrade does not support these state observations in backtesting; disable hold
+potential and state observations for backtests. Live action masks use the real
+open position even when state observations are disabled. Live frame stacks and
+recurrent states persist per pair and model only across adjacent candles. Gaps,
+repeated candles and model replacement start a new sequence; historical live
+position features are not reconstructed. Prediction validity covers every source
+row in the observation and every retained frame, not only the final candle.
+
+### Training and HPO
+
+Optuna HPO trains fresh candidates with the selected parameters. With continual
+learning, the final fit resumes the deployed model in its frozen feature
+coordinates, including its discount factor. Otherwise, it trains a fresh model
+with the selected parameters. An explicitly sampled `target_kl=null` disables
+the KL stopping threshold even when `model_training_parameters` specifies a
+numeric value. Environment prices remain raw regardless of
+`drop_ohlc_from_features`. Training returns the best checkpoint saved by the
+current evaluation run when available. DQN/QRDQN HPO rejects warmup budgets that
+leave no gradient update and trials that finish without learning. A zero-sized
+holdout remains supported when HPO is disabled, including with raw OHLC feature
+removal.
+
+### Reward and portfolio accounting
+
+The reward logic and tunables are documented in the
[reward space analysis](./ReforceXY/reward_space_analysis/README.md).
+Environment diagnostics `most_recent_return` (log return) and
+`most_recent_profit` (simple return) measure changes in liquidation equity,
+including unrealized PnL and Freqtrade's staking convention. Actions fill at
+`open[t]` while the observation window ends at candle `t-1`; equity marks, PnL
+features and trade durations in the returned observation refer to candle `t+1`.
+Round-trip fees are provisioned at entry; exits realize at the fill price without
+charging fees again. `portfolio_log_returns` stores the same log returns.
+Non-positive or non-finite equity produces NaN diagnostics rather than a zero
+return. These diagnostics do not change the training reward or realized capital.
+Rewards combine the fill-time base components with a potential-based shaping
+delta over the returned next observation. Termination liquidates any remaining
+position once and clears the terminal potential. `get_env_history()` returns one
+metrics/price row per transition. Its `execution_tick` is the transition/action/fill
+key before the tick increment; its `tick` is the returned post-increment price and
+observation row (normally `execution_tick + 1`). Ordered trade events remain
+separate in `trade_history`, where each event's `tick` equals the history row's
+`execution_tick`; multiple events may share that key. `terminal_liquidation` and
+`exit_pnl` remain on the transition history row.
+
## Development
-**Run repository quality checks from the repository root:**
+### Runtime regressions
+
+Run the runtime training, inference and accounting regressions inside the
+ReforceXY QA image, with the repository mounted at `/workspace` and `/workspace`
+as the working directory:
+
+```shell
+python -m unittest discover -s ReforceXY/tests -v
+```
+
+### Quality checks
+
+Run repository quality checks from the repository root:
Ruff does not need the Freqtrade runtime or project dependencies:
"diagnostics": [
{
"endCharacter": 58,
- "endLine": 337,
+ "endLine": 324,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Type \"RewardParamValue\" is not assignable to declared type \"bool | None\"\n Type \"RewardParamValue\" is not assignable to type \"bool | None\"\n Type \"float\" is not assignable to type \"bool | None\"\n \"float\" is not assignable to \"bool\"\n \"float\" is not assignable to \"None\"",
"rule": "reportAssignmentType",
"severity": "error",
"startCharacter": 18,
- "startLine": 337
+ "startLine": 324
},
{
"endCharacter": 92,
- "endLine": 564,
+ "endLine": 551,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"RewardParamValue\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"RewardParamValue\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 22,
- "startLine": 564
+ "startLine": 551
},
{
"endCharacter": 92,
- "endLine": 564,
+ "endLine": 551,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"RewardParamValue\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"RewardParamValue\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 22,
- "startLine": 564
+ "startLine": 551
},
{
"endCharacter": 5,
- "endLine": 1889,
+ "endLine": 2047,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "No overloads for \"cut\" match the provided arguments",
"rule": "reportCallIssue",
"severity": "error",
"startCharacter": 17,
- "startLine": 1884
+ "startLine": 2042
},
{
"endCharacter": 21,
- "endLine": 1886,
+ "endLine": 2044,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"_Array1D[Any]\" cannot be assigned to parameter \"bins\" of type \"int | Sequence[float] | Index[int] | Index[float] | IntervalIndex[Interval[Any]] | Series[Any]\" in function \"cut\"\n Type \"_Array1D[Any]\" is not assignable to type \"int | Sequence[float] | Index[int] | Index[float] | IntervalIndex[Interval[Any]] | Series[Any]\"\n \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"int\"\n \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"Sequence[float]\"\n \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"Index[int]\"\n \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"Index[float]\"\n \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"IntervalIndex[Interval[Any]]\"\n \"ndarray[tuple[int], dtype[Any]]\" is not assignable to \"Series[Any]\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 13,
- "startLine": 1886
+ "startLine": 2044
},
{
"endCharacter": 5,
- "endLine": 1911,
+ "endLine": 2069,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to declared type \"RewardParams\"\n Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to type \"RewardParams\"\n \"dict[bytes, bytes]\" is not assignable to \"dict[str, RewardParamValue]\"\n Type parameter \"_KT@dict\" is invariant, but \"bytes\" is not the same as \"str\"\n Type parameter \"_VT@dict\" is invariant, but \"bytes\" is not the same as \"RewardParamValue\"",
"rule": "reportAssignmentType",
"severity": "error",
"startCharacter": 34,
- "startLine": 1907
+ "startLine": 2065
},
{
"endCharacter": 43,
- "endLine": 1908,
+ "endLine": 2066,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "No overloads for \"__init__\" match the provided arguments",
"rule": "reportCallIssue",
"severity": "error",
"startCharacter": 8,
- "startLine": 1908
+ "startLine": 2066
},
{
"endCharacter": 42,
- "endLine": 1908,
+ "endLine": 2066,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"iterable\" of type \"Iterable[list[bytes]]\" in function \"__init__\"\n Type \"Any | None\" is not assignable to type \"Iterable[list[bytes]]\"\n \"None\" is incompatible with protocol \"Iterable[list[bytes]]\"\n \"__iter__\" is not present",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 13,
- "startLine": 1908
+ "startLine": 2066
},
{
"endCharacter": 60,
- "endLine": 2167,
+ "endLine": 2326,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"importances_mean\" for class \"dict[Unknown, Bunch]\"\n Attribute \"importances_mean\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 44,
- "startLine": 2167
+ "startLine": 2326
},
{
"endCharacter": 58,
- "endLine": 2168,
+ "endLine": 2327,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"importances_std\" for class \"dict[Unknown, Bunch]\"\n Attribute \"importances_std\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 43,
- "startLine": 2168
+ "startLine": 2327
},
{
"endCharacter": 88,
- "endLine": 2187,
+ "endLine": 2346,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"columns\" for class \"NDArray[Unknown]\"\n Attribute \"columns\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 81,
- "startLine": 2187
+ "startLine": 2346
},
{
"endCharacter": 88,
- "endLine": 2187,
+ "endLine": 2346,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"columns\" for class \"list[Unknown]\"\n Attribute \"columns\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 81,
- "startLine": 2187
+ "startLine": 2346
},
{
"endCharacter": 17,
- "endLine": 2196,
+ "endLine": 2355,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Object of type \"None\" cannot be called",
"rule": "reportOptionalCall",
"severity": "error",
"startCharacter": 28,
- "startLine": 2190
+ "startLine": 2349
},
{
"endCharacter": 40,
- "endLine": 2410,
+ "endLine": 2567,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"Any | object | NAType | Unknown\" cannot be assigned to parameter \"arg1\" of type \"SupportsRichComparisonT@min\" in function \"min\"\n Type \"Any | object | NAType | Unknown\" is not assignable to type \"SupportsRichComparison\"\n Type \"Any | object | NAType | Unknown\" is not assignable to type \"SupportsRichComparison\"\n Type \"object\" is not assignable to type \"SupportsRichComparison\"\n \"object\" is incompatible with protocol \"SupportsDunderLT[Any]\"\n \"__lt__\" is not present\n \"object\" is incompatible with protocol \"SupportsDunderGT[Any]\"\n \"__gt__\" is not present",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 22,
- "startLine": 2410
+ "startLine": 2567
},
{
"endCharacter": 38,
- "endLine": 2410,
+ "endLine": 2567,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"min\" for class \"ExtensionArray\"\n Attribute \"min\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 35,
- "startLine": 2410
+ "startLine": 2567
},
{
"endCharacter": 59,
- "endLine": 2410,
+ "endLine": 2567,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"Any | object | NAType | Unknown\" cannot be assigned to parameter \"arg2\" of type \"SupportsRichComparisonT@min\" in function \"min\"\n Type \"Any | object | NAType | Unknown\" is not assignable to type \"SupportsRichComparison\"\n Type \"Any | object | NAType | Unknown\" is not assignable to type \"SupportsRichComparison\"\n Type \"object\" is not assignable to type \"SupportsRichComparison\"\n \"object\" is incompatible with protocol \"SupportsDunderLT[Any]\"\n \"__lt__\" is not present\n \"object\" is incompatible with protocol \"SupportsDunderGT[Any]\"\n \"__gt__\" is not present",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 42,
- "startLine": 2410
+ "startLine": 2567
},
{
"endCharacter": 57,
- "endLine": 2410,
+ "endLine": 2567,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"min\" for class \"ExtensionArray\"\n Attribute \"min\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 54,
- "startLine": 2410
+ "startLine": 2567
},
{
"endCharacter": 40,
- "endLine": 2411,
+ "endLine": 2568,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"Any | object | NAType | Unknown\" cannot be assigned to parameter \"arg1\" of type \"SupportsRichComparisonT@max\" in function \"max\"\n Type \"Any | object | NAType | Unknown\" is not assignable to type \"SupportsRichComparison\"\n Type \"Any | object | NAType | Unknown\" is not assignable to type \"SupportsRichComparison\"\n Type \"object\" is not assignable to type \"SupportsRichComparison\"\n \"object\" is incompatible with protocol \"SupportsDunderLT[Any]\"\n \"__lt__\" is not present\n \"object\" is incompatible with protocol \"SupportsDunderGT[Any]\"\n \"__gt__\" is not present",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 22,
- "startLine": 2411
+ "startLine": 2568
},
{
"endCharacter": 38,
- "endLine": 2411,
+ "endLine": 2568,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"max\" for class \"ExtensionArray\"\n Attribute \"max\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 35,
- "startLine": 2411
+ "startLine": 2568
},
{
"endCharacter": 59,
- "endLine": 2411,
+ "endLine": 2568,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"Any | object | NAType | Unknown\" cannot be assigned to parameter \"arg2\" of type \"SupportsRichComparisonT@max\" in function \"max\"\n Type \"Any | object | NAType | Unknown\" is not assignable to type \"SupportsRichComparison\"\n Type \"Any | object | NAType | Unknown\" is not assignable to type \"SupportsRichComparison\"\n Type \"object\" is not assignable to type \"SupportsRichComparison\"\n \"object\" is incompatible with protocol \"SupportsDunderLT[Any]\"\n \"__lt__\" is not present\n \"object\" is incompatible with protocol \"SupportsDunderGT[Any]\"\n \"__gt__\" is not present",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 42,
- "startLine": 2411
+ "startLine": 2568
},
{
"endCharacter": 57,
- "endLine": 2411,
+ "endLine": 2568,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"max\" for class \"ExtensionArray\"\n Attribute \"max\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 54,
- "startLine": 2411
+ "startLine": 2568
},
{
"endCharacter": 76,
- "endLine": 2431,
+ "endLine": 2584,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "No overloads for \"histogram\" match the provided arguments",
"rule": "reportCallIssue",
"severity": "error",
"startCharacter": 24,
- "startLine": 2431
+ "startLine": 2584
},
{
"endCharacter": 49,
- "endLine": 2431,
+ "endLine": 2584,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"a\" of type \"_ArrayLikeComplex_co\" in function \"histogram\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"_ArrayLikeComplex_co\"\n Type \"ExtensionArray\" is not assignable to type \"_ArrayLikeComplex_co\"\n \"ExtensionArray\" is incompatible with protocol \"_SupportsArray[dtype[numpy.bool[builtins.bool] | number[Any, Any]]]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"_NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool] | number[Any, Any]]]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 37,
- "startLine": 2431
+ "startLine": 2584
},
{
"endCharacter": 74,
- "endLine": 2432,
+ "endLine": 2585,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "No overloads for \"histogram\" match the provided arguments",
"rule": "reportCallIssue",
"severity": "error",
"startCharacter": 23,
- "startLine": 2432
+ "startLine": 2585
},
{
"endCharacter": 47,
- "endLine": 2432,
+ "endLine": 2585,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"a\" of type \"_ArrayLikeComplex_co\" in function \"histogram\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"_ArrayLikeComplex_co\"\n Type \"ExtensionArray\" is not assignable to type \"_ArrayLikeComplex_co\"\n \"ExtensionArray\" is incompatible with protocol \"_SupportsArray[dtype[numpy.bool[builtins.bool] | number[Any, Any]]]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"_NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool] | number[Any, Any]]]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 36,
- "startLine": 2432
+ "startLine": 2585
},
{
"endCharacter": 51,
- "endLine": 2447,
+ "endLine": 2600,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"u_values\" of type \"ToFloatND\" in function \"wasserstein_distance\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"ToFloatND\"\n Type \"ExtensionArray\" is not assignable to type \"ToFloatND\"\n \"ExtensionArray\" is incompatible with protocol \"_CanArrayND[floating_co]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"SequenceND[py_float | _CanArray[floating_co]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 39,
- "startLine": 2447
+ "startLine": 2600
},
{
"endCharacter": 64,
- "endLine": 2447,
+ "endLine": 2600,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"v_values\" of type \"ToFloatND\" in function \"wasserstein_distance\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"ToFloatND\"\n Type \"ExtensionArray\" is not assignable to type \"ToFloatND\"\n \"ExtensionArray\" is incompatible with protocol \"_CanArrayND[floating_co]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"SequenceND[py_float | _CanArray[floating_co]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 53,
- "startLine": 2447
+ "startLine": 2600
},
{
"endCharacter": 68,
- "endLine": 2450,
+ "endLine": 2603,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "No overloads for \"ks_2samp\" match the provided arguments",
"rule": "reportCallIssue",
"severity": "error",
"startCharacter": 27,
- "startLine": 2450
+ "startLine": 2603
},
{
"endCharacter": 54,
- "endLine": 2450,
+ "endLine": 2603,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"data1\" of type \"ToFloatND\" in function \"ks_2samp\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"ToFloatND\"\n Type \"ExtensionArray\" is not assignable to type \"ToFloatND\"\n \"ExtensionArray\" is incompatible with protocol \"_CanArrayND[floating_co]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"SequenceND[py_float | _CanArray[floating_co]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 42,
- "startLine": 2450
+ "startLine": 2603
},
{
"endCharacter": 67,
- "endLine": 2450,
+ "endLine": 2603,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"data2\" of type \"ToFloatND\" in function \"ks_2samp\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"ToFloatND\"\n Type \"ExtensionArray\" is not assignable to type \"ToFloatND\"\n \"ExtensionArray\" is incompatible with protocol \"_CanArrayND[floating_co]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"SequenceND[py_float | _CanArray[floating_co]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 56,
- "startLine": 2450
+ "startLine": 2603
},
{
"endCharacter": 29,
- "endLine": 2722,
+ "endLine": 2903,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "No overloads for \"ptp\" match the provided arguments",
"rule": "reportCallIssue",
"severity": "error",
"startCharacter": 11,
- "startLine": 2722
+ "startLine": 2903
},
{
"endCharacter": 28,
- "endLine": 2722,
+ "endLine": 2903,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"a\" of type \"_ArrayLikeNumeric_co\" in function \"ptp\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"_ArrayLikeNumeric_co\"\n Type \"ExtensionArray\" is not assignable to type \"_ArrayLikeNumeric_co\"\n \"ExtensionArray\" is incompatible with protocol \"_SupportsArray[dtype[number[Any, Any] | numpy.bool[builtins.bool] | object_ | timedelta64[Any]]]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"_NestedSequence[_SupportsArray[dtype[number[Any, Any] | numpy.bool[builtins.bool] | object_ | timedelta64[Any]]]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 18,
- "startLine": 2722
+ "startLine": 2903
},
{
"endCharacter": 65,
- "endLine": 2735,
+ "endLine": 2912,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"mean\" for class \"Categorical[object]\"\n Attribute \"mean\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 61,
- "startLine": 2735
+ "startLine": 2912
},
{
"endCharacter": 65,
- "endLine": 2735,
+ "endLine": 2912,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Cannot access attribute \"mean\" for class \"ExtensionArray\"\n Attribute \"mean\" is unknown",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 61,
- "startLine": 2735
- },
- {
- "endCharacter": 56,
- "endLine": 2815,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "No overloads for \"mean\" match the provided arguments",
- "rule": "reportCallIssue",
- "severity": "error",
- "startCharacter": 43,
- "startLine": 2815
- },
- {
- "endCharacter": 55,
- "endLine": 2815,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"a\" of type \"_ArrayLikeNumeric_co\" in function \"mean\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"_ArrayLikeNumeric_co\"\n Type \"ExtensionArray\" is not assignable to type \"_ArrayLikeNumeric_co\"\n \"ExtensionArray\" is incompatible with protocol \"_SupportsArray[dtype[number[Any, Any] | numpy.bool[builtins.bool] | object_ | timedelta64[Any]]]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"_NestedSequence[_SupportsArray[dtype[number[Any, Any] | numpy.bool[builtins.bool] | object_ | timedelta64[Any]]]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 51,
- "startLine": 2815
- },
- {
- "endCharacter": 62,
- "endLine": 2816,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "No overloads for \"std\" match the provided arguments",
- "rule": "reportCallIssue",
- "severity": "error",
- "startCharacter": 42,
- "startLine": 2816
- },
- {
- "endCharacter": 53,
- "endLine": 2816,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"a\" of type \"_ArrayLikeNumeric_co\" in function \"std\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"_ArrayLikeNumeric_co\"\n Type \"ExtensionArray\" is not assignable to type \"_ArrayLikeNumeric_co\"\n \"ExtensionArray\" is incompatible with protocol \"_SupportsArray[dtype[number[Any, Any] | numpy.bool[builtins.bool] | object_ | timedelta64[Any]]]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"_NestedSequence[_SupportsArray[dtype[number[Any, Any] | numpy.bool[builtins.bool] | object_ | timedelta64[Any]]]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 49,
- "startLine": 2816
- },
- {
- "endCharacter": 39,
- "endLine": 2817,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "No overloads for \"skew\" match the provided arguments",
- "rule": "reportCallIssue",
- "severity": "error",
- "startCharacter": 23,
- "startLine": 2817
- },
- {
- "endCharacter": 38,
- "endLine": 2817,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"a\" of type \"ToFloatND\" in function \"skew\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"ToFloatND\"\n Type \"ExtensionArray\" is not assignable to type \"ToFloatND\"\n \"ExtensionArray\" is incompatible with protocol \"_CanArrayND[floating_co]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"SequenceND[py_float | _CanArray[floating_co]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 34,
- "startLine": 2817
- },
- {
- "endCharacter": 56,
- "endLine": 2818,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "No overloads for \"kurtosis\" match the provided arguments",
- "rule": "reportCallIssue",
- "severity": "error",
- "startCharacter": 23,
- "startLine": 2818
- },
- {
- "endCharacter": 42,
- "endLine": 2818,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"a\" of type \"ToFloatND\" in function \"kurtosis\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"ToFloatND\"\n Type \"ExtensionArray\" is not assignable to type \"ToFloatND\"\n \"ExtensionArray\" is incompatible with protocol \"_CanArrayND[floating_co]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"SequenceND[py_float | _CanArray[floating_co]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 38,
- "startLine": 2818
- },
- {
- "endCharacter": 50,
- "endLine": 2829,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "No overloads for \"shapiro\" match the provided arguments",
- "rule": "reportCallIssue",
- "severity": "error",
- "startCharacter": 31,
- "startLine": 2829
- },
- {
- "endCharacter": 49,
- "endLine": 2829,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"x\" of type \"ToFloat | ToFloatND\" in function \"shapiro\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"ToFloat | ToFloatND\"\n Type \"ExtensionArray\" is not assignable to type \"ToFloat | ToFloatND\"\n \"ExtensionArray\" is not assignable to \"float\"\n \"ExtensionArray\" is not assignable to \"floating[Any]\"\n \"ExtensionArray\" is not assignable to \"integer[Any]\"\n \"ExtensionArray\" is not assignable to \"numpy.bool[builtins.bool]\"\n \"ExtensionArray\" is incompatible with protocol \"_CanArrayND[floating_co]\"\n \"__array__\" is not present\n ...",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 45,
- "startLine": 2829
- },
- {
- "endCharacter": 39,
- "endLine": 2834,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"x\" of type \"ToFloatND\" in function \"anderson\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"ToFloatND\"\n Type \"ExtensionArray\" is not assignable to type \"ToFloatND\"\n \"ExtensionArray\" is incompatible with protocol \"_CanArrayND[floating_co]\"\n \"__array__\" is not present\n \"ExtensionArray\" is incompatible with protocol \"SequenceND[py_float | _CanArray[floating_co]]\"\n \"__reversed__\" is not present\n \"count\" is not present\n \"index\" is not present\n ...",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 35,
- "startLine": 2834
- },
- {
- "endCharacter": 61,
- "endLine": 2841,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Argument of type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" cannot be assigned to parameter \"x\" of type \"ToFloat | ToFloatND\" in function \"probplot\"\n Type \"np_1darray[Any] | ExtensionArray | Categorical[object]\" is not assignable to type \"ToFloat | ToFloatND\"\n Type \"ExtensionArray\" is not assignable to type \"ToFloat | ToFloatND\"\n \"ExtensionArray\" is not assignable to \"float\"\n \"ExtensionArray\" is not assignable to \"floating[Any]\"\n \"ExtensionArray\" is not assignable to \"integer[Any]\"\n \"ExtensionArray\" is not assignable to \"numpy.bool[builtins.bool]\"\n \"ExtensionArray\" is incompatible with protocol \"_CanArrayND[floating_co]\"\n \"__array__\" is not present\n ...",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 57,
- "startLine": 2841
- },
- {
- "endCharacter": 17,
- "endLine": 3640,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Declaration \"reward_params\" is obscured by a declaration of the same name",
- "rule": "reportRedeclaration",
- "severity": "error",
- "startCharacter": 4,
- "startLine": 3640
+ "startLine": 2912
},
{
"endCharacter": 5,
- "endLine": 3644,
+ "endLine": 3803,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to declared type \"RewardParams\"\n Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to type \"RewardParams\"\n \"dict[bytes, bytes]\" is not assignable to \"dict[str, RewardParamValue]\"\n Type parameter \"_KT@dict\" is invariant, but \"bytes\" is not the same as \"str\"\n Type parameter \"_VT@dict\" is invariant, but \"bytes\" is not the same as \"RewardParamValue\"",
"rule": "reportAssignmentType",
"severity": "error",
"startCharacter": 34,
- "startLine": 3640
+ "startLine": 3799
},
{
"endCharacter": 43,
- "endLine": 3641,
+ "endLine": 3800,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "No overloads for \"__init__\" match the provided arguments",
"rule": "reportCallIssue",
"severity": "error",
"startCharacter": 8,
- "startLine": 3641
+ "startLine": 3800
},
{
"endCharacter": 42,
- "endLine": 3641,
+ "endLine": 3800,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"iterable\" of type \"Iterable[list[bytes]]\" in function \"__init__\"\n Type \"Any | None\" is not assignable to type \"Iterable[list[bytes]]\"\n \"None\" is incompatible with protocol \"Iterable[list[bytes]]\"\n \"__iter__\" is not present",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 13,
- "startLine": 3641
- },
- {
- "endCharacter": 9,
- "endLine": 3787,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to declared type \"RewardParams\"\n Type \"dict[bytes, bytes] | dict[str, RewardParamValue]\" is not assignable to type \"RewardParams\"\n \"dict[bytes, bytes]\" is not assignable to \"dict[str, RewardParamValue]\"\n Type parameter \"_KT@dict\" is invariant, but \"bytes\" is not the same as \"str\"\n Type parameter \"_VT@dict\" is invariant, but \"bytes\" is not the same as \"RewardParamValue\"",
- "rule": "reportAssignmentType",
- "severity": "error",
- "startCharacter": 38,
- "startLine": 3783
- },
- {
- "endCharacter": 47,
- "endLine": 3784,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "No overloads for \"__init__\" match the provided arguments",
- "rule": "reportCallIssue",
- "severity": "error",
- "startCharacter": 12,
- "startLine": 3784
- },
- {
- "endCharacter": 46,
- "endLine": 3784,
- "file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
- "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"iterable\" of type \"Iterable[list[bytes]]\" in function \"__init__\"\n Type \"Any | None\" is not assignable to type \"Iterable[list[bytes]]\"\n \"None\" is incompatible with protocol \"Iterable[list[bytes]]\"\n \"__iter__\" is not present",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 17,
- "startLine": 3784
+ "startLine": 3800
},
{
"endCharacter": 14,
- "endLine": 4496,
+ "endLine": 4647,
"file": "ReforceXY/reward_space_analysis/reward_space_analysis.py",
"message": "Declaration \"sim_params\" is obscured by a declaration of the same name",
"rule": "reportRedeclaration",
"severity": "error",
"startCharacter": 4,
- "startLine": 4496
- },
- {
- "endCharacter": 60,
- "endLine": 755,
- "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 30,
- "startLine": 755
- },
- {
- "endCharacter": 60,
- "endLine": 755,
- "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 30,
- "startLine": 755
+ "startLine": 4647
},
{
"endCharacter": 22,
- "endLine": 827,
+ "endLine": 914,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Cannot assign to attribute \"train_env\" for class \"ReforceXY*\"\n Expression of type \"VecEnv\" cannot be assigned to attribute \"train_env\" of class \"ReforceXY\"\n Type \"VecEnv\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n \"VecEnv\" is not assignable to \"VecMonitor\"\n \"VecEnv\" is not assignable to \"SubprocVecEnv\"\n \"VecEnv\" is not assignable to \"Env[Unknown, Unknown]\"",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 13,
- "startLine": 827
+ "startLine": 914
},
{
"endCharacter": 37,
- "endLine": 827,
+ "endLine": 914,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Cannot assign to attribute \"eval_env\" for class \"ReforceXY*\"\n Expression of type \"VecEnv\" cannot be assigned to attribute \"eval_env\" of class \"ReforceXY\"\n Type \"VecEnv\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n \"VecEnv\" is not assignable to \"VecMonitor\"\n \"VecEnv\" is not assignable to \"SubprocVecEnv\"\n \"VecEnv\" is not assignable to \"Env[Unknown, Unknown]\"",
+ "message": "Cannot assign to attribute \"eval_env\" for class \"ReforceXY*\"\n Expression of type \"VecEnv | None\" cannot be assigned to attribute \"eval_env\" of class \"ReforceXY\"\n Type \"VecEnv | None\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n Type \"VecEnv\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n \"VecEnv\" is not assignable to \"VecMonitor\"\n \"VecEnv\" is not assignable to \"SubprocVecEnv\"\n \"VecEnv\" is not assignable to \"Env[Unknown, Unknown]\"",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 29,
- "startLine": 827
+ "startLine": 914
+ },
+ {
+ "endCharacter": 24,
+ "endLine": 1173,
+ "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
+ "message": "Argument of type \"VecEnv\" cannot be assigned to parameter \"eval_env\" of type \"BaseEnvironment\" in function \"__init__\"\n \"VecEnv\" is not assignable to \"BaseEnvironment\"",
+ "rule": "reportArgumentType",
+ "severity": "error",
+ "startCharacter": 16,
+ "startLine": 1173
},
{
"endCharacter": 38,
- "endLine": 1260,
+ "endLine": 1378,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"obj\" of type \"Sized\" in function \"len\"\n Type \"Any | None\" is not assignable to type \"Sized\"\n \"None\" is incompatible with protocol \"Sized\"\n \"__len__\" is not present",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 30,
- "startLine": 1260
+ "startLine": 1378
},
{
"endCharacter": 36,
- "endLine": 1264,
+ "endLine": 1382,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"obj\" of type \"Sized\" in function \"len\"\n Type \"Any | None\" is not assignable to type \"Sized\"\n \"None\" is incompatible with protocol \"Sized\"\n \"__len__\" is not present",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 29,
- "startLine": 1264
+ "startLine": 1382
},
{
"endCharacter": 80,
- "endLine": 1267,
+ "endLine": 1385,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"timeframe\" of type \"str\" in function \"steps_to_days\"\n Type \"Any | None\" is not assignable to type \"str\"\n \"None\" is not assignable to \"str\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 52,
- "startLine": 1267
+ "startLine": 1385
},
{
"endCharacter": 78,
- "endLine": 1268,
+ "endLine": 1386,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"timeframe\" of type \"str\" in function \"steps_to_days\"\n Type \"Any | None\" is not assignable to type \"str\"\n \"None\" is not assignable to \"str\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 50,
- "startLine": 1268
+ "startLine": 1386
},
{
"endCharacter": 80,
- "endLine": 1269,
+ "endLine": 1387,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"timeframe\" of type \"str\" in function \"steps_to_days\"\n Type \"Any | None\" is not assignable to type \"str\"\n \"None\" is not assignable to \"str\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 52,
- "startLine": 1269
+ "startLine": 1387
},
{
- "endCharacter": 83,
- "endLine": 1341,
+ "endCharacter": 56,
+ "endLine": 1472,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"timeframe\" of type \"str\" in function \"steps_to_days\"\n Type \"Any | None\" is not assignable to type \"str\"\n \"None\" is not assignable to \"str\"",
+ "message": "Argument of type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\" cannot be assigned to parameter \"eval_env\" of type \"VecEnv | None\" in function \"get_callbacks\"\n Type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\" is not assignable to type \"VecEnv | None\"\n Type \"Env[Unknown, Unknown]\" is not assignable to type \"VecEnv | None\"\n \"Env[Unknown, Unknown]\" is not assignable to \"VecEnv\"\n \"Env[Unknown, Unknown]\" is not assignable to \"None\"",
"rule": "reportArgumentType",
"severity": "error",
- "startCharacter": 55,
- "startLine": 1341
+ "startCharacter": 43,
+ "startLine": 1472
},
{
- "endCharacter": 52,
- "endLine": 1382,
+ "endCharacter": 84,
+ "endLine": 1492,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\" cannot be assigned to parameter \"eval_env\" of type \"BaseEnvironment\" in function \"get_callbacks\"\n Type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\" is not assignable to type \"BaseEnvironment\"\n \"SubprocVecEnv\" is not assignable to \"BaseEnvironment\"",
+ "message": "Argument of type \"float | list[float]\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | list[float]\" is not assignable to type \"ConvertibleToFloat\"\n Type \"list[float]\" is not assignable to type \"ConvertibleToFloat\"\n \"list[float]\" is not assignable to \"str\"\n \"list[float]\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"list[float]\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"list[float]\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
- "startCharacter": 39,
- "startLine": 1382
+ "startCharacter": 67,
+ "startLine": 1492
},
{
- "endCharacter": 50,
- "endLine": 1611,
+ "endCharacter": 84,
+ "endLine": 1492,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"NDArray[float32]\" cannot be assigned to parameter \"x\" of type \"float32\" in function \"append\"\n \"ndarray[_AnyShape, dtype[float32]]\" is not assignable to \"floating[_32Bit]\"",
+ "message": "Argument of type \"float | list[float]\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | list[float]\" is not assignable to type \"ConvertibleToFloat\"\n Type \"list[float]\" is not assignable to type \"ConvertibleToFloat\"\n \"list[float]\" is not assignable to \"str\"\n \"list[float]\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"list[float]\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"list[float]\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
- "startCharacter": 36,
- "startLine": 1611
+ "startCharacter": 67,
+ "startLine": 1492
},
{
"endCharacter": 44,
- "endLine": 2372,
+ "endLine": 2586,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "\"best_trial_params\" is possibly unbound",
"rule": "reportPossiblyUnboundVariable",
"severity": "error",
"startCharacter": 27,
- "startLine": 2372
+ "startLine": 2586
},
{
"endCharacter": 12,
- "endLine": 2392,
+ "endLine": 2606,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Parameter declaration \"seed\" is obscured by a declaration of the same name",
"rule": "reportRedeclaration",
"severity": "error",
"startCharacter": 8,
- "startLine": 2392
+ "startLine": 2606
},
{
"endCharacter": 16,
- "endLine": 2393,
+ "endLine": 2607,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Parameter declaration \"env_info\" is obscured by a declaration of the same name",
"rule": "reportRedeclaration",
"severity": "error",
"startCharacter": 8,
- "startLine": 2393
- },
- {
- "endCharacter": 47,
- "endLine": 2432,
- "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"list[() -> BaseEnvironment]\" cannot be assigned to parameter \"env_fns\" of type \"list[() -> Env[Unknown, Unknown]]\" in function \"__init__\"\n \"list[() -> BaseEnvironment]\" is not assignable to \"list[() -> Env[Unknown, Unknown]]\"\n Type parameter \"_T@list\" is invariant, but \"() -> BaseEnvironment\" is not the same as \"() -> Env[Unknown, Unknown]\"\n Consider switching from \"list\" to \"Sequence\" which is covariant",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 38,
- "startLine": 2432
- },
- {
- "endCharacter": 45,
- "endLine": 2434,
- "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"list[() -> BaseEnvironment]\" cannot be assigned to parameter \"env_fns\" of type \"list[() -> Env[Unknown, Unknown]]\" in function \"__init__\"\n \"list[() -> BaseEnvironment]\" is not assignable to \"list[() -> Env[Unknown, Unknown]]\"\n Type parameter \"_T@list\" is invariant, but \"() -> BaseEnvironment\" is not the same as \"() -> Env[Unknown, Unknown]\"\n Consider switching from \"list\" to \"Sequence\" which is covariant",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 36,
- "startLine": 2434
+ "startLine": 2607
},
{
- "endCharacter": 45,
- "endLine": 2436,
+ "endCharacter": 51,
+ "endLine": 2650,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"list[() -> BaseEnvironment]\" cannot be assigned to parameter \"env_fns\" of type \"list[() -> Env[Unknown, Unknown]]\" in function \"__init__\"\n \"list[() -> BaseEnvironment]\" is not assignable to \"list[() -> Env[Unknown, Unknown]]\"\n Type parameter \"_T@list\" is invariant, but \"() -> BaseEnvironment\" is not the same as \"() -> Env[Unknown, Unknown]\"\n Consider switching from \"list\" to \"Sequence\" which is covariant",
"rule": "reportArgumentType",
"severity": "error",
- "startCharacter": 37,
- "startLine": 2436
+ "startCharacter": 42,
+ "startLine": 2650
},
{
- "endCharacter": 43,
- "endLine": 2438,
+ "endCharacter": 53,
+ "endLine": 2656,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"list[() -> BaseEnvironment]\" cannot be assigned to parameter \"env_fns\" of type \"list[() -> Env[Unknown, Unknown]]\" in function \"__init__\"\n \"list[() -> BaseEnvironment]\" is not assignable to \"list[() -> Env[Unknown, Unknown]]\"\n Type parameter \"_T@list\" is invariant, but \"() -> BaseEnvironment\" is not the same as \"() -> Env[Unknown, Unknown]\"\n Consider switching from \"list\" to \"Sequence\" which is covariant",
"rule": "reportArgumentType",
"severity": "error",
- "startCharacter": 35,
- "startLine": 2438
+ "startCharacter": 45,
+ "startLine": 2656
},
{
"endCharacter": 22,
- "endLine": 2486,
+ "endLine": 2719,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Operator \"*\" not supported for \"None\"",
"rule": "reportOptionalOperand",
"severity": "error",
"startCharacter": 15,
- "startLine": 2486
+ "startLine": 2719
},
{
"endCharacter": 96,
- "endLine": 2488,
+ "endLine": 2721,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Operator \"*\" not supported for \"None\"",
"rule": "reportOptionalOperand",
"severity": "error",
"startCharacter": 89,
- "startLine": 2488
+ "startLine": 2721
},
{
"endCharacter": 23,
- "endLine": 2491,
+ "endLine": 2724,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Operator \"*\" not supported for \"None\"",
"rule": "reportOptionalOperand",
"severity": "error",
"startCharacter": 16,
- "startLine": 2491
+ "startLine": 2724
},
{
"endCharacter": 98,
- "endLine": 2493,
+ "endLine": 2726,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Operator \"*\" not supported for \"None\"",
"rule": "reportOptionalOperand",
"severity": "error",
"startCharacter": 91,
- "startLine": 2493
+ "startLine": 2726
},
{
"endCharacter": 44,
- "endLine": 2505,
+ "endLine": 2738,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Operator \"*\" not supported for types \"Any | None\" and \"Any | int | None\"\n Operator \"*\" not supported for types \"None\" and \"None\"\n Operator \"*\" not supported for types \"None\" and \"int\"",
"rule": "reportOperatorIssue",
"severity": "error",
"startCharacter": 15,
- "startLine": 2505
+ "startLine": 2738
},
{
"endCharacter": 133,
- "endLine": 2507,
+ "endLine": 2740,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Operator \"*\" not supported for types \"Any | None\" and \"Any | int | None\"\n Operator \"*\" not supported for types \"None\" and \"None\"\n Operator \"*\" not supported for types \"None\" and \"int\"",
"rule": "reportOperatorIssue",
"severity": "error",
"startCharacter": 106,
- "startLine": 2507
+ "startLine": 2740
},
{
"endCharacter": 30,
- "endLine": 2510,
+ "endLine": 2743,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Operator \">\" not supported for \"None\"",
"rule": "reportOptionalOperand",
"severity": "error",
"startCharacter": 15,
- "startLine": 2510
+ "startLine": 2743
},
{
- "endCharacter": 47,
- "endLine": 2560,
+ "endCharacter": 54,
+ "endLine": 2808,
+ "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
+ "message": "\"is_pruned\" is not a known attribute of \"None\"",
+ "rule": "reportOptionalMemberAccess",
+ "severity": "error",
+ "startCharacter": 45,
+ "startLine": 2808
+ },
+ {
+ "endCharacter": 67,
+ "endLine": 2811,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"VecEnv\" cannot be assigned to parameter \"eval_env\" of type \"BaseEnvironment\" in function \"get_callbacks\"\n \"VecEnv\" is not assignable to \"BaseEnvironment\"",
+ "message": "\"use_masking\" is not a known attribute of \"None\"",
+ "rule": "reportOptionalMemberAccess",
+ "severity": "error",
+ "startCharacter": 56,
+ "startLine": 2811
+ },
+ {
+ "endCharacter": 60,
+ "endLine": 2819,
+ "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
+ "message": "\"update_best_reward\" is not a known attribute of \"None\"",
+ "rule": "reportOptionalMemberAccess",
+ "severity": "error",
+ "startCharacter": 42,
+ "startLine": 2819
+ },
+ {
+ "endCharacter": 84,
+ "endLine": 2819,
+ "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
+ "message": "Argument of type \"float | list[float]\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | list[float]\" is not assignable to type \"ConvertibleToFloat\"\n Type \"list[float]\" is not assignable to type \"ConvertibleToFloat\"\n \"list[float]\" is not assignable to \"str\"\n \"list[float]\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"list[float]\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"list[float]\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
- "startCharacter": 39,
- "startLine": 2560
+ "startCharacter": 67,
+ "startLine": 2819
+ },
+ {
+ "endCharacter": 84,
+ "endLine": 2819,
+ "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
+ "message": "Argument of type \"float | list[float]\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | list[float]\" is not assignable to type \"ConvertibleToFloat\"\n Type \"list[float]\" is not assignable to type \"ConvertibleToFloat\"\n \"list[float]\" is not assignable to \"str\"\n \"list[float]\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"list[float]\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"list[float]\" is incompatible with protocol \"SupportsIndex\"\n ...",
+ "rule": "reportArgumentType",
+ "severity": "error",
+ "startCharacter": 67,
+ "startLine": 2819
},
{
"endCharacter": 46,
- "endLine": 2617,
+ "endLine": 2873,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "\"is_pruned\" is not a known attribute of \"None\"",
"rule": "reportOptionalMemberAccess",
"severity": "error",
"startCharacter": 37,
- "startLine": 2617
+ "startLine": 2873
},
{
"endCharacter": 57,
- "endLine": 2620,
+ "endLine": 2876,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "\"best_mean_reward\" is not a known attribute of \"None\"",
"rule": "reportOptionalMemberAccess",
"severity": "error",
"startCharacter": 41,
- "startLine": 2620
+ "startLine": 2876
},
{
"endCharacter": 37,
- "endLine": 2630,
+ "endLine": 2886,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Cannot assign to attribute \"train_env\" for class \"ReforceXY*\"\n Type \"None\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n \"None\" is not assignable to \"VecMonitor\"\n \"None\" is not assignable to \"SubprocVecEnv\"\n \"None\" is not assignable to \"Env[Unknown, Unknown]\"",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 33,
- "startLine": 2630
+ "startLine": 2886
},
{
"endCharacter": 36,
- "endLine": 2635,
+ "endLine": 2891,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Cannot assign to attribute \"eval_env\" for class \"ReforceXY*\"\n Type \"None\" is not assignable to type \"VecMonitor | SubprocVecEnv | Env[Unknown, Unknown]\"\n \"None\" is not assignable to \"VecMonitor\"\n \"None\" is not assignable to \"SubprocVecEnv\"\n \"None\" is not assignable to \"Env[Unknown, Unknown]\"",
"rule": "reportAttributeAccessIssue",
"severity": "error",
"startCharacter": 32,
- "startLine": 2635
+ "startLine": 2891
},
{
"endCharacter": 7,
- "endLine": 2667,
+ "endLine": 2936,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Declaration \"MyRLEnv\" is obscured by a declaration of the same name",
"rule": "reportRedeclaration",
"severity": "error",
"startCharacter": 0,
- "startLine": 2667
- },
- {
- "endCharacter": 34,
- "endLine": 2679,
- "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Declaration \"_last_closed_position\" is obscured by a declaration of the same name",
- "rule": "reportRedeclaration",
- "severity": "error",
- "startCharacter": 13,
- "startLine": 2679
+ "startLine": 2936
},
{
"endCharacter": 36,
- "endLine": 2680,
+ "endLine": 2948,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Declaration \"_last_closed_trade_tick\" is obscured by a declaration of the same name",
"rule": "reportRedeclaration",
"severity": "error",
"startCharacter": 13,
- "startLine": 2680
+ "startLine": 2948
},
{
"endCharacter": 13,
- "endLine": 3516,
+ "endLine": 3540,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Method \"reset\" overrides class \"BaseEnvironment\" in an incompatible manner\n Return type mismatch: base method returns type \"tuple[DataFrame, dict[Unknown, Unknown]]\", override returns type \"tuple[NDArray[float32], dict[str, Any]]\"\n \"tuple[NDArray[float32], dict[str, Any]]\" is not assignable to \"tuple[DataFrame, dict[Unknown, Unknown]]\"\n Tuple entry 1 is incorrect type\n \"ndarray[_AnyShape, dtype[float32]]\" is not assignable to \"DataFrame\"",
"rule": "reportIncompatibleMethodOverride",
"severity": "error",
"startCharacter": 8,
- "startLine": 3516
+ "startLine": 3540
},
{
"endCharacter": 26,
- "endLine": 3538,
+ "endLine": 3561,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Type \"tuple[DataFrame, dict[Unknown, Unknown]]\" is not assignable to return type \"tuple[NDArray[float32], dict[str, Any]]\"\n \"DataFrame\" is not assignable to \"ndarray[_AnyShape, dtype[float32]]\"",
"rule": "reportReturnType",
"severity": "error",
"startCharacter": 15,
- "startLine": 3538
+ "startLine": 3561
},
{
"endCharacter": 55,
- "endLine": 3867,
+ "endLine": 3869,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 44,
- "startLine": 3867
+ "startLine": 3869
},
{
"endCharacter": 55,
- "endLine": 3867,
+ "endLine": 3869,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 44,
- "startLine": 3867
+ "startLine": 3869
},
{
"endCharacter": 59,
- "endLine": 3891,
+ "endLine": 3893,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 48,
- "startLine": 3891
+ "startLine": 3893
},
{
"endCharacter": 59,
- "endLine": 3891,
+ "endLine": 3893,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 48,
- "startLine": 3891
+ "startLine": 3893
},
{
"endCharacter": 24,
- "endLine": 3936,
+ "endLine": 3922,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Method \"_get_observation\" overrides class \"BaseEnvironment\" in an incompatible manner\n Return type mismatch: base method returns type \"DataFrame\", override returns type \"NDArray[float32]\"\n \"ndarray[_AnyShape, dtype[float32]]\" is not assignable to \"DataFrame\"",
"rule": "reportIncompatibleMethodOverride",
"severity": "error",
"startCharacter": 8,
- "startLine": 3936
- },
- {
- "endCharacter": 53,
- "endLine": 4004,
- "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "\"name\" is not a known attribute of \"None\"",
- "rule": "reportOptionalMemberAccess",
- "severity": "error",
- "startCharacter": 49,
- "startLine": 4004
+ "startLine": 3922
},
{
"endCharacter": 12,
- "endLine": 4038,
+ "endLine": 4024,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Method \"step\" overrides class \"Base5ActionRLEnv\" in an incompatible manner\n Return type mismatch: base method returns type \"tuple[DataFrame, float, bool, Literal[False], dict[str, Unknown]]\", override returns type \"tuple[NDArray[float32], float, bool, bool, dict[str, Any]]\"\n \"tuple[NDArray[float32], float, bool, bool, dict[str, Any]]\" is not assignable to \"tuple[DataFrame, float, bool, Literal[False], dict[str, Unknown]]\"\n Tuple entry 1 is incorrect type\n \"ndarray[_AnyShape, dtype[float32]]\" is not assignable to \"DataFrame\"",
"rule": "reportIncompatibleMethodOverride",
"severity": "error",
"startCharacter": 8,
- "startLine": 4038
+ "startLine": 4024
},
{
"endCharacter": 20,
- "endLine": 4151,
+ "endLine": 4207,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Method \"action_masks\" overrides class \"BaseEnvironment\" in an incompatible manner\n Return type mismatch: base method returns type \"list[bool]\", override returns type \"NDArray[bool_]\"\n \"ndarray[_AnyShape, dtype[bool_]]\" is not assignable to \"list[bool]\"",
"rule": "reportIncompatibleMethodOverride",
"severity": "error",
"startCharacter": 8,
- "startLine": 4151
- },
- {
- "endCharacter": 65,
- "endLine": 4344,
- "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Type \"Any | None\" is not assignable to return type \"float\"\n Type \"Any | None\" is not assignable to type \"float\"\n \"None\" is not assignable to \"float\"",
- "rule": "reportReturnType",
- "severity": "error",
- "startCharacter": 15,
- "startLine": 4344
+ "startLine": 4207
},
{
"endCharacter": 40,
- "endLine": 4383,
+ "endLine": 4355,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "\"Figure\" is not exported from module \"matplotlib.pyplot\"",
"rule": "reportPrivateImportUsage",
"severity": "error",
"startCharacter": 34,
- "startLine": 4383
+ "startLine": 4355
},
{
"endCharacter": 54,
- "endLine": 4881,
+ "endLine": 4858,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 45,
- "startLine": 4881
+ "startLine": 4858
},
{
"endCharacter": 54,
- "endLine": 4881,
+ "endLine": 4858,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 45,
- "startLine": 4881
+ "startLine": 4858
},
{
"endCharacter": 50,
- "endLine": 4893,
+ "endLine": 4870,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"object\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"object\" is not assignable to type \"ConvertibleToFloat\"\n \"object\" is not assignable to \"str\"\n \"object\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"object\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"object\" is incompatible with protocol \"SupportsIndex\"\n \"__index__\" is not present",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 37,
- "startLine": 4893
+ "startLine": 4870
},
{
"endCharacter": 72,
- "endLine": 4904,
+ "endLine": 4881,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 70,
- "startLine": 4904
+ "startLine": 4881
},
{
"endCharacter": 72,
- "endLine": 4904,
+ "endLine": 4881,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 70,
- "startLine": 4904
+ "startLine": 4881
},
{
"endCharacter": 73,
- "endLine": 4913,
+ "endLine": 4890,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 71,
- "startLine": 4913
+ "startLine": 4890
},
{
"endCharacter": 73,
- "endLine": 4913,
+ "endLine": 4890,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"float | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"float | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 71,
- "startLine": 4913
+ "startLine": 4890
},
{
"endCharacter": 58,
- "endLine": 4922,
+ "endLine": 4899,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 56,
- "startLine": 4922
+ "startLine": 4899
},
{
"endCharacter": 58,
- "endLine": 4922,
+ "endLine": 4899,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 56,
- "startLine": 4922
+ "startLine": 4899
},
{
"endCharacter": 50,
- "endLine": 5146,
+ "endLine": 5130,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Mapping[Unknown, Unknown]\" cannot be assigned to parameter \"src\" of type \"dict[str, Any]\" in function \"deepmerge\"\n \"Mapping[Unknown, Unknown]\" is not assignable to \"dict[str, Any]\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 49,
- "startLine": 5146
+ "startLine": 5130
},
{
"endCharacter": 20,
- "endLine": 5323,
+ "endLine": 5307,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 18,
- "startLine": 5323
+ "startLine": 5307
},
{
"endCharacter": 20,
- "endLine": 5323,
+ "endLine": 5307,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 18,
- "startLine": 5323
+ "startLine": 5307
},
{
"endCharacter": 59,
- "endLine": 5328,
+ "endLine": 5312,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 31,
- "startLine": 5328
+ "startLine": 5312
},
{
"endCharacter": 59,
- "endLine": 5328,
+ "endLine": 5312,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 31,
- "startLine": 5328
+ "startLine": 5312
},
{
"endCharacter": 65,
- "endLine": 5329,
+ "endLine": 5313,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 34,
- "startLine": 5329
+ "startLine": 5313
},
{
"endCharacter": 65,
- "endLine": 5329,
+ "endLine": 5313,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 34,
- "startLine": 5329
+ "startLine": 5313
},
{
"endCharacter": 57,
- "endLine": 5330,
+ "endLine": 5314,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 31,
- "startLine": 5330
+ "startLine": 5314
},
{
"endCharacter": 57,
- "endLine": 5330,
+ "endLine": 5314,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 31,
- "startLine": 5330
+ "startLine": 5314
},
{
"endCharacter": 63,
- "endLine": 5332,
+ "endLine": 5316,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 34,
- "startLine": 5332
+ "startLine": 5316
},
{
"endCharacter": 63,
- "endLine": 5332,
+ "endLine": 5316,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 34,
- "startLine": 5332
+ "startLine": 5316
},
{
"endCharacter": 61,
- "endLine": 5334,
+ "endLine": 5318,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 32,
- "startLine": 5334
+ "startLine": 5318
},
{
"endCharacter": 61,
- "endLine": 5334,
+ "endLine": 5318,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 32,
- "startLine": 5334
+ "startLine": 5318
},
{
"endCharacter": 67,
- "endLine": 5335,
+ "endLine": 5319,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 36,
- "startLine": 5335
+ "startLine": 5319
},
{
"endCharacter": 67,
- "endLine": 5335,
+ "endLine": 5319,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 36,
- "startLine": 5335
+ "startLine": 5319
},
{
"endCharacter": 73,
- "endLine": 5336,
+ "endLine": 5320,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 39,
- "startLine": 5336
+ "startLine": 5320
},
{
"endCharacter": 73,
- "endLine": 5336,
+ "endLine": 5320,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 39,
- "startLine": 5336
+ "startLine": 5320
},
{
"endCharacter": 61,
- "endLine": 5337,
+ "endLine": 5321,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 33,
- "startLine": 5337
+ "startLine": 5321
},
{
"endCharacter": 61,
- "endLine": 5337,
+ "endLine": 5321,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 33,
- "startLine": 5337
- },
- {
- "endCharacter": 76,
- "endLine": 5341,
- "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 46,
- "startLine": 5341
- },
- {
- "endCharacter": 76,
- "endLine": 5341,
- "file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
- "message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
- "rule": "reportArgumentType",
- "severity": "error",
- "startCharacter": 46,
- "startLine": 5341
+ "startLine": 5321
},
{
"endCharacter": 89,
- "endLine": 5343,
+ "endLine": 5328,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 52,
- "startLine": 5343
+ "startLine": 5328
},
{
"endCharacter": 89,
- "endLine": 5343,
+ "endLine": 5328,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 52,
- "startLine": 5343
+ "startLine": 5328
},
{
"endCharacter": 83,
- "endLine": 5344,
+ "endLine": 5329,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 49,
- "startLine": 5344
+ "startLine": 5329
},
{
"endCharacter": 83,
- "endLine": 5344,
+ "endLine": 5329,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 49,
- "startLine": 5344
+ "startLine": 5329
},
{
"endCharacter": 57,
- "endLine": 5369,
+ "endLine": 5354,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 31,
- "startLine": 5369
+ "startLine": 5354
},
{
"endCharacter": 57,
- "endLine": 5369,
+ "endLine": 5354,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 31,
- "startLine": 5369
+ "startLine": 5354
},
{
"endCharacter": 65,
- "endLine": 5370,
+ "endLine": 5355,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 34,
- "startLine": 5370
+ "startLine": 5355
},
{
"endCharacter": 65,
- "endLine": 5370,
+ "endLine": 5355,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 34,
- "startLine": 5370
+ "startLine": 5355
},
{
"endCharacter": 67,
- "endLine": 5372,
+ "endLine": 5357,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 35,
- "startLine": 5372
+ "startLine": 5357
},
{
"endCharacter": 67,
- "endLine": 5372,
+ "endLine": 5357,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 35,
- "startLine": 5372
+ "startLine": 5357
},
{
"endCharacter": 87,
- "endLine": 5375,
+ "endLine": 5360,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 46,
- "startLine": 5375
+ "startLine": 5360
},
{
"endCharacter": 87,
- "endLine": 5375,
+ "endLine": 5360,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 46,
- "startLine": 5375
+ "startLine": 5360
},
{
"endCharacter": 93,
- "endLine": 5376,
+ "endLine": 5361,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 49,
- "startLine": 5376
+ "startLine": 5361
},
{
"endCharacter": 93,
- "endLine": 5376,
+ "endLine": 5361,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 49,
- "startLine": 5376
+ "startLine": 5361
},
{
"endCharacter": 89,
- "endLine": 5377,
+ "endLine": 5362,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 47,
- "startLine": 5377
+ "startLine": 5362
},
{
"endCharacter": 89,
- "endLine": 5377,
+ "endLine": 5362,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToFloat\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToFloat\"\n Type \"None\" is not assignable to type \"ConvertibleToFloat\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsFloat\"\n \"__float__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 47,
- "startLine": 5377
+ "startLine": 5362
},
{
"endCharacter": 89,
- "endLine": 5378,
+ "endLine": 5363,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 46,
- "startLine": 5378
+ "startLine": 5363
},
{
"endCharacter": 89,
- "endLine": 5378,
+ "endLine": 5363,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 46,
- "startLine": 5378
+ "startLine": 5363
},
{
"endCharacter": 75,
- "endLine": 5379,
+ "endLine": 5364,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 39,
- "startLine": 5379
+ "startLine": 5364
},
{
"endCharacter": 75,
- "endLine": 5379,
+ "endLine": 5364,
"file": "ReforceXY/user_data/freqaimodels/ReforceXY.py",
"message": "Argument of type \"Any | None\" cannot be assigned to parameter \"x\" of type \"ConvertibleToInt\" in function \"__new__\"\n Type \"Any | None\" is not assignable to type \"ConvertibleToInt\"\n Type \"None\" is not assignable to type \"ConvertibleToInt\"\n \"None\" is not assignable to \"str\"\n \"None\" is incompatible with protocol \"Buffer\"\n \"__buffer__\" is not present\n \"None\" is incompatible with protocol \"SupportsInt\"\n \"__int__\" is not present\n \"None\" is incompatible with protocol \"SupportsIndex\"\n ...",
"rule": "reportArgumentType",
"severity": "error",
"startCharacter": 39,
- "startLine": 5379
+ "startLine": 5364
},
{
"endCharacter": 33,
- PBRS modes: canonical, non_canonical, progressive_release, spike_cancel,
retain_previous
- Feature importance & optional partial dependence
-- Statistical tests (hypothesis, bootstrap CIs, distribution diagnostics)
-- Real vs synthetic shift metrics
+- Descriptive distribution diagnostics and real-vs-synthetic shift metrics;
+ inferential tests and bootstrap intervals are API-only for explicitly independent observations
- Manifest + parameter hash
## Quick Start
- **`--real_episodes`** (path, optional) – Episodes pickle for real vs synthetic
distribution shift metrics. (Simulation-only; triggers additional outputs when
provided).
-- **`--unrealized_pnl`** (flag, default: false) – Simulate unrealized PnL
- accrual during holds for potential Φ. (Simulation-only; affects PBRS
- components).
+- **`--unrealized_pnl`** (flag, default: false) – Transform the retained
+ in-position synthetic price/PnL trajectory using fee-aware unrealized PnL.
+ This affects extrema and all enabled base, PBRS, and additive reward terms
+ that depend on PnL. (Simulation-only.)
### Hybrid Simulation Scalars
hide reward drift or invariance violations.
- **`--strict_validation`** (flag, default: true) – Enforce parameter bounds and
finite checks; raises instead of silent clamp/discard when enabled.
-- **`--strict_diagnostics`** (flag, default: false) – Fail-fast on degenerate
- statistical diagnostics (zero-width CIs, undefined distribution metrics)
- instead of graceful fallbacks.
+- **`--strict_diagnostics`** (flag, default: false) – Raise on extreme distribution
+ moments instead of warning. In both modes, constants retain exact mean/std,
+ while undefined higher moments, normality tests and Q-Q fits remain N/A.
- **`--exit_factor_threshold`** (float, default: 1000.0) – Emits a warning if
the absolute value of the exit factor exceeds the threshold.
-- **`--pvalue_adjust`** (none|benjamini_hochberg, default: none) – Multiple
- testing p-value adjustment method.
-- **`--bootstrap_resamples`** (int, default: 10000) – Bootstrap iterations for
- confidence intervals; lower for speed (e.g. 500) during smoke tests.
- **`--skip_feature_analysis`** / **`--skip_partial_dependence`** – Skip feature
importance or PD grids (see Skipping Feature Analysis section); influence
runtime only.
- **`--rf_n_jobs`** / **`--perm_n_jobs`** (int, default: -1) – Parallel worker
counts for RandomForest and permutation importance (-1 = all cores).
+Inferential helpers are available through the programmatic API only and require
+`independent_observations=True`. Programmatic callers may pass `stats_seed`
+to isolate bootstrap resampling from the simulation seed; the descriptive CLI
+does not expose this option. In that mode, bootstrap percentile intervals retain
+finite ordered bounds, including exact zero-width intervals for constants; the
+interval need not contain the original sample mean.
+
### Overrides
- **`--out_dir`** (path, default: reward_space_outputs) – Output directory
**Formula:**
-Let `pnl_target = profit_aim · risk_reward_ratio`,
-`pnl_ratio = pnl / pnl_target`.
+Let `pnl_target = profit_aim · risk_reward_ratio` and
+`pnl_ratio = pnl / pnl_target`. On the loss branch,
+`loss_threshold = pnl_target / risk_reward_ratio` and
+`loss_ratio = |pnl| / loss_threshold = |pnl_ratio| · risk_reward_ratio`.
- If `pnl_target ≤ 0`: `pnl_target_coefficient = 1.0`
- If `pnl_ratio > 1.0`:
`pnl_target_coefficient = 1.0 + win_reward_factor · tanh(pnl_amplification_sensitivity · (pnl_ratio - 1.0))`
-- If `pnl_ratio < -(1.0 / risk_reward_ratio)`:
- `pnl_target_coefficient = 1.0 + (win_reward_factor · risk_reward_ratio) · tanh(pnl_amplification_sensitivity · (|pnl_ratio| - 1.0))`
+- If `pnl < -loss_threshold`:
+ `pnl_target_coefficient = 1.0 + (win_reward_factor · risk_reward_ratio) · tanh(pnl_amplification_sensitivity · (loss_ratio - 1.0))`
- Else: `pnl_target_coefficient = 1.0`
##### Efficiency
| Parameter | Default | Description |
| ---------------------------- | ------- | -------------------------- |
| `max_trade_duration_candles` | 128 | Trade duration cap |
-| `max_idle_duration_candles` | None | Fallback 4× trade duration |
+| `max_idle_duration_candles` | None | Idle hazard threshold (4× trade duration fallback); the idle clock keeps counting past it |
| `idle_penalty_ratio` | 1.0 | Idle penalty ratio |
| `idle_penalty_power` | 1.025 | Idle penalty exponent |
| `hold_penalty_ratio` | 1.0 | Hold penalty ratio |
| `entry_fee_rate` | 0.0 | Entry fee rate (`price · (1 + fee)`) |
| `exit_fee_rate` | 0.0 | Exit fee rate (`price / (1 + fee)`) |
-PBRS invariance holds when: `exit_potential_mode=canonical`.
+PBRS verification is evidence-based, never a raw shaping sum: complete ordered
+episodes (contiguous `transition_index`, single terminal), the local identity
+`reward_shaping = gamma * next_potential - prev_potential`, temporal potential
+continuity, and the discounted terminal boundary residual must all hold with
+sufficient ordered data; otherwise the report classifies the observed PBRS as
+"Not verified" even in canonical configuration.
In canonical mode, the entry/exit additive terms are suppressed even if the
corresponding `*_additive_enabled` flags are set.
-Note: PBRS telescoping/zero-sum shaping is a property of coherent trajectories
-(episodes). `simulate_samples()` generates synthetic trajectories (state carried
+Note: `simulate_samples()` generates synthetic trajectories (state carried
across samples) and does not apply any drift correction in post-processing.
+Trade duration is zero on entry and advances before each subsequent in-position
+reward, including an immediate exit on the next candle. The report summary uses
+the same verified/not-verified classification as the detailed PBRS section;
+a zero numerical correction alone does not establish canonical invariance.
#### Hold Potential Transforms
| Mode | Formula | Monotonic | Notes | Use Case |
| --------- | ----------------------------- | --------- | ------------------------------------------- | ------------------------------------ |
-| legacy | step: 1.5 if r\* ≤ 1 else 0.5 | No | Non-monotonic legacy mode (not recommended) | Backward compatibility only |
| sqrt | 1 / √(1 + r\*) | Yes | Sub-linear decay | Gentle long-trade penalty |
| linear | 1 / (1 + slope · r\*) | Yes | slope = `exit_linear_slope` | Balanced duration penalty (default) |
| power | (1 + r\*)^(-alpha) | Yes | alpha = -ln(tau)/ln(2); tau=1 ⇒ alpha=0 | Tunable decay rate via tau parameter |
| `softsign` | x / (1 + \|x\|) | (-1, 1) | Linear near 0 | Less aggressive saturation |
| `arctan` | (2/π) · arctan(x) | (-1, 1) | Slower saturation | Wide dynamic range |
| `sigmoid` | 2σ(x) - 1, σ(x) = 1/(1 + e^(-x)) | (-1, 1) | Standard sigmoid | Generic shaping |
-| `asinh` | x / √(1 + x²) | (-1, 1) | Outlier robust | Extreme stability |
+| `softsign_sqrt` | x / √(1 + x²) | (-1, 1) | Outlier robust | Extreme stability |
| `clip` | clip(x, -1, 1) | [-1, 1] | Hard clipping | Preserve linearity |
### Skipping Feature Analysis
### Reproducibility
-| Component | Controlled By | Notes |
-| ------------------------------------- | ---------------------------------- | ----------------------------------- |
-| Sample simulation | `--seed` | Drives action sampling & PnL noise |
-| Statistical tests / bootstrap | `--stats_seed` (fallback `--seed`) | Isolated RNG |
-| RandomForest & permutation importance | `--seed` | Identical splits and trees |
-| Partial dependence grids | Deterministic | Depends only on fitted model & data |
+| Component | Controlled By | Notes |
+| ------------------------------------- | ------------- | ------------------------------------------ |
+| Synthetic sampling | `--seed` | Drives action sampling and PnL noise |
+| RandomForest & permutation importance | `--seed` | Reproducible splits, trees, and importance |
+| Partial dependence grids | Deterministic | Depends only on fitted model and data |
-Patterns:
+The CLI uses `--seed` for every randomized analysis it exposes.
```shell
-uv run python reward_space_analysis.py --num_samples 50000 --seed 123 --stats_seed 9001 --out_dir run_stats1
-uv run python reward_space_analysis.py --num_samples 50000 --seed 123 --stats_seed 9002 --out_dir run_stats2
-# Fully deterministic
uv run python reward_space_analysis.py --num_samples 50000 --seed 777
```
`--params` wins on conflicts.
**Simulation** (not allowed in `--params`): `num_samples`, `seed`,
-`trading_mode`, `max_duration_ratio`, `out_dir`, `stats_seed`, `pnl_base_std`,
+`trading_mode`, `max_duration_ratio`, `out_dir`, `pnl_base_std`,
`pnl_duration_vol_scale`, `real_episodes`, `unrealized_pnl`,
-`strict_diagnostics`, `strict_validation`, `bootstrap_resamples`,
-`skip_feature_analysis`, `skip_partial_dependence`, `rf_n_jobs`, `perm_n_jobs`,
-`pvalue_adjust`.
+`strict_diagnostics`, `strict_validation`, `skip_feature_analysis`,
+`skip_partial_dependence`, `rf_n_jobs`, `perm_n_jobs`.
**Hybrid simulation/params** allowed in `--params`: `profit_aim`,
`risk_reward_ratio`, `action_masking`.
--num_samples 50000 \
--profit_aim 0.05 \
--trading_mode futures \
- --bootstrap_resamples 5000 \
--out_dir custom_analysis
# PBRS potential shaping analysis
uv run python reward_space_analysis.py \
### Main Report (`statistical_analysis.md`)
-Includes: global stats, representativity, component + PBRS analysis, feature
-importance/PD, statistical validation (tests, CIs, diagnostics), optional shift
-metrics, summary.
+Includes run configuration, global and component statistics, PBRS trajectory
+diagnostics, feature importance/partial dependence, descriptive distribution
+diagnostics, optional distribution-shift metrics, and a summary. Hypothesis
+tests, bootstrap confidence intervals, and inferential p-values appear only for
+programmatic calls with `independent_observations=True`; the CLI report is
+descriptive.
### Data Exports
| `num_samples` | int | Synthetic samples count |
| `seed` | int | Master random seed |
| `pnl_target` | float | Profit target |
-| `pvalue_adjust_method` | string | Multiple testing correction mode |
| `parameter_adjustments` | object | Bound clamp adjustments (if any) |
| `reward_params` | object | Final reward params |
| `simulation_params` | object | All simulation inputs |
| `*_js_distance` | √(0.5 KL(p_s‖m) + 0.5 KL(p_r‖m)) | Symmetric, [0,1] |
| `*_wasserstein` | 1D Earth Mover's Distance | Units of feature |
| `*_ks_statistic` | KS two-sample statistic | [0,1]; higher ⇒ divergence |
-| `*_ks_pvalue` | KS test p-value | High ⇒ cannot reject equality |
+| `*_ks_pvalue` | KS test p-value | API-only with `independent_observations=True`; omitted by the descriptive CLI |
-Implementation: 50-bin hist; add ε=1e-10; constants ⇒ zero divergence & KS
-p=1.0.
+Implementation: 50-bin histograms with ε=1e-10; constants have zero divergence.
+Inferential KS p-values are available only under the programmatic independence contract.
---
import warnings
from enum import Enum, IntEnum
from pathlib import Path
-from typing import TYPE_CHECKING, Any, Literal
+from typing import TYPE_CHECKING, Any, Final, Literal
import numpy as np
import pandas as pd
train_test_split = None
AttenuationMode = Literal["sqrt", "linear", "power", "half_life"]
-TransformFunction = Literal["tanh", "softsign", "arctan", "sigmoid", "clip", "asinh"]
+TransformFunction = Literal["tanh", "softsign", "arctan", "sigmoid", "clip", "softsign_sqrt"]
ExitPotentialMode = Literal[
"canonical", "non_canonical", "progressive_release", "spike_cancel", "retain_previous"
]
DEFAULT_IDLE_DURATION_MULTIPLIER = 4
-# Tolerance for PBRS invariance classification.
-#
-# When `reward_invariance_correction` is available (reward_shaping - reward_pbrs_delta),
-# canonical PBRS should satisfy max|correction| < PBRS_INVARIANCE_TOL.
-#
-# When that diagnostic column is not available (e.g., reporting from partial datasets),
-# we fall back to the weaker heuristic |Σ shaping| < PBRS_INVARIANCE_TOL.
+# Tolerance for observed PBRS identities, boundaries, and zero additive components.
PBRS_INVARIANCE_TOL: float = 1e-6
# Default discount factor γ for potential-based reward shaping
POTENTIAL_GAMMA_DEFAULT: float = 0.95
# Supported attenuation modes
ATTENUATION_MODES: tuple[str, ...] = ("sqrt", "linear", "power", "half_life")
-ATTENUATION_MODES_WITH_LEGACY: tuple[str, ...] = ("legacy", *ATTENUATION_MODES)
# Internal numeric guards and behavior toggles
INTERNAL_GUARDS: dict[str, float] = {
- "degenerate_ci_epsilon": 1e-9,
- "distribution_constant_fallback_moment": 0.0,
- "distribution_constant_fallback_qq_r2": 1.0,
"moment_extreme_threshold": 1e4,
"bootstrap_min_recommended": 200,
"sim_pnl_conservation_tol": 1e-10,
"sim_zero_reward_epsilon": 1e-12,
"sim_extreme_pnl_threshold": 0.2,
"histogram_epsilon": 1e-10,
- "distribution_identity_epsilon": 1e-12,
"efficiency_min_range_epsilon": 1e-6,
"efficiency_min_range_fraction": 0.01,
}
"softsign",
"arctan",
"sigmoid",
- "asinh",
+ "softsign_sqrt",
"clip",
}
ALLOWED_EXIT_POTENTIAL_MODES = {
# Supported p-value adjustment methods
ADJUST_METHODS: tuple[str, ...] = ("none", "benjamini_hochberg")
-# Alias without underscore for convenience
-_ADJUST_METHODS_ALIASES: frozenset[str] = frozenset({"benjaminihochberg"})
DEFAULT_MODEL_REWARD_PARAMETERS: RewardParams = {
"idle_penalty_power": "Idle penalty exponent",
"idle_penalty_ratio": "Idle penalty ratio",
"max_trade_duration_candles": "Trade duration cap (candles)",
- "max_idle_duration_candles": "Idle duration cap (candles)",
+ "max_idle_duration_candles": "Idle duration threshold (candles)",
"hold_penalty_ratio": "Hold penalty ratio",
"hold_penalty_power": "Hold penalty exponent",
- "exit_attenuation_mode": "Exit kernel (legacy|sqrt|linear|power|half_life)",
+ "exit_attenuation_mode": "Exit kernel (sqrt|linear|power|half_life)",
"exit_plateau": "Use plateau before attenuation",
"exit_plateau_grace": "Plateau grace duration ratio",
"exit_linear_slope": "Linear kernel slope",
"validation_mode": "strict" if strict else "relaxed",
}
+ # The coefficient must stay non-negative at both boundaries of the normalized
+ # efficiency interval [0, 1]. Reject combinations that require a runtime clamp.
+ if "efficiency_weight" in sanitized or "efficiency_center" in sanitized:
+ default_weight = _get_float_param(DEFAULT_MODEL_REWARD_PARAMETERS, "efficiency_weight")
+ default_center = _get_float_param(DEFAULT_MODEL_REWARD_PARAMETERS, "efficiency_center")
+ weight_value = sanitized.get("efficiency_weight", default_weight)
+ center_value = sanitized.get("efficiency_center", default_center)
+ weight = float(weight_value) if isinstance(weight_value, (int, float)) else default_weight
+ center = float(center_value) if isinstance(center_value, (int, float)) else default_center
+ if weight * max(center, 1.0 - center) > 1.0:
+ message = (
+ f"Param: efficiency_weight={weight} violates efficiency_weight * "
+ "max(efficiency_center, 1 - efficiency_center) <= 1 for "
+ f"efficiency_center={center}"
+ )
+ if strict:
+ raise ValueError(message)
+ warnings.warn(
+ message + "; falling back to weight=0.0",
+ RewardDiagnosticsWarning,
+ stacklevel=2,
+ )
+ sanitized["efficiency_weight"] = 0.0
+ adjustments["efficiency_weight"] = {
+ "original": weight,
+ "adjusted": 0.0,
+ "reason": "negative_efficiency_guard",
+ "validation_mode": "relaxed",
+ }
+
return sanitized, adjustments
parser.add_argument(
f"--{key}",
type=str,
- choices=sorted(ATTENUATION_MODES_WITH_LEGACY),
+ choices=sorted(ATTENUATION_MODES),
default=None,
help=help_text,
)
idle_penalty: float = 0.0
hold_penalty: float = 0.0
exit_component: float = 0.0
+ terminal_liquidation: bool = False
+ exit_pnl: float | None = None
# PBRS components
reward_shaping: float = 0.0
entry_additive: float = 0.0
)
exit_linear_slope = 1.0
- def _legacy_kernel(dr: float) -> float:
- return 1.5 if dr <= 1.0 else 0.5
-
def _sqrt_kernel(dr: float) -> float:
return 1.0 / math.sqrt(1.0 + dr)
return math.pow(2.0, -dr / hl)
kernels = {
- "legacy": _legacy_kernel,
"sqrt": _sqrt_kernel,
"linear": _linear_kernel,
"power": _power_kernel,
_warn_unknown_mode(
"exit_attenuation_mode",
exit_attenuation_mode,
- ATTENUATION_MODES_WITH_LEGACY,
+ ATTENUATION_MODES,
"linear",
stacklevel=2,
)
rr = risk_reward_ratio if risk_reward_ratio > 0 else RISK_REWARD_RATIO_DEFAULT
pnl_ratio = pnl / pnl_target
- if abs(pnl_ratio) > 1.0:
- base_pnl_target_coefficient = math.tanh(
- pnl_amplification_sensitivity * (abs(pnl_ratio) - 1.0)
- )
- if pnl_ratio > 1.0:
- pnl_target_coefficient = 1.0 + win_reward_factor * base_pnl_target_coefficient
- elif pnl_ratio < -(1.0 / rr):
+
+ if pnl_ratio > 1.0:
+ gain_coefficient = math.tanh(pnl_amplification_sensitivity * (pnl_ratio - 1.0))
+ pnl_target_coefficient = 1.0 + win_reward_factor * gain_coefficient
+ else:
+ loss_threshold = pnl_target / rr
+ if pnl < -loss_threshold:
+ loss_ratio = (-pnl) / loss_threshold
+ loss_coefficient = math.tanh(pnl_amplification_sensitivity * (loss_ratio - 1.0))
loss_penalty_factor = win_reward_factor * rr
- pnl_target_coefficient = 1.0 + loss_penalty_factor * base_pnl_target_coefficient
+ pnl_target_coefficient = 1.0 + loss_penalty_factor * loss_coefficient
return pnl_target_coefficient
min_pnl = min(context.min_unrealized_profit, pnl)
range_pnl = max_pnl - min_pnl
# Guard against division explosion when max_pnl ≈ min_pnl
- eps = float(INTERNAL_GUARDS.get("efficiency_min_range_epsilon", 1e-6))
- frac = float(INTERNAL_GUARDS.get("efficiency_min_range_fraction", 0.01))
+ eps = float(INTERNAL_GUARDS["efficiency_min_range_epsilon"])
+ frac = float(INTERNAL_GUARDS["efficiency_min_range_fraction"])
min_meaningful_range = max(eps, frac * pnl_target)
if np.isfinite(range_pnl) and range_pnl >= min_meaningful_range:
efficiency_ratio = (pnl - min_pnl) / range_pnl
short_allowed: bool,
action_masking: bool,
prev_potential: float = np.nan,
+ next_context: RewardContext | None = None,
+ terminated: bool = False,
) -> RewardBreakdown:
"""Calculate complete reward with base reward and PBRS shaping.
risk_reward_ratio,
)
breakdown.exit_component = base_reward
+ breakdown.exit_pnl = context.current_pnl
else:
base_reward = 0.0
next_duration_ratio = _compute_duration_ratio(
context.trade_duration, max_trade_duration_candles
)
- # Optionally simulate unrealized PnL during holds to feed Φ(s)
- if _get_bool_param(params, "unrealized_pnl", False):
- center_unrealized = 0.5 * (
- context.max_unrealized_profit + context.min_unrealized_profit
- )
- beta = _get_float_param(params, "pnl_amplification_sensitivity")
- next_pnl = float(center_unrealized * math.tanh(beta * next_duration_ratio))
- else:
- next_pnl = current_pnl
+ next_pnl = current_pnl
elif is_exit:
next_pnl = 0.0
next_duration_ratio = 0.0
next_pnl = current_pnl
next_duration_ratio = current_duration_ratio
+ if next_context is not None:
+ next_pnl = next_context.current_pnl
+ next_duration_ratio = _compute_duration_ratio(
+ next_context.trade_duration, max_trade_duration_candles
+ )
+ terminal_context = next_context if next_context is not None else context
+ breakdown.terminal_liquidation = bool(
+ terminated and next_position in (Positions.Long, Positions.Short)
+ )
+ if breakdown.terminal_liquidation:
+ liquidation_reward = _compute_exit_reward(
+ base_factor,
+ pnl_target,
+ next_duration_ratio,
+ terminal_context,
+ params,
+ risk_reward_ratio,
+ )
+ base_reward += liquidation_reward
+ breakdown.base_reward = float(base_reward)
+ breakdown.exit_component += liquidation_reward
+ breakdown.exit_pnl = terminal_context.current_pnl
+
# Apply PBRS only if enabled and not neutral self-loop
exit_mode = _get_str_param(params, "exit_potential_mode")
prev_potential = float(prev_potential) if np.isfinite(prev_potential) else 0.0
if is_neutral:
- # Neutral self-loop keeps stored potential unchanged.
+ # Neutral self-loops retain potential except at the terminal boundary.
breakdown.prev_potential = prev_potential
- breakdown.next_potential = prev_potential
- breakdown.total = base_reward
+ breakdown.next_potential = 0.0 if terminated else prev_potential
+ breakdown.reward_shaping = -prev_potential if terminated else 0.0
+ breakdown.pbrs_delta = breakdown.reward_shaping
+ breakdown.total = base_reward + breakdown.reward_shaping
return breakdown
reward_shaping, next_potential, pbrs_delta, entry_additive, exit_additive = (
params=params,
risk_reward_ratio=risk_reward_ratio,
base_factor=base_factor,
+ entry_pnl=_compute_unrealized_pnl_estimate(
+ next_position, entry_open=1.0, current_open=1.0, params=params
+ )
+ if is_entry
+ else 0.0,
)
)
+ if breakdown.terminal_liquidation and exit_additive_enabled:
+ exit_additive += _compute_exit_additive(
+ terminal_context.current_pnl, pnl_target, next_duration_ratio, params, base_factor
+ )
+ if terminated:
+ next_potential = 0.0
+ reward_shaping = -prev_potential
+ pbrs_delta = reward_shaping
breakdown.reward_shaping = reward_shaping
breakdown.prev_potential = prev_potential
prev_potential = float(prev_potential) if np.isfinite(prev_potential) else 0.0
breakdown.prev_potential = prev_potential
- breakdown.next_potential = prev_potential
+ breakdown.next_potential = 0.0 if terminated else prev_potential
breakdown.total = base_reward
return breakdown
return action, entry_prob, exit_prob, neutral_prob
+_HYBRID_REWARD_KEYS: Final[frozenset[str]] = frozenset(
+ {"profit_aim", "risk_reward_ratio", "action_masking"}
+)
+_SIMULATION_ONLY_KEYS: Final[frozenset[str]] = frozenset(
+ {
+ "unrealized_pnl",
+ "num_samples",
+ "seed",
+ "trading_mode",
+ "max_duration_ratio",
+ "pnl_base_std",
+ "pnl_duration_vol_scale",
+ "real_episodes",
+ "out_dir",
+ }
+)
+
+
def parse_overrides(overrides: Iterable[str]) -> RewardParams:
+ """Parse KEY=VALUE overrides restricted to supported reward parameters.
+
+ Only reward tunables (the canonical defaults) plus the hybrid simulation
+ scalars are accepted. Unknown keys, empty keys, and simulation-only options
+ are rejected before any artifact is produced.
+ """
parsed: RewardParams = {}
for override in overrides:
if "=" not in override:
raise ValueError(f"CLI: invalid override format '{override}'. Expected 'key=value'")
key, value = override.split("=", 1)
+ if not key:
+ raise ValueError(f"CLI: invalid override '{override}': empty parameter name")
+ if key in _SIMULATION_ONLY_KEYS:
+ raise ValueError(
+ f"CLI: override '{key}' is simulation-only and cannot be set via --params"
+ )
+ elif key not in DEFAULT_MODEL_REWARD_PARAMETERS and key not in _HYBRID_REWARD_KEYS:
+ raise ValueError(
+ f"CLI: unknown or unsupported reward parameter '{key}' in override '{override}'"
+ )
try:
parsed[key] = float(value)
except ValueError:
)
max_trade_duration_cap = int(max_trade_duration_candles * max_duration_ratio)
- samples: list[dict[str, float]] = []
+ samples: list[dict[str, float | None]] = []
prev_potential: float = 0.0
# Stateful trajectory variables
entry_open = current_open
for _ in range(num_samples):
+ action, sample_entry_prob, sample_exit_prob, sample_neutral_prob = _sample_action(
+ position,
+ rng,
+ short_allowed=short_allowed,
+ trade_duration=trade_duration,
+ max_trade_duration_candles=max_trade_duration_candles,
+ idle_duration=idle_duration,
+ max_idle_duration_candles=max_idle_duration_candles,
+ )
+
+ context = RewardContext(
+ current_pnl=pnl,
+ trade_duration=trade_duration,
+ idle_duration=idle_duration,
+ max_unrealized_profit=max_unrealized_profit,
+ min_unrealized_profit=min_unrealized_profit,
+ position=position,
+ action=action,
+ )
+
+ if position == Positions.Neutral:
+ if action == Actions.Long_enter:
+ position = Positions.Long
+ trade_duration = 0
+ idle_duration = 0
+ entry_open = current_open
+ max_unrealized_profit = -np.inf
+ min_unrealized_profit = np.inf
+ elif action == Actions.Short_enter and short_allowed:
+ position = Positions.Short
+ trade_duration = 0
+ idle_duration = 0
+ entry_open = current_open
+ max_unrealized_profit = -np.inf
+ min_unrealized_profit = np.inf
+ else:
+ idle_duration = 0
+ if action in (Actions.Long_exit, Actions.Short_exit):
+ position = Positions.Neutral
+ trade_duration = 0
+ idle_duration = 0
+ entry_open = current_open
+ if position in (Positions.Long, Positions.Short):
+ trade_duration = min(trade_duration + 1, max_trade_duration_cap)
+ else:
+ idle_duration += 1
# Simulate synthetic open-price movement.
duration_ratio = (
_compute_duration_ratio(trade_duration, max_trade_duration_candles)
step_return = float(np.clip(step_return, -0.95, 0.95))
current_open = float(max(1e-6, current_open * (1.0 + step_return)))
-
- # Compute fee-aware unrealized PnL from (entry_open, current_open)
+ # Always sample the random-walk price so both modes consume the same RNG stream.
+ # Unrealized-PnL mode replaces it below with the fee-aware price implied by
+ # the target PnL before reward calculation.
if position in (Positions.Long, Positions.Short):
- pnl = _compute_unrealized_pnl_estimate(
- position,
- entry_open=entry_open,
- current_open=current_open,
- params=params,
+ candidate_pnl = float(
+ np.clip(
+ _compute_unrealized_pnl_estimate(
+ position,
+ entry_open=entry_open,
+ current_open=current_open,
+ params=params,
+ ),
+ -0.15,
+ 0.15,
+ )
)
- pnl = float(np.clip(pnl, -0.15, 0.15))
+ if _get_bool_param(params, "unrealized_pnl", False):
+ # Let the sampled market move shape the next retained PnL without
+ # storing the discarded candidate in exit-efficiency extrema.
+ prospective_max = max(max_unrealized_profit, candidate_pnl)
+ prospective_min = min(min_unrealized_profit, candidate_pnl)
+ center_unrealized = 0.5 * (prospective_max + prospective_min)
+ beta = _get_float_param(params, "pnl_amplification_sensitivity")
+ hold_ratio = _compute_duration_ratio(trade_duration, max_trade_duration_candles)
+ target_pnl = float(
+ np.clip(center_unrealized * math.tanh(beta * hold_ratio), -0.15, 0.15)
+ )
+ entry_fee_rate, exit_fee_rate = _get_fee_rates(params)
+ if position == Positions.Long:
+ current_open = (
+ entry_open * (1 + entry_fee_rate) * (1 + exit_fee_rate) * (1 + target_pnl)
+ )
+ else:
+ current_open = (
+ entry_open * (1 - target_pnl) / ((1 + entry_fee_rate) * (1 + exit_fee_rate))
+ )
+ current_open = float(max(1e-6, current_open))
+ pnl = float(
+ np.clip(
+ _compute_unrealized_pnl_estimate(
+ position,
+ entry_open=entry_open,
+ current_open=current_open,
+ params=params,
+ ),
+ -0.15,
+ 0.15,
+ )
+ )
+ else:
+ pnl = candidate_pnl
max_unrealized_profit = max(max_unrealized_profit, pnl)
min_unrealized_profit = min(min_unrealized_profit, pnl)
else:
max_unrealized_profit = 0.0
min_unrealized_profit = 0.0
- action, sample_entry_prob, sample_exit_prob, sample_neutral_prob = _sample_action(
- position,
- rng,
- short_allowed=short_allowed,
- trade_duration=trade_duration,
- max_trade_duration_candles=max_trade_duration_candles,
- idle_duration=idle_duration,
- max_idle_duration_candles=max_idle_duration_candles,
- )
-
- context = RewardContext(
+ next_context = RewardContext(
current_pnl=pnl,
trade_duration=trade_duration,
idle_duration=idle_duration,
max_unrealized_profit=max_unrealized_profit,
min_unrealized_profit=min_unrealized_profit,
position=position,
- action=action,
+ action=Actions.Neutral,
)
-
breakdown = calculate_reward(
context,
params,
short_allowed=short_allowed,
action_masking=action_masking,
prev_potential=prev_potential,
+ next_context=next_context,
+ terminated=_ == num_samples - 1,
)
prev_potential = breakdown.next_potential
+ if breakdown.terminal_liquidation:
+ next_context = dataclasses.replace(
+ next_context,
+ position=Positions.Neutral,
+ current_pnl=0.0,
+ trade_duration=0,
+ idle_duration=0,
+ max_unrealized_profit=0.0,
+ min_unrealized_profit=0.0,
+ )
idle_ratio = context.idle_duration / max(1, max_idle_duration_candles)
samples.append(
{
+ "episode_id": 0,
+ "transition_index": _,
"pnl": context.current_pnl,
"trade_duration": context.trade_duration,
"idle_duration": context.idle_duration,
+ "next_pnl": next_context.current_pnl,
+ "next_trade_duration": next_context.trade_duration,
+ "next_position": float(next_context.position.value),
+ "terminated": _ == num_samples - 1,
+ "terminal_liquidation": breakdown.terminal_liquidation,
+ "exit_pnl": (np.nan if breakdown.exit_pnl is None else breakdown.exit_pnl),
"duration_ratio": _compute_duration_ratio(
context.trade_duration, max_trade_duration_candles
),
}
)
- # Transition state
- if position == Positions.Neutral:
- if action == Actions.Neutral:
- idle_duration = min(idle_duration + 1, max_idle_duration_candles)
- elif action == Actions.Long_enter:
- position = Positions.Long
- trade_duration = 0
- idle_duration = 0
- entry_open = current_open
- pnl = _compute_unrealized_pnl_estimate(
- Positions.Long,
- entry_open=entry_open,
- current_open=current_open,
- params=params,
- )
- max_unrealized_profit = pnl
- min_unrealized_profit = pnl
- elif action == Actions.Short_enter and short_allowed:
- position = Positions.Short
- trade_duration = 0
- idle_duration = 0
- entry_open = current_open
- pnl = _compute_unrealized_pnl_estimate(
- Positions.Short,
- entry_open=entry_open,
- current_open=current_open,
- params=params,
- )
- max_unrealized_profit = pnl
- min_unrealized_profit = pnl
- else:
- idle_duration = 0
- if action == Actions.Neutral:
- trade_duration = min(trade_duration + 1, max_trade_duration_cap)
- elif action in (Actions.Long_exit, Actions.Short_exit):
- position = Positions.Neutral
- trade_duration = 0
- idle_duration = 0
- entry_open = current_open
-
df = pd.DataFrame(samples)
df.attrs["reward_params"] = dict(params)
def _validate_simulation_invariants(df: pd.DataFrame) -> None:
"""Fail fast if simulation violates action/state invariants."""
- eps_pnl = float(INTERNAL_GUARDS.get("sim_zero_pnl_epsilon", 1e-12))
- eps_reward = float(INTERNAL_GUARDS.get("sim_zero_reward_epsilon", 1e-12))
- thr_extreme = float(INTERNAL_GUARDS.get("sim_extreme_pnl_threshold", 0.2))
+ eps_pnl = float(INTERNAL_GUARDS["sim_zero_pnl_epsilon"])
+ eps_reward = float(INTERNAL_GUARDS["sim_zero_reward_epsilon"])
+ thr_extreme = float(INTERNAL_GUARDS["sim_extreme_pnl_threshold"])
# INVARIANT 1: Action-position compatibility
long_exits = df[(df["action"] == 2.0) & (df["position"] != 1.0)]
if len(neutral_with_pnl) > 0:
raise AssertionError(f"Sim: {len(neutral_with_pnl)} Neutral positions with non-zero pnl")
- # INVARIANT 4: Exit rewards only appear on exit actions
+ # Economic exits belong to voluntary exits or a proven terminal liquidation.
+ liquidation = df.get("terminal_liquidation", pd.Series(False, index=df.index)).eq(True)
+ valid_liquidation = (
+ df.get("terminated", pd.Series(False, index=df.index)).eq(True)
+ & df.get("next_position", pd.Series(np.nan, index=df.index)).eq(Positions.Neutral.value)
+ & (
+ (
+ df["position"].isin([Positions.Long.value, Positions.Short.value])
+ & ~df["action"].isin([Actions.Long_exit.value, Actions.Short_exit.value])
+ )
+ | (
+ df["position"].eq(Positions.Neutral.value)
+ & df["action"].isin([Actions.Long_enter.value, Actions.Short_enter.value])
+ )
+ )
+ & np.isfinite(df.get("exit_pnl", pd.Series(np.nan, index=df.index)))
+ )
+ if (liquidation & ~valid_liquidation).any():
+ raise AssertionError("Sim: terminal liquidation lacks a terminal open-position transition")
non_exit_with_exit_reward = df[
- (~df["action"].isin([2.0, 4.0])) & (df["reward_exit"].abs() > eps_reward)
+ (~df["action"].isin([2.0, 4.0])) & ~liquidation & (df["reward_exit"].abs() > eps_reward)
]
if len(non_exit_with_exit_reward) > 0:
raise AssertionError(
max_trade_duration_candles = _get_int_param(reward_params, "max_trade_duration_candles")
idle_bins = np.linspace(0, max_trade_duration_candles * 3.0, 13)
trade_bins = np.linspace(0, max_trade_duration_candles * 3.0, 13)
- pnl_min = float(df["pnl"].min())
- pnl_max = float(df["pnl"].max())
+ exit_pnl = df.get("exit_pnl", df["pnl"]).fillna(df["pnl"])
+ pnl_min = float(exit_pnl.min())
+ pnl_max = float(exit_pnl.max())
if np.isclose(pnl_min, pnl_max):
pnl_max = pnl_min + 1e-6
pnl_bins = np.linspace(pnl_min, pnl_max, 13)
idle_stats = _binned_stats(df, "idle_duration", "reward_idle", idle_bins)
hold_stats = _binned_stats(df, "trade_duration", "reward_hold", trade_bins)
- exit_stats = _binned_stats(df, "pnl", "reward_exit", pnl_bins)
+ exit_stats = _binned_stats(df.assign(exit_pnl=exit_pnl), "exit_pnl", "reward_exit", pnl_bins)
idle_stats = idle_stats.round(6)
hold_stats = hold_stats.round(6)
# Keep optional list stable and explicit
numeric_optional = {
"reward_exit",
+ "exit_pnl",
+ "terminal_liquidation",
+ "execution_tick",
"reward_idle",
"reward_hold",
"reward_invalid",
if opt_col not in df.columns:
df[opt_col] = np.nan
- # Drop exact duplicates (rare but can appear after flattening)
- before_dupes = len(df)
- df = df.drop_duplicates()
- if len(df) != before_dupes:
- warnings.warn(
- f"Data: dropped {before_dupes - len(df)} duplicate row(s) when loading '{path}'",
- RewardDiagnosticsWarning,
- stacklevel=2,
- )
-
return df
def compute_distribution_shift_metrics(
synthetic_df: pd.DataFrame,
real_df: pd.DataFrame,
+ *,
+ independent_observations: bool = False,
) -> dict[str, float]:
"""Compute distribution shift metrics between synthetic and real samples.
Returns KL divergence, JS distance, Wasserstein distance, and KS test
results for continuous features (pnl, trade_duration, idle_duration).
+ The inferential KS p-value is reported only with
+ ``independent_observations=True``; the descriptive KS statistic is always
+ reported. Constants yield exact zero distances and, inferentially, p=1.0.
"""
metrics = {}
continuous_features = ["pnl", "trade_duration", "idle_duration"]
# Guard against degenerate distributions (all values identical)
if not np.isfinite(min_val) or not np.isfinite(max_val):
continue
- if np.isclose(
- max_val,
- min_val,
- rtol=0,
- atol=float(INTERNAL_GUARDS.get("distribution_identity_epsilon", 1e-12)),
- ):
+ if max_val == min_val:
# All mass at a single point -> shift metrics are all zero by definition
metrics[f"{feature}_kl_divergence"] = 0.0
metrics[f"{feature}_js_distance"] = 0.0
metrics[f"{feature}_wasserstein"] = 0.0
metrics[f"{feature}_ks_statistic"] = 0.0
- metrics[f"{feature}_ks_pvalue"] = 1.0
+ if independent_observations:
+ metrics[f"{feature}_ks_pvalue"] = 1.0
continue
- bins = np.linspace(min_val, max_val, 50)
+ bins = np.unique(np.linspace(min_val, max_val, 50))
# Use density=False to get counts, then normalize to probabilities
hist_synth, _ = np.histogram(synth_values, bins=bins, density=False)
hist_real, _ = np.histogram(real_values, bins=bins, density=False)
# Add small epsilon to avoid log(0) in KL divergence
- epsilon = float(INTERNAL_GUARDS.get("histogram_epsilon", 1e-10))
+ epsilon = float(INTERNAL_GUARDS["histogram_epsilon"])
hist_synth = hist_synth + epsilon
hist_real = hist_real + epsilon
# Normalize to create probability distributions (sum to 1)
ks_stat, ks_pval = stats.ks_2samp(synth_values, real_values)
metrics[f"{feature}_ks_statistic"] = float(ks_stat)
- metrics[f"{feature}_ks_pvalue"] = float(ks_pval)
+ if independent_observations:
+ metrics[f"{feature}_ks_pvalue"] = float(ks_pval)
# Validate distribution shift metrics bounds
_validate_distribution_metrics(metrics)
def statistical_hypothesis_tests(
- df: pd.DataFrame, *, adjust_method: str = ADJUST_METHODS[0], seed: int = 42
+ df: pd.DataFrame,
+ *,
+ independent_observations: bool,
+ adjust_method: str = ADJUST_METHODS[0],
+ seed: int = 42,
) -> dict[str, Any]:
"""Statistical hypothesis tests (Spearman, Kruskal-Wallis, Mann-Whitney).
+ Requires ``independent_observations=True``; non-finite p-values become
+ explicitly non-applicable and are excluded from the finite-only
+ Benjamini-Hochberg family.
+
Parameters
----------
df : pd.DataFrame
seed : int
Random seed for bootstrap resampling.
"""
+ if independent_observations is not True:
+ raise ValueError("Stats: hypothesis tests require independent_observations=True")
results = {}
alpha = 0.05
position_groups = [g for g in position_groups if len(g) >= 10]
if len(position_groups) >= 2:
- h_stat, p_val = stats.kruskal(*position_groups)
+ if all(np.all(g == position_groups[0][0]) for g in position_groups):
+ h_stat, p_val = np.nan, np.nan
+ else:
+ h_stat, p_val = stats.kruskal(*position_groups)
n_total = sum(len(g) for g in position_groups)
epsilon_sq = h_stat / (n_total - 1) if n_total > 1 else 0.0
"effect_size_rank_biserial": float(rb) if np.isfinite(rb) else np.nan,
}
+ for result in results.values():
+ result["applicable"] = bool(np.isfinite(result["p_value"]))
+ if not result["applicable"]:
+ result["significant"] = None
+ result["interpretation"] = "N/A (undefined test statistic)"
+ result["p_value_adj"] = np.nan
+ result["significant_adj"] = None
# Optional multiple testing correction (Benjamini-Hochberg)
- _valid_adjust = set(ADJUST_METHODS) | _ADJUST_METHODS_ALIASES
+ _valid_adjust = set(ADJUST_METHODS)
if adjust_method not in _valid_adjust:
raise ValueError(
f"Stats: unsupported adjust_method '{adjust_method}'. "
)
if adjust_method in _valid_adjust - {ADJUST_METHODS[0]} and results:
# Collect p-values
- items = list(results.items())
+ items = [(name, res) for name, res in results.items() if res["applicable"]]
pvals = np.array([v[1]["p_value"] for v in items])
m = len(pvals)
order = np.argsort(pvals)
res["p_value_adj"] = float(p_adj)
res["significant_adj"] = bool(p_adj < alpha)
results[name] = res
-
# Validate hypothesis test results
_validate_hypothesis_test_results(results)
def _validate_hypothesis_test_results(results: dict[str, Any]) -> None:
"""Validate statistical properties of hypothesis test results."""
for test_name, result in results.items():
+ if result.get("applicable") is False:
+ continue
# All p-values must be in [0, 1] or NaN (for cases like constant input)
if "p_value" in result:
p_val = result["p_value"]
confidence_level: float = 0.95,
seed: int = 42,
*,
+ independent_observations: bool,
strict_diagnostics: bool = False,
) -> dict[str, tuple[float, float, float]]:
"""Compute bootstrap confidence intervals for metric means.
- Returns percentile-based CIs, skipping metrics with <10 samples.
+ Requires ``independent_observations=True``; skips metrics with <10 samples.
+ Bounds are raw finite percentile estimates ordered low<=high and are not
+ forced to contain the point estimate; constant metrics retain their exact
+ degenerate interval in both strict and relaxed modes.
"""
+ if independent_observations is not True:
+ raise ValueError("Stats: bootstrap intervals require independent_observations=True")
alpha = 1 - confidence_level
lower_percentile = 100 * alpha / 2
upper_percentile = 100 * (1 - alpha / 2)
results = {}
# Advisory: very low bootstrap counts produce unstable CI widths
- min_rec = int(INTERNAL_GUARDS.get("bootstrap_min_recommended", 200))
+ min_rec = int(INTERNAL_GUARDS["bootstrap_min_recommended"])
if n_bootstrap < min_rec:
warnings.warn(
f"Stats: n_bootstrap={n_bootstrap} < {min_rec}; confidence intervals may be unstable",
if data_array.size == 0:
continue
if np.ptp(data_array) == 0: # zero range -> constant
- if strict_diagnostics:
- # In strict mode, skip constant metrics entirely to avoid degenerate CI raise.
- continue
- # Graceful mode: record degenerate CI; validator will widen.
results[metric] = (point_est, point_est, point_est)
continue
def _validate_bootstrap_results(
results: dict[str, tuple[float, float, float]], *, strict_diagnostics: bool
) -> None:
- """Validate each bootstrap CI: finite bounds, ordered, positive width (adjust or raise)."""
+ """Validate finite estimates and ordered percentile bounds without altering them."""
for metric, (mean, ci_low, ci_high) in results.items():
# CI bounds must be finite
if not (np.isfinite(mean) and np.isfinite(ci_low) and np.isfinite(ci_high)):
f"(mean={mean}, ci_low={ci_low}, ci_high={ci_high})"
)
- # CI must be properly ordered
- if not (ci_low <= mean <= ci_high):
+ # Percentile intervals need not contain the original sample mean.
+ if ci_low > ci_high:
raise AssertionError(
f"Stats: bootstrap CI for {metric}: ordering violation "
- f"({ci_low:.6f} <= {mean:.6f} <= {ci_high:.6f})"
- )
-
- # CI width should be positive (non-degenerate)
- width = ci_high - ci_low
- if width <= 0:
- if strict_diagnostics:
- raise AssertionError(
- f"Stats: bootstrap CI for {metric}: non-positive width {width:.6f}"
- )
- # Graceful mode: expand interval symmetrically
- epsilon = INTERNAL_GUARDS["degenerate_ci_epsilon"] if width == 0 else abs(width) * 1e-06
- center = mean
- # Adjust only if current bounds are identical; otherwise enforce ordering minimally.
- if ci_low == ci_high:
- ci_low = center - epsilon
- ci_high = center + epsilon
- else:
- # Ensure proper ordering if inverted or collapsed negatively.
- lower = min(ci_low, ci_high) - epsilon
- upper = max(ci_low, ci_high) + epsilon
- ci_low, ci_high = lower, upper
- results[metric] = (mean, ci_low, ci_high)
- warnings.warn(
- f"Stats: bootstrap_ci for '{metric}' degenerate (width={width:.6g}); falling back to epsilon={epsilon:.6g}",
- RewardDiagnosticsWarning,
- stacklevel=2,
+ f"({ci_low:.6f} > {ci_high:.6f})"
)
def distribution_diagnostics(
df: pd.DataFrame,
*,
+ independent_observations: bool = False,
seed: int | None = None,
strict_diagnostics: bool = False,
) -> dict[str, Any]:
- """Return mapping col-> diagnostics (tests, moments, entropy, divergences).
+ """Return moments and normality diagnostics for available numeric columns.
- Skips missing columns; selects Shapiro-Wilk when n<=5000 else K2; ignores non-finite intermediates.
+ Normality tests are inferential and only run with
+ ``independent_observations=True``; the Q-Q fit and higher moments are
+ descriptive. Constants retain exact mean/std, mark higher moments as N/A
+ and omit tests and Q-Q fits in both strict and relaxed modes.
"""
diagnostics = {}
_ = seed # placeholder to keep signature for future reproducibility extensions
if col not in df.columns:
continue
- data = df[col].dropna().values
+ data = df[col].dropna().to_numpy(dtype=float)
if len(data) < 10:
continue
diagnostics[f"{col}_mean"] = float(np.mean(data))
diagnostics[f"{col}_std"] = float(np.std(data, ddof=1))
+ if np.ptp(data) == 0:
+ diagnostics[f"{col}_skewness"] = None
+ diagnostics[f"{col}_kurtosis"] = None
+ diagnostics[f"{col}_constant"] = True
+ continue
skew_v = float(stats.skew(data))
kurt_v = float(stats.kurtosis(data, fisher=True))
diagnostics[f"{col}_skewness"] = skew_v
diagnostics[f"{col}_kurtosis"] = kurt_v
- thr = INTERNAL_GUARDS.get("moment_extreme_threshold", 1e4)
+ thr = INTERNAL_GUARDS["moment_extreme_threshold"]
if abs(skew_v) > thr or abs(kurt_v) > thr:
msg = f"Stats: extreme moment(s) for {col}: skew={skew_v:.3e}, kurtosis={kurt_v:.3e} exceeds threshold {thr}"
if strict_diagnostics:
raise AssertionError(msg)
warnings.warn(msg, RewardDiagnosticsWarning, stacklevel=2)
- if len(data) < 5000:
- sw_stat, sw_pval = stats.shapiro(data)
- diagnostics[f"{col}_shapiro_stat"] = float(sw_stat)
- diagnostics[f"{col}_shapiro_pval"] = float(sw_pval)
- diagnostics[f"{col}_is_normal_shapiro"] = bool(sw_pval > 0.05)
-
- ad_result = stats.anderson(data, dist="norm")
- diagnostics[f"{col}_anderson_stat"] = float(ad_result.statistic)
- diagnostics[f"{col}_anderson_critical_5pct"] = float(ad_result.critical_values[2])
- diagnostics[f"{col}_is_normal_anderson"] = bool(
- ad_result.statistic < ad_result.critical_values[2]
- )
+ if independent_observations:
+ if len(data) < 5000:
+ sw_stat, sw_pval = stats.shapiro(data)
+ diagnostics[f"{col}_shapiro_stat"] = float(sw_stat)
+ diagnostics[f"{col}_shapiro_pval"] = float(sw_pval)
+ diagnostics[f"{col}_is_normal_shapiro"] = bool(sw_pval > 0.05)
+
+ ad_result = stats.anderson(data, dist="norm")
+ diagnostics[f"{col}_anderson_stat"] = float(ad_result.statistic)
+ diagnostics[f"{col}_anderson_critical_5pct"] = float(ad_result.critical_values[2])
+ diagnostics[f"{col}_is_normal_anderson"] = bool(
+ ad_result.statistic < ad_result.critical_values[2]
+ )
(_osm, _osr), (_slope, _intercept, r) = probplot(data, dist="norm", plot=None)
diagnostics[f"{col}_qq_r_squared"] = float(r**2)
def _validate_distribution_diagnostics(diag: dict[str, Any], *, strict_diagnostics: bool) -> None:
- """Validate mathematical properties of distribution diagnostics.
-
- Ensures all reported statistics are finite and within theoretical bounds where applicable.
- Invoked automatically inside distribution_diagnostics(); raising AssertionError on violation
- enforces fail-fast semantics consistent with other validation helpers.
- """
- # Pre-compute zero-variance flags to allow graceful handling of undefined higher moments.
- zero_var_columns = set()
- for k, v in diag.items():
- if k.endswith("_std") and (not np.isfinite(v) or v == 0):
- prefix = k[: -len("_std")]
- zero_var_columns.add(prefix)
-
- for key, value in list(diag.items()):
- if any(suffix in key for suffix in ["_mean", "_std", "_skewness", "_kurtosis"]): # noqa: SIM102
- if not np.isfinite(value):
- # Graceful degradation for constant distributions: skewness/kurtosis become NaN.
- constant_problem = any(
- key.startswith(prefix)
- and (key.endswith("_skewness") or key.endswith("_kurtosis"))
- for prefix in zero_var_columns
- )
- if constant_problem and not strict_diagnostics:
- fallback = INTERNAL_GUARDS.get("distribution_constant_fallback_moment", 0.0)
- diag[key] = fallback
- warnings.warn(
- f"Stats: {key} undefined (constant distribution); falling back to {fallback}",
- RewardDiagnosticsWarning,
- stacklevel=2,
- )
- else:
- raise AssertionError(
- f"Stats: distribution diagnostic {key} is not finite: {value}"
- )
+ """Validate reported diagnostics without fabricating undefined statistics."""
+ for key, value in diag.items():
+ if value is None and key.endswith(("_skewness", "_kurtosis")):
+ column = key.rsplit("_", 1)[0]
+ if diag.get(f"{column}_constant") is True:
+ continue
+ if value is None or not np.isfinite(value):
+ raise AssertionError(f"Stats: distribution diagnostic {key} is not finite: {value}")
if key.endswith("_shapiro_pval") and not (0 <= value <= 1):
- raise AssertionError(f"Stats: Shapiro p-value {key} must be in [0,1], got {value}")
- if key.endswith("_anderson_stat") or key.endswith("_anderson_critical_5pct"): # noqa: SIM102
- if not np.isfinite(value):
- prefix = key.rsplit("_", 2)[0]
- if prefix in zero_var_columns and not strict_diagnostics:
- fallback = INTERNAL_GUARDS.get("distribution_constant_fallback_moment", 0.0)
- diag[key] = fallback
- warnings.warn(
- f"Stats: {key} undefined (constant distribution); falling back to {fallback}",
- RewardDiagnosticsWarning,
- stacklevel=2,
- )
- continue
- raise AssertionError(f"Stats: Anderson statistic {key} must be finite, got {value}")
- if key.endswith("_qq_r_squared"): # noqa: SIM102
- if not (isinstance(value, (int, float)) and np.isfinite(value) and 0 <= value <= 1):
- prefix = key[: -len("_qq_r_squared")]
- if prefix in zero_var_columns and not strict_diagnostics:
- fallback_r2 = INTERNAL_GUARDS.get("distribution_constant_fallback_qq_r2", 1.0)
- diag[key] = fallback_r2
- warnings.warn(
- f"Stats: {key} undefined (constant distribution); falling back to {fallback_r2}",
- RewardDiagnosticsWarning,
- stacklevel=2,
- )
- else:
- raise AssertionError(f"Stats: Q-Q R^2 {key} must be in [0,1], got {value}")
+ raise AssertionError(f"Stats: Shapiro p-value {key} must be in [0, 1], got {value}")
+ if key.endswith("_qq_r_squared") and not (0 <= value <= 1):
+ raise AssertionError(f"Stats: Q-Q R^2 {key} must be in [0, 1], got {value}")
"""PBRS (Potential-Based Reward Shaping) transforms & helpers."""
return 1.0 if x > 0 else -1.0
-def _apply_transform_asinh(value: float) -> float:
- """asinh: x / sqrt(1 + x^2) in (-1, 1)."""
+def _apply_transform_softsign_sqrt(value: float) -> float:
+ """softsign_sqrt: x / sqrt(1 + x^2) in (-1, 1)."""
return float(value / math.hypot(1.0, value))
"softsign": _apply_transform_softsign,
"arctan": _apply_transform_arctan,
"sigmoid": _apply_transform_sigmoid,
- "asinh": _apply_transform_asinh,
+ "softsign_sqrt": _apply_transform_softsign_sqrt,
"clip": _apply_transform_clip,
}
"""Return validated potential_gamma.
Process:
- - If NaN -> default POTENTIAL_GAMMA_DEFAULT with warning (missing or unparsable).
+ - If missing, unparsable, or non-finite -> POTENTIAL_GAMMA_DEFAULT with warning.
- If outside [0,1] -> clamp + warning including original value.
- Guarantee returned float ∈ [0,1].
"""
+ raw_gamma = params.get("potential_gamma")
gamma = _get_float_param(params, "potential_gamma", np.nan)
if not np.isfinite(gamma):
+ if "potential_gamma" in params:
+ message = (
+ f"PBRS: potential_gamma={raw_gamma!r} is invalid or non-finite; "
+ f"using default {POTENTIAL_GAMMA_DEFAULT}"
+ )
+ else:
+ message = (
+ f"PBRS: potential_gamma not specified; using default {POTENTIAL_GAMMA_DEFAULT}"
+ )
warnings.warn(
- f"PBRS: potential_gamma not specified; falling back to {POTENTIAL_GAMMA_DEFAULT}",
+ message,
RewardDiagnosticsWarning,
stacklevel=2,
)
gamma, reason_parts = _clamp_float_to_bounds("potential_gamma", raw_gamma, strict=False)
if reason_parts:
warnings.warn(
- f"PBRS: potential_gamma={raw_gamma} outside [0,1]; falling back to {gamma}",
+ f"PBRS: potential_gamma={raw_gamma} outside [0,1]; clamped to {gamma}",
RewardDiagnosticsWarning,
stacklevel=2,
)
)
-def _compute_exit_potential(prev_potential: float, params: RewardParams) -> float:
- """Exit potential per mode (canonical/non_canonical -> 0; others transform Φ(prev))."""
+def _compute_exit_potential(
+ prev_potential: float,
+ params: RewardParams,
+ gamma: float,
+) -> float:
+ """Return exit potential using the selected mode and validated PBRS gamma."""
mode = _get_str_param(params, "exit_potential_mode")
if mode == "canonical" or mode == "non_canonical":
return _fail_safely("canonical_exit_potential")
if mode == "progressive_release":
decay = _get_float_param(params, "exit_potential_decay")
- if not np.isfinite(decay) or decay < 0.0:
+ if not np.isfinite(decay):
+ warnings.warn(
+ f"PBRS: exit_potential_decay={decay} is non-finite; using 0.0",
+ RewardDiagnosticsWarning,
+ stacklevel=2,
+ )
+ decay = 0.0
+ elif decay < 0.0:
warnings.warn(
- f"PBRS: exit_potential_decay={decay} invalid or < 0; falling back to 0.0",
+ f"PBRS: exit_potential_decay={decay} below 0; clamped to 0.0",
RewardDiagnosticsWarning,
stacklevel=2,
)
decay = 0.0
if decay > 1.0:
warnings.warn(
- f"PBRS: exit_potential_decay={decay} > 1; falling back to 1.0",
+ f"PBRS: exit_potential_decay={decay} above 1; clamped to 1.0",
RewardDiagnosticsWarning,
stacklevel=2,
)
decay = 1.0
next_potential = prev_potential * (1.0 - decay)
elif mode == "spike_cancel":
- gamma = _get_potential_gamma(params)
if gamma <= 0.0 or not np.isfinite(gamma):
next_potential = prev_potential
else:
base_factor: float,
risk_reward_ratio: float,
prev_potential: float,
+ entry_pnl: float,
is_exit: bool = False,
is_entry: bool = False,
) -> tuple[float, float, float, float, float]:
Then:
Φ_hold(s) = scale · 0.5 · [T_pnl(g·r_pnl) + sign(r_pnl)·m_dur·T_dur(g·r_dur)]
+ Loss-side bound: |Φ_hold| <= scale * (1 + risk_reward_ratio) / 2.
+ Global bound uses scale * max(1, (1 + risk_reward_ratio) / 2).
Returns
-------
- reward_shaping: Δ(s,a,s') = γ·Φ(s') - Φ(s), the PBRS shaping term
- next_potential: Φ(s'), the potential function value for next state
- - pbrs_delta: Same as reward_shaping (kept for backward compatibility)
+ - pbrs_delta: Same as reward_shaping
- entry_additive: Optional non-PBRS entry bonus (0.0 if disabled or not entry)
- exit_additive: Optional non-PBRS exit bonus (0.0 if disabled or not exit)
hold_potential_enabled = _get_bool_param(params, "hold_potential_enabled")
if is_exit:
- next_potential = _compute_exit_potential(prev_potential, params)
+ next_potential = _compute_exit_potential(prev_potential, params, gamma)
pbrs_delta = gamma * next_potential - prev_potential
reward_shaping = pbrs_delta
else:
entry_additive = 0.0
exit_additive = 0.0
else:
- cand_entry_add = _compute_entry_additive(
- next_pnl, pnl_target, next_duration_ratio, params, base_factor
- )
+ cand_entry_add = _compute_entry_additive(entry_pnl, pnl_target, 0.0, params, base_factor)
cand_exit_add = _compute_exit_additive(
current_pnl, pnl_target, current_duration_ratio, params, base_factor
)
base_factor: float,
risk_reward_ratio: float,
prev_potential: float,
+ entry_pnl: float,
is_exit: bool = False,
is_entry: bool = False,
) -> tuple[float, float, float, float, float, float]:
- """Compute shaped reward and PBRS diagnostics.
-
- .. deprecated::
- This function exists only for backward compatibility with existing tests.
- New code should use :func:`compute_pbrs_components` and compute the total reward manually.
-
- This is a thin wrapper around `compute_pbrs_components()` that adds PBRS and
- optional additive terms to the provided `base_reward`.
+ """Compute total shaped reward and its PBRS/additive components.
Returns
-------
prev_potential=prev_potential,
is_exit=is_exit,
is_entry=is_entry,
+ entry_pnl=entry_pnl,
)
)
default=-1,
help="Number of parallel jobs for permutation_importance (default: -1 for all CPUs).",
)
- parser.add_argument(
- "--stats_seed",
- type=int,
- default=None,
- help="Optional separate seed for statistical analyses (default: same as --seed).",
- )
parser.add_argument(
"--base_factor",
type=float,
default=None,
help="Path to real episodes pickle for distribution shift analysis (optional).",
)
- parser.add_argument(
- "--pvalue_adjust",
- type=str.lower,
- choices=list(ADJUST_METHODS),
- default=ADJUST_METHODS[0],
- help="Multiple testing correction method for hypothesis tests (default: none).",
- )
parser.add_argument(
"--strict_diagnostics",
action="store_true",
help=(
- "Enable fail-fast mode for statistical diagnostics: raise on zero-width bootstrap CIs or undefined "
- "skewness/kurtosis/Anderson/Q-Q metrics produced by constant distributions instead of applying graceful replacements."
+ "Raise on extreme distribution moments instead of warning. Constant distributions "
+ "retain N/A diagnostics in either mode."
),
)
parser.add_argument(
default=True,
help="Enable strict parameter validation (raise on out-of-bounds or non-finite reward parameters). Default: enabled.",
)
- parser.add_argument(
- "--bootstrap_resamples",
- type=int,
- default=10000,
- metavar="N",
- help=(
- "Number of bootstrap resamples for confidence intervals (default: 10000). "
- "Lower this (e.g. 200-1000) for faster smoke tests; increase for more stable CI width estimates."
- ),
- )
parser.add_argument(
"--unrealized_pnl",
action="store_true",
- help="Simulate unrealized PnL during holds to feed Φ(s) (optional; default: disabled).",
+ help=(
+ "Simulate fee-aware unrealized PnL while a position is open. This transforms the "
+ "retained synthetic price/PnL trajectory and affects unrealized extrema, base and "
+ "PBRS rewards when enabled. Simulation-only; disabled by default."
+ ),
)
return parser
+def verify_pbrs_trajectory(df: pd.DataFrame, gamma: float) -> dict[str, Any]:
+ """Verify complete ordered episodes, local shaping and discounted telescoping.
+
+ Evidence requires episode_id, zero-based contiguous transition_index,
+ terminated, prev_potential, next_potential and reward_shaping. Verification
+ concerns these observed trajectories, not a proof of policy invariance.
+ """
+ required = {
+ "episode_id",
+ "transition_index",
+ "terminated",
+ "prev_potential",
+ "next_potential",
+ "reward_shaping",
+ }
+ if not required.issubset(df.columns) or df.empty:
+ return {"verified": False, "reason": "Insufficient ordered trajectory evidence"}
+ if not np.isfinite(gamma) or not 0 <= gamma <= 1 or df["episode_id"].isna().any():
+ return {"verified": False, "reason": "Invalid discount or episode identity"}
+ max_residual = 0.0
+ for _, episode in df.groupby("episode_id", sort=False):
+ n = len(episode)
+ if not np.array_equal(episode["transition_index"], np.arange(n)):
+ return {"verified": False, "reason": "Missing, repeated or unordered transitions"}
+ terminal = episode["terminated"].to_numpy()
+ if not np.array_equal(terminal, np.arange(n) == n - 1):
+ return {"verified": False, "reason": "Incomplete episode boundaries"}
+ prev = episode["prev_potential"].to_numpy(dtype=float)
+ nxt = episode["next_potential"].to_numpy(dtype=float)
+ shaping = episode["reward_shaping"].to_numpy(dtype=float)
+ if not all(np.isfinite(x).all() for x in (prev, nxt, shaping)):
+ return {"verified": False, "reason": "Non-finite transition evidence"}
+ if not np.allclose(nxt[:-1], prev[1:], atol=PBRS_INVARIANCE_TOL, rtol=0):
+ return {"verified": False, "reason": "Potential discontinuity between transitions"}
+ if not np.allclose(shaping, gamma * nxt - prev, atol=PBRS_INVARIANCE_TOL, rtol=0):
+ return {"verified": False, "reason": "Local shaping identity violated"}
+ residual = float(np.dot(gamma ** np.arange(n), shaping) + prev[0] - gamma**n * nxt[-1])
+ max_residual = max(max_residual, abs(residual))
+ if abs(residual) > PBRS_INVARIANCE_TOL or abs(nxt[-1]) > PBRS_INVARIANCE_TOL:
+ return {
+ "verified": False,
+ "reason": "Discounted boundary or terminal potential violated",
+ }
+ return {
+ "verified": True,
+ "reason": "Local identity, continuity and discounted terminal boundary verified",
+ "max_discounted_residual": max_residual,
+ }
+
+
def write_complete_statistical_analysis(
df: pd.DataFrame,
output_dir: Path,
seed: int,
real_df: pd.DataFrame | None = None,
*,
+ independent_observations: bool = False,
adjust_method: str = ADJUST_METHODS[0],
stats_seed: int | None = None,
strict_diagnostics: bool = False,
if isinstance(df.attrs.get("reward_params"), dict)
else {}
)
+ classification_keys = {
+ "exit_potential_mode",
+ "entry_additive_enabled",
+ "exit_additive_enabled",
+ }
+ classification_metadata_available = classification_keys.issubset(reward_params)
max_trade_duration_candles = _get_int_param(reward_params, "max_trade_duration_candles")
# Helpers: consistent Markdown table renderers
test_seed = (
stats_seed if isinstance(stats_seed, int) else (seed if isinstance(seed, int) else 42)
)
- hypothesis_tests = statistical_hypothesis_tests(df, adjust_method=adjust_method, seed=test_seed)
+ hypothesis_tests = (
+ statistical_hypothesis_tests(
+ df, independent_observations=True, adjust_method=adjust_method, seed=test_seed
+ )
+ if independent_observations
+ else {}
+ )
metrics_for_ci = [
"reward",
"reward_idle",
if col in df.columns
]
metrics_for_ci.extend(extra_ci_cols)
- bootstrap_ci = bootstrap_confidence_intervals(
+ bootstrap_ci = (
+ bootstrap_confidence_intervals(
+ df,
+ metrics_for_ci,
+ independent_observations=True,
+ n_bootstrap=int(bootstrap_resamples),
+ seed=test_seed,
+ strict_diagnostics=strict_diagnostics,
+ )
+ if independent_observations
+ else {}
+ )
+ dist_diagnostics = distribution_diagnostics(
df,
- metrics_for_ci,
- n_bootstrap=int(bootstrap_resamples),
seed=test_seed,
strict_diagnostics=strict_diagnostics,
- )
- dist_diagnostics = distribution_diagnostics(
- df, seed=test_seed, strict_diagnostics=strict_diagnostics
+ independent_observations=independent_observations,
)
distribution_shift = None
if real_df is not None:
- distribution_shift = compute_distribution_shift_metrics(df, real_df)
+ distribution_shift = compute_distribution_shift_metrics(
+ df, real_df, independent_observations=independent_observations
+ )
# Write comprehensive report
with report_path.open("w", encoding="utf-8") as f:
# Blank separator to visually group core simulation vs PBRS parameters
f.write("| | |\n")
# Core PBRS parameters exposed in run configuration if present
- reward_params: RewardParams = (
- dict(df.attrs.get("reward_params"))
- if isinstance(df.attrs.get("reward_params"), dict)
- else {}
+ exit_mode = (
+ _get_str_param(reward_params, "exit_potential_mode")
+ if "exit_potential_mode" in reward_params
+ else "unknown"
+ )
+ potential_gamma: float | str = (
+ _get_potential_gamma(reward_params) if "potential_gamma" in reward_params else "unknown"
)
- exit_mode = _get_str_param(reward_params, "exit_potential_mode")
- potential_gamma = _get_potential_gamma(reward_params)
f.write(f"| exit_potential_mode | {exit_mode} |\n")
f.write(f"| potential_gamma | {potential_gamma} |\n")
# Additional configuration details
f.write(f"| skip_partial_dependence | {skip_partial_dependence} |\n")
f.write(f"| rf_n_jobs | {rf_n_jobs} |\n")
f.write(f"| perm_n_jobs | {perm_n_jobs} |\n")
- f.write(f"| bootstrap_resamples | {bootstrap_resamples} |\n")
- f.write(f"| pvalue_adjust_method | {adjust_method} |\n")
- # Blank separator before overrides block
+ if independent_observations:
+ f.write(f"| bootstrap_resamples | {bootstrap_resamples} |\n")
+ f.write(f"| pvalue_adjust_method | {adjust_method} |\n")
f.write("| | |\n")
overrides_pairs: list[str] = []
"reward_exit_additive",
]
pbrs_present = all(col in df.columns for col in pbrs_components)
+ invariance_status: str | None = None
if pbrs_present:
# PBRS activation rates
exit_add_total = df.get("reward_exit_additive", pd.Series([0])).sum()
# Get configuration for proper invariance assessment
- reward_params = df.attrs.get("reward_params", {}) if hasattr(df, "attrs") else {}
- exit_potential_mode = _get_str_param(reward_params, "exit_potential_mode", "canonical")
- entry_additive_enabled_raw = _get_bool_param(
- reward_params, "entry_additive_enabled", False
- )
- exit_additive_enabled_raw = _get_bool_param(
- reward_params, "exit_additive_enabled", False
- )
-
- (
- entry_additive_effective,
- exit_additive_effective,
- additives_suppressed,
- ) = _resolve_additive_enablement(
- exit_potential_mode,
- entry_additive_enabled_raw,
- exit_additive_enabled_raw,
+ if classification_metadata_available:
+ exit_potential_mode: str | None = _get_str_param(
+ reward_params, "exit_potential_mode"
+ )
+ entry_additive_enabled_raw: bool | None = _get_bool_param(
+ reward_params, "entry_additive_enabled", False
+ )
+ exit_additive_enabled_raw: bool | None = _get_bool_param(
+ reward_params, "exit_additive_enabled", False
+ )
+ (
+ entry_additive_effective,
+ exit_additive_effective,
+ additives_suppressed,
+ ) = _resolve_additive_enablement(
+ exit_potential_mode,
+ entry_additive_enabled_raw,
+ exit_additive_enabled_raw,
+ )
+ else:
+ exit_potential_mode = None
+ entry_additive_enabled_raw = None
+ exit_additive_enabled_raw = None
+ entry_additive_effective = False
+ exit_additive_effective = False
+ additives_suppressed = False
+
+ evidence = (
+ verify_pbrs_trajectory(df, _get_potential_gamma(reward_params))
+ if classification_metadata_available
+ else {
+ "verified": False,
+ "reason": "Trajectory verification skipped without reward configuration evidence",
+ }
)
-
- # True PBRS invariance classification:
- # - Canonical requires canonical mode AND no effective additives.
- # - When `reward_invariance_correction` is present, we use it as the primary
- # diagnostic (reward_shaping - reward_pbrs_delta).
- # - Otherwise, we fall back to the weaker heuristic |Σ shaping| ≈ 0.
- is_theoretically_invariant = exit_potential_mode == "canonical" and not (
- entry_additive_effective or exit_additive_effective
+ canonical_configuration = classification_metadata_available and (
+ exit_potential_mode == "canonical"
+ and not (entry_additive_effective or exit_additive_effective)
)
-
- has_inv_correction = "reward_invariance_correction" in df.columns
- max_abs_inv_correction: float | None
- if has_inv_correction:
- max_abs_inv_correction = float(df["reward_invariance_correction"].abs().max())
- correction_near_zero = max_abs_inv_correction < PBRS_INVARIANCE_TOL
+ observed_additive_issues = []
+ for column in ("reward_entry_additive", "reward_exit_additive"):
+ if column not in df.columns:
+ observed_additive_issues.append(f"{column} is missing")
+ continue
+ observed = pd.to_numeric(df[column], errors="coerce").to_numpy(dtype=float)
+ if not np.isfinite(observed).all():
+ observed_additive_issues.append(f"{column} contains non-finite values")
+ elif not np.allclose(observed, 0.0, atol=PBRS_INVARIANCE_TOL, rtol=0):
+ observed_additive_issues.append(f"{column} contains non-zero values")
+ canonical_observations = not observed_additive_issues
+
+ if not classification_metadata_available:
+ invariance_status = "Not verified"
+ elif not canonical_configuration:
+ invariance_status = "Non-canonical: not verified"
+ elif evidence["verified"] and canonical_observations:
+ invariance_status = "Canonical: observed PBRS verified"
else:
- max_abs_inv_correction = None
- correction_near_zero = None
- shaping_near_zero = abs(total_shaping) < PBRS_INVARIANCE_TOL
-
- suppression_note = ""
- if additives_suppressed:
- suppression_note = (
- " Additives are suppressed in canonical mode"
- f" (requested entry_additive_enabled={bool(entry_additive_enabled_raw)},"
- f" exit_additive_enabled={bool(exit_additive_enabled_raw)})."
- )
+ invariance_status = "Not verified"
- # Prepare invariance summary markdown block
- if is_theoretically_invariant:
- if correction_near_zero is True:
- invariance_status = "✅ Canonical"
- invariance_note = (
- "Theoretical invariance preserved (canonical mode, no additives, max|correction|≈0)."
- + suppression_note
- )
- elif correction_near_zero is False:
- invariance_status = "⚠️ Canonical (with warning)"
- invariance_note = (
- "Canonical mode but invariance correction is non-zero"
- f" (max|correction|={max_abs_inv_correction:.6e})." + suppression_note
- )
- else:
- # Fallback: without invariance correction, use Σ shaping as a heuristic.
- if shaping_near_zero:
- invariance_status = "✅ Canonical"
- invariance_note = (
- "Theoretical invariance preserved (canonical mode, no additives, Σ≈0)."
- + suppression_note
- )
- else:
- invariance_status = "⚠️ Canonical (with warning)"
- invariance_note = (
- "Canonical mode but Σ shaping is non-zero"
- f" (Σ={total_shaping:.6f}; correction column unavailable)."
- + suppression_note
- )
- else:
- invariance_status = "❌ Non-canonical"
+ invariance_note = evidence["reason"] + ". Raw shaping sums do not certify invariance."
+ if not classification_metadata_available:
+ invariance_note += " Reward configuration evidence is missing or incomplete."
+ elif not canonical_configuration:
reasons = []
if exit_potential_mode != "canonical":
reasons.append(f"exit_potential_mode='{exit_potential_mode}'")
if exit_additive_effective:
additive_types.append("exit")
reasons.append(f"additives={additive_types}")
- invariance_note = f"Modified for flexibility: {', '.join(reasons)}"
+ invariance_note += f" Modified for flexibility: {', '.join(reasons)}"
+ elif observed_additive_issues:
+ invariance_note += (
+ " Observed additive components invalid: "
+ + "; ".join(observed_additive_issues)
+ + "."
+ )
+ elif additives_suppressed:
+ invariance_note += " Additives are suppressed in canonical mode."
# Summarize PBRS invariance
f.write("**PBRS Invariance Summary:**\n\n")
f.write("| Field | Value |\n")
f.write("|-------|-------|\n")
f.write(f"| Invariance Status | {invariance_status} |\n")
f.write(f"| Analysis Note | {invariance_note} |\n")
- f.write(f"| Exit Potential Mode | {exit_potential_mode} |\n")
- f.write(f"| Entry Additive Enabled | {bool(entry_additive_enabled_raw)} |\n")
- f.write(f"| Exit Additive Enabled | {bool(exit_additive_enabled_raw)} |\n")
- f.write(f"| Entry Additive Effective | {bool(entry_additive_effective)} |\n")
- f.write(f"| Exit Additive Effective | {bool(exit_additive_effective)} |\n")
+ f.write(
+ f"| Exit Potential Mode | {exit_potential_mode if exit_potential_mode is not None else 'unknown'} |\n"
+ )
+ f.write(
+ f"| Entry Additive Enabled | {bool(entry_additive_enabled_raw) if entry_additive_enabled_raw is not None else 'unknown'} |\n"
+ )
+ f.write(
+ f"| Exit Additive Enabled | {bool(exit_additive_enabled_raw) if exit_additive_enabled_raw is not None else 'unknown'} |\n"
+ )
+ f.write(
+ f"| Entry Additive Effective | {bool(entry_additive_effective) if classification_metadata_available else 'unknown'} |\n"
+ )
+ f.write(
+ f"| Exit Additive Effective | {bool(exit_additive_effective) if classification_metadata_available else 'unknown'} |\n"
+ )
f.write(f"| Σ Shaping Reward | {total_shaping:.6f} |\n")
f.write(f"| Abs Σ Shaping Reward | {abs(total_shaping):.6e} |\n")
f.write(f"| Σ Entry Additive | {entry_add_total:.6f} |\n")
)
else:
f.write(
- "Machine learning analysis to identify which features most influence total reward.\n\n"
+ "Random Forest within-trajectory descriptive analysis, not independent generalization evidence.\n\n"
)
f.write("**Model:** Random Forest Regressor (400 trees) \n")
f.write(f"**R² Score:** {analysis_stats['r2_score']:.4f}\n\n")
"- Partial dependence plots: (skipped via --skip_partial_dependence)\n\n"
)
- # Section 5: Statistical Validation
- if hypothesis_tests:
+ # Section 5: Statistical Analysis
+ if hypothesis_tests or bootstrap_ci or dist_diagnostics or distribution_shift:
f.write("---\n\n")
- f.write("## 5. Statistical Validation\n\n")
- f.write("Rigorous statistical tests to validate reward behavior and relationships.\n\n")
-
- f.write("### 5.1 Hypothesis Tests\n\n")
-
- if "idle_correlation" in hypothesis_tests:
- h = hypothesis_tests["idle_correlation"]
- f.write("#### 5.1.1 Idle Duration → Idle Penalty Correlation\n\n")
- f.write(f"**Test Method:** {h['test']}\n\n")
- f.write(f"- Spearman ρ: **{h['rho']:.4f}**\n") # noqa: RUF001
- f.write(f"- p-value: {h['p_value']:.4g}\n")
- if "p_value_adj" in h:
- f.write(
- f"- p-value (adj BH): {h['p_value_adj']:.4g} -> {'✅ Yes' if h['significant_adj'] else '❌ No'} (α=0.05)\n" # noqa: RUF001
- )
- f.write(f"- 95% CI: [{h['ci_95'][0]:.4f}, {h['ci_95'][1]:.4f}]\n")
- f.write(f"- CI width: {(h['ci_95'][1] - h['ci_95'][0]):.4f}\n")
- f.write(f"- Sample size: {h['n_samples']:,}\n")
- f.write(f"- Significant (α=0.05): {'✅ Yes' if h['significant'] else '❌ No'}\n") # noqa: RUF001
- f.write(f"- **Interpretation:** {h['interpretation']}\n\n")
-
- if "position_reward_difference" in hypothesis_tests:
- h = hypothesis_tests["position_reward_difference"]
- f.write("#### 5.1.2 Position-Based Reward Differences\n\n")
- f.write(f"**Test Method:** {h['test']}\n\n")
- f.write(f"- H-statistic: **{h['statistic']:.4f}**\n")
- f.write(f"- p-value: {h['p_value']:.4g}\n")
- if "p_value_adj" in h:
- f.write(
- f"- p-value (adj BH): {h['p_value_adj']:.4g} -> {'✅ Yes' if h['significant_adj'] else '❌ No'} (α=0.05)\n" # noqa: RUF001
- )
- f.write(f"- Effect size (ε²): {h['effect_size_epsilon_sq']:.4f}\n")
- f.write(f"- Number of groups: {h['n_groups']}\n")
- f.write(f"- Significant (α=0.05): {'✅ Yes' if h['significant'] else '❌ No'}\n") # noqa: RUF001
- f.write(f"- **Interpretation:** {h['interpretation']} effect\n\n")
-
- if "pnl_sign_reward_difference" in hypothesis_tests:
- h = hypothesis_tests["pnl_sign_reward_difference"]
- f.write("#### 5.1.3 Positive vs Negative PnL Comparison\n\n")
- f.write(f"**Test Method:** {h['test']}\n\n")
- f.write(f"- U-statistic: **{h['statistic']:.4f}**\n")
- f.write(f"- p-value: {h['p_value']:.4g}\n")
- if "p_value_adj" in h:
- f.write(
- f"- p-value (adj BH): {h['p_value_adj']:.4g} -> {'✅ Yes' if h['significant_adj'] else '❌ No'} (α=0.05)\n" # noqa: RUF001
- )
- f.write(f"- Median (PnL+): {h['median_pnl_positive']:.4f}\n")
- f.write(f"- Median (PnL-): {h['median_pnl_negative']:.4f}\n")
- f.write(f"- Significant (α=0.05): {'✅ Yes' if h['significant'] else '❌ No'}\n\n") # noqa: RUF001
+ f.write("## 5. Statistical Analysis\n\n")
+ if not independent_observations:
+ f.write(
+ "Descriptive trajectory analysis: inferential tests, p-values and confidence intervals are suppressed because independent observations were not declared.\n\n"
+ )
+
+ if hypothesis_tests:
+ f.write("### 5.1 Hypothesis Tests\n\n")
+ for name, result in hypothesis_tests.items():
+ f.write(f"#### {name.replace('_', ' ').title()}\n\n")
+ if not result["applicable"]:
+ f.write(
+ "N/A (undefined test statistic); excluded from the correction family.\n\n"
+ )
+ continue
+ f.write("| Metric | Value |\n|--------|-------|\n")
+ for key, value in result.items():
+ f.write(f"| {key} | {_fmt_val(value)} |\n")
+ f.write("\n")
# Bootstrap CI
if bootstrap_ci:
f.write("### 5.2 Confidence Intervals\n\n")
f.write(
- "Bootstrap confidence intervals (95%, 10,000 resamples) for key metrics:\n\n"
+ f"Bootstrap confidence intervals (95%, {bootstrap_resamples:,} resamples) for key metrics:\n\n"
)
f.write("| Metric | Mean | 95% CI Lower | 95% CI Upper | Width |\n")
f.write("|--------|------|--------------|--------------|-------|\n")
# Distribution diagnostics
if dist_diagnostics:
- f.write("### 5.3 Distribution Normality Tests\n\n")
- f.write("Statistical tests for normality of key distributions:\n\n")
+ f.write("### 5.3 Distribution Diagnostics\n\n")
+ f.write(
+ "Moments and descriptive Q-Q fit; normality tests require independent observations.\n\n"
+ )
for col in ["reward", "pnl", "trade_duration", "idle_duration"]:
if f"{col}_mean" in dist_diagnostics:
f.write(f"#### {col.replace('_', ' ').title()}\n\n")
f.write("|--------|-------|\n")
f.write(f"| Mean | {dist_diagnostics[f'{col}_mean']:.4f} |\n")
f.write(f"| Std Dev | {dist_diagnostics[f'{col}_std']:.4f} |\n")
- f.write(f"| Skewness | {dist_diagnostics[f'{col}_skewness']:.4f} |\n")
- f.write(f"| Kurtosis | {dist_diagnostics[f'{col}_kurtosis']:.4f} |\n")
+ for label, suffix in (("Skewness", "skewness"), ("Kurtosis", "kurtosis")):
+ value = dist_diagnostics[f"{col}_{suffix}"]
+ rendered = (
+ "N/A (constant distribution)" if value is None else f"{value:.4f}"
+ )
+ f.write(f"| {label} | {rendered} |\n")
+ if dist_diagnostics.get(f"{col}_constant", False):
+ f.write("| Normality Tests | N/A (constant distribution) |\n")
+ f.write("| Q-Q Plot R² | N/A (constant distribution) |\n")
if f"{col}_shapiro_pval" in dist_diagnostics:
is_normal = (
"✅ Yes"
js = distribution_shift.get(f"{feature}_js_distance", np.nan)
ws = distribution_shift.get(f"{feature}_wasserstein", np.nan)
ks_stat = distribution_shift.get(f"{feature}_ks_statistic", np.nan)
- ks_p = distribution_shift.get(f"{feature}_ks_pvalue", np.nan)
+ ks_p = distribution_shift.get(f"{feature}_ks_pvalue")
+ ks_p_text = "N/A" if ks_p is None else f"{ks_p:.4g}"
f.write(
- f"| {feature} | {kl:.4f} | {js:.4f} | {ws:.4f} | {ks_stat:.4f} | {ks_p:.4g} |\n"
+ f"| {feature} | {kl:.4f} | {js:.4f} | {ws:.4f} | {ks_stat:.4f} | {ks_p_text} |\n"
)
f.write("\n")
f.write("**Interpretation Guide:**\n\n")
f.write("|--------|-----------|--------|\n")
f.write("| KL Divergence | < 0.3 | ✅ Yes: Good representativeness |\n")
f.write("| JS Distance | < 0.2 | ✅ Yes: Similar distributions |\n")
- f.write("| KS p-value | > 0.05 | ✅ Yes: No significant difference |\n\n")
+ if independent_observations:
+ f.write(
+ "| KS p-value | > 0.05 | Does not reject equality; not proof of equivalence |\n\n"
+ )
else:
# Placeholder keeps numbering stable and explicit
f.write("### 5.4 Distribution Shift Analysis\n\n")
)
else:
f.write("4. **Feature Importance** - Machine learning analysis of key drivers\n")
- f.write("5. **Statistical Validation** - Hypothesis tests and confidence intervals\n")
+ f.write(
+ "5. **Statistical Analysis** - "
+ + (
+ "Independent-observation tests and confidence intervals\n"
+ if independent_observations
+ else "Descriptive diagnostics only; inference suppressed\n"
+ )
+ )
if distribution_shift:
f.write("6. **Distribution Shift** - Comparison with real trading data\n")
else:
f.write("6. **Distribution Shift** - Not performed (no real episodes provided)\n")
- if "reward_shaping" in df.columns:
- _total_shaping = float(df["reward_shaping"].sum())
- if "reward_invariance_correction" in df.columns:
- _max_abs_corr = float(df["reward_invariance_correction"].abs().max())
- _canonical = _max_abs_corr < PBRS_INVARIANCE_TOL
- _pbrs_summary = (
- "Canonical (max|correction| ≈ 0)"
- if _canonical
- else f"Canonical (with warning; max|correction|={_max_abs_corr:.6e})"
- )
- else:
- _canonical = abs(_total_shaping) < PBRS_INVARIANCE_TOL
- _pbrs_summary = (
- "Canonical (Σ shaping ≈ 0)"
- if _canonical
- else f"Canonical (with warning; Σ shaping={_total_shaping:.6f})"
- )
-
- f.write("7. **PBRS Invariance** - " + _pbrs_summary + "\n")
+ if invariance_status is not None:
+ f.write("7. **PBRS Invariance** - " + invariance_status + "\n")
f.write("\n")
f.write("**Generated Files:**\n")
- f.write("- `reward_samples.csv` - Raw synthetic samples\n")
if not skip_feature_analysis and len(df) >= 4:
f.write("- `feature_importance.csv` - Complete feature importance rankings\n")
f.write("- `partial_dependence_*.csv` - Partial dependence data for visualization\n")
]
print("CLI: Parameter adjustments applied\n" + "\n".join(adj_lines))
+ # Effective values: defaults < explicit flags < --params, resolved once.
base_factor = _get_float_param(params, "base_factor", float(args.base_factor))
profit_aim = _get_float_param(params, "profit_aim", float(args.profit_aim))
- risk_reward_ratio = _get_float_param(
- params,
- "risk_reward_ratio",
- _get_float_param(params, "rr", float(args.risk_reward_ratio)),
- )
-
- cli_action_masking = _to_bool(args.action_masking)
- if "action_masking" in params:
- params["action_masking"] = _to_bool(params["action_masking"])
- else:
- params["action_masking"] = cli_action_masking
- params["unrealized_pnl"] = bool(getattr(args, "unrealized_pnl", False))
- # Propagate strict flag into params for downstream runtime guards
- params["strict_validation"] = bool(getattr(args, "strict_validation", True))
-
+ risk_reward_ratio = _get_float_param(params, "risk_reward_ratio", float(args.risk_reward_ratio))
+ params["action_masking"] = _to_bool(params.get("action_masking", args.action_masking))
+ params["unrealized_pnl"] = bool(args.unrealized_pnl)
# Deterministic seeds cascade
random.seed(args.seed)
np.random.seed(args.seed)
+ real_df = None
+ if args.real_episodes is not None:
+ # Fail before any artifact is written for an explicitly requested file.
+ print(f"CLI: Loading real episodes from {args.real_episodes}")
+ real_df = load_real_episodes(args.real_episodes)
df = simulate_samples(
num_samples=args.num_samples,
"perm_n_jobs",
"skip_feature_analysis",
"skip_partial_dependence",
- "stats_seed",
"strict_diagnostics",
- "bootstrap_resamples",
- "pvalue_adjust",
"real_episodes",
"unrealized_pnl",
"action_masking",
df.to_csv(csv_path, index=False)
sample_output_message = f"Samples saved to {csv_path}"
- # Load real episodes if provided
- real_df = None
- if args.real_episodes and args.real_episodes.exists():
- print(f"CLI: Loading real episodes from {args.real_episodes}")
- real_df = load_real_episodes(args.real_episodes)
-
# Generate consolidated statistical analysis report (with enhanced tests)
print("CLI: Generating statistical analysis")
risk_reward_ratio=risk_reward_ratio,
seed=args.seed,
real_df=real_df,
- adjust_method=args.pvalue_adjust,
- stats_seed=(args.stats_seed if getattr(args, "stats_seed", None) is not None else None),
strict_diagnostics=bool(getattr(args, "strict_diagnostics", False)),
- bootstrap_resamples=getattr(args, "bootstrap_resamples", 10000),
skip_partial_dependence=bool(getattr(args, "skip_partial_dependence", False)),
skip_feature_analysis=bool(getattr(args, "skip_feature_analysis", False)),
rf_n_jobs=int(getattr(args, "rf_n_jobs", -1)),
"num_samples": len(df),
"seed": int(args.seed),
"pnl_target": float(profit_aim * risk_reward_ratio),
- "pvalue_adjust_method": args.pvalue_adjust,
"parameter_adjustments": adjustments,
"reward_params": resolved_reward_params,
+ "effective": {
+ "base_factor": float(base_factor),
+ "profit_aim": float(profit_aim),
+ "risk_reward_ratio": float(risk_reward_ratio),
+ },
}
sim_params_dict = df.attrs.get("simulation_params", {})
if not isinstance(sim_params_dict, dict):
* Per-scenario timing and aggregate statistics (mean / min / max / median / p95 seconds).
* Warning counting based on header lines plus a breakdown of distinct warning headers.
* Log tail truncation controlled via `--tail_chars` (characters) or full logs via `--full_logs`.
-* Direct CLI forwarding of bootstrap resample count to the child process.
Usage
-----
python test_reward_space_analysis_cli.py --num_samples 50 --out_dir ../sample_run_output \
- --shuffle_seed 123 --strict_sample 3 --bootstrap_resamples 200
+ --shuffle_seed 123 --strict_sample 3
JSON Summary fields
-------------------
"spike_cancel",
"retain_previous",
]
- exit_attenuation_modes = ["sqrt", "linear", "power", "half_life", "legacy"]
+ exit_attenuation_modes = ["sqrt", "linear", "power", "half_life"]
potential_gammas = [0.0, 0.5, 0.95, 0.999]
hold_enabled = [0, 1]
entry_additive_enabled = [0, 1]
num_samples: int,
conf: ConfigTuple,
strict: bool,
- bootstrap_resamples: int,
timeout: int,
skip_feature_analysis: bool = False,
skip_partial_dependence: bool = False,
"--seed",
str(100 + idx),
]
- # Forward bootstrap resamples explicitly
- cmd += ["--bootstrap_resamples", str(bootstrap_resamples)]
if skip_feature_analysis:
cmd.append("--skip_feature_analysis")
if skip_partial_dependence:
default=40,
help="Maximum number of (non-strict) scenarios before strict duplication",
)
- parser.add_argument(
- "--bootstrap_resamples",
- type=int,
- default=120,
- help="Number of bootstrap resamples to pass to child processes (speed/perf tradeoff)",
- )
parser.add_argument(
"--per_scenario_timeout",
type=int,
parser.add_argument(
"--unrealized_pnl",
action="store_true",
- help="Forward --unrealized_pnl to child process to exercise hold Φ(s) path.",
+ help=(
+ "Forward --unrealized_pnl so the retained fee-aware PnL trajectory affects "
+ "extrema and PnL-dependent rewards."
+ ),
)
parser.add_argument(
"--params",
parser.error("--num_samples must be >= 4 unless --skip_feature_analysis is set")
if args.strict_sample < 0:
parser.error("--strict_sample must be >= 0")
- if args.bootstrap_resamples <= 0:
- parser.error("--bootstrap_resamples must be > 0")
if args.tail_chars < 0:
parser.error("--tail_chars must be >= 0")
if args.per_scenario_timeout <= 0:
num_samples=args.num_samples,
conf=conf,
strict=strict,
- bootstrap_resamples=args.bootstrap_resamples,
timeout=args.per_scenario_timeout,
skip_feature_analysis=args.skip_feature_analysis,
skip_partial_dependence=args.skip_partial_dependence,
"shuffle_seed": args.shuffle_seed,
"strict_sample": args.strict_sample,
"max_scenarios": args.max_scenarios,
- "bootstrap_resamples": args.bootstrap_resamples,
},
"metadata": {
"timestamp_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
- Verify: reward_shaping ≈ -prev_potential (canonical exit)
- Checks proper potential release at terminal
- 3. Part C: Cumulative shaping magnitude
- - Track cumulative shaping over 100-episode sequence
- - Assert: Bounded magnitude (no systematic bias accumulation)
-
+ 3. Part C: Trajectory identity verification
+ - Verify each local shaping identity and adjacent potential continuity
+ - Verify the discounted finite-trajectory boundary identity
+ - Require terminal next potential to be zero
**Setup:**
- Exit modes: [progressive_release, spike_cancel, canonical]
- Gamma values: [0.9, 0.95, 0.99]
**Assertions:**
- Terminal potential: |next_potential| < TOLERANCE.IDENTITY_RELAXED
- - Shaping recovery: |shaping + prev_potential| < TOLERANCE.IDENTITY_RELAXED
- - Cumulative sum bound: |sum(shaping)| < 10 * TOLERANCE.IDENTITY_RELAXED
+ - Local identity: shaping_t ≈ gamma * next_potential_t - prev_potential_t
+ - Continuity: next_potential_t ≈ prev_potential_(t+1)
+ - Discounted boundary: sum(gamma**t * shaping_t) + prev_potential_0
+ - gamma**T * next_potential_(T-1) ≈ 0
**Tolerance rationale:**
- IDENTITY_RELAXED: PBRS calculations involve gamma discounting,
`assertFinite`, `assertLess`, etc.)
- **Custom assertions**: Project-specific helpers (e.g.,
`assert_component_sum_integrity`) built on unittest base
-- **Backward compatibility**: Gradual migration path from pure unittest
### Base Class
| ID | Category | Description | Owning File | Notes |
| -------------------------------------------- | ----------- | ----------------------------------------------------------------------------------- | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
-| report-abs-shaping-line-091 | integration | Abs Σ Shaping Reward line present & formatted | integration/test_report_formatting.py:4 | Module docstring; primary test at line 95. PBRS report may render line; formatting owned here |
| report-additives-deterministic-092 | components | Additives deterministic report section | components/test_additives.py:4 | Integration/PBRS may reference outcome non-owning |
| robustness-decomposition-integrity-101 | robustness | Single active core component equals total reward under mutually exclusive scenarios | robustness/test_robustness.py:43 | Scenarios: idle, hold, exit, invalid; non-owning refs integration/test_reward_calculation.py |
| robustness-exit-mode-fallback-102 | robustness | Unknown exit_attenuation_mode falls back to linear w/ warning | robustness/test_robustness.py:654 | Comment line (function at :655) |
| robustness-negative-grace-clamp-103 | robustness | Negative exit_plateau_grace clamps to 0.0 w/ warning | robustness/test_robustness.py:696 | |
| robustness-invalid-power-tau-104 | robustness | Invalid power tau falls back alpha=1.0 w/ warning | robustness/test_robustness.py:747 | |
| robustness-near-zero-half-life-105 | robustness | Near-zero half life yields no attenuation (factor≈base) | robustness/test_robustness.py:792 | |
-| pbrs-canonical-exit-semantic-106 | pbrs | Canonical exit uses shaping=-prev_potential and next_potential=0.0 | pbrs/test_pbrs.py:374 | Uses stored potential across steps; no drift correction applied |
-| pbrs-canonical-near-zero-report-116 | pbrs | Canonical near-zero cumulative shaping classification | pbrs/test_pbrs.py:1223 | Full report classification |
-| statistics-partial-deps-skip-107 | statistics | skip_partial_dependence => empty PD structures | statistics/test_statistics.py:42 | Docstring line |
-| helpers-duplicate-rows-drop-108 | helpers | Duplicate rows dropped w/ warning counting removals | helpers/test_utilities.py:27 | Docstring line |
-| helpers-missing-cols-fill-109 | helpers | Missing required columns filled with NaN + single warning | helpers/test_utilities.py:51 | Docstring line |
-| statistics-binned-stats-min-edges-110 | statistics | <2 bin edges raises ValueError | statistics/test_statistics.py:60 | Docstring line |
-| statistics-constant-cols-exclusion-111 | statistics | Constant columns excluded & listed | statistics/test_statistics.py:71 | Docstring line |
-| statistics-degenerate-distribution-shift-112 | statistics | Degenerate dist: zero shift metrics & KS p=1.0 | statistics/test_statistics.py:87 | Docstring line |
-| statistics-constant-dist-widened-ci-113a | statistics | Non-strict: widened CI with warning | statistics/test_statistics.py:551 | Test docstring labels "Invariant 113 (non-strict)" |
-| statistics-constant-dist-strict-omit-113b | statistics | Strict: omit metrics (no widened CI) | statistics/test_statistics.py:583 | Test docstring labels "Invariant 113 (strict)" |
-| statistics-fallback-diagnostics-115 | statistics | Fallback diagnostics constant distribution (qq_r2=1.0 etc.) | statistics/test_statistics.py:191 | Docstring line |
-| robustness-exit-pnl-only-117 | robustness | Only exit actions have non-zero PnL | robustness/test_robustness.py:127 | Comment line |
-| pbrs-absence-shift-placeholder-118 | pbrs | Placeholder shift line present (absence displayed) | pbrs/test_pbrs.py:1523 | Ensures placeholder appears when shaping shift absent |
-| components-pbrs-breakdown-fields-119 | components | PBRS breakdown fields finite and mathematically aligned | components/test_reward_components.py:783 | Tests base_reward, pbrs_delta, invariance_correction fields and their alignment |
-| integration-pbrs-metrics-section-120 | integration | PBRS Metrics section present in report with tracing metrics | integration/test_report_formatting.py:155 | Verifies PBRS Metrics (Tracing) subsection rendering in statistical_analysis.md |
-| cli-pbrs-csv-columns-121 | cli | PBRS columns in reward_samples.csv when shaping enabled | cli/test_cli_params_and_csv.py:221 | Ensures reward_base, reward_pbrs_delta, reward_invariance_correction columns exist and contain finite values |
+| pbrs-canonical-exit-semantic-106 | pbrs | Canonical exit uses shaping=-prev_potential and next_potential=0.0 | pbrs/test_pbrs.py:374 | Uses stored potential across steps; no drift correction applied |
+| statistics-partial-deps-skip-107 | statistics | skip_partial_dependence => empty PD structures | statistics/test_statistics.py:42 | Docstring line |
+| helpers-transitions-preserve-multiplicity-108 | helpers | Repeated transitions retain their empirical multiplicity | helpers/test_utilities.py:26 | |
+| helpers-missing-cols-fill-109 | helpers | Missing required columns filled with NaN + single warning | helpers/test_utilities.py:51 | Docstring line |
+| statistics-binned-stats-min-edges-110 | statistics | <2 bin edges raises ValueError | statistics/test_statistics.py:60 | Docstring line |
+| statistics-constant-cols-exclusion-111 | statistics | Constant columns excluded & listed | statistics/test_statistics.py:71 | Docstring line |
+| statistics-degenerate-distribution-shift-112 | statistics | Constants: zero distances; KS p only with declared independent observations | statistics/test_statistics.py:87 | Docstring line |
+| statistics-constant-dist-exact-ci-113a | statistics | Both modes retain exact constant CI bounds | statistics/test_statistics.py:606 | |
+| statistics-percentile-outside-mean-113b | statistics | Percentile bounds need not contain the sample mean | statistics/test_statistics.py:620 | |
+| statistics-constant-diagnostics-115 | statistics | Constants have N/A higher moments, normality tests and Q-Q fits in both modes | statistics/test_statistics.py:194 | |
+| pbrs-canonical-near-zero-report-116 | pbrs | Canonical trajectories with valid evidence are classified as verified | pbrs/test_pbrs.py:1509 | Requires local identity, continuity, discounted terminal boundary, and zero observed additives; the non-owning boundary test also covers a complete singleton terminal episode |
+| robustness-exit-pnl-only-117 | robustness | Only exit actions have non-zero PnL | robustness/test_robustness.py:127 | Comment line |
+| pbrs-absence-shift-placeholder-118 | pbrs | Placeholder shift line present when shaping shift is absent | pbrs/test_pbrs.py:1877 | |
+| components-pbrs-breakdown-fields-119 | components | PBRS breakdown fields finite and mathematically aligned | components/test_reward_components.py:830 | Tests base_reward, pbrs_delta and invariance_correction alignment |
+| integration-pbrs-metrics-section-120 | integration | PBRS Metrics section present in report with tracing metrics | integration/test_report_formatting.py:137 | |
+| cli-pbrs-csv-columns-121 | cli | PBRS columns in reward_samples.csv when shaping enabled | cli/test_cli_params_and_csv.py:347 | Verifies finite reward_base, reward_pbrs_delta and reward_invariance_correction values |
+| statistics-bh-finite-family-122 | statistics | Undefined tests excluded from finite-only BH family; marked non-applicable | statistics/test_statistics.py:499 | |
+| statistics-independence-contract-123 | statistics | Inferential helpers require independent_observations=True | statistics/test_statistics.py:636 | Covers hypothesis tests and bootstrap intervals |
+| report-independent-sections-124 | integration | CI, diagnostics and shift sections do not depend on hypothesis-test output | integration/test_report_formatting.py:28 | Also verifies the reported bootstrap resample count |
+| pbrs-discounted-evidence-125 | pbrs | Verification requires local identity, continuity and discounted terminal boundary | pbrs/test_pbrs.py:1699 | Discontinuous potentials are not verified |
### Non-Owning Smoke / Reference Checks
| integration/test_reward_calculation.py | 44 | Decomposition identity (sum components) | robustness/test_robustness.py:43 |
| components/test_reward_components.py | 551 | Exit factor finiteness & plateau behavior | robustness/test_robustness.py:43+ |
| pbrs/test_pbrs.py | 1053 | Canonical vs non-canonical classification formatting | robustness/test_robustness.py:43, robustness/test_robustness.py:127 |
-| pbrs/test_pbrs.py | 1222,1292,1415 | Abs Σ Shaping Reward line formatting | integration/test_report_formatting.py:95 |
-| pbrs/test_pbrs.py | 1222 | Canonical near-zero cumulative shaping classification | robustness/test_robustness.py:43 |
-| pbrs/test_pbrs.py | 1292 | Canonical warning classification (Σ shaping > tolerance) | robustness/test_robustness.py:43 |
| pbrs/test_pbrs.py | 1415 | Non-canonical full report reason aggregation | robustness/test_robustness.py:43 |
| pbrs/test_pbrs.py | 1469 | Non-canonical mode-only reason (additives disabled) | robustness/test_robustness.py:43 |
| statistics/test_statistics.py | 292 | Mean decomposition consistency | robustness/test_robustness.py:43 |
## Coverage Parity Notes
Detailed assertions reside in targeted directories (components, robustness)
-while integration tests focus on report formatting. Ownership IDs (e.g.
-091–095, 106) reflect current scope (multi-path when noted).
+while integration tests focus on report formatting. The mapping above defines
+current ownership; multi-path and non-owning references are called out explicitly.
## When to Run Tests
examples for minimal, standard, and complex tests
- **`constants.py`** - Single source of truth for all test constants (frozen
dataclasses with comprehensive documentation)
-- **`helpers/assertions.py`** - 20+ custom assertion functions for invariant
- validation
+- **`helpers/assertions.py`** - Custom assertion helpers for invariant validation
- **`test_base.py`** - Base class with common utilities (`make_ctx`, `seed_all`,
etc.)
self.assertGreater(high_idle_rate_spot, low_idle_rate_spot)
def test_parse_overrides(self):
- """Test parse overrides."""
- overrides = ["alpha=1.5", "mode=linear", "limit=42"]
- result = parse_overrides(overrides)
- self.assertEqual(result["alpha"], 1.5)
- self.assertEqual(result["mode"], "linear")
- self.assertEqual(result["limit"], 42.0)
- with self.assertRaises(ValueError):
- parse_overrides(["badpair"])
+ """Overrides accept canonical keys and reject unsupported keys."""
+ result = parse_overrides(
+ ["win_reward_factor=4.0", "risk_reward_ratio=2.5", "exit_potential_mode=tanh"]
+ )
+ self.assertEqual(result["win_reward_factor"], 4.0)
+ self.assertEqual(result["risk_reward_ratio"], 2.5)
+ self.assertEqual(result["exit_potential_mode"], "tanh")
+ for invalid in (
+ ["rr=1.5"],
+ ["alpha=1.5"],
+ ["num_samples=1"],
+ ["unrealized_pnl=true"],
+ ["=5"],
+ ["badpair"],
+ ):
+ with self.subTest(invalid=invalid), self.assertRaises(ValueError):
+ parse_overrides(invalid)
def test_api_simulation_and_reward_smoke(self):
"""Test api simulation and reward smoke."""
breakdown = calculate_reward_with_defaults(ctx, self.DEFAULT_PARAMS)
self.assertFinite(breakdown.total)
+ def test_simulate_samples_singleton_terminal_neutral_transition(self):
+ """Return a valid terminal neutral row for a singleton simulation."""
+ df = simulate_samples_with_defaults(
+ self.base_params(),
+ num_samples=SCENARIOS.SAMPLE_SIZE_SINGLETON,
+ seed=SEEDS.BASE,
+ trading_mode="spot",
+ )
+
+ self.assertEqual(len(df), SCENARIOS.SAMPLE_SIZE_SINGLETON)
+ row = df.iloc[0]
+ self.assertTrue(row["terminated"])
+ self.assertFalse(row["terminal_liquidation"])
+ self.assertTrue(pd.isna(row["exit_pnl"]))
+ self.assertEqual(row["next_position"], Positions.Neutral.value)
+
def test_simulate_samples_trading_modes_spot_vs_margin(self):
"""simulate_samples coverage: spot should forbid shorts, margin should allow them."""
df_spot = simulate_samples_with_defaults(
self.assertIn("PBRS Invariance", content)
def test_strict_diagnostics_constant_distribution_succeeds(self):
- """Run with --strict_diagnostics and low num_samples; expect success, exercising assertion branches before graceful fallback paths."""
+ """Strict diagnostics accepts constant distributions without fabricated statistics."""
out_dir = self.output_path / "strict_diagnostics"
result = _run_cli(
out_dir=out_dir,
"--strict_diagnostics",
],
)
- # Should not raise; if constant distributions occur they should assert before graceful fallback paths, exercising assertion branches.
+ # Constant distributions remain valid in strict mode.
self.assertEqual(
result.returncode,
0,
int(rp["max_trade_duration_candles"]), SCENARIOS.CLI_MAX_TRADE_DURATION_PARAMS
)
+ def test_missing_real_episodes_fails_before_artifacts(self):
+ """An explicitly requested but missing episodes file fails the run with no artifacts."""
+ out_dir = self.output_path / "missing_real"
+ missing = self.output_path / "no_such_episodes.pkl"
+ result = _run_cli(
+ out_dir=out_dir,
+ args=["--num_samples", "50", "--real_episodes", str(missing)],
+ )
+ self.assertNotEqual(result.returncode, 0)
+ self.assertIn(str(missing), result.stderr + result.stdout)
+ self.assertFalse(out_dir.exists())
+
+ def test_invalid_real_episodes_pickle_fails_before_artifacts(self):
+ """A corrupt episodes pickle fails the run with a diagnosed path and no artifacts."""
+ out_dir = self.output_path / "invalid_real"
+ invalid = self.output_path / "corrupt.pkl"
+ invalid.write_bytes(b"not a pickle")
+ result = _run_cli(
+ out_dir=out_dir,
+ args=["--num_samples", "50", "--real_episodes", str(invalid)],
+ )
+ self.assertNotEqual(result.returncode, 0)
+ self.assertIn(str(invalid), result.stderr + result.stdout)
+ self.assertFalse(out_dir.exists())
+
+ def test_valid_real_episodes_produce_real_metrics(self):
+ """A valid episodes pickle loads before simulation and enables real metrics."""
+ out_dir = self.output_path / "valid_real"
+ import pickle
+
+ episodes = [
+ {
+ "transitions": [
+ {
+ "pnl": 0.01 * (1 if index % 2 else -1),
+ "trade_duration": 2 + index % 4,
+ "idle_duration": index % 5,
+ "position": 1.0 if index % 2 else 0.5,
+ "action": 0,
+ "reward": 0.5 - 0.05 * index,
+ }
+ for index in range(20)
+ ]
+ }
+ ]
+ episodes_path = self.output_path / "episodes.pkl"
+ with episodes_path.open("wb") as fh:
+ pickle.dump(episodes, fh)
+ result = _run_cli(
+ out_dir=out_dir,
+ args=["--num_samples", "50", "--real_episodes", str(episodes_path)],
+ )
+ _assert_cli_success(self, result)
+ report = (out_dir / "statistical_analysis.md").read_text(encoding="utf-8")
+ self.assertNotIn("Not performed (no real episodes provided)", report)
+
+ def test_params_override_flags_and_manifest_reflects_effective(self):
+ """--params beats explicit flags; manifest effective values follow resolution."""
+ out_dir = self.output_path / "params_beat_flags"
+ result = _run_cli(
+ out_dir=out_dir,
+ args=[
+ "--num_samples",
+ str(SCENARIOS.CLI_NUM_SAMPLES_FAST),
+ "--profit_aim",
+ "0.05",
+ "--base_factor",
+ "150.0",
+ "--params",
+ "profit_aim=0.02",
+ "risk_reward_ratio=1.5",
+ ],
+ )
+ _assert_cli_success(self, result)
+ with (out_dir / "manifest.json").open() as f:
+ manifest = json.load(f)
+ effective = manifest["effective"]
+ self.assertEqual(effective["profit_aim"], 0.02)
+ self.assertEqual(effective["risk_reward_ratio"], 1.5)
+ self.assertEqual(effective["base_factor"], 150.0)
+ self.assertAlmostEqual(manifest["pnl_target"], 0.03)
+
+ def test_simulation_only_params_rejected_before_artifacts(self):
+ """Simulation-only keys fail the run before any artifact is written."""
+ for key, value in (("num_samples", "1"), ("unrealized_pnl", "true")):
+ out_dir = self.output_path / f"rejected_{key}"
+ result = _run_cli(out_dir=out_dir, args=["--params", f"{key}={value}"])
+ self.assertNotEqual(result.returncode, 0)
+ self.assertFalse(out_dir.exists())
+
+ def test_unrealized_pnl_flag_changes_simulated_trajectory(self):
+ common_args = [
+ "--num_samples",
+ str(SCENARIOS.CLI_NUM_SAMPLES_STANDARD),
+ "--seed",
+ str(SEEDS.BASE),
+ "--skip_feature_analysis",
+ "--skip_partial_dependence",
+ ]
+ default_dir = self.output_path / "unrealized_default"
+ enabled_dir = self.output_path / "unrealized_enabled"
+ default_result = _run_cli(out_dir=default_dir, args=common_args)
+ enabled_result = _run_cli(out_dir=enabled_dir, args=[*common_args, "--unrealized_pnl"])
+ _assert_cli_success(self, default_result)
+ _assert_cli_success(self, enabled_result)
+ default_pnl = pd.read_csv(default_dir / "reward_samples.csv")["pnl"]
+ enabled_pnl = pd.read_csv(enabled_dir / "reward_samples.csv")["pnl"]
+ self.assertFalse(default_pnl.equals(enabled_pnl))
+
+ def test_inferential_options_rejected_for_dependent_trajectory(self):
+ for option, value in (
+ ("--bootstrap_resamples", "200"),
+ ("--pvalue_adjust", "benjamini_hochberg"),
+ ):
+ out_dir = self.output_path / option.removeprefix("--")
+ result = _run_cli(out_dir=out_dir, args=[option, value])
+ self.assertNotEqual(result.returncode, 0)
+ self.assertFalse(out_dir.exists())
+
+ def test_unknown_params_rejected_before_artifacts(self):
+ """Unknown keys fail the run before any artifact is written."""
+ out_dir = self.output_path / "rejected_unknown"
+ result = _run_cli(out_dir=out_dir, args=["--params", "win_reward_factr=2.0"])
+ self.assertNotEqual(result.returncode, 0)
+ self.assertFalse(out_dir.exists())
+
def test_max_trade_duration_candles_propagation_flag(self):
"""Dynamic flag --max_trade_duration_candles X propagates identically."""
out_dir = self.output_path / "mtd_flag"
}
)
base_reward = 0.05
+ # Entry additives consume fill-time PnL, not the next observation PnL.
+ entry_pnl = 0.012
ctx = {
"current_pnl": 0.01,
"pnl_target": PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
params=base,
base_factor=PARAMS.BASE_FACTOR,
prev_potential=0.0,
+ entry_pnl=entry_pnl,
**ctx,
)
t0 = base_reward + s0 + _entry0 + _exit0
params=with_add,
base_factor=PARAMS.BASE_FACTOR,
prev_potential=0.0,
+ entry_pnl=entry_pnl,
**ctx,
)
t1 = base_reward + s1 + _entry1 + _exit1
def test_pnl_target_coefficient_below_loss_threshold(self):
"""PnL target coefficient amplifies penalty for excessive losses.
- Validates that losses exceeding risk-adjusted threshold produce
- coefficient > 1.0 to amplify negative reward signal. Penalty applies
- when BOTH conditions met: abs(pnl_ratio) > 1.0 AND pnl_ratio < -(1/rr).
+ Validates that losses exceeding the risk-adjusted threshold
+ (pnl_target / risk_reward_ratio) produce coefficient > 1.0 to
+ amplify the negative reward signal, with the loss progression
+ normalized by that threshold.
**Setup:**
- PnL: -0.06 (exceeds pnl_target magnitude)
- pnl_target: 0.045 (profit_aim=0.03 * risk_reward_ratio=1.5)
- - Penalty threshold: pnl < -pnl_target = -0.045
+ - Penalty threshold: pnl < -pnl_target / 1.5 = -0.03
- Parameters: win_reward_factor=2.0, pnl_amplification_sensitivity=0.5
**Assertions:**
coefficient, 1.0, "Excessive loss should amplify penalty with coefficient > 1.0"
)
+ def test_pnl_target_coefficient_amplifies_between_gain_and_loss_thresholds(self):
+ """Losses past the risk threshold amplify before reaching the gain target magnitude.
+
+ With profit_aim=0.03 and risk_reward_ratio=1.5: target=0.045,
+ risk threshold=0.03. A -0.04 loss sits between both thresholds and
+ must already be amplified; the coefficient stays continuous (1.0) at
+ the exact threshold and grows with loss depth.
+ """
+ params = self.base_params(win_reward_factor=2.0, pnl_amplification_sensitivity=0.5)
+ profit_aim = 0.03
+ risk_reward_ratio = 1.5
+ pnl_target = profit_aim * risk_reward_ratio
+ threshold = pnl_target / risk_reward_ratio
+
+ at_threshold = _compute_pnl_target_coefficient(
+ params, pnl=-threshold, pnl_target=pnl_target, risk_reward_ratio=risk_reward_ratio
+ )
+ between = _compute_pnl_target_coefficient(
+ params,
+ pnl=-(threshold * 1.2),
+ pnl_target=pnl_target,
+ risk_reward_ratio=risk_reward_ratio,
+ )
+ deep = _compute_pnl_target_coefficient(
+ params, pnl=-(threshold * 4), pnl_target=pnl_target, risk_reward_ratio=risk_reward_ratio
+ )
+ inside = _compute_pnl_target_coefficient(
+ params,
+ pnl=-(threshold * 0.5),
+ pnl_target=pnl_target,
+ risk_reward_ratio=risk_reward_ratio,
+ )
+
+ self.assertAlmostEqualFloat(at_threshold, 1.0, tolerance=TOLERANCE.GENERIC_EQ)
+ self.assertGreater(between, 1.0)
+ self.assertGreater(deep, between)
+ self.assertAlmostEqualFloat(inside, 1.0, tolerance=TOLERANCE.GENERIC_EQ)
+
def test_efficiency_coefficient_zero_weight(self):
"""Efficiency coefficient returns neutral value when efficiency disabled.
msg="invariance_correction should be ~0 in canonical mode",
)
- def test_rr_alias_matches_risk_reward_ratio(self):
- """`rr` param alias matches `risk_reward_ratio` runtime naming."""
- context = self.make_ctx(
- pnl=0.02,
- trade_duration=40,
- idle_duration=0,
- max_unrealized_profit=0.03,
- min_unrealized_profit=0.01,
- position=Positions.Long,
- action=Actions.Long_exit,
- )
- rr_value = 1.75
-
- # Canonical spelling
- params_ratio = self.base_params(
- exit_potential_mode="canonical",
- risk_reward_ratio=rr_value,
- )
- params_ratio.pop("rr", None)
-
- # Runtime spelling
- params_rr = self.base_params(
- exit_potential_mode="canonical",
- rr=rr_value,
- )
- params_rr.pop("risk_reward_ratio", None)
-
- br_ratio = calculate_reward_with_defaults(
- context, params_ratio, risk_reward_ratio=PARAMS.RISK_REWARD_RATIO
- )
- br_rr = calculate_reward_with_defaults(
- context, params_rr, risk_reward_ratio=PARAMS.RISK_REWARD_RATIO
- )
-
- self.assertAlmostEqualFloat(
- br_rr.total,
- br_ratio.total,
- tolerance=TOLERANCE.IDENTITY_STRICT,
- msg="Total reward should match when using rr alias",
- )
- self.assertAlmostEqualFloat(
- br_rr.exit_component,
- br_ratio.exit_component,
- tolerance=TOLERANCE.IDENTITY_STRICT,
- msg="Exit component should match when using rr alias",
- )
-
if __name__ == "__main__":
unittest.main()
CANONICAL_SWEEP: Seed for canonical sweep tests (123)
# PBRS-specific seeds
- PBRS_INVARIANCE_1: Seed for PBRS invariance test case 1 (913)
- PBRS_INVARIANCE_2: Seed for PBRS invariance test case 2 (515)
PBRS_TERMINAL: Seed for PBRS terminal potential tests (777)
# Feature analysis failure seeds
CANONICAL_SWEEP: int = 123
# PBRS-specific seeds
- PBRS_INVARIANCE_1: int = 913
- PBRS_INVARIANCE_2: int = 515
PBRS_TERMINAL: int = 777
# Feature analysis failure seeds
PNL_SMALL: Small profit/loss value (0.02)
PNL_SHORT_PROFIT: Short profit/loss value (0.03)
PNL_MEDIUM: Medium profit/loss value (0.05)
- PNL_LARGE: Large profit/loss value (0.10)
# Common duration values
TRADE_DURATION_SHORT: Short trade duration in steps (50)
ADDITIVE_RATIO_DEFAULT: Default additive ratio (0.0625)
ADDITIVE_GAIN_DEFAULT: Default additive gain (1.0)
- # PBRS hold potential parameters
- HOLD_POTENTIAL_RATIO_DEFAULT: Default hold potential ratio (0.001)
"""
BASE_FACTOR: float = 90.0
PNL_SMALL: float = 0.02
PNL_SHORT_PROFIT: float = 0.03
PNL_MEDIUM: float = 0.05
- PNL_LARGE: float = 0.10
# Common duration values
TRADE_DURATION_SHORT: int = 50
ADDITIVE_RATIO_DEFAULT: float = 0.0625
ADDITIVE_GAIN_DEFAULT: float = 1.0
- # PBRS hold potential parameters
- HOLD_POTENTIAL_RATIO_DEFAULT: float = 0.001
-
@dataclass(frozen=True)
class TestScenarios:
DURATION_MEDIUM: Medium duration scenario (200)
DURATION_LONG: Long duration scenario (300)
DURATION_SCENARIOS: Standard duration test sequence
+ SAMPLE_SIZE_SINGLETON: Single-row sample size for boundary tests (1)
SAMPLE_SIZE_TINY: Tiny sample size for smoke tests (50)
SAMPLE_SIZE_SMALL: Small sample size for quick tests (100)
SAMPLE_SIZE_MEDIUM: Medium sample size for standard tests (400)
DURATION_SHORT: int = 150
DURATION_MEDIUM: int = 200
DURATION_LONG: int = 300
- DURATION_SCENARIOS: tuple[int, ...] = (150, 200, 300)
+ DURATION_SCENARIOS: tuple[int, ...] = (DURATION_SHORT, DURATION_MEDIUM, DURATION_LONG)
+ SAMPLE_SIZE_SINGLETON: int = 1
SAMPLE_SIZE_TINY: int = 50
SAMPLE_SIZE_SMALL: int = 100
SAMPLE_SIZE_MEDIUM: int = 400
KS_STATISTIC_IDENTITY: KS statistic threshold for identical distributions (5e-3)
CORRELATION_SIGNIFICANCE: Minimum correlation for significance (0.1)
VARIANCE_RATIO_THRESHOLD: Minimum variance ratio for heteroscedasticity (0.8)
- CI_WIDTH_EPSILON: Minimum CI width for degenerate distributions (3e-9)
"""
DISTRIBUTION_SHIFT: float = 5e-4
KS_STATISTIC_IDENTITY: float = 5e-3
CORRELATION_SIGNIFICANCE: float = 0.1
VARIANCE_RATIO_THRESHOLD: float = 0.8
- CI_WIDTH_EPSILON: float = 3e-9
# Global singleton instances for easy import
-"""Helpers package for reward_space_analysis tests.
-
-Exposes shared assertion utilities, configuration dataclasses, and warning
-capture helpers, centralizing test infrastructure and reducing duplication.
-"""
+"""Helpers package for reward_space_analysis tests."""
from .assertions import (
- assert_adjustment_reason_contains,
- assert_almost_equal_list,
assert_component_sum_integrity,
assert_exit_factor_attenuation_modes,
assert_exit_factor_invariant_suite,
- assert_exit_factor_kernel_fallback,
assert_exit_factor_plateau_behavior,
assert_exit_mode_mathematical_validation,
- assert_finite,
assert_hold_penalty_threshold_behavior,
- assert_monotonic_nonincreasing,
- assert_monotonic_nonnegative,
- assert_multi_parameter_sensitivity,
- assert_non_canonical_shaping_exceeds,
- assert_parameter_sensitivity_behavior,
- assert_pbrs_canonical_sum_within_tolerance,
assert_pbrs_invariance_report_classification,
assert_progressive_scaling_behavior,
assert_relaxed_multi_reason_aggregation,
assert_reward_calculation_scenarios,
assert_single_active_component,
assert_single_active_component_with_additives,
- assert_trend,
build_validation_case,
calculate_reward_with_defaults,
execute_validation_batch,
- get_exit_factor_with_defaults,
make_idle_penalty_test_contexts,
run_relaxed_validation_adjustment_cases,
run_strict_validation_failure_cases,
- safe_float,
simulate_samples_with_defaults,
)
from .configs import (
ValidationConfig,
WarningCaptureConfig,
)
-from .warnings import (
- assert_diagnostic_warning,
- assert_no_warnings,
- capture_warnings,
- validate_warning_content,
-)
+from .warnings import assert_diagnostic_warning, capture_warnings
__all__ = [
"DEFAULT_REWARD_CONFIG",
"ValidationCallback",
"ValidationConfig",
"WarningCaptureConfig",
- "assert_adjustment_reason_contains",
- "assert_almost_equal_list",
"assert_component_sum_integrity",
"assert_diagnostic_warning",
"assert_exit_factor_attenuation_modes",
"assert_exit_factor_invariant_suite",
- "assert_exit_factor_kernel_fallback",
"assert_exit_factor_plateau_behavior",
"assert_exit_mode_mathematical_validation",
- "assert_finite",
"assert_hold_penalty_threshold_behavior",
- "assert_monotonic_nonincreasing",
- "assert_monotonic_nonnegative",
- "assert_multi_parameter_sensitivity",
- "assert_no_warnings",
- "assert_non_canonical_shaping_exceeds",
- "assert_parameter_sensitivity_behavior",
- "assert_pbrs_canonical_sum_within_tolerance",
"assert_pbrs_invariance_report_classification",
"assert_progressive_scaling_behavior",
"assert_relaxed_multi_reason_aggregation",
"assert_reward_calculation_scenarios",
"assert_single_active_component",
"assert_single_active_component_with_additives",
- "assert_trend",
"build_validation_case",
"calculate_reward_with_defaults",
"capture_warnings",
"execute_validation_batch",
- "get_exit_factor_with_defaults",
"make_idle_penalty_test_contexts",
"run_relaxed_validation_adjustment_cases",
"run_strict_validation_failure_cases",
- "safe_float",
"simulate_samples_with_defaults",
- "validate_warning_content",
]
calculate_reward,
)
-from ..constants import PARAMS, TOLERANCE
+from ..constants import PARAMS
from .configs import (
DEFAULT_REWARD_CONFIG,
DEFAULT_SIMULATION_CONFIG,
)
-def safe_float(value: Any, default: float = 0.0) -> float:
- """Coerce value to float safely for test parameter handling.
-
- Rules:
- - None, '' -> default
- - Numeric types pass through
- - String numeric forms ('3', '3.5', 'nan', 'inf') handled; nan/inf return default
- - Non-numeric strings return default
- Avoids direct float(...) exceptions leaking into tests that target relaxed validation behaviors.
- """
- try:
- if value is None or value == "":
- return default
- coerced = float(value)
- if coerced != coerced or coerced in (float("inf"), float("-inf")):
- return default
- return coerced
- except (TypeError, ValueError):
- return default
-
-
-def assert_monotonic_nonincreasing(
- test_case,
- values: Sequence[float],
- tolerance: float = 0.0,
- msg: str = "Values should be non-increasing",
-):
- """Assert that a sequence is monotonically non-increasing.
-
- Validates that each element in the sequence is less than or equal to the
- previous element, with an optional tolerance for floating-point comparisons.
-
- Args:
- test_case: Test case instance with assertion methods
- values: Sequence of numeric values to validate
- tolerance: Numerical tolerance for comparisons (default: 0.0)
- msg: Custom error message for assertion failures
-
- Example:
- assert_monotonic_nonincreasing(self, [5.0, 4.0, 3.0, 3.0, 2.0])
- # Validates: 4.0 <= 5.0, 3.0 <= 4.0, 3.0 <= 3.0, 2.0 <= 3.0
- """
- for i in range(1, len(values)):
- test_case.assertLessEqual(values[i], values[i - 1] + tolerance, msg)
-
-
-def assert_monotonic_nonnegative(
- test_case,
- values: Sequence[float],
- tolerance: float = 0.0,
- msg: str = "Values should be non-negative",
-):
- """Assert that all values in a sequence are non-negative.
-
- Validates that each element is greater than or equal to zero, with an
- optional tolerance for floating-point comparisons.
-
- Args:
- test_case: Test case instance with assertion methods
- values: Sequence of numeric values to validate
- tolerance: Numerical tolerance for comparisons (default: 0.0)
- msg: Custom error message for assertion failures
-
- Example:
- assert_monotonic_nonnegative(self, [0.0, 1.5, 2.3, 0.1])
- """
- for v in values:
- test_case.assertGreaterEqual(v + tolerance, 0.0, msg)
-
-
-def assert_finite(test_case, values: Sequence[float], msg: str = "Values must be finite"):
- """Assert that all values are finite (not NaN or infinity).
-
- Validates that no element in the sequence is NaN, positive infinity, or
- negative infinity. Essential for numerical stability checks.
-
- Args:
- test_case: Test case instance with assertion methods
- values: Sequence of numeric values to validate
- msg: Custom error message for assertion failures
-
- Example:
- assert_finite(self, [1.0, 2.5, -3.7, 0.0]) # Passes
- assert_finite(self, [1.0, float('nan')]) # Fails
- assert_finite(self, [1.0, float('inf')]) # Fails
- """
- for v in values:
- test_case.assertTrue((v == v) and (v not in (float("inf"), float("-inf"))), msg)
-
-
-def assert_almost_equal_list(
- test_case,
- values: Sequence[float],
- target: float,
- delta: float,
- msg: str = "Values should be near target",
-):
- """Assert that all values in a sequence are approximately equal to a target.
-
- Validates that each element is within a specified tolerance (delta) of the
- target value. Useful for checking plateau behavior or constant outputs.
-
- Args:
- test_case: Test case instance with assertion methods
- values: Sequence of numeric values to validate
- target: Target value for comparison
- delta: Maximum allowed deviation from target
- msg: Custom error message for assertion failures
-
- Example:
- assert_almost_equal_list(self, [1.0, 1.01, 0.99], 1.0, delta=0.02)
- """
- for v in values:
- test_case.assertAlmostEqual(v, target, delta=delta, msg=msg)
-
-
-def assert_trend(
- test_case,
- values: Sequence[float],
- trend: str,
- tolerance: float,
- msg_prefix: str = "Trend validation failed",
-):
- """Assert that a sequence follows a specific trend pattern.
-
- Generic trend validation supporting increasing, decreasing, or constant
- patterns. More flexible than specialized monotonic assertions.
-
- Args:
- test_case: Test case instance with assertion methods
- values: Sequence of numeric values to validate
- trend: Expected trend: "increasing", "decreasing", or "constant"
- tolerance: Numerical tolerance for comparisons
- msg_prefix: Prefix for error messages
-
- Raises:
- ValueError: If trend parameter is not one of the supported values
-
- Example:
- assert_trend(self, [1.0, 2.0, 3.0], "increasing", 1e-09)
- assert_trend(self, [5.0, 5.0, 5.0], "constant", 1e-09)
- """
- if trend not in {"increasing", "decreasing", "constant"}:
- raise ValueError(f"Unsupported trend '{trend}'")
- if trend == "increasing":
- for i in range(1, len(values)):
- test_case.assertGreaterEqual(
- values[i], values[i - 1] - tolerance, f"{msg_prefix}: expected increasing"
- )
- elif trend == "decreasing":
- for i in range(1, len(values)):
- test_case.assertLessEqual(
- values[i], values[i - 1] + tolerance, f"{msg_prefix}: expected decreasing"
- )
- else: # constant
- base = values[0]
- for v in values[1:]:
- test_case.assertAlmostEqual(
- v, base, delta=tolerance, msg=f"{msg_prefix}: expected constant"
- )
-
-
def assert_component_sum_integrity(
test_case,
breakdown,
validation_fn(test_case, breakdown, description, config.tolerance_relaxed)
-def assert_parameter_sensitivity_behavior(
- test_case,
- parameter_variations: list[dict[str, Any]],
- base_context,
- base_params: dict[str, Any],
- component_name: str,
- expected_trend: str,
- config: RewardScenarioConfig,
-):
- """Validate that a component responds predictably to parameter changes.
-
- Tests component sensitivity by applying parameter variations and verifying
- the component value follows the expected trend (increasing, decreasing, or constant).
- Uses RewardScenarioConfig to simplify parameter passing.
-
- Args:
- test_case: Test case instance with assertion methods
- parameter_variations: List of parameter dicts to merge with base_params
- base_context: Context object for reward calculation
- base_params: Base parameter dictionary
- component_name: Name of component to track (e.g., "exit_component")
- expected_trend: Expected trend: "increasing", "decreasing", or "constant"
- config: RewardScenarioConfig with calculation parameters
-
- Example:
- config = RewardScenarioConfig(
- base_factor=PARAMS.BASE_FACTOR,
- profit_aim=PARAMS.PROFIT_AIM,
- risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
- tolerance_relaxed=TOLERANCE.IDENTITY_RELAXED
- )
- variations = [
- {"exit_additive": 0.0},
- {"exit_additive": 5.0},
- {"exit_additive": 10.0},
- ]
- assert_parameter_sensitivity_behavior(
- self, variations, ctx, params, "exit_component", "increasing", config
- )
- """
- results = []
- for param_variation in parameter_variations:
- params = base_params.copy()
- params.update(param_variation)
- breakdown = calculate_reward(
- base_context,
- params,
- base_factor=config.base_factor,
- profit_aim=config.profit_aim,
- risk_reward_ratio=config.risk_reward_ratio,
- short_allowed=config.short_allowed,
- action_masking=config.action_masking,
- )
- component_value = getattr(breakdown, component_name)
- results.append(component_value)
- if expected_trend == "increasing":
- for i in range(1, len(results)):
- test_case.assertGreaterEqual(
- results[i],
- results[i - 1] - config.tolerance_relaxed,
- f"{component_name} should increase with parameter variations",
- )
- elif expected_trend == "decreasing":
- for i in range(1, len(results)):
- test_case.assertLessEqual(
- results[i],
- results[i - 1] + config.tolerance_relaxed,
- f"{component_name} should decrease with parameter variations",
- )
- elif expected_trend == "constant":
- baseline = results[0]
- for result in results[1:]:
- test_case.assertAlmostEqual(
- result,
- baseline,
- delta=config.tolerance_relaxed,
- msg=f"{component_name} should remain constant with parameter variations",
- )
-
-
def make_idle_penalty_test_contexts(
context_factory_fn,
idle_duration_scenarios: Sequence[int],
test_case.assertGreater(len(unique_rewards), 1)
-def assert_multi_parameter_sensitivity(
- test_case,
- parameter_test_cases: list[tuple[float, float, str]],
- context_factory_fn,
- base_params: dict[str, Any],
- config: RewardScenarioConfig,
-):
- """Validate reward behavior across multiple parameter combinations.
-
- Tests reward calculation with various profit_aim and risk_reward_ratio
- combinations, ensuring consistent behavior including edge cases like
- zero profit_aim. Uses RewardScenarioConfig to simplify parameter passing.
-
- Args:
- test_case: Test case instance with assertion methods
- parameter_test_cases: List of (profit_aim, risk_reward_ratio, description) tuples
- context_factory_fn: Factory function for creating context objects
- base_params: Base parameter dictionary
- config: RewardScenarioConfig with base calculation parameters
-
- Example:
- config = RewardScenarioConfig(
- base_factor=PARAMS.BASE_FACTOR,
- profit_aim=PARAMS.PROFIT_AIM,
- risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
- tolerance_relaxed=TOLERANCE.IDENTITY_RELAXED
- )
- test_cases = [
- (0.0, PARAMS.RISK_REWARD_RATIO, "zero profit target"),
- (PARAMS.PROFIT_AIM, PARAMS.RISK_REWARD_RATIO, "standard parameters"),
- (0.03, 2.0, "high risk/reward ratio"),
- ]
- assert_multi_parameter_sensitivity(
- self, test_cases, make_context, params, config
- )
- """
- for profit_aim, risk_reward_ratio, description in parameter_test_cases:
- with test_case.subTest(
- profit_aim=profit_aim, risk_reward_ratio=risk_reward_ratio, desc=description
- ):
- idle_context = context_factory_fn(context_type="idle")
- breakdown = calculate_reward(
- idle_context,
- base_params,
- base_factor=config.base_factor,
- profit_aim=profit_aim,
- risk_reward_ratio=risk_reward_ratio,
- short_allowed=config.short_allowed,
- action_masking=config.action_masking,
- )
- if profit_aim == 0.0:
- test_case.assertEqual(breakdown.idle_penalty, 0.0)
- test_case.assertEqual(breakdown.total, 0.0)
- else:
- test_case.assertLess(breakdown.idle_penalty, 0.0)
- if profit_aim > 0:
- exit_context = context_factory_fn(context_type="exit", profit_aim=profit_aim)
- exit_breakdown = calculate_reward(
- exit_context,
- base_params,
- base_factor=config.base_factor,
- profit_aim=profit_aim,
- risk_reward_ratio=risk_reward_ratio,
- short_allowed=config.short_allowed,
- action_masking=config.action_masking,
- )
- test_case.assertNotEqual(exit_breakdown.exit_component, 0.0)
-
-
def assert_hold_penalty_threshold_behavior(
test_case,
context_factory_fn,
test_case.assertIsInstance(sanitized, dict)
-def assert_adjustment_reason_contains(
- test_case, adjustments: dict[str, dict[str, Any]], key: str, expected_substrings: Sequence[str]
-):
- """Assert adjustment reason contains all expected substrings.
-
- Validates that all expected substrings appear in the adjustment reason
- message for a specific parameter key, regardless of order.
-
- Args:
- test_case: Test case instance with assertion methods
- adjustments: Dictionary of adjustment information from validation
- key: Parameter key to check in adjustments dict
- expected_substrings: List of substrings that must appear in reason
-
- Example:
- adjustments = {
- "exit_plateau_grace": {
- "reason": "clamped to valid range [0.0, 1.0]",
- "validation_mode": "relaxed"
- }
- }
- assert_adjustment_reason_contains(
- self, adjustments, "exit_plateau_grace", ["clamped", "valid range"]
- )
- """
- test_case.assertIn(key, adjustments, f"Adjustment key '{key}' missing")
- reason = adjustments[key].get("reason", "")
- for sub in expected_substrings:
- test_case.assertIn(sub, reason, f"Missing substring '{sub}' in reason for key '{key}'")
-
-
def run_strict_validation_failure_cases(
test_case, failure_params_list: Sequence[dict[str, Any]], validate_fn
):
test_case.fail(f"Unknown expectation '{exp}' in exit factor suite case")
-def assert_exit_factor_kernel_fallback(
- test_case,
- exit_factor_fn,
- base_factor: float,
- pnl: float,
- pnl_target: float,
- duration_ratio: float,
- context,
- bad_params: dict[str, Any],
- reference_params: dict[str, Any],
- risk_reward_ratio: float,
-):
- """Validate exit factor fallback behavior on kernel failure.
-
- Tests that when an attenuation kernel fails (e.g., invalid parameters),
- the system falls back to linear mode and produces numerically equivalent
- results. Caller must monkeypatch the kernel to trigger failure before calling.
-
- Args:
- test_case: Test case instance with assertion methods
- exit_factor_fn: Exit factor calculation function (e.g., _get_exit_factor)
- base_factor: Base scaling factor
- pnl: Realized profit/loss
- pnl_target: Target PnL (profit_aim * risk_reward_ratio)
- duration_ratio: Duration ratio
- context: RewardContext instance
- bad_params: Parameters that trigger kernel failure
- reference_params: Reference linear mode parameters for comparison
- risk_reward_ratio: Risk/reward ratio
-
- Validates:
- 1. Fallback produces non-negative result
- 2. Fallback result matches linear reference within tight tolerance (1e-12)
-
- Note:
- Warning emission should be validated separately with warning context managers.
-
- Example:
- # After monkeypatching kernel to fail:
- test_context = make_ctx(pnl=0.08, ...)
- assert_exit_factor_kernel_fallback(
- self, _get_exit_factor, 90.0, 0.08, 0.03, 0.5, test_context,
- bad_params={"exit_attenuation_mode": "power", "exit_power_tau": -1.0},
- reference_params={"exit_attenuation_mode": "linear"},
- risk_reward_ratio=PARAMS.RISK_REWARD_RATIO
- )
- """
-
- f_bad = exit_factor_fn(
- base_factor, pnl, pnl_target, duration_ratio, context, bad_params, risk_reward_ratio
- )
- f_ref = exit_factor_fn(
- base_factor, pnl, pnl_target, duration_ratio, context, reference_params, risk_reward_ratio
- )
- test_case.assertAlmostEqual(f_bad, f_ref, delta=TOLERANCE.IDENTITY_STRICT)
- test_case.assertGreaterEqual(f_bad, 0.0)
-
-
def assert_relaxed_multi_reason_aggregation(
test_case,
validate_fn,
Args:
test_case: Test case instance with assertion methods
content: Report content string to validate
- expected_status: Expected classification: "Canonical",
- "Canonical (with warning)", or "Non-canonical"
+ expected_status: Expected classification: "Canonical: observed PBRS verified",
+ "Not verified", or "Non-canonical: not verified"
expect_additives: Whether additive components should be mentioned
Example:
)
if expect_additives:
test_case.assertRegex(
- content, r"additives=\['entry', 'exit'\]|additives=\['exit', 'entry'\]"
+ content,
+ r"\| Entry Additive Effective \| True \|\n\| Exit Additive Effective \| True \|",
)
else:
- test_case.assertNotRegex(content, r"additives=\[")
-
-
-def assert_pbrs_canonical_sum_within_tolerance(test_case, total_shaping: float, tolerance: float):
- """Validate cumulative shaping is small.
-
- In canonical PBRS, the per-step shaping corresponds to a telescoping term.
- Over a full, closed episode it may cancel, but across many partial trajectories
- or with resets/discounting it does not need to be exactly zero.
-
- This helper remains as a *diagnostic* check for constructed test cases that
- intentionally enforce small cumulative shaping.
-
- Args:
- test_case: Test case instance with assertion methods.
- total_shaping: Total cumulative shaping value.
- tolerance: Maximum allowed absolute deviation from zero.
-
- Example:
- assert_pbrs_canonical_sum_within_tolerance(self, 5e-10, 1e-9)
- """
- test_case.assertLess(abs(total_shaping), tolerance)
-
-
-def assert_non_canonical_shaping_exceeds(
- test_case, total_shaping: float, tolerance_multiple: float
-):
- """Validate non-trivial shaping magnitude.
-
- In non-canonical PBRS modes or when additives are effective, the shaping
- trajectory is expected to deviate from the pure telescoping term more often.
-
- Note: cumulative shaping being large is not a strict correctness proof; it is
- a useful smoke-signal for test fixtures that intentionally construct such cases.
-
- Args:
- test_case: Test case instance with assertion methods.
- total_shaping: Total cumulative shaping value.
- tolerance_multiple: Threshold value for the given test fixture.
- """
- test_case.assertGreater(abs(total_shaping), tolerance_multiple)
+ test_case.assertNotRegex(content, r"\| (?:Entry|Exit) Additive Effective \| True \|")
def assert_exit_factor_plateau_behavior(
)
-def get_exit_factor_with_defaults(
- pnl: float,
- duration_ratio: float,
- context,
- params: dict[str, Any],
- base_factor: float | None = None,
- pnl_target: float | None = None,
- risk_reward_ratio: float | None = None,
-):
- """Calculate exit factor with standard test defaults.
-
- Reduces boilerplate by providing sensible defaults for common parameters.
- This wrapper is particularly useful for tests that need to call _get_exit_factor
- repeatedly with varying pnl and duration_ratio values.
-
- Args:
- pnl: Realized profit/loss
- duration_ratio: Ratio of current to maximum duration
- context: RewardContext for efficiency coefficient calculation
- params: Parameter dictionary
- base_factor: Base scaling factor (defaults to PARAMS.BASE_FACTOR)
- pnl_target: Target profit threshold (defaults to PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO)
- risk_reward_ratio: Risk/reward ratio (defaults to PARAMS.RISK_REWARD_RATIO)
-
- Returns:
- Exit factor value from _get_exit_factor()
-
- Example:
- # Using all defaults
- factor = get_exit_factor_with_defaults(0.05, 0.5, ctx, params)
-
- # Overriding specific parameters
- factor = get_exit_factor_with_defaults(
- 0.05, 0.5, ctx, params, base_factor=PARAMS.BASE_FACTOR
- )
- """
- if base_factor is None:
- base_factor = PARAMS.BASE_FACTOR
- if risk_reward_ratio is None:
- risk_reward_ratio = PARAMS.RISK_REWARD_RATIO
- if pnl_target is None:
- pnl_target = PARAMS.PROFIT_AIM * risk_reward_ratio
-
- return _get_exit_factor(
- base_factor,
- pnl,
- pnl_target,
- duration_ratio,
- context,
- params,
- risk_reward_ratio,
- )
-
-
def simulate_samples_with_defaults(
params: dict[str, Any],
config: SimulationConfig | None = None,
-import math
-
-import numpy as np
-
-from reward_space_analysis import (
- Actions,
- Positions,
- RewardParams,
- _get_bool_param,
-)
-
-from ..constants import PARAMS
-from ..test_base import make_ctx
-from . import calculate_reward_with_defaults
+from reward_space_analysis import RewardParams, _get_bool_param
def test_get_bool_param_none_and_invalid_literal():
params_invalid: RewardParams = {"check_invariants": "not_a_bool"}
# Invalid literal triggers ValueError in _to_bool; fallback returns default (True)
assert _get_bool_param(params_invalid, "check_invariants", True) is True
-
-
-def test_calculate_reward_unrealized_pnl_hold_path():
- """Verify unrealized PnL branch activates during hold action.
-
- Tests that when hold_potential_enabled and unrealized_pnl are both True,
- the reward calculation uses max/min unrealized profit to compute next_pnl
- via the tanh transformation path.
-
- **Setup:**
- - Position: Long, Action: Neutral (hold)
- - PnL: 0.01, max_unrealized_profit: 0.02, min_unrealized_profit: -0.01
- - Parameters: hold_potential_enabled=True, unrealized_pnl=True
- - Trade duration: 5 steps
-
- **Assertions:**
- - Both prev_potential and next_potential are finite
- - At least one potential is non-zero (shaping should activate)
- """
- # Exercise unrealized_pnl branch during hold to cover next_pnl tanh path
- context = make_ctx(
- pnl=0.01,
- trade_duration=5,
- idle_duration=0,
- max_unrealized_profit=0.02,
- min_unrealized_profit=-0.01,
- position=Positions.Long,
- action=Actions.Neutral,
- )
- params = {
- "hold_potential_enabled": True,
- "unrealized_pnl": True,
- "pnl_amplification_sensitivity": 0.5,
- }
- breakdown = calculate_reward_with_defaults(
- context,
- params,
- base_factor=100.0,
- profit_aim=0.05,
- risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
- prev_potential=np.nan,
- )
- assert math.isfinite(breakdown.prev_potential)
- assert math.isfinite(breakdown.next_potential)
- # shaping should activate (non-zero or zero after potential difference)
- assert breakdown.prev_potential != 0.0 or breakdown.next_potential != 0.0
class TestLoadRealEpisodes(RewardSpaceTestBase):
"""Unit tests for load_real_episodes."""
- def test_drop_exact_duplicates_warns(self):
- """Invariant 108: duplicate rows dropped with warning showing count removed."""
+ def test_repeated_transitions_preserve_multiplicity(self):
+ """Repeated values can be distinct transitions and must retain empirical weight."""
df = pd.DataFrame(
{
"pnl": [0.01, 0.01, -0.02], # first two duplicate
)
p = Path(self.temp_dir) / "dupes.pkl"
self.write_pickle(df, p)
- with warnings.catch_warnings(record=True) as w:
- warnings.simplefilter("always")
- loaded = load_real_episodes(p)
- self.assertEqual(len(loaded), 2, "Expected duplicate row removal to reduce length")
- msgs = [str(warning.message) for warning in w]
- dup_msgs = [m for m in msgs if "duplicate" in m.lower()]
- self.assertTrue(
- any("dropped" in m for m in dup_msgs), f"No duplicate removal warning found in: {msgs}"
- )
+ loaded = load_real_episodes(p)
+ pd.testing.assert_frame_equal(loaded[df.columns], df)
def test_missing_multiple_required_columns_single_warning(self):
"""Invariant 109: enforce_columns=False fills all missing required cols with NaN and single warning."""
import warnings
from contextlib import contextmanager
-from typing import Any
import reward_space_analysis
)
-@contextmanager
-def assert_no_warnings(warning_category: type[Warning] = Warning):
- """Context manager that asserts no warnings are raised.
-
- Useful for validating that clean code paths don't emit unexpected warnings.
-
- Args:
- warning_category: Warning category to check (default: all warnings)
-
- Yields:
- None
-
- Raises:
- AssertionError: If any warnings of the specified category are captured
-
- Example:
- with assert_no_warnings(RewardDiagnosticsWarning):
- result = function_that_should_not_warn()
- """
- with warnings.catch_warnings(record=True) as caught:
- warnings.simplefilter("always", warning_category)
- yield
-
- filtered = [w for w in caught if issubclass(w.category, warning_category)]
- if filtered:
- messages = [str(w.message) for w in filtered]
- raise AssertionError(
- f"Expected no {warning_category.__name__} but {len(filtered)} were raised: {messages}"
- )
-
-
-def validate_warning_content(
- caught_warnings: list[Any],
- warning_category: type[Warning],
- expected_substrings: list[str],
- strict_mode: bool = True,
-) -> None:
- """Validate captured warnings contain expected content.
-
- Helper function for manual validation of warning content when using
- a standard catch_warnings context.
-
- Args:
- caught_warnings: List of captured warning objects from catch_warnings
- warning_category: Expected warning category
- expected_substrings: List of substrings that should appear in messages
- strict_mode: If True, all substrings must be present; if False, at least one
-
- Raises:
- AssertionError: If validation fails
- """
- filtered = [w for w in caught_warnings if issubclass(w.category, warning_category)]
-
- if not filtered:
- raise AssertionError(
- f"No warnings of type {warning_category.__name__} captured. "
- f"Total warnings: {len(caught_warnings)}"
- )
-
- all_messages = " ".join(str(w.message) for w in filtered)
-
- if strict_mode:
- missing = [s for s in expected_substrings if s not in all_messages]
- if missing:
- raise AssertionError(
- f"Missing expected substrings: {missing}. Captured messages: {all_messages}"
- )
- else:
- found = any(s in all_messages for s in expected_substrings)
- if not found:
- raise AssertionError(
- f"None of the expected substrings {expected_substrings} found. "
- f"Captured messages: {all_messages}"
- )
-
-
__all__ = [
"assert_diagnostic_warning",
- "assert_no_warnings",
"capture_warnings",
- "validate_warning_content",
]
#!/usr/bin/env python3
-"""Report formatting focused tests moved from helpers/test_utilities.py.
-
-Owns invariant: report-abs-shaping-line-091 (integration category)
-"""
+"""Report formatting focused tests moved from helpers/test_utilities.py."""
import re
import unittest
PARAMS,
SCENARIOS,
SEEDS,
- TOLERANCE,
)
from ..test_base import RewardSpaceTestBase
class TestReportFormatting(RewardSpaceTestBase):
- def test_statistical_validation_section_absent_when_no_hypothesis_tests(self):
- """Section 5 omitted entirely when no hypothesis tests qualify (idle<30, groups<2, pnl sign groups<30)."""
+ def test_statistical_sections_do_not_depend_on_hypothesis_tests(self):
+ """Diagnostics and bootstrap sections survive when no hypothesis tests qualify."""
# Construct df with idle_duration always zero -> reward_idle all zeros so idle_mask.sum()==0
# Position has only one unique value -> groups<2
# pnl all zeros so no positive/negative groups with >=30 each
"position": np.zeros(n),
}
)
- content = self._write_report(df, real_df=None)
- # Hypothesis section header should be absent
- self.assertNotIn("## 5. Statistical Validation", content)
- # Summary numbering still includes Statistical Validation line (always written)
- self.assertIn("5. **Statistical Validation**", content)
- # Distribution shift subsection appears only inside Section 5; since Section 5 omitted it should be absent.
- self.assertNotIn("### 5.4 Distribution Shift Analysis", content)
- self.assertNotIn("_Not performed (no real episodes provided)._", content)
+ content = self._write_report(df, independent_observations=True, skip_feature_analysis=True)
+ self.assertNotIn("### 5.1 Hypothesis Tests", content)
+ self.assertIn("### 5.2 Confidence Intervals", content)
+ self.assertIn("### 5.3 Distribution Diagnostics", content)
+ self.assertIn("N/A (constant distribution)", content)
+ self.assertIn(f"{SCENARIOS.SAMPLE_SIZE_SMALL:,} resamples", content)
+ descriptive = self._write_report(df, real_df=df.copy(), skip_feature_analysis=True)
+ self.assertNotIn("### 5.2 Confidence Intervals", descriptive)
+ self.assertNotIn("Normal? (Shapiro-Wilk)", descriptive)
+ self.assertIn("### 5.4 Distribution Shift Analysis", descriptive)
+ self.assertNotIn("_Not performed (no real episodes provided)._", descriptive)
+ self.assertIn("inference suppressed", descriptive)
def _write_report(
self, df: pd.DataFrame, *, real_df: pd.DataFrame | None = None, **kwargs
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
seed=SEEDS.BASE,
real_df=real_df,
+ independent_observations=kwargs.get("independent_observations", False),
adjust_method="none",
strict_diagnostics=False,
bootstrap_resamples=SCENARIOS.SAMPLE_SIZE_SMALL, # keep test fast
report_path = out_dir / "statistical_analysis.md"
return report_path.read_text(encoding="utf-8")
- def test_abs_shaping_line_present_and_constant(self):
- """Abs Σ Shaping Reward line present, formatted, uses constant not literal."""
- df = pd.DataFrame(
- {
- "reward_shaping": [TOLERANCE.IDENTITY_STRICT, -TOLERANCE.IDENTITY_STRICT],
- "reward_entry_additive": [0.0, 0.0],
- "reward_exit_additive": [0.0, 0.0],
- }
- )
- total_shaping = df["reward_shaping"].sum()
- self.assertLess(abs(total_shaping), PBRS_INVARIANCE_TOL)
- lines = [f"| Abs Σ Shaping Reward | {abs(total_shaping):.6e} |"]
- content = "\n".join(lines)
- m = re.search("\\| Abs Σ Shaping Reward \\| ([0-9]+\\.[0-9]{6}e[+-][0-9]{2}) \\|", content)
- self.assertIsNotNone(m, "Abs Σ Shaping Reward line missing or misformatted")
- val = float(m.group(1)) if m else None
- if val is not None:
- self.assertLess(val, TOLERANCE.NEGLIGIBLE + TOLERANCE.IDENTITY_STRICT)
-
def test_distribution_shift_section_present_with_real_episodes(self):
"""Distribution Shift section renders metrics table when real episodes provided."""
# Synthetic df (ensure >=10 non-NaN per feature)
#!/usr/bin/env python3
"""Tests for Potential-Based Reward Shaping (PBRS) mechanics."""
-import re
+import math
import unittest
+from unittest.mock import patch
import numpy as np
import pandas as pd
from reward_space_analysis import (
DEFAULT_IDLE_DURATION_MULTIPLIER,
DEFAULT_MODEL_REWARD_PARAMETERS,
+ INTERNAL_GUARDS,
PBRS_INVARIANCE_TOL,
Actions,
Positions,
_compute_exit_potential,
_compute_hold_potential,
_compute_unrealized_pnl_estimate,
- _get_float_param,
+ _get_potential_gamma,
apply_potential_shaping,
get_max_idle_duration_candles,
simulate_samples,
pytestmark = pytest.mark.pbrs
+class TestSimulationParity(RewardSpaceTestBase):
+ """Synthetic durations and reports follow runtime transition semantics."""
+
+ def test_neutral_terminal_closes_stored_potential(self):
+ context = reward_space_analysis.RewardContext(
+ current_pnl=0.0,
+ trade_duration=0,
+ idle_duration=2,
+ max_unrealized_profit=0.0,
+ min_unrealized_profit=0.0,
+ position=Positions.Neutral,
+ action=Actions.Neutral,
+ )
+ for enabled in (False, True):
+ params = self.base_params(
+ hold_potential_enabled=enabled,
+ exit_potential_mode="retain_previous",
+ entry_additive_enabled=False,
+ exit_additive_enabled=False,
+ )
+ ongoing = reward_space_analysis.calculate_reward(
+ context,
+ params,
+ 100.0,
+ 0.03,
+ 2.0,
+ short_allowed=True,
+ action_masking=False,
+ prev_potential=0.25,
+ terminated=False,
+ )
+ terminal = reward_space_analysis.calculate_reward(
+ context,
+ params,
+ 100.0,
+ 0.03,
+ 2.0,
+ short_allowed=True,
+ action_masking=False,
+ prev_potential=0.25,
+ terminated=True,
+ )
+ self.assertAlmostEqualFloat(
+ ongoing.next_potential, 0.25, tolerance=TOLERANCE.GENERIC_EQ
+ )
+ self.assertAlmostEqualFloat(
+ terminal.next_potential, 0.0, tolerance=TOLERANCE.GENERIC_EQ
+ )
+ self.assertAlmostEqualFloat(
+ terminal.reward_shaping, -0.25 if enabled else 0.0, tolerance=TOLERANCE.GENERIC_EQ
+ )
+ self.assertAlmostEqualFloat(
+ terminal.pbrs_delta, terminal.reward_shaping, tolerance=TOLERANCE.GENERIC_EQ
+ )
+ self.assertAlmostEqualFloat(
+ terminal.total - ongoing.total,
+ -0.25 if enabled else 0.0,
+ tolerance=TOLERANCE.GENERIC_EQ,
+ )
+ self.assertEqual(terminal.entry_additive + terminal.exit_additive, 0.0)
+
+ def test_idle_clock_runs_past_threshold_and_hazard_saturates(self):
+ from reward_space_analysis import _SAMPLE_DURATION_HAZARD_MAX_PROBABILITY
+
+ params = self.base_params(max_trade_duration_candles=100, max_idle_duration_candles=2)
+ df = simulate_samples(
+ params=params,
+ num_samples=400,
+ seed=SEEDS.BASE,
+ base_factor=PARAMS.BASE_FACTOR,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ max_duration_ratio=2.0,
+ trading_mode="futures",
+ pnl_base_std=PARAMS.PNL_STD,
+ pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
+ )
+ over = df[df["idle_duration"] > params["max_idle_duration_candles"]]
+ self.assertFalse(over.empty, "idle clock must keep counting past its threshold")
+ self.assertTrue(
+ (
+ over["sample_entry_prob"]
+ >= _SAMPLE_DURATION_HAZARD_MAX_PROBABILITY - TOLERANCE.IDENTITY_RELAXED
+ ).all(),
+ "entry hazard must saturate once the clock exceeds the threshold",
+ )
+ within = df[df["idle_duration"].between(1, params["max_idle_duration_candles"])]
+ self.assertTrue(
+ (
+ within["sample_entry_prob"]
+ < _SAMPLE_DURATION_HAZARD_MAX_PROBABILITY - TOLERANCE.IDENTITY_RELAXED
+ ).all()
+ )
+ stretched = df["idle_duration"] / params["max_idle_duration_candles"]
+ self.assertTrue((df["idle_ratio"] - stretched).abs().max() < TOLERANCE.IDENTITY_STRICT)
+ self.assertTrue((df["idle_ratio"] > 1.0).any())
+
+ def test_simulate_durations_match_runtime_step(self):
+ """Every in-position candle advances duration, including an immediate exit."""
+ df = simulate_samples(
+ params=self.base_params(max_trade_duration_candles=100),
+ num_samples=200,
+ seed=SEEDS.BASE,
+ base_factor=PARAMS.BASE_FACTOR,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ max_duration_ratio=2.0,
+ trading_mode="futures",
+ pnl_base_std=PARAMS.PNL_STD,
+ pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
+ )
+ expected_duration = 0
+ for row in df.itertuples():
+ if row.position != Positions.Neutral.value:
+ expected_duration += 1
+ else:
+ expected_duration = 0
+ self.assertEqual(row.trade_duration, expected_duration)
+
+ def test_first_retained_pnl_is_only_exit_extremum(self):
+ """A first-candle exit excludes the fill-time PnL from its extrema."""
+ params = self.base_params(
+ unrealized_pnl=False,
+ max_trade_duration_candles=100,
+ entry_fee_rate=0.0,
+ exit_fee_rate=0.0,
+ hold_potential_enabled=False,
+ entry_additive_enabled=False,
+ exit_additive_enabled=False,
+ )
+ actions = [
+ (Actions.Long_enter, 1.0, 0.0, 0.0),
+ (Actions.Long_exit, 0.0, 1.0, 0.0),
+ ]
+ with (
+ patch.object(reward_space_analysis, "_sample_action", side_effect=actions),
+ patch.object(
+ reward_space_analysis.random.Random,
+ "gauss",
+ side_effect=[0.02, 0.0],
+ ),
+ ):
+ df = simulate_samples(
+ params=params,
+ num_samples=2,
+ seed=SEEDS.BASE,
+ base_factor=PARAMS.BASE_FACTOR,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ max_duration_ratio=2.0,
+ trading_mode="futures",
+ pnl_base_std=PARAMS.PNL_STD,
+ pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
+ )
+
+ exit_row = df.iloc[1]
+ pnl = float(exit_row["pnl"])
+ self.assertGreater(pnl, 0.0)
+ runtime_context = reward_space_analysis.RewardContext(
+ current_pnl=pnl,
+ trade_duration=int(exit_row["trade_duration"]),
+ idle_duration=0,
+ max_unrealized_profit=pnl,
+ min_unrealized_profit=pnl,
+ position=Positions.Long,
+ action=Actions.Long_exit,
+ )
+ expected_exit = reward_space_analysis.calculate_reward(
+ runtime_context,
+ params,
+ PARAMS.BASE_FACTOR,
+ PARAMS.PROFIT_AIM,
+ PARAMS.RISK_REWARD_RATIO,
+ short_allowed=True,
+ action_masking=True,
+ ).exit_component
+ self.assertAlmostEqualFloat(
+ float(exit_row["reward_exit"]),
+ expected_exit,
+ tolerance=TOLERANCE.IDENTITY_RELAXED,
+ rtol=TOLERANCE.RELATIVE,
+ )
+
+ def test_unrealized_pnl_uses_each_sampled_market_move(self):
+ """Later Gaussian innovations affect later retained PnL without becoming extrema."""
+ params = self.base_params(
+ unrealized_pnl=True,
+ max_trade_duration_candles=100,
+ entry_fee_rate=0.0,
+ exit_fee_rate=0.0,
+ )
+ actions = [
+ (Actions.Long_enter, 1.0, 0.0, 0.0),
+ (Actions.Neutral, 0.0, 0.0, 0.0),
+ (Actions.Neutral, 0.0, 0.0, 0.0),
+ (Actions.Long_exit, 0.0, 1.0, 0.0),
+ ]
+
+ def run(second_return: float) -> pd.DataFrame:
+ with (
+ patch.object(reward_space_analysis, "_sample_action", side_effect=actions),
+ patch.object(
+ reward_space_analysis.random.Random,
+ "gauss",
+ side_effect=[0.02, second_return, 0.0, 0.0],
+ ),
+ ):
+ return simulate_samples(
+ params=params,
+ num_samples=4,
+ seed=SEEDS.BASE,
+ base_factor=PARAMS.BASE_FACTOR,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ max_duration_ratio=2.0,
+ trading_mode="futures",
+ pnl_base_std=PARAMS.PNL_STD,
+ pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
+ )
+
+ baseline = run(0.0)
+ changed = run(0.03)
+ for column in ("action", "position", "trade_duration"):
+ self.assertTrue(baseline[column].equals(changed[column]))
+ self.assertAlmostEqualFloat(
+ float(baseline.iloc[0]["next_pnl"]),
+ float(changed.iloc[0]["next_pnl"]),
+ tolerance=TOLERANCE.IDENTITY_STRICT,
+ )
+ self.assertFalse(
+ np.isclose(
+ float(baseline.iloc[1]["next_pnl"]),
+ float(changed.iloc[1]["next_pnl"]),
+ atol=TOLERANCE.IDENTITY_STRICT,
+ rtol=0.0,
+ )
+ )
+
+ def test_unrealized_pnl_mode_shapes_hold_trajectory(self):
+ """The flag derives hold prices from a target PnL instead of only feeding Phi."""
+ params = self.base_params(
+ hold_potential_enabled=True,
+ exit_potential_mode="non_canonical",
+ max_trade_duration_candles=100,
+ unrealized_pnl=True,
+ entry_fee_rate=0.0015,
+ exit_fee_rate=0.0015,
+ )
+ df = simulate_samples(
+ params=params,
+ num_samples=400,
+ seed=SEEDS.BASE,
+ base_factor=PARAMS.BASE_FACTOR,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ max_duration_ratio=2.0,
+ trading_mode="futures",
+ pnl_base_std=PARAMS.PNL_STD,
+ pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
+ )
+ retained = df.loc[
+ df["next_position"].isin([Positions.Long.value, Positions.Short.value]),
+ "next_pnl",
+ ]
+ self.assertFalse(retained.empty)
+ derivable = retained[retained.abs() > TOLERANCE.IDENTITY_STRICT]
+ self.assertGreater(len(derivable), 0, "synthetic holds must accrue target PnL")
+ self.assertTrue((derivable > 0.0).any(), "market innovations must permit profitable PnL")
+ self.assertTrue((derivable < 0.0).any(), "market innovations must permit losing PnL")
+ self.assertTrue((derivable.abs() <= 0.15).all())
+ replay = simulate_samples(
+ params=params,
+ num_samples=400,
+ seed=SEEDS.BASE,
+ base_factor=PARAMS.BASE_FACTOR,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ max_duration_ratio=2.0,
+ trading_mode="futures",
+ pnl_base_std=PARAMS.PNL_STD,
+ pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
+ )
+ self.assertTrue(replay["next_pnl"].equals(df["next_pnl"]))
+
+ def test_unrealized_pnl_exit_rewards_use_only_retained_extrema(self):
+ """Exit rewards use extrema reconstructed from the retained PnL trajectory."""
+ params = self.base_params(
+ unrealized_pnl=True,
+ max_trade_duration_candles=100,
+ entry_fee_rate=0.0015,
+ exit_fee_rate=0.0015,
+ exit_attenuation_mode="linear",
+ exit_plateau=False,
+ hold_potential_enabled=False,
+ )
+ df = simulate_samples(
+ params=params,
+ num_samples=400,
+ seed=SEEDS.BASE,
+ base_factor=PARAMS.BASE_FACTOR,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ max_duration_ratio=2.0,
+ trading_mode="futures",
+ pnl_base_std=PARAMS.PNL_STD,
+ pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
+ )
+
+ max_unrealized = -np.inf
+ min_unrealized = np.inf
+ checked_exits = 0
+ risk_reward_ratio = float(
+ params.get("risk_reward_ratio", params.get("rr", PARAMS.RISK_REWARD_RATIO))
+ )
+ pnl_target = float(params.get("profit_aim", PARAMS.PROFIT_AIM)) * risk_reward_ratio
+ base_factor = float(params.get("base_factor", PARAMS.BASE_FACTOR))
+ min_range = max(
+ INTERNAL_GUARDS["efficiency_min_range_epsilon"],
+ INTERNAL_GUARDS["efficiency_min_range_fraction"] * pnl_target,
+ )
+
+ for row in df.itertuples():
+ position = Positions(row.position)
+ action = Actions(row.action)
+ if position == Positions.Neutral and action in (
+ Actions.Long_enter,
+ Actions.Short_enter,
+ ):
+ max_unrealized = -np.inf
+ min_unrealized = np.inf
+
+ if position in (Positions.Long, Positions.Short) and action in (
+ Actions.Long_exit,
+ Actions.Short_exit,
+ ):
+ pnl = float(row.pnl)
+ duration_ratio = row.trade_duration / params["max_trade_duration_candles"]
+ time_coefficient = 1.0 / (1.0 + params["exit_linear_slope"] * duration_ratio)
+ pnl_coefficient = 1.0
+ pnl_ratio = pnl / pnl_target
+ if pnl_ratio > 1.0:
+ pnl_coefficient += params["win_reward_factor"] * math.tanh(
+ params["pnl_amplification_sensitivity"] * (pnl_ratio - 1.0)
+ )
+ else:
+ loss_threshold = pnl_target / risk_reward_ratio
+ if pnl < -loss_threshold:
+ loss_ratio = abs(pnl) / loss_threshold
+ pnl_coefficient += (
+ params["win_reward_factor"]
+ * risk_reward_ratio
+ * math.tanh(
+ params["pnl_amplification_sensitivity"] * (loss_ratio - 1.0)
+ )
+ )
+
+ high = max(max_unrealized, pnl)
+ low = min(min_unrealized, pnl)
+ efficiency = 1.0
+ if high - low >= min_range and not np.isclose(pnl, 0.0):
+ ratio = (pnl - low) / (high - low)
+ if pnl > 0.0:
+ efficiency += params["efficiency_weight"] * (
+ ratio - params["efficiency_center"]
+ )
+ else:
+ efficiency += params["efficiency_weight"] * (
+ params["efficiency_center"] - ratio
+ )
+ expected = pnl * base_factor * time_coefficient * pnl_coefficient * efficiency
+ self.assertAlmostEqualFloat(
+ row.reward_exit,
+ expected,
+ tolerance=TOLERANCE.IDENTITY_RELAXED,
+ rtol=TOLERANCE.RELATIVE,
+ )
+ checked_exits += 1
+
+ next_position = Positions(row.next_position)
+ if next_position in (Positions.Long, Positions.Short):
+ max_unrealized = max(max_unrealized, row.next_pnl)
+ min_unrealized = min(min_unrealized, row.next_pnl)
+ else:
+ max_unrealized = min_unrealized = 0.0
+
+ self.assertGreater(checked_exits, 0)
+
+ def test_canonical_report_rejects_offsetting_observed_additives(self):
+ """Offsetting non-zero additives cannot certify a canonical trajectory."""
+ df = simulate_samples(
+ params=self.base_params(
+ exit_potential_mode="canonical",
+ entry_additive_enabled=False,
+ exit_additive_enabled=False,
+ hold_potential_enabled=True,
+ ),
+ num_samples=40,
+ seed=SEEDS.BASE,
+ base_factor=PARAMS.BASE_FACTOR,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ max_duration_ratio=2.0,
+ trading_mode="futures",
+ pnl_base_std=PARAMS.PNL_STD,
+ pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
+ )
+ df["reward_entry_additive"] = np.tile([1.0, -1.0], len(df) // 2)
+ out_dir = self.output_path / "canonical_observed_additives"
+ write_complete_statistical_analysis(
+ df,
+ output_dir=out_dir,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ seed=SEEDS.BASE,
+ skip_feature_analysis=True,
+ skip_partial_dependence=True,
+ bootstrap_resamples=SCENARIOS.BOOTSTRAP_MINIMAL_ITERATIONS,
+ )
+ content = (out_dir / "statistical_analysis.md").read_text(encoding="utf-8")
+ self.assertIn("| Invariance Status | Not verified |", content)
+ self.assertIn("reward_entry_additive contains non-zero values", content)
+ self.assertIn("| Σ Entry Additive | 0.000000 |", content)
+
+ def test_non_canonical_report_classifies_both_outputs(self):
+ """Zero correction cannot certify a non-canonical potential mode."""
+ df = simulate_samples(
+ params=self.base_params(
+ exit_potential_mode="retain_previous",
+ hold_potential_enabled=True,
+ entry_additive_enabled=False,
+ exit_additive_enabled=False,
+ max_trade_duration_candles=100,
+ ),
+ num_samples=200,
+ seed=SEEDS.BASE,
+ base_factor=PARAMS.BASE_FACTOR,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ max_duration_ratio=2.0,
+ trading_mode="futures",
+ pnl_base_std=PARAMS.PNL_STD,
+ pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
+ )
+ out_dir = self.output_path / "non_canonical_classifies_both"
+ write_complete_statistical_analysis(
+ df=df,
+ output_dir=out_dir,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ seed=SEEDS.BASE,
+ skip_feature_analysis=True,
+ skip_partial_dependence=True,
+ bootstrap_resamples=SCENARIOS.SAMPLE_SIZE_SMALL,
+ )
+ content = (out_dir / "statistical_analysis.md").read_text(encoding="utf-8")
+ self.assertRegex(content, r"\| Invariance Status \| [^\n]*Non-canonical")
+ self.assertRegex(content, r"PBRS Invariance\*\* - [^\n]*Non-canonical")
+
+
class TestPBRS(RewardSpaceTestBase):
"""PBRS mechanics tests (transforms, parameters, potentials, invariance)."""
pnl_target=profit_aim * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=current_dur,
next_pnl=0.0,
+ entry_pnl=0.0,
next_duration_ratio=0.0,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
PARAMS.BASE_FACTOR,
)
- gamma = _get_float_param(
- params,
- "potential_gamma",
- DEFAULT_MODEL_REWARD_PARAMETERS.get("potential_gamma", 0.95),
- )
+ gamma = _get_potential_gamma(params)
expected_next_potential = (
prev_potential / gamma if gamma not in (0.0, None) else prev_potential
)
pnl_target=profit_aim * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=current_dur,
next_pnl=0.0,
+ entry_pnl=0.0,
next_duration_ratio=0.0,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=0.0,
next_pnl=0.01,
+ entry_pnl=0.01,
next_duration_ratio=0.0,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=0.4,
next_pnl=0.02,
+ entry_pnl=0.02,
next_duration_ratio=0.41,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=current_duration_ratio,
next_pnl=next_pnl,
+ entry_pnl=next_pnl,
next_duration_ratio=next_duration_ratio,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
base_factor=PARAMS.BASE_FACTOR,
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=0.0,
next_pnl=0.02,
+ entry_pnl=0.02,
next_duration_ratio=0.0,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
pnl_target=profit_aim * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=current_dur,
next_pnl=0.0,
+ entry_pnl=0.0,
next_duration_ratio=0.0,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=0.0,
next_pnl=0.0,
+ entry_pnl=0.0,
next_duration_ratio=0.0,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
base_factor=PARAMS.BASE_FACTOR,
self.assertPlacesEqual(
next_potential, prev_potential, places=TOLERANCE.DECIMAL_PLACES_STRICT
)
- raw_gamma = DEFAULT_MODEL_REWARD_PARAMETERS.get("potential_gamma", 0.95)
- gamma_fallback = 0.95 if raw_gamma is None else raw_gamma
- try:
- gamma = float(gamma_fallback)
- except Exception:
- gamma = 0.95
+ gamma = _get_potential_gamma(params)
# PBRS shaping Δ = γ·Φ(next) - Φ(prev). Here Φ(next)=Φ(prev) since decay clamps to 0.
self.assertLessEqual(
abs(shaping - ((gamma - 1.0) * prev_potential)),
def test_potential_gamma_nan_fallback(self):
"""Verifies potential_gamma=NaN fallback to default value."""
base_params_dict = self.base_params()
- default_gamma = base_params_dict.get("potential_gamma", 0.95)
+ default_gamma = _get_potential_gamma(base_params_dict)
params_nan = self.base_params(potential_gamma=np.nan, hold_potential_enabled=True)
res_nan = apply_potential_shaping(
base_reward=0.1,
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=0.2,
next_pnl=0.035,
+ entry_pnl=0.035,
next_duration_ratio=0.25,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=0.2,
next_pnl=0.035,
+ entry_pnl=0.035,
next_duration_ratio=0.25,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
)
def test_simulate_samples_initializes_pnl_on_entry(self):
- """simulate_samples() sets in-position pnl to fee-aware entry estimate."""
+ """First in-position candle includes fees and its duration-dependent drift."""
params = self.base_params(
exit_potential_mode="non_canonical",
hold_potential_enabled=True,
expected_pnl = _compute_unrealized_pnl_estimate(
Positions.Long,
entry_open=1.0,
- current_open=1.0,
+ current_open=1.0 + 0.001 / params["max_trade_duration_candles"],
params=params,
)
post_entry_pnl = float(enter_pos.iloc[next_pos]["pnl"])
post_entry_pnl,
expected_pnl,
tolerance=TOLERANCE.IDENTITY_STRICT,
- msg="Expected pnl after entry to match entry fee estimate",
+ msg="Expected first in-position PnL to include fees and the first candle move",
)
def test_calculate_reward_hold_uses_current_duration_ratio(self):
PARAMS.BASE_FACTOR,
)
self.assertFinite(prev_phi, name="prev_phi")
- next_phi_can = _compute_exit_potential(prev_phi, params_can)
+ next_phi_can = _compute_exit_potential(prev_phi, params_can, gamma)
self.assertAlmostEqualFloat(
next_phi_can,
0.0,
msg="Canonical delta mismatch",
)
params_spike = self.base_params(exit_potential_mode="spike_cancel", **base_common)
- next_phi_spike = _compute_exit_potential(prev_phi, params_spike)
+ next_phi_spike = _compute_exit_potential(prev_phi, params_spike, gamma)
shaping_spike = gamma * next_phi_spike - prev_phi
self.assertNearZero(
shaping_spike,
exit_additive_enabled=False,
potential_gamma=0.9,
)
- gamma = _get_float_param(
- params,
- "potential_gamma",
- DEFAULT_MODEL_REWARD_PARAMETERS.get("potential_gamma", 0.95),
- )
- rng = np.random.default_rng(555)
+ gamma = _get_potential_gamma(params)
+ rng = np.random.default_rng(SEEDS.ALTERNATE_1)
potentials = rng.uniform(0.05, 0.85, size=220)
deltas = [gamma * p - p for p in potentials]
cumulative = float(np.sum(deltas))
pnl_target=profit_aim * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=current_dur,
next_pnl=0.0,
+ entry_pnl=0.0,
next_duration_ratio=0.0,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
def test_normality_invariance_under_scaling(self):
"""Skewness & excess kurtosis invariant under positive scaling of normal sample."""
- rng = np.random.default_rng(808)
+ rng = np.random.default_rng(SEEDS.ALTERNATE_2)
base = rng.normal(0.0, 1.0, size=7000)
scaled = 5.0 * base
# ---------------- Report classification / formatting ---------------- #
- # Non-owning smoke; ownership: robustness/test_robustness.py:43 (robustness-decomposition-integrity-101), robustness/test_robustness.py:127 (robustness-exit-pnl-only-117)
- @pytest.mark.smoke
- def test_pbrs_non_canonical_report_generation(self):
- """Synthetic invariance section: Non-canonical classification formatting."""
-
- df = pd.DataFrame(
- {
- "reward_shaping": [0.01, -0.002],
- "reward_entry_additive": [0.0, 0.0],
- "reward_exit_additive": [0.001, 0.0],
- }
- )
- total_shaping = df["reward_shaping"].sum()
- self.assertGreater(abs(total_shaping), PBRS_INVARIANCE_TOL)
- invariance_status = "❌ Non-canonical"
- section = []
- section.append("**PBRS Invariance Summary:**\n")
- section.append("| Field | Value |\n")
- section.append("|-------|-------|\n")
- section.append(f"| Invariance | {invariance_status} |\n")
- section.append(f"| Note | Total shaping = {total_shaping:.6f} (non-zero) |\n")
- section.append(f"| Σ Shaping Reward | {total_shaping:.6f} |\n")
- section.append(f"| Abs Σ Shaping Reward | {abs(total_shaping):.6e} |\n")
- section.append(f"| Σ Entry Additive | {df['reward_entry_additive'].sum():.6f} |\n")
- section.append(f"| Σ Exit Additive | {df['reward_exit_additive'].sum():.6f} |\n")
- content = "".join(section)
- assert_pbrs_invariance_report_classification(
- self, content, "Non-canonical", expect_additives=False
- )
- self.assertRegex(content, "Σ Shaping Reward \\| 0\\.008000 \\|")
- m_abs = re.search("Abs Σ Shaping Reward \\| ([0-9.]+e[+-][0-9]{2}) \\|", content)
- self.assertIsNotNone(m_abs)
- if m_abs:
- val = float(m_abs.group(1))
- self.assertAlmostEqual(abs(total_shaping), val, places=TOLERANCE.DECIMAL_PLACES_STRICT)
-
def test_potential_gamma_boundary_values_stability(self):
"""Potential gamma boundary values (0 and ≈1) produce bounded shaping."""
for gamma in [0.0, 0.999999]:
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=0.3,
next_pnl=0.025,
+ entry_pnl=0.025,
next_duration_ratio=0.35,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
base_factor=PARAMS.BASE_FACTOR,
exit_additive_enabled=False,
exit_potential_mode="canonical",
)
- gamma = _get_float_param(
- params,
- "potential_gamma",
- DEFAULT_MODEL_REWARD_PARAMETERS.get("potential_gamma", 0.95),
- )
- rng = np.random.default_rng(321)
+ gamma = _get_potential_gamma(params)
+ rng = np.random.default_rng(SEEDS.REPORT_FORMAT_2)
prev_potential = 0.0
telescoping_sum = 0.0
max_abs_step = 0.0
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=current_dur,
next_pnl=next_pnl,
+ entry_pnl=next_pnl,
next_duration_ratio=next_dur,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
base_factor=PARAMS.BASE_FACTOR,
exit_potential_mode="progressive_release",
exit_potential_decay=0.25,
)
- rng = np.random.default_rng(321)
+ rng = np.random.default_rng(SEEDS.REPORT_FORMAT_2)
prev_potential = 0.0
shaping_sum = 0.0
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=float(rng.uniform(0, 1)),
next_pnl=next_pnl,
+ entry_pnl=next_pnl,
next_duration_ratio=next_dur,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
base_factor=PARAMS.BASE_FACTOR,
# Owns invariant: pbrs-canonical-near-zero-report-116
@pytest.mark.smoke
def test_pbrs_canonical_near_zero_report(self):
- """Invariant 116: canonical near-zero cumulative shaping classified in full report."""
+ """Canonical trajectories with valid PBRS evidence are classified as verified."""
- small_vals = [1.0e-7, -2.0e-7, 3.0e-7] # sum = 2.0e-7 < tolerance
- total_shaping = float(sum(small_vals))
- self.assertLess(
- abs(total_shaping),
- PBRS_INVARIANCE_TOL,
- f"Total shaping {total_shaping} exceeds invariance tolerance",
- )
- inv_corr_vals = [1.0e-7, -1.0e-7, 2.0e-7]
- max_abs_corr = float(np.max(np.abs(inv_corr_vals)))
- self.assertLess(max_abs_corr, PBRS_INVARIANCE_TOL)
+ gamma = DEFAULT_MODEL_REWARD_PARAMETERS["potential_gamma"]
+ prev_potentials = np.array([0.0, -1e-7, 1e-7])
+ next_potentials = np.array([-1e-7, 1e-7, 0.0])
+ small_vals = (gamma * next_potentials - prev_potentials).tolist()
+ inv_corr_vals = [0.0] * len(small_vals)
n = len(small_vals)
df = pd.DataFrame(
"reward_invalid": np.zeros(n),
"duration_ratio": np.random.uniform(0.2, 1.0, n),
"idle_ratio": np.zeros(n),
+ "episode_id": 0,
+ "transition_index": np.arange(n),
+ "terminated": np.array([False, False, True]),
+ "prev_potential": prev_potentials,
+ "next_potential": next_potentials,
}
)
df.attrs["reward_params"] = {
self.assertTrue(report_path.exists(), "Report file missing for canonical near-zero test")
content = report_path.read_text(encoding="utf-8")
assert_pbrs_invariance_report_classification(
- self, content, "Canonical", expect_additives=False
- )
- self.assertRegex(content, r"\| Σ Shaping Reward \| 0\.000000 \|")
- m_abs = re.search(r"\| Abs Σ Shaping Reward \| ([0-9.]+e[+-][0-9]{2}) \|", content)
- self.assertIsNotNone(m_abs)
- if m_abs:
- val_abs = float(m_abs.group(1))
- self.assertAlmostEqual(
- abs(total_shaping), val_abs, places=TOLERANCE.DECIMAL_PLACES_STRICT
- )
- self.assertIn("max|correction|≈0", content)
+ self, content, "Canonical: observed PBRS verified", expect_additives=False
+ )
+ self.assertIn("Raw shaping sums do not certify invariance", content)
# Non-owning smoke; ownership: robustness/test_robustness.py:43 (robustness-decomposition-integrity-101)
@pytest.mark.smoke
non-canonical additives involvement.
"""
- small_vals = [1.0e-7, -2.0e-7, 3.0e-7] # sum = 2.0e-7 < tolerance
- total_shaping = float(sum(small_vals))
- self.assertLess(abs(total_shaping), PBRS_INVARIANCE_TOL)
- inv_corr_vals = [1.0e-7, -1.0e-7, 2.0e-7]
- max_abs_corr = float(np.max(np.abs(inv_corr_vals)))
- self.assertLess(max_abs_corr, PBRS_INVARIANCE_TOL)
+ gamma = DEFAULT_MODEL_REWARD_PARAMETERS["potential_gamma"]
+ prev_potentials = np.array([0.0, -1e-7, 1e-7])
+ next_potentials = np.array([-1e-7, 1e-7, 0.0])
+ small_vals = (gamma * next_potentials - prev_potentials).tolist()
+ inv_corr_vals = [0.0] * len(small_vals)
n = len(small_vals)
df = pd.DataFrame(
"reward_invalid": np.zeros(n),
"duration_ratio": np.random.uniform(0.2, 1.0, n),
"idle_ratio": np.zeros(n),
+ "episode_id": 0,
+ "transition_index": np.arange(3),
+ "terminated": np.array([False, False, True]),
+ "prev_potential": prev_potentials,
+ "next_potential": next_potentials,
}
)
df.attrs["reward_params"] = {
self.assertTrue(report_path.exists(), "Report file missing for canonical additives test")
content = report_path.read_text(encoding="utf-8")
assert_pbrs_invariance_report_classification(
- self, content, "Canonical", expect_additives=False
+ self, content, "Canonical: observed PBRS verified", expect_additives=False
)
self.assertIn("Additives are suppressed in canonical mode", content)
self.assertIn("| Entry Additive Enabled | True |", content)
self.assertIn("| Entry Additive Effective | False |", content)
self.assertIn("| Exit Additive Effective | False |", content)
- def test_pbrs_canonical_warning_report(self):
- """Canonical mode + no additives but max|invariance_correction| > tolerance -> warning."""
+ # Non-owning boundary; ownership: pbrs/test_pbrs.py:1509
+ # Invariant: pbrs-canonical-near-zero-report-116
+ def test_singleton_terminal_canonical_report_is_verified(self):
+ """A complete one-transition terminal episode is sufficient PBRS evidence."""
+ params = self.base_params(
+ exit_potential_mode="canonical",
+ entry_additive_enabled=False,
+ exit_additive_enabled=False,
+ )
+ previous_potential = 0.25
+ shaping = -previous_potential
+ df = pd.DataFrame(
+ {
+ "reward": [shaping],
+ "reward_base": [0.0],
+ "reward_pbrs_delta": [shaping],
+ "reward_shaping": [shaping],
+ "reward_invariance_correction": [0.0],
+ "reward_invalid": [0.0],
+ "reward_idle": [0.0],
+ "reward_hold": [0.0],
+ "reward_exit": [0.0],
+ "reward_entry_additive": [0.0],
+ "reward_exit_additive": [0.0],
+ "pnl": [0.0],
+ "exit_pnl": [0.0],
+ "trade_duration": [0],
+ "idle_duration": [0],
+ "duration_ratio": [0.0],
+ "idle_ratio": [0.0],
+ "position": [0.0],
+ "action": [0],
+ "episode_id": [0],
+ "transition_index": [0],
+ "terminated": [True],
+ "prev_potential": [previous_potential],
+ "next_potential": [0.0],
+ }
+ )
+ df.attrs["reward_params"] = params
+ out_dir = self.output_path / "canonical_singleton_terminal"
+ write_complete_statistical_analysis(
+ df,
+ output_dir=out_dir,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ seed=SEEDS.SMOKE_TEST,
+ skip_feature_analysis=True,
+ skip_partial_dependence=True,
+ bootstrap_resamples=SCENARIOS.BOOTSTRAP_MINIMAL_ITERATIONS,
+ )
+ content = (out_dir / "statistical_analysis.md").read_text(encoding="utf-8")
+ assert_pbrs_invariance_report_classification(
+ self, content, "Canonical: observed PBRS verified", expect_additives=False
+ )
+ self.assertIn(
+ "Local identity, continuity and discounted terminal boundary verified", content
+ )
- shaping_vals = [1.2e-4, 1.3e-4, 8.0e-5, -2.0e-5, 1.4e-4] # Σ not near 0
- total_shaping = float(sum(shaping_vals))
- self.assertGreater(abs(total_shaping), PBRS_INVARIANCE_TOL)
+ def test_report_without_reward_params_is_not_canonical(self):
+ """Canonical certification requires explicit reward configuration evidence."""
+ previous_potential = 0.25
+ source = pd.DataFrame(
+ {
+ "reward": [-previous_potential],
+ "reward_base": [0.0],
+ "reward_pbrs_delta": [-previous_potential],
+ "reward_shaping": [-previous_potential],
+ "reward_invariance_correction": [0.0],
+ "reward_invalid": [0.0],
+ "reward_idle": [0.0],
+ "reward_hold": [0.0],
+ "reward_exit": [0.0],
+ "reward_entry_additive": [0.0],
+ "reward_exit_additive": [0.0],
+ "pnl": [0.0],
+ "exit_pnl": [0.0],
+ "trade_duration": [0],
+ "idle_duration": [0],
+ "duration_ratio": [0.0],
+ "idle_ratio": [0.0],
+ "position": [0.0],
+ "action": [0],
+ "episode_id": [0],
+ "transition_index": [0],
+ "terminated": [True],
+ "prev_potential": [previous_potential],
+ "next_potential": [0.0],
+ }
+ )
+ source.attrs["reward_params"] = {
+ "exit_potential_mode": "non_canonical",
+ "entry_additive_enabled": False,
+ "exit_additive_enabled": False,
+ }
+ csv_path = self.output_path / "noncanonical_trajectory.csv"
+ source.to_csv(csv_path, index=False)
+ reloaded = pd.read_csv(csv_path)
+ self.assertEqual(reloaded.attrs, {})
+
+ out_dir = self.output_path / "missing_reward_params"
+ write_complete_statistical_analysis(
+ reloaded,
+ output_dir=out_dir,
+ profit_aim=PARAMS.PROFIT_AIM,
+ risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
+ seed=SEEDS.SMOKE_TEST,
+ skip_feature_analysis=True,
+ skip_partial_dependence=True,
+ bootstrap_resamples=SCENARIOS.BOOTSTRAP_MINIMAL_ITERATIONS,
+ )
+ content = (out_dir / "statistical_analysis.md").read_text(encoding="utf-8")
+ assert_pbrs_invariance_report_classification(
+ self, content, "Not verified", expect_additives=False
+ )
+ self.assertIn("Reward configuration evidence is missing or incomplete", content)
+ self.assertIn("| Exit Potential Mode | unknown |", content)
+ self.assertNotIn("Canonical: observed PBRS verified", content)
- inv_corr_vals = [1.0e-4, -2.0e-4, 1.5e-4, -1.2e-4, 7.0e-5]
- max_abs_corr = float(np.max(np.abs(inv_corr_vals)))
- self.assertGreater(max_abs_corr, PBRS_INVARIANCE_TOL)
+ # Owns invariant: pbrs-discounted-evidence-125
+ def test_pbrs_canonical_discontinuous_potentials_report(self):
+ """Potential discontinuity is sufficient to reject otherwise local PBRS evidence."""
+ gamma = DEFAULT_MODEL_REWARD_PARAMETERS["potential_gamma"]
+ prev_potentials = np.zeros(5)
+ next_potentials = np.array([-1.2e-4, 1.0e-4, -2.0e-4, 1.5e-4, 0.0])
+ shaping_vals = (gamma * next_potentials - prev_potentials).tolist()
+ inv_corr_vals = [0.0] * len(shaping_vals)
n = len(shaping_vals)
df = pd.DataFrame(
{
"reward_invalid": np.zeros(n),
"duration_ratio": np.random.uniform(0.2, 1.2, n),
"idle_ratio": np.zeros(n),
+ "episode_id": 0,
+ "transition_index": np.arange(5),
+ "terminated": np.array([False, False, False, False, True]),
+ "prev_potential": prev_potentials,
+ "next_potential": next_potentials,
}
)
df.attrs["reward_params"] = {
self.assertTrue(report_path.exists(), "Report file missing for canonical warning test")
content = report_path.read_text(encoding="utf-8")
assert_pbrs_invariance_report_classification(
- self, content, "Canonical (with warning)", expect_additives=False
+ self, content, "Not verified", expect_additives=False
)
- expected_corr_fragment = f"{max_abs_corr:.6e}"
- self.assertIn(expected_corr_fragment, content)
+ self.assertIn("Raw shaping sums do not certify invariance", content)
+ self.assertIn("Potential discontinuity between transitions", content)
# Non-owning smoke; ownership: robustness/test_robustness.py:43 (robustness-decomposition-integrity-101)
@pytest.mark.smoke
def test_pbrs_non_canonical_full_report_reason_aggregation(self):
- """Full report: Non-canonical classification aggregates mode + additives reasons."""
+ """Non-canonical configurations never certify observed PBRS."""
shaping_vals = [0.02, -0.005, 0.007]
entry_add_vals = [0.003, 0.0, 0.004]
"reward_invalid": np.zeros(n),
"duration_ratio": np.random.uniform(0.1, 1.0, n),
"idle_ratio": np.zeros(n),
+ "episode_id": 0,
+ "transition_index": np.arange(3),
+ "terminated": np.array([False, False, True]),
+ "prev_potential": np.array([0.0, 0.02, -0.005]),
+ "next_potential": np.array([0.02, -0.005, 0.0]),
}
)
df.attrs["reward_params"] = {
)
content = report_path.read_text(encoding="utf-8")
assert_pbrs_invariance_report_classification(
- self, content, "Non-canonical", expect_additives=True
+ self, content, "Non-canonical: not verified", expect_additives=True
)
- self.assertIn("exit_potential_mode='progressive_release'", content)
+ self.assertIn("| Exit Potential Mode | progressive_release |", content)
# Non-owning smoke; ownership: robustness/test_robustness.py:43 (robustness-decomposition-integrity-101)
@pytest.mark.smoke
def test_pbrs_non_canonical_mode_only_reason(self):
- """Non-canonical exit mode with additives disabled -> reason excludes additive list."""
+ """Non-canonical exit mode with additives disabled -> not verified."""
shaping_vals = [0.002, -0.0005, 0.0012]
total_shaping = sum(shaping_vals)
"reward_invalid": np.zeros(n),
"duration_ratio": np.random.uniform(0.2, 1.2, n),
"idle_ratio": np.zeros(n),
+ "episode_id": 0,
+ "transition_index": np.arange(n),
+ "terminated": np.array([False, False, True]),
+ "prev_potential": np.array([0.0, 0.002, -0.0005]),
+ "next_potential": np.array([0.002, -0.0005, 0.0]),
}
)
df.attrs["reward_params"] = {
)
content = report_path.read_text(encoding="utf-8")
assert_pbrs_invariance_report_classification(
- self, content, "Non-canonical", expect_additives=False
+ self, content, "Non-canonical: not verified", expect_additives=False
)
- self.assertIn("exit_potential_mode='retain_previous'", content)
+ self.assertIn("| Exit Potential Mode | retain_previous |", content)
# Owns invariant: pbrs-absence-shift-placeholder-118
def test_pbrs_absence_and_distribution_shift_placeholder(self):
{"exit_power_tau": 1.5},
{"exit_half_life": 0.0},
{"exit_half_life": float("nan")},
+ {"efficiency_weight": 2.0, "efficiency_center": 0.75},
]
run_strict_validation_failure_cases(adapter, failure_params, validate_reward_parameters)
({"exit_linear_slope": "not_a_number", "strict_validation": False}, ["non_numeric_reset"]),
({"exit_power_tau": float("inf"), "strict_validation": False}, ["non_numeric_reset"]),
({"max_idle_duration_candles": "bad", "strict_validation": False}, ["derived_default"]),
+ (
+ {
+ "efficiency_weight": 2.0,
+ "efficiency_center": 0.75,
+ "strict_validation": False,
+ },
+ ["negative_efficiency_guard"],
+ ),
]
run_relaxed_validation_adjustment_cases(
_PyTestAdapter(), relaxed_cases, validate_reward_parameters
"context": make_context(-0.02),
"duration_ratio": 2.0,
"params": {
- "exit_attenuation_mode": "legacy",
+ "exit_attenuation_mode": "linear",
"exit_plateau": False,
"check_invariants": True,
},
from reward_space_analysis import (
ATTENUATION_MODES,
- ATTENUATION_MODES_WITH_LEGACY,
Actions,
Positions,
RewardContext,
self.assertFinite(observed_ratio, name="observed_ratio")
self.assertLess(
abs(observed_ratio - expected_ratio),
- 5e-12 if tau == 1.0 else 5e-09,
+ TOLERANCE.ALPHA_ATTENUATION_STRICT
+ if tau == 1.0
+ else TOLERANCE.ALPHA_ATTENUATION_RELAXED,
f"Alpha attenuation mismatch tau={tau} alpha={alpha} obs_ratio={observed_ratio} exp_ratio={expected_ratio}",
)
def test_exit_attenuation_modes_enumeration(self):
"""All exit attenuation modes produce finite rewards without errors.
- Smoke test ensuring each exit attenuation mode (including legacy modes)
- executes successfully and produces finite reward components. This validates
- that mode enumeration is complete and all modes are correctly implemented.
+ Smoke test ensuring each canonical exit attenuation mode executes
+ successfully and produces finite reward components. This validates that
+ mode enumeration is complete and all modes are correctly implemented.
**Setup:**
- - Modes tested: All values in ATTENUATION_MODES_WITH_LEGACY
+ - Modes tested: All values in ATTENUATION_MODES
- Context: Long exit with pnl=0.02, duration=50, profit extrema=[0.01, 0.03]
- Uses subTest for mode-specific failure isolation
**Tolerance rationale:**
- Uses assertFinite which checks for non-NaN, non-Inf values only
"""
- modes = ATTENUATION_MODES_WITH_LEGACY
+ modes = ATTENUATION_MODES
for mode in modes:
with self.subTest(mode=mode):
test_params = self.base_params(exit_attenuation_mode=mode)
)
from ..constants import (
+ CONTINUITY,
PARAMS,
SCENARIOS,
SEEDS,
STATISTICAL,
TOLERANCE,
)
-from ..helpers import assert_diagnostic_warning
from ..test_base import RewardSpaceTestBase
_perform_feature_analysis = getattr(reward_space_analysis, "_perform_feature_analysis", None)
"idle_duration": np.ones(n) * 3.0,
}
)
- metrics = compute_distribution_shift_metrics(df_const, df_const.copy())
+ metrics = compute_distribution_shift_metrics(
+ df_const, df_const.copy(), independent_observations=True
+ )
# Each feature should have zero metrics and ks_pvalue=1.0
for feature in ["pnl", "trade_duration", "idle_duration"]:
for suffix in ["kl_divergence", "js_distance", "wasserstein", "ks_statistic"]:
df1 = self._make_idle_variance_df(100)
df2 = self._make_idle_variance_df(100)
df2["reward"] += 0.1
- metrics = compute_distribution_shift_metrics(df1, df2)
+ metrics = compute_distribution_shift_metrics(df1, df2, independent_observations=True)
expected_keys = {
"pnl_kl_divergence",
"pnl_js_distance",
def test_statistics_distribution_shift_identity_null_metrics(self):
"""Identity distributions -> near-zero shift metrics."""
df = self._make_idle_variance_df(180)
- metrics_id = compute_distribution_shift_metrics(df, df.copy())
+ metrics_id = compute_distribution_shift_metrics(
+ df, df.copy(), independent_observations=True
+ )
for name, val in metrics_id.items():
if name.endswith(("_kl_divergence", "_js_distance", "_wasserstein")):
self.assertLess(
negative_ratio, 0.5, "Most idle rewards should be negative (penalties)"
)
- def test_statistics_distribution_constant_fallback_diagnostics(self):
- """Invariant 115: constant distribution triggers fallback diagnostics (zero moments, qq_r2=1.0)."""
+ def test_statistics_distribution_constant_diagnostics(self):
+ """Invariant 115: constant distributions keep exact mean/std and mark undefined diagnostics."""
# Build constant reward/pnl columns to force degenerate stats
n = 60
df_const = pd.DataFrame(
"pnl_raw": np.zeros(n),
}
)
- diagnostics = distribution_diagnostics(df_const)
- # Mean and std for constant arrays
- for key in ["reward_mean", "reward_std", "pnl_mean", "pnl_std"]:
- if key in diagnostics:
- self.assertAlmostEqualFloat(
- float(diagnostics[key]), 0.0, tolerance=TOLERANCE.IDENTITY_RELAXED
- )
- # Skewness & kurtosis fallback to INTERNAL_GUARDS['distribution_constant_fallback_moment'] (0.0)
- for key in ["reward_skewness", "reward_kurtosis", "pnl_skewness", "pnl_kurtosis"]:
- if key in diagnostics:
- self.assertAlmostEqualFloat(
- float(diagnostics[key]), 0.0, tolerance=TOLERANCE.IDENTITY_RELAXED
- )
- # Q-Q plot r2 fallback value
- qq_key = next((k for k in diagnostics if k.endswith("_qq_r2")), None)
- if qq_key is not None:
- self.assertAlmostEqualFloat(
- float(diagnostics[qq_key]), 1.0, tolerance=TOLERANCE.IDENTITY_RELAXED
- )
- # All diagnostic values finite
- for k, v in diagnostics.items():
- self.assertFinite(v, name=k)
+ for strict in (False, True):
+ with self.subTest(strict_diagnostics=strict):
+ diagnostics = distribution_diagnostics(df_const, strict_diagnostics=strict)
+ for col in ["reward", "pnl"]:
+ self.assertEqual(diagnostics[f"{col}_mean"], 0.0)
+ self.assertEqual(diagnostics[f"{col}_std"], 0.0)
+ self.assertIsNone(diagnostics[f"{col}_skewness"])
+ self.assertIsNone(diagnostics[f"{col}_kurtosis"])
+ self.assertTrue(diagnostics[f"{col}_constant"])
+ for suffix in ("shapiro_stat", "shapiro_pval", "anderson_stat", "qq_r_squared"):
+ self.assertNotIn(f"{col}_{suffix}", diagnostics)
def test_stats_distribution_diagnostics(self):
"""Distribution diagnostics."""
"""Test statistical_hypothesis_tests API integration with synthetic data."""
base = self.make_stats_df(n=200, seed=SEEDS.BASE, idle_pattern="mixed")
base.loc[:149, ["reward_idle", "reward_hold", "reward_exit"]] = 0.0
- results = statistical_hypothesis_tests(base)
+ results = statistical_hypothesis_tests(base, independent_observations=True)
self.assertIsInstance(results, dict)
def test_stats_js_distance_symmetry_violin(self):
"""JS distance symmetry d(P,Q)==d(Q,P)."""
df1 = self._shift_scale_df(300, shift=0.0)
df2 = self._shift_scale_df(300, shift=0.3)
- metrics = compute_distribution_shift_metrics(df1, df2)
+ metrics = compute_distribution_shift_metrics(df1, df2, independent_observations=True)
self.assertIn("pnl_js_distance", metrics)
- metrics_swapped = compute_distribution_shift_metrics(df2, df1)
+ metrics_swapped = compute_distribution_shift_metrics(
+ df2, df1, independent_observations=True
+ )
self.assertIn("pnl_js_distance", metrics_swapped)
self.assertAlmostEqualFloat(
df2 = df1.copy()
df2["pnl"] *= scale
df1["pnl"] *= scale
- metrics = compute_distribution_shift_metrics(df1, df2)
+ metrics = compute_distribution_shift_metrics(df1, df2, independent_observations=True)
for k, v in metrics.items():
if k.endswith("_kl_divergence") or k.endswith("_js_distance"):
self.assertLess(
df["reward_idle"] = rng.normal(0, 1, n) * 0.001
df["position"] = rng.choice([0.0, 1.0], size=n)
df["action"] = rng.choice([0.0, 2.0], size=n)
- tests = statistical_hypothesis_tests(df)
+ tests = statistical_hypothesis_tests(df, independent_observations=True)
flags: list[bool] = []
for v in tests.values():
if isinstance(v, dict):
y_noisy = y + rng.normal(0, 0.0001, len(y))
window = 5
y_smooth = np.convolve(y_noisy, np.ones(window) / window, mode="valid")
- self.assertMonotonic(y_smooth, non_increasing=True, tolerance=1e-05)
+ self.assertMonotonic(y_smooth, non_increasing=True, tolerance=CONTINUITY.EPS_LARGE)
def test_stats_hypothesis_seed_reproducibility(self):
"""Seed reproducibility for statistical_hypothesis_tests + bootstrap."""
df = self.make_stats_df(n=300, seed=SEEDS.BASE, idle_pattern="mixed")
- r1 = statistical_hypothesis_tests(df, seed=SEEDS.REPRODUCIBILITY)
- r2 = statistical_hypothesis_tests(df, seed=SEEDS.REPRODUCIBILITY)
+ r1 = statistical_hypothesis_tests(
+ df, seed=SEEDS.REPRODUCIBILITY, independent_observations=True
+ )
+ r2 = statistical_hypothesis_tests(
+ df, seed=SEEDS.REPRODUCIBILITY, independent_observations=True
+ )
self.assertEqual(set(r1.keys()), set(r2.keys()))
for k in r1:
for field in ("p_value", "significant"):
self.assertEqual(v1, v2, f"Mismatch for {k}:{field}")
metrics = ["reward", "pnl"]
ci_a = bootstrap_confidence_intervals(
- df, metrics, n_bootstrap=STATISTICAL.BOOTSTRAP_DEFAULT_ITERATIONS, seed=SEEDS.BOOTSTRAP
+ df,
+ metrics,
+ n_bootstrap=STATISTICAL.BOOTSTRAP_DEFAULT_ITERATIONS,
+ seed=SEEDS.BOOTSTRAP,
+ independent_observations=True,
)
ci_b = bootstrap_confidence_intervals(
- df, metrics, n_bootstrap=STATISTICAL.BOOTSTRAP_DEFAULT_ITERATIONS, seed=SEEDS.BOOTSTRAP
+ df,
+ metrics,
+ n_bootstrap=STATISTICAL.BOOTSTRAP_DEFAULT_ITERATIONS,
+ seed=SEEDS.BOOTSTRAP,
+ independent_observations=True,
)
for metric in metrics:
m_a, lo_a, hi_a = ci_a[metric]
"idle_duration": np.random.gamma(2.5, 6, 500),
}
)
- metrics = compute_distribution_shift_metrics(df1, df2)
+ metrics = compute_distribution_shift_metrics(df1, df2, independent_observations=True)
for feature in ["pnl", "trade_duration", "idle_duration"]:
for suffix, upper in [
("kl_divergence", None),
self.assertPValue(
diagnostics[f"{col}_shapiro_pval"], msg=f"Shapiro p-value bounds for {col}"
)
- hypothesis_results = statistical_hypothesis_tests(df, seed=SEEDS.BASE)
+ hypothesis_results = statistical_hypothesis_tests(
+ df, seed=SEEDS.BASE, independent_observations=True
+ )
for test_name, result in hypothesis_results.items():
if "p_value" in result:
self.assertPValue(result["p_value"], msg=f"p-value bounds for {test_name}")
self.assertFinite(rho, name=f"rho[{test_name}]")
self.assertWithin(rho, -1.0, 1.0, name="rho")
+ def test_bh_excludes_undefined_tests_from_finite_family(self):
+ """Undefined constant-input correlation cannot contaminate valid adjusted p-values."""
+ rng = np.random.default_rng(SEEDS.BASE)
+ n = 120
+ df = pd.DataFrame(
+ {
+ "reward_idle": np.full(n, -1.0),
+ "idle_duration": np.ones(n),
+ "position": np.repeat([0.0, 1.0], n // 2),
+ "pnl": np.tile([-1.0, 1.0], n // 2),
+ "reward": rng.normal(size=n),
+ }
+ )
+ results = statistical_hypothesis_tests(
+ df, independent_observations=True, adjust_method="benjamini_hochberg"
+ )
+ undefined = results["idle_correlation"]
+ self.assertFalse(undefined["applicable"])
+ self.assertIsNone(undefined["significant"])
+ self.assertIsNone(undefined["significant_adj"])
+ self.assertTrue(np.isnan(undefined["p_value_adj"]))
+ valid = sorted(
+ [results["position_reward_difference"], results["pnl_sign_reward_difference"]],
+ key=lambda result: result["p_value"],
+ )
+ self.assertAlmostEqual(
+ valid[0]["p_value_adj"], min(2 * valid[0]["p_value"], valid[1]["p_value"], 1.0)
+ )
+ self.assertAlmostEqual(valid[1]["p_value_adj"], valid[1]["p_value"])
+ for result in valid:
+ self.assertTrue(result["applicable"])
+ self.assertTrue(np.isfinite(result["p_value_adj"]))
+ self.assertEqual(result["significant_adj"], result["p_value_adj"] < 0.05)
+
def test_stats_benjamini_hochberg_adjustment(self):
"""BH adjustment adds p_value_adj & significant_adj with valid bounds."""
pnl_duration_vol_scale=PARAMS.PNL_DUR_VOL_SCALE,
)
results_adj = statistical_hypothesis_tests(
- df, adjust_method="benjamini_hochberg", seed=SEEDS.REPRODUCIBILITY
+ df,
+ adjust_method="benjamini_hochberg",
+ seed=SEEDS.REPRODUCIBILITY,
+ independent_observations=True,
)
self.assertGreater(len(results_adj), 0)
for _name, res in results_adj.items():
def test_bootstrap_confidence_intervals_bounds_ordering(self):
"""Test bootstrap confidence intervals return ordered finite bounds."""
- test_data = self.make_stats_df(n=100, seed=SEEDS.BASE)
- results = bootstrap_confidence_intervals(test_data, ["reward", "pnl"], n_bootstrap=100)
+ test_data = self.make_stats_df(n=SCENARIOS.SAMPLE_SIZE_SMALL, seed=SEEDS.BASE)
+ results = bootstrap_confidence_intervals(
+ test_data,
+ ["reward", "pnl"],
+ n_bootstrap=STATISTICAL.BOOTSTRAP_DEFAULT_ITERATIONS,
+ independent_observations=True,
+ )
for metric, (mean, ci_low, ci_high) in results.items():
self.assertFinite(mean, name=f"mean[{metric}]")
self.assertFinite(ci_low, name=f"ci_low[{metric}]")
small = self._shift_scale_df(SCENARIOS.SAMPLE_SIZE_SMALL - 20)
large = self._shift_scale_df(SCENARIOS.SAMPLE_SIZE_LARGE)
- res_small = bootstrap_confidence_intervals(small, ["reward"], n_bootstrap=400)
- res_large = bootstrap_confidence_intervals(large, ["reward"], n_bootstrap=400)
+ res_small = bootstrap_confidence_intervals(
+ small, ["reward"], n_bootstrap=400, independent_observations=True
+ )
+ res_large = bootstrap_confidence_intervals(
+ large, ["reward"], n_bootstrap=400, independent_observations=True
+ )
_, lo_s, hi_s = next(iter(res_small.values()))
_, lo_l, hi_l = next(iter(res_large.values()))
hw_small = (hi_s - lo_s) / 2.0
self.assertFinite(hw_large, name="hw_large")
self.assertLess(hw_large, hw_small * 0.55)
- # Owns invariant: statistics-constant-dist-widened-ci-113a
- def test_stats_bootstrap_constant_distribution_widening(self):
- """Invariant 113 (non-strict): constant distribution CI widened with warning (positive epsilon width)."""
-
- df = self._const_df(80)
- with assert_diagnostic_warning(
- ["degenerate", "bootstrap", "CI"],
- warning_category=RewardDiagnosticsWarning,
- strict_mode=False,
- ):
- res = bootstrap_confidence_intervals(
+ def test_stats_bootstrap_constant_distribution_exact_bounds(self):
+ """Constants retain their exact degenerate interval in both diagnostic modes."""
+ df = pd.DataFrame({"reward": np.full(40, 2.5)})
+ for strict in (False, True):
+ with self.subTest(strict_diagnostics=strict):
+ res = bootstrap_confidence_intervals(
+ df,
+ ["reward"],
+ n_bootstrap=SCENARIOS.BOOTSTRAP_EXTENDED_ITERATIONS,
+ strict_diagnostics=strict,
+ independent_observations=True,
+ )
+ self.assertEqual(res, {"reward": (2.5, 2.5, 2.5)})
+
+ def test_stats_bootstrap_percentiles_need_not_contain_mean(self):
+ """A single resample yields its own mean, not a widened interval around the estimate."""
+ df = pd.DataFrame({"reward": np.arange(10, dtype=float)})
+ for strict in (False, True):
+ with self.subTest(strict_diagnostics=strict):
+ with self.assertWarns(RewardDiagnosticsWarning):
+ res = bootstrap_confidence_intervals(
+ df,
+ ["reward"],
+ n_bootstrap=1,
+ seed=SEEDS.BASE,
+ strict_diagnostics=strict,
+ independent_observations=True,
+ )
+ self.assertEqual(res["reward"], (4.5, 3.7, 3.7))
+
+ def test_inference_helpers_require_independent_observations(self):
+ """Inferential helpers reject dependent trajectory observations."""
+ df = pd.DataFrame({"reward": np.arange(10, dtype=float)})
+ with self.assertRaisesRegex(ValueError, "independent_observations=True"):
+ statistical_hypothesis_tests(df, independent_observations=False)
+ with self.assertRaisesRegex(ValueError, "independent_observations=True"):
+ bootstrap_confidence_intervals(
df,
- ["reward", "pnl"],
- n_bootstrap=200,
- confidence_level=0.95,
- strict_diagnostics=False,
- )
- for _metric, (mean, lo, hi) in res.items():
- self.assertLess(
- lo,
- hi,
- "Degenerate CI should be widened (lo < hi) under non-strict diagnostics",
- )
- width = hi - lo
- self.assertGreater(width, 0.0)
- self.assertLessEqual(
- width, STAT_TOL.CI_WIDTH_EPSILON, "Width should be small epsilon range"
+ ["reward"],
+ n_bootstrap=SCENARIOS.BOOTSTRAP_EXTENDED_ITERATIONS,
+ independent_observations=False,
)
- # Mean should be centered (approx) within widened bounds
- self.assertGreaterEqual(mean, lo)
- self.assertLessEqual(mean, hi)
-
- # Owns invariant: statistics-constant-dist-strict-omit-113b
- def test_stats_bootstrap_constant_distribution_strict_diagnostics(self):
- """Invariant 113 (strict): constant distribution metrics are omitted (no widened CI returned)."""
- df = self._const_df(60)
- res = bootstrap_confidence_intervals(
- df, ["reward", "pnl"], n_bootstrap=150, confidence_level=0.95, strict_diagnostics=True
- )
- # Strict mode should omit constant metrics entirely
- self.assertTrue(
- all(m not in res for m in ["reward", "pnl"]),
- f"Strict diagnostics should omit constant metrics; got keys: {list(res.keys())}",
- )
+
+ def test_stats_bootstrap_rejects_invalid_bounds(self):
+ """Non-finite or reversed bounds are errors, never repaired by the validator."""
+ for bounds in (
+ (0.0, 1.0, -1.0),
+ (np.nan, 0.0, 1.0),
+ (0.0, -np.inf, 1.0),
+ (0.0, 0.0, np.inf),
+ ):
+ for strict in (False, True):
+ with (
+ self.subTest(bounds=bounds, strict_diagnostics=strict),
+ self.assertRaises(AssertionError),
+ ):
+ reward_space_analysis._validate_bootstrap_results(
+ {"reward": bounds}, strict_diagnostics=strict
+ )
+
+ def test_stats_diagnostics_rejects_fabricated_constant_fallbacks(self):
+ """A constant marker cannot turn invalid statistics into synthetic moments or R²."""
+ for key in ("reward_skewness", "reward_anderson_stat", "reward_qq_r_squared"):
+ for strict in (False, True):
+ with (
+ self.subTest(key=key, strict_diagnostics=strict),
+ self.assertRaises(AssertionError),
+ ):
+ reward_space_analysis._validate_distribution_diagnostics(
+ {"reward_constant": True, "reward_std": 0.0, key: np.nan},
+ strict_diagnostics=strict,
+ )
if __name__ == "__main__":
pnl_target=PARAMS.PROFIT_AIM * PARAMS.RISK_REWARD_RATIO,
current_duration_ratio=current_dur,
next_pnl=next_pnl,
+ entry_pnl=0.0,
next_duration_ratio=next_dur,
base_factor=PARAMS.BASE_FACTOR,
risk_reward_ratio=PARAMS.RISK_REWARD_RATIO,
("tanh", [0.0, 1.0, -1.0], [0.0, math.tanh(1.0), math.tanh(-1.0)]),
# softsign transform: x / (1 + |x|) in (-1, 1)
("softsign", [0.0, 1.0, -1.0], [0.0, 0.5, -0.5]),
- # asinh transform: x / sqrt(1 + x^2) in (-1, 1)
- ("asinh", [0.0], [0.0]), # More complex calculations tested separately
+ # softsign_sqrt: x / sqrt(1 + x^2) in (-1, 1)
+ ("softsign_sqrt", [0.0, 1.0, -1.0], [0.0, 1 / math.sqrt(2), -1 / math.sqrt(2)]),
# arctan transform: (2/π) · arctan(x) in (-1, 1)
("arctan", [0.0, 1.0], [0.0, 2.0 / math.pi * math.atan(1.0)]),
# sigmoid transform: 2σ(x) - 1, σ(x) = 1/(1 + e^(-x)) in (-1, 1)
msg=f"{transform_name}(0.0) should equal 0.0",
)
- def test_transform_asinh_symmetry(self):
- """Test asinh transform symmetry: asinh(x) = -asinh(-x)."""
+ def test_transform_softsign_sqrt_symmetry(self):
+ """The bounded square-root softsign transform is odd."""
test_values = [1.2345, 2.0, 5.0, 0.1]
for test_val in test_values:
with self.subTest(input=test_val):
- pos_result = apply_transform("asinh", test_val)
- neg_result = apply_transform("asinh", -test_val)
+ pos_result = apply_transform("softsign_sqrt", test_val)
+ neg_result = apply_transform("softsign_sqrt", -test_val)
self.assertAlmostEqualFloat(
pos_result,
-neg_result,
tolerance=TOLERANCE.IDENTITY_STRICT,
- msg=f"asinh({test_val}) should equal -asinh({-test_val})",
+ msg=f"softsign_sqrt({test_val}) should equal -softsign_sqrt({-test_val})",
)
def test_transform_sigmoid_extreme_behavior(self):
transform_descriptions = [
("tanh", "Hyperbolic tangent"),
("softsign", "Softsign activation"),
- ("asinh", "Inverse hyperbolic sine normalized"),
+ ("softsign_sqrt", "Square-root softsign"),
("arctan", "Scaled arctangent"),
("sigmoid", "Scaled sigmoid"),
("clip", "Hard clipping"),
--- /dev/null
+"""Runtime regressions for liquidation-equity returns; requires the RL QA image."""
+
+import math
+import unittest
+
+import numpy as np
+import pandas as pd
+
+from ReforceXY.user_data.freqaimodels.ReforceXY import Actions, MyRLEnv
+
+
+class PortfolioReturnsTest(unittest.TestCase):
+ def make_env(self, prices, *, compound=True, fee=0.0015):
+ frame = pd.DataFrame({"open": prices})
+ env = MyRLEnv(
+ df=frame.copy(),
+ prices=frame,
+ df_raw=frame.copy(),
+ window_size=1,
+ reward_kwargs={"rr": 2.0, "profit_aim": 0.03},
+ fee=fee,
+ can_short=True,
+ config={
+ "stake_amount": "unlimited" if compound else 100.0,
+ "freqai": {
+ "rl_config": {
+ "add_state_info": False,
+ "max_training_drawdown_pct": 0.99,
+ "model_reward_parameters": {},
+ }
+ },
+ },
+ live=True,
+ )
+ self.addCleanup(env.close)
+ env.reset()
+ return env
+
+ def test_entry_does_not_capture_pre_entry_move(self):
+ env = self.make_env([100.0, 110.0, 110.0, 110.0, 110.0])
+ _, _, _, _, entry = env.step(Actions.Long_enter.value)
+ self.assertAlmostEqual(entry["most_recent_return"], -2 * math.log1p(env.fee), places=5)
+ for action in (Actions.Neutral, Actions.Long_exit):
+ _, _, _, _, info = env.step(action.value)
+ self.assertEqual(info["most_recent_return"], 0.0)
+ self.assertEqual(info["most_recent_profit"], 0.0)
+ self.assertAlmostEqual(np.expm1(env.portfolio_log_returns.sum()), env._total_profit - 1)
+
+ def test_transitions_match_equity_for_both_staking_modes_and_directions(self):
+ prices = [100.0, 100.0, 110.0, 105.0, 115.0, 120.0, 118.0, 117.0, 117.0]
+ for short in (False, True):
+ for compound in (False, True):
+ with self.subTest(short=short, compound=compound):
+ env = self.make_env(prices, compound=compound)
+ enter = Actions.Short_enter if short else Actions.Long_enter
+ exit_action = Actions.Short_exit if short else Actions.Long_exit
+ capital = previous_equity = 1.0
+ entry_price = None
+ for action in (
+ enter,
+ Actions.Neutral,
+ exit_action,
+ Actions.Neutral,
+ enter,
+ exit_action,
+ ):
+ _, _, _, _, info = env.step(action.value)
+ fill_price = prices[info["tick"] - 1]
+ mark_price = prices[info["tick"]]
+ fee_factor = (1 + env.fee) ** 2
+ if action == enter:
+ entry_price = fill_price
+ if action == exit_action:
+ price_ratio = fill_price / entry_price
+ exit_pnl = (
+ 1 - price_ratio * fee_factor
+ if short
+ else price_ratio / fee_factor - 1
+ )
+ equity = capital * (1 + exit_pnl) if compound else capital + exit_pnl
+ capital = equity
+ entry_price = None
+ else:
+ pnl = 0.0
+ if entry_price is not None:
+ price_ratio = mark_price / entry_price
+ pnl = (
+ 1 - price_ratio * fee_factor
+ if short
+ else price_ratio / fee_factor - 1
+ )
+ equity = capital * (1 + pnl) if compound else capital + pnl
+ expected_log = math.log(equity / previous_equity)
+ self.assertAlmostEqual(
+ env.portfolio_log_returns[info["tick"]], expected_log
+ )
+ self.assertEqual(info["most_recent_return"], round(expected_log, 5))
+ self.assertEqual(
+ info["most_recent_profit"], round(equity / previous_equity - 1, 5)
+ )
+ previous_equity = equity
+ self.assertAlmostEqual(np.expm1(env.portfolio_log_returns.sum()), capital - 1)
+ self.assertAlmostEqual(env._total_profit, capital)
+ self.assertEqual(env.get_most_recent_return(), 0.0)
+ self.assertEqual(env.get_most_recent_profit(), 0.0)
+
+ def test_terminal_liquidation_realizes_reward_and_equity_once(self):
+ for short in (False, True):
+ for compound in (False, True):
+ for drawdown in (False, True):
+ with self.subTest(short=short, compound=compound, drawdown=drawdown):
+ terminal_price = 110.0 if short else 90.0
+ prices = [100.0, 100.0, 100.0, terminal_price]
+ if drawdown:
+ prices.append(terminal_price)
+ env = self.make_env(prices, compound=compound)
+ env.max_drawdown = 0.95 if drawdown else 0.01
+ env.step((Actions.Short_enter if short else Actions.Long_enter).value)
+ _, reward, terminated, _, info = env.step(Actions.Neutral.value)
+ fee_factor = (1 + env.fee) ** 2
+ expected_pnl = (
+ 1 - terminal_price / 100 * fee_factor
+ if short
+ else terminal_price / 100 / fee_factor - 1
+ )
+ self.assertTrue(terminated)
+ self.assertEqual(env._position.name, "Neutral")
+ self.assertEqual(env._position_history[-1].name, "Neutral")
+ self.assertLess(reward, 0.0)
+ self.assertAlmostEqual(env._total_profit, 1 + expected_pnl)
+ self.assertAlmostEqual(
+ np.expm1(env.portfolio_log_returns.sum()), expected_pnl
+ )
+ self.assertAlmostEqual(info["exit_pnl"], expected_pnl)
+ self.assertTrue(info["terminal_liquidation"])
+ self.assertEqual(len(env.trade_history), 2)
+ self.assertEqual(
+ env.trade_history[-1]["type"],
+ "short_exit" if short else "long_exit",
+ )
+ self.assertEqual(env.trade_history[-1]["tick"], info["execution_tick"])
+ self.assertEqual(env.trade_history[-1]["price"], terminal_price)
+ self.assertAlmostEqual(env.trade_history[-1]["profit"], expected_pnl)
+ self.assertAlmostEqual(
+ reward,
+ sum(
+ info[key]
+ for key in (
+ "reward_exit",
+ "reward_hold",
+ "reward_idle",
+ "reward_invalid",
+ "reward_shaping",
+ "reward_entry_additive",
+ "reward_exit_additive",
+ )
+ ),
+ places=4,
+ )
+ history = env.get_env_history()
+ terminal_row = history.loc[
+ history["execution_tick"] == info["execution_tick"]
+ ].iloc[0]
+ self.assertEqual(terminal_row["reward_exit"], info["reward_exit"])
+ self.assertTrue(terminal_row["terminal_liquidation"])
+
+ def test_history_keeps_transitions_separate_from_trade_events(self):
+ env = self.make_env([100.0, 100.0, 110.0, 110.0])
+ _, _, _, _, entry_info = env.step(Actions.Long_enter.value)
+ _, _, _, _, exit_info = env.step(Actions.Long_exit.value)
+ history = env.get_env_history()
+ self.assertEqual(len(history), 2)
+ self.assertNotIn("type", history.columns)
+ self.assertNotIn("profit", history.columns)
+ entry_row = history.loc[history["execution_tick"] == 1].iloc[0]
+ self.assertEqual(entry_row["reward_exit"], entry_info["reward_exit"])
+ self.assertEqual(entry_row["reward_exit"], 0.0)
+ self.assertEqual(entry_row["action"], Actions.Long_enter.value)
+ exit_row = history.loc[history["execution_tick"] == 2].iloc[0]
+ self.assertGreater(exit_row["reward_exit"], 0.0)
+ self.assertEqual(exit_row["reward_exit"], exit_info["reward_exit"])
+ self.assertEqual(
+ [(event["tick"], event["type"]) for event in env.trade_history],
+ [(1, "long_enter"), (2, "long_exit")],
+ )
+
+ def test_terminal_entry_preserves_two_events_and_plot_markers(self):
+ env = self.make_env([100.0, 100.0, 90.0])
+ _, _, terminated, truncated, info = env.step(Actions.Long_enter.value)
+ expected_pnl = 90.0 / 100.0 / (1.0 + env.fee) ** 2 - 1.0
+
+ self.assertTrue(terminated)
+ self.assertFalse(truncated)
+ self.assertTrue(info["terminal_liquidation"])
+ history = env.get_env_history()
+ self.assertEqual(len(history), 1)
+ self.assertEqual(history.iloc[0]["execution_tick"], info["execution_tick"])
+ self.assertEqual(history.iloc[0]["open"], 90.0)
+
+ self.assertEqual(len(env.trade_history), 2)
+ self.assertEqual(
+ [(event["tick"], event["type"]) for event in env.trade_history],
+ [(info["execution_tick"], "long_enter"), (info["execution_tick"], "long_exit")],
+ )
+ self.assertEqual(
+ [event["price"] for event in env.trade_history],
+ [100.0, 90.0],
+ )
+ self.assertEqual(env.trade_history[0]["profit"], 0.0)
+ self.assertAlmostEqual(env.trade_history[1]["profit"], expected_pnl)
+
+ figure = env.get_env_plot()
+ marker_lines = [
+ line
+ for line in figure.axes[0].lines
+ if line.get_linestyle() == "None" and len(line.get_xdata()) == 1
+ ]
+ self.assertEqual([line.get_marker() for line in marker_lines], ["^", "."])
+ self.assertEqual([line.get_xdata()[0] for line in marker_lines], [1, 1])
+
+ def test_nonpositive_equity_is_not_reported_as_zero_return(self):
+ env = self.make_env([100.0, 100.0, 100.0, 300.0, 300.0], fee=0.0)
+ env.step(Actions.Short_enter.value)
+ _, _, terminated, _, info = env.step(Actions.Neutral.value)
+ self.assertTrue(terminated)
+ self.assertTrue(math.isnan(info["most_recent_return"]))
+ self.assertTrue(math.isnan(info["most_recent_profit"]))
+
+
+if __name__ == "__main__":
+ unittest.main()
--- /dev/null
+"""Regressions for live observations, HPO options and historic prediction alignment."""
+
+import copy
+import math
+import tempfile
+import unittest
+from datetime import datetime as dt
+from datetime import timedelta, timezone
+from pathlib import Path
+from types import SimpleNamespace
+from unittest import mock
+
+import numpy as np
+import pandas as pd
+from freqtrade.enums import RunMode
+from freqtrade.freqai.data_drawer import FreqaiDataDrawer
+from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
+from sb3_contrib import MaskablePPO
+
+from ReforceXY.user_data.freqaimodels.ReforceXY import (
+ Actions,
+ MyRLEnv,
+ ReforceXY,
+ convert_optuna_params_to_model_params,
+ deepmerge,
+)
+
+
+class RecordingPolicy:
+ def __init__(self):
+ self.observations = []
+ self.masks = []
+
+ def predict(self, observation, **kwargs):
+ self.observations.append(observation.copy())
+ self.masks.append(kwargs.get("action_masks"))
+ return np.array([0]), None
+
+
+def model_config(path):
+ return {
+ "user_data_dir": Path(path),
+ "timeframe": "5m",
+ "stake_amount": "unlimited",
+ "runmode": RunMode.DRY_RUN,
+ "exchange": {"pair_whitelist": ["BTC/USDT"]},
+ "freqai": {
+ "enabled": True,
+ "identifier": "contract-test",
+ "train_period_days": 1,
+ "backtest_period_days": 1,
+ "conv_width": 1,
+ "activate_tensorboard": False,
+ "feature_parameters": {
+ "include_timeframes": ["5m"],
+ "include_corr_pairlist": [],
+ "label_period_candles": 1,
+ "principal_component_analysis": False,
+ "noise_standard_deviation": 0,
+ "buffer_train_data_candles": 0,
+ "shuffle_after_split": False,
+ },
+ "data_split_parameters": {"test_size": 0.25, "shuffle": False},
+ "model_training_parameters": {
+ "n_steps": 8,
+ "batch_size": 8,
+ "n_epochs": 1,
+ "device": "cpu",
+ "policy_kwargs": {"net_arch": [8]},
+ },
+ "rl_config": {
+ "model_type": "MaskablePPO",
+ "policy_type": "MlpPolicy",
+ "cpu_count": 1,
+ "drop_ohlc_from_features": False,
+ "model_reward_parameters": {"rr": 2.0, "profit_aim": 0.03},
+ "train_cycles": 1,
+ "n_envs": 1,
+ "n_eval_envs": 1,
+ "n_eval_steps": 16,
+ "n_eval_episodes": 1,
+ "check_envs": False,
+ "add_state_info": False,
+ },
+ },
+ }
+
+
+class ReviewContractsTest(unittest.TestCase):
+ def model(self, *, hold=False, hpo=False):
+ temp = tempfile.TemporaryDirectory()
+ self.addCleanup(temp.cleanup)
+ config = model_config(temp.name)
+ config["freqai"]["rl_config"]["model_reward_parameters"]["hold_potential_enabled"] = hold
+ config["freqai"]["continual_learning"] = hpo
+ config["freqai"]["rl_config_optuna"] = {"enabled": hpo}
+ model = ReforceXY(config=config)
+ model.live = True
+ model.can_short = True
+ model.get_state_info = lambda pair: (1.0, 0.05, 12)
+ self.addCleanup(model.close_envs)
+ return model
+
+ def test_training_preserves_raw_prices_and_returns_best_checkpoint(self):
+ for drop in (False, True):
+ with self.subTest(drop_ohlc_from_features=drop), tempfile.TemporaryDirectory() as temp:
+ config = model_config(temp)
+ config["freqai"]["rl_config"]["drop_ohlc_from_features"] = drop
+ model = ReforceXY(config=config)
+ model.live = True
+ model.can_short = False
+ model.get_state_info = lambda pair: (0.5, 0.0, 0)
+ self.addCleanup(model.close_envs)
+ dk = FreqaiDataKitchen(config, live=True, pair="BTC/USDT")
+ dk.data_path = Path(temp) / "fit"
+ dk.data_path.mkdir()
+ dk.model_filename = "cb_btc_123"
+ dk.label_list = ["&-action"]
+ frame = pd.DataFrame(
+ {
+ "date": pd.date_range("2026-01-01", periods=64, freq="5min", tz="UTC"),
+ "%-feature": np.sin(np.arange(64)),
+ "&-action": np.zeros(64),
+ }
+ )
+ for column in ("open", "high", "low", "close"):
+ frame[f"%-raw_{column}"] = 100 + np.arange(64) * 0.1
+ dk.training_features_list = [column for column in frame if column.startswith("%")]
+ from sb3_contrib.common.maskable.evaluation import evaluate_policy
+
+ evaluated_updates = []
+
+ def evaluate_final(policy, environment, updates=evaluated_updates, **kwargs):
+ updates.append(policy._n_updates)
+ return evaluate_policy(policy, environment, **kwargs)
+
+ with mock.patch(
+ "ReforceXY.user_data.freqaimodels.ReforceXY.evaluate_policy",
+ side_effect=evaluate_final,
+ ):
+ trained = model.train(frame, dk.pair, dk)
+ self.assertEqual(len(evaluated_updates), 1)
+ self.assertGreater(evaluated_updates[0], 0)
+ checkpoint = MaskablePPO.load(dk.data_path / "best_model.zip")
+ for name, value in trained.policy.state_dict().items():
+ np.testing.assert_array_equal(
+ value.cpu().numpy(), checkpoint.policy.state_dict()[name].cpu().numpy()
+ )
+ prediction = model.rl_model_predict(
+ dk.data_dictionary["train_features"].tail(1), dk, trained
+ )
+ self.assertIn(int(prediction.iloc[0, 0]), (0, 1))
+ train_env, eval_env = model._get_train_and_eval_environments(
+ dk,
+ train_df=dk.data_dictionary["train_features"],
+ test_df=dk.data_dictionary["test_features"],
+ prices_train=dk.data_dictionary["train_prices"],
+ prices_test=dk.data_dictionary["test_prices"],
+ model_params={"gamma": 0.91},
+ )
+ try:
+ np.testing.assert_allclose(
+ train_env.get_attr("prices")[0]["open"], frame["%-raw_open"].iloc[:48]
+ )
+ self.assertEqual(train_env.get_attr("_potential_gamma"), [0.91])
+ finally:
+ train_env.close()
+ eval_env.close()
+
+ def test_training_without_holdout_keeps_all_rows_and_no_eval_environment(self):
+ with tempfile.TemporaryDirectory() as temp:
+ config = model_config(temp)
+ config["freqai"]["data_split_parameters"]["test_size"] = 0
+ model = ReforceXY(config=config)
+ model.live = True
+ model.can_short = False
+ model.get_state_info = lambda pair: (0.5, 0.0, 0)
+ self.addCleanup(model.close_envs)
+ dk = FreqaiDataKitchen(config, live=True, pair="BTC/USDT")
+ dk.data_path = Path(temp) / "fit"
+ dk.data_path.mkdir()
+ dk.model_filename = "cb_btc_no_holdout"
+ dk.label_list = ["&-action"]
+ frame = pd.DataFrame(
+ {
+ "date": pd.date_range("2026-01-01", periods=64, freq="5min", tz="UTC"),
+ "%-feature": np.sin(np.arange(64)),
+ "&-action": np.zeros(64),
+ }
+ )
+ for column in ("open", "high", "low", "close"):
+ frame[f"%-raw_{column}"] = 100 + np.arange(64) * 0.1
+ dk.training_features_list = [column for column in frame if column.startswith("%")]
+
+ trained = model.train(frame, dk.pair, dk)
+
+ self.assertIsNotNone(trained)
+ self.assertEqual(len(dk.data_dictionary["train_features"]), len(frame))
+ self.assertEqual(len(dk.data_dictionary["test_features"]), 0)
+ self.assertIsNone(model.eval_env)
+
+ def test_live_stacking_preserves_frames_and_resets_for_new_model(self):
+ model = self.model()
+ model.frame_stacking = 2
+ policy = RecordingPolicy()
+ dk = SimpleNamespace(pair="BTC/USDT", label_list=["&-action"], data_dictionary={})
+ dk.data_dictionary["prediction_dates"] = pd.DataFrame(
+ {"date": pd.date_range("2026-01-01", periods=1, tz="UTC")}
+ )
+ model.rl_model_predict(pd.DataFrame({"f": [10.0]}), dk, policy)
+ dk.data_dictionary["prediction_dates"] += pd.Timedelta(minutes=5)
+ model.rl_model_predict(pd.DataFrame({"f": [20.0]}), dk, policy)
+ np.testing.assert_array_equal(policy.observations[-1], [[[10.0, 20.0]]])
+ replacement = RecordingPolicy()
+ model.rl_model_predict(pd.DataFrame({"f": [30.0]}), dk, replacement)
+ np.testing.assert_array_equal(replacement.observations[-1], [[[0.0, 30.0]]])
+ dk.do_predict = np.ones(2, dtype=int)
+ dk.data_dictionary["prediction_dates"] = pd.DataFrame(
+ {"date": pd.date_range("2026-01-02", periods=2, freq="5min", tz="UTC")}
+ )
+ model.rl_model_predict(pd.DataFrame({"f": [40.0, 50.0]}), dk, replacement)
+ np.testing.assert_array_equal(replacement.observations[-2], [[[0.0, 40.0]]])
+ np.testing.assert_array_equal(replacement.observations[-1], [[[40.0, 50.0]]])
+ dk.pair = "ETH/USDT"
+ dk.do_predict = np.ones(1, dtype=int)
+ dk.data_dictionary["prediction_dates"] = dk.data_dictionary["prediction_dates"][-1:]
+ model.rl_model_predict(pd.DataFrame({"f": [60.0]}), dk, replacement)
+ np.testing.assert_array_equal(replacement.observations[-1], [[[0.0, 60.0]]])
+
+ def test_live_mask_uses_open_position_without_state_features(self):
+ model = self.model()
+ policy = RecordingPolicy()
+ model.rl_model_predict(
+ pd.DataFrame({"f": [1.0]}),
+ SimpleNamespace(pair="BTC/USDT", label_list=["&-action"]),
+ policy,
+ )
+ np.testing.assert_array_equal(policy.masks[-1], [True, False, True, False, False])
+
+ def test_hold_potential_has_matching_training_and_inference_shapes(self):
+ model = self.model(hold=True)
+ features = pd.DataFrame({"f": np.arange(16, dtype=float)})
+ prices = pd.DataFrame({"open": np.full(16, 100.0)})
+ env = MyRLEnv(df=features, prices=prices, **model.pack_env_dict("BTC/USDT"))
+ self.addCleanup(env.close)
+ policy = MaskablePPO("MlpPolicy", env, n_steps=8, batch_size=8, device="cpu")
+ prediction = model.rl_model_predict(
+ features.tail(1), SimpleNamespace(pair="BTC/USDT", label_list=["&-action"]), policy
+ )
+ self.assertIn(int(prediction.iloc[0, 0]), (0, 2))
+ model.live = False
+ with self.assertRaises(ValueError):
+ model.pack_env_dict("BTC/USDT")
+
+ def test_hpo_uses_cold_candidate_and_continues_in_frozen_coordinates(self):
+ from freqtrade.freqai.data_drawer import FEATURE_PIPELINE, METADATA
+ from optuna import create_study
+ from sb3_contrib.common.maskable.evaluation import evaluate_policy
+
+ with tempfile.TemporaryDirectory() as temp:
+ config = model_config(temp)
+ config["freqai"]["continual_learning"] = True
+ config["freqai"]["model_training_parameters"]["gamma"] = 0.91
+
+ def kitchen(window):
+ dk = FreqaiDataKitchen(config, live=True, pair="BTC/USDT")
+ dk.data_path = Path(temp) / window
+ dk.data_path.mkdir()
+ dk.model_filename = f"cb_btc_{window}"
+ dk.label_list = ["&-action"]
+ dk.training_features_list = [column for column in frame if column.startswith("%")]
+ return dk
+
+ frame = pd.DataFrame(
+ {
+ "date": pd.date_range("2026-01-01", periods=64, freq="5min", tz="UTC"),
+ "%-feature": np.sin(np.arange(64)),
+ "&-action": np.zeros(64),
+ }
+ )
+ for column in ("open", "high", "low", "close"):
+ frame[f"%-raw_{column}"] = 100 + np.arange(64) * 0.1
+
+ initial = ReforceXY(config=config)
+ initial.live = True
+ initial.can_short = False
+ self.addCleanup(initial.close_envs)
+ previous_dk = kitchen("initial")
+ deployed = initial.train(frame, previous_dk.pair, previous_dk)
+ deployed_updates = deployed._n_updates
+ deployed_pipeline = previous_dk.feature_pipeline
+ initial_features = previous_dk.data_dictionary["train_features"].copy()
+
+ config["freqai"]["rl_config_optuna"] = {"enabled": True}
+ model = ReforceXY(config=config)
+ model.live = True
+ model.can_short = False
+ self.addCleanup(model.close_envs)
+ model.dd.model_dictionary[previous_dk.pair] = deployed
+ model.dd.meta_data_dictionary[previous_dk.pair] = {
+ METADATA: {
+ **previous_dk.data,
+ "training_features_list": previous_dk.training_features_list,
+ "label_list": previous_dk.label_list,
+ },
+ FEATURE_PIPELINE: deployed_pipeline,
+ }
+
+ shifted = frame.copy()
+ shifted["date"] += pd.Timedelta(days=1)
+ shifted["%-feature"] += 10
+ for column in ("open", "high", "low", "close"):
+ shifted[f"%-raw_{column}"] += 20
+ dk = kitchen("continued")
+ expected_features, _, _ = deployed_pipeline.transform(
+ shifted[dk.training_features_list].copy()
+ )
+ selected_params = {**model.get_model_params(), "gamma": 0.999}
+ evaluations = []
+
+ def evaluate_final(policy, environment, **kwargs):
+ training_env = policy.get_env()
+ evaluations.append(
+ {
+ "policy": policy,
+ "updates": policy._n_updates,
+ "gamma": policy.gamma,
+ "train_features": training_env.get_attr("signal_features")[0].copy(),
+ "test_features": environment.get_attr("signal_features")[0].copy(),
+ "train_prices": training_env.get_attr("prices")[0].copy(),
+ "test_prices": environment.get_attr("prices")[0].copy(),
+ "train_gamma": training_env.get_attr("_potential_gamma"),
+ "test_gamma": environment.get_attr("_potential_gamma"),
+ }
+ )
+ return evaluate_policy(policy, environment, **kwargs)
+
+ def optimize_once(current_dk, total_timesteps, prices_train, prices_test):
+ trial = create_study(direction="maximize").ask()
+ with mock.patch.object(model, "get_optuna_params", return_value=selected_params):
+ score = model.objective(
+ trial, current_dk, total_timesteps, prices_train, prices_test
+ )
+ self.assertTrue(np.isfinite(score))
+ return selected_params
+
+ with (
+ mock.patch.object(model, "optimize", side_effect=optimize_once),
+ mock.patch(
+ "ReforceXY.user_data.freqaimodels.ReforceXY.evaluate_policy",
+ side_effect=evaluate_final,
+ ),
+ ):
+ continued = model.train(shifted, dk.pair, dk)
+
+ self.assertEqual(len(evaluations), 2)
+ candidate, resumed = evaluations
+ self.assertIsNot(candidate["policy"], deployed)
+ self.assertEqual(candidate["gamma"], 0.999)
+ self.assertGreater(candidate["updates"], 0)
+ self.assertIsNot(resumed["policy"], deployed)
+ self.assertEqual(resumed["gamma"], 0.91)
+ self.assertGreater(resumed["updates"], deployed_updates)
+ self.assertEqual(continued.gamma, 0.91)
+ np.testing.assert_allclose(candidate["train_features"], initial_features, atol=1e-12)
+ self.assertFalse(np.allclose(candidate["train_features"], resumed["train_features"]))
+ for split, rows in (("train", slice(None, 48)), ("test", slice(48, None))):
+ np.testing.assert_allclose(
+ resumed[f"{split}_features"], expected_features.iloc[rows]
+ )
+ np.testing.assert_allclose(
+ dk.data_dictionary[f"{split}_features"], expected_features.iloc[rows]
+ )
+ for evaluation in evaluations:
+ self.assertEqual(evaluation[f"{split}_gamma"], [evaluation["gamma"]])
+ for column in ("open", "high", "low", "close"):
+ np.testing.assert_allclose(
+ evaluation[f"{split}_prices"][column],
+ shifted[f"%-raw_{column}"].iloc[rows],
+ )
+ frozen_after, _, _ = dk.feature_pipeline.transform(
+ shifted[dk.training_features_list].copy()
+ )
+ np.testing.assert_allclose(frozen_after, expected_features)
+
+ def test_continuation_does_not_mutate_cached_deployment_on_success_or_failure(self):
+ from freqtrade.freqai.data_drawer import FEATURE_PIPELINE, METADATA
+ from sb3_contrib.common.maskable.evaluation import evaluate_policy
+
+ model = self.model()
+ model.continual_learning = True
+ model.can_short = False
+ frame = pd.DataFrame(
+ {
+ "date": pd.date_range("2026-01-01", periods=64, freq="5min", tz="UTC"),
+ "%-feature": np.sin(np.arange(64)),
+ "&-action": np.zeros(64),
+ }
+ )
+ for column in ("open", "high", "low", "close"):
+ frame[f"%-raw_{column}"] = 100 + np.arange(64) * 0.1
+
+ def kitchen(window):
+ dk = FreqaiDataKitchen(model.config, live=True, pair="BTC/USDT")
+ dk.data_path = Path(model.config["user_data_dir"]) / window
+ dk.data_path.mkdir()
+ dk.model_filename = f"cb_btc_{window}"
+ dk.label_list = ["&-action"]
+ dk.training_features_list = [column for column in frame if column.startswith("%")]
+ return dk
+
+ previous_dk = kitchen("initial")
+ deployed = model.train(frame, previous_dk.pair, previous_dk)
+ deployed.save(previous_dk.data_path / "deployed.zip")
+ model.dd.model_dictionary[previous_dk.pair] = deployed
+ model.dd.meta_data_dictionary[previous_dk.pair] = {
+ METADATA: {
+ **previous_dk.data,
+ "training_features_list": previous_dk.training_features_list,
+ "label_list": previous_dk.label_list,
+ },
+ FEATURE_PIPELINE: previous_dk.feature_pipeline,
+ }
+ weights = copy.deepcopy(deployed.policy.state_dict())
+ optimizer = copy.deepcopy(deployed.policy.optimizer.state_dict())
+ updates, timesteps, environment = deployed._n_updates, deployed.num_timesteps, deployed.env
+ observation = previous_dk.data_dictionary["train_features"].iloc[:1].to_numpy()
+ prediction = deployed.predict(observation, deterministic=True)[0]
+
+ for fail in (True, False):
+ with self.subTest(failure_after_gradients=fail):
+ dk = kitchen("failed" if fail else "continued")
+ evaluated = []
+
+ def evaluate_final(policy, env, evaluated=evaluated, fail=fail, **kwargs):
+ self.assertGreater(policy._n_updates, updates)
+ self.assertTrue(
+ any(
+ not np.array_equal(value.cpu().numpy(), weights[name].cpu().numpy())
+ for name, value in policy.policy.state_dict().items()
+ )
+ )
+ evaluated.append(policy)
+ if fail:
+ raise RuntimeError("Final evaluation failed after gradients")
+ return evaluate_policy(policy, env, **kwargs)
+
+ with mock.patch(
+ "ReforceXY.user_data.freqaimodels.ReforceXY.evaluate_policy",
+ side_effect=evaluate_final,
+ ):
+ if fail:
+ with self.assertRaises(RuntimeError):
+ model.train(frame, dk.pair, dk)
+ else:
+ continued = model.train(frame, dk.pair, dk)
+ self.assertIsNot(continued, deployed)
+ self.assertGreater(continued._n_updates, updates)
+ self.assertEqual(len(evaluated), 1)
+ self.assertIsNot(evaluated[0], deployed)
+ self.assertIs(model.dd.model_dictionary[dk.pair], deployed)
+ self.assertIs(deployed.env, environment)
+ self.assertEqual(
+ (deployed._n_updates, deployed.num_timesteps), (updates, timesteps)
+ )
+ np.testing.assert_array_equal(
+ deployed.predict(observation, deterministic=True)[0], prediction
+ )
+ for name, value in deployed.policy.state_dict().items():
+ np.testing.assert_array_equal(value.cpu().numpy(), weights[name].cpu().numpy())
+ current_optimizer = deployed.policy.optimizer.state_dict()
+ self.assertEqual(current_optimizer["param_groups"], optimizer["param_groups"])
+ for parameter, state in optimizer["state"].items():
+ for name, value in state.items():
+ np.testing.assert_array_equal(
+ current_optimizer["state"][parameter][name].cpu().numpy(),
+ value.cpu().numpy(),
+ )
+ transformed, _, _ = previous_dk.feature_pipeline.transform(
+ frame[previous_dk.training_features_list].copy()
+ )
+ np.testing.assert_allclose(
+ transformed.iloc[:48], previous_dk.data_dictionary["train_features"]
+ )
+
+ def test_loss_amplification_starts_at_configured_risk_threshold(self):
+ model = self.model()
+ features = pd.DataFrame({"f": np.zeros(6)})
+ prices = pd.DataFrame({"open": np.full(6, 100.0)})
+ env = MyRLEnv(df=features, prices=prices, **model.pack_env_dict("BTC/USDT"))
+ self.addCleanup(env.close)
+ params = {"win_reward_factor": 2.0}
+ target = env._pnl_target
+ threshold = target / env.rr
+ self.assertEqual(env._compute_pnl_target_coefficient(-threshold, target, params), 1.0)
+ self.assertGreater(env._compute_pnl_target_coefficient(1.01 * target, target, params), 1.0)
+ slight = env._compute_pnl_target_coefficient(-(threshold * 1.001), target, params)
+ self.assertGreater(slight, 1.0)
+ self.assertLess(slight, 1.1)
+ deep = env._compute_pnl_target_coefficient(-target, target, params)
+ self.assertGreater(deep, slight)
+ self.assertEqual(env._compute_pnl_target_coefficient(-0.5 * threshold, target, params), 1.0)
+
+ def test_step_returns_python_bools_for_each_termination_path(self):
+ model = self.model()
+
+ def make_env(prices):
+ frame = pd.DataFrame({"open": prices})
+ env = MyRLEnv(df=frame.copy(), prices=frame, **model.pack_env_dict("BTC/USDT"))
+ env.fee = 0.0
+ self.addCleanup(env.close)
+ env.reset()
+ return env
+
+ candle_env = make_env([100.0, 100.0, 100.0])
+ _, _, terminated, truncated, candle_info = candle_env.step(Actions.Neutral.value)
+ self.assertIs(type(terminated), bool)
+ self.assertTrue(terminated)
+ self.assertIs(type(truncated), bool)
+ self.assertFalse(truncated)
+ self.assertFalse(candle_info["terminal_liquidation"])
+
+ realized_env = make_env([100.0, 100.0, 100.0, 50.0, 50.0, 50.0])
+ realized_env.max_drawdown = -10.0
+ realized_env.step(Actions.Long_enter.value)
+ realized_env.step(Actions.Neutral.value)
+ realized_env.max_drawdown = 0.75
+ _, _, terminated, truncated, realized_info = realized_env.step(Actions.Long_exit.value)
+ self.assertIs(type(terminated), bool)
+ self.assertTrue(terminated)
+ self.assertIs(type(truncated), bool)
+ self.assertFalse(truncated)
+ self.assertLess(realized_env._total_profit, realized_env.max_drawdown)
+ self.assertFalse(realized_info["terminal_liquidation"])
+
+ unrealized_env = make_env([100.0, 100.0, 100.0, 50.0, 50.0, 50.0])
+ unrealized_env.max_drawdown = 0.75
+ _, _, first_terminated, first_truncated, _ = unrealized_env.step(Actions.Long_enter.value)
+ self.assertFalse(first_terminated)
+ self.assertFalse(first_truncated)
+ _, _, terminated, truncated, unrealized_info = unrealized_env.step(Actions.Neutral.value)
+ self.assertIs(type(terminated), bool)
+ self.assertTrue(terminated)
+ self.assertIs(type(truncated), bool)
+ self.assertFalse(truncated)
+ self.assertTrue(unrealized_info["terminal_liquidation"])
+
+ def test_null_target_kl_overrides_user_value(self):
+ params = {
+ "learning_rate": 0.0003,
+ "clip_range": 0.2,
+ "n_steps": 8,
+ "batch_size": 8,
+ "gamma": 0.95,
+ "ent_coef": 0.0,
+ "n_epochs": 1,
+ "gae_lambda": 0.95,
+ "max_grad_norm": 0.5,
+ "vf_coef": 0.5,
+ "target_kl": None,
+ }
+ effective = deepmerge(
+ {"target_kl": 0.03}, convert_optuna_params_to_model_params("PPO", params)
+ )
+ self.assertIsNone(effective["target_kl"])
+ del params["target_kl"]
+ effective = deepmerge(
+ {"target_kl": 0.03}, convert_optuna_params_to_model_params("PPO", params)
+ )
+ self.assertEqual(effective["target_kl"], 0.03)
+
+ def test_historic_hole_does_not_shift_actions(self):
+ dates = pd.date_range("2026-01-01", periods=4, freq="5min", tz="UTC")
+ drawer = object.__new__(FreqaiDataDrawer)
+ drawer.historic_predictions = {
+ "BTC/USDT": pd.DataFrame(
+ {"date_pred": dates[[0, 3]], "close_price": [100.0, 103.0], "&-action": [1, 2]}
+ )
+ }
+ drawer.model_return_values = {"BTC/USDT": drawer.historic_predictions["BTC/USDT"].copy()}
+ frame = pd.DataFrame(
+ {
+ "date": dates,
+ "close": [100.0, 101.0, 102.0, 103.0],
+ "high": [100.0, 101.0, 102.0, 103.0],
+ "low": [100.0, 101.0, 102.0, 103.0],
+ }
+ )
+ result = drawer.attach_return_values_to_return_dataframe("BTC/USDT", frame)
+ self.assertEqual(result["&-action"].iloc[0], 1)
+ self.assertTrue(pd.isna(result["&-action"].iloc[1]))
+ self.assertTrue(pd.isna(result["&-action"].iloc[2]))
+ self.assertEqual(result["&-action"].iloc[3], 2)
+ pd.testing.assert_series_equal(result["date"], frame["date"])
+
+ def test_historic_initialization_does_not_shift_actions(self):
+ dates = pd.date_range("2026-01-01", periods=3, freq="5min", tz="UTC")
+ drawer = object.__new__(FreqaiDataDrawer)
+ drawer.historic_predictions = {
+ "BTC/USDT": pd.DataFrame(
+ {"date_pred": dates[[0, 2]], "close_price": [100.0, 102.0], "&-action": [1, 2]}
+ )
+ }
+ drawer.model_return_values = {}
+ frame = pd.DataFrame({"date": dates, "close": [100.0, 101.0, 102.0]})
+ drawer.set_initial_return_values("BTC/USDT", pd.DataFrame({"&-action": [0, 0, 0]}), frame)
+ result = drawer.attach_return_values_to_return_dataframe("BTC/USDT", frame)
+ self.assertEqual(result["&-action"].iloc[0], 1)
+ self.assertTrue(pd.isna(result["&-action"].iloc[1]))
+ self.assertEqual(result["&-action"].iloc[2], 2)
+
+ def test_rejected_prediction_does_not_advance_virtual_position(self):
+ model = self.model()
+ model.live = False
+ model.CONV_WIDTH = 2
+
+ class EntryPolicy(RecordingPolicy):
+ def predict(self, observation, **kwargs):
+ super().predict(observation, **kwargs)
+ return np.array([1 if kwargs["action_masks"][1] else 0]), None
+
+ policy = EntryPolicy()
+ dk = SimpleNamespace(
+ pair="BTC/USDT", label_list=["&-action"], do_predict=np.array([1, 0, 1, 1])
+ )
+ prediction = model.rl_model_predict(
+ pd.DataFrame({"f": [10.0, 20.0, 30.0, 40.0]}, index=[11, 22, 33, 44]), dk, policy
+ )
+ np.testing.assert_array_equal(prediction["&-action"].iloc[1:], [1, 1, 1])
+ np.testing.assert_array_equal(dk.do_predict, [0, 0, 0, 1])
+ self.assertTrue(policy.masks[1][1])
+ self.assertTrue(policy.masks[2][1])
+ np.testing.assert_array_equal(policy.observations[1], [[[20.0], [30.0]]])
+
+ def test_execution_and_potential_match_returned_observation(self):
+ model = self.model(hold=True)
+ model.CONV_WIDTH = 2
+ features = pd.DataFrame({"f": np.arange(7, dtype=float) + 10})
+ prices = pd.DataFrame({"open": [100.0, 100.0, 100.0, 110.0, 99.0, 105.0, 106.0]})
+ env = MyRLEnv(df=features, prices=prices, **model.pack_env_dict("BTC/USDT"))
+ env.fee = 0.0
+ self.addCleanup(env.close)
+ observation, _ = env.reset()
+ np.testing.assert_array_equal(observation[:, 0], [10.0, 11.0])
+ observation, entry_reward, done, truncated, _ = env.step(1)
+ self.assertFalse(done)
+ self.assertFalse(truncated)
+ self.assertEqual(env.trade_history[-1]["tick"], 2)
+ self.assertEqual(env.trade_history[-1]["price"], 100.0)
+ np.testing.assert_array_equal(observation[:, 0], [11.0, 12.0])
+ observed_pnl = float(observation[-1, 1])
+ observed_duration = float(observation[-1, 3])
+ self.assertAlmostEqual(observed_pnl, 0.1)
+ self.assertEqual(observed_duration, 1.0)
+ hold_scale = ReforceXY.DEFAULT_HOLD_POTENTIAL_RATIO * ReforceXY.DEFAULT_BASE_FACTOR
+ expected_potential = (
+ hold_scale
+ * 0.5
+ * (
+ math.tanh(ReforceXY.DEFAULT_HOLD_POTENTIAL_GAIN * observed_pnl / env._pnl_target)
+ + math.tanh(
+ ReforceXY.DEFAULT_HOLD_POTENTIAL_GAIN
+ * observed_duration
+ / env.max_trade_duration_candles
+ )
+ )
+ )
+ self.assertAlmostEqual(entry_reward, env._potential_gamma * expected_potential)
+
+ _, exit_reward, exit_done, exit_truncated, _ = env.step(2)
+ self.assertFalse(exit_done)
+ self.assertFalse(exit_truncated)
+ self.assertEqual(env.trade_history[-1]["tick"], 3)
+ self.assertEqual(env.trade_history[-1]["price"], 110.0)
+ pnl_coefficient = 1.0 + 2.0 * math.tanh(2.0 * (observed_pnl / env._pnl_target - 1.0))
+ # The entry transition retains one PnL observation, so max equals min
+ # and the independent efficiency formula remains at its neutral value 1.0.
+ expected_base_exit = observed_pnl * ReforceXY.DEFAULT_BASE_FACTOR * pnl_coefficient
+ self.assertAlmostEqual(exit_reward, expected_base_exit - expected_potential, places=5)
+
+ env.step(1)
+ _, _, done, truncated, info = env.step(0)
+ self.assertTrue(done)
+ self.assertFalse(truncated)
+ self.assertEqual(env._current_tick, 6)
+ self.assertEqual(info["next_potential"], 0.0)
+
+ def test_pbrs_history_preserves_verifier_precision(self):
+ model = self.model(hold=True)
+ model.CONV_WIDTH = 2
+ features = pd.DataFrame({"f": np.arange(5, dtype=float)})
+ prices = pd.DataFrame({"open": np.full(5, 100.0)})
+ env = MyRLEnv(df=features, prices=prices, **model.pack_env_dict("BTC/USDT"))
+ env.fee = 0.0
+ self.addCleanup(env.close)
+ env.reset()
+
+ with mock.patch.object(env, "_compute_hold_potential", return_value=0.123456):
+ env.step(Actions.Long_enter.value)
+
+ row = env.get_env_history().iloc[-1]
+ expected_shaping = env._potential_gamma * row["next_potential"] - row["prev_potential"]
+ self.assertLessEqual(abs(row["reward_shaping"] - expected_shaping), 1e-6)
+ self.assertEqual(row["next_potential"], 0.123456)
+
+ def test_state_info_normalizes_leveraged_profit_ratio(self):
+ trade = SimpleNamespace(
+ pair="BTC/USDT",
+ is_short=False,
+ leverage=2.0,
+ open_date_utc=dt.now(timezone.utc),
+ calc_profit_ratio=lambda rate: 0.04,
+ )
+
+ exchange = SimpleNamespace(get_rate=lambda *args, **kwargs: 102.0)
+ other_pair = SimpleNamespace(pair="ETH/USDT", is_short=False, leverage=1.0)
+
+ model = self.model()
+ # Exercise the class implementation instead of the fixture's instance stub.
+ del model.get_state_info
+ model.data_provider = SimpleNamespace(_exchange=exchange)
+ with mock.patch(
+ "ReforceXY.user_data.freqaimodels.ReforceXY.Trade.get_trades_proxy",
+ return_value=[trade, other_pair],
+ ) as trades:
+ side, profit, duration = model.get_state_info("BTC/USDT")
+ trades.assert_called_once_with(is_open=True)
+ self.assertEqual(side, 1.0)
+ # calc_profit_ratio includes leverage (2x): 4% / leverage 2 -> normalized 2%.
+ self.assertAlmostEqual(profit, 0.02, places=10)
+ self.assertEqual(duration, 0)
+
+ def test_state_info_preserves_long_side_without_exchange(self):
+ trade = SimpleNamespace(
+ pair="BTC/USDT",
+ is_short=False,
+ open_date_utc=dt.now(timezone.utc) - timedelta(minutes=10),
+ )
+ model = self.model()
+ del model.get_state_info
+ model.data_provider = None
+ with mock.patch(
+ "ReforceXY.user_data.freqaimodels.ReforceXY.Trade.get_trades_proxy",
+ return_value=[trade],
+ ):
+ side, profit, duration = model.get_state_info("BTC/USDT")
+ self.assertEqual(side, 1.0)
+ self.assertEqual(profit, 0.0)
+ self.assertEqual(duration, 2)
+
+ def test_negative_efficiency_coefficient_is_clamped(self):
+ features = pd.DataFrame({"f": np.zeros(6)})
+ prices = pd.DataFrame({"open": [100.0, 100.0, 100.0, 90.0, 98.0, 100.0]})
+ env = MyRLEnv(
+ df=features,
+ prices=prices,
+ df_raw=features.copy(),
+ window_size=1,
+ reward_kwargs={"rr": 2.0, "profit_aim": 0.03},
+ fee=0.0,
+ can_short=False,
+ config={
+ "stake_amount": "unlimited",
+ "freqai": {
+ "rl_config": {
+ "add_state_info": False,
+ "max_training_drawdown_pct": 0.99,
+ "model_reward_parameters": {
+ "efficiency_weight": 2.0,
+ "efficiency_center": 0.0,
+ },
+ }
+ },
+ },
+ live=True,
+ )
+ self.addCleanup(env.close)
+ env.reset()
+ env.step(1)
+ env.step(0)
+ env.step(0)
+ # weight=2, center=0 with a partially recovered loss: raw coefficient -0.6.
+ self.assertAlmostEqual(
+ env._compute_efficiency_coefficient(
+ -0.02, {"efficiency_weight": 2.0, "efficiency_center": 0.0}
+ ),
+ 0.0,
+ )
+ exit_info = env.step(2)[-1]
+ self.assertAlmostEqual(exit_info["reward_exit"], 0.0)
--- /dev/null
+"""Regression contracts for learning eligibility, replay and temporal observations."""
+
+import tempfile
+import unittest
+from pathlib import Path
+from types import SimpleNamespace
+from unittest import mock
+
+import numpy as np
+import pandas as pd
+from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
+from optuna import TrialPruned, create_study
+
+from ReforceXY.tests.test_review_contracts import RecordingPolicy, model_config
+from ReforceXY.user_data.freqaimodels.ReforceXY import ReforceXY
+
+
+class TrainingObservationsTest(unittest.TestCase):
+ def test_dqn_eligibility_restart_and_chronology(self):
+ for algorithm in ("DQN", "QRDQN"):
+ with self.subTest(algorithm=algorithm), tempfile.TemporaryDirectory() as temp:
+ config = model_config(temp)
+ info = config["freqai"]
+ info["continual_learning"] = True
+ info["feature_parameters"]["shuffle_after_split"] = True
+ info["rl_config"]["model_type"] = algorithm
+ info["model_training_parameters"] = {
+ "learning_starts": 0,
+ "buffer_size": 128,
+ "batch_size": 8,
+ "train_freq": 4,
+ "gradient_steps": 1,
+ "device": "cpu",
+ "policy_kwargs": {"net_arch": [8]},
+ }
+ model = ReforceXY(config=config)
+ model.live = True
+ model.can_short = False
+ self.addCleanup(model.close_envs)
+ frame = pd.DataFrame(
+ {
+ "date": pd.date_range("2026-01-01", periods=64, freq="5min", tz="UTC"),
+ "%-feature": np.sin(np.arange(64)),
+ "&-action": np.zeros(64),
+ }
+ )
+ for column in ("open", "high", "low", "close"):
+ frame[f"%-raw_{column}"] = 100 + np.arange(64) * 0.1
+ dk = FreqaiDataKitchen(config, live=True, pair="BTC/USDT")
+ dk.data_path = Path(temp) / "initial"
+ dk.model_filename = "cb_btc_initial"
+ dk.label_list = ["&-action"]
+ dk.training_features_list = [c for c in frame if c.startswith("%")]
+ trained = model.train(frame, dk.pair, dk)
+ np.testing.assert_allclose(
+ dk.data_dictionary["train_prices"]["open"], frame["%-raw_open"][:48]
+ )
+ model.dd.pair_dict[dk.pair] = {}
+ model.dd.save_data(trained, dk.pair, dk)
+ replay = trained.replay_buffer.observations.copy()
+ size = trained.replay_buffer.size()
+ self.assertGreater(size, 0)
+ model.dd.model_dictionary.clear()
+ model.dd.meta_data_dictionary.clear()
+ restored = model.dd.load_data(dk.pair, dk)
+ self.assertEqual(restored.replay_buffer.size(), size)
+ np.testing.assert_array_equal(restored.replay_buffer.observations, replay)
+ clone, _ = model._resolve_deployment_state(dk, dk.pair)
+ self.assertEqual(clone.replay_buffer.size(), size)
+ self.assertIsNot(clone.replay_buffer, restored.replay_buffer)
+ clone.replay_buffer.observations.flat[0] += 123.0
+ self.assertNotEqual(
+ clone.replay_buffer.observations.flat[0],
+ restored.replay_buffer.observations.flat[0],
+ )
+ np.testing.assert_array_equal(restored.replay_buffer.observations, replay)
+ self.assertIs(model.dd.load_data(dk.pair, dk), restored)
+ model.dd.model_dictionary.clear()
+ (dk.data_path / dk.data["reforcexy_replay"]).unlink()
+ for _ in range(2):
+ with self.assertRaises(FileNotFoundError):
+ model.dd.load_data(dk.pair, dk)
+ self.assertNotIn(dk.pair, model.dd.model_dictionary)
+ params = model.get_model_params()
+ trial = create_study(direction="maximize").ask()
+ for starts in (64, 50000):
+ with (
+ mock.patch.object(
+ model,
+ "get_optuna_params",
+ return_value={
+ **params,
+ "learning_starts": starts,
+ "buffer_size": 100000,
+ },
+ ),
+ self.assertRaises(TrialPruned),
+ ):
+ model.objective(
+ trial,
+ dk,
+ 64,
+ dk.data_dictionary["train_prices"],
+ dk.data_dictionary["test_prices"],
+ )
+ with mock.patch.object(model, "get_optuna_params", return_value=params):
+ score = model.objective(
+ trial,
+ dk,
+ 64,
+ dk.data_dictionary["train_prices"],
+ dk.data_dictionary["test_prices"],
+ )
+ self.assertTrue(np.isfinite(score))
+
+ def test_frame_validity_and_gap_reset(self):
+ with tempfile.TemporaryDirectory() as temp:
+ model = ReforceXY(config=model_config(temp))
+ self.addCleanup(model.close_envs)
+ model.live = True
+ model.frame_stacking = 2
+ model.get_state_info = lambda pair: (0.5, 0.0, 0)
+ policy = RecordingPolicy()
+ dk = SimpleNamespace(pair="BTC/USDT", label_list=["&-action"], data_dictionary={})
+
+ def predict(value, minute, valid=1):
+ dk.do_predict = np.array([valid])
+ dk.data_dictionary["prediction_dates"] = pd.DataFrame(
+ {
+ "date": pd.DatetimeIndex(
+ [pd.Timestamp("2026-01-01", tz="UTC") + pd.Timedelta(minutes=minute)]
+ )
+ }
+ )
+ model.rl_model_predict(pd.DataFrame({"f": [value]}), dk, policy)
+ return dk.do_predict[0]
+
+ self.assertEqual(predict(10, 0, 0), 0)
+ self.assertEqual(predict(20, 5), 0)
+ self.assertEqual(predict(30, 10), 1)
+ self.assertEqual(predict(40, 30), 1)
+ np.testing.assert_array_equal(policy.observations[-1], [[[0, 40]]])
+ predict(50, 30)
+ np.testing.assert_array_equal(policy.observations[-1], [[[0, 50]]])
+ model.CONV_WIDTH = 3
+ model.frame_stacking = 0
+ dk.do_predict = np.ones(5, dtype=int)
+ dk.data_dictionary["prediction_dates"] = pd.DataFrame(
+ {
+ "date": pd.DatetimeIndex(
+ pd.to_datetime(
+ [
+ "2026-01-01 00:00Z",
+ "2026-01-01 00:05Z",
+ "2026-01-01 00:20Z",
+ "2026-01-01 00:25Z",
+ "2026-01-01 00:30Z",
+ ]
+ )
+ )
+ }
+ )
+ model.rl_model_predict(pd.DataFrame({"f": np.arange(5)}), dk, policy)
+ np.testing.assert_array_equal(dk.do_predict, [0, 0, 0, 0, 1])
from datetime import datetime, timezone
from functools import wraps
from inspect import iscoroutinefunction, unwrap
+from io import BytesIO
from pathlib import Path
from typing import (
Any,
BaseReinforcementLearningModel,
)
from freqtrade.freqai.tensorboard.TensorboardCallback import TensorboardCallback
+from freqtrade.persistence import Trade
from freqtrade.strategy import timeframe_to_minutes
from gymnasium.spaces import Box
from joblib.externals import cloudpickle
from optuna.trial import TrialState
from pandas import DataFrame, merge
from sb3_contrib.common.maskable.callbacks import MaskableEvalCallback
+from sb3_contrib.common.maskable.evaluation import evaluate_policy
from sb3_contrib.common.maskable.utils import is_masking_supported
from stable_baselines3.common.callbacks import (
BaseCallback,
VecMonitor,
)
+
+def _update_eval_best_reward(callback: Any, mean_reward: float, model: Any) -> None:
+ """Update plain-fit final-policy bookkeeping and save an improved checkpoint."""
+ if not np.isfinite(mean_reward):
+ return
+ callback.last_mean_reward = mean_reward
+ if not hasattr(callback, "best_mean_reward") or mean_reward > callback.best_mean_reward:
+ callback.best_mean_reward = mean_reward
+ if callback.best_model_save_path is not None and model is not None:
+ model.save(Path(callback.best_model_save_path) / "best_model.zip")
+
+
_DATE_PRED_DEDUP_SENTINEL = "_freqai_strategies_date_pred_repair_patched"
Freqtrade fills downtime rows with zeros/NaNs, without a candle close or a
prediction status. Those rows must not replace recorded predictions, including
zero predictions and rejected predictions (do_predict == 0 with a candle close).
- Indistinguishable legacy rows use last-write-wins; label magnitudes never rank rows.
+ Indistinguishable rows use last-write-wins; label magnitudes never rank rows.
Invalid dates cannot match a candle and are discarded.
"""
date_pred = pd.to_datetime(frame["date_pred"], utc=True, errors="coerce", format="mixed")
def _install_date_pred_dedup_patch() -> None:
- """Repair persisted history and duplicates produced by older Freqtrade writers.
+ """Normalize persisted history and duplicate predictions before upstream writes.
- Normalize before upstream positional writes and after legacy duplicate writes.
- Normalize before upstream disk repair can discard a recorded duplicate.
- Already-clean upstream results are preserved. Recheck these synchronous
- method contracts on Freqtrade upgrades.
+ Normalize before upstream positional writes and before disk repair can discard
+ a recorded duplicate. Already-clean upstream results are preserved. Recheck
+ these synchronous method contracts on Freqtrade upgrades.
"""
names = (
"set_initial_return_values",
self.historic_predictions[pair]
)
if self.historic_predictions[pair].empty and not strat_df.empty:
- # Legacy append requires an initialized row; let upstream construct it.
+ # Append requires an initialized row; let upstream construct it.
original_set_initial(
self,
pair,
"spike_cancel",
"retain_previous",
]
-TransformFunction = Literal["tanh", "softsign", "arctan", "sigmoid", "asinh", "clip"]
-ExitAttenuationMode = Literal["legacy", "sqrt", "linear", "power", "half_life"]
+TransformFunction = Literal["tanh", "softsign", "arctan", "sigmoid", "softsign_sqrt", "clip"]
+ExitAttenuationMode = Literal["sqrt", "linear", "power", "half_life"]
ActivationFunction = Literal["relu", "tanh", "elu", "leaky_relu"]
OptimizerClassOptuna = Literal["adamw", "rmsprop"]
OptimizerClass = OptimizerClassOptuna | Literal["adam"]
Continual learning keeps the deployed policy's fitted feature coordinates frozen,
including feature selection and scaling. Reset trained models or use a new
- freqai.identifier to change those coordinates or migrate legacy artifacts.
+ freqai.identifier to change those coordinates.
First training and continual_learning=false use fresh pipelines and policies.
Hyperopt always uses fresh current-window pipelines and cold policies; final
continuation uses the original raw split in the deployed coordinates. Selected
_LOG_2: Final[float] = math.log(2.0)
_DEPLOYMENT_COORDINATE_MARKER_KEY: Final[str] = "reforcexy_deployment_coordinates"
- _DEPLOYMENT_COORDINATE_GENERATION: Final[str] = "frozen-pipelines-v1"
+ _DEPLOYMENT_COORDINATE_GENERATION: Final[str] = "chronological-frozen-pipelines-v2"
DEFAULT_BASE_FACTOR: Final[float] = 100.0
"softsign",
"arctan",
"sigmoid",
- "asinh",
+ "softsign_sqrt",
"clip",
)
_TRANSFORM_FUNCTIONS_SET: Final[frozenset[TransformFunction]] = frozenset(_TRANSFORM_FUNCTIONS)
_EXIT_ATTENUATION_MODES: Final[tuple[ExitAttenuationMode, ...]] = (
- "legacy",
"sqrt",
"linear",
"power",
_QUARANTINE_TIE_BREAK_LIMIT: Final[int] = 99
_BEST_PARAMS_LOCK_FILENAME: Final[str] = ".hyperopt-best-params.lock"
# Bump on objective changes that make persisted trials or best params incompatible.
- _OPTUNA_OBJECTIVE_IDENTITY: Final[str] = "raw-market-prices-v1"
+ _OPTUNA_OBJECTIVE_IDENTITY: Final[str] = (
+ "terminal-liquidation-risk-normalized-trained-policy-v3"
+ )
_PPO_N_STEPS: Final[tuple[int, ...]] = (512, 1024, 2048, 4096)
_PPO_N_STEPS_MIN: Final[int] = min(_PPO_N_STEPS)
_PPO_N_STEPS_MAX: Final[int] = max(_PPO_N_STEPS)
+ _DQN_TRAIN_FREQS: Final[tuple[int, ...]] = (2, 4, 8, 16, 32, 64, 128, 256, 512, 1024)
_HYPEROPT_EVAL_FREQ_REDUCTION_FACTOR: Final[float] = 4.0
_action_masks_cache: ClassVar[dict[tuple[bool, float], NDArray[np.bool_]]] = {}
self.optuna_purge_period: int = int(self.rl_config_optuna.get("purge_period", 0))
self.optuna_eval_callback: MaskableTrialEvalCallback | None = None
self._model_params_cache: dict[str, Any] | None = None
+ self._frame_buffers: dict[str, tuple[Any, deque[NDArray[np.float32]]]] = {}
self._lstm_states_cache: dict[
str,
tuple[
] = {}
self.unset_unsupported()
self._configure_gpu_memory()
+ self._install_replay_persistence()
+
+ def _install_replay_persistence(self) -> None:
+ save_data = self.dd.save_data
+ load_data = self.dd.load_data
+
+ @wraps(save_data)
+ def save_with_replay(model, coin, dk):
+ dk.data = dk.data.copy()
+ dk.data.pop("reforcexy_replay", None)
+ if hasattr(model, "save_replay_buffer"):
+ dk.data_path.mkdir(parents=True, exist_ok=True)
+ filename = f"{dk.model_filename}_replay_{uuid4().hex}.pkl"
+ destination = dk.data_path / filename
+ temporary = destination.with_suffix(".tmp")
+ try:
+ model.save_replay_buffer(temporary)
+ temporary.replace(destination)
+ finally:
+ temporary.unlink(missing_ok=True)
+ dk.data["reforcexy_replay"] = filename
+ return save_data(model, coin, dk)
+
+ @wraps(load_data)
+ def load_with_replay(coin, dk):
+ cached = self.dd.model_dictionary.get(coin) if dk.live else None
+ model = load_data(coin, dk)
+ if model is not None and model is not cached:
+ try:
+ self._restore_replay(model, dk.data, dk.data_path, coin)
+ except Exception:
+ if self.dd.model_dictionary.get(coin) is model:
+ self.dd.model_dictionary.pop(coin, None)
+ raise
+ return model
+
+ self.dd.save_data = save_with_replay
+ self.dd.load_data = load_with_replay
+
+ @staticmethod
+ def _restore_replay(model: Any, metadata: dict[str, Any], directory: Path, pair: str) -> None:
+ if not hasattr(model, "load_replay_buffer"):
+ return
+ filename = metadata.get("reforcexy_replay")
+ if not isinstance(filename, str) or Path(filename).name != filename:
+ raise DependencyException(
+ f"Training [{pair}]: DQN/QRDQN replay metadata is missing or incompatible; "
+ "reset trained models or use a new freqai.identifier"
+ )
+ model.load_replay_buffer(directory / filename)
+ replay = model.replay_buffer
+ if (
+ replay.observation_space != model.observation_space
+ or replay.action_space != model.action_space
+ or replay.n_envs != model.n_envs
+ ):
+ raise DependencyException(
+ f"Training [{pair}]: DQN/QRDQN replay buffer is incompatible; reset trained "
+ "models or use a new freqai.identifier"
+ )
def _configure_gpu_memory(self) -> None:
"""
"Config [global]: purge_period has no effect when continuous=True; defaulting to 0"
)
self.optuna_purge_period = 0
- add_state_info = self.rl_config.get("add_state_info", False)
- if not add_state_info:
+ hold_potential_enabled = self.rl_config.get("model_reward_parameters", {}).get(
+ "hold_potential_enabled", ReforceXY.DEFAULT_HOLD_POTENTIAL_ENABLED
+ )
+ if MyRLEnv.is_unsupported_pbrs_config(
+ hold_potential_enabled, self.rl_config.get("add_state_info", False)
+ ):
logger.warning(
- "Config [global]: add_state_info=False may lead to desynchronized trade states after restart"
+ "Config [global]: hold_potential_enabled=True requires add_state_info=True; "
+ "enabling add_state_info"
)
+ self.rl_config["add_state_info"] = True
tensorboard_throttle = self.rl_config.get("tensorboard_throttle", 1)
if not isinstance(tensorboard_throttle, int) or tensorboard_throttle < 1:
logger.warning(
self.frame_stacking,
)
+ def unset_outlier_removal(self) -> None:
+ super().unset_outlier_removal()
+ # RL transitions, durations and rewards require chronological candles;
+ # a post-split shuffle would silently corrupt the learned trajectory.
+ if self.ft_params.get("shuffle_after_split", False):
+ self.ft_params.update({"shuffle_after_split": False})
+ logger.warning(
+ "Config [global]: shuffle_after_split=True reorders RL transitions; "
+ "setting shuffle_after_split to False"
+ )
+
def pack_env_dict(
self, pair: str, model_params: dict[str, Any] | None = None
) -> dict[str, Any]:
- env_info = super().pack_env_dict(pair)
-
- config = env_info.setdefault("config", {})
- freqai_cfg = config.setdefault("freqai", {})
- rl_cfg = freqai_cfg.setdefault("rl_config", {})
- model_reward_parameters = rl_cfg.setdefault("model_reward_parameters", {})
-
- gamma: float | None = None
-
- if model_params and isinstance(model_params.get("gamma"), (int, float)):
- gamma = float(model_params["gamma"])
- elif self.hyperopt:
- best_trial_params = self.load_best_trial_params(pair)
- if best_trial_params and isinstance(best_trial_params.get("gamma"), (int, float)):
- gamma = float(best_trial_params.get("gamma"))
-
- if (
- gamma is None
- and hasattr(self.model, "gamma")
- and isinstance(self.model.gamma, (int, float))
+ if not self.live and self.rl_config.get("model_reward_parameters", {}).get(
+ "hold_potential_enabled", ReforceXY.DEFAULT_HOLD_POTENTIAL_ENABLED
):
- gamma = float(self.model.gamma)
-
- if gamma is None:
- model_params_gamma = self.get_model_params().get("gamma")
- if isinstance(model_params_gamma, (int, float)):
- gamma = float(model_params_gamma)
+ raise ValueError(
+ f"Config [{pair}]: backtesting does not support hold_potential_enabled=True "
+ "because add_state_info is unavailable"
+ )
+ env_info = super().pack_env_dict(pair)
+ # Each environment owns its effective parameters; do not mutate global config.
+ config = copy.deepcopy(env_info["config"])
+ env_info["config"] = config
+ model_reward_parameters = config["freqai"]["rl_config"].setdefault(
+ "model_reward_parameters", {}
+ )
+ effective_params = self.get_model_params() if model_params is None else model_params
+ gamma = effective_params.get("gamma")
if gamma is not None:
model_reward_parameters["potential_gamma"] = gamma
"""
Set training and evaluation environments
"""
+ data_dictionary["train_prices"] = prices_train
+ data_dictionary["test_prices"] = prices_test
if self.train_env is not None or self.eval_env is not None:
logger.info("Env [%s]: closing environments", dk.pair)
self.close_envs()
train_df = data_dictionary["train_features"]
test_df = data_dictionary["test_features"]
- env_dict = self.pack_env_dict(dk.pair, model_params=model_params)
- seed = self.get_model_params().get("seed", 42)
+ env_dict = self.pack_env_dict(dk.pair, model_params)
+ effective_params = self.get_model_params() if model_params is None else model_params
+ seed = effective_params.get("seed", 42)
if self.check_envs:
logger.info("Env [%s]: checking environments", dk.pair)
check_env(_train_env_check)
finally:
_train_env_check.close()
- _eval_env_check = MyRLEnv(
- df=test_df,
- prices=prices_test,
- id="eval_env_check",
- seed=seed + 10_000,
- **env_dict,
- )
- try:
- check_env(_eval_env_check)
- finally:
- _eval_env_check.close()
+ if self.data_split_parameters.get("test_size", 0.1) > 0:
+ _eval_env_check = MyRLEnv(
+ df=test_df,
+ prices=prices_test,
+ id="eval_env_check",
+ seed=seed + 10_000,
+ **env_dict,
+ )
+ try:
+ check_env(_eval_env_check)
+ finally:
+ _eval_env_check.close()
logger.info(
"Env [%s]: populating %s train and %s eval environments",
def get_callbacks(
self,
- eval_env: BaseEnvironment,
+ eval_env: VecEnv | None,
eval_freq: int,
data_path: str,
trial: Trial | None = None,
Get the model specific callbacks
"""
callbacks: list[BaseCallback] = []
+ self.eval_callback = None
+ self.optuna_eval_callback = None
+ self.progressbar_callback = None
no_improvement_callback = None
rollout_plot_callback = None
verbose = self.get_model_params().get("verbose", 0)
self.progressbar_callback = ProgressBarCallback()
callbacks.append(self.progressbar_callback)
+ if eval_env is None:
+ return callbacks
+
use_masking = self.action_masking and is_masking_supported(eval_env)
if not trial:
self.eval_callback = MaskableEvalCallback(
def _resolve_deployment_state(
self, dk: FreqaiDataKitchen, pair: str
) -> tuple[Any, Pipeline] | None:
- """Restore the deployed policy and an independent fitted feature pipeline."""
+ """Restore independent training copies of the deployed policy and feature pipeline."""
if not self.continual_learning:
return None
model = self.dd.model_dictionary.get(pair)
+ cached_model = model
previous = self.dd.pair_dict.get(pair, {})
if model is None and not previous.get("model_filename"):
return None
feature_pipeline = cloudpickle.load(fp)
if model is None:
model = self.MODELCLASS.load(Path(f"{prefix}_model"))
+ self._restore_replay(model, metadata, previous_dk.data_path, pair)
if (
metadata.get(self._DEPLOYMENT_COORDINATE_MARKER_KEY)
!= self._DEPLOYMENT_COORDINATE_GENERATION
):
if list(expected) != list(actual):
raise ValueError(f"persisted {name} differ in names or order")
+ if cached_model is not None:
+ # SB3 archives preserve policy/optimizer state but exclude envs and loggers.
+ with BytesIO() as archive:
+ model.save(archive)
+ archive.seek(0)
+ model = self.MODELCLASS.load(archive, device=cached_model.device)
+ # Off-policy experience is excluded from SB3 model archives.
+ if getattr(cached_model, "replay_buffer", None) is not None:
+ model.replay_buffer = copy.deepcopy(cached_model.replay_buffer)
state = model, copy.deepcopy(feature_pipeline)
except Exception as exc:
raise DependencyException(
"Reset trained models or use a new freqai.identifier."
) from exc
logger.info(
- f"Training [{pair}]: continuing deployment in the persisted feature coordinate system; "
- "reset trained models to change pipeline configuration"
+ "Training [%s]: continuing deployment in the persisted feature coordinate system; "
+ "reset trained models to change pipeline configuration",
+ pair,
)
return state
unfiltered_df, dk.training_features_list, dk.label_list, training_filter=True
)
raw_data = dk.make_train_test_datasets(features_filtered, labels_filtered)
+ if (
+ not self.hyperopt
+ and self.data_split_parameters.get("test_size") == 0
+ and len(raw_data["train_features"]) > 0
+ ):
+ # A zero-size split keeps the schema but must yield no eval environment.
+ raw_data["test_features"] = raw_data["train_features"].iloc[:0].copy()
+ raw_data["test_labels"] = raw_data["train_labels"].iloc[:0].copy()
+ raw_data["test_weights"] = np.asarray(raw_data["train_weights"])[:0].copy()
+ raw_data["train_dates"] = raw_data["train_dates"]
+ raw_data["test_dates"] = raw_data["train_dates"].iloc[:0]
self.df_raw = copy.deepcopy(raw_data["train_features"])
dk.fit_labels()
# Capture prices once, before normalization and optional raw-OHLC removal.
raw_data, dk, deployment_state=None if self.hyperopt else deployment_state
)
logger.info(
- f"Training [{pair}]: model on {len(dd['train_features'].columns)}"
- f" features and {len(dd['train_features'])} data points"
+ "Training [%s]: model on %d features and %d data points",
+ pair,
+ len(dd["train_features"].columns),
+ len(dd["train_features"]),
)
model = self.fit(
dd,
)
logger.info("Model [%s]: %s params: %s", dk.pair, self.model_type, model_params)
- # "PPO"
- if ReforceXY._MODEL_TYPES[0] in self.model_type:
- n_steps = model_params.get("n_steps", 0)
- min_timesteps = 2 * n_steps * self.n_envs
- if total_timesteps <= min_timesteps:
- logger.warning(
- "Training [%s]: total_timesteps=%s is less than or equal to 2*n_steps*n_envs=%s. This may lead to suboptimal training results for model %s",
- dk.pair,
- total_timesteps,
- min_timesteps,
- self.model_type,
- )
- if n_steps > 0:
- rollout = n_steps * self.n_envs
- aligned_total_timesteps = ReforceXY._ceil_to_multiple(total_timesteps, rollout)
- if aligned_total_timesteps != total_timesteps:
- total_timesteps = aligned_total_timesteps
- logger.info(
- "Training [%s]: aligned total %s steps (%s days) for model %s",
- dk.pair,
- total_timesteps,
- steps_to_days(total_timesteps, self.config.get("timeframe")),
- self.model_type,
- )
-
if self.activate_tensorboard:
tensorboard_log_path = Path(self.full_path / "tensorboard" / Path(dk.data_path).name)
else:
tensorboard_log_path = None
- # Reward shaping must use the resumed policy's discount gamma,
- # not the configured or HPO-selected value.
- resumed_gamma: dict[str, Any] | None = None
- if deployment_state is not None:
- learner_gamma = getattr(deployment_state[0], "gamma", None)
- if isinstance(learner_gamma, (int, float)) and np.isfinite(learner_gamma):
- resumed_gamma = {"gamma": float(learner_gamma)}
- self.set_train_and_eval_environments(
- data_dictionary,
- prices_train,
- prices_test,
- dk,
- model_params=resumed_gamma,
- )
-
model = deployment_state[0] if deployment_state is not None else None
+ effective_params = dict(model_params)
if model is not None:
- logger.info(
- "Training [%s]: continual training activated, starting from previously trained model state",
- dk.pair,
- )
- model.tb_logger = getattr(self, "tb_logger", None)
- model.set_env(self.train_env)
- else:
- model = self.MODELCLASS(
- self.policy_type,
- self.train_env,
- tensorboard_log=tensorboard_log_path,
- **model_params,
+ learner_gamma = getattr(model, "gamma", None)
+ if isinstance(learner_gamma, (int, float)) and np.isfinite(learner_gamma):
+ effective_params["gamma"] = float(learner_gamma)
+ # Preserve raw prices and the resumed learner's discount in its environments.
+ try:
+ self.set_train_and_eval_environments(
+ data_dictionary,
+ prices_train,
+ prices_test,
+ dk,
+ model_params=effective_params,
)
+ if model is not None:
+ logger.info("Training [%s]: continuing from deployed model state", dk.pair)
+ model.tb_logger = getattr(self, "tb_logger", None)
+ model.set_env(self.train_env)
+ else:
+ model = self.MODELCLASS(
+ self.policy_type,
+ self.train_env,
+ tensorboard_log=tensorboard_log_path,
+ **model_params,
+ )
+ total_timesteps = self._align_model_budget(model, total_timesteps)
- eval_freq = self.get_eval_freq(total_timesteps, model_params=model_params)
- callbacks = self.get_callbacks(self.eval_env, eval_freq, str(dk.data_path))
- try:
+ eval_freq = self.get_eval_freq(total_timesteps, model_params=model_params)
+ callbacks = self.get_callbacks(self.eval_env, eval_freq, str(dk.data_path))
logger.debug(
"Training [%s]: starting model.learn with total_timesteps=%d, eval_freq=%d",
dk.pair,
)
model.learn(total_timesteps=total_timesteps, callback=callbacks)
logger.debug("Training [%s]: model.learn completed", dk.pair)
+ if self.eval_env is not None and self.eval_callback is not None:
+ # Evaluate final weights before teardown; the periodic callback may not run
+ # within a single-rollout budget.
+ use_masking = self.eval_callback.use_masking
+ final_mean_reward, _ = evaluate_policy(
+ model,
+ self.eval_env,
+ n_eval_episodes=self.n_eval_episodes,
+ warn=False,
+ use_masking=use_masking,
+ )
+ _update_eval_best_reward(self.eval_callback, float(final_mean_reward), model)
except KeyboardInterrupt:
pass
finally:
if self.progressbar_callback:
self.progressbar_callback.on_training_end()
self.close_envs()
- if hasattr(model, "env") and model.env is not None:
+ if model is not None and hasattr(model, "env") and model.env is not None:
model.env.close()
time_spent = time.time() - start_time
self.dd.update_metric_tracker("fit_time", time_spent, dk.pair)
logger.info("Model [%s]: found best model at %s", dk.pair, model_filepath)
try:
best_model = self.MODELCLASS.load(model_filepath)
- # SB3 archives exclude replay buffers; restore the final buffer so
- # the next cached retrain keeps the experience accumulated this fit.
+ # SB3 archives exclude replay buffers; preserve experience for continuation.
if (
- hasattr(model, "replay_buffer")
+ model is not None
+ and hasattr(model, "replay_buffer")
and getattr(best_model, "replay_buffer", None) is not None
):
best_model.replay_buffer = model.replay_buffer
warmed_up = remaining <= 0
if not warmed_up:
logger.warning(
- f"Predict [{pair}]: fit live predictions not warmed up; {remaining} produced observations until warmup completion"
+ "Predict [%s]: fit live predictions not warmed up; "
+ "%d more produced observations required for warmup completion",
+ pair,
+ remaining,
)
pred_df = history.tail(fit_live_predictions_candles).reset_index(drop=True)
f[1],
)
+ def get_state_info(self, pair: str) -> tuple[float, float, int]:
+ """Read the live market side, unlevered profit ratio and candle duration.
+
+ ``Trade.calc_profit_ratio`` includes effective leverage. Dividing by a
+ finite positive leverage matches the environment's unlevered PnL proxy.
+ """
+ market_side = 0.5
+ current_profit = 0.0
+ trade_duration = 0
+ for trade in Trade.get_trades_proxy(is_open=True):
+ if trade.pair != pair:
+ continue
+ market_side = 0 if trade.is_short else 1
+ now = datetime.now(timezone.utc).timestamp()
+ trade_duration = int((now - trade.open_date_utc.timestamp()) / self.base_tf_seconds)
+ if self.data_provider is None or self.data_provider._exchange is None:
+ logger.warning(
+ "StateInfo [%s]: data provider or exchange unavailable; using profit=0",
+ pair,
+ )
+ return market_side, 0.0, trade_duration
+ current_rate = self.data_provider._exchange.get_rate(
+ pair, refresh=False, side="exit", is_short=trade.is_short
+ )
+ profit_ratio = trade.calc_profit_ratio(current_rate)
+ leverage = float(trade.leverage)
+ if np.isfinite(leverage) and leverage > 0.0:
+ current_profit = profit_ratio / leverage
+ else:
+ current_profit = profit_ratio
+ return market_side, current_profit, int(trade_duration)
+
+ def predict(
+ self, unfiltered_df: DataFrame, dk: FreqaiDataKitchen, **kwargs
+ ) -> tuple[DataFrame, NDArray[np.int_]]:
+ """Feed source dates to inference; parent predict performs the filtering."""
+ dk.data_dictionary["prediction_dates"] = unfiltered_df[["date"]]
+ try:
+ return super().predict(unfiltered_df, dk, **kwargs)
+ finally:
+ dk.data_dictionary.pop("prediction_dates", None)
+
def rl_model_predict(
self, dataframe: DataFrame, dk: FreqaiDataKitchen, model: Any
) -> DataFrame:
add_state_info: bool = self.rl_config.get("add_state_info", False)
virtual_position: Positions = Positions.Neutral
virtual_trade_duration: int = 0
- if add_state_info and self.live:
+ if self.live and (add_state_info or (self.action_masking and self.inference_masking)):
position, _, trade_duration = self.get_state_info(dk.pair)
virtual_position = ReforceXY._normalize_position(position)
virtual_trade_duration = trade_duration
frame_stacking: int = self.frame_stacking
frame_stacking_enabled: bool = bool(frame_stacking) and frame_stacking > 1
inference_masking: bool = self.action_masking and self.inference_masking
+ source_valid = np.asarray(getattr(dk, "do_predict", np.ones(n))) == 1
+ source_valid = source_valid & np.isfinite(np_dataframe).all(axis=1)
+ dk.do_predict = np.zeros(n, dtype=np.int_)
+ dates = getattr(dk, "data_dictionary", {}).get("prediction_dates")
+ dates = (
+ pd.DatetimeIndex(pd.to_datetime(dates["date"], utc=True)).as_unit("ns")
+ if dates is not None
+ else None
+ )
+ if dates is not None and len(dates) != n:
+ raise ValueError(
+ f"Predict [{dk.pair}]: prediction dates must align with feature rows: "
+ f"expected {n}, got {len(dates)}"
+ )
+ step = pd.Timedelta(seconds=self.base_tf_seconds)
+ if not hasattr(self, "_observation_cache"):
+ self._observation_cache = {}
+ cached = self._observation_cache.get(dk.pair)
+ adjacent = (
+ dates is not None
+ and len(dates) == n
+ and n == window_size
+ and cached is not None
+ and cached[0] is model
+ and dates[-1] - cached[1] == step
+ )
+ frame_validity: deque[bool] = deque(
+ cached[2] if adjacent and cached is not None else (), maxlen=max(1, frame_stacking)
+ )
+ if not adjacent:
+ self._frame_buffers.pop(dk.pair, None)
+ self._lstm_states_cache.pop(dk.pair, None)
if window_size <= 0 or n < window_size:
return DataFrame(
return current_virtual_trade_duration + 1
return 0
- frame_buffer: deque[np.float32] = deque(
+ frame_buffer: deque[NDArray[np.float32]] = deque(
maxlen=frame_stacking if frame_stacking_enabled else None
)
+ if self.live and frame_stacking_enabled:
+ cached_frames = self._frame_buffers.get(dk.pair)
+ if cached_frames is not None and cached_frames[0] is model and n == window_size:
+ frame_buffer = cached_frames[1]
zero_frame: NDArray[np.float32] | None = None
model_id = id(model)
lstm_states_cache_valid = (
self.live
and self.recurrent
+ and n == window_size
and dk.pair in self._lstm_states_cache
and self._lstm_states_cache[dk.pair][0] == model_id
)
action_mask_position = virtual_position
if frame_stacking_enabled:
- frame_buffer.append(np_observation)
+ # Own only the retained window, not the full prediction dataframe.
+ frame_buffer.append(np_observation.copy())
if len(frame_buffer) < frame_stacking:
pad_count = frame_stacking - len(frame_buffer)
if zero_frame is None:
observations = np_observation.reshape(
1, np_observation.shape[0], np_observation.shape[1]
)
+ if not np.isfinite(observations).all():
+ dk.do_predict[end_idx - 1] = 0
+ lstm_states = None
+ episode_start = np.array([True], dtype=bool)
+ return Actions.Neutral.value
+ if self.recurrent and dk.do_predict[end_idx - 1] != 1:
+ lstm_states = None
+ episode_start = np.array([True], dtype=bool)
+ return Actions.Neutral.value
if inference_masking:
action_masks_param["action_masks"] = ReforceXY.get_action_masks(
predicted_actions: list[int] = []
for start_idx in range(0, n - window_size + 1):
+ end = start_idx + window_size
+ contiguous = dates is None or (
+ not dates[start_idx:end].hasnans
+ and (
+ np.diff(dates[start_idx:end].to_numpy(dtype="datetime64[ns]"))
+ == step.to_timedelta64()
+ ).all()
+ )
+ if dates is not None and start_idx > 0 and dates[end - 1] - dates[end - 2] != step:
+ frame_buffer.clear()
+ frame_validity.clear()
+ lstm_states = None
+ episode_start = np.array([True], dtype=bool)
+ valid = bool(source_valid[start_idx:end].all() and contiguous)
+ frame_validity.append(valid)
+ dk.do_predict[end - 1] = int(
+ valid and (not frame_stacking_enabled or all(frame_validity))
+ )
action = _predict(start_idx)
predicted_actions.append(action)
- previous_virtual_position = virtual_position
- virtual_position = _update_virtual_position(action, virtual_position)
- virtual_trade_duration = _update_virtual_trade_duration(
- virtual_position,
- previous_virtual_position,
- virtual_trade_duration,
- )
+ if not self.live:
+ previous_virtual_position = virtual_position
+ prediction_index = start_idx + window_size - 1
+ do_predict = getattr(dk, "do_predict", None)
+ if do_predict is None or do_predict[prediction_index] == 1:
+ virtual_position = _update_virtual_position(action, virtual_position)
+ virtual_trade_duration = _update_virtual_trade_duration(
+ virtual_position,
+ previous_virtual_position,
+ virtual_trade_duration,
+ )
pad_count = max(0, n - len(predicted_actions))
actions_list = ([np.nan] * pad_count) + predicted_actions
if self.live and self.recurrent:
self._lstm_states_cache[dk.pair] = (model_id, lstm_states, episode_start)
+ if self.live and frame_stacking_enabled:
+ self._frame_buffers[dk.pair] = (model, frame_buffer)
+ if self.live and dates is not None and len(dates) == n:
+ self._observation_cache[dk.pair] = (model, dates[-1], tuple(frame_validity))
return DataFrame(dict.fromkeys(dk.label_list, actions_df["action"]))
def _ceil_to_multiple(value: int, multiple: int) -> int:
return ((value + multiple - 1) // multiple) * multiple
+ @staticmethod
+ def _align_model_budget(model: Any, total_timesteps: int) -> int:
+ """Use the constructed algorithm's collection window, including defaults."""
+ if hasattr(model, "train_freq"):
+ frequency = model.train_freq
+ if frequency.unit.value != "step":
+ return total_timesteps
+ steps = frequency.frequency
+ else:
+ steps = model.n_steps
+ return ReforceXY._ceil_to_multiple(total_timesteps, steps * model.n_envs)
+
@staticmethod
def _ppo_resources(total_timesteps: int, n_envs: int, reduction_factor: int) -> tuple[int, int]:
min_n_steps = ReforceXY._PPO_N_STEPS_MIN
min_resource, max_resource = ReforceXY._ppo_resources(
total_timesteps, n_envs, reduction_factor
)
- else:
+ else: # "DQN"/"QRDQN": gradient windows of train_freq * n_envs
+ window = max(ReforceXY._DQN_TRAIN_FREQS)
min_resource = max(
2 * reduction_factor,
self.get_eval_freq(total_timesteps, hyperopt=True) * n_envs,
)
- max_resource = max(min_resource, total_timesteps + (n_envs - 1))
+ max_resource = max(
+ min_resource,
+ ReforceXY._ceil_to_multiple(total_timesteps, window * n_envs),
+ )
direction = StudyDirection.MAXIMIZE
load_if_exists = not continuous and not pair_purge_triggered
study.best_trial.number,
study.best_trial.value,
)
+ if self.model_type == ReforceXY._MODEL_TYPES[1] and self.get_model_params().get(
+ "policy_kwargs", {}
+ ).get("shared_lstm", False):
+ best_trial_params = {**best_trial_params, "enable_critic_lstm": False}
logger.info("Hyperopt [%s]: best params: %s", study_name, best_trial_params)
self.save_best_trial_params(best_trial_params, dk.pair)
def _best_trial_params_path(self, pair: str) -> Path:
return self.full_path / f"hyperopt-best-params-{ReforceXY._sanitize_pair(pair)}.json"
- def _resolve_legacy_best_trial_params(
- self, pair: str, best_trial_params_path: Path
- ) -> Path | None:
- base = pair.split("/")[0]
- legacy_path = self.full_path / f"hyperopt-best-params-{base}.json"
- if (
- legacy_path == best_trial_params_path
- or legacy_path.is_symlink()
- or not legacy_path.is_file()
- ):
- return None
- base_pair_count = sum(1 for configured in self.pairs if configured.split("/")[0] == base)
- if base_pair_count == 1:
- return legacy_path
- logger.warning(
- "Hyperopt [%s]: ignoring ambiguous legacy best params at %s: "
- "filename does not encode the complete pair identity",
- pair,
- legacy_path,
- )
- return None
-
@staticmethod
@contextmanager
def _locked_best_trial_params(
with self._locked_best_trial_params(best_trial_params_path, exclusive=False):
self._reject_best_trial_params_symlink(best_trial_params_path)
if not best_trial_params_path.is_file():
- legacy_path = self._resolve_legacy_best_trial_params(pair, best_trial_params_path)
- if legacy_path is None:
- return None
- best_trial_params_path = legacy_path
+ return None
logger.info(
"Hyperopt [%s]: loading best params from %s",
pair,
env_info: dict[str, Any] | None = None,
trial: Trial | None = None,
model_params: dict[str, Any] | None = None,
- ) -> tuple[VecEnv, VecEnv]:
+ ) -> tuple[VecEnv, VecEnv | None]:
seed: int = self.get_model_params().get("seed", 42) if seed is None else seed
if trial is not None:
seed += trial.number
prices_test,
env_info=env_info,
)
- for i in range(self.n_eval_envs)
+ for i in range(
+ self.n_eval_envs if self.data_split_parameters.get("test_size", 0.1) > 0 else 0
+ )
]
- if self.multiprocessing and self.n_envs > 1:
- train_env = SubprocVecEnv(train_fns, start_method="spawn")
- else:
- train_env = DummyVecEnv(train_fns)
- if self.eval_multiprocessing and self.n_eval_envs > 1:
- eval_env = SubprocVecEnv(eval_fns, start_method="spawn")
- else:
- eval_env = DummyVecEnv(eval_fns)
-
- if bool(self.frame_stacking) and self.frame_stacking > 1:
- train_env = VecFrameStack(train_env, n_stack=self.frame_stacking)
- eval_env = VecFrameStack(eval_env, n_stack=self.frame_stacking)
-
- train_env = VecMonitor(train_env)
- eval_env = VecMonitor(eval_env)
-
- return train_env, eval_env
+ train_env = eval_env = None
+ try:
+ if self.multiprocessing and self.n_envs > 1:
+ train_env = SubprocVecEnv(train_fns, start_method="spawn")
+ else:
+ train_env = _make_dummy_vec_env(train_fns)
+ eval_env = None
+ if eval_fns:
+ if self.eval_multiprocessing and self.n_eval_envs > 1:
+ eval_env = SubprocVecEnv(eval_fns, start_method="spawn")
+ else:
+ eval_env = _make_dummy_vec_env(eval_fns)
+
+ if bool(self.frame_stacking) and self.frame_stacking > 1:
+ train_env = VecFrameStack(train_env, n_stack=self.frame_stacking)
+ if eval_env is not None:
+ eval_env = VecFrameStack(eval_env, n_stack=self.frame_stacking)
+
+ train_env = VecMonitor(train_env)
+ if eval_env is not None:
+ eval_env = VecMonitor(eval_env)
+
+ return train_env, eval_env
+ except BaseException:
+ if train_env is not None:
+ train_env.close()
+ if eval_env is not None:
+ eval_env.close()
+ raise
def get_optuna_params(self, trial: Trial) -> dict[str, Any]:
# "RecurrentPPO"
if ReforceXY._MODEL_TYPES[1] in self.model_type:
- return sample_params_recurrentppo(trial)
+ return sample_params_recurrentppo(
+ trial,
+ shared_lstm=self.get_model_params()
+ .get("policy_kwargs", {})
+ .get("shared_lstm", False),
+ )
# "PPO"
elif ReforceXY._MODEL_TYPES[0] in self.model_type:
return sample_params_ppo(trial)
params["seed"] = params.get("seed", 42) + trial.number
logger.info("Hyperopt [%s]: trial #%d params: %s", study_name, trial.number, params)
- # "PPO"
- if ReforceXY._MODEL_TYPES[0] in self.model_type:
- n_steps = params.get("n_steps", 0)
- if n_steps > 0:
- rollout = n_steps * self.n_envs
- aligned_total_timesteps = ReforceXY._ceil_to_multiple(total_timesteps, rollout)
- if aligned_total_timesteps != total_timesteps:
- total_timesteps = aligned_total_timesteps
-
nan_encountered = False
+ # shared_lstm=True forbids a separate critic LSTM; sampling
+ # enable_critic_lstm=True would crash RecurrentPPO at construction
+ # before any pruning logic can run, so the merged trial parameters
+ # never carry the incompatible pair.
+ if (
+ ReforceXY._MODEL_TYPES[1] in self.model_type # "RecurrentPPO"
+ and bool((params.get("policy_kwargs") or {}).get("shared_lstm", False))
+ ):
+ params.setdefault("policy_kwargs", {})["enable_critic_lstm"] = False
+
if self.activate_tensorboard:
tensorboard_log_path = Path(
self.full_path
else:
tensorboard_log_path = None
- train_env, eval_env = self._get_train_and_eval_environments(
- dk,
- train_df=dk.data_dictionary["train_features"],
- test_df=dk.data_dictionary["test_features"],
- prices_train=prices_train,
- prices_test=prices_test,
- trial=trial,
- model_params=params,
- )
+ train_env = eval_env = model = None
+ try:
+ train_env, eval_env = self._get_train_and_eval_environments(
+ dk,
+ train_df=dk.data_dictionary["train_features"],
+ test_df=dk.data_dictionary["test_features"],
+ prices_train=prices_train,
+ prices_test=prices_test,
+ trial=trial,
+ model_params=params,
+ )
- model = self.MODELCLASS(
- self.policy_type,
- train_env,
- tensorboard_log=tensorboard_log_path,
- **params,
- )
+ model = self.MODELCLASS(
+ self.policy_type,
+ train_env,
+ tensorboard_log=tensorboard_log_path,
+ **params,
+ )
+ total_timesteps = self._align_model_budget(model, total_timesteps)
+ off_policy = hasattr(model, "replay_buffer")
+ if off_policy and model.learning_starts >= total_timesteps:
+ raise TrialPruned(
+ f"Hyperopt [{study_name}]: learning_starts={model.learning_starts} "
+ f"leaves no update within total_timesteps={total_timesteps}"
+ )
+ initial_updates = getattr(model, "_n_updates", 0)
- eval_freq = self.get_eval_freq(total_timesteps, hyperopt=True, model_params=params)
- callbacks = self.get_callbacks(eval_env, eval_freq, str(dk.data_path), trial)
- try:
+ eval_freq = self.get_eval_freq(total_timesteps, hyperopt=True, model_params=params)
+ callbacks = self.get_callbacks(eval_env, eval_freq, str(dk.data_path), trial)
model.learn(total_timesteps=total_timesteps, callback=callbacks)
+ if off_policy and model._n_updates <= initial_updates:
+ raise TrialPruned(f"Hyperopt [{study_name}]: trial completed without learning")
+ if not self.optuna_eval_callback.is_pruned and eval_env is not None:
+ # Evaluate final weights before teardown; the periodic callback may not run
+ # within a single-rollout budget.
+ use_masking = self.optuna_eval_callback.use_masking
+ final_mean_reward, _ = evaluate_policy(
+ model,
+ eval_env,
+ n_eval_episodes=self.n_eval_episodes,
+ warn=False,
+ use_masking=use_masking,
+ )
+ self.optuna_eval_callback.update_best_reward(float(final_mean_reward), model)
except AssertionError as e:
logger.warning(
"Hyperopt [%s]: trial #%d encountered NaN (AssertionError): %r",
finally:
if self.progressbar_callback:
self.progressbar_callback.on_training_end()
- train_env.close()
- eval_env.close()
- if hasattr(model, "env") and model.env is not None:
- model.env.close()
- del model, train_env, eval_env
+ if train_env is not None:
+ train_env.close()
+ if eval_env is not None:
+ eval_env.close()
if nan_encountered:
raise TrialPruned(f"Hyperopt [{study_name}]: NaN encountered during training")
self.eval_env = None
+def _make_dummy_vec_env(env_fns: list[Callable[[], BaseEnvironment]]) -> DummyVecEnv:
+ """Release already-created environments if a later factory or wrapper fails."""
+ environments = []
+ try:
+ for factory in env_fns:
+ environments.append(factory())
+ return DummyVecEnv([lambda env=env: env for env in environments])
+ except BaseException:
+ for environment in environments:
+ environment.close()
+ raise
+
+
def make_env(
MyRLEnv: type[BaseEnvironment],
env_id: str,
self.action_masking: bool = self.rl_config.get("action_masking", False)
# === INTERNAL STATE ===
- self._last_closed_position: Positions | None = None
self._last_closed_trade_tick: int = 0
self._max_unrealized_profit: float = -np.inf
self._min_unrealized_profit: float = np.inf
if MyRLEnv.is_unsupported_pbrs_config(
self._hold_potential_enabled, getattr(self, "add_state_info", False)
):
- logger.warning(
- "PBRS [%s]: hold_potential_enabled=True requires add_state_info=True, enabling",
- self.id,
+ raise ValueError(
+ f"PBRS [{self.id}]: hold_potential_enabled=True requires add_state_info=True "
+ "before environment construction"
)
- self.add_state_info = True
- self._set_observation_space()
# === PNL TARGET ===
self._pnl_target = float(self.profit_aim * self.rr)
)
self._pnl_target = 0.01
- def _get_next_position(self, action: int) -> Positions:
- if action == Actions.Long_enter.value and self._position == Positions.Neutral:
- return Positions.Long
- if (
- action == Actions.Short_enter.value
- and self._position == Positions.Neutral
- and self.can_short
- ):
- return Positions.Short
- if action == Actions.Long_exit.value and self._position == Positions.Long:
- return Positions.Neutral
- if action == Actions.Short_exit.value and self._position == Positions.Short:
- return Positions.Neutral
- return self._position
-
- def _get_entry_unrealized_profit(self, next_position: Positions) -> float:
- current_open = self.prices.iloc[self._current_tick].open
- if not isinstance(current_open, (int, float, np.floating)) or not np.isfinite(current_open):
- return 0.0
-
- next_pnl = 0.0
- if next_position == Positions.Long:
- current_price = self.add_exit_fee(current_open)
- last_trade_price = self.add_entry_fee(current_open)
- if not np.isclose(last_trade_price, 0.0) and np.isfinite(last_trade_price):
- next_pnl = (current_price - last_trade_price) / last_trade_price
- elif next_position == Positions.Short:
- current_price = self.add_entry_fee(current_open)
- last_trade_price = self.add_exit_fee(current_open)
- if not np.isclose(last_trade_price, 0.0) and np.isfinite(last_trade_price):
- next_pnl = (last_trade_price - current_price) / last_trade_price
-
- if not np.isfinite(next_pnl):
- return 0.0
- return float(next_pnl)
-
- def _get_next_transition_state(
- self,
- action: int,
- trade_duration: float,
- current_pnl: float,
- ) -> tuple[Positions, int, float]:
- """Compute next transition state tuple (next_position, next_duration, next_pnl).
-
- Parameters
- ----------
- action : int
- Action taken by the agent.
- trade_duration : float
- Trade duration at current tick.
- current_pnl : float
- Unrealized PnL at current tick.
-
- Returns
- -------
- tuple[Positions, int, float]
- (next_position, next_trade_duration, next_pnl) for the transition s -> s'.
- """
- next_position = self._get_next_position(action)
-
- # Entry: Neutral -> Long/Short
- if self._position == Positions.Neutral and next_position in (
- Positions.Long,
- Positions.Short,
- ):
- return next_position, 0, self._get_entry_unrealized_profit(next_position)
-
- # Exit: Long/Short -> Neutral
- if (
- self._position in (Positions.Long, Positions.Short)
- and next_position == Positions.Neutral
- ):
- return next_position, 0, 0.0
-
- # Hold: Long/Short -> Long/Short
- if self._position in (Positions.Long, Positions.Short) and next_position in (
- Positions.Long,
- Positions.Short,
- ):
- return next_position, int(trade_duration), current_pnl
-
- # Neutral self-loop
- return next_position, 0, 0.0
-
@staticmethod
def _loss_duration_multiplier(pnl_ratio: float, risk_reward_ratio: float) -> float:
if not np.isfinite(pnl_ratio) or pnl_ratio >= 0.0:
except OverflowError:
return 1.0 if x > 0 else -1.0
- if name == ReforceXY._TRANSFORM_FUNCTIONS[4]: # "asinh"
+ if name == ReforceXY._TRANSFORM_FUNCTIONS[4]: # "softsign_sqrt"
return x / math.hypot(1.0, x)
if name == ReforceXY._TRANSFORM_FUNCTIONS[5]: # "clip"
def _compute_pbrs_components(
self,
*,
- action: int,
+ previous_position: Positions,
+ next_position: Positions,
+ next_trade_duration: float,
+ next_pnl: float,
+ entry_pnl: float,
trade_duration: float,
max_trade_duration: float,
current_pnl: float,
entry_additive_scale: float,
exit_additive_scale: float,
) -> tuple[float, float, float]:
- """Compute potential-based reward shaping (PBRS) components.
-
- This method computes the PBRS shaping terms.
-
- Canonical PBRS Formula
- ----------------------
- R'(s,a,s') = R(s,a,s') + Δ(s,a,s')
-
- Non-Canonical PBRS Formula
- --------------------------
- R'(s,a,s') = R(s,a,s') + Δ(s,a,s') + entry_additive + exit_additive
-
- where:
- Δ(s,a,s') = γ·Φ(s') - Φ(s) (PBRS shaping term)
-
- Notation
- --------
- **States & Actions:**
- s : current state
- s' : next state
- a : action
-
- **Reward Components:**
- R(s,a,s') : base reward
- R'(s,a,s') : shaped reward
- Δ(s,a,s') : PBRS shaping term = γ·Φ(s') - Φ(s)
-
- **Potential Function:**
- Φ(s) : potential at state s
- γ : discount factor for shaping (gamma)
-
- **State Variables:**
- r_pnl : pnl / pnl_target (PnL ratio)
- r_dur : duration / max_duration (duration ratio, max 0)
- scale : scale parameter
- g : gain parameter
- T_x : transform function (tanh, softsign, etc.)
-
- **Hold Potential Formula:**
- m_dur = 1.0 if r_pnl >= 0 else loss_duration_multiplier(r_pnl, rr)
- Φ(s) = scale · 0.5 · [T_pnl(g·r_pnl) + sign(r_pnl)·m_dur·T_dur(g·r_dur)]
-
- PBRS Theory & Compliance
- ------------------------
- - Ng et al. 1999: potential-based shaping preserves optimal policy
- - Wiewiora et al. 2003: terminal states must have Φ(terminal) = 0
- - Invariance holds ONLY in canonical mode with additives disabled
- - Theorem: Canonical + no additives ⇒ Σ_t γ^t·Δ_t = 0 over episodes
-
- Architecture & Transitions
- --------------------------
- **Three mutually exclusive transition types:**
-
- 1. **Entry** (Neutral → Long/Short):
- - Φ(s) = 0 (neutral state has no potential)
- - Φ(s') = hold_potential(s')
- - Δ(s,a,s') = γ·Φ(s') - 0 = γ·Φ(s')
- - Optional entry additive (breaks invariance)
-
- 2. **Hold** (Long/Short → Long/Short):
- - Φ(s) = hold_potential(s)
- - Φ(s') = hold_potential(s')
- - Δ(s,a,s') = γ·Φ(s') - Φ(s)
- - Φ(s') reflects updated PnL and duration
-
- 3. **Exit** (Long/Short → Neutral):
- - Φ(s) = hold_potential(s)
- - Φ(s') depends on exit_potential_mode:
- * **canonical**: Φ(s') = 0 → Δ = -Φ(s)
- * **heuristic**: Φ(s') = f(Φ(s)) → Δ = γ·Φ(s') - Φ(s)
- - Optional exit additive (breaks invariance)
-
- Exit Potential Modes
- --------------------
- **canonical** (PBRS-compliant):
- Φ(s') = 0
- Δ = γ·0 - Φ(s) = -Φ(s)
- Additives disabled automatically
-
- **non_canonical**:
- Φ(s') = 0
- Δ = -Φ(s)
- Additives allowed (breaks invariance)
-
- **progressive_release** (heuristic):
- Φ(s') = Φ(s)·(1 - d) where d = decay_factor
- Δ = γ·Φ(s)·(1-d) - Φ(s)
-
- **spike_cancel** (heuristic):
- Φ(s') = Φ(s)/γ
- Δ = γ·(Φ(s)/γ) - Φ(s) = 0
-
- **retain_previous** (heuristic):
- Φ(s') = Φ(s)
- Δ = γ·Φ(s) - Φ(s) = (γ-1)·Φ(s)
-
- Additive Terms (Non-PBRS)
- --------------------------
- Entry and exit additives are **optional bonuses** that break PBRS invariance:
- - Entry additive: applied on Neutral→Long/Short transitions
- - Exit additive: applied on Long/Short→Neutral transitions
- - These do NOT persist in Φ(s) storage
-
- Invariance & Validation
- -----------------------
- **Theoretical Guarantee:**
- Canonical + no additives ⇒ Σ_t γ^t·Δ_t = 0
- (Φ(start) = Φ(end) = 0)
-
- **Deviations from Theory:**
- - Heuristic exit modes violate invariance
- - Entry/exit additives break policy invariance
- - Non-canonical modes introduce path dependence
-
- **Robustness:**
- - All transforms bounded: |T_x| ≤ 1
- - Validation: |Φ(s)| ≤ scale
- - Bounds: |Δ(s,a,s')| ≤ (1+γ)·scale
- - Terminal enforcement: Φ(s) = 0 when terminated
-
- Implementation Details
- ----------------------
- This method wraps the core PBRS logic for use in the RL environment:
- - Reads Φ(s) from self._last_potential (previous state potential)
- - Reads γ from self._potential_gamma
- - Reads configuration from self._exit_potential_mode, self._entry_additive_enabled, etc.
- - Computes next_position, next_duration_ratio, is_entry, is_exit internally
- - Stores Φ(s') to self._last_potential for next step
- - Updates diagnostic accumulators (_total_reward_shaping, _total_entry_additive, etc.)
+ """Compute PBRS and optional additive components for one transition.
- Parameters
- ----------
- action : int
- Action taken: determines transition type (entry/hold/exit)
- trade_duration : float
- Trade duration at current tick.
- This is the duration for state s'.
- max_trade_duration : float
- Maximum allowed trade duration (for normalization)
- current_pnl : float
- Unrealized PnL at current tick.
- This is the PnL for state s'.
- pnl_target : float
- Target PnL for ratio normalization: r_pnl = pnl / pnl_target
- hold_potential_scale : float
- Magnitude scale for hold potential (= hold_potential_ratio * base_factor)
- entry_additive_scale : float
- Magnitude scale for entry additive (= entry_additive_ratio * base_factor)
- exit_additive_scale : float
- Magnitude scale for exit additive (= exit_additive_ratio * base_factor)
+ ``previous_position`` and ``next_position`` identify entry, hold, exit,
+ or neutral self-loop transitions. ``next_pnl`` and
+ ``next_trade_duration`` describe the returned observation s', while
+ ``current_pnl`` and ``trade_duration`` describe s at the fill.
+ ``entry_pnl`` is the fee-aware PnL provisioned on an entry fill.
+
+ The shaping component is Δ = γ·Φ(s') - Φ(s). In canonical mode,
+ ``step()`` enforces a zero terminal potential. Therefore an observed
+ trajectory satisfies Σ γ^t·Δ_t = -Φ(s_0) + γ^T·Φ(s_T), which is zero
+ only when both boundary potentials are zero. Entry and exit additives
+ are returned separately and are not PBRS terms.
Returns
-------
tuple[float, float, float]
- (reward_shaping, entry_additive, exit_additive)
-
- - reward_shaping: Δ(s,a,s') = γ·Φ(s') - Φ(s), the PBRS shaping term
- - entry_additive: optional non-PBRS entry bonus (0.0 if disabled or not entry)
- - exit_additive: optional non-PBRS exit bonus (0.0 if disabled or not exit)
-
- Notes
- -----
- **State Management:**
- - Current Φ(s): read from self._last_potential
- - Next Φ(s'): computed and stored to self._last_potential
- - Transition type: inferred from self._position and action
-
- **Configuration Sources:**
- - γ: self._potential_gamma
- - Exit mode: self._exit_potential_mode
- - Additives: self._entry_additive_enabled, self._exit_additive_enabled
- - Transforms: self._hold_potential_transform_pnl, etc.
-
- **Recommendations:**
- - Use canonical mode for policy-invariant shaping
- - Monitor Σ_t γ^t·Δ_t ≈ 0 per episode in canonical mode
- - Disable additives to preserve theoretical PBRS guarantees
+ ``(reward_shaping, entry_additive, exit_additive)``.
"""
- prev_potential = float(self._last_potential)
+ prev_potential = float(self._last_potential)
if not self._hold_potential_enabled and not (
self._entry_additive_enabled or self._exit_additive_enabled
):
self._last_reward_shaping = 0.0
return 0.0, 0.0, 0.0
- next_position, next_trade_duration, next_pnl = self._get_next_transition_state(
- action=action, trade_duration=trade_duration, current_pnl=current_pnl
- )
if max_trade_duration <= 0:
next_duration_ratio = 0.0
else:
next_duration_ratio = next_trade_duration / max_trade_duration
- is_entry = self._position == Positions.Neutral and next_position in (
+ is_entry = previous_position == Positions.Neutral and next_position in (
Positions.Long,
Positions.Short,
)
is_exit = (
- self._position in (Positions.Long, Positions.Short)
+ previous_position in (Positions.Long, Positions.Short)
and next_position == Positions.Neutral
)
- is_hold = self._position in (
+ is_hold = previous_position in (
Positions.Long,
Positions.Short,
) and next_position in (Positions.Long, Positions.Short)
if is_entry and self._entry_additive_enabled and not self.is_pbrs_invariant_mode():
entry_additive = self._compute_entry_additive(
- next_pnl,
+ entry_pnl,
pnl_target,
- next_duration_ratio,
+ 0.0,
entry_additive_scale,
)
self._total_entry_additive += float(entry_additive)
Reset is called at the beginning of every episode
"""
observation, history = super().reset(seed, **kwargs)
- self._last_closed_position: Positions | None = None
self._last_closed_trade_tick: int = 0
self._max_unrealized_profit = -np.inf
self._min_unrealized_profit = np.inf
) -> float:
"""
Calculate time-based attenuation coefficient using configurable strategy
- (legacy/sqrt/linear/power/half_life). Optionally apply plateau grace period.
+ (sqrt/linear/power/half_life). Optionally apply plateau grace period.
"""
if duration_ratio < 0.0:
duration_ratio = 0.0
exit_attenuation_mode = str(
model_reward_parameters.get(
- "exit_attenuation_mode", ReforceXY._EXIT_ATTENUATION_MODES[2]
+ "exit_attenuation_mode", ReforceXY._EXIT_ATTENUATION_MODES[1]
) # "linear"
)
exit_plateau = bool(
)
exit_plateau_grace = 0.0
- def _legacy(dr: float, p: Mapping[str, Any]) -> float:
- return 1.5 if dr <= 1.0 else 0.5
-
def _sqrt(dr: float, p: Mapping[str, Any]) -> float:
return 1.0 / math.sqrt(1.0 + dr)
return math.pow(2.0, -dr / hl)
strategies: dict[str, Callable[[float, Mapping[str, Any]], float]] = {
- ReforceXY._EXIT_ATTENUATION_MODES[0]: _legacy,
- ReforceXY._EXIT_ATTENUATION_MODES[1]: _sqrt,
- ReforceXY._EXIT_ATTENUATION_MODES[2]: _linear,
- ReforceXY._EXIT_ATTENUATION_MODES[3]: _power,
- ReforceXY._EXIT_ATTENUATION_MODES[4]: _half_life,
+ ReforceXY._EXIT_ATTENUATION_MODES[0]: _sqrt,
+ ReforceXY._EXIT_ATTENUATION_MODES[1]: _linear,
+ ReforceXY._EXIT_ATTENUATION_MODES[2]: _power,
+ ReforceXY._EXIT_ATTENUATION_MODES[3]: _half_life,
}
if exit_plateau:
"PBRS [%s]: exit_attenuation_mode=%r invalid; defaulting to %r. Valid: %s",
self.id,
exit_attenuation_mode,
- ReforceXY._EXIT_ATTENUATION_MODES[2], # "linear"
+ ReforceXY._EXIT_ATTENUATION_MODES[1], # "linear"
", ".join(ReforceXY._EXIT_ATTENUATION_MODES),
)
strategy_fn = _linear
self.id,
exit_attenuation_mode,
e,
- ReforceXY._EXIT_ATTENUATION_MODES[2], # "linear"
+ ReforceXY._EXIT_ATTENUATION_MODES[1], # "linear"
effective_dr,
exc_info=True,
)
)
)
pnl_ratio = pnl / pnl_target
-
- if abs(pnl_ratio) > 1.0:
- base_pnl_target_coefficient = math.tanh(
- pnl_amplification_sensitivity * (abs(pnl_ratio) - 1.0)
- )
- win_reward_factor = float(
- model_reward_parameters.get(
- "win_reward_factor", ReforceXY.DEFAULT_WIN_REWARD_FACTOR
- )
+ win_reward_factor = float(
+ model_reward_parameters.get(
+ "win_reward_factor", ReforceXY.DEFAULT_WIN_REWARD_FACTOR
)
+ )
- if pnl_ratio > 1.0:
- pnl_target_coefficient = 1.0 + win_reward_factor * base_pnl_target_coefficient
- elif pnl_ratio < -(1.0 / self.rr):
+ if pnl_ratio > 1.0:
+ gain_coefficient = math.tanh(pnl_amplification_sensitivity * (pnl_ratio - 1.0))
+ pnl_target_coefficient = 1.0 + win_reward_factor * gain_coefficient
+ else:
+ loss_threshold = pnl_target / self.rr
+ if pnl < -loss_threshold:
+ loss_ratio = (-pnl) / loss_threshold
+ loss_coefficient = math.tanh(pnl_amplification_sensitivity * (loss_ratio - 1.0))
loss_penalty_factor = win_reward_factor * self.rr
- pnl_target_coefficient = 1.0 + loss_penalty_factor * base_pnl_target_coefficient
+ pnl_target_coefficient = 1.0 + loss_penalty_factor * loss_coefficient
return pnl_target_coefficient
) -> float:
"""
Compute exit efficiency coefficient (typically 0.5-1.5) based on exit timing quality.
+
+ The coefficient is clamped to be nonnegative so an aggressive weight or
+ center can never flip the sign of a losing exit's penalty.
"""
efficiency_weight = float(
model_reward_parameters.get("efficiency_weight", ReforceXY.DEFAULT_EFFICIENCY_WEIGHT)
efficiency_coefficient = 1.0 + efficiency_weight * (
efficiency_center - efficiency_ratio
)
+ if efficiency_coefficient < 0.0:
+ logger.warning(
+ "PBRS [%s]: efficiency_coefficient=%.5f < 0; clamping to 0",
+ self.id,
+ efficiency_coefficient,
+ )
+ efficiency_coefficient = 0.0
return efficiency_coefficient
def calculate_reward(self, action: int) -> float:
- """Compute per-step reward and apply potential-based reward shaping (PBRS).
-
- Reward Pipeline:
- 1. Invalid action penalty
- 2. Idle penalty
- 3. Hold overtime penalty
- 4. Exit reward
- 5. Default fallback (0.0 if no specific reward)
- 6. PBRS computation and application: R'(s,a,s') = R(s,a,s') + Δ(s,a,s') + entry_additive + exit_additive
-
- The final shaped reward is what the RL agent receives for learning.
- In canonical PBRS mode, the learned policy is theoretically equivalent
- to training on base rewards only (policy invariance).
-
- Parameters
- ----------
- action : int
- Action index taken by the agent
+ """Compute base reward at the current action's execution price.
- Returns
- -------
- float
- Shaped reward R'(s,a,s') = R(s,a,s') + Δ(s,a,s') + entry_additive + exit_additive
-
- Implementation: base_reward + reward_shaping + entry_additive + exit_additive
-
- where:
- - R(s,a,s') / base_reward: Base reward (invalid/idle/hold penalty or exit reward)
- - Δ(s,a,s') / reward_shaping: PBRS delta term = γ·Φ(s') - Φ(s)
- - entry_additive: Optional entry bonus (breaks PBRS invariance)
- - exit_additive: Optional exit bonus (breaks PBRS invariance)
+ Shaping is applied by step after execution and advancement, when the
+ actual next observation's position, duration and PnL are available.
"""
model_reward_parameters = self.rl_config.get("model_reward_parameters", {})
base_reward: float | None = None
if base_reward is None:
base_reward = 0.0
- # 6. Potential-based reward shaping
- hold_potential_scale = self._hold_potential_ratio * base_factor
- entry_additive_scale = self._entry_additive_ratio * base_factor
- exit_additive_scale = self._exit_additive_ratio * base_factor
-
- reward_shaping, entry_additive, exit_additive = self._compute_pbrs_components(
- action=action,
- trade_duration=trade_duration,
- max_trade_duration=max_trade_duration,
- current_pnl=pnl,
- pnl_target=self._pnl_target,
- hold_potential_scale=hold_potential_scale,
- entry_additive_scale=entry_additive_scale,
- exit_additive_scale=exit_additive_scale,
- )
-
- return base_reward + reward_shaping + entry_additive + exit_additive
+ return base_reward
def _get_observation(self) -> NDArray[np.float32]:
- start_idx = max(self._start_tick, self._current_tick - self.window_size)
+ start_idx = max(0, self._current_tick - self.window_size)
end_idx = min(self._current_tick, len(self.signal_features))
features_window = self.signal_features.iloc[start_idx:end_idx]
features_window_array = features_window.to_numpy(dtype=np.float32, copy=False)
def _exit_trade(self) -> None:
self._update_total_profit()
- self._last_closed_position = self._position
self._position = Positions.Neutral
self._last_trade_tick = None
self._last_closed_trade_tick = self._current_tick
# Exit trade based on action
if action in (Actions.Long_exit.value, Actions.Short_exit.value):
+ closed_position = self._position
self._exit_trade()
- return f"{self._last_closed_position.name}_exit"
+ return f"{closed_position.name}_exit"
return None
"""
Take a step in the environment based on the provided action
"""
- self._current_tick += 1
- self._update_unrealized_total_profit()
+ previous_equity = self._get_portfolio_equity(self.get_unrealized_profit())
+ previous_position = self._position
+ pre_trade_duration = self.get_trade_duration()
pre_pnl = self.get_unrealized_profit()
- self._update_portfolio_log_returns()
+ execution_tick = self._current_tick
+ exit_pnl = None
reward = self.calculate_reward(action)
trade_type = self.execute_trade(action)
+ entry_pnl = self.get_unrealized_profit() if previous_position == Positions.Neutral else 0.0
if trade_type is not None:
- self.append_trade_history(trade_type, self.current_price(), pre_pnl)
+ self.append_trade_history(
+ trade_type, self.current_price(), pre_pnl, execution_tick=execution_tick
+ )
+ if self._position == Positions.Neutral:
+ exit_pnl = pre_pnl
elif action != Actions.Neutral.value:
logger.warning(
"Env [%s]: invalid action=%s (%d) in position=%s at tick=%d",
self._position.name,
self._current_tick,
)
- self._position_history.append(self._position)
+ self._current_tick += 1
+ self._update_unrealized_total_profit()
+ base_factor = float(
+ self.rl_config.get("model_reward_parameters", {}).get(
+ "base_factor", ReforceXY.DEFAULT_BASE_FACTOR
+ )
+ )
+ shaping, entry_additive, exit_additive = self._compute_pbrs_components(
+ previous_position=previous_position,
+ next_position=self._position,
+ next_trade_duration=self.get_trade_duration(),
+ next_pnl=self.get_unrealized_profit(),
+ entry_pnl=entry_pnl,
+ trade_duration=pre_trade_duration,
+ max_trade_duration=max(1, self.max_trade_duration_candles),
+ current_pnl=pre_pnl,
+ pnl_target=self._pnl_target,
+ hold_potential_scale=self._hold_potential_ratio * base_factor,
+ entry_additive_scale=self._entry_additive_ratio * base_factor,
+ exit_additive_scale=self._exit_additive_ratio * base_factor,
+ )
+ reward += shaping + entry_additive + exit_additive
terminated = self.is_terminated()
+ terminal_liquidation = terminated and self._position in (Positions.Long, Positions.Short)
+ if terminal_liquidation:
+ terminal_pnl = self.get_unrealized_profit()
+ terminal_duration_ratio = self.get_trade_duration() / max(
+ 1, self.max_trade_duration_candles
+ )
+ self._update_max_unrealized_profit(terminal_pnl)
+ self._update_min_unrealized_profit(terminal_pnl)
+ liquidation_reward = terminal_pnl * self._get_exit_factor(
+ base_factor,
+ terminal_pnl,
+ terminal_duration_ratio,
+ self.rl_config.get("model_reward_parameters", {}),
+ )
+ reward += liquidation_reward
+ self._last_exit_reward += liquidation_reward
+ if self._exit_additive_enabled and not self.is_pbrs_invariant_mode():
+ liquidation_additive = self._compute_exit_additive(
+ terminal_pnl,
+ self._pnl_target,
+ terminal_duration_ratio,
+ self._exit_additive_ratio * base_factor,
+ )
+ reward += liquidation_additive
+ self._last_exit_additive += liquidation_additive
+ self._total_exit_additive += liquidation_additive
+ exit_pnl = terminal_pnl
+ closed_position = self._position
+ self._exit_trade()
+ self.append_trade_history(
+ f"{closed_position.name}_exit",
+ self.current_price(),
+ terminal_pnl,
+ execution_tick=execution_tick,
+ )
if terminated:
reward = self._apply_terminal_pbrs_correction(reward)
self._last_potential = 0.0
+ self._position_history.append(self._position)
self.total_reward += reward
pnl = self.get_unrealized_profit()
+ self._update_portfolio_log_returns(previous_equity, self._get_portfolio_equity(pnl))
self._update_max_unrealized_profit(pnl)
self._update_min_unrealized_profit(pnl)
delta_pnl = pnl - pre_pnl
trade_duration = self.get_trade_duration()
info = {
"tick": self._current_tick,
+ "execution_tick": execution_tick,
+ "terminal_liquidation": bool(terminal_liquidation),
+ "exit_pnl": exit_pnl,
"position": float(self._position.value),
"action": action,
"pre_pnl": round(pre_pnl, 5),
"most_recent_return": round(self.get_most_recent_return(), 5),
"most_recent_profit": round(self.get_most_recent_profit(), 5),
"total_profit": round(self._total_profit, 5),
- "prev_potential": round(self._last_prev_potential, 5),
- "next_potential": round(self._last_next_potential, 5),
+ "prev_potential": self._last_prev_potential,
+ "next_potential": self._last_next_potential,
"reward_entry_additive": round(self._last_entry_additive, 5),
"reward_exit_additive": round(self._last_exit_additive, 5),
- "reward_shaping": round(self._last_reward_shaping, 5),
+ "reward_shaping": self._last_reward_shaping,
"total_reward_shaping": round(self._total_reward_shaping, 5),
"reward_invalid": round(self._last_invalid_penalty, 5),
"reward_idle": round(self._last_idle_penalty, 5),
info,
)
- def append_trade_history(self, trade_type: str, price: float, profit: float) -> None:
+ def append_trade_history(
+ self, trade_type: str, price: float, profit: float, *, execution_tick: int
+ ) -> None:
self.trade_history.append(
{
- "tick": self._current_tick,
+ "tick": execution_tick,
"type": trade_type.lower(),
"price": price,
"profit": profit,
)
def is_terminated(self) -> bool:
- return (
+ return bool(
self._current_tick == self._end_tick
or self._total_profit < self.max_drawdown
or self._total_unrealized_profit < self.max_drawdown
):
self._min_unrealized_profit = pnl
+ def _get_portfolio_equity(self, pnl: float) -> float:
+ """Mark equity to liquidation using the upstream fee and staking conventions."""
+ if self.compound_trades:
+ return self._total_profit * (1.0 + pnl)
+ return self._total_profit + pnl
+
def get_most_recent_return(self) -> float:
- """
- Calculate tick-to-tick log-return for the current position.
+ """Return the stored log equity change for the last completed transition."""
+ return float(self.portfolio_log_returns[self._current_tick])
- Entry fees are applied on position transitions only (Neutral/opposite → current).
+ def _update_portfolio_log_returns(self, previous_equity: float, current_equity: float) -> None:
+ """Record log(E_after / E_before), provisioning round-trip fees at entry.
- Returns
- -------
- float
- Log-return: ln(current/previous)
- - Long: positive when price rises
- - Short: positive when price falls
- - 0.0 if no trade, neutral position, or invalid prices
+ Equity includes unrealized liquidation PnL while a position is open and
+ realized capital after exit. This accounts for the final price move
+ without charging liquidation fees twice. Non-positive or non-finite
+ equity has no finite log return and is reported as NaN, not zero.
"""
- if self._last_trade_tick is None:
- return 0.0
- if self._position == Positions.Neutral:
- return 0.0
-
- elif self._position == Positions.Long:
- current_price = self.current_price()
- previous_price = self.previous_price()
- previous_tick = self.previous_tick()
- if (
- self._position_history[previous_tick] == Positions.Short
- or self._position_history[previous_tick] == Positions.Neutral
- ):
- previous_price = self.add_entry_fee(previous_price)
-
- if (
- previous_price <= 0.0
- or not np.isfinite(previous_price)
- or current_price <= 0.0
- or not np.isfinite(current_price)
- ):
- return 0.0
-
- return np.log(current_price) - np.log(previous_price)
-
- elif self._position == Positions.Short:
- current_price = self.current_price()
- previous_price = self.previous_price()
- previous_tick = self.previous_tick()
- if (
- self._position_history[previous_tick] == Positions.Long
- or self._position_history[previous_tick] == Positions.Neutral
- ):
- previous_price = self.add_exit_fee(previous_price)
-
- if (
- previous_price <= 0.0
- or not np.isfinite(previous_price)
- or current_price <= 0.0
- or not np.isfinite(current_price)
- ):
- return 0.0
-
- return np.log(previous_price) - np.log(current_price)
-
- return 0.0
-
- def _update_portfolio_log_returns(self):
- self.portfolio_log_returns[self._current_tick] = self.get_most_recent_return()
+ if (
+ not np.isfinite(previous_equity)
+ or not np.isfinite(current_equity)
+ or previous_equity <= 0.0
+ or current_equity <= 0.0
+ ):
+ value = np.nan
+ logger.warning(
+ "Env [%s]: undefined portfolio log return at tick=%d: equity %s -> %s",
+ self.id,
+ self._current_tick,
+ previous_equity,
+ current_equity,
+ )
+ else:
+ value = math.log(current_equity) - math.log(previous_equity)
+ self.portfolio_log_returns[self._current_tick] = value
def get_most_recent_profit(self) -> float:
- """
- Calculate tick-to-tick unrealized profit ratio with fees.
-
- Returns simple return: (current - previous) / previous
- Entry/exit fees are always applied to simulate closing the position.
-
- Returns
- -------
- float
- Profit ratio (not log-return)
- - Long: (current_with_exit_fee - previous_with_entry_fee) / previous
- - Short: (previous_with_exit_fee - current_with_entry_fee) / previous
- - 0.0 if no trade, neutral position, or invalid prices
- """
- if self._last_trade_tick is None:
- return 0.0
- if self._position == Positions.Neutral:
- return 0.0
-
- elif self._position == Positions.Long:
- current_price = self.add_exit_fee(self.current_price())
- previous_price = self.add_entry_fee(self.previous_price())
-
- if (
- previous_price <= 0.0
- or not np.isfinite(previous_price)
- or current_price <= 0.0
- or not np.isfinite(current_price)
- ):
- return 0.0
-
- return (current_price - previous_price) / previous_price
-
- elif self._position == Positions.Short:
- current_price = self.add_entry_fee(self.current_price())
- previous_price = self.add_exit_fee(self.previous_price())
-
- if (
- previous_price <= 0.0
- or not np.isfinite(previous_price)
- or current_price <= 0.0
- or not np.isfinite(current_price)
- ):
- return 0.0
-
- return (previous_price - current_price) / previous_price
-
- return 0.0
-
- def previous_tick(self) -> int:
- return max(self._current_tick - 1, self._start_tick)
-
- def previous_price(self) -> float:
- return self.prices.iloc[self.previous_tick()].get("open")
+ """Return the simple equity change corresponding to the stored log return."""
+ return float(np.expm1(self.get_most_recent_return()))
def get_env_history(self) -> DataFrame:
- """
- Get environment data aligned on ticks, including optional trade events
+ """Return one metrics row per transition, joined only with price data.
+
+ Trade events remain normalized in ``trade_history``. It contains one
+ row per economic event, and ordered events may share an execution tick.
"""
if not self.history:
logger.warning("Env [%s]: history is empty", self.id)
logger.warning("Env [%s]: 'tick' column missing from history", self.id)
return DataFrame()
- _rollout_history = _history_df.copy()
- if self.trade_history:
- _trade_history_df = DataFrame(self.trade_history)
- if "tick" in _trade_history_df.columns:
- _rollout_history = merge(_rollout_history, _trade_history_df, on="tick", how="left")
-
try:
history = merge(
- _rollout_history,
+ _history_df,
self.prices,
left_on="tick",
right_index=True,
axs[0].plot(ticks, history_open, linewidth=1, color="orchid", zorder=1)
- history_type = history.get("type")
- history_price = history.get("price")
+ trades = DataFrame(self.trade_history)
+ if not trades.empty:
+ trades = trades[
+ trades["tick"].between(history["execution_tick"].min(), ticks.max())
+ ]
+ history_type = trades.get("type")
+ history_price = trades.get("price")
if history_type is not None and history_price is not None:
trade_markers_config = [
("long_enter", "^", "forestgreen", 5, -0.1, "Long enter"),
) in trade_markers_config:
mask = history_type == type_name
if mask.any():
- xs = ticks[mask]
- ys = history.loc[mask, "price"]
+ xs = trades.loc[mask, "tick"]
+ ys = trades.loc[mask, "price"]
plot_markers(axs[0], xs, ys, marker, color, size, offset)
)
self.is_pruned = True
return False
-
return True
+ def update_best_reward(self, mean_reward: float, model: Any) -> None:
+ """Update final-policy bookkeeping without reporting another trial step."""
+ _update_eval_best_reward(self, mean_reward, model)
+
class SimpleLinearSchedule:
"""
- Linear schedule (from initial value to zero),
- simpler than sb3 LinearSchedule.
+ Linear schedule from the initial value to zero. Unlike SB3's LinearSchedule,
+ the progress factor is clamped to [0, 1] so overshooting the aligned training
+ budget never produces a negative learning rate or clip range.
:param initial_value: (float or str) The initial value for the schedule
"""
self.initial_value = float(initial_value)
def __call__(self, progress_remaining: float) -> float:
- return progress_remaining * self.initial_value
+ # SB3 can request negative progress when overshooting the aligned
+ # budget (e.g. DQN finishing its train_freq); never emit a negative
+ # learning rate or clip range.
+ return min(1.0, max(0.0, float(progress_remaining))) * self.initial_value
def __repr__(self) -> str:
return f"SimpleLinearSchedule(initial_value={self.initial_value})"
"vf_coef": float(optuna_params.get("vf_coef")),
}
)
- if optuna_params.get("target_kl") is not None:
- model_params["target_kl"] = float(optuna_params.get("target_kl"))
+ if "target_kl" in optuna_params:
+ target_kl = optuna_params["target_kl"]
+ model_params["target_kl"] = None if target_kl is None else float(target_kl)
if ReforceXY._MODEL_TYPES[1] in model_type: # "RecurrentPPO"
policy_kwargs["lstm_hidden_size"] = int(optuna_params.get("lstm_hidden_size"))
policy_kwargs["n_lstm_layers"] = int(optuna_params.get("n_lstm_layers"))
)
-def sample_params_recurrentppo(trial: Trial) -> dict[str, Any]:
+def sample_params_recurrentppo(trial: Trial, *, shared_lstm: bool = False) -> dict[str, Any]:
"""
Sampler for RecurrentPPO hyperparams
"""
{
"n_lstm_layers": trial.suggest_int("n_lstm_layers", 1, 2),
"lstm_hidden_size": trial.suggest_categorical("lstm_hidden_size", [64, 128, 256, 512]),
- "enable_critic_lstm": trial.suggest_categorical("enable_critic_lstm", [True, False]),
+ "enable_critic_lstm": False
+ if shared_lstm
+ else trial.suggest_categorical("enable_critic_lstm", [True, False]),
}
)
- return convert_optuna_params_to_model_params("RecurrentPPO", ppo_optuna_params)
+ return convert_optuna_params_to_model_params(ReforceXY._MODEL_TYPES[1], ppo_optuna_params)
def get_common_dqn_optuna_params(trial: Trial) -> dict[str, Any]:
else:
min_fraction = 0.05
return {
- "train_freq": trial.suggest_categorical(
- "train_freq", [2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
- ),
+ "train_freq": trial.suggest_categorical("train_freq", ReforceXY._DQN_TRAIN_FREQS),
"subsample_steps": trial.suggest_categorical("subsample_steps", [2, 4, 8, 16]),
"gamma": trial.suggest_categorical(
"gamma", [0.93, 0.95, 0.97, 0.98, 0.99, 0.995, 0.997, 0.999, 0.9999]
"""Ensure a date series is datetime64[ms, UTC], following freqtrade's data handler pattern."""
if series is None:
raise ValueError(
- "Expected a date Series but received None. "
+ "Data: expected a date Series but received None. "
"The 'date' column is missing from the dataframe."
)
if pd.api.types.is_integer_dtype(series):
probe = int(sample.iat[0])
if not (_EPOCH_MS_MIN <= probe <= _EPOCH_MS_MAX):
raise ValueError(
- f"Integer date column value {probe} is outside the expected epoch-ms "
+ f"Data: integer date column value {probe} is outside the expected epoch-ms "
f"range [{_EPOCH_MS_MIN}, {_EPOCH_MS_MAX}]. "
"Data is likely corrupted or uses a different unit."
)
repositories / freqai-strategies.git / commitdiff
