From: Jérôme Benoit Date: Sat, 10 Oct 2026 16:02:23 +0000 (+0200) Subject: docs: align configuration and reward analysis references X-Git-Url: https://git.piment-noir.org/?a=commitdiff_plain;h=1389717f4fbb956efc032889bdfceae0bebbee98;p=freqai-strategies.git docs: align configuration and reward analysis references --- diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bbc0584..92d2eae 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -56,15 +56,22 @@ do not pin incidental wording, number formatting or arbitrary message lengths. ## Coverage gate -Both strategy suites enforce their own runtime coverage gates. ReforceXY -measures the complete `ReforceXY/user_data` tree with a 70% minimum, including -branches and non-imported namespace files. Tests, the analytical package and -QuickAdapter do not contribute to that denominator. The standalone analysis -suite has a separate 85% gate; see [its testing documentation](ReforceXY/reward_space_analysis/tests/README.md). +Both strategy suites measure their complete `user_data` trees, including branches +and non-imported namespace files. Their gates are defined in the linked coverage +configurations: + +| Suite | Coverage source | Minimum | +| ------------ | ---------------------------------------------------- | ------- | +| QuickAdapter | [`quickadapter/user_data`](quickadapter/.coveragerc) | 68% | +| ReforceXY | [`ReforceXY/user_data`](ReforceXY/.coveragerc) | 70% | + +Each strategy denominator excludes tests, the analytical package and the other +strategy. The standalone analysis suite has a separate 85% gate; see +[its testing documentation](ReforceXY/reward_space_analysis/tests/README.md). Each strategy's `.coveragerc` is selected explicitly because coverage.py does not search parent directories. Set `strategy=quickadapter` in its matching QA -image and use the same canonical runner command above for its unchanged gate. +image and use the same canonical runner command above for its own gate. The shared runner gives both coverage commands the same environment. When running `coverage run` and `coverage report` separately, export both variables for both diff --git a/ReforceXY/reward_space_analysis/README.md b/ReforceXY/reward_space_analysis/README.md index 2e5f5dd..018c9ce 100644 --- a/ReforceXY/reward_space_analysis/README.md +++ b/ReforceXY/reward_space_analysis/README.md @@ -188,6 +188,16 @@ not enum objects. Missing required columns fail the CLI. Nonnumeric values and infinities become NaN with warnings; this is not full semantic validation of a trading trajectory. +For comparisons with synthetic samples, use the same units and state semantics: + +- `pnl` is the current fee-adjusted fractional return (`0.03` means 3%), not + currency or percentage points. Include unrealized PnL during open-position + holds, not only realized exit results; neutral-position rows have zero PnL. +- `trade_duration` and `idle_duration` count candles, not minutes or seconds. + Use a comparable candle timeframe in both datasets. +- Loading coerces numeric values but does not convert units, resample transitions + or reconstruct missing open-position PnL. + Accepted envelopes: - A pandas DataFrame or other DataFrame-compatible records/column arrays. @@ -301,8 +311,11 @@ be overridden via `--params`. ### Diagnostics & Validation -- **`--check_invariants`** (numeric 0/1, default: true) – Enable runtime - invariant checks; `0` makes diagnostics advisory and can hide violations. +- **`--check_invariants`** (numeric 0/1, default: true) – Enable post-kernel + exit-factor guards for non-finite factors and negative factors with nonnegative + PnL, plus excessive-factor and negative-efficiency warnings. `0` disables these + guards and warnings; simulation assertions, strict parameter validation and + statistical validation remain active. - **`--strict_validation`** (bare flag, always true in CLI) – Enforce parameter bounds, finite checks and exact `exit_potential_mode` choices. The CLI cannot disable this. Relaxed Python API validation may clamp/canonicalize with @@ -429,6 +442,8 @@ Let `max_u = max_unrealized_profit`, `min_u = min_unrealized_profit`, `range = max_u - min_u`, `ratio = (pnl - min_u)/range`, `min_range = max(1e-6, 0.01 · pnl_target)`. Then: +- If `np.isclose(pnl, 0.0)` (`abs(pnl) <= 1e-8`): + `efficiency_coefficient = 1`; otherwise apply the following cases. - If `range < min_range`: `efficiency_coefficient = 1` (guard against division explosion) - If `pnl > 0`: @@ -542,12 +557,14 @@ The hold potential combines PnL and duration signals with an asymmetric duration multiplier for loss-side holds: ``` -Φ_hold(s) = scale · 0.5 · [T_pnl(g·r_pnl) + sign(r_pnl)·m_dur·T_dur(g·r_dur)] +Φ_hold(s) = scale · 0.5 · [T_pnl(g·r_pnl) + sign_eps(r_pnl)·m_dur·T_dur(g·r_dur)] ``` where: - `r_pnl = pnl / pnl_target` +- `sign_eps(r_pnl) = 0` when `abs(r_pnl) <= 1e-8`, otherwise `sign(r_pnl)`; + this is the near-zero guard applied by `np.isclose(r_pnl, 0.0)` - `r_dur = max(duration_ratio, 0)` - `scale = base_factor · hold_potential_ratio` - `g = hold_potential_gain` @@ -779,7 +796,8 @@ Within the same analyzer revision, identical `params_hash` values mean the resol | `*_ks_pvalue` | KS test p-value | API-only with `independent_observations=True`; omitted by the descriptive CLI | Implementation: up to 50 evenly spaced histogram edges (normally 49 bins) with -ε=1e-10; constants have zero divergence. +ε=1e-10. Identical constant distributions have zero distances; constants at +different values are compared normally and can have nonzero distances. Non-finite numeric values in real episodes are marked missing. Each feature is compared only when both synthetic and real data contain at least 10 finite diff --git a/ReforceXY/reward_space_analysis/reward_space_analysis.py b/ReforceXY/reward_space_analysis/reward_space_analysis.py index 355efa7..be956b6 100644 --- a/ReforceXY/reward_space_analysis/reward_space_analysis.py +++ b/ReforceXY/reward_space_analysis/reward_space_analysis.py @@ -2610,7 +2610,8 @@ def load_real_episodes( Returns ------- pd.DataFrame - Normalized transitions with required columns. + Transitions with required columns and numeric coercion; PnL units and + duration timeframes are not converted. """ try: @@ -2789,7 +2790,8 @@ def compute_distribution_shift_metrics( results for continuous features (pnl, trade_duration, idle_duration). The inferential KS p-value is reported only with ``independent_observations=True``; the descriptive KS statistic is always - reported. Constants yield exact zero distances and, inferentially, p=1.0. + reported. Identical constant distributions yield exact zero distances + and, inferentially, p=1.0; constants at different values do not. """ metrics = {} continuous_features = _DISTRIBUTION_SHIFT_FEATURES diff --git a/ReforceXY/reward_space_analysis/tests/README.md b/ReforceXY/reward_space_analysis/tests/README.md index f5b4a5a..0197533 100644 --- a/ReforceXY/reward_space_analysis/tests/README.md +++ b/ReforceXY/reward_space_analysis/tests/README.md @@ -153,7 +153,6 @@ Selective markers: uv run --locked --extra dev pytest -m pbrs -q uv run --locked --extra dev pytest -m robustness -q uv run --locked --extra dev pytest -m "components or robustness" -q -uv run --locked --extra dev pytest -m "not slow" -q ``` Coverage reports: @@ -163,12 +162,17 @@ uv run --locked --extra dev pytest --cov=reward_space_analysis --cov-report=term uv run --locked --extra dev pytest --cov=reward_space_analysis --cov-report=html && open htmlcov/index.html ``` -Slow statistical tests: +Focused statistical tests without the full-suite coverage gate: ```shell -uv run --locked --extra dev pytest -m "statistics and slow" -q +uv run --locked --extra dev pytest -m statistics --no-cov -q ``` +Focused runs do not establish the full-suite coverage gate. Add `--no-cov` to +other focused commands when debugging without that gate. The registered `slow` +marker is reserved: no current tests carry it, so `statistics and slow` selects +none and `not slow` excludes none. + ## Coverage Mapping (Invariant Ownership) Columns: @@ -181,52 +185,52 @@ Columns: stable across unrelated line insertions. - Notes: Sub-modes, non-owning references and multi-path coverage. -| ID | Category | Description | Owning test | Notes | -| --------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| report-additives-deterministic-092 | components | Additives deterministic report section | components/test_additives.py::test_additive_activation_deterministic_contribution | Integration/PBRS may reference outcome non-owning | -| robustness-decomposition-integrity-101 | robustness | Single active core component equals total reward under mutually exclusive scenarios | robustness/test_robustness.py::test_decomposition_integrity | Scenarios: idle, hold, exit, invalid; non-owning refs integration/test_reward_calculation.py | -| robustness-exit-mode-fallback-102 | robustness | Unknown exit_attenuation_mode falls back to linear w/ warning | robustness/test_robustness.py::test_robustness_102_unknown_exit_mode_fallback_linear | | -| robustness-negative-grace-clamp-103 | robustness | Negative exit_plateau_grace clamps to 0.0 w/ warning | robustness/test_robustness.py::test_robustness_103_negative_plateau_grace_clamped | | -| robustness-invalid-power-tau-104 | robustness | Invalid power tau falls back alpha=1.0 w/ warning | robustness/test_robustness.py::test_robustness_104_invalid_power_tau_fallback_alpha_one | | -| robustness-near-zero-half-life-105 | robustness | Near-zero half life yields no attenuation (factor≈base) | robustness/test_robustness.py::test_robustness_105_half_life_near_zero_fallback | | -| pbrs-canonical-exit-semantic-106 | pbrs | Canonical exit uses shaping=-prev_potential and next_potential=0.0 | pbrs/test_pbrs.py::test_exit_step_shaping_matches_exit_step_rules | Uses stored potential across steps; no drift correction applied | -| statistics-partial-deps-skip-107 | statistics | skip_partial_dependence => empty PD structures | statistics/test_statistics.py::test_statistics_feature_analysis_skip_partial_dependence | | -| helpers-transitions-preserve-multiplicity-108 | helpers | Repeated transitions retain their empirical multiplicity | helpers/test_utilities.py::test_repeated_transitions_preserve_multiplicity | | -| helpers-missing-cols-fill-109 | helpers | Missing required columns filled with NaN + single warning | helpers/test_utilities.py::test_missing_multiple_required_columns_single_warning | | -| statistics-binned-stats-min-edges-110 | statistics | <2 bin edges raises ValueError | statistics/test_statistics.py::test_statistics_binned_stats_invalid_bins_raises | Docstring line | -| statistics-constant-cols-exclusion-111 | statistics | Constant columns excluded & listed | statistics/test_statistics.py::test_statistics_correlation_dropped_constant_columns | Docstring line | -| statistics-degenerate-distribution-shift-112 | statistics | Constants: zero distances; KS p only with declared independent observations | statistics/test_statistics.py::test_statistics_distribution_shift_metrics_degenerate_zero | Docstring line | -| statistics-constant-dist-exact-ci-113a | statistics | Both modes retain exact constant CI bounds | statistics/test_statistics.py::test_stats_bootstrap_constant_distribution_exact_bounds | | -| statistics-percentile-outside-mean-113b | statistics | Percentile bounds need not contain the sample mean | statistics/test_statistics.py::test_stats_bootstrap_percentiles_need_not_contain_mean | | -| statistics-constant-diagnostics-115 | statistics | Constants have N/A higher moments, normality tests and Q-Q fits in both modes | statistics/test_statistics.py::test_statistics_distribution_constant_diagnostics | | -| pbrs-canonical-near-zero-report-116 | pbrs | Canonical trajectories with valid evidence are classified as verified | pbrs/test_pbrs.py::test_pbrs_canonical_near_zero_report | Requires local identity, continuity, discounted terminal boundary, and zero observed additives; the non-owning boundary test also covers a complete singleton terminal episode | -| robustness-exit-pnl-only-117 | robustness | Only exit actions have non-zero PnL | robustness/test_robustness.py::test_pnl_invariant_exit_only | | -| pbrs-absence-shift-placeholder-118 | pbrs | Placeholder shift line present when shaping shift is absent | pbrs/test_pbrs.py::test_pbrs_absence_and_distribution_shift_placeholder | | -| components-pbrs-breakdown-fields-119 | components | PBRS breakdown fields finite and mathematically aligned | components/test_reward_components.py::test_pbrs_breakdown_fields_finite_and_aligned | Tests base_reward, pbrs_delta and invariance_correction alignment | -| cli-pbrs-csv-columns-121 | cli | PBRS columns in reward_samples.csv when shaping enabled | cli/test_cli_params_and_csv.py::test_csv_contains_pbrs_columns_when_shaping_present | Verifies finite reward_base, reward_pbrs_delta and reward_invariance_correction values | -| statistics-bh-finite-family-122 | statistics | Undefined tests excluded from finite-only BH family; marked non-applicable | statistics/test_statistics.py::test_bh_excludes_undefined_tests_from_finite_family | | -| statistics-independence-contract-123 | statistics | Inferential helpers require independent_observations=True | statistics/test_statistics.py::test_inference_helpers_require_independent_observations | Covers hypothesis tests and bootstrap intervals | -| pbrs-discounted-evidence-125 | pbrs | Verification requires local identity, continuity and discounted terminal boundary | pbrs/test_pbrs.py::test_pbrs_canonical_discontinuous_potentials_report | Discontinuous potentials are not verified | -| statistics-proportional-histograms-126 | statistics | Proportional histograms ignore sample count; moved mass yields positive KL/JS | statistics/test_statistics.py::test_distribution_shift_proportional_histograms_ignore_sample_count | KL and JS remain finite and non-negative | -| pbrs-near-bound-clamp-127 | pbrs | Relaxed near-bound clamps apply exact endpoints and retain all reasons | pbrs/test_pbrs.py::test_validate_reward_parameters_records_near_bound_clamps_exactly | Includes numeric-string coercion | -| pbrs-exit-mode-validation-128 | pbrs | Exit-potential choices are strict or canonicalized; direct calls fail safe | pbrs/test_pbrs.py::test_invalid_exit_mode_warns_at_direct_and_simulation_boundaries | Direct calculation and simulation boundaries warn before fallback; PBRS calls suppress additives | -| cli-invalid-exit-mode-129 | cli | Invalid `--params exit_potential_mode` fails before artifacts | cli/test_cli_params_and_csv.py::test_invalid_exit_potential_mode_params_fails_before_artifacts | Strict CLI validation | -| cli-warning-header-recognition-130 | cli | Warning counts accept only anchored Python warning header formats | cli/test_cli_params_and_csv.py::test_warning_header_positive_and_negative_formats | Covers POSIX, relative, synthetic and Windows source locations | -| pbrs-invalid-mode-provenance-131 | pbrs | Invalid imported exit-mode metadata cannot certify canonical invariance | pbrs/test_pbrs.py::test_report_rejects_invalid_exit_mode_provenance | Preserves the invalid raw value and reports effective additive settings as unknown | -| pbrs-synthetic-fee-floor-132 | pbrs | High-fee entry loss remains in long/short synthetic PnL and exit rewards | pbrs/test_pbrs.py::test_synthetic_fee_loss_extrema_match_retained_pnl | Direct and transformed trajectories | -| pbrs-synthetic-profitable-mark-133 | pbrs | Favorable long/short marks remain profitable after high-fee synthetic transformation | pbrs/test_pbrs.py::test_synthetic_high_fee_winner_retains_profitable_mark | Covers immediate gains and recovery after an initial loss on the independent sampled market path | -| pbrs-synthetic-fee-boundary-134 | pbrs | Fee-boundary rounding is accepted while materially extreme PnL is rejected | pbrs/test_pbrs.py::test_synthetic_high_fee_short_boundary_rejects_real_excess | Non-unit entry price exposes floating-point roundoff | -| pbrs-synthetic-latent-price-135 | pbrs | Clipped candidate PnL does not erase the sampled market path for later holds | pbrs/test_pbrs.py::test_unrealized_pnl_retains_sampled_market_path_after_candidate_cap | Equal first retained marks from distinct market prices diverge after the same adverse return | -| api-unmasked-invalid-actions-136 | api | Unmasked invalid actions keep positions and receive penalties | api/test_api_helpers.py::test_unmasked_simulation_samples_invalid_actions_without_changing_position | Includes spot, futures and zero-penalty invalid actions | -| api-terminal-invalid-exit-137 | api | Wrong-side terminal exits liquidate the held side | api/test_api_helpers.py::test_invalid_terminal_exit_liquidates_the_held_position | Prevents dropped terminal trades | -| cli-stale-generated-artifacts-138 | cli | Skipped analyses remove only reserved outputs from an earlier run | cli/test_cli_params_and_csv.py::test_skipped_analysis_removes_only_stale_generated_artifacts | Preserves unrelated files | -| helpers-nonfinite-episodes-139 | helpers | Infinite real observations become missing without dropping rows | helpers/test_utilities.py::test_nonfinite_numeric_episodes_are_marked_missing | Retains transition multiplicity | -| statistics-finite-shift-140 | statistics | Finite observations remain comparable despite an infinite value | statistics/test_statistics.py::test_distribution_shift_uses_finite_observations | Rejects all-nonfinite features | -| statistics-rank-direction-141 | statistics | Rank-biserial effect follows the named first-group advantage | statistics/test_statistics.py::test_pnl_rank_biserial_direction_matches_named_first_group | Checks both directions | -| statistics-bootstrap-count-142 | statistics | Zero and negative resample counts fail for variable and constant data | statistics/test_statistics.py::test_bootstrap_rejects_nonpositive_resample_count | Rejects missing bootstrap | -| api-unmasked-sample-probabilities-144 | api | Unmasked sample probabilities match marginal valid-action frequencies | api/test_api_helpers.py::test_unmasked_sampling_probabilities_match_action_frequencies | Spot/futures entries, long/short exits, and neutral probability | -| statistics-report-inference-projection-146 | statistics | Constant confidence intervals require declared independence, even with an empty hypothesis map | statistics/test_statistics.py::test_report_projects_constant_intervals_only_with_declared_independence | Numeric projection; True → False → default rewrites the same report; non-owning bounds reference: statistics-constant-dist-exact-ci-113a | -| statistics-report-feature-availability-147 | statistics | Complete importance exports require a fitted model and finite means and standard deviations | statistics/test_statistics.py::test_report_certifies_feature_importance_only_from_finite_estimates | Real finite-zero, single-feature, empty-feature and one-test-row states; numerical CSV and availability. | +| ID | Category | Description | Owning test | Notes | +| --------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| report-additives-deterministic-092 | components | Additives deterministic report section | components/test_additives.py::test_additive_activation_deterministic_contribution | Integration/PBRS may reference outcome non-owning | +| robustness-decomposition-integrity-101 | robustness | Single active core component equals total reward under mutually exclusive scenarios | robustness/test_robustness.py::test_decomposition_integrity | Scenarios: idle, hold, exit, invalid; non-owning refs integration/test_reward_calculation.py | +| robustness-exit-mode-fallback-102 | robustness | Unknown exit_attenuation_mode falls back to linear w/ warning | robustness/test_robustness.py::test_robustness_102_unknown_exit_mode_fallback_linear | | +| robustness-negative-grace-clamp-103 | robustness | Negative exit_plateau_grace clamps to 0.0 w/ warning | robustness/test_robustness.py::test_robustness_103_negative_plateau_grace_clamped | | +| robustness-invalid-power-tau-104 | robustness | Invalid power tau falls back alpha=1.0 w/ warning | robustness/test_robustness.py::test_robustness_104_invalid_power_tau_fallback_alpha_one | | +| robustness-near-zero-half-life-105 | robustness | Near-zero half life yields no attenuation (factor≈base) | robustness/test_robustness.py::test_robustness_105_half_life_near_zero_fallback | | +| pbrs-canonical-exit-semantic-106 | pbrs | Canonical exit uses shaping=-prev_potential and next_potential=0.0 | pbrs/test_pbrs.py::test_exit_step_shaping_matches_exit_step_rules | Uses stored potential across steps; no drift correction applied | +| statistics-partial-deps-skip-107 | statistics | skip_partial_dependence => empty PD structures | statistics/test_statistics.py::test_statistics_feature_analysis_skip_partial_dependence | | +| helpers-transitions-preserve-multiplicity-108 | helpers | Repeated transitions retain their empirical multiplicity | helpers/test_utilities.py::test_repeated_transitions_preserve_multiplicity | | +| helpers-missing-cols-fill-109 | helpers | Missing required columns filled with NaN + single warning | helpers/test_utilities.py::test_missing_multiple_required_columns_single_warning | | +| statistics-binned-stats-min-edges-110 | statistics | <2 bin edges raises ValueError | statistics/test_statistics.py::test_statistics_binned_stats_invalid_bins_raises | Docstring line | +| statistics-constant-cols-exclusion-111 | statistics | Constant columns excluded & listed | statistics/test_statistics.py::test_statistics_correlation_dropped_constant_columns | Docstring line | +| statistics-degenerate-distribution-shift-112 | statistics | Identical constant distributions: zero distances; KS p only with declared independent observations | statistics/test_statistics.py::test_statistics_distribution_shift_metrics_degenerate_zero | Docstring line | +| statistics-constant-dist-exact-ci-113a | statistics | Both modes retain exact constant CI bounds | statistics/test_statistics.py::test_stats_bootstrap_constant_distribution_exact_bounds | | +| statistics-percentile-outside-mean-113b | statistics | Percentile bounds need not contain the sample mean | statistics/test_statistics.py::test_stats_bootstrap_percentiles_need_not_contain_mean | | +| statistics-constant-diagnostics-115 | statistics | Constants have N/A higher moments, normality tests and Q-Q fits in both modes | statistics/test_statistics.py::test_statistics_distribution_constant_diagnostics | | +| pbrs-canonical-near-zero-report-116 | pbrs | Canonical trajectories with valid evidence are classified as verified | pbrs/test_pbrs.py::test_pbrs_canonical_near_zero_report | Requires local identity, continuity, discounted terminal boundary, and zero observed additives; the non-owning boundary test also covers a complete singleton terminal episode | +| robustness-exit-pnl-only-117 | robustness | Neutral-position samples have zero PnL; open positions may carry unrealized PnL | robustness/test_robustness.py::test_pnl_invariant_exit_only | | +| pbrs-absence-shift-placeholder-118 | pbrs | Placeholder shift line present when shaping shift is absent | pbrs/test_pbrs.py::test_pbrs_absence_and_distribution_shift_placeholder | | +| components-pbrs-breakdown-fields-119 | components | PBRS breakdown fields finite and mathematically aligned | components/test_reward_components.py::test_pbrs_breakdown_fields_finite_and_aligned | Tests base_reward, pbrs_delta and invariance_correction alignment | +| cli-pbrs-csv-columns-121 | cli | PBRS columns in reward_samples.csv when shaping enabled | cli/test_cli_params_and_csv.py::test_csv_contains_pbrs_columns_when_shaping_present | Verifies finite reward_base, reward_pbrs_delta and reward_invariance_correction values | +| statistics-bh-finite-family-122 | statistics | Undefined tests excluded from finite-only BH family; marked non-applicable | statistics/test_statistics.py::test_bh_excludes_undefined_tests_from_finite_family | | +| statistics-independence-contract-123 | statistics | Inferential helpers require independent_observations=True | statistics/test_statistics.py::test_inference_helpers_require_independent_observations | Covers hypothesis tests and bootstrap intervals | +| pbrs-discounted-evidence-125 | pbrs | Verification requires local identity, continuity and discounted terminal boundary | pbrs/test_pbrs.py::test_pbrs_canonical_discontinuous_potentials_report | Discontinuous potentials are not verified | +| statistics-proportional-histograms-126 | statistics | Proportional histograms ignore sample count; moved mass yields positive KL/JS | statistics/test_statistics.py::test_distribution_shift_proportional_histograms_ignore_sample_count | KL and JS remain finite and non-negative | +| pbrs-near-bound-clamp-127 | pbrs | Relaxed near-bound clamps apply exact endpoints and retain all reasons | pbrs/test_pbrs.py::test_validate_reward_parameters_records_near_bound_clamps_exactly | Includes numeric-string coercion | +| pbrs-exit-mode-validation-128 | pbrs | Exit-potential choices are strict or canonicalized; direct calls fail safe | pbrs/test_pbrs.py::test_invalid_exit_mode_warns_at_direct_and_simulation_boundaries | Direct calculation and simulation boundaries warn before fallback; PBRS calls suppress additives | +| cli-invalid-exit-mode-129 | cli | Invalid `--params exit_potential_mode` fails before artifacts | cli/test_cli_params_and_csv.py::test_invalid_exit_potential_mode_params_fails_before_artifacts | Strict CLI validation | +| cli-warning-header-recognition-130 | cli | Warning counts accept only anchored Python warning header formats | cli/test_cli_params_and_csv.py::test_warning_header_positive_and_negative_formats | Covers POSIX, relative, synthetic and Windows source locations | +| pbrs-invalid-mode-provenance-131 | pbrs | Invalid imported exit-mode metadata cannot certify canonical invariance | pbrs/test_pbrs.py::test_report_rejects_invalid_exit_mode_provenance | Preserves the invalid raw value and reports effective additive settings as unknown | +| pbrs-synthetic-fee-floor-132 | pbrs | High-fee entry loss remains in long/short synthetic PnL and exit rewards | pbrs/test_pbrs.py::test_synthetic_fee_loss_extrema_match_retained_pnl | Direct and transformed trajectories | +| pbrs-synthetic-profitable-mark-133 | pbrs | Favorable long/short marks remain profitable after high-fee synthetic transformation | pbrs/test_pbrs.py::test_synthetic_high_fee_winner_retains_profitable_mark | Covers immediate gains and recovery after an initial loss on the independent sampled market path | +| pbrs-synthetic-fee-boundary-134 | pbrs | Fee-boundary rounding is accepted while materially extreme PnL is rejected | pbrs/test_pbrs.py::test_synthetic_high_fee_short_boundary_rejects_real_excess | Non-unit entry price exposes floating-point roundoff | +| pbrs-synthetic-latent-price-135 | pbrs | Clipped candidate PnL does not erase the sampled market path for later holds | pbrs/test_pbrs.py::test_unrealized_pnl_retains_sampled_market_path_after_candidate_cap | Equal first retained marks from distinct market prices diverge after the same adverse return | +| api-unmasked-invalid-actions-136 | api | Unmasked invalid actions keep positions and receive penalties | api/test_api_helpers.py::test_unmasked_simulation_samples_invalid_actions_without_changing_position | Includes spot, futures and zero-penalty invalid actions | +| api-terminal-invalid-exit-137 | api | Wrong-side terminal exits liquidate the held side | api/test_api_helpers.py::test_invalid_terminal_exit_liquidates_the_held_position | Prevents dropped terminal trades | +| cli-stale-generated-artifacts-138 | cli | Skipped analyses remove only reserved outputs from an earlier run | cli/test_cli_params_and_csv.py::test_skipped_analysis_removes_only_stale_generated_artifacts | Preserves unrelated files | +| helpers-nonfinite-episodes-139 | helpers | Infinite real observations become missing without dropping rows | helpers/test_utilities.py::test_nonfinite_numeric_episodes_are_marked_missing | Retains transition multiplicity | +| statistics-finite-shift-140 | statistics | Finite observations remain comparable despite an infinite value | statistics/test_statistics.py::test_distribution_shift_uses_finite_observations | Rejects all-nonfinite features | +| statistics-rank-direction-141 | statistics | Rank-biserial effect follows the named first-group advantage | statistics/test_statistics.py::test_pnl_rank_biserial_direction_matches_named_first_group | Checks both directions | +| statistics-bootstrap-count-142 | statistics | Zero and negative resample counts fail for variable and constant data | statistics/test_statistics.py::test_bootstrap_rejects_nonpositive_resample_count | Rejects missing bootstrap | +| api-unmasked-sample-probabilities-144 | api | Unmasked sample probabilities match marginal valid-action frequencies | api/test_api_helpers.py::test_unmasked_sampling_probabilities_match_action_frequencies | Spot/futures entries, long/short exits, and neutral probability | +| statistics-report-inference-projection-146 | statistics | Constant confidence intervals require declared independence, even with an empty hypothesis map | statistics/test_statistics.py::test_report_projects_constant_intervals_only_with_declared_independence | Numeric projection; True → False → default rewrites the same report; non-owning bounds reference: statistics-constant-dist-exact-ci-113a | +| statistics-report-feature-availability-147 | statistics | Complete importance exports require a fitted model and finite means and standard deviations | statistics/test_statistics.py::test_report_certifies_feature_importance_only_from_finite_estimates | Real finite-zero, single-feature, empty-feature and one-test-row states; numerical CSV and availability. | | cli-effective-reward-boundaries-145 | cli | Effective profit-target fallback and explicit zero idle denominator reach artifacts | cli/test_cli_params_and_csv.py::test_zero_target_and_idle_cap_reach_rewards_and_manifest | Metadata and near-target market coverage agree with runtime boundaries | | pbrs-native-integer-durations-150 | pbrs | Native integer durations and adjustment metadata retain exact values | pbrs/test_pbrs.py::test_native_integer_durations_preserve_exact_values | Python, signed NumPy and unsigned NumPy integers; strict and relaxed | diff --git a/ReforceXY/user_data/config-template.json b/ReforceXY/user_data/config-template.json index be796b5..131d50a 100644 --- a/ReforceXY/user_data/config-template.json +++ b/ReforceXY/user_data/config-template.json @@ -144,7 +144,7 @@ "rr": 2.0, "profit_aim": 0.025, "win_reward_factor": 2.0, - "max_trade_duration_candles": 96, // Maximum trade duration in candles + "max_trade_duration_candles": 96, // Reward duration scale in candles; not a live exit timeout "idle_penalty_ratio": 0.0 }, "train_cycles": 25, diff --git a/quickadapter/README.md b/quickadapter/README.md index b06a3dd..51776fa 100644 --- a/quickadapter/README.md +++ b/quickadapter/README.md @@ -77,11 +77,17 @@ no entries occur. Do not disable safety gates merely to increase trade count. ## Supplied profile versus runtime fallbacks [config-template.json](user_data/config-template.json) is the supplied profile, -not a copy of every runtime fallback. An explicit configured value takes -precedence; the reference below describes the fallback when a value is absent -or falls back through its documented validation. Review normalized startup logs +not a copy of every runtime fallback. Explicit configured values take precedence +over runtime fallbacks, subject to validation and the persisted label-state +exception in [Models and HPO](#models-and-hpo). Review normalized startup logs and Freqtrade overrides for the effective configuration. +Validation of QuickAdapter-specific settings is implemented in +[Utils.py](user_data/strategies/Utils.py), +[QuickAdapterRegressorV3.py](user_data/freqaimodels/QuickAdapterRegressorV3.py) and +[QuickAdapterV3.py](user_data/strategies/QuickAdapterV3.py). +The utility and regressor modules also define the canonical parameter fallbacks. + The template enables dry-run and spot trading on 5m candles. It selects XGBoost, 21-day training windows, 2-day backtest prediction windows, a 0.333 holdout, Kaiser–Bessel-derived label smoothing (window 6, beta 10), Isodata thresholding, @@ -117,6 +123,12 @@ for economic comparison and the limitations of threshold replay in native backtests. Changing fitted pipelines, label weighting or study schemas can require a new `freqai.identifier` or an intentional reset; see the reference. +Live and dry-run initialization restores accepted per-pair label parameters +(`label_period_candles` and `label_natr_multiplier`) saved under +`freqai.identifier` before using their configured values. Disabling HPO or +`optuna_hyperopt.warm_start` does not discard these saved parameters. To adopt +fixed label settings without restoring old state, use a new isolated identifier. + ## Configuration formats The `label_weighting`, `label_smoothing`, `label_pipeline` and @@ -166,10 +178,8 @@ in the current session keeps its in-memory score. ## Configuration reference -Runtime validation and canonical fallback definitions live in -[Utils.py](user_data/strategies/Utils.py) and -[QuickAdapterRegressorV3.py](user_data/freqaimodels/QuickAdapterRegressorV3.py). -This guide covers QuickAdapter behavior, overrides and SDK boundaries. The +The tables below describe QuickAdapter-specific settings, accepted values and +runtime fallbacks. The shipped JSON profile is the authoritative example of explicit overrides. For standard framework settings, use the [Freqtrade configuration reference][freqtrade-configuration] and the [FreqAI parameter reference][freqai-parameters]. @@ -239,11 +249,19 @@ and the [FreqAI parameter reference][freqai-parameters]. ### Data split parameters +`timeseries_split` uses only the final fold, but every split must fit its test +block. For `N` filtered rows and a test block of `T` rows, require +`N >= n_splits + 1` and `N - gap - n_splits * T > 0`. A fractional `test_size` +sets `T = int(N * test_size)`; an integer sets `T` directly. When the key is +omitted, sklearn sets `T = N // (n_splits + 1)`. Reduce or omit the supplied +`test_size=0.333` override before switching to `timeseries_split` with the +default five splits. + | Path | Runtime fallback | Type / Range | Description | | ------------------------------------------- | ------------------------------------------------------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | freqai.data_split_parameters.method | `train_test_split` | enum {`train_test_split`,`timeseries_split`} | Data splitting strategy. `train_test_split` for sequential split, `timeseries_split` for chronological split with configurable gap. | | freqai.data_split_parameters.test_size | 0.1 (`train_test_split`); dynamic (`timeseries_split`, key omitted) | float [0,1) \| int >= 0 | Outer holdout size; `0` disables the holdout (single-stage fit, `train_test_split` only). The same parameter reserves the chronological tail of the remaining training rows as inner validation for HPO and early stopping; a fractional value is relative to those remaining rows, not the original window. The holdout is predicted once and reported as weighted `holdout_rmse` in the original label scale; it measures the cold-started pre-refit selection model, not the refitted deployed model. Omitting `test_size` with `timeseries_split` uses sklearn dynamic outer sizing; inner validation then falls back to `0.1`. Explicitly supplied Freqtrade configuration values must be numeric. | -| freqai.data_split_parameters.n_splits | 5 | int >= 2 | Controls train/test proportions for `timeseries_split` (higher = larger train set). | +| freqai.data_split_parameters.n_splits | 5 | int >= 2 | Number of splits whose test blocks must fit in `timeseries_split`; only the final fold is used. Increasing it reduces the default holdout only when `test_size` is omitted. | | freqai.data_split_parameters.gap | 0 | int >= 0 | Samples to exclude between train/test for `timeseries_split`. `0` auto-derives the gap (source and lower-bound rule depend on `causal_mode`; see `causal_mode`). Not used by `train_test_split`. | | freqai.data_split_parameters.max_train_size | null | int >= 1 \| null | Maximum training set size for `timeseries_split`. When set, creates a sliding window instead of expanding train set. `null` = no limit. | @@ -255,7 +273,7 @@ and the [FreqAI parameter reference][freqai-parameters]. | freqai.label_smoothing.window_candles | 5 | int >= 1 | Requested smoothing window in candles. Runtime raises values below 3; Gaussian, Kaiser, triangular, SMM and SMA use the next odd length, `kaiser_bessel_derived` uses the next even length, and `savgol` uses an odd length greater than `polyorder`. `none` does not smooth. For `gaussian_filter1d`, this value only gates series shorter than the requested window; `sigma` defines the kernel. | | freqai.label_smoothing.beta | 8.0 | float > 0 | Shape parameter for `kaiser` and `kaiser_bessel_derived` kernels. | | freqai.label_smoothing.polyorder | 3 | int >= 0 | Polynomial order for `savgol` smoothing. | -| freqai.label_smoothing.mode | `mirror` | `savgol`: enum {`mirror`,`constant`,`nearest`,`wrap`,`interp`}; `gaussian_filter1d`: enum {`mirror`,`constant`,`nearest`,`wrap`} | Boundary mode for `savgol` and `gaussian_filter1d`; ignored otherwise. | +| freqai.label_smoothing.mode | `mirror` | `savgol`: enum {`mirror`,`constant`,`nearest`,`wrap`,`interp`}; `gaussian_filter1d`: enum {`mirror`,`constant`,`nearest`,`wrap`} | Boundary mode for `savgol` and `gaussian_filter1d`; ignored otherwise. `wrap` is rejected when `feature_parameters.causal_mode=true`. | | freqai.label_smoothing.sigma | 1.0 | float > 0 | Gaussian `sigma` for `gaussian_filter1d` smoothing. | ### Label weighting @@ -304,14 +322,14 @@ and the [FreqAI parameter reference][freqai-parameters]. | freqai.feature_parameters.min_label_natr_multiplier | 9.0 | float > 0 | Minimum labeling NATR multiplier used for reversals labeling HPO. | | freqai.feature_parameters.max_label_natr_multiplier | 12.0 | float > 0 | Maximum labeling NATR multiplier used for reversals labeling HPO. | | freqai.feature_parameters.label_frequency_candles | `auto` | int [2, 10000] \| `auto` | Reversals labeling frequency. `auto` = max(2, 2 \* number of whitelisted pairs). | -| freqai.feature_parameters.label_weights | uniform | list of 7 finite floats >= 0; sum > 0 | Per-objective weights for trial selection methods, normalized internally. Objectives: (1) number of detected reversals, (2) median swing amplitude, (3) median (swing amplitude / median volatility-threshold ratio), (4) median swing ratio of mean volume per candle to median volume (dimensionless), (5) median swing speed, (6) median swing efficiency ratio, (7) median swing volume-weighted efficiency ratio. Zero-weight objectives do not contribute to power means, including negative and zero orders; positive-weight zero values retain their usual power-mean semantics. Clustering excludes zero-weight dimensions before estimating the cluster count and forming clusters; positive weights score clusters and candidates without rescaling cluster coordinates. Weights are validated before constant-objective projection; an all-zero projected vector falls back to uniform weights. | +| freqai.feature_parameters.label_weights | uniform | list of 7 finite floats >= 0; sum > 0 | Per-objective weights for trial selection methods, normalized internally. Objectives: (1) number of detected reversals, (2) median swing amplitude, (3) median (swing amplitude / median volatility-threshold ratio), (4) median swing ratio of mean volume per candle to median volume (dimensionless), (5) median swing speed, (6) median swing efficiency ratio, (7) median swing volume-weighted efficiency ratio. Zero-weight objectives do not contribute to power means, including negative and zero orders; positive-weight zero values retain their usual power-mean semantics. Clustering excludes zero-weight dimensions before estimating the cluster count and forming clusters; positive weights score clusters and candidates without rescaling cluster coordinates. Across distance-based selection methods, `seuclidean` and `mahalanobis` warn and ignore custom distance-weight magnitudes; clustering still excludes zero-weight dimensions. Weights are validated before constant-objective projection; an all-zero projected vector falls back to uniform weights. | | freqai.feature_parameters.label_p_order | null | `minkowski`: finite float > 0; `power_mean`: finite float; null otherwise | Lp exponent for parameterized distance metrics. Used by `minkowski` distance (default 2.0) and `power_mean` distance (default 1.0). The KNN `power_mean` aggregation exponent is configured by `label_density_aggregation_param`. Ignored by other metrics. | | freqai.feature_parameters.label_method | `compromise_programming` | enum {`compromise_programming`,`topsis`,`kmeans`,`kmeans2`,`knn`,`medoid`} | HPO `label` Pareto front trial selection method. `kmedoids` is unavailable in the current Python 3.14 image. | -| freqai.feature_parameters.label_distance_metric | `euclidean` | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`,`harmonic_mean`,`geometric_mean`,`arithmetic_mean`,`quadratic_mean`,`cubic_mean`,`power_mean`,`weighted_sum`} | Distance metric for `compromise_programming` and `topsis` methods. Invalid values warn and fall back to `euclidean`. | -| freqai.feature_parameters.label_cluster_metric | `euclidean` | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`} | Distance metric for `kmeans` and `kmeans2`. Invalid values warn and fall back to `euclidean`. | +| freqai.feature_parameters.label_distance_metric | `euclidean` | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`,`harmonic_mean`,`geometric_mean`,`arithmetic_mean`,`quadratic_mean`,`cubic_mean`,`power_mean`,`weighted_sum`} | Distance metric for `compromise_programming` and `topsis` methods. Invalid values warn and fall back to `euclidean`. See `label_weights` for weighting exceptions. | +| freqai.feature_parameters.label_cluster_metric | `euclidean` | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`} | Distance metric for `kmeans` and `kmeans2`. Invalid values warn and fall back to `euclidean`. See `label_weights` for weighting exceptions. | | freqai.feature_parameters.label_cluster_selection_method | `topsis` | enum {`compromise_programming`,`topsis`} | Cluster selection method for clustering-based label methods. | | freqai.feature_parameters.label_cluster_trial_selection_method | `topsis` | enum {`compromise_programming`,`topsis`} | Best cluster trial selection method for clustering-based label methods. | -| freqai.feature_parameters.label_density_metric | method-dependent | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`} | Distance metric for `knn` and `medoid` methods. Invalid values warn and fall back to the method's natural default (`minkowski` for `knn`, `euclidean` for `medoid`). | +| freqai.feature_parameters.label_density_metric | method-dependent | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`} | Distance metric for `knn` and `medoid` methods. Invalid values warn and fall back to the method's natural default (`minkowski` for `knn`, `euclidean` for `medoid`). See `label_weights` for weighting exceptions. | | freqai.feature_parameters.label_density_aggregation | `power_mean` | enum {`power_mean`,`quantile`,`min`,`max`} | Aggregation method for KNN neighbor distances. | | freqai.feature_parameters.label_density_n_neighbors | 5 | int >= 1 | Number of neighbors for KNN. | | freqai.feature_parameters.label_density_aggregation_param | aggregation-dependent | `power_mean`: finite float; `quantile`: float [0,1]; null otherwise | Tunable for KNN neighbor distance aggregation: Lp exponent (`power_mean`) or quantile value (`quantile`). |