From b28d13b3d080b9f50e10f956c7f9887aeda0e433 Mon Sep 17 00:00:00 2001 From: =?utf8?q?J=C3=A9r=C3=B4me=20Benoit?= Date: Thu, 30 Jul 2026 03:07:34 +0200 Subject: [PATCH] fix(quickadapter): guard causal pivot-weight imputation (#158) * fix(quickadapter): guard causal pivot-weight imputation Defer non-finite pivot weights and their Gaussian bands until the frame boundary when legacy full-frame imputation can change across prefixes. * refactor(quickadapter): dedupe combined label-weight selection and align imputation-mask naming Address initial-review findings on the causal pivot-weight imputation guard (behaviour-preserving, verified bit-for-bit against the prior tree): - F1: extract _select_combined_metrics so _compute_combined_label_weights and the imputation-dependency mask share one selection path; each caller applies its own imputer, keeping the mask on the legacy full-frame _impute_weights. - F3: rename compute_label_weight_imputation_mask -> compute_label_weight_imputation_dependency_mask for naming coherence with imputation_dependency_mask / _nonfinite_imputation_dependency_mask. - F5: raise an explicit ValueError for strategy='none', mirroring compute_label_weights instead of falling through to the generic message. - F2: rewrite the label_weighting.strategy README sentence to fix a garden-path reading. * refactor(quickadapter): finish combined-weight dedup and tighten mask docstrings Address residual re-review nits (behaviour-preserving, verified bit-for-bit): - N1: extract _aggregate_imputed_metrics so _compute_combined_label_weights and the imputation-dependency mask share the vstack/aggregate step (no recompute). - N2: make _select_combined_metrics docstring describe the primitive generically instead of one consumer's causal-imputer rationale. - N3: tighten _nonfinite_imputation_dependency_mask docstring to the essential prefix-instability rationale. * refactor(quickadapter): dedupe weight-strategy error message and fix mask docstring Address residual re-review nits (behaviour-preserving, error text byte-identical): - NEW-1: extract _invalid_weight_strategy_message shared by compute_label_weight_imputation_dependency_mask and _compute_label_weight_values (removes the verbatim duplicated ValueError text). - NEW-2: correct _nonfinite_imputation_dependency_mask docstring to enumerate all prefix-unstable imputation modes (boundary default/zero, interior median). * docs(quickadapter): drop causal-availability note from label_weighting.strategy The sentence documented internal causal-availability behaviour (pivot deferral to the frame boundary), not a choice or value of the strategy tunable, and used implementation terms undefined in the README. Reverting the strategy row to main also removes the table re-padding churn, so the PR no longer touches README. * docs(quickadapter): consolidate cross-tunable behaviour into governing tunables Apply a global README rule: each tunable row documents that tunable's own role; behaviour conditioned on another tunable's value lives in the governing tunable's row (with a short pointer instead of duplication). - causal_mode: single home for all causal split-guard behaviour (split-option rejections; timeseries_split gap auto-set/rejection; train_test_split fixed purge; label-aware availability row removal incl. Zigzag confirmation, centered-smoothing availability, k-NN bandwidth confirmation, and PR #158 non-finite imputation-dependency deferral; causal epsilon baseline). - gap / fill_method / fill_sigma_candles / label_horizon_candles / support_policy / continual_learning: keep role-focused text, point to causal_mode / test_size. - Fix factual defects: optuna_hyperopt.n_jobs default (1, not CPU threads/4 which is only a cap); gpu_vram_gb type (int > 0, floored to nearest tier, not enum); label_smoothing.window_candles range (int >= 1, floored to 3 at runtime); label_frequency_candles range (int [2, 10000] | auto). * docs(quickadapter): correct causal_mode guard attribution and range constraints - causal_mode: shuffle_after_split is only causal-gated under train_test_split; it is rejected structurally under timeseries_split and held-out evaluation regardless of causal_mode, so the false-mode note no longer implies all guards are lifted (shuffle and reverse_train_test_order remain fully causal-gated). - label_pipeline.minmax_range / feature_parameters.range: document the low < high constraint (enforced by _RangeValidator / MinMaxScaler), matching robust_quantiles. * docs(quickadapter): tighten the causal band-skip guard comment Comment-only: condense the fill_radius skip-guard rationale (7 -> 4 lines) while keeping the non-obvious causal invariant (an imputation-dependent pivot is not skipped despite weight_avail == n, since its all-non-finite metric imputes to the non-zero legacy default and its band must defer to the frame boundary). * docs(quickadapter): document toggle-independent shuffle-family rejections in causal_mode NF-A: the held-out evaluation (test_size != 0) rejects shuffle, shuffle_after_split and reverse_train_test_order unconditionally, and timeseries_split rejects shuffle_after_split structurally -- both independent of causal_mode. The row now states these under clause (1) and clarifies the false branch (only the causal split-guard rejections are lifted; the toggle-independent ones remain). * perf(quickadapter): release leading stable imputations before frame boundary For single-metric weight strategies, a leading non-finite run imputes to 0.0 and becomes stable once the first finite pivot's weight is confirmed. Deferring these pivots to the frame boundary over-purged causally valid training rows. compute_label_weight_imputation_dependency_mask now returns a second leading_stable_mask (subset of dependency_mask) marking that run; empty for uniform, combined, and all-non-finite metrics. The consumer releases those pivots at weight_availability[first_finite] (the closing pivot's backfilled confirmation, not the leaky idx[first_finite]) under an identity-order guard, and skips their zero-weight Gaussian bands. combined stays deferred to the frame boundary (prefix aggregation can still shift a leading pivot), preserving causal safety. --- README.md | 220 ++++++++-------- .../user_data/strategies/QuickAdapterV3.py | 15 ++ quickadapter/user_data/strategies/Utils.py | 237 +++++++++++++++--- 3 files changed, 332 insertions(+), 140 deletions(-) diff --git a/README.md b/README.md index cd2163f..a3ae17a 100644 --- a/README.md +++ b/README.md @@ -37,116 +37,116 @@ docker compose up -d --build ### Configuration tunables -| Path | Default | Type / Range | Description | -| -------------------------------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| _Protections_ | | | | -| custom_protections.trade_duration_candles | 72 | int >= 1 | Estimated trade duration in candles. Scales protections stop duration candles and trade limit. | -| custom_protections.lookback_period_fraction | 0.5 | float (0,1] | Fraction of `fit_live_predictions_candles` used to calculate `lookback_period_candles` for _MaxDrawdown_ and _StoplossGuard_ protections. | -| custom_protections.cooldown.enabled | true | bool | Enable/disable _CooldownPeriod_ protection. | -| custom_protections.cooldown.stop_duration_candles | 4 | int >= 1 | Number of candles to wait before allowing new trades after a trade is closed. | -| custom_protections.drawdown.enabled | true | bool | Enable/disable _MaxDrawdown_ protection. | -| custom_protections.drawdown.max_allowed_drawdown | 0.2 | float (0,1) | Maximum allowed drawdown. | -| custom_protections.stoploss.enabled | true | bool | Enable/disable _StoplossGuard_ protection. | -| _Leverage_ | | | | -| leverage | `proposed_leverage` | float [1.0, max_leverage] | Leverage. Fallback to `proposed_leverage` for the pair. | -| _Exit pricing_ | | | | -| exit_pricing.trade_price_target_method | `moving_average` | enum {`moving_average`,`quantile_interpolation`,`weighted_average`} | Trade NATR computation method. | -| _Reversal confirmation_ | | | | -| reversal_confirmation.lookback_period_candles | 0 | int >= 0 | Prior confirming candles; 0 = none. | -| reversal_confirmation.decay_fraction | 0.5 | float (0,1] | Geometric per-candle volatility adjusted reversal threshold relaxation factor. | -| reversal_confirmation.min_natr_multiplier_fraction | 0.0095 | float [0,1] | Lower bound fraction (< upper bound) for volatility adjusted reversal threshold. | -| reversal_confirmation.max_natr_multiplier_fraction | 0.0125 | float [0,1] | Upper bound fraction (> lower bound) for volatility adjusted reversal threshold. | -| _Regressor model_ | | | | -| freqai.regressor | `xgboost` | enum {`xgboost`,`lightgbm`,`histgradientboostingregressor`,`ngboost`,`catboost`} | Machine learning regressor algorithm. | -| freqai.continual_learning | false | bool | Continue XGBoost or LightGBM training from the previously deployed model, so its booster grows at every retrain; delete trained models to reset. With `test_size` two-stage selection, HPO and the pre-refit selection model always cold-start (including when causal purging leaves no scorable holdout rows) and only the final refit continues, growing by the selection model's round count. Other regressors ignore any prior model. | -| _Model training parameters_ | | | | -| freqai.model_training_parameters.gpu_vram_gb | 80 | enum {8,10,12,16,24,32,40,48,64,80} | Available GPU VRAM (GB) for CatBoost, not total. Constrains `depth`, `border_count`, and `max_ctr_complexity` ranges. | -| _Data split parameters_ | | | | -| freqai.data_split_parameters.method | `train_test_split` | enum {`train_test_split`,`timeseries_split`} | Data splitting strategy. `train_test_split` for sequential split, `timeseries_split` for chronological split with configurable gap. | -| freqai.data_split_parameters.test_size | 0.1 | float [0,1) \| int >= 0 \| None | Outer holdout size; `0` disables the holdout (single-stage fit, `train_test_split` only). The same parameter reserves the chronological tail of the remaining training rows as inner validation for HPO and early stopping; a fractional value is relative to those remaining rows, not the original window. The holdout is predicted once and reported as weighted `holdout_rmse` in the original label scale; it measures the cold-started pre-refit selection model, not the refitted deployed model. `None` (sklearn dynamic sizing) applies only to `timeseries_split`; `train_test_split` requires a float or int; inner validation then falls back to `0.1`. | -| freqai.data_split_parameters.n_splits | 5 | int >= 2 | Controls train/test proportions for `timeseries_split` (higher = larger train set). | -| freqai.data_split_parameters.gap | 0 | int >= 0 | Samples to exclude between train/test for `timeseries_split`. When `0` and `causal_mode=true` (default), auto-set from `label_horizon_candles`; when `0` and `causal_mode=false`, auto-set from `label_period_candles`. Under `causal_mode=true`, an explicit `gap= 1 \| None | Maximum training set size for `timeseries_split`. When set, creates a sliding window instead of expanding train set. None = no limit. | -| _Label smoothing_ | | | | -| freqai.label_smoothing.method | `gaussian` | enum {`none`,`gaussian`,`kaiser`,`kaiser_bessel_derived`,`triang`,`smm`,`sma`,`savgol`,`gaussian_filter1d`} | Label smoothing method (`kaiser_bessel_derived` uses an even-length Kaiser-Bessel-derived zero-phase kernel; `smm`=median, `sma`=mean, `savgol`=Savitzky–Golay). | -| freqai.label_smoothing.window_candles | 5 | int >= 3 | Smoothing window length (candles). | -| freqai.label_smoothing.beta | 8.0 | float > 0 | Shape parameter for `kaiser` and `kaiser_bessel_derived` kernels. | -| freqai.label_smoothing.polyorder | 3 | int >= 0 | Polynomial order for `savgol` smoothing. | -| freqai.label_smoothing.mode | `mirror` | `savgol`: enum {`mirror`,`constant`,`nearest`,`wrap`,`interp`}; `gaussian_filter1d`: enum {`mirror`,`constant`,`nearest`,`wrap`}; ignored otherwise | Boundary mode for `savgol` and `gaussian_filter1d`. | -| freqai.label_smoothing.sigma | 1.0 | float > 0 | Gaussian `sigma` for `gaussian_filter1d` smoothing. | -| _Label weighting_ | | | | -| freqai.label_weighting.strategy | `none` | enum {`none`,`uniform`,`amplitude`,`amplitude_threshold_ratio`,`volume_rate`,`speed`,`efficiency_ratio`,`volume_weighted_efficiency_ratio`,`combined`} | Label weighting metric: none (`none`), uniform unit weight on every detected pivot (`uniform`), swing amplitude (`amplitude`), swing amplitude / median volatility-threshold ratio (`amplitude_threshold_ratio`), swing volume per candle (`volume_rate`), swing speed (`speed`), swing efficiency ratio (`efficiency_ratio`), swing volume-weighted efficiency ratio (`volume_weighted_efficiency_ratio`), or combined metrics aggregation (`combined`). Switching between `none` and any other strategy requires deleting trained models to realign training emphasis. | -| freqai.label_weighting.metric_coefficients | {} | dict[str, float] | Per-metric coefficients for `combined` strategy. Keys: `amplitude`, `amplitude_threshold_ratio`, `volume_rate`, `speed`, `efficiency_ratio`, `volume_weighted_efficiency_ratio`. | -| freqai.label_weighting.aggregation | `arithmetic_mean` | enum {`arithmetic_mean`,`geometric_mean`,`harmonic_mean`,`quadratic_mean`,`weighted_median`,`softmax`} | Metric aggregation method for `combined` strategy. `arithmetic_mean`=(Σ(w·m)/Σ(w)), `geometric_mean`=(∏(m^w))^(1/Σw), `harmonic_mean`=Σ(w)/(Σ(w/m)), `quadratic_mean`=(Σ(w·m²)/Σ(w))^(1/2), `weighted_median`=Q₀.₅(m,w), `softmax`=Σ(m·s_i) where s_i=w_i·exp(m_i/T)/Σ(w_j·exp(m_j/T)). | -| freqai.label_weighting.softmax_temperature | 1.0 | float > 0 | Temperature T for `softmax` aggregation, controls distribution sharpness. | -| freqai.label_weighting.fill_method | `zero` | enum {`zero`,`epsilon`,`gaussian`,`epsilon_gaussian`} | Off-pivot weighting scheme. `zero` hard-zeros off-pivot rows; `epsilon` applies the epsilon floor `fill_epsilon * (pivot_weights)`; in causal mode, each row's baseline uses only pivot weights available with that row's label. `gaussian` applies per-pivot Gaussian bumps; `epsilon_gaussian` sums the `epsilon` floor and the `gaussian` bumps. Pivot rows take the max of their raw weight and the off-pivot field at their index (no-op for `zero`). Switching away from `zero` may require retuning tree-leaf regularization (`min_child_weight`, `lambda`) and resetting any prior Optuna study. Changing this parameter requires deleting trained models. | -| freqai.label_weighting.fill_epsilon | 0.000001 | float [0,1] | Off-pivot fraction of the pivot baseline. Ignored when `fill_method` not in {`epsilon`,`epsilon_gaussian`}. | -| freqai.label_weighting.fill_epsilon_baseline | `mean` | enum {`mean`,`median`} | Pivot baseline statistic. `mean` tracks central tendency; `median` is robust against pivot-weight skew. Ignored when `fill_method` not in {`epsilon`,`epsilon_gaussian`}. | -| freqai.label_weighting.fill_sigma_candles | 25.0 | float >= 0.5 | Gaussian standard deviation in candles for the per-pivot bumps. Acts as the upper bound on per-pivot sigma when `fill_bandwidth == "knn"`. Under `causal_mode=true`, bumps are zero outside the finite support `ceil(4 * fill_sigma_candles)` tracked by exact availability; non-causal baselines retain the legacy unbounded Gaussian tails. Lower bound 0.5 prevents severe underflow inside the causal support. Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`}. | -| freqai.label_weighting.fill_sigma_min_candles | 0.5 | float >= 0.5 | Lower bound on per-pivot sigma in candles when `fill_bandwidth == "knn"`. Clipped to `fill_sigma_candles` when larger. Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`} or `fill_bandwidth != "knn"`. | -| freqai.label_weighting.fill_bandwidth | `fixed` | enum {`fixed`,`knn`} | Per-pivot Gaussian bandwidth selector. `fixed` applies a constant `fill_sigma_candles` to every pivot (legacy behavior). `knn` adapts each pivot's sigma to local pivot density via `sigma_p = clip(fill_bandwidth_alpha * d_k(p), fill_sigma_min_candles, fill_sigma_candles)` where `d_k(p)` is the index distance to the `k`-th nearest pivot neighbor (Loftsgaarden & Quesenberry 1965; Silverman 1986, §5.2). Mitigates the crushing of weaker pivots by stronger neighbors in dense clusters. Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`}. | -| freqai.label_weighting.fill_bandwidth_neighbors | 1 | int >= 1 | `k` for the k-nearest-neighbor bandwidth selector. Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`} or `fill_bandwidth != "knn"`. | -| freqai.label_weighting.fill_bandwidth_alpha | 0.5 | float > 0 | Multiplicative factor on the k-th neighbor distance. Smaller values produce sharper, more separated Gaussians; larger values approach the `fixed` behavior. Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`} or `fill_bandwidth != "knn"`. | -| freqai.label_weighting.support_policy | `fallback` | enum {`fallback`,`raise`} | Policy when active label weighting fails support checks (after causal split guards and feature-pipeline row filtering). `raise` aborts the fit with `ValueError`; `fallback` logs a `WARNING` and uses sanitized base sample weights for that fit. Eval (test/val) weights bypass this policy and always fall back on composition errors. | -| freqai.label_weighting.min_pivot_equivalent_count | 3 | int >= 1 | Minimum number of surviving pivot-equivalent label weights required after filtering. Pivot-equivalent rows are weights at least 10% of the surviving maximum label weight. | -| freqai.label_weighting.min_positive_label_weight_fraction | 0.01 | float [0,1] | Minimum fraction of filtered training rows with finite positive label weights. | -| freqai.label_weighting.min_effective_sample_size | 3.0 | float >= 1 | Minimum Kish effective sample size of the final composed training weights. | -| _Label pipeline_ | | | | -| freqai.label_pipeline.standardization | `none` | enum {`none`,`zscore`,`robust`,`mmad`,`power_yj`} | Standardization method applied to labels before normalization. `none`=w, `zscore`=(w-μ)/σ, `robust`=(w-median)/(Q₃-Q₁), `mmad`=(w-median)/(MAD·k), `power_yj`=YJ(w). | -| freqai.label_pipeline.robust_quantiles | [0.25, 0.75] | list[float] where 0 <= Q1 < Q3 <= 1 | Quantile range for robust standardization, Q1 and Q3. | -| freqai.label_pipeline.mmad_scaling_factor | 1.4826 | float > 0 | Scaling factor for MMAD standardization. | -| freqai.label_pipeline.normalization | `maxabs` | enum {`maxabs`,`minmax`,`sigmoid`,`none`} | Normalization method applied to labels. `maxabs`=w/max(\|w\|), `minmax`=low+(w-min)/(max-min)·(high-low), `sigmoid`=2·σ(scale·w)-1, `none`=w. | -| freqai.label_pipeline.minmax_range | [-1.0, 1.0] | list[float] | Target range for `minmax` normalization, min and max. | -| freqai.label_pipeline.sigmoid_scale | 1.0 | float > 0 | Scale parameter for `sigmoid` normalization, controls steepness. | -| freqai.label_pipeline.gamma | 1.0 | float (0,10] | Contrast exponent applied to labels after normalization: >1 emphasizes extrema, values between 0 and 1 soften. | -| _Feature parameters_ | | | | -| freqai.feature_parameters.label_period_candles | min/max midpoint | int >= 1 | Zigzag labeling NATR period. | -| freqai.feature_parameters.label_horizon_candles | `label_period_candles` | int >= 1 | Conservative fixed purge used by causal train/test guards and as the default `timeseries_split` gap. Zigzag labels additionally expose their exact row-wise confirmation time; centered smoothing composes the maximum availability time across each kernel support. When unset, falls back to `label_period_candles`. | -| freqai.feature_parameters.causal_mode | true | bool | Causal split guard toggle. When `true` (default): rejects `data_split_parameters.shuffle=true`, `shuffle_after_split=true`, `reverse_train_test_order=true`; for `timeseries_split` auto-sets `gap=label_horizon_candles` when unset/`0` (rejects explicit `gap= 1 | Minimum labeling NATR period used for reversals labeling HPO. | -| freqai.feature_parameters.max_label_period_candles | 24 | int >= 1 | Maximum labeling NATR period used for reversals labeling HPO. | -| freqai.feature_parameters.label_natr_multiplier | min/max midpoint | float > 0 | Zigzag labeling NATR multiplier. | -| freqai.feature_parameters.min_label_natr_multiplier | 9.0 | float > 0 | Minimum labeling NATR multiplier used for reversals labeling HPO. | -| freqai.feature_parameters.max_label_natr_multiplier | 12.0 | float > 0 | Maximum labeling NATR multiplier used for reversals labeling HPO. | -| freqai.feature_parameters.label_frequency_candles | `auto` | int >= 2 \| `auto` | Reversals labeling frequency. `auto` = max(2, 2 \* number of whitelisted pairs). | -| freqai.feature_parameters.label_weights | [1/7,1/7,1/7,1/7,1/7,1/7,1/7] | list[float] | Per-objective weights for trial selection methods. Objectives: (1) number of detected reversals, (2) median swing amplitude, (3) median (swing amplitude / median volatility-threshold ratio), (4) median swing volume per candle, (5) median swing speed, (6) median swing efficiency ratio, (7) median swing volume-weighted efficiency ratio. | -| freqai.feature_parameters.label_p_order | None | float \| None | Lp exponent for parameterized metrics. Used by `minkowski` distance (default 2.0) and `power_mean` aggregation (default 1.0). Ignored by other metrics. | -| freqai.feature_parameters.label_method | `compromise_programming` | enum {`compromise_programming`,`topsis`,`kmeans`,`kmeans2`,`kmedoids`,`knn`,`medoid`} | HPO `label` Pareto front trial selection method. | -| freqai.feature_parameters.label_distance_metric | `euclidean` | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`,`harmonic_mean`,`geometric_mean`,`arithmetic_mean`,`quadratic_mean`,`cubic_mean`,`power_mean`,`weighted_sum`} | Distance metric for `compromise_programming` and `topsis` methods. Invalid values warn and fall back to `euclidean`. | -| freqai.feature_parameters.label_cluster_metric | `euclidean` | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`} | Distance metric for `kmeans`, `kmeans2`, and `kmedoids` methods. Invalid values warn and fall back to `euclidean`. | -| freqai.feature_parameters.label_cluster_selection_method | `topsis` | enum {`compromise_programming`,`topsis`} | Cluster selection method for clustering-based label methods. | -| freqai.feature_parameters.label_cluster_trial_selection_method | `topsis` | enum {`compromise_programming`,`topsis`} | Best cluster trial selection method for clustering-based label methods. | -| freqai.feature_parameters.label_density_metric | method-dependent | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`} | Distance metric for `knn` and `medoid` methods. Invalid values warn and fall back to the method's natural default (`minkowski` for `knn`, `euclidean` for `medoid`). | -| freqai.feature_parameters.label_density_aggregation | `power_mean` | enum {`power_mean`,`quantile`,`min`,`max`} | Aggregation method for KNN neighbor distances. | -| freqai.feature_parameters.label_density_n_neighbors | 5 | int >= 1 | Number of neighbors for KNN. | -| freqai.feature_parameters.label_density_aggregation_param | aggregation-dependent | float \| None | Tunable for KNN neighbor distance aggregation: Lp exponent (`power_mean`) or quantile value (`quantile`). | -| freqai.feature_parameters.scaler | `minmax` | enum {`minmax`,`maxabs`,`standard`,`robust`} | Feature scaling method. `minmax`=MinMaxScaler, `maxabs`=MaxAbsScaler, `standard`=StandardScaler, `robust`=RobustScaler. Changing this parameter requires deleting trained models. | -| freqai.feature_parameters.range | [-1.0, 1.0] | list[float] | Target range for `minmax` scaler, min and max. Changing this parameter requires deleting trained models. | -| _Label prediction_ | | | | -| freqai.label_prediction.method | `thresholding` | enum {`none`,`thresholding`} | Prediction method. `none` disables threshold computation, `thresholding` enables adaptive threshold calculation. | -| freqai.label_prediction.selection_method | `rank_extrema` | enum {`rank_extrema`,`rank_peaks`,`partition`} | Extrema selection method. `rank_extrema` ranks extrema values, `rank_peaks` ranks detected peak values, `partition` uses sign-based partitioning. | -| freqai.label_prediction.threshold_method | `mean` | enum {`mean`,`isodata`,`li`,`minimum`,`otsu`,`triangle`,`yen`,`median`,`soft_extremum`} | Thresholding method for prediction thresholds. | -| freqai.label_prediction.soft_extremum_alpha | 12.0 | float >= 0 | Alpha for `soft_extremum` threshold method. | -| freqai.label_prediction.outlier_quantile | 0.999 | float (0,1) | Quantile threshold for predictions outlier filtering. | -| freqai.label_prediction.keep_fraction | 0.0075 | float (0,1] | Fraction of extrema used for thresholds. 1 uses all, lower values keep only most significant. Applies to `rank_extrema` and `rank_peaks`; ignored for `partition`. | -| _Optuna / HPO_ | | | | -| freqai.optuna_hyperopt.enabled | false | bool | Enables regressor and dynamic label HPO. | -| freqai.optuna_hyperopt.sampler | `tpe` | enum {`tpe`,`auto`} | HPO sampler algorithm for `hp` namespace. `tpe` uses [TPESampler](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html) with multivariate, group, and constant_liar (when multiple workers), `auto` uses [AutoSampler](https://hub.optuna.org/samplers/auto_sampler). | -| freqai.optuna_hyperopt.label_sampler | `auto` | enum {`auto`,`tpe`,`nsgaii`,`nsgaiii`} | HPO sampler algorithm for multi-objective `label` namespace. `nsgaii` uses [NSGAIISampler](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html), `nsgaiii` uses [NSGAIIISampler](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html). | -| freqai.optuna_hyperopt.storage | `file` | enum {`file`,`sqlite`} | HPO storage backend. | -| freqai.optuna_hyperopt.continuous | true | bool | Continuous HPO. Forced for both namespaces in backtest and hyperopt, resetting the study on each optimization. | -| freqai.optuna_hyperopt.warm_start | true | bool | Warm start HPO with previous best value(s). Persisted values are loaded and saved only in live and dry-run modes; non-live runs reuse only values produced earlier in the same run. | -| freqai.optuna_hyperopt.n_startup_trials | 15 | int >= 0 | HPO startup trials. | -| freqai.optuna_hyperopt.n_trials | 50 | int >= 1 | Maximum HPO trials. | -| freqai.optuna_hyperopt.n_jobs | CPU threads / 4 | int >= 1 | Parallel HPO workers. | -| freqai.optuna_hyperopt.timeout | 7200 | int >= 0 | HPO wall-clock timeout in seconds. | -| freqai.optuna_hyperopt.label_candles_step | 1 | int >= 1 | Step for Zigzag NATR period `label` search space. | -| freqai.optuna_hyperopt.space_reduction | false | bool | Enable/disable `hp` search space reduction based on previous best parameters. | -| freqai.optuna_hyperopt.space_fraction | 0.4 | float [0,1] | Fraction of the `hp` search space to use with `space_reduction`. Lower values create narrower search ranges around the best parameters. | -| freqai.optuna_hyperopt.min_resource | 3 | int >= 1 | Minimum resource per [HyperbandPruner](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.HyperbandPruner.html) rung. | -| freqai.optuna_hyperopt.seed | 1 | int >= 0 | HPO RNG seed used by the Optuna samplers and label-candle shuffling. | -| freqai.optuna_hyperopt.reset_label_study_on_schema_mismatch | true | bool | Reset a persisted `label` study when its selection schema is missing, invalid, or incompatible. `true` keeps the historical destructive reset, deleting the study before recreating it; `false` preserves its trials and stored metadata, permits caller-managed reuse in memory, and does not persist selected params until the schema is reconciled. Both fail closed: an inspection error, or (under `true`) a deletion error, aborts study creation. Has no effect when `continuous=true` or outside live/dry-run modes, where studies are always reset. | -| freqai.optuna_hyperopt.vary_model_seed_by_trial | true | bool | Add `trial.number` to each regressor's configured model seed (or its default seed of `1`) during HPO. `true` samples model randomness across trials and preserves the historical behavior; `false` evaluates every trial and the final fit with the same model seed. This does not change `freqai.optuna_hyperopt.seed`. | +| Path | Default | Type / Range | Description | +| -------------------------------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| _Protections_ | | | | +| custom_protections.trade_duration_candles | 72 | int >= 1 | Estimated trade duration in candles. Scales protections stop duration candles and trade limit. | +| custom_protections.lookback_period_fraction | 0.5 | float (0,1] | Fraction of `fit_live_predictions_candles` used to calculate `lookback_period_candles` for _MaxDrawdown_ and _StoplossGuard_ protections. | +| custom_protections.cooldown.enabled | true | bool | Enable/disable _CooldownPeriod_ protection. | +| custom_protections.cooldown.stop_duration_candles | 4 | int >= 1 | Number of candles to wait before allowing new trades after a trade is closed. | +| custom_protections.drawdown.enabled | true | bool | Enable/disable _MaxDrawdown_ protection. | +| custom_protections.drawdown.max_allowed_drawdown | 0.2 | float (0,1) | Maximum allowed drawdown. | +| custom_protections.stoploss.enabled | true | bool | Enable/disable _StoplossGuard_ protection. | +| _Leverage_ | | | | +| leverage | `proposed_leverage` | float [1.0, max_leverage] | Leverage. Fallback to `proposed_leverage` for the pair. | +| _Exit pricing_ | | | | +| exit_pricing.trade_price_target_method | `moving_average` | enum {`moving_average`,`quantile_interpolation`,`weighted_average`} | Trade NATR computation method. | +| _Reversal confirmation_ | | | | +| reversal_confirmation.lookback_period_candles | 0 | int >= 0 | Prior confirming candles; 0 = none. | +| reversal_confirmation.decay_fraction | 0.5 | float (0,1] | Geometric per-candle volatility adjusted reversal threshold relaxation factor. | +| reversal_confirmation.min_natr_multiplier_fraction | 0.0095 | float [0,1] | Lower bound fraction (< upper bound) for volatility adjusted reversal threshold. | +| reversal_confirmation.max_natr_multiplier_fraction | 0.0125 | float [0,1] | Upper bound fraction (> lower bound) for volatility adjusted reversal threshold. | +| _Regressor model_ | | | | +| freqai.regressor | `xgboost` | enum {`xgboost`,`lightgbm`,`histgradientboostingregressor`,`ngboost`,`catboost`} | Machine learning regressor algorithm. | +| freqai.continual_learning | false | bool | Continue XGBoost or LightGBM training from the previously deployed model, so its booster grows at every retrain; delete trained models to reset. Under `test_size` two-stage selection, HPO and the pre-refit selection model cold-start and only the final refit continues, growing by the selection model's round count (see `test_size`). Other regressors ignore any prior model. | +| _Model training parameters_ | | | | +| freqai.model_training_parameters.gpu_vram_gb | 80 | int > 0 | Available GPU VRAM (GB) for CatBoost, not total. Any positive value is floored to the nearest supported tier `<= value` (tiers 8, 10, 12, 16, 24, 32, 40, 48, 64, 80; values below 8 use tier 8). Constrains `depth`, `border_count`, and `max_ctr_complexity` ranges. | +| _Data split parameters_ | | | | +| freqai.data_split_parameters.method | `train_test_split` | enum {`train_test_split`,`timeseries_split`} | Data splitting strategy. `train_test_split` for sequential split, `timeseries_split` for chronological split with configurable gap. | +| freqai.data_split_parameters.test_size | 0.1 | float [0,1) \| int >= 0 \| None | Outer holdout size; `0` disables the holdout (single-stage fit, `train_test_split` only). The same parameter reserves the chronological tail of the remaining training rows as inner validation for HPO and early stopping; a fractional value is relative to those remaining rows, not the original window. The holdout is predicted once and reported as weighted `holdout_rmse` in the original label scale; it measures the cold-started pre-refit selection model, not the refitted deployed model. `None` (sklearn dynamic sizing) applies only to `timeseries_split`; `train_test_split` requires a float or int; inner validation then falls back to `0.1`. | +| freqai.data_split_parameters.n_splits | 5 | int >= 2 | Controls train/test proportions for `timeseries_split` (higher = larger train set). | +| freqai.data_split_parameters.gap | 0 | int >= 0 | Samples to exclude between train/test for `timeseries_split`. `0` auto-derives the gap (source and lower-bound rule depend on `causal_mode`; see `causal_mode`). Not used by `train_test_split`. | +| freqai.data_split_parameters.max_train_size | None | int >= 1 \| None | Maximum training set size for `timeseries_split`. When set, creates a sliding window instead of expanding train set. None = no limit. | +| _Label smoothing_ | | | | +| freqai.label_smoothing.method | `gaussian` | enum {`none`,`gaussian`,`kaiser`,`kaiser_bessel_derived`,`triang`,`smm`,`sma`,`savgol`,`gaussian_filter1d`} | Label smoothing method (`kaiser_bessel_derived` uses an even-length Kaiser-Bessel-derived zero-phase kernel; `smm`=median, `sma`=mean, `savgol`=Savitzky–Golay). | +| freqai.label_smoothing.window_candles | 5 | int >= 1 | Smoothing window length (candles). Values below 3 are raised to 3 at runtime. | +| freqai.label_smoothing.beta | 8.0 | float > 0 | Shape parameter for `kaiser` and `kaiser_bessel_derived` kernels. | +| freqai.label_smoothing.polyorder | 3 | int >= 0 | Polynomial order for `savgol` smoothing. | +| freqai.label_smoothing.mode | `mirror` | `savgol`: enum {`mirror`,`constant`,`nearest`,`wrap`,`interp`}; `gaussian_filter1d`: enum {`mirror`,`constant`,`nearest`,`wrap`}; ignored otherwise | Boundary mode for `savgol` and `gaussian_filter1d`. | +| freqai.label_smoothing.sigma | 1.0 | float > 0 | Gaussian `sigma` for `gaussian_filter1d` smoothing. | +| _Label weighting_ | | | | +| freqai.label_weighting.strategy | `none` | enum {`none`,`uniform`,`amplitude`,`amplitude_threshold_ratio`,`volume_rate`,`speed`,`efficiency_ratio`,`volume_weighted_efficiency_ratio`,`combined`} | Label weighting metric: none (`none`), uniform unit weight on every detected pivot (`uniform`), swing amplitude (`amplitude`), swing amplitude / median volatility-threshold ratio (`amplitude_threshold_ratio`), swing volume per candle (`volume_rate`), swing speed (`speed`), swing efficiency ratio (`efficiency_ratio`), swing volume-weighted efficiency ratio (`volume_weighted_efficiency_ratio`), or combined metrics aggregation (`combined`). Switching between `none` and any other strategy requires deleting trained models to realign training emphasis. | +| freqai.label_weighting.metric_coefficients | {} | dict[str, float] | Per-metric coefficients for `combined` strategy. Keys: `amplitude`, `amplitude_threshold_ratio`, `volume_rate`, `speed`, `efficiency_ratio`, `volume_weighted_efficiency_ratio`. | +| freqai.label_weighting.aggregation | `arithmetic_mean` | enum {`arithmetic_mean`,`geometric_mean`,`harmonic_mean`,`quadratic_mean`,`weighted_median`,`softmax`} | Metric aggregation method for `combined` strategy. `arithmetic_mean`=(Σ(w·m)/Σ(w)), `geometric_mean`=(∏(m^w))^(1/Σw), `harmonic_mean`=Σ(w)/(Σ(w/m)), `quadratic_mean`=(Σ(w·m²)/Σ(w))^(1/2), `weighted_median`=Q₀.₅(m,w), `softmax`=Σ(m·s_i) where s_i=w_i·exp(m_i/T)/Σ(w_j·exp(m_j/T)). | +| freqai.label_weighting.softmax_temperature | 1.0 | float > 0 | Temperature T for `softmax` aggregation, controls distribution sharpness. | +| freqai.label_weighting.fill_method | `zero` | enum {`zero`,`epsilon`,`gaussian`,`epsilon_gaussian`} | Off-pivot weighting scheme. `zero` hard-zeros off-pivot rows; `epsilon` applies the epsilon floor `fill_epsilon * (pivot_weights)`; `gaussian` applies per-pivot Gaussian bumps; `epsilon_gaussian` sums the `epsilon` floor and the `gaussian` bumps. Pivot rows take the max of their raw weight and the off-pivot field at their index (no-op for `zero`). Under `causal_mode=true` the epsilon baseline is computed causally (see `causal_mode`). Switching away from `zero` may require retuning tree-leaf regularization (`min_child_weight`, `lambda`) and resetting any prior Optuna study. Changing this parameter requires deleting trained models. | +| freqai.label_weighting.fill_epsilon | 0.000001 | float [0,1] | Off-pivot fraction of the pivot baseline. Ignored when `fill_method` not in {`epsilon`,`epsilon_gaussian`}. | +| freqai.label_weighting.fill_epsilon_baseline | `mean` | enum {`mean`,`median`} | Pivot baseline statistic. `mean` tracks central tendency; `median` is robust against pivot-weight skew. Ignored when `fill_method` not in {`epsilon`,`epsilon_gaussian`}. | +| freqai.label_weighting.fill_sigma_candles | 25.0 | float >= 0.5 | Gaussian standard deviation in candles for the per-pivot bumps. Acts as the upper bound on per-pivot sigma when `fill_bandwidth == "knn"`. Lower bound 0.5 prevents severe underflow in the Gaussian tail. Under `causal_mode=true` the bumps use a finite support `ceil(4 * fill_sigma_candles)` (see `causal_mode`). Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`}. | +| freqai.label_weighting.fill_sigma_min_candles | 0.5 | float >= 0.5 | Lower bound on per-pivot sigma in candles when `fill_bandwidth == "knn"`. Clipped to `fill_sigma_candles` when larger. Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`} or `fill_bandwidth != "knn"`. | +| freqai.label_weighting.fill_bandwidth | `fixed` | enum {`fixed`,`knn`} | Per-pivot Gaussian bandwidth selector. `fixed` applies a constant `fill_sigma_candles` to every pivot (legacy behavior). `knn` adapts each pivot's sigma to local pivot density via `sigma_p = clip(fill_bandwidth_alpha * d_k(p), fill_sigma_min_candles, fill_sigma_candles)` where `d_k(p)` is the index distance to the `k`-th nearest pivot neighbor (Loftsgaarden & Quesenberry 1965; Silverman 1986, §5.2). Mitigates the crushing of weaker pivots by stronger neighbors in dense clusters. Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`}. | +| freqai.label_weighting.fill_bandwidth_neighbors | 1 | int >= 1 | `k` for the k-nearest-neighbor bandwidth selector. Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`} or `fill_bandwidth != "knn"`. | +| freqai.label_weighting.fill_bandwidth_alpha | 0.5 | float > 0 | Multiplicative factor on the k-th neighbor distance. Smaller values produce sharper, more separated Gaussians; larger values approach the `fixed` behavior. Ignored when `fill_method` not in {`gaussian`,`epsilon_gaussian`} or `fill_bandwidth != "knn"`. | +| freqai.label_weighting.support_policy | `fallback` | enum {`fallback`,`raise`} | Policy when active label weighting fails support checks (evaluated on the training rows surviving upstream filtering). `raise` aborts the fit with `ValueError`; `fallback` logs a `WARNING` and uses sanitized base sample weights for that fit. Eval (test/val) weights bypass this policy and always fall back on composition errors. | +| freqai.label_weighting.min_pivot_equivalent_count | 3 | int >= 1 | Minimum number of surviving pivot-equivalent label weights required after filtering. Pivot-equivalent rows are weights at least 10% of the surviving maximum label weight. | +| freqai.label_weighting.min_positive_label_weight_fraction | 0.01 | float [0,1] | Minimum fraction of filtered training rows with finite positive label weights. | +| freqai.label_weighting.min_effective_sample_size | 3.0 | float >= 1 | Minimum Kish effective sample size of the final composed training weights. | +| _Label pipeline_ | | | | +| freqai.label_pipeline.standardization | `none` | enum {`none`,`zscore`,`robust`,`mmad`,`power_yj`} | Standardization method applied to labels before normalization. `none`=w, `zscore`=(w-μ)/σ, `robust`=(w-median)/(Q₃-Q₁), `mmad`=(w-median)/(MAD·k), `power_yj`=YJ(w). | +| freqai.label_pipeline.robust_quantiles | [0.25, 0.75] | list[float] where 0 <= Q1 < Q3 <= 1 | Quantile range for robust standardization, Q1 and Q3. | +| freqai.label_pipeline.mmad_scaling_factor | 1.4826 | float > 0 | Scaling factor for MMAD standardization. | +| freqai.label_pipeline.normalization | `maxabs` | enum {`maxabs`,`minmax`,`sigmoid`,`none`} | Normalization method applied to labels. `maxabs`=w/max(\|w\|), `minmax`=low+(w-min)/(max-min)·(high-low), `sigmoid`=2·σ(scale·w)-1, `none`=w. | +| freqai.label_pipeline.minmax_range | [-1.0, 1.0] | list[float], low < high | Target range for `minmax` normalization, min and max. | +| freqai.label_pipeline.sigmoid_scale | 1.0 | float > 0 | Scale parameter for `sigmoid` normalization, controls steepness. | +| freqai.label_pipeline.gamma | 1.0 | float (0,10] | Contrast exponent applied to labels after normalization: >1 emphasizes extrema, values between 0 and 1 soften. | +| _Feature parameters_ | | | | +| freqai.feature_parameters.label_period_candles | min/max midpoint | int >= 1 | Zigzag labeling NATR period. | +| freqai.feature_parameters.label_horizon_candles | `label_period_candles` | int >= 1 | Conservative fixed purge horizon in candles: the magnitude of the causal guards' purge and of the default `timeseries_split` gap (see `causal_mode` for how the guards consume it). When unset, falls back to `label_period_candles`. | +| freqai.feature_parameters.causal_mode | true | bool | Causal split-guard master toggle. When `true` (default): (1) rejects `data_split_parameters.shuffle=true`, `feature_parameters.shuffle_after_split=true`, and `feature_parameters.reverse_train_test_order=true` (two of these rejections are independent of this toggle: `timeseries_split` rejects `shuffle_after_split` structurally, and an active holdout `test_size != 0` rejects all three at evaluation); (2) for `timeseries_split`, auto-sets `gap=label_horizon_candles` when `gap` is unset or `0` and rejects an explicit `gap= 1 | Minimum labeling NATR period used for reversals labeling HPO. | +| freqai.feature_parameters.max_label_period_candles | 24 | int >= 1 | Maximum labeling NATR period used for reversals labeling HPO. | +| freqai.feature_parameters.label_natr_multiplier | min/max midpoint | float > 0 | Zigzag labeling NATR multiplier. | +| freqai.feature_parameters.min_label_natr_multiplier | 9.0 | float > 0 | Minimum labeling NATR multiplier used for reversals labeling HPO. | +| freqai.feature_parameters.max_label_natr_multiplier | 12.0 | float > 0 | Maximum labeling NATR multiplier used for reversals labeling HPO. | +| freqai.feature_parameters.label_frequency_candles | `auto` | int [2, 10000] \| `auto` | Reversals labeling frequency. `auto` = max(2, 2 \* number of whitelisted pairs). | +| freqai.feature_parameters.label_weights | [1/7,1/7,1/7,1/7,1/7,1/7,1/7] | list[float] | Per-objective weights for trial selection methods. Objectives: (1) number of detected reversals, (2) median swing amplitude, (3) median (swing amplitude / median volatility-threshold ratio), (4) median swing volume per candle, (5) median swing speed, (6) median swing efficiency ratio, (7) median swing volume-weighted efficiency ratio. | +| freqai.feature_parameters.label_p_order | None | float \| None | Lp exponent for parameterized metrics. Used by `minkowski` distance (default 2.0) and `power_mean` aggregation (default 1.0). Ignored by other metrics. | +| freqai.feature_parameters.label_method | `compromise_programming` | enum {`compromise_programming`,`topsis`,`kmeans`,`kmeans2`,`kmedoids`,`knn`,`medoid`} | HPO `label` Pareto front trial selection method. | +| freqai.feature_parameters.label_distance_metric | `euclidean` | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`,`harmonic_mean`,`geometric_mean`,`arithmetic_mean`,`quadratic_mean`,`cubic_mean`,`power_mean`,`weighted_sum`} | Distance metric for `compromise_programming` and `topsis` methods. Invalid values warn and fall back to `euclidean`. | +| freqai.feature_parameters.label_cluster_metric | `euclidean` | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`} | Distance metric for `kmeans`, `kmeans2`, and `kmedoids` methods. Invalid values warn and fall back to `euclidean`. | +| freqai.feature_parameters.label_cluster_selection_method | `topsis` | enum {`compromise_programming`,`topsis`} | Cluster selection method for clustering-based label methods. | +| freqai.feature_parameters.label_cluster_trial_selection_method | `topsis` | enum {`compromise_programming`,`topsis`} | Best cluster trial selection method for clustering-based label methods. | +| freqai.feature_parameters.label_density_metric | method-dependent | enum {`euclidean`,`minkowski`,`chebyshev`,`cityblock`,`sqeuclidean`,`seuclidean`,`mahalanobis`} | Distance metric for `knn` and `medoid` methods. Invalid values warn and fall back to the method's natural default (`minkowski` for `knn`, `euclidean` for `medoid`). | +| freqai.feature_parameters.label_density_aggregation | `power_mean` | enum {`power_mean`,`quantile`,`min`,`max`} | Aggregation method for KNN neighbor distances. | +| freqai.feature_parameters.label_density_n_neighbors | 5 | int >= 1 | Number of neighbors for KNN. | +| freqai.feature_parameters.label_density_aggregation_param | aggregation-dependent | float \| None | Tunable for KNN neighbor distance aggregation: Lp exponent (`power_mean`) or quantile value (`quantile`). | +| freqai.feature_parameters.scaler | `minmax` | enum {`minmax`,`maxabs`,`standard`,`robust`} | Feature scaling method. `minmax`=MinMaxScaler, `maxabs`=MaxAbsScaler, `standard`=StandardScaler, `robust`=RobustScaler. Changing this parameter requires deleting trained models. | +| freqai.feature_parameters.range | [-1.0, 1.0] | list[float], low < high | Target range for `minmax` scaler, min and max. Changing this parameter requires deleting trained models. | +| _Label prediction_ | | | | +| freqai.label_prediction.method | `thresholding` | enum {`none`,`thresholding`} | Prediction method. `none` disables threshold computation, `thresholding` enables adaptive threshold calculation. | +| freqai.label_prediction.selection_method | `rank_extrema` | enum {`rank_extrema`,`rank_peaks`,`partition`} | Extrema selection method. `rank_extrema` ranks extrema values, `rank_peaks` ranks detected peak values, `partition` uses sign-based partitioning. | +| freqai.label_prediction.threshold_method | `mean` | enum {`mean`,`isodata`,`li`,`minimum`,`otsu`,`triangle`,`yen`,`median`,`soft_extremum`} | Thresholding method for prediction thresholds. | +| freqai.label_prediction.soft_extremum_alpha | 12.0 | float >= 0 | Alpha for `soft_extremum` threshold method. | +| freqai.label_prediction.outlier_quantile | 0.999 | float (0,1) | Quantile threshold for predictions outlier filtering. | +| freqai.label_prediction.keep_fraction | 0.0075 | float (0,1] | Fraction of extrema used for thresholds. 1 uses all, lower values keep only most significant. Applies to `rank_extrema` and `rank_peaks`; ignored for `partition`. | +| _Optuna / HPO_ | | | | +| freqai.optuna_hyperopt.enabled | false | bool | Enables regressor and dynamic label HPO. | +| freqai.optuna_hyperopt.sampler | `tpe` | enum {`tpe`,`auto`} | HPO sampler algorithm for `hp` namespace. `tpe` uses [TPESampler](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html) with multivariate, group, and constant_liar (when multiple workers), `auto` uses [AutoSampler](https://hub.optuna.org/samplers/auto_sampler). | +| freqai.optuna_hyperopt.label_sampler | `auto` | enum {`auto`,`tpe`,`nsgaii`,`nsgaiii`} | HPO sampler algorithm for multi-objective `label` namespace. `nsgaii` uses [NSGAIISampler](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIISampler.html), `nsgaiii` uses [NSGAIIISampler](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.NSGAIIISampler.html). | +| freqai.optuna_hyperopt.storage | `file` | enum {`file`,`sqlite`} | HPO storage backend. | +| freqai.optuna_hyperopt.continuous | true | bool | Continuous HPO. Forced for both namespaces in backtest and hyperopt, resetting the study on each optimization. | +| freqai.optuna_hyperopt.warm_start | true | bool | Warm start HPO with previous best value(s). Persisted values are loaded and saved only in live and dry-run modes; non-live runs reuse only values produced earlier in the same run. | +| freqai.optuna_hyperopt.n_startup_trials | 15 | int >= 0 | HPO startup trials. | +| freqai.optuna_hyperopt.n_trials | 50 | int >= 1 | Maximum HPO trials. | +| freqai.optuna_hyperopt.n_jobs | 1 | int >= 1 | Parallel HPO workers. The effective value is capped at `max(1, CPU threads // 4)`; that cap is not the default. | +| freqai.optuna_hyperopt.timeout | 7200 | int >= 0 | HPO wall-clock timeout in seconds. | +| freqai.optuna_hyperopt.label_candles_step | 1 | int >= 1 | Step for Zigzag NATR period `label` search space. | +| freqai.optuna_hyperopt.space_reduction | false | bool | Enable/disable `hp` search space reduction based on previous best parameters. | +| freqai.optuna_hyperopt.space_fraction | 0.4 | float [0,1] | Fraction of the `hp` search space to use with `space_reduction`. Lower values create narrower search ranges around the best parameters. | +| freqai.optuna_hyperopt.min_resource | 3 | int >= 1 | Minimum resource per [HyperbandPruner](https://optuna.readthedocs.io/en/stable/reference/generated/optuna.pruners.HyperbandPruner.html) rung. | +| freqai.optuna_hyperopt.seed | 1 | int >= 0 | HPO RNG seed used by the Optuna samplers and label-candle shuffling. | +| freqai.optuna_hyperopt.reset_label_study_on_schema_mismatch | true | bool | Reset a persisted `label` study when its selection schema is missing, invalid, or incompatible. `true` keeps the historical destructive reset, deleting the study before recreating it; `false` preserves its trials and stored metadata, permits caller-managed reuse in memory, and does not persist selected params until the schema is reconciled. Both fail closed: an inspection error, or (under `true`) a deletion error, aborts study creation. Has no effect when `continuous=true` or outside live/dry-run modes, where studies are always reset. | +| freqai.optuna_hyperopt.vary_model_seed_by_trial | true | bool | Add `trial.number` to each regressor's configured model seed (or its default seed of `1`) during HPO. `true` samples model randomness across trials and preserves the historical behavior; `false` evaluates every trial and the final fit with the same model seed. This does not change `freqai.optuna_hyperopt.seed`. | ## ReforceXY diff --git a/quickadapter/user_data/strategies/QuickAdapterV3.py b/quickadapter/user_data/strategies/QuickAdapterV3.py index 2b2c30d..7078f3a 100644 --- a/quickadapter/user_data/strategies/QuickAdapterV3.py +++ b/quickadapter/user_data/strategies/QuickAdapterV3.py @@ -55,6 +55,7 @@ from Utils import ( bottom_log_return, calculate_quantile, compose_label_lookahead, + compute_label_weight_imputation_dependency_mask, compute_label_weight_known_at_lookahead, compute_label_weights, ensure_datetime_series, @@ -996,6 +997,18 @@ class QuickAdapterV3(IStrategy): ), ) if label_data.known_at_lookahead is not None: + if causal_mode: + ( + imputation_dependency_mask, + imputation_leading_stable_mask, + ) = compute_label_weight_imputation_dependency_mask( + len(label_data.indices), + label_data.metrics, + col_weighting_config, + ) + else: + imputation_dependency_mask = None + imputation_leading_stable_mask = None dataframe[ label_weight_known_at_lookahead_column_name(label_col) ] = compute_label_weight_known_at_lookahead( @@ -1003,6 +1016,8 @@ class QuickAdapterV3(IStrategy): indices=label_data.indices, fill_radius=weight_fill_radius(col_weighting_config), weighting_config=col_weighting_config, + imputation_dependency_mask=imputation_dependency_mask, + imputation_leading_stable_mask=imputation_leading_stable_mask, ) if label_col == EXTREMA_COLUMN: diff --git a/quickadapter/user_data/strategies/Utils.py b/quickadapter/user_data/strategies/Utils.py index 29e9388..3f4e4ac 100644 --- a/quickadapter/user_data/strategies/Utils.py +++ b/quickadapter/user_data/strategies/Utils.py @@ -2566,43 +2566,162 @@ def _aggregate_metrics( ) -def _compute_combined_label_weights( +def _invalid_weight_strategy_message( + strategy: str, metrics: dict[str, list[float]] +) -> str: + return ( + f"Invalid weighting strategy value {strategy!r}: " + f"supported values are {', '.join(WEIGHT_STRATEGIES)} or metric names {', '.join(metrics.keys())}" + ) + + +def _select_combined_metrics( metrics: dict[str, list[float]], metric_coefficients: dict[str, Any], - aggregation: CombinedAggregation, - softmax_temperature: float, - *, - impute: Callable[[NDArray[np.floating]], NDArray[np.floating]] = _impute_weights, -) -> NDArray[np.floating]: - if len(metrics) == 0: - return np.asarray([], dtype=float) +) -> list[tuple[str, NDArray[np.floating], float]]: + """Select the components feeding ``combined`` aggregation. + Shared selection logic (coefficient parsing, the all-unit default, the skip + rules for unselected or empty metrics), returning raw pre-imputation value + arrays paired with metric name and coefficient in ``metrics`` iteration + order. Imputation is left to the caller. + """ coefficients = _parse_metric_coefficients(metric_coefficients) if len(coefficients) == 0: coefficients = {k: 1.0 for k in metrics.keys()} - imputed_metrics: list[NDArray[np.floating]] = [] - coefficients_list: list[float] = [] - + selected: list[tuple[str, NDArray[np.floating], float]] = [] for metric_name, metric_values in metrics.items(): if metric_name not in coefficients: continue - coefficient = coefficients[metric_name] values_array = np.asarray(metric_values, dtype=float) if values_array.size == 0: continue - imputed_metrics.append(impute(values_array)) - coefficients_list.append(float(coefficient)) + selected.append((metric_name, values_array, float(coefficients[metric_name]))) + return selected - if len(imputed_metrics) == 0: + +def _aggregate_imputed_metrics( + imputed_metrics: list[NDArray[np.floating]], + coefficients: list[float], + aggregation: CombinedAggregation, + softmax_temperature: float, +) -> NDArray[np.floating]: + return _aggregate_metrics( + np.vstack(imputed_metrics), + np.asarray(coefficients, dtype=float), + aggregation, + softmax_temperature, + ) + + +def _compute_combined_label_weights( + metrics: dict[str, list[float]], + metric_coefficients: dict[str, Any], + aggregation: CombinedAggregation, + softmax_temperature: float, + *, + impute: Callable[[NDArray[np.floating]], NDArray[np.floating]] = _impute_weights, +) -> NDArray[np.floating]: + selected = _select_combined_metrics(metrics, metric_coefficients) + if len(selected) == 0: return np.asarray([], dtype=float) - stacked_metrics = np.vstack(imputed_metrics) - coefficients_array = np.asarray(coefficients_list, dtype=float) + return _aggregate_imputed_metrics( + [impute(values) for _, values, _ in selected], + [coefficient for _, _, coefficient in selected], + aggregation, + softmax_temperature, + ) - return _aggregate_metrics( - stacked_metrics, coefficients_array, aggregation, softmax_temperature + +def _nonfinite_imputation_dependency_mask( + values: NDArray[np.floating], +) -> NDArray[np.bool_]: + """Mark values whose legacy full-frame imputation is not prefix-stable. + + A non-finite value's imputation is prefix-unstable (default or zero at + boundaries, interior median otherwise) as later pivots arrive, so it stays + unavailable until the frame boundary. + """ + return ~np.isfinite(values) + + +def compute_label_weight_imputation_dependency_mask( + n_indices: int, + metrics: dict[str, list[float]], + weighting_config: dict[str, Any], +) -> tuple[NDArray[np.bool_], NDArray[np.bool_]]: + """Identify pivot weights whose non-finite imputation can change by prefix. + + Returns ``(dependency_mask, leading_stable_mask)``. A ``dependency_mask`` + pivot remains causally unavailable until the frame boundary; for + ``combined``, dependency propagates from every selected component and from + the aggregate before its final imputation. ``leading_stable_mask`` (a subset + of ``dependency_mask``) marks the leading non-finite run of a single-metric + strategy: those pivots impute to ``0.0`` and stabilize once the first finite + pivot's weight is known, so they need not defer to the frame boundary. It is + empty for ``uniform``, ``combined``, and all-non-finite metrics. + """ + label_weighting = {**DEFAULTS_LABEL_WEIGHTING, **weighting_config} + strategy = label_weighting["strategy"] + if strategy == WEIGHT_STRATEGIES[0]: # "none" + raise ValueError( + "compute_label_weight_imputation_dependency_mask must not be called " + f"with strategy={strategy!r}; callers must skip invocation when " + "weighting is disabled" + ) + if strategy == WEIGHT_STRATEGIES[1]: # "uniform" + return np.zeros(n_indices, dtype=bool), np.zeros(n_indices, dtype=bool) + if strategy in metrics: + values = np.asarray(metrics[strategy], dtype=float) + if values.size == 0: + return np.zeros(n_indices, dtype=bool), np.zeros(n_indices, dtype=bool) + if values.shape != (n_indices,): + raise ValueError( + f"Invalid metric {strategy!r} shape {values.shape}: " + f"must be ({n_indices},)" + ) + dependency = _nonfinite_imputation_dependency_mask(values) + leading_stable = np.zeros(n_indices, dtype=bool) + finite = ~dependency + if finite.any(): + leading_stable[: int(np.argmax(finite))] = True + return dependency, leading_stable + if strategy != WEIGHT_STRATEGIES[8]: # "combined" + raise ValueError(_invalid_weight_strategy_message(strategy, metrics)) + + dependency_mask = np.zeros(n_indices, dtype=bool) + imputed_metrics: list[NDArray[np.floating]] = [] + coefficients_list: list[float] = [] + for metric_name, values_array, coefficient in _select_combined_metrics( + metrics, label_weighting["metric_coefficients"] + ): + if values_array.shape != (n_indices,): + raise ValueError( + f"Invalid metric {metric_name!r} shape {values_array.shape}: " + f"must be ({n_indices},)" + ) + dependency_mask |= _nonfinite_imputation_dependency_mask(values_array) + imputed_metrics.append(_impute_weights(values_array)) + coefficients_list.append(coefficient) + + if len(imputed_metrics) == 0: + return dependency_mask, np.zeros(n_indices, dtype=bool) + + combined_weights = _aggregate_imputed_metrics( + imputed_metrics, + coefficients_list, + label_weighting["aggregation"], + label_weighting["softmax_temperature"], ) + if combined_weights.shape != (n_indices,): + raise ValueError( + f"Invalid combined weights shape {combined_weights.shape}: " + f"must be ({n_indices},)" + ) + dependency_mask |= _nonfinite_imputation_dependency_mask(combined_weights) + return dependency_mask, np.zeros(n_indices, dtype=bool) def _compute_epsilon_floor( @@ -2655,10 +2774,7 @@ def _compute_label_weight_values( impute=impute, ) else: - raise ValueError( - f"Invalid weighting strategy value {strategy!r}: " - f"supported values are {', '.join(WEIGHT_STRATEGIES)} or metric names {', '.join(metrics.keys())}" - ) + raise ValueError(_invalid_weight_strategy_message(strategy, metrics)) return impute(weights) @@ -3119,6 +3235,8 @@ def compute_label_weight_known_at_lookahead( indices: Sequence[int] | NDArray[np.integer], fill_radius: int = 0, *, + imputation_dependency_mask: Sequence[bool] | NDArray[np.bool_] | None = None, + imputation_leading_stable_mask: Sequence[bool] | NDArray[np.bool_] | None = None, weighting_config: dict[str, Any] | None = None, ) -> pd.Series: """Per-row causal availability (in candles) of the label WEIGHT column. @@ -3145,6 +3263,15 @@ def compute_label_weight_known_at_lookahead( off-center bands still wait for the pivot confirmation and sigma. Other strategies and additive fills keep their existing competing-band dependencies. + + ``imputation_dependency_mask`` marks pivot weights whose non-finite + imputation can change as the available prefix grows. Those pivots and their + Gaussian bands are unavailable until the frame boundary. An unresolved + trailing pivot is excluded only when it has no such dependency. + ``imputation_leading_stable_mask`` (a subset) marks a leading non-finite run + that imputes to 0.0 and stabilizes at the first finite pivot's confirmation; + those pivots are released there instead of at the frame boundary and their + zero-weight bands are skipped. """ n = len(known_at_lookahead) positions, known_at_lookahead_values = _sanitize_known_at_lookahead( @@ -3153,8 +3280,34 @@ def compute_label_weight_known_at_lookahead( if n == 0: return pd.Series(positions, index=known_at_lookahead.index, dtype=np.int64) known_at_positions = positions + known_at_lookahead_values - idx = np.asarray(indices, dtype=int) - idx = np.sort(idx[(idx >= 0) & (idx < n)]) + raw_idx = np.asarray(indices, dtype=int) + + def _validate_pivot_mask( + mask: Sequence[bool] | NDArray[np.bool_] | None, name: str + ) -> NDArray[np.bool_]: + if mask is None: + return np.zeros(raw_idx.size, dtype=bool) + arr = np.asarray(mask) + if arr.shape != raw_idx.shape: + raise ValueError( + f"Invalid {name} shape {arr.shape}: must be {raw_idx.shape}" + ) + if arr.dtype != np.bool_: + raise ValueError(f"Invalid {name} dtype {arr.dtype}: must be bool") + return arr + + raw_dependency_mask = _validate_pivot_mask( + imputation_dependency_mask, "imputation_dependency_mask" + ) + raw_leading_stable_mask = _validate_pivot_mask( + imputation_leading_stable_mask, "imputation_leading_stable_mask" + ) + valid_mask = (raw_idx >= 0) & (raw_idx < n) + idx = raw_idx[valid_mask] + order = np.argsort(idx, kind="stable") + idx = idx[order] + dependency_mask = raw_dependency_mask[valid_mask][order] + leading_stable_mask = raw_leading_stable_mask[valid_mask][order] base = known_at_positions.copy() if idx.size: weight_availability = np.empty(idx.size, dtype=np.int64) @@ -3189,17 +3342,41 @@ def compute_label_weight_known_at_lookahead( n, ) np.maximum(avail_pivot, sigma_availability, out=avail_pivot) + avail_pivot[dependency_mask] = n + if leading_stable_mask.any() and np.array_equal(order, np.arange(idx.size)): + # Leading non-finite run imputes to 0.0, stable once the first finite + # pivot's weight is known (backfilled confirmation weight_availability + # [first_finite]), not at the frame boundary. Guarded to the sorted + # (identity-order) case where the run is a contiguous prefix. + first_finite = int(leading_stable_mask.sum()) + if first_finite < weight_availability.size: + avail_pivot[leading_stable_mask] = int( + weight_availability[first_finite] + ) base[idx] = np.maximum(base[idx], avail_pivot) if fill_radius > 0: - for pivot_pos, pivot_avail, weight_avail in zip( + for ( + pivot_pos, + pivot_avail, + weight_avail, + pivot_dependency, + pivot_leading, + ) in zip( idx.tolist(), avail_pivot.tolist(), band_weight_availability.tolist(), + dependency_mask.tolist(), + leading_stable_mask.tolist(), ): - # A metric-based trailing pivot never resolves in-frame and its - # Gaussian bump is 0. Pure-Gaussian uniform pivots use their own - # confirmation, so only that path retains the trailing band. - if weight_avail >= n: + # A leading-run pivot imputes to 0.0 (zero bump): its own row is + # released above; it spreads no band. + if pivot_leading: + continue + # Skip pivots whose band weight never resolves in-frame + # (weight_avail == n): their Gaussian bump is zero. Exception: an + # imputation-dependent pivot keeps the non-zero legacy default for + # an all-non-finite metric, so its band must defer to n. + if weight_avail >= n and not pivot_dependency: continue lo = max(0, pivot_pos - fill_radius) hi = min(n, pivot_pos + fill_radius + 1) -- 2.53.0