From: Jérôme Benoit Date: Thu, 8 Jan 2026 11:13:16 +0000 (+0100) Subject: feat(quickadapter): add CatBoost regressor with RMSE loss function (#34) X-Git-Url: https://git.piment-noir.org/?a=commitdiff_plain;h=ad61c24d488bbf9e9869b270949ca9e31a3bdca8;p=freqai-strategies.git feat(quickadapter): add CatBoost regressor with RMSE loss function (#34) * feat: add CatBoost regressor with RMSE loss function Add CatBoost as the 5th regressor option using standard RMSE loss function. Changes: - Add catboost==1.2.8 to Dockerfile dependencies - Update Regressor type literal to include 'catboost' - Implement fit_regressor branch for CatBoost with: - RMSE loss function (default) - Early stopping and validation set handling - Verbosity parameter mapping - Sample weights support - Optuna CatBoostPruningCallback for trial pruning - Add Optuna hyperparameter optimization with 6 parameters: - iterations: [100, 2000] (log-scaled) - learning_rate: [0.001, 0.3] (log-scaled) - depth: [4, 10] (tree depth) - l2_leaf_reg: [1, 10] (L2 regularization) - bagging_temperature: [0, 10] (Bayesian bootstrap) - random_strength: [1, 20] (split randomness) - Update README.md regressor enum documentation CatBoost advantages: - Better accuracy than XGBoost/LightGBM (2024 benchmarks) - GPU support for faster training - Better categorical feature handling - Strong overfitting resistance (ordered boosting) - Production-ready at scale - Optuna pruning callback for efficient hyperparameter search * feat(catboost): add GPU/CPU differentiation for training and Optuna hyperparameters - Add task_type-aware parameter handling in fit_regressor - GPU mode: set devices, max_ctr_complexity=4, remove n_jobs - CPU mode: propagate n_jobs to thread_count, max_ctr_complexity=2 - Trust CatBoost defaults for border_count (CPU=254, GPU=128) - Differentiate Optuna hyperparameter search spaces by task_type - GPU: depth=(4,12), border_count=(32,254), bootstrap=[Bayesian,Bernoulli] - CPU: depth=(4,10), bootstrap=[Bayesian,Bernoulli,MVS] - Add GPU-specific parameters: border_count, max_ctr_complexity - Expand search space: min_data_in_leaf, grow_policy, model_size_reg, rsm, subsample - Use CatBoost Pool for training with proper eval_set handling * refactor(catboost): remove devices default to allow GPU auto-discovery Trust CatBoost's automatic GPU device detection (default: all available GPUs). Users can still explicitly set devices='0' or devices='0:1' in config if needed. * fix(quickadapter): avoid forcing CatBoost thread_count when n_jobs unset Remove nonstandard thread_count=-1 default in CPU CatBoost path.\nLet CatBoost select threads automatically unless n_jobs is provided.\nImproves consistency and avoids potential performance misinterpretation. * Apply suggestion from @Copilot Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix(catboost): properly omit incompatible bootstrap parameters CatBoost strictly validates bootstrap parameters and rejects: - subsample with Bayesian bootstrap - bagging_temperature with non-Bayesian bootstrap Even passing 'neutral' values (0 or 1.0) causes runtime errors. Changed from ternary expressions (which always pass params) to conditional dict building (which omits incompatible params entirely). Also fixed: border_count min_val 16→1 per CatBoost documentation. * refactor(quickadapter): replace plotting column names with constants Replace string literals "minima", "maxima", and "smoothed-extrema" with MINIMA_COLUMN, MAXIMA_COLUMN, and SMOOTHED_EXTREMA_COLUMN constants following the existing *_COLUMN naming convention. This improves maintainability and prevents typos when referencing these DataFrame column names throughout the codebase. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- diff --git a/README.md b/README.md index 575f20d..5380209 100644 --- a/README.md +++ b/README.md @@ -58,7 +58,7 @@ docker compose up -d --build | reversal_confirmation.min_natr_multiplier_fraction | 0.0095 | float [0,1] | Lower bound fraction for volatility adjusted reversal threshold. (Deprecated alias: `reversal_confirmation.min_natr_ratio_percent`) | | reversal_confirmation.max_natr_multiplier_fraction | 0.075 | float [0,1] | Upper bound fraction (>= lower bound) for volatility adjusted reversal threshold. (Deprecated alias: `reversal_confirmation.max_natr_ratio_percent`) | | _Regressor model_ | | | | -| freqai.regressor | `xgboost` | enum {`xgboost`,`lightgbm`,`histgradientboostingregressor`} | Machine learning regressor algorithm. | +| freqai.regressor | `xgboost` | enum {`xgboost`,`lightgbm`,`histgradientboostingregressor`,`catboost`} | Machine learning regressor algorithm. | | _Extrema smoothing_ | | | | | freqai.extrema_smoothing.method | `gaussian` | enum {`gaussian`,`kaiser`,`triang`,`smm`,`sma`,`savgol`,`gaussian_filter1d`} | Extrema smoothing method (`smm`=median, `sma`=mean, `savgol`=Savitzky–Golay). | | freqai.extrema_smoothing.window_candles | 5 | int >= 3 | Smoothing window length (candles). (Deprecated alias: `freqai.extrema_smoothing.window`) | diff --git a/quickadapter/Dockerfile b/quickadapter/Dockerfile index f5f3db2..413f4cc 100644 --- a/quickadapter/Dockerfile +++ b/quickadapter/Dockerfile @@ -3,6 +3,7 @@ FROM freqtradeorg/freqtrade:stable_freqai ARG optuna_version=4.6.0 ARG scikit_learn_extra_version=0.3.0 ARG scikit_image_version=0.26.0 +ARG catboost_version=1.2.8 USER root RUN apt-get update \ @@ -16,7 +17,8 @@ RUN pip install --user --no-cache-dir \ optunahub \ -r https://hub.optuna.org/samplers/auto_sampler/requirements.txt \ scikit-learn-extra==${scikit_learn_extra_version} \ - scikit-image==${scikit_image_version} + scikit-image==${scikit_image_version} \ + catboost==${catboost_version} LABEL org.opencontainers.image.source="freqai-strategies" \ org.opencontainers.image.title="freqtrade-quickadapter" \ diff --git a/quickadapter/user_data/freqaimodels/QuickAdapterRegressorV3.py b/quickadapter/user_data/freqaimodels/QuickAdapterRegressorV3.py index 0774932..a8b49b5 100644 --- a/quickadapter/user_data/freqaimodels/QuickAdapterRegressorV3.py +++ b/quickadapter/user_data/freqaimodels/QuickAdapterRegressorV3.py @@ -3275,6 +3275,7 @@ def hp_objective( trial, regressor, model_training_best_parameters, + model_training_parameters, space_reduction, space_fraction, ) diff --git a/quickadapter/user_data/strategies/QuickAdapterV3.py b/quickadapter/user_data/strategies/QuickAdapterV3.py index 23db9bb..7f47761 100644 --- a/quickadapter/user_data/strategies/QuickAdapterV3.py +++ b/quickadapter/user_data/strategies/QuickAdapterV3.py @@ -26,12 +26,16 @@ from pandas import DataFrame, Series, isna from scipy.stats import pearsonr, t from technical.pivots_points import pivots_points +from ExtremaWeightingTransformer import COMBINED_AGGREGATIONS from Utils import ( DEFAULT_FIT_LIVE_PREDICTIONS_CANDLES, DEFAULTS_EXTREMA_SMOOTHING, EXTREMA_COLUMN, + MAXIMA_COLUMN, MAXIMA_THRESHOLD_COLUMN, + MINIMA_COLUMN, MINIMA_THRESHOLD_COLUMN, + SMOOTHED_EXTREMA_COLUMN, SMOOTHING_METHODS, SMOOTHING_MODES, TRADE_PRICE_TARGETS, @@ -57,7 +61,6 @@ from Utils import ( zigzag, zlema, ) -from ExtremaWeightingTransformer import COMBINED_AGGREGATIONS TradeDirection = Literal["long", "short"] InterpolationDirection = Literal["direct", "inverse"] @@ -194,9 +197,9 @@ class QuickAdapterV3(IStrategy): EXTREMA_COLUMN: {"color": "orange", "type": "line"}, }, "min_max": { - "smoothed-extrema": {"color": "wheat", "type": "line"}, - "maxima": {"color": "red", "type": "bar"}, - "minima": {"color": "green", "type": "bar"}, + SMOOTHED_EXTREMA_COLUMN: {"color": "wheat", "type": "line"}, + MAXIMA_COLUMN: {"color": "red", "type": "bar"}, + MINIMA_COLUMN: {"color": "green", "type": "bar"}, }, }, } @@ -897,8 +900,8 @@ class QuickAdapterV3(IStrategy): minutes=len(dataframe) * self.get_timeframe_minutes() ) dataframe[EXTREMA_COLUMN] = 0.0 - dataframe["minima"] = 0.0 - dataframe["maxima"] = 0.0 + dataframe[MINIMA_COLUMN] = 0.0 + dataframe[MAXIMA_COLUMN] = 0.0 if len(pivots_indices) == 0: logger.warning( @@ -928,12 +931,12 @@ class QuickAdapterV3(IStrategy): if not np.isfinite(plot_eps): plot_eps = 0.0 plot_eps = max(float(plot_eps) * 0.5, QuickAdapterV3._PLOT_EXTREMA_MIN_EPS) - dataframe["maxima"] = ( + dataframe[MAXIMA_COLUMN] = ( weighted_extrema.where(extrema_direction.gt(0), 0.0) .clip(lower=0.0) .mask(extrema_direction.gt(0) & weighted_extrema.eq(0.0), plot_eps) ) - dataframe["minima"] = ( + dataframe[MINIMA_COLUMN] = ( weighted_extrema.where(extrema_direction.lt(0), 0.0) .clip(upper=0.0) .mask(extrema_direction.lt(0) & weighted_extrema.eq(0.0), -plot_eps) @@ -950,7 +953,7 @@ class QuickAdapterV3(IStrategy): ) dataframe[EXTREMA_COLUMN] = smoothed_extrema - dataframe["smoothed-extrema"] = smoothed_extrema + dataframe[SMOOTHED_EXTREMA_COLUMN] = smoothed_extrema return dataframe diff --git a/quickadapter/user_data/strategies/Utils.py b/quickadapter/user_data/strategies/Utils.py index 04dec08..bd74786 100644 --- a/quickadapter/user_data/strategies/Utils.py +++ b/quickadapter/user_data/strategies/Utils.py @@ -48,6 +48,10 @@ EXTREMA_COLUMN: Final = "&s-extrema" MAXIMA_THRESHOLD_COLUMN: Final = "&s-maxima_threshold" MINIMA_THRESHOLD_COLUMN: Final = "&s-minima_threshold" +MAXIMA_COLUMN: Final = "maxima" +MINIMA_COLUMN: Final = "minima" +SMOOTHED_EXTREMA_COLUMN: Final = "smoothed_extrema" + SmoothingKernel = Literal["gaussian", "kaiser", "triang"] SMOOTHING_KERNELS: Final[tuple[SmoothingKernel, ...]] = ( "gaussian", @@ -1608,11 +1612,12 @@ def zigzag( ) -Regressor = Literal["xgboost", "lightgbm", "histgradientboostingregressor"] +Regressor = Literal["xgboost", "lightgbm", "histgradientboostingregressor", "catboost"] REGRESSORS: Final[tuple[Regressor, ...]] = ( "xgboost", "lightgbm", "histgradientboostingregressor", + "catboost", ) RegressorCallback = Union[Callable[..., Any], XGBoostTrainingCallback] @@ -1785,6 +1790,66 @@ def fit_regressor( y_val=y_val, sample_weight_val=sample_weight_val, ) + elif regressor == REGRESSORS[3]: # "catboost" + from catboost import CatBoostRegressor, Pool + + model_training_parameters.setdefault("random_seed", 1) + model_training_parameters.setdefault("loss_function", "RMSE") + + task_type = model_training_parameters.get("task_type", "CPU") + if task_type == "GPU": + model_training_parameters.setdefault("max_ctr_complexity", 4) + model_training_parameters.pop("n_jobs", None) + else: + n_jobs = model_training_parameters.pop("n_jobs", None) + if n_jobs is not None: + model_training_parameters.setdefault("thread_count", n_jobs) + model_training_parameters.setdefault("max_ctr_complexity", 2) + + early_stopping_rounds = None + if has_eval_set: + early_stopping_rounds = model_training_parameters.pop( + "early_stopping_rounds", _EARLY_STOPPING_ROUNDS_DEFAULT + ) + else: + model_training_parameters.pop("early_stopping_rounds", None) + + verbosity = model_training_parameters.pop("verbosity", None) + if "verbose" not in model_training_parameters and verbosity is not None: + model_training_parameters["verbose"] = verbosity + + if trial is not None: + model_training_parameters["random_seed"] = ( + model_training_parameters["random_seed"] + trial.number + ) + + pruning_callback = None + if trial is not None and has_eval_set: + pruning_callback = optuna.integration.CatBoostPruningCallback(trial, "RMSE") + fit_callbacks.append(pruning_callback) + + model = CatBoostRegressor(**model_training_parameters) + + model.fit( + Pool(data=X, label=y, weight=train_weights), + eval_set=Pool( + data=eval_set[0][0], + label=eval_set[0][1], + weight=eval_weights[0] if eval_weights else None, + ) + if has_eval_set + else None, + early_stopping_rounds=early_stopping_rounds + if early_stopping_rounds is not None and has_eval_set + else None, + use_best_model=True + if early_stopping_rounds is not None and has_eval_set + else False, + callbacks=fit_callbacks if fit_callbacks else None, + ) + + if pruning_callback is not None: + pruning_callback.check_pruned() else: raise ValueError( f"Invalid regressor value {regressor!r}: supported values are {', '.join(REGRESSORS)}" @@ -1833,6 +1898,7 @@ def get_optuna_study_model_parameters( trial: optuna.trial.Trial, regressor: Regressor, model_training_best_parameters: dict[str, Any], + model_training_parameters: dict[str, Any], space_reduction: bool, space_fraction: float, ) -> dict[str, Any]: @@ -2189,6 +2255,128 @@ def get_optuna_study_model_parameters( ), } + elif regressor == REGRESSORS[3]: # "catboost" + # Parameter order: boosting -> tree structure -> regularization -> sampling + task_type = model_training_parameters.get("task_type", "CPU") + if task_type == "GPU": + default_ranges: dict[str, tuple[float, float]] = { + # Boosting/Training + "iterations": (100, 2000), + "learning_rate": (0.001, 0.3), + # Tree structure + "depth": (4, 12), + "min_data_in_leaf": (1, 20), + "border_count": (32, 254), + "max_ctr_complexity": (2, 6), + # Regularization + "l2_leaf_reg": (1, 10), + "model_size_reg": (0.0, 1.0), + # Sampling/Randomization + "bagging_temperature": (0, 10), + "random_strength": (1, 20), + "rsm": (0.5, 1.0), + "subsample": (0.6, 1.0), + } + bootstrap_options = ["Bayesian", "Bernoulli"] + else: # CPU + default_ranges: dict[str, tuple[float, float]] = { + # Boosting/Training + "iterations": (100, 2000), + "learning_rate": (0.001, 0.3), + # Tree structure + "depth": (4, 10), + "min_data_in_leaf": (1, 20), + # Regularization + "l2_leaf_reg": (1, 10), + "model_size_reg": (0.0, 1.0), + # Sampling/Randomization + "bagging_temperature": (0, 10), + "random_strength": (1, 20), + "rsm": (0.5, 1.0), + "subsample": (0.6, 1.0), + } + bootstrap_options = ["Bayesian", "Bernoulli", "MVS"] + + log_scaled_params = { + "iterations", + "learning_rate", + } + + ranges = _build_ranges(default_ranges, log_scaled_params) + + bootstrap_type = trial.suggest_categorical("bootstrap_type", bootstrap_options) + + params = { + # Boosting/Training + "iterations": _optuna_suggest_int_from_range( + trial, "iterations", ranges["iterations"], min_val=1, log=True + ), + "learning_rate": trial.suggest_float( + "learning_rate", + ranges["learning_rate"][0], + ranges["learning_rate"][1], + log=True, + ), + # Tree structure + "depth": _optuna_suggest_int_from_range( + trial, "depth", ranges["depth"], min_val=1 + ), + "min_data_in_leaf": _optuna_suggest_int_from_range( + trial, "min_data_in_leaf", ranges["min_data_in_leaf"], min_val=1 + ), + "grow_policy": trial.suggest_categorical( + "grow_policy", ["SymmetricTree", "Depthwise", "Lossguide"] + ), + # Regularization + "l2_leaf_reg": trial.suggest_float( + "l2_leaf_reg", ranges["l2_leaf_reg"][0], ranges["l2_leaf_reg"][1] + ), + "model_size_reg": trial.suggest_float( + "model_size_reg", + ranges["model_size_reg"][0], + ranges["model_size_reg"][1], + ), + # Sampling/Randomization + "bootstrap_type": bootstrap_type, + "random_strength": trial.suggest_float( + "random_strength", + ranges["random_strength"][0], + ranges["random_strength"][1], + ), + "rsm": trial.suggest_float( + "rsm", + ranges["rsm"][0], + ranges["rsm"][1], + ), + } + + if bootstrap_type == "Bayesian": + params["bagging_temperature"] = trial.suggest_float( + "bagging_temperature", + ranges["bagging_temperature"][0], + ranges["bagging_temperature"][1], + ) + + if bootstrap_type in ["Bernoulli", "MVS"]: + params["subsample"] = trial.suggest_float( + "subsample", + ranges["subsample"][0], + ranges["subsample"][1], + ) + + if task_type == "GPU": + params["border_count"] = _optuna_suggest_int_from_range( + trial, "border_count", ranges["border_count"], min_val=1 + ) + params["max_ctr_complexity"] = _optuna_suggest_int_from_range( + trial, + "max_ctr_complexity", + ranges["max_ctr_complexity"], + min_val=1, + ) + + return params + else: raise ValueError( f"Invalid regressor value {regressor!r}: supported values are {', '.join(REGRESSORS)}"