feat(weights): per-label sample weights propagated to model.fit(sample_weight=...) (#72)
* chore(quickadapter): bump strategy and regressor version 3.11.8 → 3.11.9
* feat(weights): add compose_sample_weights helper with mean=1 multiplicative composition
AFML §4.10 / mlfinpy canonical: per-label mean=1 normalization, multiplicative composition with temporal decay, geometric-mean aggregation for multi-label, NaN/inf handling, all-zero degenerate fallback. Validated locally with pytest (evidence: .omo/evidence/task-5-{red,green}.txt).
* fix(weights): persist label weights into <label>_weight column instead of rescaling target
Removes statistically incorrect target rescaling (label = direction × weight). Persists raw direction labels and a separate <label>_weight column for downstream sample_weight composition. Validated locally with pytest (evidence: .omo/evidence/task-6-{red,green}.txt).
* feat(weights): add _strip_label_weight_columns helper for find_labels collision avoidance
* feat(weights): compose per-label weights with temporal decay before model.fit
* feat(weights): integrate sample_weight composition into both train() data split paths
Add _train_default() mirroring BaseRegressionModel.train() with _compose_train_weights inserted between make_train_test_datasets and _apply_pipelines. Routes train_test_split path through _train_default instead of super().train(). Inserts _compose_train_weights before _apply_pipelines in timeseries_split path. Calls _strip_label_weight_columns(dk) at top of train() for both branches. Validated locally with pytest + structural AST checks (evidence: .omo/evidence/task-9-{pytest,structural}.txt).
* refactor(weights): align _train with BaseRegressionModel.train
Rename _train_default to _train to match the upstream method name (with
underscore prefix to mark it as the internal mirror, since the public
train() method routes between data split paths).
Mirror BaseRegressionModel.train line-for-line with _compose_train_weights
as the single intentional insertion between make_train_test_datasets and
the pipeline application:
- Drop ensure_datetime_series wrapper around unfiltered_df['date']:
upstream calls .iloc[].strftime() directly.
- Drop **kwargs from self.fit(dd, dk) to match upstream signature.
- Use dk.data_dictionary['train_features'].columns for feature count log,
matching upstream source of truth.
- Apply the same cosmetic alignment to the timeseries_split path for
consistency between both train code paths.
- Add docstring documenting the mirror relationship and the single
functional difference.
* refactor(weights): drop dead apply_label_weighting wrapper
Now that QuickAdapterV3.set_freqai_targets persists raw label direction
into the label column and weights into <label>_weight (consumed by
sample_weight downstream), the apply_label_weighting wrapper that
multiplied label values by their weights is no longer used.
- Drop Utils.apply_label_weighting (returned (weighted_label, weights)).
- Drop Utils._apply_label_weights (the values × weights helper).
- Switch QuickAdapterV3.set_freqai_targets to call compute_label_weights
directly (already used internally by the removed wrapper).
* refactor(weights): simplify MAXIMA/MINIMA plot columns for binary direction
Now that the label column holds raw direction in {-1, 0, +1}, the MAXIMA
and MINIMA plot columns reduce to a where(direction>0, 0.0) /
where(direction<0, 0.0) projection.
The previous magnitude-aware logic (plot_eps padding, mask of zero values
on positive direction, etc.) was tailored to the weighted label
amplitudes and is now dead code:
- extrema.abs().where(extrema.ne(0.0)).min() always evaluates to 1.0,
so plot_eps is always max(0.5, _PLOT_EXTREMA_MIN_EPS) = 0.5.
- direction.gt(0) & extrema.eq(0.0) is always False because direction>0
implies extrema = +1, never 0. The .mask() branches never trigger.
Drop the now-unused _PLOT_EXTREMA_MIN_EPS class constant.
* refactor(weights): rename smooth_label to smooth for genericity
The function applies generic smoothing kernels (gaussian, kaiser, triang,
smm, sma, savgol, gaussian_filter1d) to any pd.Series. The 'label'
suffix narrowed it to label-specific use, but it now also smooths the
<label>_weight column (next commit). Drop the suffix; the smoothing
config dict is still named label_smoothing because the per-column
config map remains label-keyed.
* feat(weights): smooth <label>_weight column with the same kernel as label
Per-label weights are pointwise: only pivot indices carry the
metric-derived weight, while non-pivot indices are filled with the
median weight (see compute_label_weights / _build_weights_array).
The label column is smoothed with smooth() to spread pivot signals
over neighbouring candles. Without smoothing the weight column, the
smoothed label values around a pivot keep the constant median weight,
so the model treats high-amplitude pivot neighbours and the pivot
itself as equally important during training.
Apply smooth() to the <label>_weight column with the exact same
per-column smoothing config as the label, so the weight profile
follows the label profile candle-for-candle.
The smooth() positional argument list was redundant with the
col_smoothing_config dict keys; collapse both calls to **kwargs
unpacking. _SMOOTHING_SPECS keys exactly match smooth() parameter
names, so the unpack is type-safe.
compose_sample_weights already replaces non-finite or non-positive
values with 1.0, which absorbs any sign overshoot from kernels like
savgol at series edges.
* feat(weights): add direction and weight subplots showing raw + smoothed signals
Replace the MAXIMA/MINIMA bar visualization with two new subplots that
show both the raw and the smoothed direction/weight curves:
- Subplot 'direction' overlays raw direction (extrema_direction) and
smoothed direction (smoothed_extrema).
- Subplot 'weight' overlays raw weight (extrema_weight) and smoothed
weight (smoothed_extrema_weight).
Each visualization column is captured at the right point in
set_freqai_targets:
- Raw columns (EXTREMA_DIRECTION_COLUMN, EXTREMA_WEIGHT_COLUMN) are
written before the smooth() call.
- Smoothed columns (SMOOTHED_EXTREMA_COLUMN, SMOOTHED_EXTREMA_WEIGHT_COLUMN)
are written after the smooth() call.
Drop MAXIMA_COLUMN and MINIMA_COLUMN constants — they were only used by
the old min_max bar subplot. The new subplots convey the same direction
information plus the per-pivot weight magnitude that the legacy
weighted_label visualization showed (before the sample-weight refactor).
All four visualization column names lack the '&' prefix, so FreqAI's
find_labels auto-detection ignores them; they cannot leak into model
targets.
* refactor(weights): align visualization column names on extrema_<axis>[_smoothed]
Three-agent audit (explore + librarian + oracle) found the previous viz
column names suffered from three inconsistencies:
- 'extrema_direction' (raw) carries the axis word, but 'smoothed_extrema'
drops it, breaking the 2x2 grid (axis × stage).
- Stage qualifier appears as PREFIX ('smoothed_extrema') for one column
and as SUFFIX ('smoothed_extrema_weight') for another.
- 'smoothed_extrema_weight' mixes prefix stage word with suffix axis word.
Production codebases (statsmodels Kalman, bukosabino/ta MACD, mlflow
NPMI, FreqAI's own _mean/_std) overwhelmingly use suffix-decorated
processed forms with the raw form as the plain base. FreqAI's internal
pattern is suffix (&s-extrema_weight, &s-extrema_mean, &s-extrema_std);
align with it.
Rename:
- SMOOTHED_EXTREMA_COLUMN ('smoothed_extrema')
-> EXTREMA_DIRECTION_SMOOTHED_COLUMN ('extrema_direction_smoothed')
- SMOOTHED_EXTREMA_WEIGHT_COLUMN ('smoothed_extrema_weight')
-> EXTREMA_WEIGHT_SMOOTHED_COLUMN ('extrema_weight_smoothed')
Result: every viz column follows extrema_<axis>[_smoothed]. The 2x2
grid is uniform, sort-order groups raw and smoothed pairs together,
and the pattern is internally consistent with FreqAI's existing
suffix-based derivations.
* fix(weights): address PR #72 review comments
Three-agent cross-validation (explore + librarian + oracle) of Copilot
review comments produced these verdicts:
C1 (Utils.py:compose_sample_weights) — REAL BUG. Replacing 0-valued
weights with 1.0 silently undoes sklearn / AFML §4.10's canonical 'drop
this sample' semantic. sklearn's _check_sample_weight_equivalence,
DecisionTree _splitter.pyx, HistGBM docs, LightGBM #5553/#905, XGBoost
#3787 and mlfinlab time-decay all converge on the same contract:
sample_weight=0 means 'this sample contributes nothing'. Preserve zeros
via a drop_mask that is OR'd across labels (any label saying 'drop'
wins), then re-applied after the geometric-mean composition. Non-zero
non-finite or negative values still collapse to 1.0 (geometric mean's
neutral element) since they represent undefined weights, not exclusions.
C3/C4 (QuickAdapterRegressorV3.py:_train, timeseries_split) — REAL
REGRESSION. Commit
9953f0c removed ensure_datetime_series with the
rationale 'mirror BaseRegressionModel.train exactly'. But
ensure_datetime_series was introduced in commit
ce843f9 specifically
as a workaround for freqtrade issue #13107 (int64 epoch-ms date
columns from feather/parquet handlers). Mirror the algorithm, retain
project-specific safety patches. Restore ensure_datetime_series in
both train paths.
C2/C7 (_label_weight_column_name unused) — DRY violation. Both call
sites in _compose_train_weights now use the helper instead of inline
f-strings.
C8 (train() docstring) — Inaccurate. The default path was claimed to
'Delegate to BaseRegressionModel.train()' but actually routes to
self._train() (a mirror with weight composition). Fix docstring to
reflect actual control flow.
C5/C6 (**kwargs forwarding) — FALSE POSITIVE. freqai_interface never
passes kwargs to model.train(); upstream BaseRegressionModel.train
also calls self.fit(dd, dk) without kwargs. The current code matches
upstream and the call chain is dead in practice.
* refactor(weights): factor train paths and relocate methods for structural coherence
Three-agent structural audit (explore + librarian + oracle) identified
five issues; fixes that don't fight existing conventions:
1. _train and the timeseries_split inline branch in train() shared
~30 lines of identical scaffolding (filter_features, dates logging,
fit_labels guard, weight composition, pipeline application, fit,
timing logs). Extract _train_common(unfiltered_df, pair, dk, split_fn)
that owns the full mirror; _train_default and _train_timeseries_split
become 4-line dispatchers passing the split callback. train() routing
collapses to a clean two-line if/elif.
2. _label_weight_column_name, _strip_label_weight_columns and
_compose_train_weights were inserted into the middle of the class
constants block (between _TEST_SIZE and _SQRT_2), interrupting the
constant-block coherence. Move them to the private instance method
zone, immediately after _apply_pipelines (their natural neighbour).
3. _compose_train_weights duplicated the train/test weight extraction
loop verbatim. Factor into a static _extract_split_weights helper
that takes a split index and returns the per-label weight map; both
train and test call sites become single expressions.
4. The four visualization column constants (EXTREMA_DIRECTION_COLUMN,
EXTREMA_DIRECTION_SMOOTHED_COLUMN, EXTREMA_WEIGHT_COLUMN,
EXTREMA_WEIGHT_SMOOTHED_COLUMN) were 75 lines below EXTREMA_COLUMN /
LABEL_COLUMNS, separated by the LabelData dataclass and label
generator registry. Move them next to EXTREMA_COLUMN where they
logically belong.
5. Revert the train() docstring bullet to its upstream form. The
modification introduced RST cross-reference syntax inconsistent with
the surrounding plain-text docstring style.
* refactor(weights): centralize LABEL_WEIGHT_SUFFIX in Utils
The "_weight" suffix was duplicated as a class constant in
QuickAdapterRegressorV3 and as 5 hardcoded f-strings in QuickAdapterV3.
Three-agent audit (explore + librarian + oracle) converged on moving
this column-naming convention to Utils.py:
- PEP 8 default for constants is module-level; class-level is the
exception for class-private semantics.
- Both consumers already import column-naming constants from Utils.py
(LABEL_COLUMNS, EXTREMA_COLUMN, EXTREMA_WEIGHT_COLUMN, etc.). The
suffix belongs with them.
- Production precedents (sklearn UNUSED/WARN/UNCHANGED, mlflow
_SAMPLE_WEIGHT/_TRAINING_PREFIX, lightgbm _DatasetNames, pandas
LOCAL_TAG) all place cross-module string tokens at module level.
- The constant describes a dataframe schema contract (column names),
not model behaviour. Schema concerns belong in the schema module.
Add LABEL_WEIGHT_SUFFIX to Utils.py next to LABEL_COLUMNS. Remove the
class-level _LABEL_WEIGHT_SUFFIX in QuickAdapterRegressorV3 and import
the module-level constant. Replace 5 hardcoded f-strings in
QuickAdapterV3 with a local label_weight_col binding using the
imported constant.
The private _label_weight_column_name helper is kept in the regressor
since it is used twice (in _strip_label_weight_columns and
_compose_train_weights) and still adds a thin DRY layer over the
suffix synthesis.
* style: apply ruff formatting
* feat(weights): add label_weight_column helper with regex prefix strip and collision assertion
* fix(weights): preserve drop_mask and prevent NaN in compose_sample_weights fallback
* refactor(weights): adopt label_weight_column helper for canonical training column
* feat(weights): add _build_per_row_weights helper for pre-split weight composition
* feat(weights): add _make_default_split_datasets mirror with sklearn-key whitelist
* refactor(weights): make timeseries split helper accept external weights parameter
* refactor(weights): refactor _train_common chain and delete obsolete weight helpers
* refactor(weights): replace train() if/elif with dispatch dict and add weight-column uniqueness check
* fix(weights): harden compose_sample_weights for degenerate inputs
Address audit findings A0-1, A0-2, A0-3, A0-4, P2 #6, P2 #9, P2 #11
on branch feat/per-label-sample-weights.
- Extract _sanitize_and_renormalize private helper with four-guard chain
(positive sum, finite sum, finite ratio, finite scaled) and uniform
fallback. Used at empty-map fast path and at final fallback site.
- Empty label_weights_map now sanitizes raw temporal (A0-3).
- Subnormal temporal no longer overflows: ratio + scaled both checked
for finiteness before returning (A0-2).
- Drop predicate unified as 'arr <= 0 or non-finite' instead of exact
zero, eliminating the discontinuity at zero from smoothing artifacts
(A0-4); negatives now drop, no longer rescued to 1.0 (subsumes A1-10).
- Surviving positive values floored at np.finfo(float).tiny to prevent
subnormal arithmetic in the geo-mean log step.
- drop_mask covering all rows now raises ValueError instead of silently
returning all-zero weights that crash XGBoost / sklearn HGBR (A0-1).
- Up-front per-label shape validation raises a precise error instead of
letting numpy broadcasting fail mid-computation (P2 #11).
* docs(weights): document compose_sample_weights contract
Address audit findings A1-2, P2 #12, A1-12 on branch
feat/per-label-sample-weights.
Add 11-line docstring covering: output invariant (mean=1), per-label
sanitization predicate, aggregation operator, drop semantics, error
conditions, and the bounded full-series-median leakage in
compute_label_weights.
* refactor(weights): inline data-split dispatch with match/case
Address audit findings A1-1, A1-7, A1-6 (obsolete), P2 #14 on branch
feat/per-label-sample-weights. Conflict C1: A1-7 wins over A1-3
(YAGNI on subclass extension; LSP traceability preserved).
- Delete _DATA_SPLIT_DISPATCH class attribute (4 LOC).
- Delete _data_split_methods_set lru_cached helper (4 LOC dead code).
- Delete _train_default and _train_timeseries_split wrappers (~30 LOC
pure boilerplate dispatching to _train_common).
- Inline match/case on method name in train(); pick the right
_make_*_split_datasets via a local split_builder; nested split_fn
closes over dk. Net -37 LOC; full LSP traceability.
- Add SplitFn module-level type alias used in _train_common signature.
* fix(weights): reject bool config values in numeric validators
Address audit findings A1-4, P2 #5, P2 #7 on branch
feat/per-label-sample-weights.
Python's `bool` is an `int` subclass, so `isinstance(True, int)` is
true and config values of `true`/`false` silently passed through as
`1`/`0` in the validators for n_splits, gap, max_train_size,
test_size and weight_factor. This commit closes that footgun:
- Add static helpers _coerce_int (always returns int, raises on bool
or non-int) and _coerce_optional_int (returns Optional[int]) to
centralize the validation; both echo the offending raw value via
`{value!r}` so the diagnostic shows True/False rather than 1/0.
- Apply _coerce_int to n_splits and gap, _coerce_optional_int to
max_train_size in _make_timeseries_split_datasets.
- Add explicit bool guard for test_size in both default-split and
timeseries-split paths; previously test_size=true would slip past
isinstance(_, int) and silently train on 1 sample.
- Add explicit bool guard for weight_factor before the >0 comparison.
* fix(weights): preserve mean=1 invariant across pipeline stages
Address audit finding A1-5 on branch feat/per-label-sample-weights.
compose_sample_weights guarantees sum(w)==N (mean(w)==1) over the full
training series, but this invariant breaks twice downstream: (1) the
train/test split partitions weights into disjoint subsets whose means
no longer equal 1, and (2) feature_pipeline.fit_transform may drop
rows via SVM/DBSCAN, drifting the means further. XGBoost
min_child_weight, LightGBM min_sum_hessian_in_leaf and L2
regularization are all sensitive to absolute weight scale.
- Add static helper _renormalize_to_unit_mean with the same four-guard
chain as _sanitize_and_renormalize (positive sum, finite sum, finite
ratio, finite scaled, uniform fallback).
- Apply at four sites: before dk.build_data_dictionary in both
_make_default_split_datasets and _make_timeseries_split_datasets,
and after feature_pipeline.fit_transform / .transform in
_apply_pipelines (train and test sides).
* feat(weights): make label-weights aggregation configurable
Address audit finding A1-11 on branch feat/per-label-sample-weights.
Conflict C3: switch default to arithmetic_mean (matching
_compute_combined_label_weights), expose all 6 aggregations via
existing _aggregate_metrics infrastructure.
The hardcoded geometric mean over per-label normalized arrays was
mathematically conservative (one weak label dominates) and inconsistent
with the project's _compute_combined_label_weights default
(arithmetic_mean). For PR #44's correlated multi-target labels
(amplitude, time_to_pivot, efficiency, natr all derived from zigzag),
geomean over-counts redundant evidence and silently degrades to ~0
when any single factor is small. AFML \xc2\xa74.4 recommends arithmetic-mean
equivalents for correlated meta-labels.
- Add aggregation parameter to compose_sample_weights with default
COMBINED_AGGREGATIONS[0] ("arithmetic_mean"); also expose
softmax_temperature.
- Delegate the row-wise aggregation step to _aggregate_metrics, reusing
the existing 6-operator infrastructure with uniform unit coefficients.
- Read both knobs from feature_parameters in _build_per_row_weights:
label_weights_aggregation and label_weights_softmax_temperature.
* refactor(weights): align naming on compose/sigil/base_weights
Address audit findings A1-9, A1-14, A1-15, A1-16 on branch
feat/per-label-sample-weights.
- Rename _LABEL_WEIGHT_PREFIX_PATTERN to _FREQAI_LABEL_SIGIL_PATTERN:
the regex strips the freqtrade-native '&' sigil, not a label-weight
prefix; the new name describes what is matched (Utils.py).
- Rename compose_sample_weights parameter `temporal` to `base_weights`:
the parameter accepts any base vector (recency weights or uniform
ones), not exclusively temporal data (Utils.py).
- Rename _build_per_row_weights to _compose_per_row_weights:
standardize on the 'compose' verb to mirror compose_sample_weights;
this helper is the orchestrator that calls the kernel
(QuickAdapterRegressorV3.py).
- Rename _build_weights_array to _scatter_weights: the function
scatters sparse pivot weights into a dense default-filled array,
not a generic 'build' (Utils.py).
- Rename eval_set_and_weights to make_test_set_and_weights: aligns
with FreqAI's 'test_*' data_dictionary vocabulary while avoiding
the 'test_' prefix that pytest auto-discovers (the verb 'make_'
also clarifies it as a constructor, not a test) (Utils.py + caller).
* refactor(weights): extract _shuffle_in_unison helper
Address audit findings A1-8 and A1-13 on branch
feat/per-label-sample-weights.
- Extract the train/test shuffle pattern into a static
_shuffle_in_unison helper. Each call shuffles features, labels and
weights with the same random seed in lockstep. The shuffle block in
_make_default_split_datasets shrinks from ~28 lines (two duplicated
5-line idioms x train+test) to two helper invocations.
- Fix the dk.data_dictionary vs dd inconsistency at the feature-count
log line: read from local dd (the pipeline's return value) rather
than dk.data_dictionary (a side-effect set by _apply_pipelines).
* chore(weights): polish naming, validators, caching
Address audit P2 polish items #2, #4, #8, #13, #15 on branch
feat/per-label-sample-weights. P2 #21 and #22 (logger telemetry)
deliberately skipped per the no-new-comments/no-new-infrastructure
constraint; the existing fail-fast ValueError on degenerate inputs
already surfaces the most critical failure modes loudly.
- Counter-based duplicate-label diagnostic now names the offending
weight columns instead of merely raising on a length mismatch
(P2 #2).
- Widen shuffle seed space from random.randint(0, 100) (101 distinct
seeds, birthday collisions at sqrt(101) ~ 10) to randint(0, 2**31-1)
at both _shuffle_in_unison call sites (P2 #4).
- dsp = dict(self.config['freqai']['data_split_parameters']) replaced
with self.data_split_parameters (the safe pre-populated FreqAI
attribute used everywhere else in the file) (P2 #8).
- Cache label_weight_column with @lru_cache(maxsize=16): the helper
is pure on its single string argument and called in tight loops at
training; matches the file's existing convention for similar helpers
(P2 #13).
- Rename loop variable w to label_values in compose_sample_weights;
the outer scope spans ~20 lines and prior single-letter w obscured
the role (P2 #15).
* docs(weights): align train() and helper docstrings with current behavior
- Rewrite train() docstring to describe match-based dispatch and the per-row
weight composition flow through _train_common; remove stale delegation claim.
- Sync _compose_per_row_weights docstring: aggregation default is
arithmetic_mean, not geometric_mean.
- Fix AFML citation in compose_sample_weights from section 7.4 to chapter 4.
- Document _aggregate_metrics softmax branch as a per-column convex
combination with explicit T->0 and T->+inf limits.
* refactor(weights): consolidate sample weight renormalization helper
- Promote Utils._sanitize_and_renormalize to public sanitize_and_renormalize.
- Drop QuickAdapterRegressorV3._renormalize_to_unit_mean (cross-file
duplication of the mean=1 invariant); replace 6 call sites with the
unified helper. Sites that previously skipped per-element sanitization
now also reject non-finite or non-positive entries.
- Collapse triple-guard ladder in sanitize_and_renormalize to a single
finite-positive total check; surviving 'safe' is provably finite-nonneg
so intermediate isfinite checks were dead code.
- Tighten _shuffle_in_unison signature from Any to concrete pd.DataFrame
and NDArray types.
- Drop empty-fold sentinel in _make_timeseries_split_datasets and raise
ValueError on degenerate generator output instead of silently producing
empty index arrays.
* feat(weights): validate label_weights tunables via label_weighting block
- _compose_per_row_weights now consumes get_label_weighting_config (which
validates aggregation against COMBINED_AGGREGATIONS and enforces
softmax_temperature > 0 via _WEIGHTING_SPECS) instead of reading raw
feature_parameters.label_weights_*.
- Add CONFIG_MIGRATIONS entries auto-migrating
freqai.feature_parameters.label_weights_aggregation and
freqai.feature_parameters.label_weights_softmax_temperature to the
freqai.label_weighting block; users get one warning per key.
- Add module-level _logger to Utils.py and warn on
compose_sample_weights silent fallback so collapsed-aggregation paths
are observable.
- _make_timeseries_split_datasets honors reverse_train_test_order for
parity with _make_default_split_datasets and raises ValueError on
shuffle_after_split=True (chronological + shuffle is incoherent and
would leak future data into training).
* fix(weights): seed shuffle deterministically from data_split_parameters.random_state
Replace global random.randint() with a random.Random instance derived from
data_split_parameters.random_state. When the user provides a random_state
(whitelisted in _SKLEARN_TRAIN_TEST_SPLIT_KEYS), train and test shuffles
become reproducible end-to-end; when absent, behavior remains
non-deterministic. The single parent RNG draws two independent sub-seeds
so train and test shuffles stay decorrelated.
* refactor(weights): rename for naming coherence
- Rename _make_default_split_datasets to _make_train_test_split_datasets
to restore the case-key/method-name grep-line in train()'s match
dispatch.
- Rename label_weight_column to label_weight_column_name across Utils.py,
QuickAdapterV3.py and QuickAdapterRegressorV3.py: the helper returns
a column-name string, not a column accessor; the new name matches the
*_COLUMN constant convention used elsewhere in Utils.py.
- Drop redundant 'dk.data_dictionary = dd' in _apply_pipelines:
build_data_dictionary already self-assigns the dict on dk and dd is
the same object reference.
* style(weights): group EXTREMA_* constants and separate LABEL_* declarations
* docs(weights): rewrite _make_train_test_split_datasets docstring without history narration
Replace the deviation list and PR-history reference with a concise
description of what the function IS (sklearn-key whitelist, honored
tunables, weight propagation contract).
* fix(weights): resolve KeyError in label_weighting config consumption
_compose_per_row_weights passed self.config (root) to
get_label_weighting_config and accessed weighting_config['aggregation']
directly; the helper expects freqai.label_weighting and returns
{default, columns}. Fix consumes self.freqai_info['label_weighting']
and reads ['default']['aggregation'] / ['default']['softmax_temperature'].
* fix(weights): honor drop_mask in sanitize_and_renormalize fallback
When total <= 0 or non-finite, the helper returned np.ones_like(arr)
ignoring drop_mask, resurrecting dropped rows with weight=1. Fallback
now zeros drop_mask rows before returning.
* fix(config): rename reverse_test_train_order to reverse_train_test_order
Match the canonical key name from upstream freqtrade and from the code
in QuickAdapterRegressorV3._make_train_test_split_datasets and
_make_timeseries_split_datasets. The previous template key was silently
ignored.
* fix(strategy): clip smoothed weight column to non-negative finite
Some smoothing methods (savgol, filtfilt) can ring negative on positive
input. Clip the smoothed weight series to >= 0 and replace non-finite
with 0 before assigning to the dataframe so compose_sample_weights does
not silently drop rows that were positive before smoothing.
* fix(weights): apply project _TEST_SIZE default when data_split_parameters omits test_size
The whitelist comprehension that builds sklearn_kwargs only preserved keys
present in data_split_parameters; the local test_size variable computed via
dsp.get(..., _TEST_SIZE) was never injected back. Configs without an explicit
test_size silently fell through to sklearn's stock 0.25 default instead of
the project's 0.1.
Replace bare 'shuffle' insertion with setdefault for both shuffle and
test_size so sklearn_kwargs always carries the project defaults.
Update _make_train_test_split_datasets docstring to reflect the actual
default behavior.
* refactor(strategy): rename smoothed_weights to smoothed_label_weights
Aligns naming with surrounding label_weights variable, EXTREMA_WEIGHT_SMOOTHED_COLUMN
constant and compute_label_weights helper.
* feat(weights): split per-row aggregation into freqai.sample_weighting block
Cross-metric (per-pivot) and cross-label (per-row) compositions are
distinct distributions. Decouple their tunables:
- freqai.label_weighting.{aggregation,softmax_temperature} stay for
cross-metric aggregation in compute_label_weights (combined strategy).
- New freqai.sample_weighting.{aggregation,softmax_temperature} for
cross-label composition in
QuickAdapterRegressorV3._compose_per_row_weights.
Add _SAMPLE_WEIGHTING_SPECS, DEFAULTS_SAMPLE_WEIGHTING and
get_sample_weighting_config helper routed through _get_label_config so
the returned shape ({default, columns}) matches the get_label_*_config
family.
* refactor(weights): pass logger as parameter and log per-label weight column status
Drop the module-level _logger introduced in Utils.py; compose_sample_weights
now takes a keyword-only logger argument, matching the caller-passes-logger
pattern used by every other helper in the file.
In _compose_per_row_weights, log per-label weight column resolution at
debug when columns are present (static across retrains) and at warning
when none are found (unexpected configuration; falls back to temporal
weights only).
* fix(weights): use shape-consistent empty containers for test_size=0 sentinels
Replace np.zeros(2)/pd.DataFrame() sentinels with iloc[:0] / weights[:0]
slices so test_features, test_labels and test_weights all have 0 rows
with preserved column names and dtype. Behavior unchanged because
_apply_pipelines skips test-side processing when test_size == 0, but
shape-consistent containers respect the declared types and avoid
surprising downstream consumers.
fix: ensure_datetime_series raises ValueError on None instead of silent corruption
refactor(zigzag): replace ad-hoc x/(x+y) normalization with pure ratios + MinMaxScaler
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-opencode)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
feat(quickadapter): add TimeSeriesSplit as alternative data splitting method (#48)
* feat(model): add TimeSeriesSplit support via train() override
* docs(readme): document TimeSeriesSplit configuration options
* fix: address PR review comments
- Use test_size parameter in TimeSeriesSplit
- Remove unused dk parameter from _make_timeseries_split_datasets()
- Assign dk.data_dictionary = dd before logging
- Fix typo: train_test_test -> train_test_split in README
* docs: integrate data_split_parameters into tunables table
Remove standalone section and add parameters to existing table
with freqai. prefix for consistency.
* refactor: use FreqAI APIs for weight calculation and data dictionary
- Use dk.set_weights_higher_recent() instead of duplicating weight formula
- Use dk.build_data_dictionary() for consistent data structure
- Respects feature_parameters.weight_factor configuration
- Fix bug: was using data_kitchen_thread_count instead of weight_factor
* refactor: extract _apply_pipelines() to reduce code duplication
- Move pipeline definition and application logic to helper method
- Reduces train() override complexity while keeping same behavior
- Helper can be reused by future custom split implementations
* style: harmonize namespace and remove inline comments
- Rename DATA_SPLIT_METHODS to _DATA_SPLIT_METHODS (private tuple pattern)
- Reference DATA_SPLIT_METHOD_DEFAULT from _DATA_SPLIT_METHODS[0]
- Remove 22 inline comments to match self-documenting codebase style
* fix: align TimeSeriesSplit weight calculation with FreqAI semantics
Calculate weights on combined train+test set before splitting to maintain
temporal weight continuity, matching FreqAI's make_train_test_datasets behavior.
* feat: add gap=0 warning and improve TimeSeriesSplit validation
- Warn when gap=0 about look-ahead bias risk (reference label_period_candles)
- Add _compute_timeseries_min_samples() for accurate minimum sample calculation
- Account for gap and test_size in minimum sample validation
- Improve error message with all relevant parameters
* style: harmonize error messages with codebase conventions
- Use 'Invalid {param} value {value!r}: {constraint}' pattern
- Align with existing validation error format (lines 718, 1145)
* style: add cached set accessor for data split methods
- Add _data_split_methods_set() with @staticmethod @lru_cache
- Use QuickAdapterRegressorV3 prefix for class attribute access
- Use cached set for O(1) membership check in validation
* fix: address PR review comments for TimeSeriesSplit
- Use dd consistently in training logs instead of dk.data_dictionary
- Use self.data_split_parameters consistently in _apply_pipelines
- Add explicit type coercion for n_splits, gap, max_train_size
- Add validation for gap >= 0 and max_train_size >= 1
- Improve test_size validation: float in (0,1) as fraction, int >= 1 as count
- Fix _compute_timeseries_min_samples formula: (n_splits+1)*test_size + n_splits*gap
- Optimize tscv.split() iteration to avoid unnecessary list materialization
* fix: correct min_samples formula to match sklearn validation
sklearn validates: n_samples - gap - (test_size * n_splits) > 0
Correct formula: test_size * n_splits + gap + 1
* feat: auto-calculate TimeSeriesSplit gap from label_period_candles
When gap=0 is configured, automatically set gap to label_period_candles
to prevent look-ahead bias from overlapping label windows. This ensures
temporal separation between train and test sets without requiring manual
configuration.
* fix: remove redundant time import shadowing module
* fix: correct min_samples formula for dynamic test_size and document test_size param
* refactor: remove redundant TimeSeriesSplit min_samples validation
* docs: clarify test_size default per split method
* refactor: move DependencyException import to file header
* style: use class name for class constant access
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* docs: use Python None instead of null in README
* docs: fix train_test_split description (sequential, not random)
* fix: use explicit None check for max_train_size validation
* docs: clarify timeseries_split as chronological split, not cross-validation
* refactor(quickadapter): shorten log prefixes and tailor empty test set error by split method
* refactor(quickadapter): use index pattern for timeseries_split method constant
Replace string literals with index access pattern following existing
codebase convention for _DATA_SPLIT_METHODS.
Also renames variables for semantic clarity:
- test_size_param -> test_size
- feat_dict -> feature_parameters
* refactor(quickadapter): use _TEST_SIZE constant instead of hardcoded 0.1
* chore(quickadapter): bump version to 3.11.2
* fix(quickadapter): restore test_size parameter in TimeSeriesSplit
The test_size variable from data_split_parameters was being
immediately overwritten by a type annotation line, making it
always None regardless of user configuration.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
repositories / freqai-strategies.git / log
