Jérôme Benoit [Fri, 28 Aug 2026 16:18:38 +0000 (18:18 +0200)]
fix(quickadapter): gate final take-profit on retracement
Replace the elapsed-time final take-profit exit with a persisted volatility-scaled retracement, harden its state invariants, and harmonize final-exit terminology.
Jérôme Benoit [Sun, 23 Aug 2026 21:14:34 +0000 (23:14 +0200)]
chore(quickadapter): update label_weighting and label_smoothing templates
- set label_smoothing method to kaiser_bessel_derived with window_candles 6
- add commented alternative label_weighting example (amplitude strategy,
epsilon_gaussian fill, knn bandwidth)
- align commented per-label beta example with canonical default (8.0)
Ports QuickAdapter's _install_date_pred_dedup_patch verbatim to ReforceXY. The duplicate-date_pred / validate="m:1" MergeError originates in the shared FreqaiDataDrawer (freqtrade 2026.7) and affects every FreqAI model; ReforceXY runs in its own process and never imports QuickAdapter, so it needs its own identical copy.
Jérôme Benoit [Wed, 12 Aug 2026 20:54:36 +0000 (22:54 +0200)]
fix(quickadapter): guard against freqtrade duplicate date_pred regeneration (#202)
* revert(scripts): drop historic-predictions dedup tool
Reverts the squash-merged PR #201 (commit c20d23a2). A one-shot on-disk
deduplication cannot durably fix the issue: freqtrade 2026.7 regenerates the
duplicate date_pred in memory on restart. It is superseded by an in-memory
guard in the QuickAdapter model (next commit).
* fix(quickadapter): guard against freqtrade duplicate date_pred regeneration
freqtrade 2026.7 FreqaiDataDrawer.set_initial_return_values trims the new
prediction window by position (new_pred.iloc[len(common_dates):]) instead of by
date_pred value. When the persisted history overlaps the tail of the analysis
window (the normal restart geometry), the trim misaligns and the following
pd.concat re-creates duplicate date_pred rows, which the validate="m:1" merge in
attach_return_values_to_return_dataframe rejects with MergeError, blocking the
pair. A one-shot on-disk dedup cannot fix this since the duplicate is
regenerated in memory on restart.
Install an idempotent in-memory guard from the model module that wraps
set_initial_return_values, append_model_predictions and
attach_return_values_to_return_dataframe to keep date_pred unique, preferring a
real prediction over a zero/NaN placeholder and preserving NaT rows. Validated
against freqtradeorg/freqtrade:stable_freqai (2026.7) across all restart
geometries.
* fix(quickadapter): keep original row order in date_pred dedup
Sort the kept rows by their original position (_order) instead of by
date_pred (_dp). The date sort pushed interior NaT rows to the tail and could
change which rows tail(N) returns to the strategy, dropping a real candle and
injecting a placeholder. Ordering by original position preserves freqtrade
insertion order minus the removed duplicates; date_pred uniqueness is already
guaranteed by drop_duplicates.
* style(quickadapter): type the date_pred dedup wrappers
Annotate the three FreqaiDataDrawer wrappers to match the module conventions
and freqtrade signatures (data_drawer.py:286/343/413).
* docs(quickadapter): sharpen date_pred duplicate attribution in guard docstring
State that set_initial_return_values counts the overlap but trims new_pred by
position, so a crash-persisted or overlapping store breaks the contiguous
head-prefix assumption and reintroduces duplicate date_pred; drop the
inaccurate "normal restart geometry" framing.
Lead with the robust primary cause: a store already holding duplicate
date_pred rows survives set_initial_return_values unchanged (its pd.concat
preserves hist_preds) and trips the validate="m:1" merge. Demote the
positional-trim path to a secondary case and drop the inaccurate claim that it
needs non-monotonic dates: a gapped or tail-overlapping store triggers it with
strictly monotonic dates too (proven). Anchor the data_drawer.py source lines.
Move the freqtrade-bug rationale from the pure _dedupe helper (contract only:
unique non-NaT date_pred, most-informative row, NaT preserved, order kept) to
_install, where the patch rationale belongs, and condense it with source
anchors. Net -13 doc lines; matches the module terse-docstring style.
* fix(quickadapter): drop NaT date_pred and restore chronological dedup order
Two correctness defects in the date_pred dedup guard reached the validate="m:1"
merge in attach_return_values_to_return_dataframe (data_drawer.py:429-431):
- The early return only checked duplicate valid dates, so a frame with >=2 NaT
date_pred was returned unchanged; the merge then raised MergeError ("Merge
keys are not unique in right dataset") because pandas counts repeated null
keys as non-unique, even with no NaT on the left.
- Ordering the kept rows by original position (_order) left the frame
non-chronological, so the tail(N) trim in set_initial_return_values and
append_model_predictions could keep stale dates and evict recent ones.
Drop NaT date_pred rows (a NaT matches no candle -- freqtrade always assigns
real dates at set_initial_return_values, data_drawer.py:305 -- and only harms
the m:1 merge) and order survivors chronologically by normalized date_pred; the
most-informative row per timestamp is still kept. This supersedes the _order
sort introduced in f4b95a7b: removing the NaT rows makes that reorder
unnecessary and the tail chronologically correct.
Add scripts/historic_predictions_deduplicate.py to repair FreqAI
historic_predictions.pkl stores that a crash-recovery backfill can leave with
duplicate date_pred rows, which break the prediction merge for that pair.
- Keeps the most informative row per date_pred (informative cells, then non-null
count, then most-recent write); deduplicates only non-NaT keys.
- Dry-run by default; --apply writes atomically (temp + fsync + os.replace,
preserving mode and best-effort owner) after copying the original aside as
.original-<stamp>. Exits 1 when any target is skipped as unreadable.
- Repairs the sibling historic_predictions.backup.pkl as well.
- Portable single-file tool with a bundled test suite run via uv.
Jérôme Benoit [Wed, 5 Aug 2026 11:46:54 +0000 (13:46 +0200)]
style(optuna): apply ruff format to best-params I/O
Reflow the lock helper, save fchown block and legacy resolver wrapping
to satisfy ruff format at the default 88-column width, matching the rest
of both files (main was format-clean). Whitespace and redundant grouping
parentheses only; the AST is unchanged.
Jérôme Benoit [Tue, 4 Aug 2026 23:16:21 +0000 (01:16 +0200)]
fix(optuna): harden best-params I/O against deployment failure modes
Address three P2 review findings; the same latent behaviors were ported
from QuickAdapter to ReforceXY, so all fixes are applied to both.
- Cross-uid save (C2): make owner preservation best-effort. A non-root
process on a bind mount with a differently owned existing file lacks
CAP_CHOWN; the fchown now swallows PermissionError (logged at debug)
so the atomic write still completes, as the previous in-place write
did. fchmod and the successful-chown path are unchanged.
- Read-only mount (C3): a shared (read) lock no longer forces O_CREAT.
On a read-only mount where the lock file is absent, the load reads
without a lock; os.replace atomicity guarantees a consistent read.
Exclusive locks keep O_CREAT and still fail closed on read-only
filesystems. S_ISREG/O_NOFOLLOW/O_NONBLOCK guards are preserved.
- Legacy warm-start (C1): a base-only legacy best-params file is now
loaded when exactly one configured pair maps to that base (read in
place, no rename, read-only safe). Ambiguous cases (more than one pair
sharing a base) still warn and return None, preserving #180 safety.
The configured pair list is threaded into the QuickAdapter module-level
loader; a legacy payload still passes the same validation.
Mirror the rationale already documented on the shared QuickAdapter
helper: _quarantine_path appends the tag and timestamp after the
complete filename (extension included) so live-artefact globs never
match quarantined files. Docstring only; no behavior change.
The best-params quarantine in Utils re-inlined the timestamp + tie-break
path computation that the journal quarantine already implemented in
QuickAdapterRegressorV3, duplicating the algorithm across the two
modules.
- Add a shared, parameterized _optuna_quarantine_path(path, now, *, tag,
limit) in Utils (the module both call sites can import).
- Delegate from _quarantine_corrupt_optuna_best_params (Utils) and from
_optuna_quarantine_journal (regressor); each keeps its own domain
constants (journal vs best-params), logging and rename handling, so
quarantine names, log output and error semantics are unchanged.
- Remove the regressor's private _optuna_quarantine_path copy.
Review follow-up (DRY, single-source-of-truth). The best-params
quarantine re-inlined the timestamp + tie-break path computation already
provided by _quarantine_path, and duplicated the journal quarantine
tag/limit constants with identical values.
- Reuse _quarantine_path from _quarantine_corrupt_best_trial_params;
the wrapper keeps its own pair-prefixed logging and rename error
handling, so both call sites' log output and error semantics are
unchanged.
- Collapse _JOURNAL_QUARANTINE_TAG/_BEST_PARAMS_QUARANTINE_TAG and their
tie-break limits into neutral shared _QUARANTINE_TAG /
_QUARANTINE_TIE_BREAK_LIMIT (same values: "corrupt" / 99); generalize
the _quarantine_path parameter name journal_path -> path.
- Regroup the constants so the _JOURNAL_* block stays contiguous, with
the shared quarantine constants after it.
Port the QuickAdapter Optuna best-params I/O hardening to ReforceXY's
save_best_trial_params / load_best_trial_params:
- Encode the complete pair identity in the filename via _sanitize_pair
instead of the base currency only, so pairs sharing a base (e.g.
BTC/USDT and BTC/USDC) no longer collide on one file.
- Warn and ignore a base-only legacy best-params file that shadows the
pair-safe path.
- Write atomically (temp file + fsync + os.replace), preserving the
previous file owner and mode.
- Serialize reads and writes with a flock lock file opened
O_NOFOLLOW|O_NONBLOCK|O_CLOEXEC and guarded by an S_ISREG check.
- Fail closed when the live path is a symlink.
- Quarantine a corrupt best-params file on load and return None instead
of raising, mirroring the existing journal quarantine.
Jérôme Benoit [Sun, 2 Aug 2026 18:39:14 +0000 (20:39 +0200)]
fix(reforcexy): give jwt_secret_key and ws_token self-documenting placeholders
Match the QuickAdapter template: empty and placeholder are equally inert while
the API server is disabled, but the placeholder signals it must be replaced
before enabling the API.
Jérôme Benoit [Sun, 2 Aug 2026 18:35:43 +0000 (20:35 +0200)]
fix(reforcexy): reinstall matplotlib and bind API to localhost
The recent stable_freqairl image no longer ships matplotlib, so ReforceXY.py
fails to import ('No module named matplotlib'). Pin matplotlib==3.11.1 (cp311-cp314
wheels exist, so it installs without a compiler) via the Dockerfile build arg.
Also bind the published API port to 127.0.0.1 by default, matching the
QuickAdapter hardening, so the REST API is not exposed on all interfaces.
Jérôme Benoit [Sun, 2 Aug 2026 00:38:36 +0000 (02:38 +0200)]
fix(quickadapter): give ws_token a self-documenting placeholder like jwt_secret_key
Makes the README 'public placeholders that provide no protection until changed'
note accurate for the WebSocket token too; empty and placeholder are equally
inert while the API server is disabled.
Jérôme Benoit [Sun, 2 Aug 2026 00:21:51 +0000 (02:21 +0200)]
docs(quickadapter): trim README tunable cells to user-facing contracts
Drop internal availability-composition from the causal_mode cell, the
ValueError type from support_policy, and legacy/historical framing from
fill_bandwidth/reset_label_study/vary_model_seed; expand NATR on first use;
unify 'Quick start'; remove the profitability aside.
Jérôme Benoit [Sat, 1 Aug 2026 20:53:46 +0000 (22:53 +0200)]
fix(quickadapter): support CatBoost continual learning (#196)
* fix(quickadapter): support CatBoost continual learning
* fix(quickadapter): drop CatBoost init_model on GPU task_type
CatBoost init_model training continuation is CPU-only and raises
"Training continuation for GPU is not yet supported" on GPU. Passing it
unconditionally crashed GPU CatBoost retrains under continual_learning.
Drop init_model on the GPU path so it cold-starts instead.
* style(quickadapter): realign README table cells after continual-learning doc edits
refactor(quickadapter): route inline enum error messages through enum_error_message (#194)
* refactor(quickadapter): relocate enum_error_message to EnumErrors module
Extract the canonical enum validation error formatter into a new
dependency-free module (strategies/EnumErrors.py) and re-export it from
Utils. This lets LabelTransformer route through the same helper without
importing Utils (which would create an import cycle, since Utils imports
from LabelTransformer).
Behavior-preserving: the helper is moved verbatim; the public import path
'from Utils import enum_error_message' is preserved via re-export.
* refactor(quickadapter): route Utils enum errors through helper
Replace 5 inline 'Invalid X value ...: supported values are ...' error
constructions in Utils with enum_error_message calls (window type,
fill_bandwidth, fill_method, and two regressor sites). Byte-identical.
* refactor(quickadapter): route LabelTransformer enum errors through helper
Import enum_error_message from the dependency-free EnumErrors module
(LabelTransformer cannot import Utils without creating an import cycle)
and route the 3 inline enum error constructions (scaler family kind,
standardization, normalization). Byte-identical.
* refactor(quickadapter): route strategy enum errors through helper
Add enum_error_message to the existing 'from Utils import' block and
route the 4 inline enum error constructions in QuickAdapterV3
(trade_price_target_method, interpolation_direction, side, trading_mode).
Byte-identical.
* refactor(quickadapter): route regressor enum errors through helper
Route the 11 canonical inline enum error constructions plus 3 that keep
byte-identical output via explicit sequence wrapping:
- data_split_parameters.method, selection_method (x2), skimage threshold
method, trial_selection_method, cluster_method, aggregation,
label_method, optuna storage_backend, optuna sampler, namespace
- namespace single-value sites wrap (_OPTUNA_NAMESPACES.label,) so join
yields the same string; optuna namespace-sampler wraps tuple(samplers)
(frozenset -> Sequence[str]) preserving the existing join order.
Byte-identical.
* fix(quickadapter): unify divergent enum error wording via helper
Route the two enum error sites that did NOT include the word "value"
through enum_error_message, aligning them with the canonical format.
INTENTIONAL, observable message change (NOT byte-identical):
- QuickAdapterRegressorV3._validate_enum_value: 'Invalid {ctx} {value!r}:'
-> 'Invalid {ctx} value {value!r}:'. The message is reused by the
logger.warning at the same helper, so the change affects both the
raised ValueError and the warning log across its 6 callers.
- Utils.get_ngboost_dist dist_name: 'Invalid dist_name {v!r}:' ->
'Invalid dist_name value {v!r}:' (dict_keys wrapped in tuple() to
satisfy Sequence[str]; join output unchanged).
No test asserts these strings (quickadapter has no test suite).
Glue the first-party EnumErrors import directly to the preceding import
block in LabelTransformer, matching the convention already used in
Utils.py and QuickAdapterV3.py (no blank line separating first-party
from third-party imports). No behavior change.
* docs(quickadapter): scope EnumErrors docstring to the canonical form
The module owns the canonical 'Invalid <ctx> value <value>: supported
values are <options>' message; messages with custom prefix/infix/suffix
are built inline at their call sites. Avoids over-claiming a single
source of truth for every enum error string.
Share the combined `select -> impute -> aggregate` pipeline between the value
path (`_compute_combined_label_weights`) and the dependency-mask path
(`compute_label_weight_imputation_dependency_mask`) via a single
`_compute_combined_label_weight_pipeline` helper returning a
`_CombinedWeightPipeline`. The mask branch now derives the per-component and
aggregate non-finite masks and the leading-stable release (#169) from the
shared result instead of re-running selection, imputation, and aggregation.
Behavior is bit-for-bit identical for `compute_label_weights` and
`compute_label_weight_imputation_dependency_mask` across all strategies,
aggregations, fill methods, and causal/non-causal modes, including raised
exception type, message, and order. The two imputers (`_impute_weights` and
`_causal_impute_weights`) stay distinct; the epsilon path keeps its causal
imputer through the shared helper.
fix(quickadapter): fail closed on unmeasurable entry history (#193)
* fix(quickadapter): fail closed on unmeasurable entry history
Reject reversal entries when configured historical confirmation is non-finite, while preserving a valid current-candle exit result for exposure reduction.
* docs(quickadapter): drop internal fallback note from lookback tunable
The reversal_confirmation.lookback_period_candles row documents user-facing
usage; the non-finite historical confirmation fallback is an internal
implementation detail already described in reversal_confirmed's docstring.
Keep the tunable description terse and avoid duplicating internals per the
repository documentation conventions.
* refactor(quickadapter): reuse shared numeric validator for Optuna config
Route the Optuna numeric configuration validation through a new public
Utils.require_numeric() helper that wraps the canonical _NumericValidator,
instead of reimplementing bool/int/range checks inline. This removes the
duplicated validation logic, harmonizes error messages with the rest of
the module, and drops the unreachable math.isfinite() branch on
space_fraction (non-finite values already fail the range check).
Also align the README seed range notation (int [0, 4294967295]) with the
existing int [2, 10000] style and the emitted error message.
Behavior-preserving: raise/pass outcomes are identical to the previous
implementation across the full option matrix, except unbounded integer
options with values >= 2**64 (n_trials, timeout, n_startup_trials) are now
rejected via the shared finiteness contract (values <= 2**63 unchanged).
Keep exact built-in type semantics when using the shared helper and treat arbitrary-precision Python integers as finite.
* refactor(quickadapter): reuse shared boolean validator for Optuna config
Route the Optuna boolean option validation through a new public
Utils.require_bool() helper that wraps the canonical _BoolValidator,
mirroring require_numeric(). This removes the last inline validation in
_optuna_config so both boolean and numeric checks now share the framework
validators, closing the residual duplication.
Behavior-preserving: raise/pass outcomes are identical to the previous
inline check across all boolean options (True/False accepted, every other
type rejected).
* docs(quickadapter): justify the int short-circuit in _is_finite_value
The comment stated a Python language truism (integers are arbitrary-precision
and finite) instead of explaining why the branch exists. Reword it to document
the load-bearing rationale: np.isfinite raises TypeError/OverflowError on Python
ints >= 2**64, so routing them through it would misclassify finite values as
non-finite. Comment-only change; no behavior change.
fix(quickadapter): prevent best-params lock hang on FIFO paths (#191)
Open the best-params lock with O_NONBLOCK so a pre-existing FIFO at
.optuna-best-params.lock no longer blocks the O_RDONLY shared-load open
until a writer appears (O_NOFOLLOW rejects symlinks but not FIFOs),
hanging strategy startup before the S_ISREG guard can reject the
non-regular file. The flag is inert on regular files and does not affect
the subsequent blocking flock() (governed by LOCK_NB, not the fd flag).
Addresses the unresolved P2 review thread on merged PR #185.
Harmonize the lone bare "outer window" in the backtest evaluation protocol nested-HPO step with the "outer prediction window" term used elsewhere in the section, removing an ambiguous synonym for the rolling-origin evaluation window (distinct from the "outer holdout").
* docs(quickadapter): harmonize backtest-protocol terminology and qualify drawdown bootstrap
- clarify the row-wise known_at_lookahead maximum spans label and weight availability values
- use code font consistently for continual_learning
- align the label-candle shuffling seed role with the configuration tunables table
- name log returns explicitly before the positivity requirement
- unify "latest retained information time" across the purge step
- note the maximum-drawdown bootstrap bound is approximate for a non-smooth path-dependent functional
* docs(quickadapter): gate step 8 effect-size bootstrap on stationarity credibility
Make the stationary-bootstrap applicability in step 8 symmetric with the
Reality Check/SPA gate in step 9: declare the result inconclusive when the
paired series' stationarity and local-dependence assumptions are not credible
under a fixed or moving-window scheme. Addresses the outstanding PR #154 review
comment on the step 8 vs step 9 stationarity asymmetry.
* docs(quickadapter): harmonize primary-metric naming and fix spelling
- name the step 9 promotion bound the "net-log-return effect", consistent with
the step 7-8 net-log-return inference and the positive-is-favorable effect
terminology (removes the one-off "net-growth" synonym for the same estimand)
- correct "relabelled" to "relabeled" for US-English consistency
Align the Optuna best-params quarantine with the established journal
quarantine sibling (_optuna_quarantine_journal): source the artifact tag
from a new _OPTUNA_BEST_PARAMS_QUARANTINE_TAG constant and use "corrupt"
consistently in the docstring and log message (the function name and
.corrupt- artifact suffix already used "corrupt"; only the docstring and
warning diverged as "malformed"). Add a docstring to
_reject_optuna_best_params_symlink for parity with its sibling helpers.
On-disk artifacts and load results are byte-identical; only the log
wording and tag source change.
Consolidate the three divergent unlimited max_open_trades predicates into a
single _is_unlimited_open_trades helper, honoring both the raw -1 sentinel and
its runtime-normalized float("inf") form (matching Freqtrade's own
config_validation idiom).
Behavior is preserved for every schema-legal max_open_trades value (config
schema minimum is -1): the sole formal divergence is that max_open_trades < -1,
already rejected by Freqtrade at config load, is no longer treated as unlimited
by max_open_trades_per_side. Also makes the confirm_trade_entry guard reject
the +inf form explicitly instead of relying on IEEE finite >= inf semantics.
Rename _is_unlimited_open_trades to _is_unlimited_max_open_trades (and the
local flag) for terminological precision with max_open_trades and
max_open_trades_per_side, and widen its parameter annotation to int | float to
match the is_trade_duration_valid precedent. No behavior change.
fix(quickadapter): make best params pair-safe (#183)
* fix(quickadapter): make best params pair-safe
* style(quickadapter): order pair_to_filename import per isort
Place the freqtrade.misc import after the straight third-party imports so the block satisfies ruff's isort ordering (I001), which the pair-safe best-params change had regressed. No behavior change.
* docs(quickadapter): drop internal best-params filename details from warm_start
The warm_start tunable description exposed the on-disk best-params filename convention and legacy base-only cold-start handling: internal implementation detail with no bearing on setting the tunable (AGENTS.md: omit internal details unless necessary for usage). Revert the row to its pre-change wording; the pair-safe fix needs no tunable-doc change.
perf(quickadapter): harmonize causal availability for provably-stable zero-weight imputations (#186)
* perf(quickadapter): harmonize causal availability for provably-stable zero-weight imputations
Release trailing single-metric and combined leading non-finite runs whose
imputation is provably fixed at 0.0 under the causal prefix at their true
stabilization candle instead of the frame boundary, matching the existing
single-metric leading treatment (#158).
- Gap 1: a trailing single-metric non-finite run imputes to 0.0 and stabilizes
at the first finite pivot's confirmation; release it there. An all-non-finite
metric keeps the nonzero legacy default (1.0) and stays deferred to n, so a
dedicated trailing_stable_mask distinguishes it from dependency_mask.
- Gap 2: for combined with every component leading with a non-finite run, the
aggregate is stably zero over [0, min_c first_finite_c); release it at the
max-over-components confirmation candle (unequal run lengths must not leak),
guarded by an empirical combined_weights[:S] == 0.0 prefix-stability check.
The producer now returns a LabelWeightImputationMasks dataclass carrying both
stable masks and a stable_release_index; min bounds the stable run while max
sets the release candle, so the release index cannot be derived from the mask
alone. The non-causal path and existing single-metric leading behavior remain
bit-for-bit unchanged.
Closes #169
* refactor(quickadapter): harden causal availability release from review feedback
Address self-review findings on the #169 harmonization without changing
production behavior (bit-for-bit over a monotone battery vs the prior commit).
- Release the combined leading run via the max over the release prefix
weight_availability[: stable_release_index + 1] instead of a single lookup.
Both are equal when weight_availability is monotone (guaranteed by the zigzag
confirmation watermark), but the prefix max stays leak-free even if that
invariant is ever violated, since every contributing component confirms at or
before stable_release_index (= max_c first_finite_c for combined).
- Guard the release on idx.size == raw_idx.size so a dropped (out-of-range)
pivot, which would desync the producer-space release index from the filtered
weight_availability, conservatively defers to the frame boundary.
- Rename all_components_finite -> every_component_has_finite: the flag is true
when no component is entirely non-finite, not when every value is finite.
- Refine docstrings to state the prefix-max release and that trailing pivots
are additionally bounded by their own label availability via the max fold.
* docs(quickadapter): align LabelWeightImputationMasks docstring with prefix-max release
The dataclass docstring still described the pre-hardening single-lookup release
(weight_availability[stable_release_index]). Match the consumer/producer
docstrings: the release candle is the max over the release prefix
weight_availability[: stable_release_index + 1], where stable_release_index is
first_finite for a single metric and the max over components for combined.
* fix(quickadapter): stop causal leak of non-terminal trailing pivot weights
trailing_stable_mask marked the whole trailing non-finite run
(last_finite+1 .. n) and released it at weight_availability[first_finite]. Only
the terminal pivot (no closing swing) is provably 0.0 there: it is always the
last pivot so its trailing-boundary classification cannot flip. A non-terminal
pivot inside a longer trailing run — reachable when an interior swing metric is
non-finite (e.g. volume_rate / efficiency_ratio / volume_weighted_efficiency_
ratio on a zero-volume or flat window) — could still become interior (median,
!= 0.0) until its own later swing confirms, so releasing it at the first finite
pivot's confirmation leaked future information (under-purge).
Restrict trailing_stable_mask to the terminal pivot; a non-terminal pivot in a
length>=2 trailing run stays deferred to the frame boundary n via
dependency_mask (conservative, no leak). Behaviour is unchanged for the common
length-1 trailing run (terminal only), and for the leading, combined, and
non-causal paths. Docstrings aligned (terminal-pivot wording; stable_release_
index == -1 semantics clarified).
* docs(quickadapter): correct stable_release_index and prefix-max descriptions
Third-round review accuracy fixes (documentation only, no behaviour change):
- stable_release_index: the docstrings claimed "-1 when both stable masks are
empty", but a single metric with any finite value reports first_finite (>= 0)
even when both masks are empty (all-finite or interior-only-NaN). It is -1
only when no pivot is finite. The consumer independently gates the release on
a non-empty stable mask, so a non-negative index with empty masks is inert.
- Consumer release comment: the prefix max equals
weight_availability[stable_release_index] only under the production invariant
(zigzag confirmation watermark => monotone weight_availability), not
unconditionally; state that explicitly and note the prefix max stays leak-free
(at worst defers later) if the invariant is ever violated.
* fix(quickadapter): stop causal leak of terminal pivot-weight imputation
The terminal non-finite pivot's 0.0 imputation is not causal-prefix stable:
add_pivot backfills its swing metric once a later closing pivot arrives, so
the 0.0 becomes a nonzero interior median. Releasing it (and skipping its
Gaussian band) before the frame boundary n leaked the fact that no later
pivot occurs. Defer the terminal pivot to n via dependency_mask and remove
the now-always-empty trailing_stable plumbing (dataclass field, consumer
parameter, and caller wiring). The leading-run and combined leading release
(both provably prefix-stable) are unchanged.
Empirically verified against the prior revision: bit-for-bit availability on
all non-terminal cases; terminal cases only ever defer later (leak-free).
* refactor(quickadapter): make stable_release_index -1 when no leading run
Return stable_release_index = -1 whenever leading_stable_mask is empty
(single-metric with first_finite == 0: all-finite or interior-only-NaN),
so the field holds the invariant 'index >= 0 iff a leading run is
released' instead of relying on the consumer's mask-emptiness gate. The
consumer already gates the release on leading_stable_mask.any(), so the
availability output is bit-for-bit unchanged (verified across single,
combined, and uniform cases). Also align docstrings/terminology
(terminal vs trailing pivot) and tighten the stable_release_index docs.
The comment claimed an all-non-finite component (imputed to the nonzero
default 1.0) makes the aggregate leading run non-zero, but geometric_mean
and harmonic_mean annihilate to 0.0 when any component is 0.0 (verified),
so that rationale is false. The real reason the leading release is blocked
is that an all-non-finite component never confirms a finite weight in-frame
and thus has no first_finite release candle; the pivots defer to n
conservatively. Comment-only; no behavior change.
Remove the consumer release-prefix formula duplicated from
compute_label_weight_known_at_lookahead (single source of truth), drop the
vague 'shared' wording, and name both the producer and consumer of the
dataclass. Docstring-only; no behavior change.
Extract the duplicated "segment-ends" idiom
`np.flatnonzero(np.r_[a[1:] != a[:-1], True])` into a pure helper
`_segment_ends` and call it from its three sites: `_causal_impute_weights`,
`_compute_causal_epsilon_fill`, and `_compute_knn_pivot_sigma_availability`
(where the flipped operand order is equivalent by symmetry of `!=`).
Route the two verbatim-duplicated `fill_epsilon_baseline` `ValueError`
messages in `_compute_epsilon_floor` and `_compute_causal_epsilon_fill`
through the existing `enum_error_message` factory; the produced string is
unchanged and the scalar vs expanding baseline computations are untouched.
Purely internal deduplication; no behavior change. Bit-for-bit equivalence
of the four affected functions verified against the prior revision by
randomized fuzzing in the quickadapter-freqtrade container.
* docs(quickadapter): clarify _segment_ends run contiguity (#171)
refactor(quickadapter): consolidate LabelTransformer scaler tail and direction core (#179)
* refactor(quickadapter): consolidate LabelTransformer scaler tail and direction core
Extract the shared scaler tail of `_standardize`/`_normalize` into a static
`_apply_registered_scaler(method, registry, type_names, kind, ...)` helper
(registry lookup, ValueError on unknown method, RuntimeError on unfitted
scaler, `_apply_scaler` dispatch). Method-specific preambles (`mmad`,
`sigmoid`, `none`) and the two registries stay separate.
Fold `transform`/`inverse_transform` onto a shared direction-parametrized
`_apply_columns` core; the public wrappers keep their exact signatures
(including the unused `outlier_check` and `**kwargs`).
Behavior-preserving: no registry merge, no `_fit_*` change, no `Utils`
import (avoids the import cycle); error messages built locally and kept
byte-identical. Outputs verified bit-for-bit unchanged across all
standardization x normalization x gamma combinations, 1D/2D inputs,
multi-column, and every error path.
Closes #175
* refactor(quickadapter): tidy LabelTransformer scaler-family descriptor and naming
Address review nits on the #175 consolidation, behavior-preserving:
- Bundle the co-varying `(registry, type_names, kind)` triple into a frozen
`_ScalerFamily` descriptor with two per-family class constants
(`_STANDARDIZATION_FAMILY`, `_NORMALIZATION_FAMILY`); `_apply_registered_scaler`
now takes one `family` argument instead of three. The two registries stay
separate. Matches the file's dataclass idiom (`_ColumnState`).
- Type the error noun as `Literal["standardization", "normalization"]` instead
of `str`, consistent with the module's enum typing.
- Rename orchestrator `_apply_columns` to `_transform_columns` to pair with
`_transform_column` and reserve the `_apply_*` prefix for the static kernels.
Error strings kept byte-identical; outputs verified bit-for-bit unchanged
(container SHA-256 across all standardization x normalization x gamma combos,
1D/2D, multi-column, and every error path). No `_fit_*` change, no `Utils`
import, no instance/state field renames.
* refactor(quickadapter): harden _ScalerFamily immutability and hashing
Re-review follow-up on the #175 consolidation, behavior-preserving:
- Set `_ScalerFamily` to `eq=False` so the frozen dataclass no longer
synthesizes a `__hash__` over its mutable `registry` mapping (which would
raise `TypeError` on hash despite advertising hashability); identity
semantics are correct for the two class-level singletons.
- Type `registry` as `Mapping[str, str]` to signal read-only intent.
- Mark `_STANDARDIZATION_SCALERS`/`_NORMALIZATION_SCALERS` as `Final`, matching
the `Final` family constants that wrap them.
The two registries stay separate; no `_fit_*` change, no `Utils` import, no
instance/state field renames. Outputs verified bit-for-bit identical to the
pre-refactor baseline (container SHA-256, full method x gamma x shape x
error-path matrix).
Document why the frozen `_ScalerFamily` descriptor sets `eq=False`: the
default `eq=True` would synthesize a `__hash__` over the fields and raise
`TypeError` on the unhashable `registry` mapping. Comment only; no behavior
change (outputs verified bit-for-bit identical to the pre-refactor baseline).
The previous comment implied the synthesized __hash__ itself raises; in fact
eq=True synthesizes a __hash__ that raises TypeError only when called (the
``registry`` mapping is unhashable). Reword to state this precisely and that
eq=False keeps identity hashing. Comment only; no behavior change (outputs
verified bit-for-bit identical to the pre-refactor baseline).
* docs(quickadapter): trim _ScalerFamily eq=False comment to the invariant
The prior comment described a __hash__ TypeError that never occurs: no _ScalerFamily instance is ever hashed (only constructed as two constants and passed by value). Reduce it to the actual invariant behind eq=False (registry is an unhashable Mapping).
Behavior-preserving deduplication of numeric helpers in Utils.py, proven
bit-for-bit identical across a broad input grid (all smooth methods,
ceil/floor/round over int/np.integer/float/non-finite/invalid inputs).
- smooth: replace the four near-identical zero-phase filter branches and
the gaussian else-fallback with a table-driven dispatch
`_SMOOTHING_FILTER_SPECS: method -> (kernel, window_selector)`, mirroring
the get_ma_fn/get_price_fn `.get(key, default)` idiom. std stays derived
from the odd window for every kernel.
- ceil_to_step/floor_to_step: extract a shared `_step_round(value, step,
int_op, float_op)` core (validation, integer fast-path, finiteness guard,
float path); lru_cache stays on the public wrappers only. round_to_step
is left untouched (distinct banker's/half-step tie-breaking).
- get_odd_window/get_even_window merge (F6) intentionally skipped: they are
used as first-class callables in the smooth dispatch table and merging
would churn a stable public API and split lru_cache capacity for no gain.
Closes #174
* refactor(quickadapter): tighten smooth dispatch comment and step typing
- _SMOOTHING_FILTER_SPECS comment: it does not mirror the get_ma_fn/
get_price_fn idiom (those build a local dict per call); it is a
module-level Final table of the _*_SPECS family consumed via
.get(method, default). Correct the wording; keep the load-bearing
invariants (std stays odd-window-derived; default reproduces the
legacy gaussian/odd else-branch).
- _step_round: annotate float_op as Callable[[float], int] since only
math.ceil/math.floor are passed and both return int.
* refactor(quickadapter): rename smooth dispatch table, fix its comment
- Rename _SMOOTHING_FILTER_SPECS -> _ZERO_PHASE_FILTER_DISPATCH. The
_*_SPECS suffix is reserved for the dict[str, _ParamSpec] config
validation family (e.g. _SMOOTHING_SPECS); this is a runtime dispatch
table feeding zero_phase_filter, and the near-homonym _SMOOTHING_SPECS
was ambiguous. Module-private symbol, both occurrences updated.
- Fix its comment: it is not a _*_SPECS family member, and both the
kernel and the window parity vary per method (the invariant is std,
which stays odd-window-derived), not "only the window parity".
Review-driven, no-op (comment only, behavior bit-for-bit unchanged):
the previous "Both the kernel and the window parity vary per method"
put kernel and parity in a false parallel. The kernel varies per method,
but the window parity is odd for every method except kaiser_bessel_derived
(even). Restate precisely; keep the std-odd-derived and .get-default notes.
Review-driven, no-op (behavior bit-for-bit unchanged): the smooth
zero-phase dispatch `.get` default duplicated the gaussian entry value
verbatim. Reference the entry (_ZERO_PHASE_FILTER_DISPATCH[SMOOTHING_METHODS[1]])
instead, matching the get_ma_fn/get_price_fn `.get(key, table[default])`
idiom and single-sourcing the fallback. The default is only reachable
for methods outside the closed SmoothingMethod enum (defensive).
* docs(quickadapter): drop redundant dispatch-table paraphrase
* docs(quickadapter): drop self-documented dispatch-table header comment
Extract `_resolve_min_max` applying the finite fallback once and reuse it
from `soft_extremum_min_max`, `median_min_max`, and `skimage_min_max`.
Fold the `safe_min_pred`/`safe_max_pred` twins into a `_safe_pred` core
with two thin wrappers passing the load-bearing +/-2.0 sentinels explicitly.
The finite branch returns the candidate unchanged (no float() coercion),
preserving dtype; only the non-finite branch routes through the unchanged
safe_*_pred fallback. Outputs are bit-for-bit unchanged (verified in
quickadapter-freqtrade:latest across 38 finite/non-finite edge cases).
Add a comment on `_resolve_min_max` capturing the load-bearing invariant
surfaced during review: finite candidates are returned without float()
coercion to preserve their dtype, and the ±2.0 sentinels are the
out-of-domain bounds of normalized labels. Behavior unchanged; bit-for-bit
equivalence re-verified in quickadapter-freqtrade:latest (38 edge cases,
0 divergence).
* docs(quickadapter): scope fallback sentinel comment to default label range
Refine the `_resolve_min_max` comment surfaced in re-review: the ±2.0
sentinels sit outside the default [-1, 1] normalized label range (not
universally, since normalization="none" and custom minmax_range are
unbounded), and the no-float()-coercion note is tied to preserving the
pre-refactor bit-for-bit behavior rather than an incidental dtype.
Comment-only; behavior re-verified bit-for-bit in the container.
Wrap `float()` in RST double backticks in the `_resolve_min_max` comment
to match the class's explanatory-comment convention for code identifiers.
Comment-only; behavior re-verified bit-for-bit in the container.
Drop the historical (pre-refactor) finite-passthrough comment in
_resolve_min_max, whose behavior is self-evident from the code, and
document the surviving non-evident invariant (±2.0 fallbacks are
out-of-[-1, 1] normalized-range sentinels) at the safe_min_pred/
safe_max_pred definition site.
Defer non-finite pivot weights and their Gaussian bands until the frame boundary when legacy full-frame imputation can change across prefixes.
* refactor(quickadapter): dedupe combined label-weight selection and align imputation-mask naming
Address initial-review findings on the causal pivot-weight imputation guard
(behaviour-preserving, verified bit-for-bit against the prior tree):
- F1: extract _select_combined_metrics so _compute_combined_label_weights and
the imputation-dependency mask share one selection path; each caller applies
its own imputer, keeping the mask on the legacy full-frame _impute_weights.
- F3: rename compute_label_weight_imputation_mask ->
compute_label_weight_imputation_dependency_mask for naming coherence with
imputation_dependency_mask / _nonfinite_imputation_dependency_mask.
- F5: raise an explicit ValueError for strategy='none', mirroring
compute_label_weights instead of falling through to the generic message.
- F2: rewrite the label_weighting.strategy README sentence to fix a
garden-path reading.
* refactor(quickadapter): finish combined-weight dedup and tighten mask docstrings
- N1: extract _aggregate_imputed_metrics so _compute_combined_label_weights and
the imputation-dependency mask share the vstack/aggregate step (no recompute).
- N2: make _select_combined_metrics docstring describe the primitive generically
instead of one consumer's causal-imputer rationale.
- N3: tighten _nonfinite_imputation_dependency_mask docstring to the essential
prefix-instability rationale.
* refactor(quickadapter): dedupe weight-strategy error message and fix mask docstring
Address residual re-review nits (behaviour-preserving, error text byte-identical):
- NEW-1: extract _invalid_weight_strategy_message shared by
compute_label_weight_imputation_dependency_mask and _compute_label_weight_values
(removes the verbatim duplicated ValueError text).
- NEW-2: correct _nonfinite_imputation_dependency_mask docstring to enumerate all
prefix-unstable imputation modes (boundary default/zero, interior median).
* docs(quickadapter): drop causal-availability note from label_weighting.strategy
The sentence documented internal causal-availability behaviour (pivot deferral to
the frame boundary), not a choice or value of the strategy tunable, and used
implementation terms undefined in the README. Reverting the strategy row to main
also removes the table re-padding churn, so the PR no longer touches README.
* docs(quickadapter): consolidate cross-tunable behaviour into governing tunables
Apply a global README rule: each tunable row documents that tunable's own role;
behaviour conditioned on another tunable's value lives in the governing tunable's
row (with a short pointer instead of duplication).
- causal_mode: single home for all causal split-guard behaviour (split-option
rejections; timeseries_split gap auto-set/rejection; train_test_split fixed
purge; label-aware availability row removal incl. Zigzag confirmation,
centered-smoothing availability, k-NN bandwidth confirmation, and PR #158
non-finite imputation-dependency deferral; causal epsilon baseline).
- gap / fill_method / fill_sigma_candles / label_horizon_candles / support_policy /
continual_learning: keep role-focused text, point to causal_mode / test_size.
- Fix factual defects: optuna_hyperopt.n_jobs default (1, not CPU threads/4 which
is only a cap); gpu_vram_gb type (int > 0, floored to nearest tier, not enum);
label_smoothing.window_candles range (int >= 1, floored to 3 at runtime);
label_frequency_candles range (int [2, 10000] | auto).
* docs(quickadapter): correct causal_mode guard attribution and range constraints
- causal_mode: shuffle_after_split is only causal-gated under train_test_split;
it is rejected structurally under timeseries_split and held-out evaluation
regardless of causal_mode, so the false-mode note no longer implies all guards
are lifted (shuffle and reverse_train_test_order remain fully causal-gated).
- label_pipeline.minmax_range / feature_parameters.range: document the low < high
constraint (enforced by _RangeValidator / MinMaxScaler), matching robust_quantiles.
* docs(quickadapter): tighten the causal band-skip guard comment
Comment-only: condense the fill_radius skip-guard rationale (7 -> 4 lines) while
keeping the non-obvious causal invariant (an imputation-dependent pivot is not
skipped despite weight_avail == n, since its all-non-finite metric imputes to the
non-zero legacy default and its band must defer to the frame boundary).
* docs(quickadapter): document toggle-independent shuffle-family rejections in causal_mode
NF-A: the held-out evaluation (test_size != 0) rejects shuffle, shuffle_after_split
and reverse_train_test_order unconditionally, and timeseries_split rejects
shuffle_after_split structurally -- both independent of causal_mode. The row now
states these under clause (1) and clarifies the false branch (only the causal
split-guard rejections are lifted; the toggle-independent ones remain).
* perf(quickadapter): release leading stable imputations before frame boundary
For single-metric weight strategies, a leading non-finite run imputes to
0.0 and becomes stable once the first finite pivot's weight is confirmed.
Deferring these pivots to the frame boundary over-purged causally valid
training rows.
compute_label_weight_imputation_dependency_mask now returns a second
leading_stable_mask (subset of dependency_mask) marking that run; empty
for uniform, combined, and all-non-finite metrics. The consumer releases
those pivots at weight_availability[first_finite] (the closing pivot's
backfilled confirmation, not the leaky idx[first_finite]) under an
identity-order guard, and skips their zero-weight Gaussian bands.
combined stays deferred to the frame boundary (prefix aggregation can
still shift a leading pivot), preserving causal safety.
fix(quickadapter): isolate holdout from continual state (#156)
Cold-start HPO trials and the pre-refit selection model whenever `test_size`
enables two-stage selection, so `holdout_rmse` is an independent diagnostic;
only the final XGBoost/LightGBM refit continues from the FreqAI-supplied
previously deployed model. Other regressors ignore any prior model state.
Give the changed HPO objective a stable semantic identity
(`candidate-cold-start-v1`) stored as the Optuna study `user_attr`
`objective_identity` and in the persisted best-params envelope, so legacy
warm-state studies/params are ineligible (reset/rejected once on upgrade).
Unify the hp and label study-lifecycle versioning behind one
`_OptunaStudyMarker` dispatch (stable study names; hp always resets on
mismatch, no new tunable; label keeps `reset_label_study_on_schema_mismatch`).
Make the canonical `continual_learning=false` explicit; document the
`continual_learning` and `test_size` rows (`0` = single-stage, train_test_split
only) and harmonize strategy comment code refs to RST double backticks.
The partial-exit minimum bound previously multiplied `min_stake` by the
stoploss/rate factor `max(current_exit_rate/current_entry_rate,
1/(1-|stoploss|))` in every runmode. In backtest/hyperopt, Freqtrade already
passes the adjusted minimum it guards the remainder against, so that factor
duplicated the stoploss reserve. Apply the factor only in live/dry-run
(is_trade_runmode == TRADE_MODES = {LIVE, DRY_RUN}), where the callback supplies
min_entry_stake (stoploss 0.0) but the guard uses the larger min_exit_stake;
keep the 0.1% numerical clearance in both modes. Live/dry-run behavior is
bit-for-bit unchanged. Freqtrade remains responsible for amount precision and
final pre-order minimum validation.
Also rename the local bound to `min_remaining_position_value` (it is the minimum
acceptable remaining position value, not an exit stake).
fix(quickadapter): make causal k-NN weight availability exact (#151)
Make the causal availability of the label-WEIGHT column exact for adaptive
k-NN Gaussian-fill bandwidths. Compute production 1D k-th-neighbor distances in
O(M log M), and propagate the earliest complete confirmation group at which each
clipped k-NN sigma is provably stable, over the confirmable Zigzag suffix
geometry: a confirmed pivot needs at least five slope observations, so
successive pivots are at least six candles apart and the last future candidate
is n-6. Initial-orientation replay is handled atomically via a position-only
successor bound (last replayed pivot + 6); later groups use the exact
confirmation frontier. Floor/ceiling/interior/singleton stability is decided on
the effective rank min(k, pivot_count-1) jointly with that geometry.
Truncate causal Gaussian fills to the finite support ceil(4*fill_sigma_candles)
tracked by availability; causal_mode=false keeps the legacy dense unbounded
Gaussian bit-identical. Pure-Gaussian uniform pivot centers keep their own label
availability (fixed, constant-clipped, adaptive); only adaptive off-center bands
additionally wait for sigma availability. Thread each label column's
weighting_config to availability.
Add the first quickadapter test harness, locking the Zigzag geometry invariants
the availability proof relies on (min pivot spacing, first-future bound, k-NN
availability bounds), runnable in the freqtrade container.
fix(quickadapter): recheck weight support after feature pipeline (#150)
Label-weight support checks (min_pivot_equivalent_count,
min_positive_label_weight_fraction, min_effective_sample_size Kish ESS)
previously ran only before feature_pipeline.fit_transform, which can remove
training rows and renormalize the surviving weights below those thresholds
without re-invoking support_policy.
Recheck support post-pipeline on the surviving rows: carry the raw base and
label weight components through fit_transform as temporary object-keyed label
columns (row-filtered in lockstep with X/y/sample_weight, since DataSieve
resets indexes), pop them, restore the label schema, and re-gate via
support_policy. _sanitize_pipeline_weights fails closed on any malformed
post-pipeline weight shape (None/scalar/2-D/wrong-length). Base-only weights
carry no label support and skip the re-gate. Selection training and final
refit share the path; eval (val/test) weights still bypass support_policy but
now fail closed on shape.
No tunable, canonical default, or public API change.
fix(quickadapter): use directional PnL momentum gate (#149)
Replace the velocity/acceleration t-statistic exit gate with a deterministic
mean per-candle PnL velocity direction rule: once the take-profit target is
reached and the recent window is complete, allow the exit iff recent[-1] <
recent[0] (equivalent to strictly negative mean velocity via telescoping).
Remove the now-dead machinery: decline_quantile, t-critical/effective-df,
acceleration and zero-variance fallbacks, get_pnl_momentum,
get_trade_unrealized_pnl_history, the thresholds_calibration config specs, and
the unrealized_pnl_timeframe_minutes field. A legacy
exit_pricing.thresholds_calibration setting now warns as obsolete and ignored.
Fail-open only where the horizon is unmeasurable (missing candle date,
incomplete window, non-finite value in the selected window); complete finite
windows are deterministic (falling exits; flat or rising blocks).
fix(quickadapter): make epsilon fill causal (#148)
* fix(quickadapter): make epsilon fill causal
Derive each causal row epsilon baseline from pivot weights available with its label while preserving the global non-causal formulation.
* fix(quickadapter): resolve undefined as_dict in set_freqai_targets
Replace the unimported as_dict() call with the dominant
self.freqai_info.get("feature_parameters", {}) idiom, matching the sibling
get_causal_mode call site and the other feature_parameters lookups in the
file. Fixes the F821/NameError that aborted set_freqai_targets on every fit,
and removes the idiom divergence.
Reformat the README configuration tunables table via prettier (table
re-padding only; no cell content changed).
feat(quickadapter): make label study schema reset configurable (#147)
* feat(quickadapter): make label study schema reset configurable
Add a boolean reset_label_study_on_schema_mismatch option that preserves the current reset behavior by default and lets operators retain incompatible label studies with a warning. Avoid rewriting incompatible selection metadata when preservation is selected, and fail closed when study inspection or deletion fails.
The `true` value keeps the historical destructive reset but no longer matches
main byte-for-byte on error paths: a study inspection error (either value) or a
deletion error (under `true`) now fails closed and aborts the optimization
instead of silently recreating or reusing the study. Reword the README tunable
description accordingly and align terminology with the neighboring entries.
Say the fail-closed paths abort study creation, matching the actual mechanism:
optuna_create_study returns None before the study is created, and the caller
then skips the optimization cycle.
fix(quickadapter): preserve take-profit JSON history (#145)
* fix(quickadapter): preserve take-profit JSON history
* refactor(quickadapter): drop over-engineered take-profit history migration
Remove the history_v2 versioned key, its legacy-migration branches, the
scalar cd_type/JSON-string recovery paths and the getter write side effect.
The history custom-data row has only ever been written as a dict, so its
cd_type is always "dict" and those defensive branches are unreachable dead
code; a stored dict round-trips as a dict with list fields, and the shared
callback wrapper logs rather than silently swallows exceptions.
_get_trade_history reverts to the minimal reader keyed by the literal
"history", matching the other custom-data row keys in this file. The
take-profit JSON fix for #134 is preserved: the _TakeProfitHistoryEntry
alias keeps the list arm and safe_append_trade_take_profit_price still
accepts a two-element tuple or list with a non-bool int stage and non-bool
numeric price.
fix(quickadapter): correct and harmonize builtin caching (#165)
Remove @lru_cache from the instance method
QuickAdapterRegressorV3.optuna_samplers_by_namespace: decorating a bound
method kept self in the process-lifetime cache, pinning regressor
instances (models, studies, dataframes) and leaking memory across
re-instantiations. The value is trivially recomputable.
Harmonize the rest of the builtin caching:
- Centralize lru_cache sizes into _CACHE_MAXSIZE_SMALL/_LARGE constants,
replacing scattered 8/64/128 literals (right-sizes the oversized 64 on
_label_aux_column_name).
- Drop ineffective @lru_cache on the float-keyed static helpers
_t_statistic/_effective_df/_t_critical: continuous per-call float keys
give a near-zero hit rate while _effective_df pays O(n) tuple hashing.
- Return a read-only ndarray from _calculate_coeffs so callers cannot
corrupt the shared cached kernel.
- Document the _df_signature process_only_new_candles coupling.
* fix(quickadapter): drop epoch-anchored phase guard in PnL continuity
The (month-1) % n phase guard on the MS path assumed a fixed January
epoch, but pandas resample anchors calendar bins on the data origin, not
a fixed epoch, so it wrongly reset valid non-January-anchored multi-month
series (e.g. a Feb/May/Aug 3M grid). is_on_offset already rejects a
non-boundary stored date and stored + offset is the exact next candle for
any anchor phase, so the membership test alone is correct and symmetric
for MS and YS (resolving the MS/YS asymmetry). Also complete the
_TradeHistory key-mirror comment with the legacy timeframe-minutes key.
Route the label_smoothing method x mode check through the shared label-kind
machinery instead of a bespoke loop in the smoothing getter:
- fold the per-kind coupled-field validator into _LABEL_KIND_REGISTRY as a
third tuple element (single source of truth), with a named
CrossFieldValidatorFn type alias matching the existing ValidateParamsFn
- get_label_kind_config runs the registered validator on each resolved
per-column config; all four label-kind getters are now symmetric one-liners
- derive the bot_start wrap/causal-mode check's mode-aware method set from
SMOOTHING_METHOD_MODES, dropping the duplicate _SMOOTHING_GAUSSIAN_FILTER1D
- README: keep the per-method mode matrix in the type column, drop the
code-behavior narration from the description
* fix(quickadapter): validate exit calibration and leverage bounds at the option layer
Two config inputs bypassed the option-layer validation their siblings use:
- exit_pricing.thresholds_calibration.decline_quantile was merged raw:
documented float (0,1) but enforced nowhere (only a consumer-side guard
that raised TypeError on non-numeric input). Route it through a validated
get_exit_thresholds_calibration_config using the shared _validate_params
machinery, with DEFAULTS_EXIT_THRESHOLDS_CALIBRATION as single source of
truth (drops the duplicate class-var default). Invalid values warn and
fall back to 0.5.
- leverage() applied only the upper bound; the documented lower bound 1.0
(README: float [1.0, max_leverage]) and non-numeric guarding were missing.
Clamp to [1.0, max_leverage], falling back to proposed_leverage on
non-numeric input.
* fix(quickadapter): validate custom_protections config at the option layer
custom_protections was the last config section read ad-hoc with hard int()/
float() casts that crashed on non-numeric input, inconsistent with the
warn-and-fall-back contract every other section uses.
Add get_custom_protections_config on the shared _validate_params machinery
(new _BoolValidator for the enabled flags; nested cooldown/drawdown/stoploss
sub-dicts validated per section) with single-source DEFAULTS_*; the
protections property consumes the validated, typed config. Invalid or
non-numeric values now warn and fall back to their documented defaults
instead of raising.
* fix(quickadapter): validate fit_live_predictions_candles at the option layer
The last strategy-side config value read with a raw int() cast (protections
and startup_candle_count) crashed on non-numeric input. Route it through a
validated get_fit_live_predictions_candles (positive int, warn and fall back
to the default) on the shared _validate_params machinery; drop the now-unused
DEFAULT_FIT_LIVE_PREDICTIONS_CANDLES import.
* refactor(quickadapter): address PR review nits
- harmonize get_exit_thresholds_calibration_config to accept the parent
exit_pricing dict and deref thresholds_calibration internally, removing the
double as_dict at the call site (mirrors get_custom_protections_config)
- reject bool in _NumericValidator: bool is not a valid numeric input,
consistent with is_finite_number / leverage() / _BoolValidator
- drop the dead commented minimal_roi block referencing the removed
DEFAULT_FIT_LIVE_PREDICTIONS_CANDLES import
- align comment terminology on 'cross-field' (matches CrossFieldValidatorFn)
- warn instead of silently resetting when a config section is present but not
a mapping: new as_config_section helper applied to custom_protections
(+ cooldown/drawdown/stoploss) and exit_pricing.thresholds_calibration
- re-add default_exit_thresholds_calibration ClassVar as a compat alias to the
canonical DEFAULTS_EXIT_THRESHOLDS_CALIBRATION (public API stability)
- move the minimal_roi rationale note directly above its assignment
- Warn on non-mapping config sections by routing the section getters
(label kinds, exit_pricing, reversal_confirmation, fit_live) through
as_config_section, matching the custom_protections pattern (N-1).
- Honor the default_exit_thresholds_calibration override via an optional
overrides argument merged over the canonical defaults; user config
still wins in _validate_params (N-2).
- Resolve fit_live_predictions_candles once through the canonical
validator in the regressor so an explicit 0 floors to 100, fixing the
.iloc[-0:] whole-frame slice (N-3).
* fix(quickadapter): warn on invalid leverage before fallback
Route the configured leverage through a cached validator that logs a
harmonized warning when the value is non-numeric or a boolean before
falling back to proposed_leverage, instead of silently discarding the
user setting. The warning fires once (cached) to avoid per-call spam.
* fix(quickadapter): warn on sub-minimum leverage; cache protections
- Warn once (via the _configured_leverage cached_property) when a numeric
leverage is below the 1.0 floor before the leverage() hook clamps it;
the per-pair max_leverage ceiling is only known at entry time, so
above-ceiling values stay clamped silently.
- Promote protections to a cached_property, aligning it with the sibling
config-derived accessors and collapsing duplicate warnings on a
malformed custom_protections/freqai section to one per strategy
instance (reload re-instantiates the strategy, so the cache is fresh).
* refactor(quickadapter): drop dead FIT_LIVE_PREDICTIONS_CANDLES_DEFAULT
Both consumers were rewired to the resolved self._fit_live_predictions_candles,
leaving the ClassVar and its DEFAULT_FIT_LIVE_PREDICTIONS_CANDLES import unused.
Remove both (not a public-library API surface).
* style(quickadapter): drop redundant comments in exit-calibration getter
The as_dict coercion and defaults merge are self-explanatory; keep
comments only where the code is not clear on its own.
* fix(quickadapter): guard leverage finiteness; warn-once on fit-live warmup
- Reject non-finite leverage (NaN/Inf) via the shared is_finite_number
guard before falling back to proposed_leverage, instead of letting it
reach the clamp silently.
- Back startup_candle_count and protections with a cached
_fit_live_predictions_candles helper so an invalid fit_live_predictions_candles
warns once instead of on every access. startup_candle_count stays a plain
property so the StrategyResolver keeps protecting it from config override
(cached_property is not a property subclass).
* style(quickadapter): drop redundant _LABEL_KIND_REGISTRY comment
The tuple type (CrossFieldValidatorFn | None) and the named unpacking
(cross_field_validator) already document the third element.
* fix(quickadapter): validate exit-calibration override before use
Route the override (e.g. a subclass default_exit_thresholds_calibration)
through _validate_params against the canonical defaults so an invalid
subclass value falls back to the canonical default instead of being
trusted blindly (previously it could be returned as-is with a misleading
warning, or crash on output_type coercion). User config still wins over
the override, which still wins over the canonical default. Also restore a
concise note on the intentional silent parent coercion.
Enforce the documented [0, 1] upper bound for both reversal NATR multiplier
fractions at the configuration boundary:
- reject config values above 1 (warn and fall back per component)
- fail fast with ValueError when canonical defaults exceed max_value
- canonicalize the out-of-range component warning to the single
"<= {max_value}" form, dropping the wrong "[-inf, x]" notation
- clarify the strict min < max ordering in the README tunables table with
symmetric "(< upper bound)" / "(> lower bound)" hints
refactor(quickadapter): group related constants (#161)
Keep canonical sources contiguous with their aliases, sets, maps, and related defaults across the strategy and regressor constant blocks. Preserve every value and consumer while removing semantic-family interleaving.
fix(quickadapter): invalidate reversal caches together (#155)
Purge threshold and deviation entries for a pair before recording its new dataframe signature, and invalidate both caches when live pair-local label period or NATR multiplier state changes.
feat(quickadapter): make HPO trial seed variation configurable (#119)
Add freqai.optuna_hyperopt.vary_model_seed_by_trial (default true) to make the
per-trial model-seed variation explicit.
- true (default) preserves the historical behavior: each HPO trial adds
trial.number to the regressor model seed. false uses the same model seed for
every trial and the final fit.
- Independent from freqai.optuna_hyperopt.seed (Optuna samplers + label-candle
shuffling); wired through the canonical defaults, config template, startup
logging, and README.
- Seed NGBoost's base DecisionTreeRegressor as well: NGBoost.random_state only
seeds subsampling/validation split, not the cloned base learner, so the base
tree needs its own random_state for reproducibility.
BREAKING CHANGE: the optuna_hyperopt boolean options (enabled, continuous,
warm_start, space_reduction, vary_model_seed_by_trial) are now validated at the
option layer and raise ValueError on non-boolean values instead of relying on
Python truthiness; a config passing a non-boolean (e.g. 1 or "false") for these
keys must use real booleans.
Trigger the take-profit partial exit at freqtrade's executable current_exit_rate
instead of the entry-side current_rate, and size the partial-exit remainder so it
survives freqtrade's minimum-stake guard.
- Evaluate the take-profit trigger and the remaining position at current_exit_rate
(the rate the exit actually fills at); the previous current_rate is the entry rate
in this callback and could fire a long exit below its bid target (symmetric short).
- freqtrade passes min_entry_stake as min_stake but rejects the partial exit when the
remaining value is below the larger min_exit_stake. Size the remainder against a
proven upper bound min_exit_stake <= min_stake * max(current_exit_rate/current_entry_rate,
1/(1-|stoploss|)) (holds for both the cost- and amount-driven minimum), with a small
rounding margin, so the shrunk remainder clears the guard. Leverage cancels; amount
precision and final validation are left to freqtrade.
fix(quickadapter): sample PnL momentum per candle (#117)
Sample the take-profit declining-PnL momentum gate once per candle instead of
per callback throttle, so velocity/acceleration are computed on a
candle-uniform series.
- Window sizing: ceil(30/tf)+1 samples (>=30 min velocity span on any
timeframe), floored to 4 so both velocity and acceleration t-statistics are
computable; warn when the nominal window is floored.
- Warm-up: fail open (never block a profitable take-profit exit) until a full
momentum window is available.
- Persist a candle-date and timeframe marker with the PnL history; reset the
series on legacy-history migration, timeframe change, or a candle
discontinuity (forward gap or backward/non-monotonic date), persisting the
reset before the append re-read.
- Fail open when the last candle lacks a valid date rather than gating on a
stale series.
- Dedupe the take-profit order tag into a helper; type the trade history via a
TypedDict.
refactor(quickadapter): harmonize patterns and consolidate common logic (#126)
* refactor(quickadapter): harmonize patterns and consolidate common logic
Behavior-preserving refactor of the QuickAdapter codebase (Utils.py,
QuickAdapterV3.py, QuickAdapterRegressorV3.py). Every change verified in the
project Docker image (freqtrade + optuna/ngboost/catboost) via import smoke and
per-change golden/behavior tests; py_compile clean, ruff E,F,W baseline unchanged
or reduced, ruff format clean.
Consolidation:
- C1: single _get_validation_size() replaces 3 identical test_size None-coerce blocks (None-vs-0 distinction preserved).
- C2: _resolve_optuna_store() registry replaces 6 near-duplicate get/set_optuna_* dispatchers; hp-only/label-only asymmetry and per-accessor defaults preserved.
- C4: as_dict() helper replaces 7 dict-guard blocks across both classes.
- C5: _distance_to_reference() extracts the 4-family distance dispatch shared by _compromise_programming_scores and _topsis_scores (apply_abs flag preserves the TOPSIS-only np.abs asymmetry); golden differential over all 17 metrics is bit-identical.
- C6: _trade_natr_window() extracts the shared NATR preamble of the weighted/quantile trade-NATR methods (single-candle sentinel preserved).
- C7: _invalidate_pair_cache() replaces 2 identical per-pair cache-invalidation blocks (rebind-on-change / same-object-on-noop preserved).
- C8: _validate_step_args() shares the guard preamble of round/ceil/floor_to_step.
- C9: _pop_early_stopping_rounds() and _apply_verbosity_alias() replace repeated fit_regressor boilerplate (verbose precedence preserved).
- C10: drop dead default 2nd-args on _optuna_config[...] lookups (merge is exhaustive; constant_liar path kept).
Harmonization:
- H2/C3: fold 3 near-identical scalar validators into one _validate_scalar(predicate, constraint); exact messages preserved (subsumes H1 predicate sharing).
- H4: single canonical source for label defaults in Utils; RV3 ClassVars reference it (fixes value duplication).
- H6: enum_error_message() builder; identical-text regressor/aggregation raises routed through it.
- H7: make the intended mode="raise" explicit for label_method and scaler validation (no behavior change).
Notes:
- H5 (DEFAULT_X vs X_DEFAULT naming): the duplicated default values were removed by H4; the remaining prefix/suffix difference is a consistent module-scalar vs class-ClassVar convention, left as-is.
- H3 (magic-index TUPLE[n] -> named members) remains as a follow-up; behavior-neutral cosmetic cleanup.
* refactor(quickadapter): replace magic-index dispatch with named constants (H3)
Every fragile TUPLE[n] dispatch/comparison site (~64 across the three files) now
reads a named Final constant defined next to its tuple, extending the existing
_OPTUNA_NAMESPACES / _REGRESSOR_SPECS idiom. Reordering a literals tuple can no
longer silently break dispatch, and the trailing `# "name"` comments are removed
as redundant.
The canonical named-derivation definitions (_UNSUPPORTED_WEIGHTS_METRICS and the
LABEL_*_DEFAULT / *_DEFAULT ClassVars) keep their index form: they are the single
source that maps index -> name. Behaviour preserved by construction (each named
constant is bound to the same tuple element); verified in the project Docker image
by asserting every constant equals its tuple value and that CP/TOPSIS scoring is
unchanged.
- M1: hoist the scipy cdist kwargs in _topsis_scores and pass them into both
_distance_to_reference calls, so _prepare_distance_kwargs (and its warn-mode
validators) run once instead of twice for scipy metrics — restores the
original single-warning behavior on an invalid label_distance_p / unsupported
weights. Distance scores stay bit-identical (kwargs are deterministic).
- N1: make _validate_scalar's predicate/constraint optional; _validate_power_mean_p
now calls it with no predicate (finite-only), dropping the dead always-true
lambda + unreachable constraint string.
- N2: type _invalidate_pair_cache with a bound TypeVar so each caller's cache
type flows through instead of a bare dict.
- N3: drop 3 redundant `# "name"` comments left next to named constants.
Verified in the project Docker image: CP/TOPSIS golden differential vs the prior
commit is bit-identical across all 17 metrics; TOPSIS invalid-p warnings 2 -> 1;
validator messages unchanged; py_compile clean, ruff E,F,W <= main baseline,
ruff format clean.
- NEW-1: rebuild the invalidated cache via type(cache)(...) in _invalidate_pair_cache
so the _PairCacheT TypeVar return is sound. pyright: prior plain-dict-comprehension
reassignment reported reportAssignmentType (1 error); type(cache)(...) reports 0.
Runtime identical (type(cache) is dict for the two concrete caches).
- NEW-2: drop the 32 remaining tautological `# "name"` comments sitting next to a
named constant (e.g. `== _TRADE_LONG # "long"`), consistent with the H3 cleanup.
Index-resolving comments on the canonical definitions (e.g. `_DISTANCE_METRICS[6]
# "mahalanobis"`) are kept — they document the literal an index maps to.
Verified in the project Docker image: CP/TOPSIS bit-identical vs prior commit across
all metrics; _invalidate_pair_cache runtime unchanged; py_compile clean, ruff E,F,W
<= main baseline, ruff format clean.
Complete NEW-2 cleanup: remove 14 tautological `# "name"` comments on
multi-line comparison/closing-paren lines that the single-line strip
missed (e.g. `): # "power_mean"`, `) # "short"`), fixing the
long-removed/short-kept asymmetry. Removing the pinning comments lets
ruff format collapse the parenthesized `if (...)` forms to single lines.
Keeps index-resolving comments (`_ARRAY[n] # "x"`) and the
LABEL_METHOD_DEFAULT value-documenting comment. Comment-only + format
change: AST verified identical to prior commit; ruff E,F,W within
baseline; container import smoke passes.
refactor(quickadapter): centralize exit_pricing and reversal_confirmation validation (#125)
Fold trade_price_target_method and reversal_confirmation inline validation onto the declarative spec-table pattern (get_exit_pricing_config, get_reversal_confirmation_config); coupled min/max NATR pair kept on validate_range. Remove the vestigial default_reversal_confirmation ClassVar. Guard np.isfinite via a shared _is_finite_value helper so every numeric validation path is total (no raise on ints >= 2**64 or non-scalar input). Cache config-derived validation properties (cached_property) in the strategy and regressor. Behavior preserved for all realistic inputs.
fix(quickadapter): refit iteration aliases, test_size None docs, live holdout_rmse (#124)
* fix(quickadapter): clear regressor iteration aliases before refit
The refit set the canonical iteration parameter on a deep copy of the user
model_training_parameters without removing configured synonyms. CatBoost aborts
on duplicate iteration aliases, so a selection fit that used n_estimators,
num_boost_round, or num_trees crashed at the post-holdout refit. Purge all known
iteration aliases per regressor before setting the canonical count.
None reaches sklearn dynamic sizing only through timeseries_split; the
train_test_split path rejects any non-int/float value. Document that None
applies to timeseries_split and that train_test_split requires a float or int.
* fix(quickadapter): restore holdout_rmse on live model reload
A live or dry-run restart that reuses a cached model without invoking fit left
_holdout_rmse at the constructor inf placeholder, so fit_live_predictions
published holdout_rmse=inf until the next retraining window. Track pairs fitted
in the current session and, when none ran, recover the last finite holdout_rmse
from historic_predictions.
* fix(quickadapter): complete lightgbm iteration aliases and guard alias-map coverage
The lightgbm alias set omitted max_iter, num_round, and num_tree. LightGBM does
not reject duplicate iteration synonyms; it silently lets a leftover alias win
over n_estimators, so a config using one of those names would train the refit at
its original capacity instead of the selected count. Complete the set to every
num_iterations synonym and add an import-time guard that the alias map covers
REGRESSORS.
* fix(quickadapter): coerce holdout_rmse to numeric before finite filter
Guard the live holdout_rmse restore against a non-float historic column: coerce
with pd.to_numeric(errors=coerce) so np.isfinite cannot raise on an object dtype,
matching the defensive handling of the non-live replay branch.
* refactor(quickadapter): consolidate per-regressor metadata into RegressorSpec
Replace the hand-maintained REGRESSORS tuple, the _REFIT_ITERATION_ALIASES map
and the scattered magic-index dispatch with a single RegressorSpec source: the
_REGRESSOR_SPECS NamedTuple singleton (mirroring _OPTUNA_NAMESPACES) carries each
regressor's canonical iteration parameter, its iteration aliases and its RNG seed
parameter. REGRESSORS, DEFAULT_REGRESSOR and the by-name lookup derive from it,
and import-time guards enforce coverage of the Regressor literal and that each
canonical iteration parameter is one of its own aliases.
get_refit, fit_regressor and get_optuna_study_model_parameters now dispatch on
named specs (regressor == _REGRESSOR_SPECS.xgboost.name) instead of REGRESSORS[i];
fit_regressor's identical per-branch seed setdefault + trial increment is hoisted
into one spec-driven prelude. Behavior preserved: same REGRESSORS values/order,
same seed handling (verified identical across all five regressors), introspection
and library-specific fit logic left in their branches.
QuickAdapterRegressorV3 uses DEFAULT_REGRESSOR instead of REGRESSORS[0].
* fix(quickadapter): recover last published holdout_rmse (incl. inf) on live reload
Restoring the last FINITE historic holdout_rmse discarded an intentional inf from
a model trained with test_size=0 and could resurrect a stale finite score from an
earlier holdout-enabled configuration. Take the last published (non-null) value
instead, so the recovered metric faithfully reflects the cached model.
* docs(quickadapter): restore inner-validation 0.1 fallback note for test_size
The test_size row documented only the outer None behavior after the applicability
fix; restore that the inner validation split falls back to 0.1, and tighten the
refit clause to keep the row within the table column width.
Raise a contextual DependencyException when an integer test_size is not
smaller than the training rows left after the outer holdout, and when the
feature pipeline (SVM/DBSCAN outlier removal) empties the validation set
after transform, instead of surfacing a low-level sklearn/predict error.
* refactor(quickadapter): normalize None test_size in _add_refit_data
Match _add_validation_split: coerce a None test_size to _TEST_SIZE before
the ==0 short-circuit, so the refit gate stays consistent with the
validation gate regardless of the _TEST_SIZE value.
repositories / freqai-strategies.git / log
