fix(quickadapter): preserve zero power mean exponent in label selection
_validate_power_mean_p returns 0.0 for p=0 (a finite float accepted by
README), and the falsy fallback silently rewrote it to 1.0, turning the
documented generalized-mean p=0 (geometric) into arithmetic. Use an
explicit None check for the fallback; map-based metrics are unchanged.
Verified in the QA image (Freqtrade 2026.8): p=0 now matches
geometric_mean selection (winner 1, identical distances), while p=None
keeps the arithmetic default.
chore(basedpyright): drop resolved reward_space_analysis ExtensionArray.size diagnostic
pandas-stubs 3.0.5.260914 resolves the previously-recorded
reportAttributeAccessIssue on ExtensionArray.size at
reward_space_analysis.py:2720, so the accepted ReforceXY snapshot no
longer matches current diagnostics. Remove the stale entry to unblock
the BasedPyright (ReforceXY) required check.
fix(qa): tolerate tool version drift in snapshot comparison
Compare snapshots with basedpyrightVersion normalized out: a tool upgrade that changes no diagnostics no longer fails the check. The recorded version stays in snapshots and --write output as provenance, is still validated nonempty on both sides, still appears in the failure diff, and both versions are reported in the success message. Diagnostics, file inventory and schema stay byte-exact, so changed output still fails closed. Covers scripts/test_check_basedpyright.py (stdlib unittest): version-only drift matches, diagnostic change mismatches, empty version and missing key rejected.
fix(renovate): anchor managed version values to digit-start
Require managed version values to start with a digit in both Dockerfile ARG and compose matchStrings. Kills the false-positive class where any *_version key with a non-version scalar (e.g. latest, stable) becomes a pypi lookup. Verified no-op on the current corpus: identical 26-capture set before/after.
chore(qa): refresh BasedPyright snapshots for 1.40.1
Repin accepted snapshots for the basedpyright 1.40.0 -> 1.40.1 bump. Diagnostics are unchanged in both projects (242 quickadapter / 159 reforcexy) — only the version marker moves. Snapshots generated with the same --pull docker build as CI.
feat(docker): manage xgboost installation and version
Pin xgboost as a managed build dependency, harmonized with the existing catboost/ngboost pattern: ARG xgboost_version in quickadapter/Dockerfile, matching build arg in docker-compose.yml, and automatic updates via the existing renovate custom regex manager (pypi datasource).
The freqtrade stable_freqai base image ships xgboost 3.2.0; this pins 3.4.1 so the QA and production images track the latest release independently of base image lag. ReforceXY is untouched: it does not use xgboost. BasedPyright snapshots unchanged (242/159 diagnostics verified with the rebuilt QA images).
chore(qa): refresh BasedPyright snapshots for 1.40.0
Repin accepted snapshots for the basedpyright 1.40.0 bump (pyright 1.1.412). QuickAdapter diagnostics are unchanged; ReforceXY keeps one message-only update from the re-pulled base image (base class step() return dict value inferred as dict[str, float] instead of dict[str, Unknown]) with the same rule, location, and severity. No added or removed diagnostics. Snapshots generated with the same --pull docker build as CI.
Jérôme Benoit [Fri, 28 Aug 2026 16:18:38 +0000 (18:18 +0200)]
fix(quickadapter): gate final take-profit on retracement
Replace the elapsed-time final take-profit exit with a persisted volatility-scaled retracement, harden its state invariants, and harmonize final-exit terminology.
Jérôme Benoit [Sun, 23 Aug 2026 21:14:34 +0000 (23:14 +0200)]
chore(quickadapter): update label_weighting and label_smoothing templates
- set label_smoothing method to kaiser_bessel_derived with window_candles 6
- add commented alternative label_weighting example (amplitude strategy,
epsilon_gaussian fill, knn bandwidth)
- align commented per-label beta example with canonical default (8.0)
Ports QuickAdapter's _install_date_pred_dedup_patch verbatim to ReforceXY. The duplicate-date_pred / validate="m:1" MergeError originates in the shared FreqaiDataDrawer (freqtrade 2026.7) and affects every FreqAI model; ReforceXY runs in its own process and never imports QuickAdapter, so it needs its own identical copy.
Jérôme Benoit [Wed, 12 Aug 2026 20:54:36 +0000 (22:54 +0200)]
fix(quickadapter): guard against freqtrade duplicate date_pred regeneration (#202)
* revert(scripts): drop historic-predictions dedup tool
Reverts the squash-merged PR #201 (commit c20d23a2). A one-shot on-disk
deduplication cannot durably fix the issue: freqtrade 2026.7 regenerates the
duplicate date_pred in memory on restart. It is superseded by an in-memory
guard in the QuickAdapter model (next commit).
* fix(quickadapter): guard against freqtrade duplicate date_pred regeneration
freqtrade 2026.7 FreqaiDataDrawer.set_initial_return_values trims the new
prediction window by position (new_pred.iloc[len(common_dates):]) instead of by
date_pred value. When the persisted history overlaps the tail of the analysis
window (the normal restart geometry), the trim misaligns and the following
pd.concat re-creates duplicate date_pred rows, which the validate="m:1" merge in
attach_return_values_to_return_dataframe rejects with MergeError, blocking the
pair. A one-shot on-disk dedup cannot fix this since the duplicate is
regenerated in memory on restart.
Install an idempotent in-memory guard from the model module that wraps
set_initial_return_values, append_model_predictions and
attach_return_values_to_return_dataframe to keep date_pred unique, preferring a
real prediction over a zero/NaN placeholder and preserving NaT rows. Validated
against freqtradeorg/freqtrade:stable_freqai (2026.7) across all restart
geometries.
* fix(quickadapter): keep original row order in date_pred dedup
Sort the kept rows by their original position (_order) instead of by
date_pred (_dp). The date sort pushed interior NaT rows to the tail and could
change which rows tail(N) returns to the strategy, dropping a real candle and
injecting a placeholder. Ordering by original position preserves freqtrade
insertion order minus the removed duplicates; date_pred uniqueness is already
guaranteed by drop_duplicates.
* style(quickadapter): type the date_pred dedup wrappers
Annotate the three FreqaiDataDrawer wrappers to match the module conventions
and freqtrade signatures (data_drawer.py:286/343/413).
* docs(quickadapter): sharpen date_pred duplicate attribution in guard docstring
State that set_initial_return_values counts the overlap but trims new_pred by
position, so a crash-persisted or overlapping store breaks the contiguous
head-prefix assumption and reintroduces duplicate date_pred; drop the
inaccurate "normal restart geometry" framing.
Lead with the robust primary cause: a store already holding duplicate
date_pred rows survives set_initial_return_values unchanged (its pd.concat
preserves hist_preds) and trips the validate="m:1" merge. Demote the
positional-trim path to a secondary case and drop the inaccurate claim that it
needs non-monotonic dates: a gapped or tail-overlapping store triggers it with
strictly monotonic dates too (proven). Anchor the data_drawer.py source lines.
Move the freqtrade-bug rationale from the pure _dedupe helper (contract only:
unique non-NaT date_pred, most-informative row, NaT preserved, order kept) to
_install, where the patch rationale belongs, and condense it with source
anchors. Net -13 doc lines; matches the module terse-docstring style.
* fix(quickadapter): drop NaT date_pred and restore chronological dedup order
Two correctness defects in the date_pred dedup guard reached the validate="m:1"
merge in attach_return_values_to_return_dataframe (data_drawer.py:429-431):
- The early return only checked duplicate valid dates, so a frame with >=2 NaT
date_pred was returned unchanged; the merge then raised MergeError ("Merge
keys are not unique in right dataset") because pandas counts repeated null
keys as non-unique, even with no NaT on the left.
- Ordering the kept rows by original position (_order) left the frame
non-chronological, so the tail(N) trim in set_initial_return_values and
append_model_predictions could keep stale dates and evict recent ones.
Drop NaT date_pred rows (a NaT matches no candle -- freqtrade always assigns
real dates at set_initial_return_values, data_drawer.py:305 -- and only harms
the m:1 merge) and order survivors chronologically by normalized date_pred; the
most-informative row per timestamp is still kept. This supersedes the _order
sort introduced in f4b95a7b: removing the NaT rows makes that reorder
unnecessary and the tail chronologically correct.
Add scripts/historic_predictions_deduplicate.py to repair FreqAI
historic_predictions.pkl stores that a crash-recovery backfill can leave with
duplicate date_pred rows, which break the prediction merge for that pair.
- Keeps the most informative row per date_pred (informative cells, then non-null
count, then most-recent write); deduplicates only non-NaT keys.
- Dry-run by default; --apply writes atomically (temp + fsync + os.replace,
preserving mode and best-effort owner) after copying the original aside as
.original-<stamp>. Exits 1 when any target is skipped as unreadable.
- Repairs the sibling historic_predictions.backup.pkl as well.
- Portable single-file tool with a bundled test suite run via uv.
Jérôme Benoit [Wed, 5 Aug 2026 11:46:54 +0000 (13:46 +0200)]
style(optuna): apply ruff format to best-params I/O
Reflow the lock helper, save fchown block and legacy resolver wrapping
to satisfy ruff format at the default 88-column width, matching the rest
of both files (main was format-clean). Whitespace and redundant grouping
parentheses only; the AST is unchanged.
Jérôme Benoit [Tue, 4 Aug 2026 23:16:21 +0000 (01:16 +0200)]
fix(optuna): harden best-params I/O against deployment failure modes
Address three P2 review findings; the same latent behaviors were ported
from QuickAdapter to ReforceXY, so all fixes are applied to both.
- Cross-uid save (C2): make owner preservation best-effort. A non-root
process on a bind mount with a differently owned existing file lacks
CAP_CHOWN; the fchown now swallows PermissionError (logged at debug)
so the atomic write still completes, as the previous in-place write
did. fchmod and the successful-chown path are unchanged.
- Read-only mount (C3): a shared (read) lock no longer forces O_CREAT.
On a read-only mount where the lock file is absent, the load reads
without a lock; os.replace atomicity guarantees a consistent read.
Exclusive locks keep O_CREAT and still fail closed on read-only
filesystems. S_ISREG/O_NOFOLLOW/O_NONBLOCK guards are preserved.
- Legacy warm-start (C1): a base-only legacy best-params file is now
loaded when exactly one configured pair maps to that base (read in
place, no rename, read-only safe). Ambiguous cases (more than one pair
sharing a base) still warn and return None, preserving #180 safety.
The configured pair list is threaded into the QuickAdapter module-level
loader; a legacy payload still passes the same validation.
Mirror the rationale already documented on the shared QuickAdapter
helper: _quarantine_path appends the tag and timestamp after the
complete filename (extension included) so live-artefact globs never
match quarantined files. Docstring only; no behavior change.
The best-params quarantine in Utils re-inlined the timestamp + tie-break
path computation that the journal quarantine already implemented in
QuickAdapterRegressorV3, duplicating the algorithm across the two
modules.
- Add a shared, parameterized _optuna_quarantine_path(path, now, *, tag,
limit) in Utils (the module both call sites can import).
- Delegate from _quarantine_corrupt_optuna_best_params (Utils) and from
_optuna_quarantine_journal (regressor); each keeps its own domain
constants (journal vs best-params), logging and rename handling, so
quarantine names, log output and error semantics are unchanged.
- Remove the regressor's private _optuna_quarantine_path copy.
Review follow-up (DRY, single-source-of-truth). The best-params
quarantine re-inlined the timestamp + tie-break path computation already
provided by _quarantine_path, and duplicated the journal quarantine
tag/limit constants with identical values.
- Reuse _quarantine_path from _quarantine_corrupt_best_trial_params;
the wrapper keeps its own pair-prefixed logging and rename error
handling, so both call sites' log output and error semantics are
unchanged.
- Collapse _JOURNAL_QUARANTINE_TAG/_BEST_PARAMS_QUARANTINE_TAG and their
tie-break limits into neutral shared _QUARANTINE_TAG /
_QUARANTINE_TIE_BREAK_LIMIT (same values: "corrupt" / 99); generalize
the _quarantine_path parameter name journal_path -> path.
- Regroup the constants so the _JOURNAL_* block stays contiguous, with
the shared quarantine constants after it.
Port the QuickAdapter Optuna best-params I/O hardening to ReforceXY's
save_best_trial_params / load_best_trial_params:
- Encode the complete pair identity in the filename via _sanitize_pair
instead of the base currency only, so pairs sharing a base (e.g.
BTC/USDT and BTC/USDC) no longer collide on one file.
- Warn and ignore a base-only legacy best-params file that shadows the
pair-safe path.
- Write atomically (temp file + fsync + os.replace), preserving the
previous file owner and mode.
- Serialize reads and writes with a flock lock file opened
O_NOFOLLOW|O_NONBLOCK|O_CLOEXEC and guarded by an S_ISREG check.
- Fail closed when the live path is a symlink.
- Quarantine a corrupt best-params file on load and return None instead
of raising, mirroring the existing journal quarantine.
Jérôme Benoit [Sun, 2 Aug 2026 18:39:14 +0000 (20:39 +0200)]
fix(reforcexy): give jwt_secret_key and ws_token self-documenting placeholders
Match the QuickAdapter template: empty and placeholder are equally inert while
the API server is disabled, but the placeholder signals it must be replaced
before enabling the API.
Jérôme Benoit [Sun, 2 Aug 2026 18:35:43 +0000 (20:35 +0200)]
fix(reforcexy): reinstall matplotlib and bind API to localhost
The recent stable_freqairl image no longer ships matplotlib, so ReforceXY.py
fails to import ('No module named matplotlib'). Pin matplotlib==3.11.1 (cp311-cp314
wheels exist, so it installs without a compiler) via the Dockerfile build arg.
Also bind the published API port to 127.0.0.1 by default, matching the
QuickAdapter hardening, so the REST API is not exposed on all interfaces.
Jérôme Benoit [Sun, 2 Aug 2026 00:38:36 +0000 (02:38 +0200)]
fix(quickadapter): give ws_token a self-documenting placeholder like jwt_secret_key
Makes the README 'public placeholders that provide no protection until changed'
note accurate for the WebSocket token too; empty and placeholder are equally
inert while the API server is disabled.
Jérôme Benoit [Sun, 2 Aug 2026 00:21:51 +0000 (02:21 +0200)]
docs(quickadapter): trim README tunable cells to user-facing contracts
Drop internal availability-composition from the causal_mode cell, the
ValueError type from support_policy, and legacy/historical framing from
fill_bandwidth/reset_label_study/vary_model_seed; expand NATR on first use;
unify 'Quick start'; remove the profitability aside.
Jérôme Benoit [Sat, 1 Aug 2026 20:53:46 +0000 (22:53 +0200)]
fix(quickadapter): support CatBoost continual learning (#196)
* fix(quickadapter): support CatBoost continual learning
* fix(quickadapter): drop CatBoost init_model on GPU task_type
CatBoost init_model training continuation is CPU-only and raises
"Training continuation for GPU is not yet supported" on GPU. Passing it
unconditionally crashed GPU CatBoost retrains under continual_learning.
Drop init_model on the GPU path so it cold-starts instead.
* style(quickadapter): realign README table cells after continual-learning doc edits
refactor(quickadapter): route inline enum error messages through enum_error_message (#194)
* refactor(quickadapter): relocate enum_error_message to EnumErrors module
Extract the canonical enum validation error formatter into a new
dependency-free module (strategies/EnumErrors.py) and re-export it from
Utils. This lets LabelTransformer route through the same helper without
importing Utils (which would create an import cycle, since Utils imports
from LabelTransformer).
Behavior-preserving: the helper is moved verbatim; the public import path
'from Utils import enum_error_message' is preserved via re-export.
* refactor(quickadapter): route Utils enum errors through helper
Replace 5 inline 'Invalid X value ...: supported values are ...' error
constructions in Utils with enum_error_message calls (window type,
fill_bandwidth, fill_method, and two regressor sites). Byte-identical.
* refactor(quickadapter): route LabelTransformer enum errors through helper
Import enum_error_message from the dependency-free EnumErrors module
(LabelTransformer cannot import Utils without creating an import cycle)
and route the 3 inline enum error constructions (scaler family kind,
standardization, normalization). Byte-identical.
* refactor(quickadapter): route strategy enum errors through helper
Add enum_error_message to the existing 'from Utils import' block and
route the 4 inline enum error constructions in QuickAdapterV3
(trade_price_target_method, interpolation_direction, side, trading_mode).
Byte-identical.
* refactor(quickadapter): route regressor enum errors through helper
Route the 11 canonical inline enum error constructions plus 3 that keep
byte-identical output via explicit sequence wrapping:
- data_split_parameters.method, selection_method (x2), skimage threshold
method, trial_selection_method, cluster_method, aggregation,
label_method, optuna storage_backend, optuna sampler, namespace
- namespace single-value sites wrap (_OPTUNA_NAMESPACES.label,) so join
yields the same string; optuna namespace-sampler wraps tuple(samplers)
(frozenset -> Sequence[str]) preserving the existing join order.
Byte-identical.
* fix(quickadapter): unify divergent enum error wording via helper
Route the two enum error sites that did NOT include the word "value"
through enum_error_message, aligning them with the canonical format.
INTENTIONAL, observable message change (NOT byte-identical):
- QuickAdapterRegressorV3._validate_enum_value: 'Invalid {ctx} {value!r}:'
-> 'Invalid {ctx} value {value!r}:'. The message is reused by the
logger.warning at the same helper, so the change affects both the
raised ValueError and the warning log across its 6 callers.
- Utils.get_ngboost_dist dist_name: 'Invalid dist_name {v!r}:' ->
'Invalid dist_name value {v!r}:' (dict_keys wrapped in tuple() to
satisfy Sequence[str]; join output unchanged).
No test asserts these strings (quickadapter has no test suite).
Glue the first-party EnumErrors import directly to the preceding import
block in LabelTransformer, matching the convention already used in
Utils.py and QuickAdapterV3.py (no blank line separating first-party
from third-party imports). No behavior change.
* docs(quickadapter): scope EnumErrors docstring to the canonical form
The module owns the canonical 'Invalid <ctx> value <value>: supported
values are <options>' message; messages with custom prefix/infix/suffix
are built inline at their call sites. Avoids over-claiming a single
source of truth for every enum error string.
Share the combined `select -> impute -> aggregate` pipeline between the value
path (`_compute_combined_label_weights`) and the dependency-mask path
(`compute_label_weight_imputation_dependency_mask`) via a single
`_compute_combined_label_weight_pipeline` helper returning a
`_CombinedWeightPipeline`. The mask branch now derives the per-component and
aggregate non-finite masks and the leading-stable release (#169) from the
shared result instead of re-running selection, imputation, and aggregation.
Behavior is bit-for-bit identical for `compute_label_weights` and
`compute_label_weight_imputation_dependency_mask` across all strategies,
aggregations, fill methods, and causal/non-causal modes, including raised
exception type, message, and order. The two imputers (`_impute_weights` and
`_causal_impute_weights`) stay distinct; the epsilon path keeps its causal
imputer through the shared helper.
fix(quickadapter): fail closed on unmeasurable entry history (#193)
* fix(quickadapter): fail closed on unmeasurable entry history
Reject reversal entries when configured historical confirmation is non-finite, while preserving a valid current-candle exit result for exposure reduction.
* docs(quickadapter): drop internal fallback note from lookback tunable
The reversal_confirmation.lookback_period_candles row documents user-facing
usage; the non-finite historical confirmation fallback is an internal
implementation detail already described in reversal_confirmed's docstring.
Keep the tunable description terse and avoid duplicating internals per the
repository documentation conventions.
* refactor(quickadapter): reuse shared numeric validator for Optuna config
Route the Optuna numeric configuration validation through a new public
Utils.require_numeric() helper that wraps the canonical _NumericValidator,
instead of reimplementing bool/int/range checks inline. This removes the
duplicated validation logic, harmonizes error messages with the rest of
the module, and drops the unreachable math.isfinite() branch on
space_fraction (non-finite values already fail the range check).
Also align the README seed range notation (int [0, 4294967295]) with the
existing int [2, 10000] style and the emitted error message.
Behavior-preserving: raise/pass outcomes are identical to the previous
implementation across the full option matrix, except unbounded integer
options with values >= 2**64 (n_trials, timeout, n_startup_trials) are now
rejected via the shared finiteness contract (values <= 2**63 unchanged).
Keep exact built-in type semantics when using the shared helper and treat arbitrary-precision Python integers as finite.
* refactor(quickadapter): reuse shared boolean validator for Optuna config
Route the Optuna boolean option validation through a new public
Utils.require_bool() helper that wraps the canonical _BoolValidator,
mirroring require_numeric(). This removes the last inline validation in
_optuna_config so both boolean and numeric checks now share the framework
validators, closing the residual duplication.
Behavior-preserving: raise/pass outcomes are identical to the previous
inline check across all boolean options (True/False accepted, every other
type rejected).
* docs(quickadapter): justify the int short-circuit in _is_finite_value
The comment stated a Python language truism (integers are arbitrary-precision
and finite) instead of explaining why the branch exists. Reword it to document
the load-bearing rationale: np.isfinite raises TypeError/OverflowError on Python
ints >= 2**64, so routing them through it would misclassify finite values as
non-finite. Comment-only change; no behavior change.
fix(quickadapter): prevent best-params lock hang on FIFO paths (#191)
Open the best-params lock with O_NONBLOCK so a pre-existing FIFO at
.optuna-best-params.lock no longer blocks the O_RDONLY shared-load open
until a writer appears (O_NOFOLLOW rejects symlinks but not FIFOs),
hanging strategy startup before the S_ISREG guard can reject the
non-regular file. The flag is inert on regular files and does not affect
the subsequent blocking flock() (governed by LOCK_NB, not the fd flag).
Addresses the unresolved P2 review thread on merged PR #185.
Harmonize the lone bare "outer window" in the backtest evaluation protocol nested-HPO step with the "outer prediction window" term used elsewhere in the section, removing an ambiguous synonym for the rolling-origin evaluation window (distinct from the "outer holdout").
* docs(quickadapter): harmonize backtest-protocol terminology and qualify drawdown bootstrap
- clarify the row-wise known_at_lookahead maximum spans label and weight availability values
- use code font consistently for continual_learning
- align the label-candle shuffling seed role with the configuration tunables table
- name log returns explicitly before the positivity requirement
- unify "latest retained information time" across the purge step
- note the maximum-drawdown bootstrap bound is approximate for a non-smooth path-dependent functional
* docs(quickadapter): gate step 8 effect-size bootstrap on stationarity credibility
Make the stationary-bootstrap applicability in step 8 symmetric with the
Reality Check/SPA gate in step 9: declare the result inconclusive when the
paired series' stationarity and local-dependence assumptions are not credible
under a fixed or moving-window scheme. Addresses the outstanding PR #154 review
comment on the step 8 vs step 9 stationarity asymmetry.
* docs(quickadapter): harmonize primary-metric naming and fix spelling
- name the step 9 promotion bound the "net-log-return effect", consistent with
the step 7-8 net-log-return inference and the positive-is-favorable effect
terminology (removes the one-off "net-growth" synonym for the same estimand)
- correct "relabelled" to "relabeled" for US-English consistency
Align the Optuna best-params quarantine with the established journal
quarantine sibling (_optuna_quarantine_journal): source the artifact tag
from a new _OPTUNA_BEST_PARAMS_QUARANTINE_TAG constant and use "corrupt"
consistently in the docstring and log message (the function name and
.corrupt- artifact suffix already used "corrupt"; only the docstring and
warning diverged as "malformed"). Add a docstring to
_reject_optuna_best_params_symlink for parity with its sibling helpers.
On-disk artifacts and load results are byte-identical; only the log
wording and tag source change.
Consolidate the three divergent unlimited max_open_trades predicates into a
single _is_unlimited_open_trades helper, honoring both the raw -1 sentinel and
its runtime-normalized float("inf") form (matching Freqtrade's own
config_validation idiom).
Behavior is preserved for every schema-legal max_open_trades value (config
schema minimum is -1): the sole formal divergence is that max_open_trades < -1,
already rejected by Freqtrade at config load, is no longer treated as unlimited
by max_open_trades_per_side. Also makes the confirm_trade_entry guard reject
the +inf form explicitly instead of relying on IEEE finite >= inf semantics.
Rename _is_unlimited_open_trades to _is_unlimited_max_open_trades (and the
local flag) for terminological precision with max_open_trades and
max_open_trades_per_side, and widen its parameter annotation to int | float to
match the is_trade_duration_valid precedent. No behavior change.
fix(quickadapter): make best params pair-safe (#183)
* fix(quickadapter): make best params pair-safe
* style(quickadapter): order pair_to_filename import per isort
Place the freqtrade.misc import after the straight third-party imports so the block satisfies ruff's isort ordering (I001), which the pair-safe best-params change had regressed. No behavior change.
* docs(quickadapter): drop internal best-params filename details from warm_start
The warm_start tunable description exposed the on-disk best-params filename convention and legacy base-only cold-start handling: internal implementation detail with no bearing on setting the tunable (AGENTS.md: omit internal details unless necessary for usage). Revert the row to its pre-change wording; the pair-safe fix needs no tunable-doc change.
perf(quickadapter): harmonize causal availability for provably-stable zero-weight imputations (#186)
* perf(quickadapter): harmonize causal availability for provably-stable zero-weight imputations
Release trailing single-metric and combined leading non-finite runs whose
imputation is provably fixed at 0.0 under the causal prefix at their true
stabilization candle instead of the frame boundary, matching the existing
single-metric leading treatment (#158).
- Gap 1: a trailing single-metric non-finite run imputes to 0.0 and stabilizes
at the first finite pivot's confirmation; release it there. An all-non-finite
metric keeps the nonzero legacy default (1.0) and stays deferred to n, so a
dedicated trailing_stable_mask distinguishes it from dependency_mask.
- Gap 2: for combined with every component leading with a non-finite run, the
aggregate is stably zero over [0, min_c first_finite_c); release it at the
max-over-components confirmation candle (unequal run lengths must not leak),
guarded by an empirical combined_weights[:S] == 0.0 prefix-stability check.
The producer now returns a LabelWeightImputationMasks dataclass carrying both
stable masks and a stable_release_index; min bounds the stable run while max
sets the release candle, so the release index cannot be derived from the mask
alone. The non-causal path and existing single-metric leading behavior remain
bit-for-bit unchanged.
Closes #169
* refactor(quickadapter): harden causal availability release from review feedback
Address self-review findings on the #169 harmonization without changing
production behavior (bit-for-bit over a monotone battery vs the prior commit).
- Release the combined leading run via the max over the release prefix
weight_availability[: stable_release_index + 1] instead of a single lookup.
Both are equal when weight_availability is monotone (guaranteed by the zigzag
confirmation watermark), but the prefix max stays leak-free even if that
invariant is ever violated, since every contributing component confirms at or
before stable_release_index (= max_c first_finite_c for combined).
- Guard the release on idx.size == raw_idx.size so a dropped (out-of-range)
pivot, which would desync the producer-space release index from the filtered
weight_availability, conservatively defers to the frame boundary.
- Rename all_components_finite -> every_component_has_finite: the flag is true
when no component is entirely non-finite, not when every value is finite.
- Refine docstrings to state the prefix-max release and that trailing pivots
are additionally bounded by their own label availability via the max fold.
* docs(quickadapter): align LabelWeightImputationMasks docstring with prefix-max release
The dataclass docstring still described the pre-hardening single-lookup release
(weight_availability[stable_release_index]). Match the consumer/producer
docstrings: the release candle is the max over the release prefix
weight_availability[: stable_release_index + 1], where stable_release_index is
first_finite for a single metric and the max over components for combined.
* fix(quickadapter): stop causal leak of non-terminal trailing pivot weights
trailing_stable_mask marked the whole trailing non-finite run
(last_finite+1 .. n) and released it at weight_availability[first_finite]. Only
the terminal pivot (no closing swing) is provably 0.0 there: it is always the
last pivot so its trailing-boundary classification cannot flip. A non-terminal
pivot inside a longer trailing run — reachable when an interior swing metric is
non-finite (e.g. volume_rate / efficiency_ratio / volume_weighted_efficiency_
ratio on a zero-volume or flat window) — could still become interior (median,
!= 0.0) until its own later swing confirms, so releasing it at the first finite
pivot's confirmation leaked future information (under-purge).
Restrict trailing_stable_mask to the terminal pivot; a non-terminal pivot in a
length>=2 trailing run stays deferred to the frame boundary n via
dependency_mask (conservative, no leak). Behaviour is unchanged for the common
length-1 trailing run (terminal only), and for the leading, combined, and
non-causal paths. Docstrings aligned (terminal-pivot wording; stable_release_
index == -1 semantics clarified).
* docs(quickadapter): correct stable_release_index and prefix-max descriptions
Third-round review accuracy fixes (documentation only, no behaviour change):
- stable_release_index: the docstrings claimed "-1 when both stable masks are
empty", but a single metric with any finite value reports first_finite (>= 0)
even when both masks are empty (all-finite or interior-only-NaN). It is -1
only when no pivot is finite. The consumer independently gates the release on
a non-empty stable mask, so a non-negative index with empty masks is inert.
- Consumer release comment: the prefix max equals
weight_availability[stable_release_index] only under the production invariant
(zigzag confirmation watermark => monotone weight_availability), not
unconditionally; state that explicitly and note the prefix max stays leak-free
(at worst defers later) if the invariant is ever violated.
* fix(quickadapter): stop causal leak of terminal pivot-weight imputation
The terminal non-finite pivot's 0.0 imputation is not causal-prefix stable:
add_pivot backfills its swing metric once a later closing pivot arrives, so
the 0.0 becomes a nonzero interior median. Releasing it (and skipping its
Gaussian band) before the frame boundary n leaked the fact that no later
pivot occurs. Defer the terminal pivot to n via dependency_mask and remove
the now-always-empty trailing_stable plumbing (dataclass field, consumer
parameter, and caller wiring). The leading-run and combined leading release
(both provably prefix-stable) are unchanged.
Empirically verified against the prior revision: bit-for-bit availability on
all non-terminal cases; terminal cases only ever defer later (leak-free).
* refactor(quickadapter): make stable_release_index -1 when no leading run
Return stable_release_index = -1 whenever leading_stable_mask is empty
(single-metric with first_finite == 0: all-finite or interior-only-NaN),
so the field holds the invariant 'index >= 0 iff a leading run is
released' instead of relying on the consumer's mask-emptiness gate. The
consumer already gates the release on leading_stable_mask.any(), so the
availability output is bit-for-bit unchanged (verified across single,
combined, and uniform cases). Also align docstrings/terminology
(terminal vs trailing pivot) and tighten the stable_release_index docs.
The comment claimed an all-non-finite component (imputed to the nonzero
default 1.0) makes the aggregate leading run non-zero, but geometric_mean
and harmonic_mean annihilate to 0.0 when any component is 0.0 (verified),
so that rationale is false. The real reason the leading release is blocked
is that an all-non-finite component never confirms a finite weight in-frame
and thus has no first_finite release candle; the pivots defer to n
conservatively. Comment-only; no behavior change.
Remove the consumer release-prefix formula duplicated from
compute_label_weight_known_at_lookahead (single source of truth), drop the
vague 'shared' wording, and name both the producer and consumer of the
dataclass. Docstring-only; no behavior change.
Extract the duplicated "segment-ends" idiom
`np.flatnonzero(np.r_[a[1:] != a[:-1], True])` into a pure helper
`_segment_ends` and call it from its three sites: `_causal_impute_weights`,
`_compute_causal_epsilon_fill`, and `_compute_knn_pivot_sigma_availability`
(where the flipped operand order is equivalent by symmetry of `!=`).
Route the two verbatim-duplicated `fill_epsilon_baseline` `ValueError`
messages in `_compute_epsilon_floor` and `_compute_causal_epsilon_fill`
through the existing `enum_error_message` factory; the produced string is
unchanged and the scalar vs expanding baseline computations are untouched.
Purely internal deduplication; no behavior change. Bit-for-bit equivalence
of the four affected functions verified against the prior revision by
randomized fuzzing in the quickadapter-freqtrade container.
* docs(quickadapter): clarify _segment_ends run contiguity (#171)
repositories / freqai-strategies.git / log
