Run multiple live configurations in parallel #3
Labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
pregno/daytrading#3
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Run Multiple Live Configurations in Parallel Implementation Plan
Goal: add a
live-manycommand that runs multiple paper/live config files concurrently in one process, with per-entry cash caps, isolated runtime artifacts, shared trading-client pooling, graceful shutdown, and a simple operational dashboard.Global constraints:
cash_capmeans maximum notional value per entry order.cash_capis required formode: paperandmode: live; it is not required formode: backtest.live-manymust always namespacejournal_path,error_log_path, andhalt_filefilenames by bot label while preserving directories.-.live-manyrequires at least two--configvalues.paperandliveconfigs are allowed; existingI_UNDERSTAND_LIVE_TRADING=yesguard remains for real live configs.Task 1: Add
cash_capconfig validationFiles:
pivotbot/config.pyconfig.example.yamlconfig.btc.yamltests/test_config.pySteps:
tests/test_config.pyprovingcash_caploads for paper configs, is required for paper/live configs, is not required for backtest configs, and rejects non-positive values.python -m pytest tests/test_config.py -vand confirm the new tests fail before implementation.cash_cap: float | None = NonetoConfiginpivotbot/config.py.load_config(), aftercfg = Config(**raw)and existing enum validation, raiseValueError("cash_cap is required when mode is paper or live")whencfg.mode in {"paper", "live"}andcfg.cash_cap is None.load_config(), raise a clearValueError("cash_cap must be positive")whencfg.cash_cap is not None and cfg.cash_cap <= 0.mode: liveandI_UNDERSTAND_LIVE_TRADING=yes.cash_cap: 10000plus a comment toconfig.example.yamlandconfig.btc.yamlexplaining it is the max notional value per entry order and must be adjusted by the user.python -m pytest tests/test_config.py -vand confirm it passes.Task 2: Apply
cash_capto live broker order sizingFiles:
pivotbot/broker.pytests/test_alpaca_broker.pySteps:
tests/test_alpaca_broker.pyfor stock order quantity capped bycash_cap / entry_price, crypto quantity capped bycash_cap / entry_price, cash cap composing with buying-power cap by taking the smaller quantity, stock whole-share flooring, and crypto 6-decimal rounding.python -m pytest tests/test_alpaca_broker.py -vand confirm the new tests fail before implementation.AlpacaBrokerhelper such as_cap_to_cash_cap(self, qty: float, ref_price: float) -> floatthat returnsqtyunchanged whencfg.cash_cap is Noneorref_price <= 0, otherwise returnsmin(qty, cfg.cash_cap / ref_price).AlpacaBroker.submit_entry(), apply both_cap_to_buying_power()and_cap_to_cash_cap()before stock flooring or crypto rounding.python -m pytest tests/test_alpaca_broker.py -vand confirm it passes.Task 3: Extract reusable live-runner construction
Files:
pivotbot/__main__.pySteps:
pivotbot/__main__.py, for examplebuild_live_runner(cfg, *, quiet: bool = False, trading_client=None, reporter=None) -> LiveRunner, that contains the current object wiring fromcmd_live().AlpacaBroker(cfg, trading_client=trading_client), useAlwaysOpen()for crypto andAlpacaClock(broker.trading_client)for stocks, constructLiveBarFeed(cfg),Journal(cfg.journal_path), andErrorLog(cfg.error_log_path), and default toQuietReporter()whenquietis true orConsoleReporter()when false unless an explicit reporter is provided.cmd_live()to load config and call the helper, thenrunner.run().python -m pytest tests/test_cli.py tests/test_live_runner.py -vand confirm existing behavior still passes.Task 4: Add label and artifact namespacing helpers
Files:
pivotbot/live_many.pytests/test_live_many.pySteps:
tests/test_live_many.pyfor label derivation:config.spy.yaml -> config-spy,config.btc.yaml -> config-btc,configs/a/config.yaml -> configs-a-config, andconfigs/b/config.yaml -> configs-b-configwhen relative to the current working directory.pivotbot.db -> pivotbot.<label>.db,logs/pivotbot.db -> logs/pivotbot.<label>.db,errors.log -> errors.<label>.log, andrun/HALT -> run/HALT.<label>.python -m pytest tests/test_live_many.py -vand confirm the tests fail before implementation.derive_label(config_path: str | Path, cwd: str | Path = Path.cwd()) -> strinpivotbot/live_many.py.ensure_unique_labels(labels: Iterable[str]) -> Nonethat raisesValueErroron duplicates.namespace_path(path: str | Path, label: str) -> strthat inserts.<label>before a file extension when one exists and appends.<label>when no extension exists.isolate_runtime_paths(cfg: Config, label: str) -> Configusingdataclasses.replace()to updatejournal_path,error_log_path, andhalt_file.python -m pytest tests/test_live_many.py -vand confirm it passes.Task 5: Add
live-manyCLI parsingFiles:
pivotbot/__main__.pypivotbot/live_many.pytests/test_cli.pySteps:
tests/test_cli.pyprovinglive-manyaccepts repeated--config, supports--quiet, and rejects fewer than two configs through parser validation or command-level validation.python -m pytest tests/test_cli.py -vand confirm the new tests fail before implementation.live-manysubparser inbuild_parser()with--configusingaction="append",required=True, and--quietusingaction="store_true".cmd_live_many(args)inpivotbot/__main__.pythat delegates topivotbot.live_many.run_live_many_command(args.config, quiet=args.quiet).run_live_many_command(), fail fast withSystemExit("live-many requires at least two --config values")when fewer than two configs are provided.python -m pytest tests/test_cli.py -vand confirm it passes.Task 6: Add trading-client pooling
Files:
pivotbot/live_many.pytests/test_live_many.pySteps:
(mode, api_key, api_secret): two configs with the same key reuse one client, different modes use different clients, and secrets are not printed or exposed by helper output.python -m pytest tests/test_live_many.py -vand confirm the new tests fail before implementation.TradingClientPool, that accepts an optional factory for tests and returns a client for each(mode, api_key, api_secret)key.alpaca.trading.client.TradingClient(api_key, api_secret, paper=(mode != "live")).LiveBarFeedinstances.python -m pytest tests/test_live_many.py -vand confirm it passes.Task 7: Make live running stop-aware without changing single
livebehaviorFiles:
pivotbot/runner.pytests/test_live_runner.pySteps:
python -m pytest tests/test_live_runner.py -vand confirm the new tests fail before implementation.LiveRunner, for examplerun_until_stopped(stop_event, sleep_seconds: int = 60), that callsreporter.startup(), logs the same start event asrun(), repeatedly calls_run_once(datetime.now(timezone.utc)), and waits usingstop_event.wait(sleep_seconds)so shutdown is interruptible.run()to preserve current behavior, either by keeping its existing infinite loop or by delegating to the new method with an event that is never set.cfg.halt_fileand does not callbroker.close_all().python -m pytest tests/test_live_runner.py -vand confirm it passes.Task 8: Implement dashboard state and reporter adapter
Files:
pivotbot/live_many.pytests/test_live_many.pySteps:
python -m pytest tests/test_live_many.py -vand confirm the new tests fail before implementation.BotStatusdataclass with fields:label,symbol,mode,status,last_pulse,consecutive_errors, andlast_update.BotStatusper label and marks itself dirty on state changes.LiveRunnerreporters:startup(cfg),day_header(day, symbol, levels),bar(char), andmarket_closed(); the adapter updates the dashboard instead of writing raw pulse output.quiet=Truebehavior that updates state but suppresses dashboard rendering.python -m pytest tests/test_live_many.py -vand confirm it passes.Task 9: Implement threaded
live-manysupervisorFiles:
pivotbot/live_many.pypivotbot/__main__.pyif helper signatures need adjustmenttests/test_live_many.pySteps:
python -m pytest tests/test_live_many.py -vand confirm the new tests fail before implementation.run_live_many_command(config_paths: list[str], *, quiet: bool = False) -> Noneto validate count, load configs, derive unique labels, isolate runtime paths, create the dashboard, create the trading-client pool, build one runner per config, and start one thread per runner.running,stopping,stopped, orfailedas appropriate.KeyboardInterrupt, set the shared stop event, wait briefly for threads to exit, render a final status if not quiet, and return without writing halt files.python -m pytest tests/test_live_many.py -vand confirm it passes.Task 10: Documentation and integration verification
Files:
README.mddocker-compose.ymlonly if documentation requires an example command; do not change default compose behavior unless necessary.Steps:
README.mdwith a shortlive-manysection showing repeated--config, explainingcash_cap, namespaced artifacts, dashboard behavior,--quiet, and graceful Ctrl-C shutdown.live-manyis for paper/live configs only, not backtest/sweep parallelization.python -m pytest -vand confirm the full test suite passes.python -m pivotbot live-many --helpand confirm the command documents repeated--configand--quiet.git status --shortand confirm only intended files changed.Final acceptance checklist
python -m pivotbot live-many --config config.spy.yaml --config config.btc.yamlstarts two bot workers when both configs are valid.live-manyrejects one or zero config paths.cash_capis mandatory forpaperandlive, optional forbacktest.cash_cap / entry_pricebefore final submission.(mode, api_key, api_secret)keys.--quietsuppresses dashboard refresh.python -m pytest -v.pippero: build failed — spawn devcontainer ENOENT. Log: /Users/pregno/projects/personal/coding-machine/.logs/pregno-daytrading/issue-3/2026-06-21T22-32-55-780Z.jsonl
pippero: build failed — git worktree add failed (255): Preparing worktree (new branch 'pippero/issue-3-run-multiple-live-configurations-in-para')
fatal: a branch named 'pippero/issue-3-run-multiple-live-configurations-in-para' already exists. Log: /Users/pregno/projects/personal/coding-machine/.logs/pregno-daytrading/issue-3/2026-06-21T22-57-55-163Z.jsonl
pippero: build failed — devcontainer up failed (1): [2026-06-21T23:04:32.201Z] @devcontainers/cli 0.87.0. Node.js v26.3.1. darwin 25.5.0 arm64.
Error: Filename must be devcontainer.json or .devcontainer.json (/Users/pregno/projects/personal/coding-machine/.worktrees/pippero-base-devcontainer.json).
at MG (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:485:3378)
at GI (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:485:5774)
at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
at async $Z (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:666:206)
at async zZ (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:665:15743)
at async /Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:485:1917. Log: /Users/pregno/projects/personal/coding-machine/.logs/pregno-daytrading/issue-3/2026-06-21T23-04-31-026Z.jsonl
pippero: build failed — devcontainer up failed (1): [2026-06-21T23:07:13.572Z] @devcontainers/cli 0.87.0. Node.js v26.3.1. darwin 25.5.0 arm64.
[2026-06-21T23:07:15.910Z] Error fetching image details: No manifest found for mcr.microsoft.com/devcontainers/universal:2.
[2026-06-21T23:07:16.390Z] Retrying (Attempt 0) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:2
'
[2026-06-21T23:07:17.784Z] Retrying (Attempt 1) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:2
'
[2026-06-21T23:07:19.187Z] Retrying (Attempt 2) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:2
'
[2026-06-21T23:07:20.577Z] Retrying (Attempt 3) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:2
'
[2026-06-21T23:07:21.964Z] Retrying (Attempt 4) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:2
'
[2026-06-21T23:07:22.966Z] Command failed: docker inspect --type image mcr.microsoft.com/devcontainers/universal:2
[2026-06-21T23:07:22.966Z] []
[2026-06-21T23:07:22.966Z] Error response from daemon: No such image: mcr.microsoft.com/devcontainers/universal:2
[2026-06-21T23:07:22.966Z] Command failed: docker pull mcr.microsoft.com/devcontainers/universal:2
Error: Command failed: docker pull mcr.microsoft.com/devcontainers/universal:2
at $V (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:467:1277)
at fG (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:467:1021)
at async f9 (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:485:4649)
at async GI (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:485:5768)
at async $Z (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:666:206)
at async zZ (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:665:15743)
at async /Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:485:1917. Log: /Users/pregno/projects/personal/coding-machine/.logs/pregno-daytrading/issue-3/2026-06-21T23-07-12-707Z.jsonl
pippero: build failed — devcontainer up failed (1): [2026-06-21T23:10:35.325Z] @devcontainers/cli 0.87.0. Node.js v26.3.1. darwin 25.5.0 arm64.
[2026-06-21T23:10:36.018Z] Error fetching image details: No manifest found for mcr.microsoft.com/devcontainers/universal:linux.
[2026-06-21T23:10:36.436Z] Retrying (Attempt 0) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:linux
'
[2026-06-21T23:10:37.846Z] Retrying (Attempt 1) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:linux
'
[2026-06-21T23:10:39.244Z] Retrying (Attempt 2) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:linux
'
[2026-06-21T23:10:40.638Z] Retrying (Attempt 3) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:linux
'
[2026-06-21T23:10:42.043Z] Retrying (Attempt 4) with error
'Command failed: docker pull mcr.microsoft.com/devcontainers/universal:linux
'
[2026-06-21T23:10:43.045Z] Command failed: docker inspect --type image mcr.microsoft.com/devcontainers/universal:linux
[2026-06-21T23:10:43.045Z] []
[2026-06-21T23:10:43.045Z] Error response from daemon: No such image: mcr.microsoft.com/devcontainers/universal:linux
[2026-06-21T23:10:43.045Z] Command failed: docker pull mcr.microsoft.com/devcontainers/universal:linux
Error: Command failed: docker pull mcr.microsoft.com/devcontainers/universal:linux
at $V (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:467:1277)
at fG (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:467:1021)
at async f9 (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:485:4649)
at async GI (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:485:5768)
at async $Z (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:666:206)
at async zZ (/Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:665:15743)
at async /Users/pregno/.node_modules/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:485:1917. Log: /Users/pregno/projects/personal/coding-machine/.logs/pregno-daytrading/issue-3/2026-06-21T23-10-34-515Z.jsonl
pippero: build failed — no valid .pippero/result.json produced by the build. Log: /Users/pregno/projects/personal/coding-machine/.logs/pregno-daytrading/issue-3/2026-06-21T23-17-36-339Z.jsonl
pippero: build failed — no valid .pippero/result.json produced by the build. Log: /Users/pregno/projects/personal/coding-machine/.logs/pregno-daytrading/issue-3/2026-06-21T23-34-09-724Z.jsonl