Agentic Framework for Secondary Analysis R&D

the DIY experience
scidm-claude: an AI agent environment built on Claude Code —
design, features, and use in research and development
Denis Kaznadzey · 2026-10-15
PLATFORMRECORDKNOWLEDGECONTEXTCODINGRESEARCHDOCUMENTClaude Code

What this talk covers

1

Background

From a bare language model to orchestrated agents.

3 slides
2

Architecture and features

AUpdate system
BAgent account
CLaunchers
DTrac + SVN
EKagebunshin
FShintenshin
GTotalrecall
HMetempsychosis
IPhoenix
JOrchestration
KProtocols
LClusters
3

Use cases on Thermo work

User experience in development tasks.

4

Outlook

From a continuously emerging prototype to a generalized framework.

1 · BACKGROUND

Four tiers of LLM-based systems

1Language modeltext in → text outWHAT PERSISTSnothing2Chat bot+ accumulating context, user loopWHAT PERSISTSthe conversation3Agent+ tools, actions, stateWHAT PERSISTSconversation + world changes4Orchestration+ agents used as toolsWHAT PERSISTSmany contexts + a shared worldinput texttransformercompletiononepassusercontextmodel12341prompt appended to the context2model reads the whole context3response appended4updated context shown → next promptagentic loopusercontextmodeltools12534steps 2 and 5 repeat, many per turnworldstate1prompt appended2model reads the context3response appended4updated context shown5tool call; result appendedorchestrator agentown context, own loopagentagentagentshared worldmulti-turn sessionsmulti-turn sessionsHigher tiers: agentic meshes, ecosystems, marketplaces, AAOs, swarms, federations
1 · BACKGROUND

From language model to agent

User sets the goal,steers, and approvesContext window= the agent's working memorySystem prompt & rulesUser requestsFiles it has readCommand outputsIts own reasoning & actionslimit: 1M tokens ≈ 1,500 pagesfills up asthe work proceedsWhen full: compacted, with detail lost — or the session ends.A new session starts empty.Language modelreads the entire context,writes the next stepno memory between sessionsknowledge has a cutoff datecan be fluently wrongHarness — Claude Coderuns the proposed action,returns the resultShell & scriptsFiles & codeWeb & literatureTrac & SVNCompute clusterOther agentsTools: interaction with the outer world1reads everything,every step2proposesan action3runs4result appended
1 · BACKGROUND

What a bare agent lacks for R&D work — and what scidm-claude supplies

AOne install per machine, configured by hand→ Update systemBFull trust — or keeps asking permissions→ Agent accountCOne endpoint and one tool set per launch→ Launchers & profilesDWork evaporates with the terminal scroll→ Trac + SVN recordEForgets everything between sessions→ Kagebunshin: shared memoryFA generic system prompt→ Shintenshin: prompt assemblyGTranscripts stay local, limited retention→ Totalrecall: archive & searchHContext bound to one session on one host→ Metempsychosis: snapshotsIContext overflow ends in lossy compaction→ Phoenix: controlled handoverJOne context for everything→ OrchestrationKProcedures drift; quality goes unchecked→ Protocols & reviewsLOne user, one machine→ Clusters, distribution & collaborationClaude Codeunmodified+ Claude model
A · DELIVERY

A continuously emerging prototype

100200300400May 18Jun 8Jun 29Jul 20Aug 10Aug 31Sep 21TotalrecallKagebunshin syncMetempsychosisShintenshin self-reviewPhoenixsemantic recallcodified protocolsCommits per week, 2026code & designs (SCIDM)memory, prompt, launchers (kage)3,383 code commits · 1,280 memory/prompt commits · 131 feature branches99.7% of the code commits made by the agent, under directionInstances on multiple hostsrnd cluster (Thermo)Thermo data, tools and cluster computehome lab workstationdirect access to the LAN and local drivesoffice hostoffice network; fast local diskscompute serversheavy computation distributedany hostalways-on hot backup when one breaks oris out of reachChanges daily, runs everywhere →delivery must be frequent and seamless
A · DELIVERY

Frequent, seamless delivery: the update system

The gap

Claude Code installs per host; its configuration is a hand-kept, per-user directory.

›
Out there

Configuration management (Ansible, Nix), vendor-managed settings, plugins — no live agent state, no secrets, no two install flavours.

›
scidm-claude

SVN as the single source; tested on branches; one command updates any host inside a transaction.

Ticketwhat & whySVN branchcopy of the treeTest hostscanonic + single-userReviews & grid14 test cellsoperatormerge✓Trunkproduction sourceoperatorper host✓Fleet14 installsupdate-claude tool brings the scidm-claude instance to the most current state, directly from the SVN repoSVNexport a clean,throw-away treeinside a deploy transactionsystemconfigsecretsnodeCLIpackagessubsystemsadminverify22 steps in 9 sections, each run selectable; --dry-run prints the planmaintenance marker: no new sessionsservices paused and restoredon failure: log + reminderEvery session knows its buildscidm-claude version:2026-Q3.2(SCIDM r5210; kage r1272)shown in the prompt'sEnvironment section
B · SECURITY

A separate account for the agent: security by the operating system

The gap

The agent runs with all of the invoking user's rights — the harness compensates with permission prompts, or with none at all.

›
Out there

Harness sandbox (shell only), containers, VMs — safe, but cut off from the LAN, shares, cluster and display.

›
scidm-claude

The agent is its own Unix user; group permissions decide what it may touch — and the kernel enforces it.

user — operatorclaude_ai — agentthe kernel decidesShared project areasgroup ai_agents, group-writableread & writeread & writeboth work hereOperator's private fileshome, keys, mail, personal datafull accessno accessrefused by the kernelAgent's own area~/.claude, secrets registry (mode 700)~via sudo onlyfull accesssecrets stay with the agentThermo git worktreesproduct and R&D repositoriesedit, commit, push~edit; no committhe operator owns historySingle-user flavour (rnd cluster — no second account possible): the same code runs under the user's own account; there the agent has the user's rights
C · LAUNCH

The launcher: where a session is assembled

The gap

The harness takes one endpoint, one tool set and one set of credentials from its environment.

›
Out there

Settings files, managed settings, per-project configuration — configuration, not session assembly.

›
scidm-claude

One launcher, many names; six-axis profiles; secrets read only after switching to the agent account.

runs as the invoking userruns as the agent (claude_ai)sudo: one permitted script, nothing elseclaudecoone script, manynames: the nameselects the profileOuter launcher• read the profile and the flags• clean the environment• hand over a launch descriptorreads no secretsInner launcher• secrets registry → credentials for the chosen endpoint• tool servers (MCP): 14 available• session secrets: SVN, Trac• system prompt assembled → Shintenshinclaudeunmodified harness+ model at the chosen endpointLauncher loopwhen the harness exits:handover requested?→ start the successor (Phoenix)exitSubordinate agentslaunched by the agentitself: already claude_ai,so no sudo — straightto the inner launcherSecrets live only on the agent side of the line; a launch command never carries them.
C · LAUNCH

Profiles and backends: same harness, many brains

A launcher profile (TOML) sets six axes — each can be overridden per launchendpoint & credentialsmodel & efforttool servers (MCP set)tool exclusionssession secretsprompt profilee.g. claudeco = corporate Opus endpoint · slim MCP set · default exclusions · default secrets · prompt scidm-default-v2.1claudecclaudecoclaudec5claudeco5Azure APIM gateway (Thermo)Anthropic models only: Sonnet, Opusclaudepapi.anthropic.compersonal subscription (OAuth)claudesolclaudeterraLiteLLM proxy → OpenAI (Thermo)GPT-5.6 sol / terraclaudep-sol6claudep-astra6claudep-ds4claudep-ds4newLiteLLM proxy → OpenRouterGPT-6 sol / astra · DeepSeek V4 ProSame harness, different brain. The GPT backends work with all tools and harness features — used for design sessions.Outside Anthropic's supported configurations; useful in practice.Background jobs: session nicknames and summaries (Metempsychosis), memory-merge resolution (Kagebunshin);they use worker profiles (Anthropic, OpenAI, Ollama).
D · RECORD

Document everything, in structured form: Trac + SVN

The gap

Agent work evaporates with the terminal scroll — decisions, reasons and reviews are lost.

›
Out there

GitHub / GitLab issues, JIRA + Confluence — hosted or heavy; not built as an agent's notebook.

›
scidm-claude

Self-hosted Trac + SVN, written by the agents: a ticket for every change, a signed comment for every step.

commit message: refs #1302signed comment: session IDthe transcript shows every commandTrac ticketwhat and why: goal, design, findings, decisions1,502 tickets · 876 on scidm-claudeSVN revisionwhat changed, atomically numbered94.5% of trunk commits cite a ticketSession transcripthow it was done, step by step747 working sessions archivedTrac: tickets, wiki, repository browser, reports, custom fields;agents read and write it through an MCP serverSVN: one revision number per change; cheap subtree branches;sparse checkoutsStandalone install bundles both, offline
D · RECORD

What a record looks like: one review, as Trac shows it

trac-personal #1302 — dual-agent review of a design, 2026-09-11, 22:37–22:5522:37Reviewerround 1: 3 findings (2 concerns, 1 suggestion)22:39commit r4569R1 fixes … refs #1295 refs #130222:39Designer"All three findings valid, all three fixed. Two of them arethe kind that only an independent reader finds."22:43Reviewerround 2: 1 finding (risk)22:44commit r4570R2 fix … refs #130222:48Reviewerround 3: 1 finding, plus: the round-2 response claimed an editthat did not land22:49commit r4571R3 fix … refs #130222:54Reviewerround 4: REVIEW IS CLEAN22:55Orchestratorclosing: 4 of 10 rounds; 5 findings, all fixedEvery agent comment is signedwhoclaude_aimodelclaude-opus-4-7[1m]roleReviewer (DUAL_AGENT_REVIEW)sessionb0563940-5ad2-4c68-…wherearchimedes:…/review-1295launcherclaudecowhen2026-09-11$ search-sessions --show b0563940→ the reviewer's full session, on any LAN hostticket ↔ commits ↔ sessions:one connected record of the work
E · MEMORY

Kagebunshin: memory shared by every instance

The gap

Built-in memory: per harness, per project, per host; a flat index; nobody reviews what gets written.

›
Out there

Memory services (Mem0, claude-mem, Letta): vector stores, often hosted, opaque, without reviewed merges.

›
scidm-claude

Plain Markdown records in an SVN "hot repository": synced to every host, merges audited, semantic search.

影分身Kage Bunshin"shadow clone"In Naruto, solid clones work on theirown. When a clone dissolves,everything it experienced returns tothe original. Many agent instances,one growing memory.kagerepository (SVN)archimedesfermatklapauciusrnd clusteratillaa new session↑ commit what it learned↓ update with what others learned
E · MEMORY

From need to mechanism

11Hot repositoryneedShare across instances onmany hostsmechanisman SVN working copy ofplain Markdown records:versioned, human-editable22Live sync engineneedKeep every instancecurrentmechanismevery 3 minutes and onevery change: commitlocal, update remote33Merge controlneedSee and approve what getsmergedmechanismconflicts become mergerequests: review, accept,resolve44HierarchyneedLong flat lists clutterthe contextmechanism14 categories withsub-indices; only a 10 KBtop index is injected55Semantic searchneedThe index lists onlytitles: the agent mustknow what to look formechanismrecords split by headingand embedded (bge-small,384-dim) into a sqlite-vecvector DB, plus a BM25keyword index; ranks fused(RRF); re-indexed on everycommit; top 3 hitsinjected at session startand on topic change;recall tool on demand251 records in 14 categories · 759 memory commits · 26,037 sync cycles in 71 days on one host · top index 30 KB → 10 KB
E · MEMORY

How the memory learns

Operator correctsthe agentAgent writes a recordrule · why · how to applyCommitted & syncedto every hostEvery later sessionbehaves differently/reflectconsolidates asession; runs in thebackground on atopic changefeedback/conversational_trap.md — a real recordtype: feedbackChallenge + question and "should be" inputs areconversational — confirming a bug is not authorization tofix it.The problem-identification trap: when discussion confirms a bugexists, the instinct is to fix it immediately. This is wrong. Statewhat was found, describe the fix needed, stop.Why:User said "This seems wrong to me. Are there any obstacles? Is thetrunk URL hardcoded?" — the agent investigated AND implemented(r2832) without being asked to implement.How to apply:Answer the question, describe the fix, stop. Act only on anexplicit command.48 feedback records, each with the incident behind it · 27 reflection logs since July
F · PROMPT

Shintenshin: the system prompt, assembled for the task

The gap

Default prompt: fixed, generic, written for coding. CLAUDE.md arrives as a message after it — it adds, never replaces.

›
Out there

Replace/append flags, output styles; prompt tools (Langfuse, promptfoo, DSPy) built for application prompts.

›
scidm-claude

Modules assembled per purpose at launch; live-synced; each change approved; the agent reviews each edition.

心転身Shintenshin"mind-body switch"In Naruto, a ninja sends his mindinto another's body. Here: a mindassembled for the task, in thevendor's harness.39 modulesprofiles pick and order thembare3agent11v227v2.134upstream1assembled prompt ≈145 KB→ claude --system-prompt-fileplaceholders filled at launch• runtime: model, host, launcher• deploy: version, revisions• derived: paths, project memory• per-host overrideskage working copy: synced fleet-wide, each change approved by the operatorbare: one-shot jobs · agent: subordinates, reviewers · v2.1: interactive · upstream: the vendor's own
F · PROMPT

Inside the current prompt

User interaction2,383 wordsDoing tasks2,162 wordsWork with care2,073 wordsSubordinateagents1,042 wordsCodingstyle828 wordsSystem786 wordsDocu-mentingwork458 wordsHow to work → D, JKage operating rules1,700 wordsPersistentmemory1,390 wordsSession start986 wordsAuto-memory411 wordsMemory → EGraphicaldisplay993 wordsKage-bunshin293 wordsKageretrieval290 wordsShintenshin259 wordsMetem-psychosisDeployment235 wordsTotal-recallProv.Subsystems → E–ISoftwareengineering544 wordsSysadmin507 wordsOther330 wordsResearch294 wordsActivities Sessionlifecycle542 wordsModels &launchersEnvironmentSession → C, Iscidm-claude395 wordsLocaldatastayslocalLANIntroIdentity & place → A, BProtocoladherence531 wordsCodifiedprotocolsProtocols → KAupdate system  ·  Bagent account  ·  Claunchers  ·  DTrac + SVN record  ·  EKagebunshin  ·  FShintenshinGTotalrecall  ·  HMetempsychosis  ·  IPhoenix  ·  Jorchestration  ·  Kprotocols & reviewsThe active prompt (scidm-default-v2.1): 34 sections, area ∝ wordsContext usage by system promptvendor default: 4,187 wordsassembled: 21,189 wordsRelevance and coherence matter more thanlength: sections contain informationessential in every session.Environment — generated per launchpowered by claude-opus-5-5runs as claude_ai on archimedeslaunched by <user> via claudeco5Claude Code 2.1.296scidm-claude 2026-Q3.2 (SCIDM r5210; kage r1272)Self-review by the agentEach new edition: "introduce yourself",then "find contradictions, gaps,ambiguities". June campaign:47 rounds in 2 days, findings per roundfalling from up to 13 to 3–6; ~70 fixed.
F · PROMPT

Rules, guidance and guardrails: three tiers

System promptalways loadedshort imperatives + 68 references to 48memory recordsMemory recordsloaded when relevantguardrails 19 · feedback 35 · protocols 23· sysadmin 3 · Trac 5Hooksenforced by coderun on harness events; can block an actionoutrightWhere the rules liveWide find sweeps"Never find /; give a startpath and a depth limit"guardrails/find-wide-sweeps:network mounts descend intoterabytes; runaway scans hangblocked unless -maxdepth ≤ 3,-xdev, or an explicit overrideQuestions are not orders"Only explicit commandstrigger actions"feedback/conversational-trap:the incident, the rule,how to apply— judgement, no hook(not every rule can beenforced by code)
G · ARCHIVE

Totalrecall: every session archived and searchable

The gap

Transcripts stay on one machine, are deleted by default after 30 days, and cannot be searched by content.

›
Out there

Longer retention; per-machine exporters and viewers of transcripts.

›
scidm-claude

Hooks archive every checkpoint to shared storage; a per-host index; full-text and lineage search.

One session, on any LAN hoststartendbefore / after compactionevery turnArchive on shared storage: every checkpoint kept, never pruned; crash-safePer-host index: title, first prompt, last reply, tags, directory, lineageStandalone search tool$ search-sessions "ITD detector"Check Python 2.7 pysam versions for…archimedes · Jun 9itd detector ref trust merge ctlarchimedes · Apr 11itd detector keep minorarchimedes · Apr 5metadata search: 0.2 sfull text over 14 GB: 27 schains: ancestors, descendantsWhy it mattersMay 12119-turn ITD detector sessionJune 11harness deletes its local copyAugustarchive copy is the only survivor747 working sessions on 15 hosts (3,810 incl. automated checks) · 169,559 tool calls · 147 M output tokens · rnd: 141 sessions in its own archive
H · CONTEXT

Metempsychosis: context that moves between sessions and hosts

The gap

Sessions live on one machine; forks start only from the current point; lineage cannot be queried.

›
Out there

Harness branch, fork and rewind; vendor-side stores (SDK session store on S3, Redis, Postgres; cloud sessions); LangGraph time travel.

›
scidm-claude

Every turn snapshotted to the shared storage: load, branch or fork any turn of any session on any host.

T1T2T3T4T5T6T7T8T9T10T11T12session on host Ahost AT4'try an alternativefrom turn 3T10fork at turn 9: a subordinate inherits the full contexthost BT8load turn 7 on another host and continuehost CShared snapshot store — every turn captured as compressed, content-addressed blobs, with session lineage18,216 snapshots · 3,686 sessions · 1.9 GB · nickname + turn as the address (e.g. scidm-claude-talk@T42)metempsychosis — "transmigration of souls"
I · CONTEXT

Phoenix: tasks longer than the context window

The gap

When the context overflows, the harness compacts it in place — lossy, often disastrous mid-task.

›
Out there

Manual compaction with instructions, a configurable window; handover tools in other agents (Cline's new_task).

›
scidm-claude

The agent writes the handover while its context is rich; the launcher starts a fresh successor automatically.

Harness default: auto-compactioncontext loadAt the limit the context issummarized in place:detail lost, mid-task,unplannedPhoenix: the agent hands over, a fresh successor continuescontext loadlink 1link 2link 3Handover: progress.md +a rich prompt; the launcherstarts the successorReminders only in thelast few percent (amber);below: silence — no"context anxiety"Example — preparing the ITD talk: 7 links · 99 hours · 6 handovers · no compaction73 handovers in 43 chains since late August · originally named CCP, the Context Chain Protocol
J · WORK

Orchestration: the agent as a team lead

The gap

One agent, one context, one model: long work bloats the context; independent work runs in series.

›
Out there

Harness subagents (a baseline agent without knowledge of the system's facilities); multi‑agent frameworks.

›
scidm-claude

Subordinates through the same launchers — full rules and memory; the right model per task; every return verified.

Orchestratordecomposes, briefs, verifiesSonnetlong procedural work: tests,deployments, sweepsOpusanalysis, design, reviews,investigationGPT (claudesol)independent views, designsessionsbrief: a fresh context+ referencesfork: inherits the wholecontext (snapshot)Every return is verified: branch, commits, files, syntax, acceptance — or halt and reportFan-outresearch: 7 agents gathered theevidence for this talk in parallelcompute, e.g.: one TMAP issue acrossTorrent Suite versionsorchestratorcompares how the issue manifestsagenton rnd20buildsTS 5.16own fixes,config, depsagenton rnd21buildsTS 5.20own fixes,config, depsagenton rnd22buildsTS 5.22own fixes,config, depseach build needs its own judgment, soeach gets its own agent and hostalso: digging for sequence features indifferently organized BAM archives338 subordinate sessions from 237 parents since late August · largest tree 15 · no blind forwarding of a job
K · PROTOCOLS

Protocols: recording what was learned

The gap

Learned techniques live in one person's head or one session's context — and are lost.

›
Out there

Agent skills, SOPs with MUST/SHOULD/MAY steps, spec-driven task lists: the agent follows them, but control flow stays in prose.

›
scidm-claude

Protocols in shared memory: each lesson recorded once, with its origin, applied in every session on every host.

Ways to record how work is doneAd-hoc promptplan: in the operator's headstate: nowhereSkill (harness)plan: an instruction file Claude followsstate: the context windowProse protocol · 20plan: a procedure in memory, every step mandatorystate: progress.md in the projectCodified protocol · 3plan: a program the agent interpretsstate: execution files + tracemoreformalA lesson learned once, applied everywhereincidentlesson + originprotocol ruleevery hoste.g. review calibration (#1256) · documentation sweep in proposals(#1351) · no credentials in protocol state (#1359)Sessions that read each protocol (403 sessions in all)review procedure231dual-agent review164small functionality test134branch-completion test grid77merge & verification46autonomous ticket handling39Phoenix handover38infrastructure proposal29
K · PROTOCOLS

Reviews: the most useful protocols

Reviews make it possible to hand over both the design and the implementation of complex, well-integrated features to agentsOperator: sets the goal, reads the record,decides on the mergeDesigner / Implementerwrites the design or codeanswers each findingfixes and commits per roundReviewerfresh context: learns everythingfrom the ticket; resumed eachround with its own test scaffoldingReview Request ticket (Trac)round 1findings: risk · concern · …responsefix + commit per findinground 2 … nuntil no findings remainREVIEW IS CLEANor the round limitseverity: bug · risk · concern · suggestionany model: Opus by defaultApplied tocodedocumentsimplementation proposalsproduction-system designssystem-prompt editions"All three findings valid, all three fixed. Two of them are the kind that only an independent reader finds."— the designer agent answering its reviewer, #130285 review-request tickets · 90 sessions acting as reviewer since June · the built-in review skill is switched off in favour of the protocols
K · PROTOCOLS

Production-system design review

Gapclosuredesigner≤ 1 roundComplexityreductiondesigner≤ 5 roundsSelf-review:standarddesigner≤ 25 roundsSelf-review:simulationdesigner≤ 15 roundsDual-agentreviewfresh reviewer≤ 10 roundsRegressionself-reviewdesigner≤ 5 roundsThe order is load-bearing: complete the design → prune it → review itMental simulation — two modes × four mandatory scenariosMode A: no-error walkenumerate every step explicitly; everyREAD needs a prior WRITEMode B: failure injectionat each step: what if THIS step failswith THIS accumulated state?Operation cycleeach entry point at three system ages: firstrun, second run, agedFresh deploymentwith an audit of implicit assumptionsDeployment onto a running systemlive state, active users, running servicesRe-deploymentidempotency: the second run changes nothing×Every proposal must contain• the deployment procedure • the test plan• a reading guide • a documentation sweepDone whenfindings reach zero — or the round limit leavesonly concerns and suggestions; a finding countthat stops falling triggers a diagnosis
K · PROTOCOLS

Why prose protocols are not enough

The gap

Prose protocols drift: steps skipped, instructions narrowed, interpretation shaped by context.

›
Out there

Workflow engines and agent frameworks: a new runtime, code first — the agent no longer reads the procedure.

›
scidm-claude

Codified protocols: a structured language the agent interprets, with execution state and trace kept in files.

Steps skipped or"descoped"An agent skipped mandatory reviewphases and test cells on its owneffort estimates. Now the promptlists the forbiddenrationalizations.#852Instructions narrowed"Every applicable local test" wasread as "dry run only"; a"tailor" clause quietly narroweda standing default.#705A halt rule ignoredA delegated test executor ran an11-bug fix-and-continue loopdespite an explicit "halt andreport".#705Context shapesinterpretationInstruction files in a workingdirectory steered the mergeagent; a standing rule overrodean explicit operator instruction.Prompt-injection-like, without anattacker.June–Sepagent reads& follows itno extraruntimelevels ofobligationaudittrailsurvivessession endWorkflow engines (LangGraph,Temporal)Multi-agent frameworks(CrewAI, MS Agent Framework)~~Claude Code Workflow tool~SkillsSOPs with MUST/SHOULD/MAYsteps (e.g. Strands)Prose protocols~~~Codified protocols~ = partly · characterization from the 2026 codification study, with updates
K · PROTOCOLS

Codified protocols: a protocol as a program

A protocol is a program — specimen from the language specification== VARSTARGET_HOSTS: list = []FAILED: list = []== END VARS== PROCEDURE main1 MANDATORY SET $TARGET_HOSTS = <@ELIGIBLE_HOSTS>2 FOR EACH HOST IN $TARGET_HOSTS2.1 MANDATORY SET $OK = CALL deploy-and-verify(HOST=$HOST)2.2 IF $OK == false2.2.A.1 MANDATORY record the failure SET $FAILED = <the failure list incl. $HOST>2.2 ENDIF2 ENDFOR3 IF $FAILED == []3.A.1 MANDATORY RETURN done3 ELSE3.B.1 MANDATORY HALT "deployment failed on $FAILED"3 ENDIF== END PROCEDUREscpexec: the state lives in files, not in the contextEXEC_STATEpointer: 2.1HOST = fermatFAILED = []call stack: main → 2.1protocol sha256EXEC_TRACE (append-only)init protocol + includes pinnedadvance 1 → 2 set TARGET_HOSTSforeach HOST = fermatcall deploy-and-verifyreturn OK = truebranch 2.2 → false… doneschematic trace for this specimen; event types as in real runsDeclare, then doeach step is declared in the trace before it runs;gate decisions and deviations are recordedwith reasons; skipping a MANDATORY step needs the operatorSurvives the sessioncontext full → Phoenix successor runsscpexec bind --take-over and continues the sameexecution from the recorded pointerStatus: in adoption — 3 of 20 protocols codified, 31 traced runs; the most used protocols still run as prose
L · CLUSTERS

Clusters: memory, archives and instances

The gap

One user, one machine: configuration and learning do not travel; no notion of a group of instances.

›
Out there

Managed settings and plugin marketplaces distribute configuration — they do not bring learning back.

›
scidm-claude

Memory clusters per kage base; archive clusters on shared storage; standalone instances without servers.

Memory clusters: one kage base eachkageSVNscidm — production14 installsseed-base — templatefor new clustersworkgroup X — possibleown members, own serverArchive clusters on shared storageLAN: NFS on eulerone transcript archive + one snapshotstore for all hosts; each host writesits own index; lock files per lineagernd: the shared homeone single-user install serves 30+nodes; the same sharing and locking,updates serialized by a lockStandalone instancelocal SVN (file://)local Traclocal memory searchoffline install bundleone host, no servers;tested, not yet in productionUsers (current, limited model)sudocanonic: several admins share one agent identity —one memory, one secrets registry; per-user recordssingle-user: each user is their own agent,with their own memory and archive
L · CLUSTERS

The fleet today

Home lab LANeuler — LAN serverSVN · Trac · NFS archivesmemory-search hub · LiteLLM proxy(not an agent host)archimedesprimary workstation1,118fermatcompute first168pascal5aristotelis61plato19newton36leibniz7atos9portos7aramis7descartes36atillamobile, via proxy235klapauciusoffice · reverse tunnel48Thermo rnd clustersingle-user install on the shared homeserves 30+ nodes · own archivesreached through the bastionrnd20rnd21rnd22rnd23rnd24rnd25rnd26rnd27rnd28rnd29bikinirnd8rnd9rnd10rnd12…141Test hosts — deployed from brancheskurdlcanonic1,585sepulkasingle-user497heteronius2014 production + 3 test installs synced to one kage repopills: sessions archived
AND MORE

And more

Session nicknames

an offline worker names each session and summarizes it; the handle in every signature

Reflection

/reflect consolidates a session into memory; runs in the background on a topic change

Secrets registry

one file per secret, readable only by the agent account; never in a launch command

Trac MCP server

our fork: custom fields, raw TracWiki, workflow actions for agents

Worker connections

named endpoints for background jobs: Anthropic, OpenAI, local Ollama

Graphical display

with a grant, the agent opens IGV and viewers on the operator's screen

Language servers

code intelligence for Python and Bash, usable by IDEs too

Time awareness

current time on every prompt and tool result

Guard hooks

block filesystem-wide find sweeps; refuse sessions during an update

Codex peer

an OpenAI Codex environment alongside, sharing group and repository

Web chat archive

full-text search over exported web-chat transcripts

Version stamps

every session knows its build: version, code and memory revisions

PUTTING IT TOGETHER

The architecture on one page

WORKIPhoenixtasks beyond one contextJOrchestrationsubordinate agentsKProtocols & reviewsrecorded ways of workingKNOWLEDGE & CONTEXTEKagebunshinshared memoryFShintenshinprompt assemblyGTotalrecallarchive & searchHMetempsychosissnapshotsPLATFORMBAgent accountOS-enforced securityAUpdate systemSVN → every hostLClusters & fleetshared storage, groupsSESSIONCLaunchers & profilesassemble every sessionClaude Codeunmodified harnessModelsAnthropic (Thermo, personal)OpenAI · DeepSeekDRecordTrac + SVNevery change:a ticket,a revision,a signedsessionwhat, why,how, by whomEverything on top of an unmodified vendor harness — upgrades land without patching; replacing it is the goal of v2
3 · USE CASE

Work track: reference trust in the ITD detector

The requestCustomers (IRS-29792, IRS-31877): report theinsert sequence the reads carry — not thereference copy — when the reads really differ.Example: an ITD with 6 extra bases CGGGGGat the copy junction.What was built · PR 67Conversion to the reference copy only above anidentity threshold and without a dense mismatchcluster; read-derived sequences on all evidencepaths; anchor-first option; consistent evidence BAM.Merged as PR 67 on 2026-04-13.Since thenIn the production pipeline (Genexus).hg38 test fixtures for the test team (July);later sessions find the work in the archive.Remaining: a pre-existing one-base shift inthe recorded INS location, now visible (#438).Apr 6 – 11archimedesdesign · code · reviewSPECProposal from the request; the planreviewed on requestREVIEWCode review by a subagent; theoperator rules on its findingsDECISIONv1 dropped, v2 designed from scratchCODEv2 implemented in a fresh sessionApr 12 – 13archimedestests · mergeTEST RUNRegression against the old version onthe lab BAM setREPORTEvery difference explained: traces,evidence BAMs, IGVMERGEPR 67 merged Apr 13Apr 15 – 20Genexusinto productionMERGEPorted to the production pipelineREVIEWCopilot suggests two small fixes atthe port; back-portedJul 28 – Octrnd clustertest fixtures · look-upsTEST DATAhg38 test fixtures built for the testteamSEARCHLater sessions find this work againfrom the archiveDEFECTA pre-existing INS-location shiftsurfaces (#438)April: Claude Code + project notes and progress.mdno tickets, no shared memory, no review protocolscidm-claude: tickets, shared memory,archive search, synthetic test data
3 · USE CASE

Reference trust: the first exchange

Operator · first prompt of the project (Apr 6)This folder contains a flt3_itd subfolder. This is a source code for the itd detector tool (entry point flt3_itd/detector/detect_itd.py).The itd detector was originally developed for detecting internal tandem duplications in FLT3 gene that are the important markersin myeloid cancers.Recently it has been extended to support detection of the long inserts (primerely duplications) in any genome locaton.It is target at detecting ITDs and other types of inserts in the sequencing data obtaind by usiing AmpliSeq based sequencingpanels with IonTorrent NGS sequencing instruments. The input data for the tool is BAM file; it looks for the insert/duplicationevents located in given genome intervals (like exones 14 and 15 of FLT3 gene). Traditional variant callers often fail to detect suchinserts. If the amplicon primer in a panel falls inside the duplicated region, the allele-carrying template amplifies to two products,the shorter one matching wildtype. It competes with the longer one that includes an extra copy of the duplicated region duringPCR, reducing the observed AF. Also the longer product often does not get sequenced end-to-end (for instance, due toexhaustion of the flow sequence), which implies that only the portion of the read from either end to the insertion point is aligned.The part that continues beyond the insertion point ends up in a softclip. The detector algorithm analyzes the softclips anddeduces the size and posiion of the insert by (1) finding the anchor matching the reference in the softclip it the read contains oneand (2) matching the softclipped prefix of the duplication with the reference (original) copy in case the insert is a duplication,effectively determnig it's size even if partial copy is present in the read. The detector compensates for possible sequencingartifacts, bulds a consensus and reports variants supported by data as VCF file.During the processing of data the detector finds primary insert evidence reads, consolidates them, and then filters the candidatevariants through a series of filters.The determination if the observed insert can be classified as 'duplication' (rather then de-novo insert) plays improtant role asdifferent stages of processing. As the template carrying true duplication might contain several features that disrupts the exactmatch between the reference and the duplication, such as sequencing errors and community variants. The detector tries tocompensate for such features and still classify the insert as duplication. One of the transformations it applies is loosely called'refernce trust': if the reference and the insert are similar enough in some metric (have edit distance not over a threshold, or hasnot more then X mismatches, etc), it is considered to be a true duplication and the resulting insert sequence is converted tomatch reference (so that the deviations are removed).The job we will be doing in this project is based on the request from customers to provide the option for the detector to report theoriginal insert sequences (not to apply reference trust).This option might already be supported through existing parametrization. However, we may need to add the way ni which slightlydeviating inserts are still classified as dups while their sequence not converted to the reference. It would be the best if theexisting parametrization through command line can be used to trigger this witout adding new parameters.First please analyze the code and report your understanding of the overall architecture, the project structure, control flow anddata flow. Please identify how the conversion to reference is controlled and when it is applied.Please propose the plan of actions to satisfy the request. Please do not do any modifications, present the plan and first. We silldiscuss it first. Please do not proceed to implementation in further turns until I explicitely ask to do so.Agent · reply after reading the code (excerpt)Critical finding: ref_trust is NOT wired into any logic. It appears only in the reportheader (ItdDetector.py:1500-1502). The conversion always happensunconditionally regardless of this parameter's value. There is a directcontradiction: the default value is documented as "no conversion" but the codealways converts.Proposed Plan of Action — Step 1: Preserve original read sequences in evidenceobjects. Step 2: Wire ref_trust into fill_in_seqs. Step 3: Change DEF_REF_TRUSTdefault. Step 4: Update report header.Open Questions Before Implementation — 1. Default behavior: ShouldDEF_REF_TRUST stay 1.0 (making "no conversion" the new default), or be changedto 0.0 to preserve the existing conversion behavior? Changing to 0.0 is the safe,non-breaking choice.Operator · reply (first paragraph)The parameter --ref_trust (as some others) were wired in as a mock-up for theplanned functionality, so the integration with front end could be done in parallelwith the back-end development. I am gradually adding the planned functionality.In this project we will be implementing the proper --ref_trust.
3 · USE CASE

Reference trust: a code review, and the locus–sequence rule

Apr 6 – 7The operator fixes the meaning of --ref_trust and how the reported sequence is chosen; the agent writes the proposal and reviews its own design on request. A compactionloses a decision — from then on plan and progress are kept in files. v1 is implemented.Operator · Apr 7Great. Please do a code review now. Use the copy of the original code in/space/code/ion/dev/csd-ts-rnd-tools/ref-trust-merge-ctl/flt3_itd. Check syntax and style,correctness, alignment with the implementation goals (plase formulate them first based on theearlier discussions within this conversation), optimality of the solution, possible side effects,safety with respect to unexpected / unsuual inputs. When complete, save your findings into anew file in the project directory (name it appropriately).Agent · the review runs in a subagent; reply (excerpt)The review is complete and saved to code_review.md. […] Two Major findings: 1.pick_insert_sequence / _sufficient_inframe_support — len(ev.read_seq) is called withoutguarding against read_seq returning None (which happens when an evidence has no acceptanceflag set). […] 2. _select_representative_sequence — pick_insert_sequence(all_evs) canreturn None […] but the very next line calls len(candidate_seq) on it.Operator · reply (first paragraph)Thanks! There is no need to correct anything. The event.read_seq being non None is a newconvention that is followed through the code. Having it unassigned is a legitimate cause of anerror, there is no need to guard it with specific check. Same applies to finding 2, The rest isinformational.Agent · Apr 7, after the first regression against the old version (excerpt)[…] The INS differences are from the new consensus-voting sequence selection, which is adeliberate improvement, not a regression.OperatorI am almost certain we see the position error. Looks like when the evidence is picked by voting as arepresentative, the position of the insert from that evidence is not propagated to become thereported position of the insert/duplication. The inserts carying rotated sequences are ok - as longas they are report4ed at proper position. In the data they always coincide with the shift of the insertposition. Otherwise different reads on which in is called with rotated dup sequence will not agreewith each other. In te case of duplication, the shift of the position reported as a point the insertcauses the rotation of insert sequence. But the rotated sequence reported without the shiftindicates the sequence itself is different. This is not happening in real data, and certainbly nothappening in the test data we use. Please check. If we pick as a representtive the read in which theactual insert position is shifted, we must use that insert position as the variant locus. (this applies toHEAD/TAIL originated evidences as well, but for them we will see shifted insert positions lessfrequently).Agent · reply (excerpt)The voted sequence comes from an evidence at a specific reference position; if that positiondiffers from representative.pos, reporting the sequence at the representative's position is alocus/sequence inconsistency. The rotation IS the shift — same insert, different phase, differentlocus.Apr 7 – 9Regression runs against the old version on the lab BAM set; each class of difference is traced into the code. The operator inspects reads in IGV himself and finds another bug;strict mode turns out broken and his own fix goes in.
3 · USE CASE

Reference trust: a goal check, then a clean redesign

Operator · Apr 9 – 11Now we have a pilot implementation of the modification. Please verify if they conform to thefollowing goals:1. implement reference trust controlled by the insert to the reference copy alignment properties(identity and presence of concentrated deviations);2. When picking representative insert for a supercluster (the one that is used in ref-trust evaluationand then reported as a variant), prefer the structure obtained by ins-based evaluator(ins_accepted), then anchor-based evaluators (anchor_accepted), and then dup_accepted. If noins_accepted and anchor_accepted evidences are present in the supercluster, assemble therepresentative insert sequence from the read-derived sequences of supporting evidences,supplementing the possibly missing middle part (or entire wing) with the correspondingreference-derived fragment.3. When picking the read-derived sequences for (2), for each wing (left and right) of the insert usethe sequence of the largest cluster of perfectly matchng softclip fragments (cut at the duplicationlength if they are longer). The shorter softclips are allowed in such as far s they match the longerones perfectly. The matching should account for possible offset of the matched fragment from theHEAD/TAIL (dist_to_head/dist_from_tail in Dup evaluators). Check for a possible not-agreeingsequences in the overlapping parts of left and right insert wings derived from dup evidence clusters- if the disagreement is significant warn about it, while choosing the sequence of the bigger clusterfor reporting.4. If the evaluation of ref_trust rules determines it should not be applied, the representativesequence that was used in ref_trust applicability determination should be reported as a variant.Please do not do any modifications: assume we are in the planning mode.Agent · reply (excerpt)I now have enough information for a thorough analysis. Here is my assessment ofconformance against each goal, without any modifications. […]Goal 1 — Status: Conforming. Goal 2 — Status: Partial — significant gap in the pure-DUPpath. Goal 3 — Status: Non-conforming — four distinct gaps. Goal 4 — Status: Conformingfor the INS/anchor path; indeterminate for pure-DUP.Operator · reply (first two of four paragraphs)Let's not try to apply new modificatons aimed at alignment to the goals on top of alreadyimplemented ones. The implemetation is already a bit messy, and this can only worsen thecode.Instead, let's plan the clean and rational way of re-implementing the new ref_trust supportfrom scratch, starting fresh from the version of he code we used in the very beginning. Icopied this version into flt3_itd_rt2 subdirectory in the current project - we will use it for allmodifications from now. Let's switch to the proposal generation/discussion mode: please donot alter any code until I explicitely ask to implement the proposal. Please keep a newproposal as a file "ref_trust_impl_proposal.v2.md. Please do not switch to thedesignated/official "planning mode" - it has rather inconvenient UI.Agent · reply (excerpt)Now I have a thorough understanding of the baseline, the fix file, the 4 goals, all the issuesdiscovered, and the user's design principles. Let me write the v2 proposal.Apr 11 – 12Two subagents review the v2 proposal; one finding is rated "not a bug" by theagent — the operator identifies it as a place overlooked in the change, and thefix ships. A handoff file carries the plan into a fresh, cheaper session, whichimplements v2.
3 · USE CASE

Reference trust v2: what was built

1 · Evaluators capture, they do not decideEvery evidence path stores the insert as the reads carry it: by insertion, by anchor, by duplication(soft-clip fragments). Decisions are deferred to consolidation.2 · One representative per featureWeighted plurality of the read sequences, priority insertion → anchor → duplication; a pureduplication is assembled from read fragments plus reference fill; homopolymer correction as one finalstep. Reported position and sequence come from the same feature.3 · Reference trust, decided lateReport the reference copy only when the Levenshtein identity exceeds the --ref_trust threshold andno window of concentrated mismatches breaks the trust; otherwise report the reads' sequence.4 · OutputVCF with the chosen sequence; evidence BAM adjusted with anchor priority, consistent with the call;every parameter in the run header.Design rulesNo spaghetti: evaluators collect data, consolidationdecides.Converted and unconverted calls share one selectionpath.Levenshtein identity replaces the difflib ratio, whichfails on repeated k-mers.Evaluator order selectable: duplication first (default) oranchor first (--prefer_anchor).v2 rebuilt from the original code; the proposal reviewedbefore coding.VerifiedRegression against the old version, 324 lab BAMs perround: the last round leaves only poly-T boundary callswaps (non-strict mode), each traced and explained.30 BAMs × 6 option variants: all pass, no falsepositives.Merged as PR 67 on Apr 13; in the production pipeline(Genexus).details ›Exposed, not introduced: with the reads' sequence reported, pre-existing read-slice code shows through —the INS location is recorded with a one-base shift. Correcting it is the remaining fix (#438).
3 · USE CASE

Reference trust: eight minutes to an IGV view — then a protocol

Operator · Apr 12, 22:56 UTClet's take batch1/110030/IonXpress_021_rawlib, baseline vs non strict always-convert. The 4thvariant, supported in both cases by 6 reads, is reported with different sequence. This is likley theresult of new cluster-based representative selection. Let's trace exactly what causes the difference,and demonstrate it. Please run both baseline and new code for this sample with same parametersplus '--trace ALL --trace_reads'. Redirect outputs to new files in new subdir under./tests/flatrun_v3.1/batch1/110030/IonXpress_021_rawlib/v31_dup_convert_always/. Find the readssupporting the variant at chr13:28608273. Check if same reads support the variant in old and newruns. List the evidences from trace and find if the sequence reported by a new code makes sense(represents most abundant cluster). Alsi create mini-bam containing only the reads supporting thisvariant (use samtools to extract header and to extract just the needed reads, and to unite them in amini-bam, this can b done in one command using pipe parameter substitution). Run both new and oldcode on mini-bam with generation of evidence BAMs enabled. PRoduce the batch file that wouldstart igv with these two files, --genome hg19, and --locus pointing to the location of insert - so thatthe differences can be visualized.Agent · 23:04, eight minutes later (excerpt)Mini-BAM has exactly 6 reads. […] Both run correctly on the mini-BAM and reproduce thesequence difference. Now create the IGV batch script: […] The 6 reads are identical in both oldand new — same reads, same individual insert sequences. The difference is entirely in theconsensus selection algorithm.Operator · 23:33You generated the .bat file, apparenlty for IGV. I was ignorent that such thngs are possible - Imeant the shell script. However, the .bat is interesting. Please educate me how to use it.Agent · 23:34 (excerpt)IGV supports a batch command scripting interface that lets you automate its GUI — loading files,navigating, taking screenshots, etc.Operator · Apr 13, 00:49 UTCPerfect. As we will be applying similar analysis (isolation of the cause of the difference, isolationof the supporting reads into a mini-bam, producing the evidence BAMs for baseline and newcode, generating a script for examining/comparing the 3 mini-bams in IGV, elaborating in detailson the data processing path leading to the differences and documenting it in the explanation.txtfile) it would make sense to store the instructions for such analysis persistently for repeated use.Would agent be a right tool for this?Agent · reply (excerpt)Yes, an agent would be the right tool — specifically storing the instructions as a memory file in theproject's persistent memory directory, or as a reusable skill document.Operator · 00:54YEs, please write both. The memory entry should indicate the purpose and what this analysisapplies to, and refer to the extrnal procedure file. (please mentio that --min-indel-cnt 1 is neededonly if single evidence is analyzed; if all reads that support a variant underinvestigation areextracted into mini-bam, there is no need to alter parameters.).Apr 13 – 30The protocol is applied to every remaining difference; the operator corrects thecode himself and the agent reviews it; he commits, PR 67 is merged and portedto the production pipeline; test cases for the tickets follow.
3 · USE CASE

Tests and test data

April · regression against the old version4 roundstesting plans v1 – v4, bothevaluator orders324lab BAMs per round; smoke runs88/88; 30-BAM mode check5 classesof difference: sequence, callswap, gain, loss, count15differences left in v4(non-strict mode): all poly-Tboundary swapsEach difference explained by protocol1isolate the reads in a mini-BAM2trace both versions3write evidence BAMs4IGV batch script → snapshot5explanation.txt per caseOperator: “trace why this variant changed”→ traces, evidence BAMs, IGV view:8 minutes (Apr 12, 22:56 → 23:04 UTC)July · hg38 test fixtures (#1012)02:59Operator: convert the reference-trusttest cases to hg3803:23New panel BED does not fit the sample →liftover researched03:36“accurate liftover of the entire BED isneeded … execute the plan”03:39540 / 540 amplicons lifted; 1.17 M readsremapped (Monitor)03:470 variants traced to hg19 default zones→ hg38 zones shipped with the test04:14Split into two feature fixtures; namingcorrected by the operator04:42Run as another user (ionadmin) →permission fix05:07“close ticket #1012”2 h 08 min from request to closed ticketrnd cluster, 2026-07-28 (UTC)October · synthetic reproductionReads written with explicit alignments — noaligner, no sequencing errors, every readmatches its allele exactly.51 bptandem duplication in FLT3 exon 14120reads with the insertion (51I),200 wild-type2 hostsrnd cluster and archimedes:identical callsOutcomeThe one remaining issue shows in isolation: theINS location is recorded with a one-base shift —pre-existing read-slice code, made visible oncethe reads' sequence is reported. The fix target ispinned down (#438); scripts, BAMs, outputs andexact commands are preserved in the ticket.
3 · USE CASE

Work track: read-end profiles for TMAP's REPAiR

The problemTMAP’s REPAiR anchors read edges to aread-end profile: per amplicon edge, thefraction of qualified reads ending there.July: per-edge sums bimodal, near 0.45 / 0.87.September (QA): sums above 1 — 6% / 24% ofamplicons (read starts / read ends).What was doneDefects fixed in the profiler: read assignmentat amplicon edges, the denominator, thequality-trim rule; process errors in how aprofile was built found and corrected.Merged as PR #79; profile rebuilt on the cohort.OutcomeMedian per-amplicon sum 0.53 → 0.965.Amplicons with sum above 1: 765 → 140.TMAP --repair sanity run on the newprofile: 2.1% of reads re-aligned, MAPQunchanged. Profile committed 2026-10-07.Jul 16 – 30rnd22investigation · first fixSPECOperator’s question; the agent readsthe pipelineDECISIONChemistry corrected by the operator;a rejected hypothesis re-testedMERGEOverlap fix committed by the operatorJul 22Jul 20 · Sep 2rnd22session lost — foundSEARCHJul 20: wrong config directory —found by a new sessionSEARCHSep 2: “I can not find any traces ofit” — restored from archive +snapshotSep 26 – 30rnd20one request, a session chainSEARCHInventory from memory and theearlier transcriptsREVIEWEach fix with a dual-agent reviewLOGISTICSPhoenix handovers carry the workMERGEPR #79 merged Sep 30Oct 7rnd20profile shippedTEST RUNFull-cohort profile; TMAP --repairsanity runREPORTProfile editions compared; a reportdeck for ThermoOperator: domain model, rulings, every commit / merge / push,full-cohort jobs on the cluster schedulerAgent: tracing, instrumentation, fixtures, fixes in the working copy,reviews, forensics, tickets and reports
3 · USE CASE

Read-end profiles: the first exchange

Operator · Jul 16, 19:08 UTC — first prompt (rnd cluster)In current directory the work on generating read-ends profiles for ngs sequence mapping was performed(Along with other tasks). The read-end profile is generated for a sequencing panel and contains theper-amplicon statistics of the locations where the ends of the reads land on the reference. This information isused to guide the alignment of indels located near amplicon edges - it allows to anchor the 3' of the read thatis known to be full-length (because it contains the digested out adapter sequence) to it's landing location onthe reference, and use the 'global' (needleman-wunch style) alignent around the amplico end. Thus, the gapsthat otherwise would cause softclipping and will escape the eye of variant caller, are getting aligned - andcalled s variants.The procedure of the profile collection is 3-step. First scans a collection of BAM files used as a training setand extracts the read end positions, performing certain filtering. Second collect the statistics on the results offirst. Third updates the panel BED file with this statistics. First step was carried out bymapq_collect.05_25_26.sh script, second by mapq_consolidate.06_04_26.sh, third bymapq_update_overrides.06_04_26.sh. The scripts normally call the tools thorough the symlinks in ~/bin/bin/ .Please follow the symlinks to the actual executable locations, read them and learn how they work.The question that we need to answer is: in the final profiles saved in BED, the sum of frequency values over allrecorded ends on the partular amplicon edge is expected to be close to 1 (100%) - as majority of the qualifiedreads do carry the read-end position that is expected to be included in the count. However, on large class ofedges, if not majoroty, this sum is much below (near 0.5 or even lower). Why? The hypothesys I have is that itis near 0.5 on the edges that are located in the overlaps with other amplicons, and the reads of overlappingamplicon participate in computed coverage which is used as denominator in fraction calculation. Thishypotheses may be correct or not.Please find what factors make the fractions low.Another observation that requires explanation is: The distribution of fraction value taken on the entirepanellooks clearly bi-model. Why? What classes of read-end positions these modes represent?Agent · 19:14, six minutes later (excerpt)The data confirms it completely. The root cause is definitive. […]The frequency values in each edge are normalized against ALL reads on the amplicon,not just the reads at that edge. […] The data confirms this exactly: per-edge mediansum ≈ 0.46, but READ_STARTS + READ_ENDS combined per amplicon ≈ 0.92 […]Your hypothesis was exactly right. The fix is to divide by e_cnt […] rather than thetotal r_cnt.Jul 16 – 17Asked next where the bimodality comes from, the agent explains it by anoffset between forward and reverse reads.
3 · USE CASE

Read-end profiles: chemistry corrected, a hypothesis returns

Operator · Jul 17, 05:12 UTCThe digestion happes before denaturation and sequencing. The primers contain uracils instead oftimidines to make digestion possible. The products of amplificaton are mostly 2-strand DNA piecescontaining primers on both ends. The digestion happens on all uracils in primers. Sometimes it isincomplete, and this leads to survival of some molecules where first digestion site remains undigested- the edge primer sequence continues to the 2nd (or sometimes even 3rd) uracil. The digested mix istreated with the single-strand exonuclease. The result is DNA pices from digestion site on one end todigestion site on another end. Then these pieces are linked to the adapters and sequenced. In thepopulation of amplicon products roughtly 1/2 molecules get sequenced from one end, and another halffrom another end. When they are aligned, they map to exactly same puece of reference - from thelocation corresponding to the digestion site at lower reference coord to the location of digestion site atthe higher reference coord, exclusive on both sides.Your explanation implies that the products of "fwd" and "rev" strand sequencing would map to differentcoordinates, which are shifted by 1 base. This looks like a complete nonsence. There is no space forsuch shift. It does not matter which strand of a double-stranded piece is sequecncef - they willproduce complementary sequence that map to the exactly same reference span.Agent · 05:18 (excerpt)You are correct, and I was wrong. Let me redo the analysis from scratch with the correct biologicalmodel. […] So the code produces no artificial FWD/REV split in the recorded positions. My previousQ3 mechanism was entirely wrong.Operator · Jul 17, 06:31Please reiterate on the overlapping amplicons hypotheses; check if it is wrong, provide the proove ifit is.Agent · 06:40 (excerpt)The overlapping amplicons hypothesis is false. Three independent lines of evidence: […]Operator · Jul 20, 19:08 (three days later)The elimination of bi-modality is not a question - if the distribution average is pushed to 1 by fixingdenomicator, it is not visible even if present. The client wants to know (And this is very reaonablerequest) where the bi-modality comes from. The inflated denominator allows the bimodality tomanifest, but does not explain it's origin. We need to investigate if if there is any feature in ampliconedges that participate in in different modes that discriminates them. Why one class of amplicon edgesgives typical fraction ~0.5 (with inflated denomicator), and another ~1.0? What is the difference in thisclasses? (my initial hypotheses was that one class may be in edges overlapped by adjacent amplicons,but you rejected it. Does you reasoning still apply?)Agent · 19:14 (excerpt)Strong result — the overlapping-amplicons hypothesis holds for OCAPlus 550, contra my earlierrejection for OFAv2. […] Mode 1 amplicons are overwhelmingly "middle-of-tile" (2 neighbours). Mode2 amplicons are overwhelmingly isolated (0 neighbours).Jul 20 – 30Instrumented runs find the missing neighbour scan at the tail edge; the operator ruleswhich count is the intended denominator; only the scan fix goes in, and he commits it.
3 · USE CASE

Read-end profiles: lost work found, the thread picked up

Operator · Sep 2, 16:20 UTC — new session (rnd cluster)In this directory the work was done on correcting the fraction value in the read-end profile thatwas initially computed improperly. The plot_*.py scripts are files that are created in the course ofthis work, along with others. This work was carried out by the scidm-claude. However, I can notfind any traces of it (sessions, trac tickets). I need to continue/complete the work. Theclaude-code sessions contain the valuabel context that is needed for continuation. Please helpme find the session or it's trancsript/context snapshots that would allow restoring the context andcontinue the work. What remains is to merge the min_mapping_qv overrides into the BED file thatalready has new read-end profiles merged, and validate the resulting BED.Agent · 16:23 (excerpt)Found the session. Here is the recovery path. […] Session ID:263fa407-787b-4dcc-9978-0f5332c541f5 […] Timespan: 2026-07-16 → 2026-07-31 […]Important: the resume-claude.sh in this directory (claudeco --resume 263fa407-...) will fail —the harness needs the local project-dir JSONL, which has been rotated out. The archives areread-only artifacts, not resume sources. […]Recommended recovery command […] metem load 'metadata/ngs-read-end-profiles@latest'This gives you the complete conversation history […] in a new session with the same CWD.Sep 2 – 26The context is restored from the snapshot, and the profile ships the same day. QAthen reports per-edge fraction sums above 1, and a code audit files defect tickets.Operator · Sep 26, 22:21 UTC — new session (rnd cluster)In the 3 sessions in this project dir we were working on fixing the read-end profiles for tmap. Pleaseread the kage orientation on thermo projects and find if there are any records relevant for theread-end profiles, read them. Also please read the transcripts of the sessions.Recently few problems that we did not find earlier were found in the profiles. First, when testing thefixed profiles, QA team found that for some amplicons the sum of fractions exceeds 1.0 . Second, 3defects were found by independent audit of the code, recoreded as egdev tickets #415, $414 and#413.These findings are the scope of work in this session: we need to fix the found bugs, figure out if theyare responsible for the sum (fractions)>1 problem or there are more defects, and generate the branchthat can be merged into master.BEfore proceeding, we need to inventory the code.1 Please find (from reading the transcripts and by direct investigation) if the branch in/rhome/dkaznadzey/code/ion/dev/csd-ts-rnd-tools/read-ends-ovl-ampls worktree was merged intomaster (checked out to /rhome/dkaznadzey/code/ion/master/csd-ts-rnd-tools).2. please read #416. My understanding is that tmap_support is the live directory andREPAiR_guidance_pipeline is stale; please check if true.Agent · 22:26 (excerpt)Inventory report […] Kagebunshin read. [[thermo-projects-handling]] — Thermo GitHubownership wall (no git commit/push/fetch/pull) […] No pre-existing kage record specifically on theread-end profile fix. […]Transcripts read. […] Prior work established: […] The succeeding-scan fix was committed asc91dfa8b "Fix for IRS-65789" on the read-ends-ovl-ampls branch […]Question 1 — is read-ends-ovl-ampls merged into master? No. […] git log master..HEAD on thebranch worktree returns only c91dfa8b — one commit ahead of master. […]Question 2 — is tmap_support/endstat live and REPAiR_guidance_pipeline/endstat stale? Yes,#416's claim is correct.
3 · USE CASE

Read-end profiles: profile reproduced, swapped rule caught

Sep 27 – 28Each defect is fixed with a test slice and a dual-agent review. Plots from the fixedcode come out far below the QA plots. The operator finds that the QA profile wasbuilt from pre-fix statistics and asks whether the July consolidator masked thebimodality.Operator · Sep 28, 09:07 UTC- Please compose (in the separate directory, out of the git worktrees) the package sufficient to run theprofile generation pipeline matching the version you expect was used for generation 07_20_26.bed- Please demonstrate the consolidator from this package produces data identical toOncomine_Focus_v2_4.0.20260211.GX.designed.re-profile.07_20_26.bed; if it does not, try to identifythe source of the differences.- Please compute the profiles using the version c91dfa8b (verify that this is the version with theoverlap fix committed, no newer fixes; checkout this revision into a separate worktree underdev/csd-ts-rnd-tools/), save the resulting BED in a separate new directory, capture the stderr/stdoutfor this computing as files in the same directory (model command afterread_ends_consolidate.07_20_26.sh). Please compute the distribution graphs on it using xiaoxi's scriptand your script, save in the same directory.Agent · 09:18 (excerpt)All three deliverables complete […] read_end_stats_consolidate.py — master's file with the 5-lineratio = count / e_cnt load-time recomputation patched in. […] identical to the .tot hand-patchedfile on itd-detector-3-proto (which is what actually produced Xiaoxi's bed on July 20).Verification — BYTE-IDENTICAL reproduction ✓ […] Same md5: 14526c796487cedff936d2039de231ef[…] diff -q: no output (files identical)What the plot shows. c91dfa8b's distribution peaks near ~0.5 with a broad spread from ~0.2 to ~0.85— the bimodal-shifted "half" cluster shape of count / r_cnt on pre-collector-fix stats.Operator · 14:07 (first paragraph)Perfect, the pre-fix distribution shape is reproduced, as expected, and the post-fix consolidator doesnot hide it.Sep 28The fraction is traced end to end. At the operator's request, tickets are filed for a newdenominator and for a quality-trim allowance whose rule he spells out; both areimplemented and reviewed as specified.Operator · Sep 28, 21:39 UTCThanks. Let us double check I did not make a mistake when specified the quality trim confition. Tmapuses (read_len >= ZA-5). What do we use?Agent · 21:39, under a minute later (excerpt)Yes, there is a discrepancy — the two are backwards. […]The two conditions have ZA and read_len swapped on the addition side — they are opposite. […]Interpretation: the ticket #423 body swapped the operands. […] If the intent of #423 was to alignqualification with tmap, the predicate should be flipped. […] Want me to apply the flip and re-run thereproducers + pilot to compare?Operator · 21:40apply the flip and re-run the reproducersSep 29 – Oct 7The automated PR review flags the pooled method's denominator; the agentconfirms and fixes it. In the full-cohort run the agent notices that theper-method consolidations are byte-identical (the method flag is missing fromthe operator's driver script) and re-runs them. PR #79 is merged Sep 30; after aTMAP --repair sanity run the profile is committed Oct 7.
4 · OUTLOOK

Outlook: Agentic Framework v2

v1 · scidm-claude todayIn daily production — and reached by accretion: a continuouslyemerging prototype.Evolutionary debtA defect gets a guard; no guard is ever removed.Each change sees only its local area; nobody owns the overall shape.Rules are prose the agents are asked to honour; nothing checksthem.Superseded mechanisms stay in place and drift apart.“The working assumption that agent training plus prose guidance inkage and shin would be sufficient guards against over-engineering andsolution redundancy has proven false.”— operator assessment, #1312v2 · Agentic Framework, design principlesCapability inventory first: inherit the ideas, not the code.Harness-agnostic core: the agent CLI becomes a replaceable driver.Enforced contracts: every invariant that matters is checked by tooling.One mechanism per concern; deletion is rewarded work.Complexity is measured — e.g. time-to-change for a fixed referencetask.Design review gates the mechanism, not only correctness.Carried over, generalizedshared memorytranscript archivesnapshots and forksprompt assemblyorchestrationcodified protocolsfleet deploymenttracker + version controlsecrets and isolationcontext chainsStatus: design stage (Epic opened 2026-09-12). v1 stays in production throughout; memory, archive and snapshots carry over; no big-bang cutover.

Thank you

Questions?
scidm-claude — Agentic Framework for Secondary Analysis R&D:
the DIY experience
Denis Kaznadzey · 2026-10-15
PLATFORMRECORDKNOWLEDGECONTEXTCODINGRESEARCHDOCUMENTClaude Code