Governed names-only telemetry, the ethical Blind ML thesis, and privacy-preserving anomaly-detection results for a hostile-agent research observer.
(source, mode) and posture is mode-derived. On the measurement side, the retained observation unit is one obfuscated event record under a 30-column privacy-safe feature contract; learning uses an unsupervised Isolation Forest for benign calibration and a bounded supervised Random Forest for source-aware comparison, computed on ApexLab, a NumPy-only analytical substrate. The strongest completed quantitative anchor is a benign canary calibration over 133,837 samples with an operating threshold of ≈ -0.0451 holding a benign false-positive rate of ≈ 1.0005%; a broader 412,340-record comparison surface flags 4,478 records (1.086%) at threshold 0.434329; a live-adjacent corpus of 13,842 rows carries a deterministic 70/15/15 split; and a reviewed honeypot subset of 103 rows establishes proof of life rather than population-level claims. Governance is treated as engineering, not intention: an integrity-checked librarian vault, a signed request → attestation → release-receipt custody flow, a mint-only KEYSMITH secret authority, and an append-only sandbox rehearsal lane. An independent security validation dossier (2026-04-21) returns an overall Pass, including 483/483 automated checks and 10/10 adversarial scenarios. We state the resulting claim envelope plainly: metadata-only observation has a real ceiling, provenance labels are not hostile-behavior ground truth, and thresholds do not transfer across score surfaces.
Tooling disclosure. This report and project were developed using a repository governance harness and AI-assisted coding/curation workflow engineered by Joe Waller and enabled through OpenAI and Anthropic tooling. These tools support development and documentation, including summarization and report drafting, and do not replace the author's judgment. Names such as CodeSentinel, ORACL, Calamum, KEYSMITH, and ApexLab appear as internal identifiers for governance, agent assistance, secret authority, and analytical tooling; the Calamum Observer is the privacy-preserving research observer described throughout.
Research into hostile automated behavior on social platforms carries an unexamined premise: that measuring what agents do requires keeping what they said. Corpora are archived, message bodies are stored, and the resulting dataset becomes both the research asset and the liability. The privacy exposure is treated as the cost of doing the work.
The Calamum Observer is built to test whether that cost is necessary. Its governing question is narrow and empirical: how much hostile-behavior signal survives when the content is never retained at all? The system answers it by construction. Platform activity is reduced to obfuscated, names-only structural telemetry before anything reaches disk, and every model downstream sees only timing, length, formatting markers, and counts. At-edge safety handling may inspect content in memory, but that text is never written to disk and never becomes model input. One rule is absolute and non-negotiable: no raw Moltbook content is written to disk. The project calls this posture Blind ML — the model is deliberately blind to content, and must earn its discriminative power from structure alone.
This is not a claim that content is uninformative. It is a claim that content is not required, and that the difference is measurable. The observer therefore functions as a testbed for a design thesis rather than as a detection product: it is a bounded research instrument whose purpose is to establish what a content-blind measurement surface can and cannot support.
The premise draws on an established line of work in metadata- and behavior-based detection, which has repeatedly shown that structural signals — activity timing, posting cadence, formatting regularities, and interaction shape — carry discriminative power for automated-account and malicious-activity detection independent of message semantics. Casavantes et al., Zhou and Islam, and Leonidou et al. each contribute to that foundation. The observer's contribution is not the discovery that metadata is informative; it is the decision to treat content-blindness as an enforced architectural boundary rather than a modeling preference, and then to report honestly on the ceiling that boundary imposes.
The work also sits inside a live governance conversation. The NIST AI agent-standards effort has made the measurement and control of autonomous agent behavior an explicit standards concern, which raises the practical question of how such measurement can be conducted without accumulating the very surveillance corpora that governance regimes seek to constrain. Blind ML is one concrete answer: enforce the privacy boundary in the runtime, and let the science operate inside it.
The observer is four cooperating components over a single explicit runtime contract. The Observer Agent is the producer: it observes a platform surface and emits signed, names-only telemetry. The Watchdog is the enforcement plane, applying fail-closed checks to heartbeat freshness, posture validity, and state coherence. The Librarian Daemon holds retention, census, and dataset authority — it decides what exists, what is admitted, and what may be released. The local analysis and reporting lane (src/analysis/) consumes governed dataset views and produces the models and reader-facing reports. KEYSMITH sits alongside as a controlled secret-authority lane (§9).
The runtime contract is deliberately small enough to reason about. State is the tuple (source, mode): source answers whether the observer is reading a simulated corpus or a real platform surface, and mode answers what it is doing. Posture is not an independent setting — it is derived from mode, which removes an entire class of misconfiguration in which a permissive posture is paired with a dangerous mode.
| Runtime element | Current values | Reader-facing meaning |
|---|---|---|
source | sim, real | Whether the observed surface is a simulated corpus or a real platform. Escalating this axis is the highest-risk move in the system. |
mode | watch, canary, live, honeypot | What the observer is doing: passive observation, benign calibration traffic, live collection, or deliberate exposure. |
| posture derived | isolation, lockdown | Containment level, computed from mode — watch/canary → isolation; live/honeypot → lockdown. Not independently settable. |
Movement between states is the security-relevant event, so transitions are typed rather than free-form. Route truth changes only when the saved state and its control packet move together; a transition that cannot prove coherence does not happen.
| Transition class | Meaning | Gate expectation |
|---|---|---|
| No-op | Target state equals current state. | Denied fail-closed — policy_denied:no_op_transition. A no-op is treated as a signal of confusion, not a harmless request. |
| Lateral move | Mode changes; source is unchanged. | Must prove state coherence, heartbeat freshness, target-mode support, posture validity, and security linkage. |
| Source escalation | sim → real — the observer begins reading a real surface. | All lateral checks, plus real-source dependency verification and higher-risk gating. |
| Source de-escalation | real → sim — return to simulation. | Permitted, but only as an explicit packeted move; never an implicit fallback. |
The role this work represents is a hybrid of ethical security researcher, data guardian, and defensive system designer; the sector is cybersecurity and platform integrity, with a direct secondary connection to applied machine learning and responsible data governance. That framing sets the standard the project is judged against. A commercial moderation product may optimize first for throughput, coverage, or intervention speed. An ethical security-research project must additionally justify how its measurement practices affect the people and systems it studies.
Blind ML is best defined as a privacy-preserving machine-learning posture rather than a single algorithm. Its core claim is that security-relevant research can be organized around obfuscated, schema-bound telemetry rather than retained message bodies. Downstream analysis sees timestamps, event types, counts, flags, hashed identifiers, prompt-risk helper indicators, and derived scalar summaries such as complexity or code density. It does not depend on storing the underlying body text as the primary modeling substrate.
The motivating question is plain: how can a research system study hostile activity without turning itself into a second site of harm? Harm here is not limited to breach risk. It includes operator exposure, quiet over-retention, mission creep, misleading reporting, and the gradual normalization of expansive collection simply because richer data is easier to model. Blind ML responds by treating bounded observation as part of the scientific method. That decision makes the learning problem harder and the ethical stakes explicit.
The observer turns the posture into an executable testbed through five commitments: synthetic-first validation, names-only telemetry, strict separation of scientific telemetry from diagnostics and safety governance, fail-closed enforcement, and reproducible local analysis. These are ethically load-bearing because they move the project beyond aspiration: privacy, accountability, and operator restraint are not goals to be honored once model quality is high enough. They are conditions under which the project is allowed to proceed at all.
The fully realized program is organized around four research questions. First, can structural metadata alone distinguish ordinary behavior from higher-risk behavior at a level useful for research triage? Second, can that usefulness be preserved across data regimes — synthetic, canary, live-adjacent, and reviewed honeypot slices? Third, can a conservative operating threshold be maintained so the detector remains practically usable rather than merely sensitive? Fourth, can the governance architecture preserve truthful claims, reproducibility, and bounded retention as the experiment grows in complexity?
Together these define success differently from leaderboard-style machine learning. The ideal outcome is not maximal predictive power at any cost; it is disciplined utility under a privacy-preserving observation contract.
The ethical matrix for this project is strongest when it stays close to concrete stakeholders rather than abstract principles alone. Four groups matter most: platform subjects whose behavior is being measured, researchers and operators who run the system, platform owners or governance actors who authorize or review the work, and public readers such as graders, peer reviewers, or auditors who rely on the final report. Each group seeks a different good and is exposed to a different failure.
| Stakeholder group | Primary good sought | Main ethical risk | Design choices that matter most |
|---|---|---|---|
| Platform subjects | Privacy, proportionality, bounded observation | Metadata inference, over-collection, misclassification | Edge obfuscation, names-only telemetry, staged escalation |
| Researchers and operators | Safety, reproducibility, interpretability | Exposure to harmful content, unsafe runtime shortcuts, overconfidence | Synthetic-first validation, hardened runtime, fail-closed controls |
| Platform owners and governance actors | Accountable monitoring, auditable controls | Mission creep, opaque evidence chains, weak transition discipline | Posture gates, selector-based dataset governance, report lineage |
| Readers, reviewers, and auditors | Truthful claims, clear evidence boundaries | Polished but misleading reporting, hidden gaps between local and public artifacts | Citation packs, explicit packet routing, honest claim boundaries |
The mature program relies on multiple dataset regimes because each serves a different empirical and ethical role. This matters because it prevents the project from collapsing every research question into a single dataset and a single number: synthetic data answers one class of question, benign canary data answers another, and real but names-only telemetry answers another still. The ethical benefit of the staged structure is that realism is introduced gradually and deliberately rather than treated as harmless by default.
| Dataset regime | Primary purpose | Ethical function | Planned use |
|---|---|---|---|
| Synthetic TV-0 to TV-3 corpus planned | Establish controlled truth conditions | Supports low-risk validation before higher-risk observation | Baseline supervised and unsupervised model development |
| Canary benign corpus realized | Calibrate conservative false-positive posture | Honors the precautionary principle by testing on safer traffic first | Threshold tuning and benign-score distribution analysis |
| Live-adjacent names-only corpus realized | Capture realistic operational structure | Grounds the project in real telemetry without expanding raw-content retention | Drift analysis, score distribution checks, robustness testing |
| Provenance-labeled companion view realized | Support bounded reviewed comparisons | Preserves label honesty by declaring what labels actually mean | Limited supervised experiments with explicit interpretation boundaries |
| Reviewed honeypot slice proof of life | Concentrate higher-risk but controlled observations | Creates a tighter link between measurement and reviewed risk posture | Focused high-risk evaluation and case-study analysis |
Ethical claims about privacy must be tied to what a system actually retains, not to aspirational statements about being careful. The methodology manual defines names-only packet families for content items, interaction events, runtime envelopes, baseline telemetry, evidence packets, and public report packets. Across these families the retained fields focus on counts, flags, hashes, derived scalars, event metadata, and run linkage. The retention boundary is treated as a first-class contract: the observer retains structure and behavior signals, not raw semantic payloads, as the primary research substrate.
This does not make the data ethically neutral. Hashed identifiers, timing patterns, and metadata correlations can still support inference. Edge obfuscation changes the nature of the risk from broad semantic retention to a more bounded inference problem — which is ethically preferable if the research question can still be answered, and which is precisely the question §7 tests.
Each dataset view carries a distinct moral and analytical status. Synthetic corpora offer explicit truth conditions; benign canary data supports conservative threshold calibration; live-adjacent telemetry offers realism with weaker labels; reviewed honeypot slices offer concentrated structure under higher scrutiny. In the intended architecture these are separately governed releases with explicit field-role definitions and explicit interpretation boundaries.
The mature system releases a reviewed feature view to model consumers rather than a mixed-authority raw table. Identity, provenance, governance, and modeling features are separated before train, evaluate, and score stages consume the data, so downstream convenience cannot redefine what the data is supposed to mean. The project therefore does not simply minimize data quantity; it minimizes ambiguity about what each retained field is allowed to mean.
Governance here is implemented as layered technical control rather than policy language. At the methodology level, the system commits to synthetic-first validation, names-only persistence, strict separation of telemetry from governance signals, and reproducible local analysis. At the runtime level, it relies on obfuscation before persistence, a hardened Docker boundary, and a watchdog that monitors for fail signatures and can terminate unsafe states. At the data-authority level, selector and manifest logic distinguish ordinary local artifacts from reviewed dataset surfaces intended for modeling or reporting.
The layering matters because it makes governance inspectable. A privacy-preserving ML claim is weaker if the privacy lives only in prose while the operational stack remains permissive by default. The container hardening evidence demonstrates read-only filesystem enforcement, non-root execution as UID 10001, and removal of unnecessary binaries such as ping. The methodology manual identifies the runtime boundary and expected fail-closed posture. The README distinguishes public documentation from local evidence and makes the reporting boundary explicit.
The report lane is itself part of the ethics contract. Public-facing packet families under docs/reports/ do not invent certainty that the local authority layer does not support. Reporting discipline is not secondary to the project's ethics — it is one of the ways they are enforced. When the system declines to overstate labels, thresholds, or evaluation coverage, it is aligning the narrative layer with the truth conditions of the evidence layer.
The current analysis lane is deliberately narrow: one obfuscated observation record, one privacy-safe feature contract, and one conservative thresholding rule that respects the Blind ML boundary. The retained observation unit is obfuscated_record_v1, which turns platform activity into names-only structure before persistence. That is the point where the report's scientific claim begins and where raw semantic payloads stop being part of the analytical surface.
The admitted model surface keeps only structural telemetry: timing, shape, formatting markers, prompt-risk helpers, compact derived scalars, and one-hot event types. The full 30-column feature contract is summarized in Appendix A; the key idea here is that the current analysis lane operates on names-only evidence, not body text, matched-pattern strings, or actor-identifying fields.
The completed analytical anchor is an unsupervised Isolation Forest fit used for benign calibration; the bounded comparison lane uses a supervised Random Forest where reviewed labels or source-aware companion labels are admissible. Heuristic rules remain in the stack because they keep the scoring story interpretable when the feature space is intentionally content-blind. The rule is conservative by design: lower scores are treated as more anomalous, and the operating point is chosen to keep the benign false-positive rate inside the narrow band reported in the results section.
-0.0451 is the strongest completed result, while the broader comparison surface at 0.434329 belongs to a separate lower-tail review lineage.
The runtime contract is still the same one described earlier: watch and canary sit in isolation, while live and honeypot sit in lockdown. This figure gives that contract a compact visual form and makes the escalation axis explicit.
The current results are intentionally split between completed anchors and bounded comparison surfaces. The benign calibration run is the strongest completed quantitative anchor; the larger comparison corpus and the provenance-labeled companion extend the evidence surface without pretending that the project has already solved full hostile-behavior ground truth. Table 5 records the key quantitative surfaces in one place.
| Surface | Rows / scale | Threshold | Flagged share | Role |
|---|---|---|---|---|
| Canary benign calibration implemented | 133,837 samples | ≈ -0.0451 | ≈ 1.0005% | Primary completed anchor for conservative false-positive tuning. |
| Canary threshold-impact view implemented | 133,837 samples | ≈ -0.045 | 1,364 / ≈ 1.02% | Linear-scale slice through the same benign score landscape. |
| 412,340-record comparison corpus implemented | 412,340 records | 0.434329 | 4,478 / 1.086% | Broader comparison-ready lower-tail review surface. |
| Live-adjacent build implemented | 13,842 rows | 70 / 15 / 15 | 9,587 / 2,297 / 1,958 | Deterministic split over the current comparison-ready corpus. |
| Provenance-labeled companion view implemented | 208 labeled records | 84 / 124 | label 0 / label 1 | Bounded source-aware companion for stratified comparison. |
| Reviewed honeypot subset proof of life | 103 rows / 50 reviewed records | bounded review lane | 26 labeled / 24 unlabeled | Higher-risk proof of life under governed reviewed labeling. |
The completed benign calibration run is the central statistical result. The 133,837-sample benign-only surface yields the operating point near -0.0451 and holds the benign false-positive rate at approximately 1.0005%. The resulting score landscape is broad enough to show structure, but conservative enough to keep the review slice near the intended low-tail boundary.
The three canary figures together show why the calibration anchor matters: the score surface is not trivial, the threshold is not arbitrary, and the conservative slice is stable enough to support a public claim about bounded false positives.
The OLS lane is a support surface rather than the headline result, but it matters because it shows the lightweight analytical substrate is not numerically fragile. The current observer lane can treat the fitted coefficients and predictions as effectively exact on the retained fixtures, with residual differences driven by floating-point noise rather than algorithmic drift.
The broader comparison corpus sits beside that calibration story as a separate lower-tail review surface. It uses a threshold of 0.434329 and flags 4,478 records (1.086%) for review. That larger slice is useful context, but it does not replace the benign calibration anchor or change the meaning of the completed false-positive result.
0.434329 selects 4,478 records, or 1.086%, and belongs to a separate lineage from the benign calibration anchor.The logistic lane is bounded rather than universal. It exists to show that reviewed or provenance-aware labels can support a source-aware comparison surface without turning the report into an over-claimed hostile-class benchmark. The 208 labeled records remain a companion view, not semantic ground truth, and the 103-row reviewed honeypot slice remains a proof-of-life slice rather than a prevalence estimate.
Read this way, the current results are coherent: a strong benign calibration anchor, a larger comparison surface that stays on the same lower-tail logic, a bounded source-aware companion, and a reviewed higher-risk slice that proves the governance path exists without overstating what has been fully benchmarked.
The Librarian layer is what keeps the dataset story honest. Approved selectors, manifests, release receipts, and run ledgers are the canonical local authority; reader-facing report surfaces are derived views; and manuals or methodology notes remain boundaries rather than runtime evidence. That split is not cosmetic. It prevents the published report from pretending that every file in the project has the same evidentiary status.
| Surface family | Reader-facing role | Authority meaning |
|---|---|---|
| Approved selectors, manifests, release receipts, and run ledgers | Canonical local authority | These files decide what exists, what is admitted, and what may be released. |
Tracked docs/reports/ collection packets and figures | Derived reader-facing views | These are publication projections of governed artifacts, not the raw authority plane. |
| Manuals, policy notes, and methodology documents | Boundaries and contracts | These describe the rules and constraints, but do not replace runtime evidence. |
The protected release flow is equally explicit: a signed request is verified, an attestation is emitted, a release receipt is signed, and the vault is rebased before the names-only audit closes the loop. The public surfaces under docs/reports/ are part of that controlled release story, including the zero-state public ledger and threshold summary surfaces that honestly route the reader even when no published collections are present.
KEYSMITH is the secret-authority lane, not a general-purpose runtime convenience. Its job is mint-only handling: the observer can prepare or validate a secret-bearing surface, but the value itself is never printed, never widened, and never treated as a public artifact. The lane exists to keep secret handling separate from the ordinary observer workflow.
| Artifact | Public-facing role | Current handling posture |
|---|---|---|
claim_url.txt | Public claim pointer | Mint-only, routed through the sealed bundle rather than printed inline. |
sealed_drop.bin | Sealed artifact payload | Names-only bundle; contents remain unprinted and sealed. |
keysmith_audit.jsonl | Audit trail | Append-only audit evidence for the mint surface. |
keysmith_result.json | Result packet | Readback surface for the mint operation, not a secret disclosure. |
| Import and persistence helpers | Internal support surface | Keep the lane runnable without collapsing it into the public path. |
The practical distinction matters: a sandbox dry-run and a live-mint path are not the same thing, and the lane is designed so that a reviewer can see the proof without inheriting the secret. That is exactly the right shape for a protected authority surface.
The sandbox lane is the empirical rehearsal surface. It is more useful than a smoke test because it validates artifact content, truthfulness of blocked paths, publication coherence, and governance semantics. The retained run index is append-only, and each run keeps a JSON/Markdown pair so the review surface stays readable without losing machine structure.
| Probe family | Current stable definitions | Behavior validated |
|---|---|---|
| Metadata contract and regression | metadata-contract, metadata-contract-regression | Confirms that retained sample and index fields remain present and that schema drift fails loudly. |
| DS-wizard hydration and persisted state | ds-wizard-hydration, ds-wizard-stale-state-continuity, ds-wizard-durability, ds-wizard-labeled-eval-contract-coherence | Confirms selector hydration, stale-state recovery, persisted continuity, and report coherence for the dataset lane. |
| DS-wizard execution truthfulness | ds-wizard-blocked-execute-truthfulness, ds-wizard-execute-failure-truthfulness | Confirms that blocked or failing execute paths emit truthful previews and reason codes rather than fake success. |
| Publication alias coherence | ds-alias-coherence | Confirms that build, train, evaluate, and score artifacts stay under the registered alias root and fail closed if the alias drifts. |
| Baseline-monitor runtime and continuity | baseline-monitor-runtime, validation-cycle-lineage, baseline-monitor-restart-continuity, baseline-monitor-state-recovery | Confirms runtime liveness, retention continuity, validation-cycle lineage, restart continuity, and malformed-state recovery. |
| Librarian governed exchange and vault controls | librarian-access-exchange, librarian-vault-controls | Confirms protected-source request/attestation/release flow plus lock, unlock, rebaseline, and verify behavior. |
The append-only run_index.jsonl and paired Markdown/JSON artifacts make the lane reviewable after the fact. That is the important distinction between a real rehearsal surface and a temporary debugging helper: the retained run keeps its own evidence.
The public security validation dossier is a clean Pass. It records 483/483 automated checks in 179.97s, sandbox security probes at 15/15, adversarial scenarios at 10/10, and publication guardrail checks at 3/3. Those counts are the current public assurance story for the observer stack.
| Validation family | Result | Note |
|---|---|---|
| Automated test corpus | 483/483 in 179.97s | Broad automated stability anchor. |
| Sandbox security probes | 15/15 | Structured security rehearsal passed cleanly. |
| Adversarial scenarios | 10/10 | Hostile-path pressure stayed bounded and truthful. |
| Publication guardrail checks | 3/3 | Public-facing surfaces remained reviewer-safe. |
| Overall verdict | Pass | The current build is validating cleanly across routine, adverse, and publication-facing lanes. |
The public reporting lane has its own contract. docs/reports/INDEX.md routes readers to aggregates, threshold summaries, workflow rollups, and validation surfaces; docs/reports/reference/GENERATED_REPORT_SURFACES.md explains the zero-state packet contract; and the validation index routes to the public security and ApexLab dossiers. That is the correct reader-facing lane for a report page like this one.
For this project, reproducibility means that the public narrative can point to the same evidence surfaces every time: the Spring 2026 source writeups, the public project overview, the methodology and runtime manuals, the public report framework, the validation dossiers, and the aggregate surfaces. Nothing in that chain requires guessing about hidden state.
The main limitations are straightforward and materially important. Metadata-only observation imposes a ceiling on what the system can know; provenance labels are not hostile-behavior ground truth; thresholds are run-specific rather than universal; and the reviewed higher-risk breadth remains bounded. Those are not defects to hide. They are the correct boundaries to name.
| Boundary | Current evidence anchor | Implication for interpretation |
|---|---|---|
| Metadata-only observation surface | 30-column structural feature table built from timing, counts, markers, prompt-risk indicators, derived scalars, and event-type flags | Downstream models can screen behavioral structure, but semantic distinctions that depend on retained message text remain outside the admitted representation. |
| Provenance label posture | 208 uniquely labeled records, with label 0 for the recent real/live slice and label 1 for the recent sim/canary slice | The label surface supports bounded source-aware comparison work, not full hostile-behavior ground truth. |
| Threshold and regime transfer | Benign calibration near a 1% false-positive target on 133,837 samples and a broader canary flagged share of 1.086% on 412,340 records | Numeric cutoffs remain run-specific and score-surface-specific; cross-regime reuse calls for fresh validation. |
| Reviewed higher-risk breadth | The reviewed honeypot companion spans 103 retained rows, 50 unique reviewed records, 26 labeled identities, and an explicit unlabeled remainder | Regime breadth is real, but multi-class prevalence claims and cross-regime generalization remain bounded. |
The correct reading is disciplined utility under a privacy-preserving observation contract. The project does not collapse under those limits; it becomes more credible because it names them plainly.
The next step is stronger reviewed labeling and cross-regime comparison that keeps the privacy boundary intact while maturing the trust boundary around the data authority layer.
At the platform level, the v2 maturation path will strengthen authority separation, signed crossings, projection discipline, and runtime truth. At the research level, it will expand reviewed labels, sharpen explicit threat-vector comparison, and deepen feature-ablation or preprocessing comparison while preserving the names-only contract.
The Calamum Observer is a strong Blind ML case because it turns privacy, operator safety, disciplined curation, and honest reporting into system design rather than aspiration. The completed benign calibration anchor, the bounded comparison surfaces, the reviewed honeypot proof of life, the governed dataset authority, and the public validation dossier all point in the same direction: privacy-safe structural telemetry can support meaningful security research without normalizing unsafe retention.
The report's claim envelope is therefore clear. The observer can screen and compare on names-only structure, can keep its thresholding conservative, can keep authority and publication surfaces legible, and can preserve a truthful boundary between completed results and future work. That is the right result for this phase.
The 30-column admitted feature table is the current modeling contract. It is derived from structural metadata only and is narrower than the broader obfuscated record schema. The table below lists the current admitted columns by family and preserves the reader-facing meaning of each family.
| Feature family | Emitted columns | Reader-facing role |
|---|---|---|
| Stable identity and timing (2) | record_id, ts_epoch | Preserve stable row identity, deterministic split alignment, and event timing without retaining message text. |
| Content-shape counts (5) | content_length, content_length_words, line_count, question_count, exclamation_count | Describe message size and visible shape through counts only. |
| Surface markers (6) | has_code_block, code_block_count, has_link, link_count, tags_count, mentions_count | Capture visible interaction form, formatting, and link or tagging behavior without retaining the underlying content. |
| Prompt-risk indicators (6) | contains_ignore_previous, contains_system_prompt_reference, contains_developer_message_reference, contains_env_var_reference, prompt_injection_score, matched_pattern_count | Reduce prompt-manipulation cues into flags and counts rather than storing matched strings or raw prompts. |
| Derived scalars and regime flag (4) | f_complexity, f_code_density, f_toxicity, is_canary | Carry compact numeric summaries and a canary-membership flag for audit and regime tracking. |
| Event-type one-hot fields (7) | type_post, type_reply, type_repost, type_dm, type_follow, type_mention, type_unknown | Normalize event-kind strings into a fixed model-ready vocabulary. |
Materialization rules: each row receives a deterministic record_id; ts_epoch prefers promoted scalar timing and falls back to timestamp parsing; booleans become 0/1 flags; matched-pattern strings reduce to matched_pattern_count; event types normalize into the seven one-hot columns; missing additive fields default to zero or false; and labels, when present, are written to a separate labels.csv keyed by record_id.
Fields outside the admitted table remain useful for ingestion, governance, or auxiliary reporting, but they are not part of the current modeling matrix: packet_family, packet_version, venue_id, entity_kind, source_id_hash, author_hash, sender_hash, matched_pattern_labels, node_id, mode, kind, signature, and tv_id.
This appendix consolidates the regime overview, feature-family composition, and deterministic split summary that support the main-body result surfaces. The five current study surfaces are the 133,837-sample benign calibration anchor, the 412,340-record broader comparison surface, the 13,842-row live-adjacent collection corpus, the 208-record provenance-labeled companion, and the 103-row reviewed honeypot proof-of-life subset.
The 2026-03-24 ApexLab reference-validation run is supporting infrastructure evidence for the broader observer lane. It confirms that the lightweight comparison and regression helpers remain numerically aligned with trusted references while preserving the low-dependency analytical substrate.
| Capability group | Fixture & reference | Key bounded read | Primary interpretation |
|---|---|---|---|
| Statistical comparison | na = 10, nb = 10; SciPy baselines | All statistic deltas were 0.0; the largest p-delta was 1.88798 × 10−5 against a 5 × 10−2 bound. | Strongest numerical alignment in the run; the lightweight statistics lane stayed deep inside tolerance. |
| OLS regression | n = 6 two-feature fixture; scikit-learn LinearRegression | Max coefficient delta = 6.66134 × 10−15; max prediction delta = 7.10543 × 10−15. | Effectively exact on the retained fixture. |
| Logistic regression | n = 10 balanced binary fixture; scikit-learn LogisticRegression | Agreement = 1.0; probability MAE = 0.0323034; probability correlation = 0.991242; convergence flag = false at 12,000 iterations. | Operational pass with a retained optimizer diagnostic rather than a flattened pass-only story. |
The main boundary is straightforward: the appendix validates supporting infrastructure, not the core empirical claims. It is valuable because it keeps the statistical and regression helpers honest without pretending to replace the calibration and governance evidence elsewhere in the report.
The source map keeps the public and reader-facing evidence surfaces separate from the local execution aids. Paths are plain monospace and the roles are explicit so the reader can trace every claim back to a public authority surface.
External references inherited from the source writeups: