instrumentation-specdraft

Instrumentation Spec: Curator session loop (pilot)

Overview

Feature: trustworthy session loop (intake → analysis → editable send) + dossier.

Analytics Goals: 1. Is the output trusted? (proxy: follow-ups sent with light edits) 2. Is the analysis reproducible and human-comparable? 3. Does it save coach time and get adopted weekly?

Analytics Platform: TBD (Amplitude/Mixpanel/Segment-class; small pilot may start with a simple events table). Naming Convention: snake_case: feature_action.

Event Inventory

session_processed

Field Value
Event Name session_processed
Trigger Analysis pipeline completes for a consented session
Description A session has produced an analysis artifact

Properties:

Property Type Required Description Example
coach_id string Yes Pseudonymous coach id c_8f2
session_id string Yes Pseudonymous session id s_41a
consent_present boolean Yes Consent recorded true
diarization_confidence number No Avg confidence 0–1 0.86
reproducibility_run boolean No Was a re-run comparison done true
extraction_f1 number No F1 vs gold (when sampled) 0.74

followup_sent

Field Value
Event Name followup_sent
Trigger Coach presses Send on a draft summary (delivery confirmed)
Description The NSM event — a trusted, client-facing follow-up went out

Properties:

Property Type Required Description Example
coach_id string Yes Pseudonymous coach id c_8f2
session_id string Yes Pseudonymous session id s_41a
edit_distance_pct number Yes % change from draft to sent (trust proxy) 12
channel string Yes Delivery channel email
time_to_send_min number No Minutes from analysis ready → sent 8

dossier_opened

Field Value
Event Name dossier_opened
Trigger Coach opens a client dossier (esp. pre-session)
Description Continuity usage signal

Properties:

Property Type Required Description Example
coach_id string Yes Pseudonymous c_8f2
pre_session boolean No Opened within prep window true

User Properties

Property Type Description Set When Example
coach_cohort string Pilot cohort/start At pilot onboarding 2026_pilot_A
active_clients_band string 6–15 etc. (self-reported) Onboarding 6-15

PII & Privacy Considerations

PII Properties

Property PII Type Handling
coach_id / session_id indirect identifier Pseudonymous ids only; map kept in protected perimeter
client name / content sensitive Never sent to analytics; analytics carries metrics, not transcript content
  • No event references session content; only consented sessions are processed at all.
  • Analytics receives metrics/flags, never transcript text or client identifiers.

Data Retention

  • Analytics events: aggregate retention per policy; raw audio/transcript governed separately (default 90d, configurable; per studio page).

Implementation Notes

SDK/Integration

  • Platform: backend (pipeline emits events) + web (UI actions like send).
  • Initialization: pseudonymous id mapping lives server-side only.

Event Timing

  • followup_sent fires only on confirmed delivery, not on button click, so it can't over-count.
  • extraction_f1 populated only when a gold sample exists (sampled, not every session).

Testing Checklist

Event Validation

  • [ ] session_processed: process a consented test session → event fires with consent_present=true.
  • [ ] followup_sent: edit + send a draft → event fires with edit_distance_pct populated.
  • [ ] dossier_opened: open a dossier pre-session → pre_session=true.

Property Validation

  • [ ] No transcript text or client identifier appears in any event payload.
  • [ ] extraction_f1 within 0–1; only present on sampled sessions.

Edge Cases

  • [ ] No event references content when consent absent (session never processed).
  • [ ] followup_sent does NOT fire on send failure.

Debug Tools

  • Server-side event log; analytics debug view (platform TBD).