SpeakTrue

Conversation Workspace Plan

Status

Proposed implementation plan for a durable, replayable Turn-Based STS workspace across web, iOS, and Android.

Product Thesis

A communication turn should not disappear when the next recording begins. SpeakTrue should preserve the useful context around completed turns while retaining the speed of the current Finish & Speak flow.

The MVP is a revoicing workspace: captured speech becomes transcript text and generated speech. It is not a translation engine. In the current implementation, sts-realtime-finalize sends committed_transcript directly to tts-generate; source and spoken text are therefore identical unless a future transformation provider is explicitly added.

User Outcome

A user can:

1. start a conversation without configuring a document or project;
2. complete multiple turns with automatic or manual turn completion;
3. see each completed turn in order;
4. replay, share, or save any turn;
5. leave and resume the conversation on another signed-in surface;
6. delete a turn or the entire conversation.

Current-State Evidence

• Web turn state and the latest interpreter summary are managed in web/python-web-app/static/js/index_core.js and web/python-web-app/templates/partials/index_slow/sts.html.
• iOS owns a single transcribedText, latestGeneratedArtifact, and generated playback state in ios/SpeakTrue/STSViewModel.swift.
• Android owns equivalent latest-session/latest-artifact state in android/app/src/main/java/com/speaktrue/features/sts/presentation/STSViewModel.kt.
• Session creation uses sts-live-interpreter-session and sts-live-interpreter-tts-session; final artifact generation uses sts-realtime-finalize.
• sts_realtime_sessions is an expiring transport/accounting record. It must not be overloaded as the user’s durable conversation model.

Scope

MVP

• ephemeral multi-turn UI before schema work;
• durable conversation and turn records for authenticated users;
• replay, share, save-to-Soundboard, resume, rename, and delete;
• same-account sync across web, iOS, and Android;
• idempotent attachment of a finalized STS artifact to one conversation turn;
• content-free operational measurement and recovery states.

Later

• two-person mirrored display mode;
• speaker labels and diarized multi-speaker sessions;
• true translation with separate source and target text;
• collaborative/shared conversations;
• automatic summaries or LLM-generated responses;
• unauthenticated cloud sync.

Non-Goals

• replacing the existing batch or realtime STS modes;
• changing the canonical artifact response fields;
• persisting microphone PCM by default;
• editing the historical transcript in place after it has been spoken;
• treating an expiring signed URL as durable state.

UX Contract

Conversation screen

• The record control remains the primary action.
• A completed turn appears only after final transcript selection and artifact finalization have succeeded.
• A pending turn shows explicit states: listening, finalizing transcript, generating speech, ready, or needs attention.
• The newest turn can autoplay according to existing settings. Older turns never autoplay unexpectedly.
• Each ready turn offers Play/Pause, Share, Save to Soundboard, and More.
• More contains Speak edited copy, Delete turn, and diagnostic retry only when relevant.
• Starting the next turn never clears completed turns.
• Leaving an unsaved ephemeral conversation prompts only when durable persistence is unavailable; authenticated persistence should be automatic.

Editing

Speak edited copy creates a derived turn or revision with a new artifact. It does not rewrite the captured transcript. The UI visually links the derived version to its source turn.

Failure recovery

• If artifact finalization succeeds but conversation attachment fails, keep the artifact response locally and retry attachment with an idempotency key.
• If the app closes during attachment, resume from the finalized transport session on next launch.
• If the signed playback URL expires, resolve a new URL from artifact_path.
• If a conversation is deleted, do not delete a Soundboard clip previously saved from one of its turns.

Durable Data Model

Add a new migration; update 001_core_schema.sql only after the additive migration is accepted.

speech_conversations

Field	Type	Notes
id	uuid	Primary key
user_id	uuid	Owner; cascade on auth-user deletion
title	text nullable	User-editable; generated default stays local until saved
workflow_mode	text	Starts with live_interpreter
source_language_code	text nullable	Capture language hint
speech_language_code	text nullable	TTS language hint; does not imply translation
status	text	active, closed, deleted
turn_count	integer	Server-maintained convenience count
created_at / updated_at / last_activity_at	timestamptz	UTC timestamps

speech_conversation_turns

Field	Type	Notes
id	uuid	Primary key
conversation_id / user_id	uuid	Ownership enforced against parent
sequence_number	integer	Unique within conversation
transport_session_id	uuid nullable	Unique idempotency seam to sts_realtime_sessions
transcript_text	text	Captured final transcript
spoken_text	text	Text sent to TTS; equal to transcript in MVP
transformation_kind	text	none in MVP; no false translation claim
artifact_id / artifact_path	text nullable	Stable artifact identity/path
clip_metadata	jsonb	Canonical generation provenance
status	text	pending, ready, failed, deleted
derived_from_turn_id	uuid nullable	Links edited/re-spoken versions
created_at / updated_at	timestamptz	UTC timestamps

Do not persist audio_path or download_path; both are signed and must be resolved on demand.

Database protections

• RLS for select/insert/update/delete by owner.
• Trigger or RPC validation that turn owner matches conversation owner.
• Unique (conversation_id, sequence_number) and unique non-null transport_session_id.
• Bounded text lengths and valid status/transformation constraints.
• Soft-delete retention only if the retention policy is approved; otherwise hard delete owned metadata while preserving separately saved Soundboard clips.

API Contract

Prefer handler factories with dependency injection and stable error codes.

New functions

• conversation-create: create an owned active conversation.
• conversation-list: cursor-paginated summaries ordered by last_activity_at.
• conversation-get: conversation plus cursor-paginated turns; resolve signed URLs only for requested ready turns.
• conversation-update: rename or close.
• conversation-delete: owner-scoped delete with explicit artifact-retention behavior.
• conversation-turn-resolve-audio: resolve a short-lived URL from an owned turn artifact path.

Existing finalize extension

Extend sts-realtime-finalize with optional fields:

{
  "conversation_id": "uuid",
  "client_turn_id": "uuid"
}

After artifact persistence, call an owner-scoped database RPC that attaches the finalized session exactly once. The response keeps all existing artifact fields and may add:

{
  "conversation_turn": {
    "id": "uuid",
    "conversation_id": "uuid",
    "sequence_number": 3,
    "status": "ready"
  }
}

Old callers that omit conversation fields receive the existing response unchanged.

Stable new errors include conversation_not_found, conversation_closed, conversation_turn_conflict, and conversation_attach_failed. Provider errors retain current codes.

File Ownership Map

Backend

• Create migration under backend/supabase/migrations/.
• Update backend/supabase/migrations/001_core_schema.sql after rollout validation.
• Extend backend/supabase/functions/sts-realtime-finalize/.
• Add backend/supabase/functions/conversation-*/ handler/index/test directories.
• Reuse auth, CORS, usage, and signed-path helpers under backend/supabase/functions/_shared/.

Web

• Timeline state and orchestration: web/python-web-app/static/js/index_core.js.
• STS markup: web/python-web-app/templates/partials/index_slow/sts.html.
• Flask edge wrappers: web/python-web-app/src/routes/ and web/python-web-app/src/services/supabase_edge_client.py.
• Focused tests: web/python-web-app/tests/api/test_sts.py, web/python-web-app/tests/test_stt_sts_studio_ui.py, and web/python-web-app/tests/js/sts_live_voice_runtime_check.mjs or a new conversation runtime check.

iOS

• Models/orchestration: ios/SpeakTrue/STSViewModel.swift and ios/SpeakTrue/AIProxyService.swift.
• UI: ios/SpeakTrue/SpeechToSpeechView.swift.
• Playback reuse: ios/SpeakTrue/LiveVoiceAudioPlayer.swift.
• Tests: ios/SpeakTrueTests/STTAndSTSViewModelTests.swift plus focused conversation coding tests.

Android

• Models/repository: android/app/src/main/java/com/speaktrue/features/sts/data/.
• State: android/app/src/main/java/com/speaktrue/features/sts/presentation/STSViewModel.kt.
• UI: android/app/src/main/java/com/speaktrue/features/sts/ui/STSScreen.kt.
• Tests: matching android/app/src/test/java/com/speaktrue/features/sts/ files.

Vertical Slices

• C01: Make STS semantics accurate across the product — Risk: medium; Depends: none; Mode: HITL.

  Replace translation claims with revoicing language where no translation contract exists, add a contract test that committed transcript is the spoken text, and record the decision about the Live Interpreter label.

  After this: a user can understand exactly what the current mode does, and future translation work has an explicit boundary.

• C02: Add an ephemeral web turn timeline — Risk: medium; Depends: C01; Mode: HITL.

  Accumulate completed turns in browser memory, render state/replay/share/save actions, preserve the existing newest-turn autoplay behavior, and add a JS runtime check for two consecutive turns.

  After this: a web user can complete and revisit several turns in one open session without a schema change.

• C03: Add an ephemeral iOS turn timeline — Risk: medium; Depends: C02 interaction review; Mode: AFK.

  Introduce native ConversationTurn state, retain prior turns when recording restarts, reuse existing player/save/share services, and cover reducer behavior with focused tests.

  After this: the validated multi-turn interaction works on iOS but remains session-local.

• C04: Add an ephemeral Android turn timeline — Risk: medium; Depends: C02 interaction review; Mode: AFK.

  Add the equivalent state/reducer/UI flow, reuse existing playback/save/share services, prove consecutive-turn ordering, and run the complete Android change gate before commit or push.

  After this: the validated multi-turn interaction works on all three clients but remains session-local.

• C05: Persist one conversation end to end on web — Risk: high; Depends: C02 and retention decision; Mode: HITL.

  Add schema, RLS, create/get/list functions, optional finalize attachment, web auto-persistence, idempotent retry, and migration/handler/web tests.

  After this: a signed-in web user can create a conversation, complete turns, reload the page, and resume it.

• C06: Add iOS conversation resume — Risk: high; Depends: C05; Mode: AFK.

  Decode additive response fields, list/resume conversations, retry orphaned attachments, and keep old-server behavior functional on iOS.

  After this: a conversation started on web can be resumed on iOS, and vice versa.

• C07: Add Android conversation resume — Risk: high; Depends: C05; Mode: AFK.

  Decode additive response fields, list/resume conversations, retry orphaned attachments, retain old-server compatibility, and run the complete Android change gate.

  After this: a conversation started on web or iOS can be resumed on Android, and vice versa.

• C08: Add per-turn reuse and immutable edit flow — Risk: medium; Depends: C06-C07; Mode: HITL.

  Support replay, URL refresh, share, Save to Soundboard, and Speak edited copy as a derived turn. Preserve artifact provenance in clip_metadata when saving.

  After this: a user can reuse any turn without losing the original captured record.

• C09: Add lifecycle controls and recovery diagnostics — Risk: medium; Depends: C05; Mode: AFK.

  Add rename/close/delete, attachment reconciliation, stable failure codes, content-free timing events, auth-expiry recovery, and operator queries.

  After this: users can manage conversation retention and support can diagnose failures without reading conversation content.

• C10: Stage cross-platform release — Risk: high; Depends: C06-C09; Mode: HITL.

  Run migration rehearsal, old-client compatibility tests, poor-network/device tests, accessibility checks, allowlisted beta, and rollback drill before default enablement.

  After this: Conversation Workspace is releasable with evidence and can be independently disabled.

Slice Verification

• Backend: focused Deno handler tests, bash scripts/coverage_backend_contracts.sh, and bash scripts/regression_migration_paths.sh for schema slices.
• Web: focused pytest, node --check web/python-web-app/static/js/index_core.js, and conversation runtime checks.
• iOS: STTAndSTSViewModelTests, coding tests, simulator build/test, and manual microphone/playback checks.
• Android: focused STS tests during development and bash scripts/verify_android_ci_local.sh before commit/push.
• Every slice: old payload fixture, auth ownership failure, expired signed URL, retry/idempotency test, git diff --check.

Acceptance Gate

• Three consecutive turns remain ordered and individually playable.
• Reloading and cross-device resume preserve turn identity and text.
• Retrying finalization or attachment does not create duplicate turns.
• Old clients can finalize an STS artifact without conversation fields.
• Saving a turn preserves the canonical artifact and clip_metadata contract.
• Delete behavior matches the published retention rule.
• Logs and product events contain no transcript text or signed URLs.
• The UI does not promise translation unless a real transformation contract is later implemented.

Rollback

Disable conversation_workspace_enabled to return clients to latest-turn behavior. Keep additive tables and optional response fields in place until all deployed clients are known to ignore or understand them. Do not roll back by deleting conversation records. A later cleanup migration can remove abandoned schema only after export and retention review.

Open Decisions

1. Final product label for live_interpreter before translation exists.
2. Default retention and whether deletion is immediate or recoverable.
3. Whether a conversation auto-closes after inactivity.
4. Whether original microphone recordings are ever opt-in attachments; MVP says no.
5. Whether true translation belongs in this plan after MVP or in a separate provider/quality plan.