Full Stack Engineer, Empat Speech

The challenge

Empat Speech connects speech-language pathologists (SLPs) and students through live video communication, interactive games, and clinical progress tracking. Because the platform handles sensitive student and clinical information, personally identifiable information (PII) could not simply be stored alongside ordinary application data in the primary databases.

At the same time, the product needed to remain frictionless for students—including supporting a workflow where a student joins a live therapy session without logging in—while guaranteeing that personal information is strictly protected across every API boundary and resilient against third-party provider outages.

My contribution

I designed and built both the PII isolation and synchronization pipeline and the platform's multi-layered authorization architecture:

First, the system stores all PII outside the main databases in a dedicated third-party provider. Whenever the platform needs to read or manage PII, it communicates with that external provider and links records through an isolated mapping table that correlates internal system IDs with external provider IDs.

To protect against provider downtime or transient communication failures without permanently storing PII in our own database, I built a queueing pipeline on AWS SQS backed by a dead-letter queue (DLQ). Synchronization events are held only temporarily in the queue, and any failed deliveries are captured in the DLQ so we can track, inspect, and replay failures safely without losing records.

Second, I implemented access control combining AWS Cognito for identity management, AWS AppSync authentication modes, and custom authorization logic. When a student accesses a therapy session without logging in, the API layer itself scopes the response payload so no personal information is ever returned to an unauthenticated client.

Decisions and trade-offs

A key architectural trade-off was choosing asynchronous queueing (AWS SQS with a DLQ) and an isolated ID-mapping table over caching or persisting PII locally. Relying on an external provider introduces network and availability dependencies, but buffering transitions in SQS with DLQ tracking derisked data loss during provider hiccups while keeping our primary databases free of permanent PII.

For access control, we deliberately enforced role and session boundaries inside AppSync resolvers and custom authorizers rather than relying on client-side UI filtering. Unauthenticated student sessions receive only the state needed to participate in the therapy activity, ensuring personal data is absent at the wire level.

Outcomes and lessons

The architecture keeps PII isolated from the primary databases, provides full operational visibility and recovery when external communication fails, and lets students join therapy sessions seamlessly without exposing sensitive personal information.

The challenge

Speech-language pathologists need engaging visual stories and scene activities tailored to the specific speech and language targets they are stimulating with each student, along with clear progress reports summarizing clinical sessions.

Two practical engineering challenges stood out: keeping characters and scenes visually consistent across multi-page generated stories while still personalizing questions to an individual student, and ensuring LLM-generated clinical progress reports remain accurate, predictably structured, and verifiable by therapists.

My contribution

I designed and implemented two focused LLM pipelines for the platform:

1. Story and scene image generation: Given therapy input, the backend uses AWS Bedrock to generate structured story context and persists metadata about the story's actors so characters and settings remain consistent across pages and illustrations. That context feeds an image generation pipeline built behind a swappable provider adapter (currently using Gemini for image generation, while allowing us to switch providers without rewriting application logic). Once a story or scene is generated, its core pages stay static, while comprehension and stimulation questions are generated dynamically using the student's personalized context and target skills. Server-Sent Events (SSE) notify the client in real time as each text segment or illustration completes.

2. Clinical progress reports: Rather than asking an LLM to do math or invent summaries from raw data, the system pre-calculates all quantitative metrics we have data for first. It then prompts AWS Bedrock—using Bedrock's structured output tooling to enforce a strict response schema—to synthesize the clinician's session notes and calculated metrics into the required report format, attaching references to the underlying therapy sessions.

Decisions and trade-offs

Separating static story/scene assets from dynamic per-student question generation gave us the best balance of consistency and personalization: stories do not need to be regenerated from scratch for every session, while prompts and questions adapt to what the student is actively practicing.

Wrapping image generation behind an adapter interface avoided locking the pipeline to a single model vendor as multimodal capabilities evolve. For progress reports, pre-calculating metrics in code and enforcing an explicit output schema in Bedrock—paired with clickable session citations—kept the LLM focused on formatting and synthesis so therapists can quickly verify every claim against the source session notes.

Outcomes and lessons

Clinicians can generate cohesive multi-page visual stories with real-time SSE progress updates, present personalized questions tailored to each student's therapy goals, and produce structured, fact-checkable progress reports grounded in real session records.