The Challenge
Build an intelligent HR platform that ingests thousands of employee events — feedback, call transcripts, compliance issues, performance reviews — and automatically generates actionable insights, recommendations, and policy documents for business owners, managers, and employees. At scale. With quality guarantees.
What I Built
Brighten — a multi-stage, event-driven AI pipeline on AWS that transformed raw HR events into production-ready business intelligence.
The Pipeline
Every HR event flowed through a 10-stage pipeline:
- Ingestion — events published to SNS, stored in a data lake
- Vectorization — embeddings generated for semantic search and clustering
- Smart Batching — dual-level batching (company + individual) with priority routing (HIGH/MEDIUM/LOW)
- Gap Detection — AI identifies missing information, pauses pipeline, triggers interactive wizards to collect it
- Insight Generation — LLM analyzes batched events, produces structured insights
- Recommendation — actionable recommendations generated from insights
- Solution Generation — full policy documents, CAPs, PIPs, job descriptions generated
- Notice Creation — user-facing notices crafted per audience (owner, manager, employee)
- Editorial Review — AI-driven content refinement and quality pass
- Broadcasting — distribution to stakeholders
Every AI generation stage passed through an LLM-as-judge evaluation gate — scoring relevance, brand alignment, clarity, and safety on a 0-1 scale. Threshold: 0.7. Below that, the stage retried with backoff. This meant every piece of AI-generated content met a consistent quality bar before reaching a human.
v1 → v2 Migration
The original architecture used pure SNS/SQS — every stage was a separate Lambda consuming from its own queue. This worked, but during HIGH event spikes, all workers competed for Lambda concurrency. Individual batch processing time was unpredictable and unmeasurable.
v2 migrated everything downstream of batch collection to AWS Step Functions. Each batch got its own isolated state machine execution — one execution per batch, with:
- Process isolation — batch A doesn’t slow batch B
- Built-in observability — exact duration per batch and per stage, visual execution graphs
- Declarative retries — no custom retry code, retry policies defined in the state machine
- Parallel fan-out — Map states for notice generation (the v1 sequential bottleneck became parallel)
Result: batch processing went from hours during spikes to minutes.
Multi-Track Architecture
v2 introduced parallel tracks targeting different audiences from the same batch:
- Owner Track — company-wide insights, full pipeline with gap detection and all solution types
- Manager Track — role-scoped recommendations, team-level notices
- Employee Track — personalized insights delivered to specific individuals
Tracks ran as independent Step Function definitions. A failure in one track didn’t block the others.
Emotional Drift Analysis
LOW-priority batches (historical imports, background analytics) ran through a separate pipeline analyzing emotional trends across a 16-emotion framework. Company drift threshold: 20%. Employee drift threshold: 25%. Significant drift automatically generated HIGH-priority events that re-entered the main pipeline.
Technical Highlights
- Event-driven architecture processing thousands of HR events across hundreds of companies
- LLM-as-judge quality gates at every AI generation stage (5 criteria, 0.7 threshold)
- AI debugger with code reflection — captured stack traces, sent source context to a fast model, produced structured debugging output (summary, root cause, suggested fix, Claude prompt)
- Dual-level batching with priority cascading (HIGH events count toward MEDIUM/LOW thresholds)
- Solution throttle: max 100 solutions per 24-hour window per company
- Snowflake analytics dashboard (15 charts tracking pipeline performance, engagement, AI quality)
- Test metadata flags for controlled failure injection and pipeline behavior testing