Altong
A Korean knowledge-sharing community serving five languages — I built its AI assistant end to end: SSE token streaming, multi-turn memory, per-turn language matching, voice input and the quota architecture that keeps LLM cost predictable.
The problem
A five-language community wanted an AI assistant that feels instant, answers every user in their own language, and survives public traffic from anonymous visitors — without letting LLM spend run away or letting a burst of AI requests degrade the rest of the platform.
The solution
An assistant module isolated behind its own thread pool and rejection policy, streaming over SSE for immediate first tokens, with multi-turn memory and per-turn language matching. Atomic Redis scripts enforce burst, daily and global spend limits, and signed guest cookies meter anonymous users without an account. Whisper powers voice input, and a two-stage classifier detects each post's language reliably enough to drive content filtering.
Overview
Altong (altong.com) is a Korean knowledge-sharing community where people ask questions, publish posts and earn an in-app currency, serving an audience across Korean, English, Japanese, Chinese and Vietnamese. I joined as a Senior AI & Full-Stack Engineer and own features end to end — frontend in Next.js and TypeScript, backend in Java 17 and Spring Boot against a sharded MySQL cluster.
My main body of work is the platform's AI assistant. It streams answers token by token over SSE so the first words appear immediately instead of after a long pause, keeps multi-turn conversation memory, and matches the user's language per turn — ask in English inside the Korean site and the reply comes back in English. The whole module lives behind its own thread pool with an explicit rejection policy, so a burst of AI traffic can never starve the request threads serving the rest of the site.
Cost and abuse control sit underneath it. Quota guards run as atomic Redis scripts covering burst limits, per-day allowances and a global spend ceiling, with signed HttpOnly cookies plus IP counters so guests are metered without an account. Sessions and messages persist with distributed IDs, and the assistant can turn a conversation into a public question — the model drafts the title, body and category, and the user posts it in one step.
Beyond the assistant I shipped voice input built on Whisper speech-to-text with a hold-to-talk interaction (drag to lock, slide to cancel), and rebuilt content-language detection as a two-stage classifier after the original single-pass approach proved unreliable on short, mixed-script posts. Alongside the features I fixed sorting, transaction-naming and locale bugs across the product, and reported several security issues to the team.
Key features
- SSE token streaming so answers start appearing immediately instead of after a full-response pause
- Per-turn language matching — the assistant replies in whatever language the user writes in
- Multi-turn conversation memory with persisted sessions and LLM-generated session titles
- Atomic Redis quota guards: burst limits, daily allowances and a global spend ceiling
- Signed HttpOnly guest cookies plus IP counters so anonymous visitors are metered without an account
- Dedicated thread pool with an explicit rejection policy, isolating AI load from the main platform
- Voice input via Whisper speech-to-text with hold-to-talk, drag-to-lock and slide-to-cancel
- Two-stage content language detection driving the multilingual content filter
- Turn a conversation into a public question — the model drafts title, body and category