Inventors: Ruhi Sarikaya, Dongchan Kim, Young-Bum Kim
Assignee: Amazon Technologies, Inc.
Patent No.: US 11,568,863 B1
Date of Patent: January 31, 2023
Official Links: USPTO / Google Patents US11568863B1 | Patent PDF
Abstract
Systems and methods are provided for shortlisting skills for natural language processing. In one embodiment, a computing system receives natural language speech audio or text from a user device, generates an utterance representation, and processes the representation using a skill shortlister to identify a subset of candidate skills from a large catalog of available skills.
The shortlister evaluates fast semantic features, user historical engagement, and device contextual parameters to rank and select a bounded candidate shortlist (top-$k$). Downstream spoken language understanding (SLU) components—including full domain classifiers, intent recognizers, and slot extractors—evaluate only the shortlisted skills, thereby drastically reducing latency and computational overhead in large-scale multi-domain conversational systems.
Technical Challenge & Architectural Solution
Intelligent personal digital assistants (IPDAs) such as Amazon Alexa host tens of thousands of first-party and third-party skills. In real-time conversational processing:
- Latency Bottleneck: Evaluating complex deep neural architectures (e.g., bidirectional LSTMs, Transformers) across tens of thousands of skills within a 50–100 millisecond conversational budget is computationally intractable.
- Overlapping Decision Spaces: Massive classification spaces create severe confusion boundaries and degrade precision.
[User Utterance & Context]
│
▼
┌──────────────────────────────────────────────┐
│ Lightweight Skill Shortlister │
│ - Fast dual-encoder embedding lookup │
│ - Historical user invocation priors │
│ - Device modality & location gating │
└──────────────────────┬───────────────────────┘
│
Top-k Candidate Skills (e.g. k=5..10)
│
▼
┌──────────────────────────────────────────────┐
│ Targeted Deep SLU Processing │
│ - Intent classification & slot extraction │
│ - Heavy neural reranker over top candidates │
└──────────────────────────────────────────────┘The patented invention implements a multi-stage routing pipeline:
- Lightweight Candidate Generation: Computes embedding projections of incoming user utterances against pre-indexed skill representations.
- Context-Sensitive Shortlisting: Integrates user session context, active device state, location parameters, and historical usage patterns to filter candidate capabilities.
- Targeted Deep Parsing: Routes candidate shortlists to specialized NLU decoders for intent and slot extraction, maximizing routing accuracy while preserving ultra-low runtime latency.
Citation
@misc{sarikaya2023skill,
title={Skill shortlister for natural language processing},
author={Sarikaya, Ruhi and Kim, Dongchan and Kim, Young-Bum},
year={2023},
month=jan # "~31",
note={US Patent 11,568,863 B1, Assignee: Amazon Technologies, Inc.}
}