Efficient Large-Scale Neural Domain Classification with Personalized Attention

Authors: Young-Bum Kim, Dongchan Kim, Anjishnu Kumar, Ruhi Sarikaya
Conference: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018, Volume 1: Long Papers), pp. 2214–2224
Preprint: arXiv:1804.08065 [cs.CL]
Location: Melbourne, Australia | July 2018
Direct Links: ACL Anthology P18-1206 | PDF Fulltext

Abstract

In this paper, we explore the task of mapping spoken language utterances to one of thousands of natural language understanding domains in intelligent personal digital assistants (IPDAs). This scenario is observed in mainstream IPDAs in industry that allow third parties to develop thousands of new domains to augment built-in first party domains to rapidly increase domain coverage and overall IPDA capabilities.

We propose a scalable neural model architecture with a shared encoder, a novel attention mechanism that incorporates personalization information, and domain-specific classifiers that solve the problem efficiently. Our architecture is designed to efficiently accommodate incremental domain additions, achieving two orders of magnitude speedup ($100\times$) compared to full model retraining. We consider the practical constraints of real-time production systems, and design to minimize memory footprint and runtime latency. We demonstrate that incorporating personalization significantly improves domain classification accuracy in a setting with thousands of overlapping domains.

Key Technical Contributions & Architecture

                       [User Utterance Tokens]
                                  │
                                  ▼
                       ┌──────────────────────┐
                       │    Shared Encoder    │
                       │ (Bidirectional LSTM) │
                       └──────────┬───────────┘
                                  │
                ┌─────────────────┴─────────────────┐
                │                                   │
                ▼                                   ▼
┌───────────────────────────────┐   ┌───────────────────────────────┐
│     Domain Classifier 1       │   │     Domain Classifier D       │
│  - Personalized Attention Head│   │  - Personalized Attention Head│
│    (User Interaction Embeds)  │   │    (User Interaction Embeds)  │
└───────────────────────────────┘   └───────────────────────────────┘

1. Personalized Attention Mechanism: Conditions attention weights on user-specific historical interaction embeddings and preferred domain contexts, effectively disambiguating polysemous user requests.
2. Decoupled Architecture for $100\times$ Speedup:
   • Shared Feature Encoder: Extracts foundational sentence representations frozen across domain updates.
   • Domain-Specific Classifiers: Lightweight, modular heads trained independently when third parties publish new capabilities, avoiding global retraining.
3. Ultra-Low Inference Latency: Optimized to operate within sub-15ms budgets on production serving clusters.
4. Empirical Performance: Evaluated across thousands of real conversational domains, achieving superior precision on overlapping voice queries.

Citation

@inproceedings{kim2018efficient,
  title={Efficient Large-Scale Neural Domain Classification with Personalized Attention},
  author={Kim, Young-Bum and Kim, Dongchan and Kumar, Anjishnu and Sarikaya, Ruhi},
  booktitle={Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  pages={2214--2224},
  year={2018},
  doi={10.18653/v1/P18-1206},
  url={https://aclanthology.org/P18-1206/}
}