Coupled Representation Learning for Domains, Intents and Slots in Spoken Language Understanding

Authors: Jihwan Lee, Dongchan Kim, Ruhi Sarikaya, Young-Bum Kim
Conference: 2018 IEEE Spoken Language Technology Workshop (SLT 2018), pp. 184–191
Preprint: arXiv:1812.06083 [cs.CL, cs.LG]
Location: Athens, Greece | December 2018
Direct Links: arXiv Abstract | PDF Fulltext

Abstract

Representation learning is a fundamental problem in a wide range of applications and it is important for performing downstream tasks successfully. In this paper, we propose a novel model that learns coupled representations of domains, intents, and slots by exploiting their inherent hierarchical dependencies within a Spoken Language Understanding (SLU) system.

Traditional spoken dialog systems treat domain classification, intent determination, and slot filling as isolated steps or through loosely coupled multi-task pipelines. These conventional designs suffer from error propagation and fail to leverage semantic constraints across hierarchy levels. Our proposed architecture jointly learns vector representations that capture semantic interactions among domains, intents, and slots, significantly improving accuracy across all three tasks on large-scale conversational benchmarks.

Key Technical Contributions & Architecture

In goal-oriented conversational systems, natural language inputs follow a natural ontological hierarchy:

                  ┌────────────────────────┐
                  │    Domain (e.g. Music) │
                  └───────────┬────────────┘
                              │ restricts
                              ▼
               ┌───────────────────────────────┐
               │ Intent (e.g. PlayArtistTrack) │
               └──────────────┬────────────────┘
                              │ constrains
                              ▼
            ┌────────────────────────────────────┐
            │ Slots (e.g. ArtistName, TrackName) │
            └────────────────────────────────────┘

Explicit Coupling Formulation: Rather than computing distinct unconditioned embeddings, the model couples latent representations such that the domain vector regularizes the intent hypothesis space, which subsequently constrains allowable slot sequences.
Unified Semantic Manifold: Projects domains, intents, and slots into a shared geometric space where semantic compatibility is enforced through joint loss optimization.
Preventing Error Cascades: In sequential pipelines, an upstream domain classification error inevitably corrupts intent and slot parsing. Coupled learning shares bidirectional gradient signals across all heads simultaneously.
Empirical Results: Evaluated on large-scale voice assistant interaction benchmarks, yielding consistent relative error reductions over independent and uncoupled multi-task baselines.

Citation

@inproceedings{lee2018coupled,
  title={Coupled Representation Learning for Domains, Intents and Slots in Spoken Language Understanding},
  author={Lee, Jihwan and Kim, Dongchan and Sarikaya, Ruhi and Kim, Young-Bum},
  booktitle={2018 IEEE Spoken Language Technology Workshop (SLT)},
  pages={184--191},
  year={2018},
  organization={IEEE},
  doi={10.1109/SLT.2018.8639556},
  url={https://arxiv.org/abs/1812.06083}
}