Fine-grained Generalization Analysis of Structured Output Prediction

Waleed Mustafa, Yunwen Lei, Antoine Ledent, Marius Kloft

Research output: Chapter in book/report/conference proceedingConference proceedingpeer-review

4 Citations (Scopus)

Abstract

In machine learning we often encounter structured output prediction problems (SOPPs), i.e. problems where the output space admits a rich internal structure. Application domains where SOPPs naturally occur include natural language processing, speech recognition, and computer vision. Typical SOPPs have an extremely large label set, which grows exponentially as a function of the size of the output. Existing generalization analysis implies generalization bounds with at least a square-root dependency on the cardinality d of the label set, which can be vacuous in practice. In this paper, we significantly improve the state of the art by developing novel high-probability bounds with a logarithmic dependency on d. Moreover, we leverage the lens of algorithmic stability to develop generalization bounds in expectation without any dependency on d. Our results therefore build a solid theoretical foundation for learning in large-scale SOPPs. Furthermore, we extend our results to learning with weakly dependent data.

Original languageEnglish
Title of host publicationProceedings of the 30th International Joint Conference on Artificial Intelligence, IJCAI 2021
EditorsZhi-Hua Zhou
PublisherInternational Joint Conferences on Artificial Intelligence
Pages2841-2847
Number of pages7
ISBN (Electronic)9780999241196
DOIs
Publication statusPublished - Aug 2021
Event30th International Joint Conference on Artificial Intelligence, IJCAI 2021 - Virtual, Online, Canada
Duration: 19 Aug 202127 Aug 2021
https://ijcai-21.org/#
https://www.ijcai.org/proceedings/2021/

Publication series

NameIJCAI International Joint Conference on Artificial Intelligence
ISSN (Print)1045-0823

Conference

Conference30th International Joint Conference on Artificial Intelligence, IJCAI 2021
Country/TerritoryCanada
CityVirtual, Online
Period19/08/2127/08/21
Internet address

Scopus Subject Areas

  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Fine-grained Generalization Analysis of Structured Output Prediction'. Together they form a unique fingerprint.

Cite this