Optimizing batched winograd convolution on GPUs

Da Yan; Wei Wang; Xiaowen Chu

doi:10.1145/3332466.3374520

Optimizing batched winograd convolution on GPUs

Da Yan, Wei Wang, Xiaowen Chu

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

44 Citations (Scopus)

Abstract

In this paper, we present an optimized implementation for single-precision Winograd convolution on NVIDIA Volta and Turing GPUs. Compared with the state-of-the-art Winograd convolution in cuDNN 7.6.1, our implementation achieves up to 2.13× speedup on Volta V100 and up to 2.65× speedup on Turing RTX2070. On both Volta and Turing GPUs, our implementation achieves up to 93% of device peak. Apart from analyzing and benchmarking different high-level optimization options, we also build a SASS assembler TuringAs for Volta and Turing that enables tuning the performance at the native assembly level. The new optimization opportunities uncovered by TuringAs not only improve the Winograd convolution but can also benefit CUDA compilers and native assembly programming. We have released TuringAs as an open-source software. To the best of our knowledge, this is the first public-available assembler for Volta and Turing GPUs.

Original language	English
Title of host publication	PPoPP 2020 - Proceedings of the 2020 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
Publisher	Association for Computing Machinery (ACM)
Pages	32-44
Number of pages	13
ISBN (Electronic)	9781450368186
DOIs	https://doi.org/10.1145/3332466.3374520
Publication status	Published - 19 Feb 2020
Event	25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2020 - San Diego, United States Duration: 22 Feb 2020 → 26 Feb 2020 https://dl.acm.org/doi/proceedings/10.1145/3332466 (Conference proceedings)

Publication series

Name	Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP

Conference

Conference	25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2020
Country/Territory	United States
City	San Diego
Period	22/02/20 → 26/02/20
Internet address	https://dl.acm.org/doi/proceedings/10.1145/3332466 (Conference proceedings)

Scopus Subject Areas

Software

User-Defined Keywords

Convolution
GPU
Performance

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1145/3332466.3374520

Cite this

Yan, D., Wang, W., & Chu, X. (2020). Optimizing batched winograd convolution on GPUs. In PPoPP 2020 - Proceedings of the 2020 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (pp. 32-44). (Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP). Association for Computing Machinery (ACM). https://doi.org/10.1145/3332466.3374520

@inproceedings{0e30d6e67eca48bf8ca0a571dac77799,

title = "Optimizing batched winograd convolution on GPUs",

abstract = "In this paper, we present an optimized implementation for single-precision Winograd convolution on NVIDIA Volta and Turing GPUs. Compared with the state-of-the-art Winograd convolution in cuDNN 7.6.1, our implementation achieves up to 2.13× speedup on Volta V100 and up to 2.65× speedup on Turing RTX2070. On both Volta and Turing GPUs, our implementation achieves up to 93% of device peak. Apart from analyzing and benchmarking different high-level optimization options, we also build a SASS assembler TuringAs for Volta and Turing that enables tuning the performance at the native assembly level. The new optimization opportunities uncovered by TuringAs not only improve the Winograd convolution but can also benefit CUDA compilers and native assembly programming. We have released TuringAs as an open-source software. To the best of our knowledge, this is the first public-available assembler for Volta and Turing GPUs.",

keywords = "Convolution, GPU, Performance",

author = "Da Yan and Wei Wang and Xiaowen Chu",

note = "Funding Information: The aforementioned mechanism is supported by the control code. Control code stores information to prevent data hazards, control reuse flag, and balance progress between warps. A detailed introduction of the control code can be found in Section 2.1 in [5]. We give a detailed description of the yield flag (at [45]), since we found this flag will affect the overall performance.; 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2020 ; Conference date: 22-02-2020 Through 26-02-2020",

year = "2020",

month = feb,

day = "19",

doi = "10.1145/3332466.3374520",

language = "English",

series = "Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP",

publisher = "Association for Computing Machinery (ACM)",

pages = "32--44",

booktitle = "PPoPP 2020 - Proceedings of the 2020 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming",

address = "United States",

url = "https://dl.acm.org/doi/proceedings/10.1145/3332466",

}

Yan, D, Wang, W & Chu, X 2020, Optimizing batched winograd convolution on GPUs. in PPoPP 2020 - Proceedings of the 2020 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP, Association for Computing Machinery (ACM), pp. 32-44, 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2020, San Diego, United States, 22/02/20. https://doi.org/10.1145/3332466.3374520

Optimizing batched winograd convolution on GPUs. / Yan, Da; Wang, Wei; Chu, Xiaowen.
PPoPP 2020 - Proceedings of the 2020 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Association for Computing Machinery (ACM), 2020. p. 32-44 (Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Optimizing batched winograd convolution on GPUs

AU - Yan, Da

AU - Wang, Wei

AU - Chu, Xiaowen

N1 - Funding Information: The aforementioned mechanism is supported by the control code. Control code stores information to prevent data hazards, control reuse flag, and balance progress between warps. A detailed introduction of the control code can be found in Section 2.1 in [5]. We give a detailed description of the yield flag (at [45]), since we found this flag will affect the overall performance.

PY - 2020/2/19

Y1 - 2020/2/19

N2 - In this paper, we present an optimized implementation for single-precision Winograd convolution on NVIDIA Volta and Turing GPUs. Compared with the state-of-the-art Winograd convolution in cuDNN 7.6.1, our implementation achieves up to 2.13× speedup on Volta V100 and up to 2.65× speedup on Turing RTX2070. On both Volta and Turing GPUs, our implementation achieves up to 93% of device peak. Apart from analyzing and benchmarking different high-level optimization options, we also build a SASS assembler TuringAs for Volta and Turing that enables tuning the performance at the native assembly level. The new optimization opportunities uncovered by TuringAs not only improve the Winograd convolution but can also benefit CUDA compilers and native assembly programming. We have released TuringAs as an open-source software. To the best of our knowledge, this is the first public-available assembler for Volta and Turing GPUs.

AB - In this paper, we present an optimized implementation for single-precision Winograd convolution on NVIDIA Volta and Turing GPUs. Compared with the state-of-the-art Winograd convolution in cuDNN 7.6.1, our implementation achieves up to 2.13× speedup on Volta V100 and up to 2.65× speedup on Turing RTX2070. On both Volta and Turing GPUs, our implementation achieves up to 93% of device peak. Apart from analyzing and benchmarking different high-level optimization options, we also build a SASS assembler TuringAs for Volta and Turing that enables tuning the performance at the native assembly level. The new optimization opportunities uncovered by TuringAs not only improve the Winograd convolution but can also benefit CUDA compilers and native assembly programming. We have released TuringAs as an open-source software. To the best of our knowledge, this is the first public-available assembler for Volta and Turing GPUs.

KW - Convolution

KW - GPU

KW - Performance

UR - http://www.scopus.com/inward/record.url?scp=85082381803&partnerID=8YFLogxK

U2 - 10.1145/3332466.3374520

DO - 10.1145/3332466.3374520

M3 - Conference proceeding

AN - SCOPUS:85082381803

T3 - Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP

SP - 32

EP - 44

BT - PPoPP 2020 - Proceedings of the 2020 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

PB - Association for Computing Machinery (ACM)

T2 - 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2020

Y2 - 22 February 2020 through 26 February 2020

ER -

Yan D, Wang W, Chu X. Optimizing batched winograd convolution on GPUs. In PPoPP 2020 - Proceedings of the 2020 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Association for Computing Machinery (ACM). 2020. p. 32-44. (Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP). doi: 10.1145/3332466.3374520

Optimizing batched winograd convolution on GPUs

Abstract

Publication series

Conference

Scopus Subject Areas

User-Defined Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this