ParaCoder: Parallel Code Generation with Large Language Model

Xiaowen Huang; Xu Zhang; Lvfang Tao; Renjie Mao; Nan Zhou; Wenxi Zhu; Minwen Deng; Jintao Meng; Yanjie Wei; Amelie Chi Zhou; Bingqiang Wang; Shengzhong Feng

doi:10.1145/3711708.3723442

ParaCoder: Parallel Code Generation with Large Language Model

Xiaowen Huang, Xu Zhang, Lvfang Tao, Renjie Mao, Nan Zhou, Wenxi Zhu, Minwen Deng, Jintao Meng^*, Yanjie Wei, Amelie Chi Zhou, Bingqiang Wang, Shengzhong Feng

^*Corresponding author for this work

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

Abstract

High-performance parallel code generation is a complex and fascinating area in computer science that focuses on producing code that executes as quickly and efficiently as possible. In our paper, we designed a new architecture for parallel code generation agent with 4 inter-connected components of LLM---Memory, Planning, Tools and Action. It also incooperated with two techniques: data augmentation, prompting and retrieval-augmented editing to improve the performance of the parallel codes. Data augmentation is implemented by extracting and processing PIE dataset, and also synthesis dataset generated by LLM models with ParEval benchmark. Finally planning-oriented prompting, code verification and retrieval augmented editing are used to promote the actual performance of the LLM generated code. The evaluation results confirm that a rough speedup of 6.06X and 5.13X are achieved using Qwen2.5-Coder-7B-Instruct, Qwen2.5-Coder-14B-Instruct LLM models.

Original language	English
Title of host publication	Proceedings of the 1st FastCode Programming Challenge, FCPC 2025
Place of Publication	New York
Publisher	Association for Computing Machinery (ACM)
Pages	1-7
Number of pages	7
ISBN (Electronic)	9798400714467
ISBN (Print)	9798400714467
DOIs	https://doi.org/10.1145/3711708.3723442
Publication status	Published - 2 May 2025
Event	Fastcode Programming Challenge 2025 - The Westin Las Vegas Hotel & Spa, Las Vegas, United States Duration: 1 Mar 2025 → 5 Mar 2025 https://dl.acm.org/doi/proceedings/10.1145/3711708 (Conference proceeding) https://ppopp25.sigplan.org/ (Conference website)

Publication series

Name	Proceedings of the FastCode Programming Challenge
Publisher	Association for Computing Machinery

Conference

Conference	Fastcode Programming Challenge 2025
Abbreviated title	PPoPP 2025
Country/Territory	United States
City	Las Vegas
Period	1/03/25 → 5/03/25
Internet address	https://dl.acm.org/doi/proceedings/10.1145/3711708 (Conference proceeding) https://ppopp25.sigplan.org/ (Conference website)

User-Defined Keywords

LLM
code generation
parallelization

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1145/3711708.3723442Licence: CC BY

Cite this

Huang, X., Zhang, X., Tao, L., Mao, R., Zhou, N., Zhu, W., Deng, M., Meng, J., Wei, Y., Zhou, A. C., Wang, B., & Feng, S. (2025). ParaCoder: Parallel Code Generation with Large Language Model. In Proceedings of the 1st FastCode Programming Challenge, FCPC 2025 (pp. 1-7). (Proceedings of the FastCode Programming Challenge). Association for Computing Machinery (ACM). https://doi.org/10.1145/3711708.3723442

@inproceedings{a29830816d80454d8a1ecde8acbbf70a,

title = "ParaCoder: Parallel Code Generation with Large Language Model",

abstract = "High-performance parallel code generation is a complex and fascinating area in computer science that focuses on producing code that executes as quickly and efficiently as possible. In our paper, we designed a new architecture for parallel code generation agent with 4 inter-connected components of LLM---Memory, Planning, Tools and Action. It also incooperated with two techniques: data augmentation, prompting and retrieval-augmented editing to improve the performance of the parallel codes. Data augmentation is implemented by extracting and processing PIE dataset, and also synthesis dataset generated by LLM models with ParEval benchmark. Finally planning-oriented prompting, code verification and retrieval augmented editing are used to promote the actual performance of the LLM generated code. The evaluation results confirm that a rough speedup of 6.06X and 5.13X are achieved using Qwen2.5-Coder-7B-Instruct, Qwen2.5-Coder-14B-Instruct LLM models.",

keywords = "LLM, code generation, parallelization",

author = "Xiaowen Huang and Xu Zhang and Lvfang Tao and Renjie Mao and Nan Zhou and Wenxi Zhu and Minwen Deng and Jintao Meng and Yanjie Wei and Zhou, {Amelie Chi} and Bingqiang Wang and Shengzhong Feng",

note = "This work is supported by the Shenzhen-HongKong Joint Funding Project (Category A) under Grant No. SGDX20230116092056010, Shenzhen Key Laboratory of Intelligent Bioinformatics under Grant No. ZDSYS20220422103800001. Publisher Copyright: {\textcopyright} 2025 Copyright is held by the owner/author(s).; Fastcode Programming Challenge 2025, PPoPP 2025 ; Conference date: 01-03-2025 Through 05-03-2025",

year = "2025",

month = may,

day = "2",

doi = "10.1145/3711708.3723442",

language = "English",

isbn = "9798400714467",

series = "Proceedings of the FastCode Programming Challenge",

publisher = "Association for Computing Machinery (ACM)",

pages = "1--7",

booktitle = "Proceedings of the 1st FastCode Programming Challenge, FCPC 2025",

address = "United States",

url = "https://dl.acm.org/doi/proceedings/10.1145/3711708, https://ppopp25.sigplan.org/",

}

Huang, X, Zhang, X, Tao, L, Mao, R, Zhou, N, Zhu, W, Deng, M, Meng, J, Wei, Y, Zhou, AC, Wang, B & Feng, S 2025, ParaCoder: Parallel Code Generation with Large Language Model. in Proceedings of the 1st FastCode Programming Challenge, FCPC 2025. Proceedings of the FastCode Programming Challenge, Association for Computing Machinery (ACM), New York, pp. 1-7, Fastcode Programming Challenge 2025, Las Vegas, Nevada, United States, 1/03/25. https://doi.org/10.1145/3711708.3723442

ParaCoder: Parallel Code Generation with Large Language Model. / Huang, Xiaowen; Zhang, Xu; Tao, Lvfang et al.
Proceedings of the 1st FastCode Programming Challenge, FCPC 2025. New York: Association for Computing Machinery (ACM), 2025. p. 1-7 (Proceedings of the FastCode Programming Challenge).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - ParaCoder

T2 - Fastcode Programming Challenge 2025

AU - Huang, Xiaowen

AU - Zhang, Xu

AU - Tao, Lvfang

AU - Mao, Renjie

AU - Zhou, Nan

AU - Zhu, Wenxi

AU - Deng, Minwen

AU - Meng, Jintao

AU - Wei, Yanjie

AU - Zhou, Amelie Chi

AU - Wang, Bingqiang

AU - Feng, Shengzhong

N1 - This work is supported by the Shenzhen-HongKong Joint Funding Project (Category A) under Grant No. SGDX20230116092056010, Shenzhen Key Laboratory of Intelligent Bioinformatics under Grant No. ZDSYS20220422103800001. Publisher Copyright: © 2025 Copyright is held by the owner/author(s).

PY - 2025/5/2

Y1 - 2025/5/2

N2 - High-performance parallel code generation is a complex and fascinating area in computer science that focuses on producing code that executes as quickly and efficiently as possible. In our paper, we designed a new architecture for parallel code generation agent with 4 inter-connected components of LLM---Memory, Planning, Tools and Action. It also incooperated with two techniques: data augmentation, prompting and retrieval-augmented editing to improve the performance of the parallel codes. Data augmentation is implemented by extracting and processing PIE dataset, and also synthesis dataset generated by LLM models with ParEval benchmark. Finally planning-oriented prompting, code verification and retrieval augmented editing are used to promote the actual performance of the LLM generated code. The evaluation results confirm that a rough speedup of 6.06X and 5.13X are achieved using Qwen2.5-Coder-7B-Instruct, Qwen2.5-Coder-14B-Instruct LLM models.

AB - High-performance parallel code generation is a complex and fascinating area in computer science that focuses on producing code that executes as quickly and efficiently as possible. In our paper, we designed a new architecture for parallel code generation agent with 4 inter-connected components of LLM---Memory, Planning, Tools and Action. It also incooperated with two techniques: data augmentation, prompting and retrieval-augmented editing to improve the performance of the parallel codes. Data augmentation is implemented by extracting and processing PIE dataset, and also synthesis dataset generated by LLM models with ParEval benchmark. Finally planning-oriented prompting, code verification and retrieval augmented editing are used to promote the actual performance of the LLM generated code. The evaluation results confirm that a rough speedup of 6.06X and 5.13X are achieved using Qwen2.5-Coder-7B-Instruct, Qwen2.5-Coder-14B-Instruct LLM models.

KW - LLM

KW - code generation

KW - parallelization

UR - http://www.scopus.com/inward/record.url?scp=105008272414&partnerID=8YFLogxK

U2 - 10.1145/3711708.3723442

DO - 10.1145/3711708.3723442

M3 - Conference proceeding

SN - 9798400714467

T3 - Proceedings of the FastCode Programming Challenge

SP - 1

EP - 7

BT - Proceedings of the 1st FastCode Programming Challenge, FCPC 2025

PB - Association for Computing Machinery (ACM)

CY - New York

Y2 - 1 March 2025 through 5 March 2025

ER -

ParaCoder: Parallel Code Generation with Large Language Model

Abstract

Publication series

Conference

User-Defined Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this