Shijie Peng (彭士杰)

UCAS & SIAT, CAS

I am a PhD student in Computer Science at the University of Chinese Academy of Sciences (UCAS), advised by Prof. Kejiang Ye (叶可江), and affiliated with the Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences.

My research focuses on systems for AI, with an emphasis on making large-model inference efficient, elastic, and reliable in real-world cloud environments. I study the interaction between model execution, serving runtimes, scheduling, and heterogeneous hardware.

Research Interests

LLM serving systems: architecture and runtime design for low-latency, high-throughput inference
Serverless and cloud systems: elastic scheduling and resource management for AI-native workloads
Operating and distributed systems: performance optimization across heterogeneous clusters
Hardware-software co-design for AI: practical acceleration strategies for production inference pipelines

Research & Collaborations

I collaborate with academic and industry teams on large-scale AI infrastructure, especially model serving, scheduling, and end-to-end system efficiency. My work has appeared in venues including EuroSys, ICPP, TPDS, SoCC, ICDCS, and ICWS; see the publications page for the complete list.

News

Aug 24, 2026	Our paper “Valve” has been conditionally accepted to EuroSys’27!
Jul 06, 2026	Our paper “ReliefServe” has been accepted to ICPP’26!
Sep 10, 2025	Our paper “FlexPipe” has been accepted to EuroSys’26!

View all news →

Selected Publications

EuroSys27

Valve: Risk-Bounded Offline Execution in Online Generative Serving

Yanying Lin , Haiying Shen , Shijie Peng, and 6 more authors

In Proceedings of the 22nd European Conference on Computer Systems, 2027

Bib

@inproceedings{lin2027valve,
  title = {Valve: Risk-Bounded Offline Execution in Online Generative Serving},
  author = {Lin, Yanying and Shen, Haiying and Peng, Shijie and Lu, Chengzhi and Ma, Chong and Shen, Min and Xu, Chengzhong and Qu, Lin and Ye, Kejiang},
  booktitle = {Proceedings of the 22nd European Conference on Computer Systems},
  year = {2027},
}

ICPP

ReliefServe: Relieving GPU Pressure in Multi-Model Serving via Selective CPU Escape

Shijie Peng , Yanying Lin , Chengzhi Lu, and 3 more authors

In Proceedings of the 55th IEEE International Conference on Parallel Processing, 2026

DOI Bib HTML PDF

@inproceedings{peng2026reliefserve,
  title = {ReliefServe: Relieving GPU Pressure in Multi-Model Serving via Selective CPU Escape},
  author = {Peng, Shijie and Lin, Yanying and Lu, Chengzhi and Wu, Shuaipeng and Xu, Chengzhong and Ye, Kejiang},
  booktitle = {Proceedings of the 55th IEEE International Conference on Parallel Processing},
  year = {2026},
  doi = {10.1145/3832810.3832827},
}

EuroSys
FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters

Yanying Lin , Shijie Peng , Chengzhi Lu, and 2 more authors

In Proceedings of the 21st European Conference on Computer Systems, 2026

Insight Bib HTML PDF

Insight: We find that for pipeline inference, the optimal online pipeline structure varies greatly depending on the incoming request distribution. Longer pipelines can absorb load spikes by dispersing demand, while shorter pipelines reduce communication overhead. Dynamically adjusting the pipeline structure according to the request distribution is an effective approach in serverless environments.
@inproceedings{lin2026flexpipe, title = {FlexPipe: Adapting Dynamic LLM Serving Through Inflight Pipeline Refactoring in Fragmented Serverless Clusters}, author = {Lin, Yanying and Peng, Shijie and Lu, Chengzhi and Xu, Chengzhong and Ye, Kejiang}, booktitle = {Proceedings of the 21st European Conference on Computer Systems}, year = {2026}, }

TPDS

Workload-Adapted Resource Allocation for LLM Distributed Serving in Serverless Clusters

Yanying Lin , Shijie Peng , Yanbo Li, and 4 more authors

IEEE Transactions on Parallel and Distributed Systems, 2026

Bib

@article{lin2026workload,
  title = {Workload-Adapted Resource Allocation for LLM Distributed Serving in Serverless Clusters},
  author = {Lin, Yanying and Peng, Shijie and Li, Yanbo and Luo, Shutian and Shen, Haiying and Ye, Kejiang and Xu, Chengzhong},
  journal = {IEEE Transactions on Parallel and Distributed Systems},
  year = {2026},
}

Cluster
ROCK: Serving Multimodal Models in Cloud with Heterogeneous-Aware Resource Orchestration for Thousands of LoRA Adapters

Shuaipeng Wu , Yanying Lin , Shijie Peng, and 6 more authors

In Proceedings of the 2025 IEEE International Conference on Cluster Computing, 2025

Insight Bib PDF

Insight: This work analyzes the performance of different request workloads in production-level environments and proposes a heterogeneous graph matching algorithm to adapt to different levels of resource characteristics and inference performance of various workloads. ROCK is a system-level resource orchestration framework designed to optimize the deployment of thousands of LoRA adapters in a cloud environment through intelligent resource allocation and workload-aware scheduling.
@inproceedings{wu2025rock, title = {ROCK: Serving Multimodal Models in Cloud with Heterogeneous-Aware Resource Orchestration for Thousands of LoRA Adapters}, author = {Wu, Shuaipeng and Lin, Yanying and Peng, Shijie and Chen, Wenyan and Ma, Chong and Shen, Min and Chen, Le and Xu, Chengzhong and Ye, Kejiang}, booktitle = {Proceedings of the 2025 IEEE International Conference on Cluster Computing}, year = {2025}, }

IEEE TSC
Serving LLM in Distributed GPU Cluster With Fine-Grain Pipeline Constraints

Yanying Lin , Shijie Peng , Shuaipeng Wu, and 4 more authors

IEEE Transactions on Services Computing, 2025

Insight Bib

Insight: LLMs are increasingly deployed in distributed GPU clusters using pipeline parallelism for efficient inference. However, existing approaches treat the entire pipeline as a monolithic unit, resulting in coarse-grained resource management and suboptimal SLO compliance. Each pipeline stage has distinct workload characteristics, but current systems cannot leverage this for better performance control. We propose fine-grained pipeline constraints that manage each stage independently, enabling precise SLO allocation at the stage level.
@article{lin2025serving, title = {Serving LLM in Distributed GPU Cluster With Fine-Grain Pipeline Constraints}, author = {Lin, Yanying and Peng, Shijie and Wu, Shuaipeng and Li, Yanbo and Lu, Chengzhi and Ye, Kejiang and Xu, Chengzhong}, journal = {IEEE Transactions on Services Computing}, year = {2025}, volume = {18}, number = {5}, }

ICDCS
QUART: Latency-Aware FaaS System for Pipelining Large Model Inference

Yanying Lin , Yanbo Li , Shijie Peng, and 5 more authors

In Proceedings of the 44th IEEE International Conference on Distributed Computing Systems, 2024

Insight Bib PDF

Insight: We find that when using pipeline inference for LLMs, each stage exhibits significant load differences due to varying resource allocation and throughput across different workloads. For example, under stable or bursty traffic patterns, throughput disparities between upstream and downstream stages can create performance-critical bottlenecks in the pipeline.
@inproceedings{lin2024quart, title = {QUART: Latency-Aware FaaS System for Pipelining Large Model Inference}, author = {Lin, Yanying and Li, Yanbo and Peng, Shijie and Tang, Yingfei and Luo, Shutian and Shen, Haiying and Xu, Chengzhong and Ye, Kejiang}, booktitle = {Proceedings of the 44th IEEE International Conference on Distributed Computing Systems}, year = {2024}, }

ICWS

Planck: Optimizing LLM Inference Performance in Pipeline Parallelism with Fine-Grained SLO Constraint

Yanying Lin , Shijie Peng , Shuaipeng Wu, and 4 more authors

In Proceedings of the 31st IEEE International Conference on Web Services, 2024

Insight

Insight: This work proposes preliminary ideas for SLO allocation in LLM pipeline parallelism. We explore fine-grained SLO constraint mechanisms to optimize inference performance by intelligently distributing service level objectives across different pipeline stages, enabling more efficient resource utilization and better meeting diverse application requirements.

CSCWD

EINS: Edge-Cloud Deep Model Inference with Network-Efficiency Schedule in Serverless

Shijie Peng , Yanying Lin , Wenyan Chen, and 3 more authors

In Proceedings of the 27th International Conference on Computer Supported Cooperative Work in Design, 2024

Bib

@inproceedings{peng2024eins,
  title = {EINS: Edge-Cloud Deep Model Inference with Network-Efficiency Schedule in Serverless},
  author = {Peng, Shijie and Lin, Yanying and Chen, Wenyan and Tang, Yingfei and Duan, Xu and Ye, Kejiang},
  booktitle = {Proceedings of the 27th International Conference on Computer Supported Cooperative Work in Design},
  year = {2024},
}