Modern large language models (LLMs) reach tens or even hundreds of billions of parameters, making it impossible to fit them into the memory of a single GPU. To perform inference for such models, multiple accelerators are used, across which the model is distributed. However, many current distributed inference frameworks require modifications to the model's source code or adaptation of its architecture. Tools that do not require model changes are typically oriented towards using tensor parallelism within a single node. In the case of inter-node interaction, inference performance with such partitioning can significantly decrease due to frequent data exchange between accelerators and limited network bandwidth. One way to reduce inter-node communication is the use of pipeline parallelism. However, existing solutions rarely provide the ability to use pipeline parallelism for distributed inference without modifying the model code. This paper presents a concept for partitioning a model between accelerators by modifying its layers without affecting the forward pass function itself. A realization of pipeline parallelism with computational overlap between graphics accelerators is also presented. The proposed approaches allow for improved distributed inference performance without changing the LLM source code.