[Feature Request] Decouple the Prefill and Decode Stage

[zhengpeirong]

The current implementation only supports the Decode function, which is a bit behind popular repos, e.g., llama.cpp.

The Prefill stage should be implemented with batched computation (GEMM) and adopt the mature communication backend e.g., OpenMPI.
This should facilitate the whole process because computing the entire prompts should be faster than decoding one by one.