Support multibatch multithreaded parsing

[shubhamugare]

If we are parsing multiple partial codes the parsers currently run sequentially in a single thread, this should be much faster parallel in multiple threads. Example Options

[victbr]

the second link says that we should use asyncio or threading only if the program is io_bound. I wonder if multithread benefit the parsing. Using multiprocess, I met the problem that if we use spawn as start method, process init takes a long time; but fork cannot be compatible with cuda environment, is there any advice? thx!

upd: using other language without GIL might be a solution?

[shubhamugare]

Hi @victbr,

Yes, you are right, I think using other languages like Rust without GIL is probably the best solution for this.