Improve decode latency

[scse-l]

I've done some profiling and find that decoding generated tokens one by one costs a lot (20% ~ 30% of total costs). Is there some work we can do to optimize this ?
PS: I've tried to decode generated tokens parallelly by process pool. But it works worse due to high overhead of cross-process conmunications.