[FIXED] Which usecases are suitable for Dispatchers.Default in Kotlin?

Issue

Based on the documentation the threadpool size of IO and Default dispatchers behave as follows:

• Dispatchers.Default: By default, the maximal level of parallelism used by this dispatcher is equal to the number of CPU cores, but is at least two.
• Dispatchers.IO: It defaults to the limit of 64 threads or the number of cores (whichever is larger).

Unless there is one piece of information that I am missing, performing lots of CPU intensive works on Default is more efficient (faster) because context switching will happen less often.

But the following code actually runs much faster on Dispatchers.IO:

fun blockingWork() {
    val startTime = System.currentTimeMillis()
    while (true) {
        Random(System.currentTimeMillis()).nextDouble()
        if (System.currentTimeMillis() - startTime > 1000) {
            return
        }
    }
}

fun main() = runBlocking {
    val startTime = System.nanoTime()
    val jobs = (1..24).map { i ->
        launch(Dispatchers.IO) { // <-- Select dispatcher here
            println("Start #$i in ${Thread.currentThread().name}")
            blockingWork()
            println("Finish #$i in ${Thread.currentThread().name}")
        }
    }
    jobs.forEach { it.join() }
    println("Finished in ${Duration.of(System.nanoTime() - startTime, ChronoUnit.NANOS)}")
}

I am running 24 jobs on a 8-core CPU (so, I can keep all the threads of Default dispatcher, busy). Here is the results on my machine:

Dispatchers.IO --> Finished in PT1.310262657S
Dispatchers.Default --> Finished in PT3.052800858S

Can you tell me what I am missing here? If IO works better, why should I use any dispatcher other than IO (or any threadpool with lots of threads).

Solution

Answering your question: Default dispatcher works best for tasks that do not feature blocking because there is no gain in exceeding maximum parallelism when executing such workloads concurrently(the-difference-between-concurrent-and-parallel-execution).

https://www.cs.uic.edu/~jbell/CourseNotes/OperatingSystems/5_CPU_Scheduling.html

---

Your experiment is flawed. As already mentioned in the comments, your blockingWork is not CPU-bound but IO-bound. It's all about waiting - periods when your task is blocked and CPU cannot execute its subsequent instructions. Your blockingWork in essence is just "wait for 1000 milliseconds" and waiting 1000ms X times in parallel is going to be faster than doing it in sequence. You perform some computation(generating random number - which in essence might also be IO-bound), but as already noted, your workers are generating more or less of those numbers, depending on how much time the underlying threads have been put to sleep.

I performed some simple experiments with generating Fibonacci numbers(often used for simulation of CPU workloads). However, after taking into the account the JIT in the JVM I couldn't easily produce any results proving that the Default dispatcher performs better. Might be that the context-switching isn't as significant as one may believe. Might be that the dispatcher wasn't creating more threads with IO dispatcher for my workload. Might be that my experiment was also flawed. Can't be certain - benchmarking on JVM is not simple by itself and adding coroutines(and their thread pools) to the mix certainly isn't making it any simpler.

However, I think there is something more important to consider here and that is blocking. Default dispatcher is more sensitive to blocking calls. With fewer threads in the pool, it is more likely that all of them become blocked and no other coroutine can execute at that time.

Your program is working in threads. If all threads are blocked, then your program isn't doing anything. Creating new threads is expensive(mostly memory-wise), so for high-load systems that feature blocking this becomes a limiting factor. Kotlin did an amazing job of introducing "suspending" functions. The concurrency of your program is not limited to the number of threads you have anymore. If one flow needs to wait, it just suspends instead of blocking the thread. However, "the world is not perfect" and not everything "suspends" - there are still "blocking" calls - how certain are you that no library that you use performs such calls under the hood? With great power comes great responsibility. With coroutines, one needs to be even more careful about deadlocks, especially when using Default dispatcher. In fact, in my opinion, IO dispatcher should be the default one.

---

EDIT

TL;DR: You might actually want to create your own dispatchers.

Looking back it came to my attention that my answer is somewhat superficial. It's technically incorrect to decide which dispatcher to use by only looking at the type of workload you want to run. Confining CPU-bound workload to a dispatcher that matches the number of CPU cores does indeed optimize for throughput, but that is not the only performance metric.

Indeed, by using only the Default for all CPU-bound workloads you might find that your application becomes unresponsive! For example, let's say we have a "CPU-bound " long-running background process that uses the Default dispatcher. Now if that process saturates the thread pool of the Default dispatcher then you might find that the coroutines that are started to handle immediate user actions (user click or client request) need to wait for a background process to finish first! You have achieved great CPU throughput but at the cost of latency and the overall performance of your application is actually degraded.

Kotlin does not force you to use predefined dispatchers. You can always create your own dispatchers custom-cut for the specific task you have for your coroutines.

Ultimately it's about:

1. Balancing resources. How many threads do you actually need? How many threads you can afford to create? Is it CPU-bound or IO-bound? Even if it is CPU-bound, are you sure you want to assign all of the CPU resources to your workload?
2. Assigning priorities. Understand what kind of workloads run on your dispatchers. Maybe some workloads need to run immediately and some other might wait?
3. Preventing starvation deadlocks. Make sure your currently running coroutines don't block waiting for a result of a coroutine that is waiting for a free thread in the same dispatcher.

Answered By - Aleksander Stelmaczonek

Answer Checked By - David Marino (PHPFixing Volunteer)