Preparing sessions with a read/write transaction using a background executor works well as long as that executor is not being overloaded. When the executor has reached its limit, it is more efficient to allow the read/write transaction to be created in-process, as that scales with the number of user threads available, instead of being limited to the fixed thread pool of the background executor.

Fixes #151

Benchmarks - Before change

Benchmark	(incStep)	Time (ms)
SessionPoolBenchmark.burstWrite	1	4683.78
SessionPoolBenchmark.burstWrite	10	4800.407
SessionPoolBenchmark.burstWrite	20	4763.488
SessionPoolBenchmark.burstWrite	25	4741.395
SessionPoolBenchmark.burstWrite	30	4687.498
SessionPoolBenchmark.burstWrite	40	4740.128
SessionPoolBenchmark.burstWrite	50	4723.155
SessionPoolBenchmark.burstWrite	100	4779.078

Benchmarks - After change

Benchmark	(incStep)	Time (ms)
SessionPoolBenchmark.burstWrite	1	1191.581
SessionPoolBenchmark.burstWrite	10	994.402
SessionPoolBenchmark.burstWrite	20	955.75
SessionPoolBenchmark.burstWrite	25	916.664
SessionPoolBenchmark.burstWrite	30	898.538
SessionPoolBenchmark.burstWrite	40	952.803
SessionPoolBenchmark.burstWrite	50	906.222
SessionPoolBenchmark.burstWrite	100	1151.83

[skuruppu]

I don't quite understand this. Why is the waiter being added to readWaiters if inProcessPrepare? Aren't we trying to create a read/write session here?

I suppose I don't understand why it makes a difference if this is done in-process.

[olavloite]

I've added a comment on why we add it to readWaiters here.

The reasons are:

1. The normal prepare process is executed using a background executor with a fixed thread pool. When the work queue for this executor has exceeded the number of threads available in the thread pool, any requester for a read/write session will have to wait for a thread to come available, and then for the prepare to actually be executed. In the meantime, the user thread is just waiting. In those cases, it's more efficient to use the user thread to execute the BeginTransaction RPC, instead of letting it wait for a thread to come available in the thread pool.
2. In this specific case, in addition to the work queue for preparing sessions being longer than the number of threads in the thread pool, there is also no session at all available in the session pool. In that case, it's more efficient to wait for any (read) session to come available, and then execute the BeginTransaction RPC using the user thread, than to wait for both a session to come available, and then for a thread in the thread pool to come available for preparing the session.