Thread safe lazy initialization using proxy

Sometimes we need to defer the object creation until it is needed.
The reasons may be various like creation is time/resource consuming (database connection etc.) and we want to speed up application startup.

This is a use case for the lazy initialization pattern.

We have a a following service interface and implementation that returns current time:


Let's imagine that the initialization is very costly.

Simple lazy initialization could look as follows:
We're creating DefaultService only once when we're invoking getTime method.
This solution is fine in single threaded environment (we will cover multi threaded environment later).

The problem arises when the service  does not have few methods but a lot of them (it may happen when you're using some legacy or not well designed library).
Such implementation would look awful in this situation - you would have to implement every method and try to initialize the object in all of them.

In that case we can create a proxy object that would lazy initialization for each method invocation.
To create such proxy we need to implement java.lang.reflect.InvocationHandler interface.
There are some pitfalls the we need to overcome (more info here), that's why it's easier to use Guava's AbstractInvocationHandler:

The only thing left is thread safety. We can implement it on our own, but the safest way is to use ConcurrentInitializer from commons-lang:

I chose LazyInitializer that uses double-checked idiom but there are other implementation that might be more useful depending on the context.

Let's run all the implementations and see how they work (notice when the info about initialization is printed on the console) :

The whole project can be found on github.

Dynamic thread pool

Sometimes we're facing the problem, that we need to solve using dynamic thread pool.

By 'dynamic' I mean, that the pool is shrinking and growing automatically but it does have upper bound (so it can't grow indefinitely).

The API of the pool would be quite simple:

We should be able to submit the task to be executed asynchronously and monitor number of threads in the pool.
Having control over maximum number of threads and their longevity would be another feature.

Let's try to document functional requirements in the form of unit tests.
That would be our test setup:
Initial number of threads should be zero:
The execution of the tasks should be done concurrently:
Number of threads should grow automatically:
The size of thread pool should shrink when there is no active tasks and keep alive time was exceeded:
If the number of submitted task is greater than maximum threads number, we should block:
Actual implementation is quite simple, we need to use ThreadPoolExecutor:
There are two important things here.
First, we are required to create our own RejectedExecutionHandler to support blocking when the maximum number is exceed.
Secondly, we need to use the blocking queue of size zero (in our case I chose SynchronousQueue) - the queue needs to be filled up before we can create more than initial number of threads (in our case zero).
The whole project can be found at github.

Testing asynchronous calls with awaitility

Testing asynchronous systems is not an easy task. Let's take a simple example:


We have a class that creates file based on a given file name. The interesting thing is, that it does it asynchronously using a thread pool and returns immediately to the caller.

Let's try to create a test for it. Our first attempt could look like that:


The horrible thing about it is that Thread.sleep() invocation. Test should be fast, making them wait unnecessary is very poor solution. And what if the test sometimes fails because of overloaded hardware? Are we going to sleep even more?

To eliminate unneeded waiting, we may come up with a concept of validator:


We no longer need to sleep for a long time, but the code has been significantly polluted. Of course we can refactor our validator, make it more reusable but why reinvent the wheel? There is a nice and small library - awaitility - that will do the same for us.


In a very expressive way, we achieve the same result. Timeout, polling delay and polling interval are of course configurable.

Getting result from thread's execution with Future

To make things go faster we parallelize our computations. What if we need to use the result of the thread's execution?
Let's say we have a service that buys some product. It needs to fetch the price and quantity of the product. Fetching the price usually takes longer, so we delegate this task to a thread, while we are dealing with quantity.
To keep things simple, our PriceChecker class will be just simulating that it does something meaningful:

Now it would be good to somehow get the result of checkPrice() invocation. Runnable's run() is a void method, so we would have to do something like this:

This approach has a lot of drawbacks. We have to check in the loop if the price has already been set. What is more, price cannot be a final variable but has to be a field instead.

To deal with this kind of a problem, the Future interface should be used. Basically, it allows to get the result of thread's execution. Let's take a look at the actual usage in the context of our example:
First of all we're using Callable which is similar to Runnable but is capable of returning the result. Notice that it can also throw exception, while run() cannot. When we submit the callable to executor service, we're getting the Future object. Its method get() blocks until the computation is finished. If we want to specify the maximum time that we want to wait, there is an overloaded version that takes waiting time settings as parameters.

The runner for both cases is pretty straightforward:

Handling read and write operations on shared object

Many times we come across a problem, when in the multithreaded environment some shared object has frequent read operations and write operations are rare. If we could use a concurrent collection that would be the best option in most of the cases, but sometimes we can't.

Let's imagine a situation when this shared object is in a 3rd party library. In our example this object is a Book class that I used in previous posts.

Most of the time we want to show information about the book, but exceptionally the author modifies the content and therefore the number of pages needs to be changed.

Using synchronized for the methods that read/write the book would be a poor solution.
Instead, we could use ReadWriteLock which allows to differentiate between read and write operations. Read operation is concurrent, which means that all the threads have access to the object. Only write operation is exclusive and blocks other threads.

Everything we do after acquiring the lock must be put in a try block, so that unlocking is done even if the exception is thrown.

Let's see it in action:

We have 10 threads that are performing 100 tasks. Only 10% of them are write tasks. When we run it, we can see that read operations are concurrent (lines about the reading thread and book info are mixed from time to time) and write operations are exclusive (info about writing thread and successful update are always in the correct order).

The example with the book is purely educational and not too realistic. In real life we would probably have some kind of a large collection. It's good to read ReentrantReadWriteLock and ReadWriteLock javadoc, because sometimes using them may bring more overhead than mutual exclusion.

Coordinating threads

In our multithreaded application we may need to coordinate the work of our threads. The most occurring situation is waiting for all the threads once they finish their work.

Let's take a simple example: we have 2 threads that are doing some mathematical operations and we need to wait for both of them before we proceed with further processing. We could do that using thread's join() method:

If the number of threads is higher, this implementation will look much more nastier, as we would need to invoke join() method on all of them.


That's why the better solution is to use CountDownLatch, "that allows one or more threads to wait until a set of operations being performed in other threads completes", as Javadoc says.

In the constructor of CountDownLatch we specify a count - as each of our threads will be doing only one countdown once work is finished, it is equal to the number of threads. await() method makes the current thread to stop, until latch has counted down to zero. It also provides possibility to wait for the specified time.

If threads must repeatedly count down in a similar way, we would need to use CyclicBarrier instead.  

Concurrent map with timed out elements using Guava library

Sometimes we have a need to use a map, in which elements live only for a specified amount of time.
We could implement such map ourselves (by overriding removeEldestEntry() from LinkedHashMap or using thread that will be removing elements periodically) but the better idea is to use already existing implementation. Google's guava library (http://code.google.com/p/guava-libraries) is the solution for us.
Let's write some simple unit tests to check the API and verify behaviour:
Map is created using the builder pattern. Apart from specifying the time of elements expiration, it allows among others, setting the maximum map's size or the listener that will be notified each time an entry is removed. As the built object is an instance of Cache, we need to create ConcurrentMap from it using asMap() method.
Two tests verify that added element is removed only when the timeout expires.

Concurrent collections

As I already mentioned, situation that requires concurrent access to a collection is pretty often. Let's say that we have a map that is accessed by many threads.
Synchronizing it like this is not the best choice, as only one thread at a time will be able to access it.
Another approach is useful only in a few cases. The factory method creates synchronized wrapper for our map that makes it thread-safe. However if we want to iterate over it, we still need a synchronization, otherwise it will throw ConcurrentModificationException when some other thread will modify it while we're iterating.

The better solution is to use ConcurrentMap  It not only gives us a concurrent access, but we can also iterate over it without synchronization.

Let's imagine a situation that we need to put some value to the map, on a condition that it isn't there already.
Doing it that way is a wrong solution: After we check that the key is not there, another thread could put some entry to the map and we will override existing value. Basically our operation in this implementation is not atomic.
So should we add the synchronization? No, the ConcurrentMap provides a method that will make this operation atomic:

ConcurrentMap provides also other methods that are able to perform atomic replacement or removal of an entry, only if it is mapped to a specified value.
There are also a concurrent implementations of List and Set - CopyOnWriteArrayList and CopyOnWriteArraySet. Every mutative operation on them creates a new copy, so they should be used with a special care. 

Blocking queues

It happens that in our system's architecture we have a situation of concurrent access to some collection. Whether it's a server that handles requests via sockets or a JMS listener, it usually stores incoming objects into a collection and other worker processes them. It can be illustrated as a well known producer-consumer problem.

An example implementation could look like that:
We have a class that serves as our producer. In an endless loop we're sending random strings to a queue. Access to it is synchronized by acquiring its monitor. We're sleeping 1 ms so that the processor is not over-killed.

Let's take a look at a consumer: It synchronizes queue in a similar way. What is more, before taking from the queue, it checks if it's not empty. It also sleeps 1 ms for the same reason as producer.

Class that glues them together:

We're using LinkedList as an implementation of our Queue. Producer and consumer are started in a separate threads.

There are at least 3 major drawbacks with this implementation.
First of all we're acquiring monitor of the queue, so we're completely blocking access to it not allowing for a concurrent access.
Secondly we need to check if the queue is not empty before taking from it.
Last but not least, consumer sleeps some specified time (sleeping when you're supposed to be working is not the best thing to do) and our performance decreases.

The remedy for above problems is BlockingQueue.

The new producer is similar to previous one, except that it operates on BlockingQueue and doesn't use any synchronization on it, because it's thread safe.

There are some changes in consumer:
Apart from not using synchronization, we don't need to check if the queue is not empty, as method take() is doing it for us. We also don't have to sleep anymore, as it is also handled by this method: as soon as an element is available it will be taken from the queue.

Class to run it:
It hasn't changed much, we're using LinkedBlockingQueue as an implementation.
There are other implementations of BlockingQueue that may be helpful for our needs (e.g. that introduces priority to elements or some delay that is needed before taking).

Synchronization pitfall

Synchronized method modifier is equivalent to block synchronized(this). Not only invoking of such methods is expensive for virtual machine, but they're also a little bit risky. Object's monitor can be taken not only during method invocation, but also by another object that has a reference to it.
Let's take a look at this example:
Class Resource contains some object and provides synchronized methods for fetching it.
getVeryImportantObject() has synchronized modifier.
getVeryImportantObjectWithMutex() uses synchronized block on private object.

We also have an evil class:
Cheater owns a reference to Resource and it blocks it in blockResource() method.

The class that glues it together:
Pitfall invokes 3 threads. The first one uses Cheater to take the monitor of created object of Resource class. Next threads start after 1 second in order to allow the first one performing its task. The second thread tries to get resource object by using method with synchronized modifier, however it never succeeds. The third thread grabs this object by using method with internal synchronized block.
Project Lombok provides an interesting way of not getting into this trap (http://projectlombok.org/features/Synchronized.html)

Running threads using Executors

There are several ways of running threads.
We can create an instance of Runnable and pass it to Thread class:

Or we can override the run() method in the Thread class directly:
However the better solution (in most cases) is using the Executor interface.
It gives us a couple of benefits. In previous cases, if we want to start again the thread that has come to the terminated state we will get IllegalThreadStateException.
With Executor we can reuse the existing Runnable :
To prevent adding any more tasks to our executor we need to call shutdown() method.

What is even more important, it provides the ability to easily switch the implementation of running: using single thread, different kinds of thread pools or even scheduled periodic task.
We would just need to use different static factory method from Executors utility class:
This time our runnable is run using two threads from executor's thread pool.

java.util.concurrent.atomic package

Classes from java.util.concurrent.atomic package allow us to use single variables concurrently without blocking access to them. The typical usage example is increasing the counter that is modified by many threads.

Instead of using the synchronized block, that blocks the access to the counter:

or the synchronized method equivalent to the block with 'synchronized(this)':

you can use AtomicLong class, that guarantees non-blocking access:

Method incrementAndGet() besides incrementing counter's value, returns it.

Another typical usage is providing non-blocking access to an object, that is shared by many threads. Instead of synchronizing getter/setters methods:

you can use AtomicReference class: