Sunday, 18 November 2012

Mutation testing with PIT

How can we know that our unit tests are good?

Code coverage can't say that we have proper unit tests - it's just a side effect of using TDD.
It says what code has been executed in the unit test - we can write a test which just executes a method and without any assertions we'll have 100% code coverage.

Let's take a look at the following class:

public class NotThoroughlyTestedClass {
public boolean isHigherOrEqualToZero(int number) {
// '=' condition no tested
return number >= 0;
}
}
It's a simple class that tells us if a given number is higher or equal to zero.

Let's take a look at the unit test:

public class NotThoroughlyTestedClassTest {
private NotThoroughlyTestedClass testedClass = new NotThoroughlyTestedClass();
private int number;
@Test
public void shouldReturnTrueWhenNumberHigherThanZero() {
// given
number = 1;
// when
boolean result = testedClass.isHigherOrEqualToZero(number);
// then
assertThat(result, is(true));
}
@Test
public void shouldReturnFalseWhenNumberLowerThanZero() {
// given
number = -1;
// when
boolean result = testedClass.isHigherOrEqualToZero(number);
// then
assertThat(result, is(false));
}
}

When we execute cobertura on this test, we will see 100% coverage, both line and branch, even though it forgets about the case where the number is 0.

Mutation testing is an idea that tries to measure the quality of unit tests. They are run against modified version of the source code (by applying various mutators - hence the name). If the application code is changed, it should produce different result, therefore the unit test should fail. If it doesn't fail, it may indicate problem in the test suite.

PIT (http://pitest.org/) is a bytecode mutation testing system that automatically applies mutators to the application code.

If we run it against above mentioned class, it will produce a nice HTML report saying that 'changed conditional boundary' has survived - which means that we didn't test all boundary conditions.

We can integrate execution of PIT into our build (there are plugins for maven, ant and gradle) or we can execute it within our IDE (plugins for Eclipse and IntelliJ - that one I've created recently).

The project is actively developed, so we can expect more features to come.

Wednesday, 3 October 2012

JUnit - hamcrest incompatibility

When we're using the latest version of JUnit (4.10) and hamcrest (1.3) together we might come across an unexpected behaviour.

Let's take a look at a simple test:

@Test
public void shouldHaveOneElementInList() {
List<String> list = Arrays.asList("firstElement", "secondElement");
assertThat(list, hasSize(1));
}
We might think that this test will fail because of the assertion, but the result is:

java.lang.NoSuchMethodError: org.hamcrest.Matcher.describeMismatch(Ljava/lang/Object;Lorg/hamcrest/Description;)V

That's because the default JUnit has a dependency to hamcrest 1.1 and also the artifact org.junit:junit has some hamcrest classes inside its jar. The hasSize matcher is not present in 1.1.

To avoid this problem we need to declare following dependencies in our project:

<dependency>
<groupId>junit</groupId>
<artifactId>junit-dep</artifactId>
<version>4.10</version>
<scope>test</scope>
<exclusions>
<exclusion>
<artifactId>hamcrest-core</artifactId>
<groupId>org.hamcrest</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
view raw HamcrestBug.xml hosted with ❤ by GitHub
We need to use JUnit distribution that doesn't have any hamcrest internals and we need to exclude hamcrest dependency as well. What's interesting, we will not bump into this behaviour unless our assertion throws an error. 

Saturday, 15 September 2012

Manipulating collections with lambdaj

Java is not that nice regarding manipulations on collections even though tasks like iterating happen to us on a daily basis.

To ease the pain a little bit we can use the lambdaj library that allows manipulating collections without using loops.

Here is a simple unit test that shows some of lambdaj API and features
(Person class is a pojo with age and name fields, and implemented hashCode and equals methods):


private List<Person> people;
private Person robin;
private Person shinji;
private Person tom;
@Before
public void setUp() {
robin = new Person("Robin van Persie", 29);
shinji = new Person("Shinji Kagawa", 23);
tom = new Person("Tom Cleverley", 23);
people = asList(robin, shinji, tom);
}
@Test
public void selectsByAge() {
List<Person> resultIteratively = new ArrayList<>();
for (Person person : people) {
if (person.getAge() == 23) {
resultIteratively.add(person);
}
}
List<Person> resultWithLambda = select(people, having(on(Person.class).getAge(), equalTo(23)));
assertThat(resultIteratively, is(asList(shinji, tom)));
assertThat(resultWithLambda, is(asList(shinji, tom)));
}
@Test
public void sumsAge() {
int resultIteratively = 0;
for (Person person : people) {
resultIteratively += person.getAge();
}
int resultWithLambda = sum(people, on(Person.class).getAge());
assertThat(resultIteratively, is(75));
assertThat(resultWithLambda, is(75));
}
As you can see the code using lambdaj is much shorter and more readable.

The features of the library go far beyond that, there are many aggregation and filtering functions, as well as grouping, sorting and possibility to invoke a method for each collection's item.

The main drawback is that it's slower than standard iterative approach, so if speed is your concern, you need to watch out. However you can always use it in the unit tests.

Saturday, 7 July 2012

Extending maven with some groovy magic

Maven is very strict regarding its conventions and the build lifecycle. Even though it has a lot of plugins that extend its functionality, it doesn't always do what we want it to do.
(If you want more freedom take a look at gradle)

Writing our own plugin is time consuming but we can easily extend maven capabilities with executing groovy code during the build.
All we need to do is add gmaven plugin into pom.xml.
Let's see an example that will make our maven build tell us which operating system we are using:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>pl.mjedynak</groupId>
<artifactId>gmaven</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.gmaven</groupId>
<artifactId>gmaven-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>execute</goal>
</goals>
<configuration>
<source>
import org.apache.commons.lang3.SystemUtils
if (SystemUtils.IS_OS_WINDOWS) {
println("running on Windows")
} else {
println("running on Unix")
}
</source>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.1</version>
</dependency>
</dependencies>
</project>
When we add the plugin we need to specify in which phase it will be run ('validate') and which goal of the plugin will be invoked ('execute'). Then in the source section we write our groovy code that will be executed. As you can see we have access to the dependencies declared in dependencies section of the pom file.

If we don't want to inline our script we can specify the file that contains it:

<source>${pom.basedir}/src/main/script/whichOS.groovy</source>
view raw specifyFile.xml hosted with ❤ by GitHub
Another interesting feature is access to maven variables that can be used in the xml configuration of the build.

<plugin>
<groupId>org.codehaus.gmaven</groupId>
<artifactId>gmaven-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>execute</goal>
</goals>
<configuration>
<source>
project.properties['findbugsSettingsDirectory'] = 'c:/findbugs'
</source>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>2.3.3</version>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
<configuration>
<includeFilterFile>${findbugsSettingsDirectory}/findbugs.xml</includeFilterFile>
</configuration>
</plugin>
The findbugsSettingsDirectory variable is set inside our script and then used in the configuration of findbugs plugin.

Monday, 4 June 2012

Testing legacy code with Mockito spies

When we're working with legacy code, before making any changes we should write unit tests that will prevent us from breaking existing functionality. However most of the time, writing tests for legacy code is not easy.

Let's take a look at a simple example, that will try to show the challenges that we may face:

public class HardToTestClass {
public Double calculateDiscount(String customerName, Integer amount) {
Double discount = 0.0;
if (amount > 10 || CustomerService.isImportantCustomer(customerName)) {
discount = 25.0;
}
return discount;
}
}
We have a class that calculates a discount for a customer based on his name and the amount of product that he's buying.

The problem hides in a static method call CustomerService.isImportantCustomer(). In the old code we can see a lot of static methods. Let's say that the one in our example is calling the database.
We need to mock it in order to write a proper unit test.

First of all, we will extract the static call to a separate method:

public class HardToTestClass {
public Double calculateDiscount(String customerName, Integer amount) {
Double discount = 0.0;
if (amount > 10 || isImportantCustomer(customerName)) {
discount = 25.0;
}
return discount;
}
boolean isImportantCustomer(String customerName) {
return CustomerService.isImportantCustomer(customerName);
}
}
Refactoring using IDE (extracting methods, renaming etc.) is considered to be safe. Once we did that we can use a nice feature of Mockito - spies. 
If we declare our tested class as a spy, we can mock its method the same way as with standard mocks. The test for our class could look like:

@RunWith(MockitoJUnitRunner.class)
public class HardToTestClassTest {
private static final double ZERO_DISCOUNT = 0.0;
@Spy
private HardToTestClass hardToTestClass;
private String customerName = "Neal Stephenson";
private Integer lowAmount = 1;
@Test
public void shouldReturnZeroDiscountIfAmountIsLowAndCustomerIsNotImportantOne() {
// given
given(hardToTestClass.isImportantCustomer(customerName)).willReturn(false);
// when
Double result = hardToTestClass.calculateDiscount(customerName, lowAmount);
// then
assertThat(result, is(ZERO_DISCOUNT));
}
}
The Mockito API for spies is similar to the one for mocks (the syntax is different for overriding void methods).

Sunday, 13 May 2012

Testing getters and setters with openpojo

Testing getters and setters is a little bit controversial topic. However, if we decide to do it or if we are forced to, we can ease the task with openpojo library.

Generally standard getters and setters can be generated by your IDE but testing them is a manual work - it takes some time and seems to be boring. So if the code was generated, why can't we automate the testing?
openpojo doesn't generate the unit test but it allows to test your POJO with a very small effort.

Let's take a look at an example. We have a Person class which is a POJO with generated getters and setters.

package pl.mjedynak.openpojo;
public class Person {
private String name;
private int age;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof Person)) return false;
Person person = (Person) o;
if (age != person.age) return false;
if (name != null ? !name.equals(person.name) : person.name != null) return false;
return true;
}
@Override
public int hashCode() {
int result = name != null ? name.hashCode() : 0;
result = 31 * result + age;
return result;
}
}
view raw Person.java hosted with ❤ by GitHub

To test it we need to create just one class:

public class PojoTest {
private static final String POJO_PACKAGE = "pl.mjedynak.openpojo";
private List<PojoClass> pojoClasses;
private PojoValidator pojoValidator;
@Before
public void setup() {
pojoClasses = PojoClassFactory.getPojoClasses(POJO_PACKAGE, new FilterPackageInfo());
pojoValidator = new PojoValidator();
pojoValidator.addRule(new GetterMustExistRule());
pojoValidator.addRule(new SetterMustExistRule());
pojoValidator.addTester(new SetterTester());
pojoValidator.addTester(new GetterTester());
}
@Test
public void testPojoStructureAndBehavior() {
for (PojoClass pojoClass : pojoClasses) {
pojoValidator.runValidation(pojoClass);
}
}
}
view raw PojoTest.java hosted with ❤ by GitHub
Our class defines which rules openpojo will apply and what tests it will execute. We also specify which package will be tested - if we have a lot of classes we can test them all in one go. In our case we're only testing if  getters and setters exist and if they behaving in a conventional way.
Running the class with any coverage tool will show us that the get and set methods are executed.
There are more rules that we can apply (like checking if there are no public fields, no primitive values etc.). We can write our own rules as well.

Thursday, 19 April 2012

JMS with Spring configured in Java

Handling JMS using standard JMS API is a tedious task - you need to create a lot of boilerplate code.
Spring provides an abstraction layer that eases this pain.
In the following example I'm using ActiveMQ (which needs to be started before running any of the code below) but Spring works with other major JMS implementations as well.

In order to send a JMS message we can use JMS template. Here's our sender that's using it:

public class JmsSender {
private JmsTemplate jmsTemplate;
public void sendText(final String text) {
jmsTemplate.send(new MessageCreator() {
public Message createMessage(Session session) throws JMSException {
return session.createTextMessage(text);
}
});
}
public void setJmsTemplate(JmsTemplate jmsTemplate) {
this.jmsTemplate = jmsTemplate;
}
}
view raw JmsSender.java hosted with ❤ by GitHub

Spring beans needs to be configured in a context. Let's define Spring configuration in Java code:

@Configuration
public class JmsSenderSpringConfig {
@Bean
public JmsTemplate jmsTemplate() {
JmsTemplate jmsTemplate = new JmsTemplate();
jmsTemplate.setDefaultDestination(new ActiveMQQueue("jms.queue"));
jmsTemplate.setConnectionFactory(connectionFactory());
return jmsTemplate;
}
@Bean
public ActiveMQConnectionFactory connectionFactory() {
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL("tcp://localhost:61616");
return activeMQConnectionFactory;
}
@Bean
public JmsSender jmsSender() {
JmsSender jmsSender = new JmsSender();
jmsSender.setJmsTemplate(jmsTemplate());
return jmsSender;
}
}

JMS template must have a connection factory with a broker url and a destination (in our case it's a queue).
The XML configuration equivalent would look like:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context.xsd">
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="defaultDestinationName" value="jms.queue"/>
</bean>
<bean id="jmsSender" class="pl.mjedynak.jms.sender.JmsSender">
<property name="jmsTemplate" ref="jmsTemplate"/>
</bean>
</beans>

Java configuration provides not only compile-time checking but in my opinion it is more straightforward and easier to control.
Unfortunately we can't escape XML completely, we need to turn on component scanning so that Spring will read configuration from classes in our package:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context.xsd">
<context:component-scan base-package="pl.mjedynak"/>
</beans>

The example code that would run the sender:

public class JmsSenderRunner {
public static void main(String[] args) {
ApplicationContext applicationContext = new ClassPathXmlApplicationContext("sender-context.xml");
JmsSender jmsSender = (JmsSender) applicationContext.getBean("jmsSender");
jmsSender.sendText("hellooooo " + new Date());
}
}
Apart from the sender we can also create the receiver. One of the solutions is to make him asynchronous by implementing MessageListener interface.

public class JmsReceiver implements MessageListener {
@Override
public void onMessage(Message message) {
System.out.println("Received: " + message);
}
}

The Spring config looks as follows:

@Configuration
public class JmsReceiverSpringConfig {
@Bean
public JmsReceiver jmsReceiver() {
return new JmsReceiver();
}
@Bean
public ActiveMQConnectionFactory connectionFactory() {
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL("tcp://localhost:61616");
return activeMQConnectionFactory;
}
@Bean
public DefaultMessageListenerContainer messageListenerContainer() {
DefaultMessageListenerContainer messageListenerContainer = new DefaultMessageListenerContainer();
messageListenerContainer.setConnectionFactory(connectionFactory());
messageListenerContainer.setDestinationName("jms.queue");
messageListenerContainer.setMessageListener(jmsReceiver());
return messageListenerContainer;
}
}

Our receiver is set in the message listener container with the connection factory and the destination.
Receiver is missing the same XML configuration as sender, and the running code is analogical.

Here's the list of maven dependencies:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>3.1.1.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jms</artifactId>
<version>3.1.1.RELEASE</version>
</dependency>
<dependency>
<groupId>cglib</groupId>
<artifactId>cglib</artifactId>
<version>2.2</version>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-core</artifactId>
<version>5.5.1</version>
</dependency>
<dependency>
<groupId>org.apache.xbean</groupId>
<artifactId>xbean-spring</artifactId>
<version>3.5</version>
</dependency>
view raw deps.xml hosted with ❤ by GitHub

Monday, 2 April 2012

Testing asynchronous calls with awaitility

Testing asynchronous systems is not an easy task. Let's take a simple example:

public class FileCreator {
private ExecutorService executor = Executors.newCachedThreadPool();
public void createFile(final String fileName) {
executor.submit(new Callable<Path>() {
@Override
public Path call() throws Exception {
// simulate delay
Thread.sleep(500);
return Files.createFile(Paths.get(fileName));
}
});
}
}

We have a class that creates file based on a given file name. The interesting thing is, that it does it asynchronously using a thread pool and returns immediately to the caller.

Let's try to create a test for it. Our first attempt could look like that:

public class FileCreatorTestWithSleeping {
private FileCreator fileCreator = new FileCreator();
private String fileName = "file.txt";
@Before
public void setUp() throws IOException {
Files.deleteIfExists(Paths.get(fileName));
}
@Test
public void shouldCreateFile() throws InterruptedException {
// when
fileCreator.createFile(fileName);
Thread.sleep(1000);
// then
verifyThatFileExists(fileName);
}
private void verifyThatFileExists(String fileName) {
Path path = Paths.get(fileName);
assertThat(Files.exists(path), is(true));
}
}

The horrible thing about it is that Thread.sleep() invocation. Test should be fast, making them wait unnecessary is very poor solution. And what if the test sometimes fails because of overloaded hardware? Are we going to sleep even more?

To eliminate unneeded waiting, we may come up with a concept of validator:

public class FileCreatorTestWithCustomValidator {
private FileCreator fileCreator = new FileCreator();
private String fileName = "file.txt";
@Before
public void setUp() throws IOException {
Files.deleteIfExists(Paths.get(fileName));
}
@Test
public void shouldCreateFile() throws InterruptedException {
// when
fileCreator.createFile(fileName);
// then
boolean fileExists = checkThatFileExists(fileName);
assertThat(fileExists, is(true));
}
private boolean checkThatFileExists(String fileName)
throws InterruptedException {
for (int i = 0; i < 100; i++) {
Path path = Paths.get(fileName);
try {
assertThat(path, exists());
return true;
} catch (AssertionError e) {
// ignore exception
}
Thread.sleep(100);
}
throw new AssertionError("Timeout exceeded");
}
}

We no longer need to sleep for a long time, but the code has been significantly polluted. Of course we can refactor our validator, make it more reusable but why reinvent the wheel? There is a nice and small library - awaitility - that will do the same for us.

public class FileCreatorTestWithAwaitility {
private FileCreator fileCreator = new FileCreator();
private String fileName = "file.txt";
@Before
public void setUp() throws IOException {
Files.deleteIfExists(Paths.get(fileName));
}
@Test
public void shouldCreateFile() throws Exception {
// when
fileCreator.createFile(fileName);
// then
await().until(fileIsCreated(fileName));
}
private Callable<Boolean> fileIsCreated(final String fileName) {
return new Callable<Boolean>() {
@Override
public Boolean call() throws Exception {
Path path = Paths.get(fileName);
return Files.exists(path);
}
};
}
}

In a very expressive way, we achieve the same result. Timeout, polling delay and polling interval are of course configurable.

Friday, 23 March 2012

logback - successor of log4j

The old and good known log4j seems to be a standard framework for logging in Java applications, despite some serious disadvantages like boilerplate configuration, lack of good documentation and overcomplicated architecture.

The authors of log4j dispatched on another journey and created its successor - logback, which addresses old problems and adds a lot of enhancements. 

Configuration is now more concise (it can be even written in groovy) and well documented.
slf4j api is used natively, so the implementation can be changed easily.
The issue with many instances of RollingFileAppender writing to the same file was also resolved.

In order to add logback to your project you need to add two dependencies:

<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.0.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.6.4</version>
</dependency>
You're basically ready to go, because default configuration is applied, when no other is found.

When following class is run:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class App {
private static Logger logger = LoggerFactory.getLogger(App.class);
public static void main(String[] args) {
logger.debug("Hello world.");
}
}

it will print something like this:

22:44:38.560 [main] DEBUG pl.mjedynak.App - Hello world.
view raw gistfile1.txt hosted with ❤ by GitHub
But the coolest feature is automatic reloading of configuration file.

When we add this example configuration (saved as logback.xml) to the classpath:

<configuration scan="true" scanPeriod="5 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT"/>
</root>
</configuration>
view raw logback.xml hosted with ❤ by GitHub
we can change it on the fly and logback will automatically apply changes (at configured interval, in our case 5 seconds) without a need to restart the application.

Sunday, 18 March 2012

Getting result from thread's execution with Future

To make things go faster we parallelize our computations. What if we need to use the result of the thread's execution?
Let's say we have a service that buys some product. It needs to fetch the price and quantity of the product. Fetching the price usually takes longer, so we delegate this task to a thread, while we are dealing with quantity.
To keep things simple, our PriceChecker class will be just simulating that it does something meaningful:
public class PriceChecker {
private static final Random RANDOM = new Random();
public Double checkPrice() {
int sleepTime = RANDOM.nextInt(3);
try {
Thread.sleep(sleepTime * 1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
return Math.random();
}
}

Now it would be good to somehow get the result of checkPrice() invocation. Runnable's run() is a void method, so we would have to do something like this:
public class Buyer {
private PriceChecker priceChecker = new PriceChecker();
private Double price;
public void buyProduct() {
new Thread(new Runnable() {
@Override
public void run() {
price = priceChecker.checkPrice();
}
}).start();
int quantity = getQuantity();
while (price == null) {
try {
Thread.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
System.out.println("Buying product, price = " + price + " , quantity = " + quantity);
}
private int getQuantity() {
return new Random().nextInt(100);
}
}
view raw Buyer.java hosted with ❤ by GitHub

This approach has a lot of drawbacks. We have to check in the loop if the price has already been set. What is more, price cannot be a final variable but has to be a field instead.

To deal with this kind of a problem, the Future interface should be used. Basically, it allows to get the result of thread's execution. Let's take a look at the actual usage in the context of our example:
public class BuyerWithFuture {
private PriceChecker priceChecker = new PriceChecker();
private ExecutorService executorService = Executors.newSingleThreadExecutor();
public void buyProduct() throws Exception {
Callable<Double> getPriceTask = new Callable<Double>() {
@Override
public Double call() throws Exception {
return priceChecker.checkPrice();
}
};
Future<Double> priceFuture = executorService.submit(getPriceTask);
Double price = priceFuture.get();
int quantity = getQuantity();
System.out.println("Buying product, price = " + price + " , quantity = " + quantity);
executorService.shutdown();
}
private int getQuantity() {
return new Random().nextInt(100);
}
}
First of all we're using Callable which is similar to Runnable but is capable of returning the result. Notice that it can also throw exception, while run() cannot. When we submit the callable to executor service, we're getting the Future object. Its method get() blocks until the computation is finished. If we want to specify the maximum time that we want to wait, there is an overloaded version that takes waiting time settings as parameters.

The runner for both cases is pretty straightforward:
public class FutureExampleRunner {
public static void main(String[] args) throws Exception {
Buyer buyerWithoutFuture = new Buyer();
buyerWithoutFuture.buyProduct();
BuyerWithFuture buyerWithFuture = new BuyerWithFuture();
buyerWithFuture.buyProduct();
}
}

Saturday, 10 March 2012

Handling read and write operations on shared object

Many times we come across a problem, when in the multithreaded environment some shared object has frequent read operations and write operations are rare. If we could use a concurrent collection that would be the best option in most of the cases, but sometimes we can't.

Let's imagine a situation when this shared object is in a 3rd party library. In our example this object is a Book class that I used in previous posts.

Most of the time we want to show information about the book, but exceptionally the author modifies the content and therefore the number of pages needs to be changed.

Using synchronized for the methods that read/write the book would be a poor solution.
Instead, we could use ReadWriteLock which allows to differentiate between read and write operations. Read operation is concurrent, which means that all the threads have access to the object. Only write operation is exclusive and blocks other threads.

public class ReadWriteLockExample {
private ReadWriteLock readWriteLock = new ReentrantReadWriteLock();
private Book book = new Book("Haruki Murakami", "1Q84", 300);
public void showInfoAboutBook() {
readWriteLock.readLock().lock();
try {
System.out.println("Reading book info from thread " + Thread.currentThread().getName());
System.out.println(book);
} finally {
readWriteLock.readLock().unlock();
}
}
public void setNumberOfPages(int pages) {
readWriteLock.writeLock().lock();
try {
System.out.println("Setting book property from thread " + Thread.currentThread().getName());
book.setPages(pages);
System.out.println("New value for book has been set");
} finally {
readWriteLock.writeLock().unlock();
}
}
}
Everything we do after acquiring the lock must be put in a try block, so that unlocking is done even if the exception is thrown.

Let's see it in action:

public class ReadWriteLockExampleRunner {
public static void main(String[] args) {
final ReadWriteLockExample example = new ReadWriteLockExample();
ExecutorService executor = Executors.newFixedThreadPool(10);
Runnable readTask = new Runnable() {
@Override
public void run() {
example.showInfoAboutBook();
}
};
Runnable writeTask = new Runnable() {
@Override
public void run() {
example.setNumberOfPages(301);
}
};
for (int i = 0; i < 100; i++) {
executor.submit(readTask);
if (i % 10 == 0) {
executor.submit(writeTask);
}
}
executor.shutdown();
}
}
We have 10 threads that are performing 100 tasks. Only 10% of them are write tasks. When we run it, we can see that read operations are concurrent (lines about the reading thread and book info are mixed from time to time) and write operations are exclusive (info about writing thread and successful update are always in the correct order).

The example with the book is purely educational and not too realistic. In real life we would probably have some kind of a large collection. It's good to read ReentrantReadWriteLock and ReadWriteLock javadoc, because sometimes using them may bring more overhead than mutual exclusion.

Friday, 24 February 2012

Overriding equals and hashCode with Guava and Objects class

We usually use IDE to override equals and hashCode methods in our classes.
Let's see how IntelliJ IDEA 11 copes with a simple class with 3 fields:
Even though the class seems simple, the methods' size is a little bit scary.
Using EqualsBuilder and HashCodeBuilder from Apache Commons Lang makes the methods much smaller, however introduces one major flaw: it uses reflection, which isn't a speed demon.
To the rescue comes Guava, which makes these methods simpler and more readable:
public class Book {
private String author;
private String title;
private int pages;
public Book(String author, String title, int pages) {
this.author = author;
this.title = title;
this.pages = pages;
}
@Override
public boolean equals(Object obj) {
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
final Book other = (Book) obj;
return Objects.equal(this.author, other.author) && Objects.equal(this.title, other.title) && Objects.equal(this.pages, other.pages);
}
@Override
public int hashCode() {
return Objects.hashCode(author, title, pages);
}
}
If you're working on a project in which you can use Java 7, it provides a similar solution without having to use any external library:
public class Book {
private String author;
private String title;
private int pages;
public Book(String author, String title, int pages) {
this.author = author;
this.title = title;
this.pages = pages;
}
@Override
public boolean equals(Object obj) {
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
final Book other = (Book) obj;
return Objects.equals(this.author, other.author) && Objects.equals(this.title, other.title) && Objects.equals(this.pages, other.pages);
}
@Override
public int hashCode() {
return Objects.hash(author, title, pages);
}
}
At the time of writing this, IDEA 11 doesn't support generating equals and hashCode using Guava or Objects class from Java 7. I created a plugin that provides such possibility:
http://plugins.intellij.net/plugin/?idea&id=6875

Sunday, 19 February 2012

Coordinating threads

In our multithreaded application we may need to coordinate the work of our threads. The most occurring situation is waiting for all the threads once they finish their work.

Let's take a simple example: we have 2 threads that are doing some mathematical operations and we need to wait for both of them before we proceed with further processing. We could do that using thread's join() method:

public class JoiningThreadsExample {
public static void main(String[] args) throws InterruptedException {
Runnable runnable = new Runnable() {
@Override
public void run() {
for (int i = 0; i < 10000000; i++) {
Math.pow(Math.random(), 2);
}
System.out.println(Thread.currentThread().getName() + " finished");
}
};
Thread thread1 = new Thread(runnable);
Thread thread2 = new Thread(runnable);
thread1.start();
thread2.start();
thread1.join();
thread2.join();
System.out.println("Both threads finished computing.");
}
}
If the number of threads is higher, this implementation will look much more nastier, as we would need to invoke join() method on all of them.


That's why the better solution is to use CountDownLatch, "that allows one or more threads to wait until a set of operations being performed in other threads completes", as Javadoc says.

public class CountdownLatchExample {
public static void main(String[] args) throws InterruptedException {
int threadsNumber = 2;
final CountDownLatch latch = new CountDownLatch(threadsNumber);
Runnable runnable = new Runnable() {
@Override
public void run() {
for (int i = 0; i < 10000000; i++) {
Math.pow(Math.random(), 2);
}
System.out.println(Thread.currentThread().getName() + " finished");
latch.countDown();
}
};
ExecutorService executorService = Executors.newFixedThreadPool(threadsNumber);
executorService.submit(runnable);
executorService.submit(runnable);
latch.await();
executorService.shutdown();
System.out.println("Both threads finished computing.");
}
}
In the constructor of CountDownLatch we specify a count - as each of our threads will be doing only one countdown once work is finished, it is equal to the number of threads. await() method makes the current thread to stop, until latch has counted down to zero. It also provides possibility to wait for the specified time.

If threads must repeatedly count down in a similar way, we would need to use CyclicBarrier instead.  

Sunday, 12 February 2012

Concurrent map with timed out elements using Guava library

Sometimes we have a need to use a map, in which elements live only for a specified amount of time.
We could implement such map ourselves (by overriding removeEldestEntry() from LinkedHashMap or using thread that will be removing elements periodically) but the better idea is to use already existing implementation. Google's guava library (http://code.google.com/p/guava-libraries) is the solution for us.
Let's write some simple unit tests to check the API and verify behaviour:
public class ConcurrentMapExampleTest {
private static final int TIMEOUT_IN_MILLIS = 500;
private static final int KEY = 1;
private static final String VALUE = "David de Gea";
private ConcurrentMap<Integer, String> concurrentMap;
@Before
public void setUp() {
Cache<Integer, String> cache = CacheBuilder.newBuilder().expireAfterWrite(TIMEOUT_IN_MILLIS, TimeUnit.MILLISECONDS).build();
concurrentMap = cache.asMap();
}
@Test
public void shouldRemoveElementWhenTimeoutExpires() throws InterruptedException {
// when
concurrentMap.put(KEY, VALUE);
Thread.sleep(TIMEOUT_IN_MILLIS * 2); // wait for the element to be removed
// then
assertThat(concurrentMap.containsKey(KEY), is(false));
assertThat(concurrentMap.containsValue(VALUE), is(false));
}
@Test
public void shouldNotRemoveElementWhenTimeoutDoesNotExpire() throws InterruptedException {
// when
concurrentMap.put(KEY, VALUE);
// then
assertThat(concurrentMap.containsKey(KEY), is(true));
assertThat(concurrentMap.containsValue(VALUE), is(true));
}
}
Map is created using the builder pattern. Apart from specifying the time of elements expiration, it allows among others, setting the maximum map's size or the listener that will be notified each time an entry is removed. As the built object is an instance of Cache, we need to create ConcurrentMap from it using asMap() method.
Two tests verify that added element is removed only when the timeout expires.

Thursday, 9 February 2012

Concurrent collections

As I already mentioned, situation that requires concurrent access to a collection is pretty often. Let's say that we have a map that is accessed by many threads.
Synchronizing it like this is not the best choice, as only one thread at a time will be able to access it.
private Map<String, Integer> map = new HashMap<>();
public Integer getKey(String key) {
synchronized (map) {
return map.get(key);
}
}

Another approach is useful only in a few cases.
private Map<String, Integer> map = Collections.synchronizedMap(new HashMap<String, Integer>());
The factory method creates synchronized wrapper for our map that makes it thread-safe. However if we want to iterate over it, we still need a synchronization, otherwise it will throw ConcurrentModificationException when some other thread will modify it while we're iterating.

The better solution is to use ConcurrentMap 
private ConcurrentMap<String, Integer> map = new ConcurrentHashMap<String, Integer>();
It not only gives us a concurrent access, but we can also iterate over it without synchronization.

Let's imagine a situation that we need to put some value to the map, on a condition that it isn't there already.
Doing it that way is a wrong solution:
// bad solution
public void addEntryIfKeyDoesNotExist(String key, Integer value) {
if (!map.containsKey(key)) {
map.put(key, value);
}
}
After we check that the key is not there, another thread could put some entry to the map and we will override existing value. Basically our operation in this implementation is not atomic.
So should we add the synchronization? No, the ConcurrentMap provides a method that will make this operation atomic:
public void addEntryIfKeyDoesNotExist(String key, Integer value) {
map.putIfAbsent(key, value);
}

ConcurrentMap provides also other methods that are able to perform atomic replacement or removal of an entry, only if it is mapped to a specified value.
There are also a concurrent implementations of List and Set - CopyOnWriteArrayList and CopyOnWriteArraySet. Every mutative operation on them creates a new copy, so they should be used with a special care. 

Wednesday, 1 February 2012

Blocking queues

It happens that in our system's architecture we have a situation of concurrent access to some collection. Whether it's a server that handles requests via sockets or a JMS listener, it usually stores incoming objects into a collection and other worker processes them. It can be illustrated as a well known producer-consumer problem.

An example implementation could look like that:
public class SynchronizedProducer {
private Queue<String> queue;
public SynchronizedProducer(Queue queue) {
this.queue = queue;
}
public void produce() throws InterruptedException {
while (true) {
putOnQueue(UUID.randomUUID().toString());
Thread.sleep(1);
}
}
private void putOnQueue(String message) {
synchronized (queue) {
queue.add(message);
}
}
}
We have a class that serves as our producer. In an endless loop we're sending random strings to a queue. Access to it is synchronized by acquiring its monitor. We're sleeping 1 ms so that the processor is not over-killed.

Let's take a look at a consumer:
public class SynchronizedConsumer {
private Queue queue;
public SynchronizedConsumer(Queue queue) {
this.queue = queue;
}
public void consume() throws InterruptedException {
while (true) {
takeFromQueue();
Thread.sleep(1);
}
}
private void takeFromQueue() {
synchronized (queue) {
if (!queue.isEmpty()) {
System.out.println(queue.remove());
}
}
}
}
It synchronizes queue in a similar way. What is more, before taking from the queue, it checks if it's not empty. It also sleeps 1 ms for the same reason as producer.

Class that glues them together:

public class SynchronizedProducerConsumerApp {
public static void main(String[] args) throws InterruptedException {
Queue<String> queue = new LinkedList<String>();
final SynchronizedConsumer synchronizedConsumer = new SynchronizedConsumer(queue);
final SynchronizedProducer synchronizedProducer = new SynchronizedProducer(queue);
Executors.newSingleThreadExecutor().submit(new Runnable() {
@Override
public void run() {
try {
synchronizedProducer.produce();
} catch (InterruptedException e) {
}
}
});
Executors.newSingleThreadExecutor().submit(new Runnable() {
@Override
public void run() {
try {
synchronizedConsumer.consume();
} catch (InterruptedException e) {
}
}
});
}
}
view raw App.java hosted with ❤ by GitHub
We're using LinkedList as an implementation of our Queue. Producer and consumer are started in a separate threads.

There are at least 3 major drawbacks with this implementation.
First of all we're acquiring monitor of the queue, so we're completely blocking access to it not allowing for a concurrent access.
Secondly we need to check if the queue is not empty before taking from it.
Last but not least, consumer sleeps some specified time (sleeping when you're supposed to be working is not the best thing to do) and our performance decreases.

The remedy for above problems is BlockingQueue.

The new producer is similar to previous one, except that it operates on BlockingQueue and doesn't use any synchronization on it, because it's thread safe.

public class BlockingQueueProducer {
private BlockingQueue<String> queue;
public BlockingQueueProducer(BlockingQueue queue) {
this.queue = queue;
}
public void produce() throws InterruptedException {
while (true) {
putOnQueue(UUID.randomUUID().toString());
Thread.sleep(1);
}
}
private void putOnQueue(String message) {
queue.add(message);
}
}
There are some changes in consumer:
public class BlockingQueueConsumer {
private BlockingQueue queue;
public BlockingQueueConsumer(BlockingQueue queue) {
this.queue = queue;
}
public void consume() throws InterruptedException {
while (true) {
takeFromQueue();
}
}
private void takeFromQueue() throws InterruptedException {
System.out.println(queue.take());
}
}
Apart from not using synchronization, we don't need to check if the queue is not empty, as method take() is doing it for us. We also don't have to sleep anymore, as it is also handled by this method: as soon as an element is available it will be taken from the queue.

Class to run it:
public class BlockingQueueConsumerProducerApp {
public static void main(String[] args) {
BlockingQueue<String> queue = new LinkedBlockingQueue<String>();
final BlockingQueueConsumer consumer = new BlockingQueueConsumer(queue);
final BlockingQueueProducer producer = new BlockingQueueProducer(queue);
Executors.newSingleThreadExecutor().submit(new Runnable() {
@Override
public void run() {
try {
producer.produce();
} catch (InterruptedException e) {
}
}
});
Executors.newSingleThreadExecutor().submit(new Runnable() {
@Override
public void run() {
try {
consumer.consume();
} catch (InterruptedException e) {
}
}
});
}
}
It hasn't changed much, we're using LinkedBlockingQueue as an implementation.
There are other implementations of BlockingQueue that may be helpful for our needs (e.g. that introduces priority to elements or some delay that is needed before taking).

Monday, 30 January 2012

Synchronization pitfall

Synchronized method modifier is equivalent to block synchronized(this). Not only invoking of such methods is expensive for virtual machine, but they're also a little bit risky. Object's monitor can be taken not only during method invocation, but also by another object that has a reference to it.
Let's take a look at this example:
public class Resource {
private Object veryImportantObject = new Object();
private final Object mutex = new Object();
public synchronized Object getVeryImportantObject() {
System.out.println("Getting object");
return veryImportantObject;
}
public Object getVeryImportantObjectWithMutex() {
synchronized(mutex) {
System.out.println("Getting object with mutex");
return veryImportantObject;
}
}
}
view raw Resource.java hosted with ❤ by GitHub
Class Resource contains some object and provides synchronized methods for fetching it.
getVeryImportantObject() has synchronized modifier.
getVeryImportantObjectWithMutex() uses synchronized block on private object.

We also have an evil class:
public class Cheater {
private final Resource resource;
public Cheater(Resource resource) {
this.resource = resource;
}
public void blockResource() {
synchronized(resource) {
System.out.println("I'm blocking resource");
while(true) {
try {
Thread.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
view raw Cheater.java hosted with ❤ by GitHub
Cheater owns a reference to Resource and it blocks it in blockResource() method.

The class that glues it together:
public class Pitfall {
public static void main(String[] args) throws Exception {
final Resource resource = new Resource();
Executors.newSingleThreadExecutor().submit(new Runnable() {
@Override
public void run() {
new Cheater(resource).blockResource();
}
});
Thread.sleep(1000);
Executors.newSingleThreadExecutor().submit(new Runnable() {
@Override
public void run() {
resource.getVeryImportantObject();
}
});
Executors.newSingleThreadExecutor().submit(new Runnable() {
@Override
public void run() {
resource.getVeryImportantObjectWithMutex();
}
});
}
}
view raw Pitfall.java hosted with ❤ by GitHub
Pitfall invokes 3 threads. The first one uses Cheater to take the monitor of created object of Resource class. Next threads start after 1 second in order to allow the first one performing its task. The second thread tries to get resource object by using method with synchronized modifier, however it never succeeds. The third thread grabs this object by using method with internal synchronized block.
Project Lombok provides an interesting way of not getting into this trap (http://projectlombok.org/features/Synchronized.html)

Saturday, 28 January 2012

Running threads using Executors

There are several ways of running threads.
We can create an instance of Runnable and pass it to Thread class:
Runnable runnable = new Runnable() {
@Override
public void run() {
System.out.println("thread 1");
}
};
Thread thread1 = new Thread(runnable);
thread1.start();

Or we can override the run() method in the Thread class directly:
Thread thread2 = new Thread() {
@Override
public void run() {
System.out.println("thread 2");
}
};
thread2.start();

However the better solution (in most cases) is using the Executor interface.
It gives us a couple of benefits. In previous cases, if we want to start again the thread that has come to the terminated state we will get IllegalThreadStateException.
With Executor we can reuse the existing Runnable :
ExecutorService executor = Executors.newSingleThreadExecutor();
Runnable runnable = new Runnable() {
@Override
public void run() {
System.out.println(Thread.currentThread().getName());
}
};
executor.execute(runnable);
// executing once again
executor.execute(runnable);
executor.shutdown();
To prevent adding any more tasks to our executor we need to call shutdown() method.

What is even more important, it provides the ability to easily switch the implementation of running: using single thread, different kinds of thread pools or even scheduled periodic task.
We would just need to use different static factory method from Executors utility class:
ExecutorService executor = Executors.newFixedThreadPool(2);
Runnable runnable = new Runnable() {
@Override
public void run() {
System.out.println(Thread.currentThread().getName());
}
};
executor.execute(runnable);
executor.execute(runnable);
executor.shutdown();
This time our runnable is run using two threads from executor's thread pool.

Saturday, 21 January 2012

java.util.concurrent.atomic package


Classes from java.util.concurrent.atomic package allow us to use single variables concurrently without blocking access to them. The typical usage example is increasing the counter that is modified by many threads.

Instead of using the synchronized block, that blocks the access to the counter:
private long counter = 0;
private final Object mutex = new Object();
public void incrementCounter() {
synchronized (mutex) {
counter++;
}
}

or the synchronized method equivalent to the block with 'synchronized(this)':
public synchronized void incrementCounter() {
counter++;
}

you can use AtomicLong class, that guarantees non-blocking access:
private AtomicLong counter = new AtomicLong();
public void incrementCounter() {
counter.incrementAndGet();
}

Method incrementAndGet() besides incrementing counter's value, returns it.

Another typical usage is providing non-blocking access to an object, that is shared by many threads. Instead of synchronizing getter/setters methods:
private Object resource = new Object();
public synchronized Object getResource() {
return resource;
}
public synchronized void setResource(Object resource) {
this.resource = resource;
}

you can use AtomicReference class: