Lesson Tuesday

Now that we've used Gradle to get everything in place, we can finally begin testing! In this lesson we'll address how to write and run JUnit tests, and how to interpret their results.

Once we have the basics down, the next lesson will build the Ping Pong application to completion, demonstrating what the entire Red, Green, Refactor workflow looks like in conjunction with JUnit.

Setup

After the the previous lesson, our project should look like this:

ping-pong
├── build.gradle
└── src
    ├── main
    │   └── java
    │       ├── App.java
    │       └── PingPong.java
    └── test
        └── java

If your project contains temporary files created by Gradle, you can use $ gradle clean to remove them. This isn't required, but can make navigating projects easier. It's safe to remove these because Gradle will rebuild them as necessary the next time you execute $ gradle compileJava.

Setting Up JUnit

Test File

Next, let's set up our test file. This file will contain our coded tests, and reside in the src/test/java directory. Test files have the same name of the file containing the code they are responsible for testing, with the word "Test" added to the end. Because our program's logic will reside in PingPong.java we call our test file PingPongTest.java.

Unit tests like those created with JUnit are meant to test back-end logic, not the user interface. Our back-end logic will reside in PingPong.java. Whereas App.java will contain the code for our front-end user interface.

We'll create our test file now:

$ touch src/test/java/PingPongTest.java

Importing JUnit

Next, we need to import the necessary packages to write JUnit tests. This is similar to the manner we've previously imported packages:

src/test/java/PingPongTest.java
import org.junit.*;
import static org.junit.Assert.*;

Here, we are importing two JUnit libraries:

  • org.junit.Test; is the main JUnit library that instructs Java how to run all of the testing procedures.

  • static org.junit.Assert.*; Is JUnit's assertion library. To "assert" in coding is essentially to "confirm". We use the methods available in this assertion library to confirm our code is returning the correct output. We'll explore this further momentarily.

Declaring the Test Class

Next, like many Java files, test files require a class declaration of the same name as the file. Because the test file is named PingPongTest.java, its class is PingPongTest:

src/test/java/PingPongTest.java
import org.junit.*;
import static org.junit.Assert.*;

public class PingPongTest {

}

Identifying the Simplest Behavior

Now, don't forget about the Red, Green, Refactor workflow. In fact, it's a good idea to keep that lesson open as you follow along here.

Before we write a coded test, we must identify the simplest possible behavior our program must exhibit.

Now, you should be familiar with what a Ping Pong application does. We enter a number. It counts up to that number, replacing numbers divisible by 3 with "ping", numbers divisible by 5 with "pong" and numbers divisible by both with "ping pong". So, what is the simplest possible behavior such a program must demonstrate?

The primary function of this application is to replace certain numbers with strings. But what behaviors will it exhibit that are even simpler than that? First off, it must be able to count to the provided number before replacing any numbers. And the simplest number our program will have to count to is 1.

Our first, simplest possible behavior the program will exhibit is counting to 1.

Anatomy of a JUnit Test

We can now begin the second step in the Red, Green, Refactor workflow. We'll write a coded test for the simplest behavior we've just identified.

Our tests will be located in the PingPongTest class declaration:

src/test/java/PingPongTest.java
import org.junit.*;
import static org.junit.Assert.*;

public class PingPongTest {

  @Test
  public void runPingPong_countUpToOne_ArrayList() {
    // contents of test will eventually go here. 
  }
}

Every test begins with @Test. This is called an annotation. Annotations are marked with an @ symbol. They are used to provide extra information to the Java compiler. In this case, it informs the compiler this is a test JUnit is responsible for running.

Next, we declare our test's method with the line public void runPingPong_countUpToOne_ArrayList(). JUnit tests are actually methods! Also, they always start with public void.

The naming convention is: public void nameOfMethodWeAreTesting_descriptionOfBehavior_expectedReturnValue(). You can read more about this here.

We called our test method runPingPong_countUpToOne_ArrayList() because runPingPong() will be the name of our method. countUpToOne is the behavior this spec is testing. And ArrayList() is the data type we're expecting runPingPong() to return.

Note on Return Types

But why use an ArrayList for a single number? We do not want to modify previous specs when implementing future specs. Even though this behavior only requires a single number, our program will eventually return an entire ArrayList like this: [1, 2, "ping"]. If we wrote our test to anticipate an Integer instead of an ArrayList we would later have to go back and change this test when we implement behaviors to count higher than one.

JUnit Assertions

We've declared our test method. Now, let's fill it with code:

src/test/java/PingPongTest.java
import org.junit.*;
import static org.junit.Assert.*;
import java.util.ArrayList;
import java.util.List;

public class PingPongTest {

  @Test
  public void runPingPong_countUpToOne_ArrayList() {
    PingPong testPingPong = new PingPong();
    List<Object> expectedOutput = new ArrayList<Object>();
    expectedOutput.add(1);
    assertEquals(expectedOutput, testPingPong.runPingPong(1));
  }

}

Here, we've done the following:

  • First, we create a new instance of our PingPong class called testPingPong.
  • Then, we create an ArrayList of the <Object> generic called expectedOutput, and add 1 to it (This is only included because our method returns an ArrayList, and this is how an ArrayList is constructed. If our method returned something else, such as a String, we wouldn't need to construct it piece-by-piece with the .add() method).
  • We import the necessary List and ArrayList packages at the top of the file.
  • Then, we write our first assertion using assertEquals(). This tells JUnit to confirm (or assert) whether the two arguments provided are equivalent.
    • The first argument, [1] represents the return value we expect. This is similar to the "Example Output" we included with our plain English specs in Intro to Programming.
    • The second argument, testPingPong.runPingPong(1), is a trial in which we call our runPingPong() method, providing it 1 as an argument.

Essentially, translated into "plain English" this test would read: "JUnit, test out the runPingPong() method. Give it the number 1 and make sure it correctly returns [1]."

By writing our first coded test, we've completed the second step in the Red, Green, Refactor workflow!

Running JUnit Tests

Let's run $ gradle test and see what happens!

$ gradle test

:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileTestJava

/Users/staff/Desktop/java/PingPong/src/test/java/PingPongTest.java:10: error: cannot find symbol
      PingPong testPingPong = new PingPong();
      ^

  symbol:   class PingPong
  location: class PingPongTest
/Users/staff/Desktop/java/PingPong/src/test/java/PingPongTest.java:10: error: cannot find symbol
      PingPong testPingPong = new PingPong();
                                  ^

  symbol:   class PingPong
  location: class PingPongTest

2 errors
:compileTestJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileTestJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 0.985 secs

Oh no, this isn't what a failing test looks like. This is a compiler error. Let's decipher it together. The first portion of the error looks like this:

/Users/staff/Desktop/java/PingPong/src/test/java/PingPongTest.java:10: error: cannot find symbol
      PingPong testPingPong = new PingPong();
      ^

It says cannot find symbol and clearly points to the PingPong class declaration. It cannot find the PingPong class. The second portion of the error says something similar:

  symbol:   class PingPong
  location: class PingPongTest
/Users/staff/Desktop/java/PingPong/src/test/java/PingPongTest.java:10: error: cannot find symbol
      PingPong testPingPong = new PingPong();
                                  ^

Again, it states cannot find symbol and points to an instance of the PingPong class.

Since this is our very first test, we have not touched a line of program logic yet. However, the $ gradle test command first compiles our source code, then tests the compiled code using JUnit. If the application cannot compile successfully, JUnit tests never even get a chance to run! Therefore, we have to make sure to address all compiler errors first.

In our case, the compiler is throwing an error because we have not declared the PingPong class being referenced in our test.

In our PingPong.java file, let's declare our PingPong class, and the runPingPong() method. But we will not add any logic until we see the test fail. We'll only declare them in order to avoid compiler errors:

ping-pong/src/main/java/PingPong.java
import java.util.ArrayList;
import java.util.List;

public class PingPong {

  public List<Object> runPingPong(int countUpTo){

  }

}
  • Here, we declare the method's return type as ArrayList because we know the return value will differ in length depending on what number the user enters (we cannot use an Array because they are unable to fluctuate in size).

  • We also include the generic <Object> because we know a ping pong application will eventually replace some numbers with Strings, and ArrayList<Object> can contain multiple differing data types.

  • We also import the ArrayList package at the top of the file.

Now, if we run $ gradle test again, we should see the following:

$ gradle test

:compileJava
/Users/staff/Desktop/java/ping-pong/src/main/java/PingPong.java:7: error: missing return statement
  }
  ^
1 error
:compileJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 0.666 secs

Alright. A different error this time! This one reads error: missing return statement. This is because our runPingPong() method isn't actually returning anything.

Again, $ gradle test compiles our project, then runs tests using the compiled logic. Therefore, to begin running tests we need to make sure the project can actually compile successfully.

At this point, you may be asking "But I thought you told us not to write code before our test fails?!".

And that's true; we shouldn't write any program logic. We do, however, need to declare any methods and classes our tests reference. If we don't, the project will not successfully compile. Instead, the compiler will throw an error. If we can't successfully compile, we cannot run tests. If we cannot run tests, we cannot confirm the test fails appropriately.

So, to move past the missing return statement error, we need to add a return statement to our runPingPong() method. We know it will return an ArrayList<Object>, so we'll construct an empty one and return it:

ping-pong/src/main/java/PingPong.java
import java.util.ArrayList;
import java.util.List;

public class PingPong {

  public List<Object> runPingPong(int countUpTo){
    List<Object> result = new ArrayList<Object>();
    return result;
  }

}

It's important to understand the difference between declaring classes and methods in order to compile successfully, and writing the actual program logic. You do not write any program logic before writing a test for that behavior, and confirming it fails appropriately.. But you do need to declare necessary elements to get beyond any compiler errors.

If we run $ gradle test once more, we should see the following:

$ gradle test
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test

PingPongTest > runPingPong_countUpToOne_ArrayList FAILED
    java.lang.AssertionError at PingPongTest.java:13

1 test completed, 1 failed
:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/staff/Desktop/java/ping-pong/build/reports/tests/test/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 1.357 secs

Awesome. This is a test failure message.

Reading JUnit Results

Notice in the JUnit message it says 1 test completed, 1 failed. Additionally, it tells us which test failed in the line PingPongTest > runPingPong_countUpToOne_ArrayList FAILED. These messages are now from JUnit. That's how we know the project has compiled successfully and tests were able to run.

Test Reports in the Browser

Additionally, notice the following line There were failing tests. See the report at: file:///Users/staff/Desktop/java/ping-pong/build/reports/tests/test/index.html.

Copy/paste the file path that looks like file:///Users/staff/Desktop/java/ping-pong/build/reports/tests/test/index.html from your tests into the browser.

You should see something like this:

junit-test-report

This is a more detailed (and easier to read) report of your JUnit tests. Each time you run tests you can refresh this page to receive updated results!

If we click on the name of our failing test...

junit-test-report-stacktrace-link-highlighted

A panel containing more details should expand:

junit-test-report-failure-stacktrace

This is a stack trace. It contains line-by-line details of exactly what caused any errors. Stack traces are useful for figuring out the cause of failing tests and other errors. The sheer number of lines they contain can be intimidating, but thankfully the most useful line is usually right on top!

The top line in our stack trace reads:

java.lang.AssertionError: expected:<[1]> but was:<[]>

Here, AssertionError refers to the assertion we created in our test (assertEquals(expectedOutput, testPingPong.runPingPong(1));) requesting JUnit to assert (or, confirm) that using the runPingPong() method with the argument 1 would successfully return the expectedOutput ArrayList. AssertionError means there was an issue with this assertion. Essentially, we did not receive the output we anticipated.

expected:<[1]> but was:<[]> means that we told JUnit that the correct output should be [1], but the output it actually received is an empty ArrayList. Because these were different, the test failed.

But remember, this is actually good news! We want our test to fail appropriately before we begin adding code.

Now that we have a general understanding of how a JUnit test works, how to run them, and where to view their results, we'll continue following the Red, Green, Refactor workflow to build our Ping Pong application in the next lesson.


Example GitHub Repo for Ping Pong

Terminology


  • Assertation Library: A collection of tools (known as a library) included as part of JUnit that are used to "assert" (or, "confirm") that code is returning the values we expect it to.

  • Annotation: Marked with an @ symbol. Most commonly they are used to tell the Java compiler something about a method it is about to run. In this case, it is telling the compiler that this is a test that should be run by JUnit.

  • Stack Trace: A report detailing what caused an error, and where it occurred. They're often very long and a little intimidating; but the most useful information is usually at the top!

Overview


  • Because our primary program logic will exist in a file called App.java, we name our test file AppTest.java. Because the test file is named AppTest.java, we name its inner class AppTest. All applications will use this same naming convention. For instance, if our primary logic existed in a file called Puppies.java, our test file would be named PuppiesTest.java, and its inner class would be named PuppiesTest.

Tips


  • Test methods should always start with public void.

  • The naming convention for test methods: public void nameOfMethod_descriptionOfBehavior_expectedReturnType().

  • If your project contains a bunch of temporary files created by gradle, you can use the command $ gradle clean to remove them. This isn't required, but can make navigating your directory easier. It's safe to remove these files with the $ gradle clean command. The next time Gradle compiles your project, it will rebuild any of these files it requires.

Example


Example JUnit test file, with a single test:

src/test/java/AppTest.java
import org.junit.*;
import static org.junit.Assert.*;
import java.util.List;
import java.util.ArrayList;

public class PingPongTest {

  @Test
  public void runPingPong_countUpToOne_ArrayList() {
    PingPong testPingPong = new PingPong();
    List<Object> expectedOutput = new ArrayList<Object>();
    expectedOutput.add(1);
    assertEquals(expectedOutput, testPingPong.runPingPong(1));
  }

}

Example JUnit test report in the browser:

junit-test-report

Example GitHub Repo for Ping Pong

Additional Resources