Testing Java solutions

To evaluate student solutions written in Java, you need to write tests using JUnit.

Basics

Let's start with a basic exercise. Your student is to implement a HelloWorld class with a function helloWorld() that returns the string "Hello World". For the sake of simplicity, it doesn't print anything.

Here is an example solution:

public class HelloWorld {
    public String helloWorld() {
        return "Hello World";
    }
}

And this is an example test of the student's code:

import org.junit.Test;
import org.junit.Assert;

public class Evaluate {
    HelloWorld hw = new HelloWorld();

    @Test
    public void testHelloWorld() {
        Assert.assertEquals("It doesn't return 'Hello World'", 
            "Hello World", hw.helloWorld());
    }
}

Let's examine the above test code.

Evaluate Class

Your test code must live inside a class named Evaluate . It is, otherwise, just a plain java class.

@Test

This is a JUnit annotation that indicates a test function. You add this annotation before each of your test cases.

assertEquals

This is a method provided by JUnit which compares the expected value and actual value. It takes 3 arguments:

  • Feedback message to be shown if values are not equal
  • Expected value
  • Actual value

You can see other assertions here.

Our test class has a test case testHelloWorld() which asserts equality of "Hello World" and student's function's output.

Testing Outputs

Let's change the previous example a little bit and ask students to print "Hello World" rather than returning the string.

Here is the example solution:

public class HelloWorld {
    public void helloWorld() {
        System.out.print("Hello World");
    }
}

To be able to test the student code, you must be able to read stdout after calling helloWorld() function. For this purpose, we have provided helper class named IOHelper.

For example:

import org.junit.Test;
import org.junit.Assert;
import com.udemy.ucp.IOHelper;

public class Evaluate {
    HelloWorld hw = new HelloWorld();
    IOHelper helper = new IOHelper();

    @Test
    public void testHelloWorld() {
        helper.resetStdOut(); // clean stdout
        hw.helloWorld();
        Assert.assertEquals("It doesn't print 'Hello World'", 
            "Hello World", helper.getOutput());
    }
}

There are a few differences between this test and the previous one:

  • We are using IOHelper to gain access to data written to stdout.
  • Calling IOHelper.resetStdOut() removes output that's been captured so far.
  • Comparing "Hello World" string with output that we read with IOHelper.getOutput().

In this specific case, calling resetStdOut() before getOutput() isn't needed. But, in general, it's a recommended practice. You can see more details about IOHelper below in other examples.

Testing Field and Method Declarations

Now let's change the example and see how to validate if students have declared fields and methods.

Let's consider that the student's task is to declare two fields (name and lastName ) and two methods (printName() and printNumber(int) ) in an Applicant class. For the sake of simplicity, methods can have empty bodies because we don't really care about the behavior here.

The solution for this problem could be implemented as follows:

public class Applicant {
    public String name;
    public String lastName;

    public void printName() {}    
    public void printNumber(int number) {}
}

To be able to test this code, you will need to know whether these fields and methods are declared or not. For this purpose, we have provided EvaluationHelper , which is explained below:


Check if a field is declared in a class

public boolean isFieldDeclared(Object obj, String fieldName)
  • obj: Instance of the class
  • fieldName: Name of the field to be checked

Check if a method is declared in a class

public boolean isMethodDeclared(Object obj, String methodName, 
                                Class<?>... parameterTypes)
  • obj: Instance of the class
  • methodName: Name of the method to be checked
  • parameterTypes: Types of parameters the method expects such as int.class

Below, you can see an example of how to test the student's code:

import org.junit.Test;
import org.junit.Assert;
import com.udemy.ucp.EvaluationHelper;

public class Evaluate {
    EvaluationHelper helper = new EvaluationHelper();
    Applicant applicant = new Applicant();

    @Test
    public void testFieldDeclarations() {
        if (!helper.isFieldDeclared(applicant, "name")) {
            Assert.fail("'name' field is not declared.");
        }
        // You can also check if the field has a specific type
        // Following line is checking if Applicant class has
        // a String field named "surname"
        if (!helper.isFieldDeclared(applicant, "surname", String.class)) {
            Assert.fail("'surname' field is not declared.");
        }
    }

    @Test
    public void testMethodDeclarations() {
        if (!helper.isMethodDeclared(applicant, "printName")) {
            Assert.fail("'printName' method is not declared.");
        }
        if (!helper.isMethodDeclared(applicant, "printNumber", int.class)) {
            Assert.fail("'printNumber' method is not declared.");
        }
    }
}

When these tests fail, they show relevant error messages to students by calling Assert.fail().

Providing Inputs

How do you test whether students are able to read input from stdin?

Consider that the student's task is to read a customer's first and last name from stdin and print the full name to stdout.

The solution for this problem can be implemented as follows:

import java.util.Scanner;

public class Customer {
    public String firstName;
    public String lastName;

    public String getFullName() {
        Scanner scanner = new Scanner(System.in);
        System.out.println("Enter your first name:");
        this.firstName = scanner.next();
        System.out.println("Enter your last name:");
        this.lastName = scanner.next();
        return this.firstName + " " + this.lastName;
    }  
}

IOHelper has a method to fill stdin with given values.

public void setStdInput(String... inputs)
  • inputs: String inputs to be entered to stdin. A line separator will be appended to each input.

The following test class enters first and last name in the console and checks whether student's code can return the full name.

import org.junit.Test;
import org.junit.Assert;

public class Evaluate {
    IOHelper helper = new IOHelper();
    Customer customer = new Customer();

    @Test
    public void testGetFullName() {
        this.helper.setStdInput("Albert", "Einstein");
        String fullName = this.customer.getFullName();
        Assert.assertEquals(
            "For inputs 'Albert' and 'Einstein' it doesn't work.", 
            "Albert Einstein", fullName);
    }
}

Testing Declared Values

Consider that the student's task is to implement a class that has two fields (SSN and age ) and a function (setSSN(String)). One other requirement is that the age field's initial value must be 0.

The solution for this problem can be implemented as follows:

public class Customer {
    public String SSN;
    public int age = 0;

    public void setSSN(String ssn) {
        this.SSN = ssn;
    }
}

The following example tests the values of the declared fields.

import org.junit.Test;
import org.junit.Assert;

public class Evaluate {
    Customer studentSolution = new Customer();

    @Test
    public void testSetSSN() {
        // Set customer's Social Security Number
        this.studentSolution.setSSN("123-45-6789");
        Assert.assertEquals("'setSSN' did not set customer's SSN properly.", 
              "123-45-6789", this.studentSolution.SSN);
    }

    @Test
    public void testInitialAgeValue() {
        // Check if customer's age is 0 if it's not set
        Assert.assertEquals("Initial age must be 0.", 0, this.studentSolution.age);
    }
}

Testing Exceptions

You can expect student code to raise Exceptions in some cases.

So let's improve the example above and ask students to throw IllegalArgumentException if the input passed to a function is not valid. setSSN() won't accept SSNs shorter than 10 characters and setAge() won't accept negative ages. And if age is negative, the Exception message must be "Age can't be a negative number.".

The solution for this problem can be implemented as follows:

public class Customer {
    private String SSN;
    private int age = 0;

    public void setSSN(String ssn) {
        if (ssn.length() < 10) {
            throw new IllegalArgumentException();
        }
        this.SSN = ssn;
    }

    public void setAge(int age) {
        if (age < 0) {
            throw new IllegalArgumentException("Age can't be a negative number.");
        }
        this.age = age;
    }
}

To check whether the student handled exception cases, you can use JUnit's exception testing. Following test class is checking exceptions.

  • **testInvalidSSN** is only checking whether the exception is thrown or not
  • **testInvalidAge** is also checking the exception message
import org.junit.Rule;
import org.junit.Test;
import org.junit.Assert;
import org.junit.rules.ExpectedException;

public class Evaluate {
    @Rule
    public ExpectedException thrown = ExpectedException.none();
    Customer studentSolution = new Customer();

    @Test(expected = IllegalArgumentException.class)
    public void testInvalidSSN() {
        // Check if it raises exception for invalid Social Security Number
        this.studentSolution.setSSN("123");     
    }

    @Test
    public void testInvalidAge() {
        thrown.expect(IllegalArgumentException.class);
        thrown.expectMessage("Age can't be a negative number.");
        // Check if it raises exception for negative age
        this.studentSolution.setAge(-5);
    }
}

You can also use a try / catch block to check whether an exception is thrown and call Assert.fail() if it is not.

Packages and Restrictions

You can ask your students to implement classes inside different packages. However, your test class Evaluate cannot be in a package.

For example, you can ask students to implement a Car class inside com.example.vehicles package.

Here is an example solution:

package com.example.vehicles;

public class Car {

}

And the evaluation of this code can be as follows:

import org.junit.Test;
import org.junit.Assert;
import com.example.vehicles.Car;

public class Evaluate {
    @Test
    public void testPackageNames() {
        // There's no need to run any tests since importing Car was successful
        Assert.assertTrue(true);
    }
}

The only logical thing the above test code does is that it imports Car class from com.example.vehicles package. If the package wasn't defined, this would raise a compile error. (If you want to avoid the compilation error, you can use the Java reflection API to dynamically lookup and instantiate the student's class).

IOHelper

Let's see all the details of IOHelper class which is provided by Udemy to help you deal with reading outputs of and entering inputs to student solutions.

package com.udemy.ucp;

public class IOHelper {

    /**
     * Enter values to standard input (stdin). 
     * A line separator will be appended to each input.
     * @param inputs String inputs to be entered to stdin. 
     */
    public void setStdInput(String... inputs) {}

    /**
     * Returns all the outputs printed to standard output after the last reset.
     * If you do not use `resetStdOut()`, 
     * this may return outputs from other test cases.
     */
    public String getOutput() {}

    /**
     * Reset (clear) the output printed to stdout.
     */
     public void resetStdOut() {}
}

EvaluationHelper

Here are all the details of EvaluationHelper class to help you test method and field declarations.

package com.udemy.ucp;

public class EvaluationHelper {

    /**
     * Check if a method is declared in a class
     * @param obj Instance of the class
     * @param methodName Name of the method to be checked
     * @param parameterTypes Types of parameters the method expects such as int.class
     */
    public boolean isMethodDeclared(Object obj, String methodName, 
                                    Class<?>... parameterTypes)

    /**
     * Checks whether a field is declared in a class
     * @param obj Instance of the class
     * @param fieldName Name of the field to be checked
     */
    public boolean isFieldDeclared(Object obj, String fieldName)

    /**
     * Checks whether a field with specific type is declared in a class
     * @param obj Instance of the class
     * @param fieldName Name of the field to be checked
     * @param fieldType Type of the field
     */
    public boolean isFieldDeclared(Object obj, String fieldName, Class<?> fieldType)

    /**
     * Return the value of a field from a given instance
     * @param obj Instance that contains the field
     * @param fieldName Name of the field whose value will be returned
     */
    public Object getFieldValue(Object obj, String fieldName)

    /**
     * Return the type of a field in a class
     * @param obj Instance of the class that contains the field
     * @param fieldName Name of the field whose value will be returned
     */
    public Class<?> getFieldType(Object obj, String fieldName)

    /**
     * Checks whether a field has private modifier or not
     * @param obj Instance of the class
     * @param fieldName Name of the field to be checked
     */
    public boolean isFieldPrivate(Object obj, String fieldName)

    /**
     * Checks whether a field has protected modifier or not
     * @param obj Instance of the class
     * @param fieldName Name of the field to be checked
     */
    public boolean isFieldProtected(Object obj, String fieldName)

    /**
     * Checks whether a field has public modifier or not
     * @param obj Instance of the class
     * @param fieldName Name of the field to be checked
     */
    public boolean isFieldPublic(Object obj, String fieldName)

    /**
     * Checks whether a field has static modifier or not
     * @param obj Instance of the class
     * @param fieldName Name of the field to be checked
     */
    public boolean isFieldStatic(Object obj, String fieldName)

    /**
     * Checks whether a field has final modifier or not
     * @param obj Instance of the class
     * @param fieldName Name of the field to be checked
     */
    public boolean isFieldFinal(Object obj, String fieldName)
}

results matching ""

    No results matching ""