Close Window

Show Source |    | About   «  17.11. Indexing   ::   Contents   ::   17.13. Lists  »

17.12. JUnit Testing

17.12.1. Testing

What is the difference between testing and debugging?

How much time have you already spent on this project testing/debugging your code?

17.12.2. JUnit testing and code coverage

  • It is a true art to be able to think in your head all the possible ways that your program could go wrong.

  • One thing that you can get from proper JUnit testing is an indication of what lines are not covered.

  • This indicates situations that you have not yet thought to test.

  • Sort of an automatic test generator!

17.12.3. Philosophy

  • Essential rule: Anything that you do in a test must be followed with assertions to verify that what you did is correct.

  • Every unit test does two things:
    1. Changes the state of the program. You can assert that these changes are correct.

    2. Covers (possibly new) lines of code

    You want these two to be in alignment.

17.12.4. A Bad test (1)

  • I see many tests like this:

    public void testMInit() {
      Memman mem = new Memman();
      Memman.main(new String[] {"25", "20", "P1SampleInput.txt"});

17.12.5. A Bad test (2)

Why is this so bad?
  • It violates our essential rule.

  • There is no testing of what running the program on input DID. Pretty much your only conclusion is that the program did not crash.

  • But worse: Lots of lines of code are “covered”. So you don’t even know what paths have NOT been tested.

WARNING: Project 2 will be unforgiving of this sort of thing.

17.12.6. Full test of output

public void testSampleInput() throws Exception {
  String[] args = new String[3];
  args[0]= "10"; args[1]= "32"; args[2]= "P1sampleInput.txt";
  assertFuzzyEquals( systemOut().getHistory(),
        "|When Summer's Through| " +
        "does not exist in the song database.\n" +
        "(0,32)\n" +
        "|Watermelon Man| is added to the song database.\n" +
        "(44,11) -> (121,4) -> (319,1)\n");

17.12.7. Selective Testing of Output

public void testEmpty()
    throws Exception {
  String[] args = new String[3];
  args[0] = "10";
  args[1] = "32";
  args[2] = "EmptyTest.txt";
  System.out.println("Empty test");

17.12.8. What would be good testing for Project 1?


17.12.9. Models

  • JUnit testing compares a model of what the program should do against what your program does do.

  • Executing commands puts your program into a certain state (expressed by the output).

  • The assertions define characterstics of what you expect from that state. This is the model.

  • The test then compares what state YOUR program is in (expressed by the output) against the model (assertions).

17.12.10. What if your model is wrong?

  • If you have a model in your head, and you write the program to that model, and you test to that model, a “properly working” program will meet that model.

  • What if your model does not match reality?

  • In this program, that most often happens when:
    • Your output text is not what is expected. BUT you should have used the sample output file to write your tests.

    • You have the wrong model about how probing works. BUT you should then see that you pass your tests, and fail the (one) Web-CAT test. Then you should be suspicious about your model if they tell you different things.

17.12.11. Regression testing

  • This means running all of your old tests on the program to make sure that any new changes don’t break anything.

  • Students sometimes add print statements to help them debug, and then forget to remove them. Then Web-CAT tells them they failed a bunch of tests.

    • Your unit tests should warn you about this.

  • If you find a bug, and your tests all pass, then update the tests to trigger on the bug.

   «  17.11. Indexing   ::   Contents   ::   17.13. Lists  »

Close Window