Com S 362 Lecture Outline ----------------------------------------------- Contracts Use cases are composed of: Success and Failure Scenarios These scenarios describe the system behavior. Will these scenarios be successful irrespective of the state in which an actor starts? ----------------------------------------------- For example, consider the checkout scenario Actor [Patron] --Goal [Checkout] --> System The actor cannot checkout books using a library administration system, if the system is not turned on. In other words, for the actor to be able to fulfill their goals, the system has to be in the [Turned On] state. A broader condition is that the library has to be open on that particular date at that particular time in order for the actor to be able to fulfill their goal. ----------------------------------------------- The operation contracts specifies the conditions that will be required to start the use case scenario (pre-condition) and the conditions that will be true after the scenario is complete (post-condition). ----------------------------------------------- A Programming Example: Let us consider the method "Pop" of a class "Stack". Calling "Pop" retrieves the top item on the stack. A pre-condition to retrieving the top item will be that the stack is non-empty. A post-condition will be that number of items in the stack will be one less than previous numbers. ----------------------------------------------- A Use Case Scenario Example: Consider a use case scenario for the patient monitoring system discussed in previous lectures. #2: A patients' pulse is being monitored. The pulse reader measures the pulse rate to be 85 pulses per minute. The monitoring machine records the current time and pulse rate. A pre-condition to this use case scenario is that the safe range for that patient is already specified. The post-condition for this scenario is that the patient record now contains the most recent reading of the patients' pulse rate. ----------------------------------------------- Defensive Programming A philosophy of programming, where we aim to build robustness into the software systems by introducing checks for erroneous conditions. The client is often not required to satisfy any pre-condition and the obligation rests on the service to satisfy clients' need in almost all possible circumstances. Programmers introduce as many checks as possible, even if these checks are redundant with the client code. - Results in more complex software system - Increased chances of error due to complexity - Often results in performance overheads due to excessive checking ----------------------------------------------- Design by Contract Builds upon the contractual obligations of the client and the service. The service is only obligated to satisfy a clients' request if the pre-conditions of the contract are met. On the other hand, if the pre-conditions are met by the client, the service is obligated to ensure the post conditions. ----------------------------------------------- ----------------------------------- | Obligation | Benefits | ----------------------------------- Client | Pre-condition | Post-condition | ----------------------------------- Service | Post-condition | Pre-condition | ----------------------------------- ----------------------------------------------- How to notify clients that the pre-conditions are not satisfied? Example: Consider a simple method: class List { collection innerList; public bool add(Element[] elist){ foreach(Element e in elist){ if(!innerList.add(e)) return false; } return true; } } ----------------------------------------------- How can we make the method add "robust"? Option 1-a: Check elist, return false in cases elist is "null" or elist has zero elements, true otherwise. - Is checking an additional overhead? - What if users don't check for the return value of add? - Checking for return values makes programs complicated. Same code is repeated every where the method add is called. Option 1-b: Check elist, and exit the program in erroneous cases. - Poor reuse - Reasoning/Debugging often difficult Option 1-c: Check elist, throw exceptions in erroneous cases. What are exceptions? -------------------- - An abnormal condition (in natural language) - In C#/Java/C++/other OO languages, an object that represents the abnormal condition. More precisely encapsulates the information about the abnormal condition. Exception mechanisms Goal: Communicate that the abnormal condition has occurred to the entity that is responsible for handling it. - Terms: throws, catch Throws often is a synonym to signaling an abnormal condition. Catch is analogous to declaring the ability to provide a solution to the abnormal condition. Four important things: - What type of abnormal condition? - Where is it occurring? - Who is responsible for handling it a.k.a handler ? - What is the scope of this handler? Example: public boolean add(Element[] elist) throws IllegalArgumentException{ try{ // Try block -- determines scope of handler foreach(Element e in elist){ if(!innerList.add(e)) return false; } }catch(NullPointerException e){ // Provides responsible handler // Previous line of code also specifies the type of // abnormal condition that this block is concerned // about. // construct an IllegalArgumentException out of this // and throw it. } return true; } Benefits: - Overall a better solution compared to 1-a and 1-b - Users who don't check will not miss. - Reasoning/debugging will not be difficult. because you will get exact stack trace of where the problem occured. Option 2: Require user to check that elist is not "null" before calling add - doesn't work for untrusted clients ------------------------------------------ Unit testing with JUnit A. what is unit testing? ------------------------------------------ WHAT IS UNIT TESTING? def: *testing* is the process of showing that a program works for certain inputs def: a *unit* is a module, or small set of modules. In Java, a unit is a class or interface, or a small set of them. E.g., an interface and 3 classes that implement it. def: *unit testing* is testing of a unit. ------------------------------------------ Do you get more confidence by running more test data? B. goals and motivation ------------------------------------------ WHY UNIT TESTING? Code isn't right if it's not tested Practical: - most programmers rely on testing - e.g., Microsoft has 1 tester per developer - you could get work as a tester Divide and conquer: - split system into units - debug units individually - narrow down places where bugs can be - don't want to chase down bugs in other units Support regression testing: - so can make changes to lots of code and know if you broke something - can make big changes with confidence ------------------------------------------ C. how to do unit testing ------------------------------------------ HOW TO DO UNIT TESTING Build system in layers - start with classes that don't depend on others - continue testing building on already tested classes Benefit: - avoids having to write stubs - when testing a module, ones it depends on are reliable ------------------------------------------ How does low coupling help testing? How does high coupling hurt it? 1. example without JUnit ------------------------------------------ PROGRAM TO TEST public class ISqrt { /** Return an integer approximation to the square root of y. */ public static int isqrt(int y) { int guess = 1; while (guess * guess < y) { guess++; } return guess; } } ------------------------------------------ ------------------------------------------ CONVENTIONAL TESTING /** Test ISqrt. */ public class TestISqrtNoJUnit { /** Run the tests. */ public static void main(String[] args) { printTestResult(0); printTestResult(1); printTestResult(2); printTestResult(3); printTestResult(4); printTestResult(7); printTestResult(9); printTestResult(100); } private static void printTestResult(int arg) { System.out.print("isqrt("); System.out.print(arg + ") ==> "); System.out.println( ISqrt.isqrt(arg)); } } ------------------------------------------ ------------------------------------------ CONVENTIONAL TEST OUTPUT isqrt(0) ==> 1 isqrt(1) ==> 1 isqrt(2) ==> 2 isqrt(3) ==> 2 isqrt(4) ==> 2 isqrt(7) ==> 3 isqrt(9) ==> 3 isqrt(100) ==> 10 ------------------------------------------ What does this say about the code? Is it right? What's the problem with this kind of test output? 2. example with JUnit ------------------------------------------ import junit.framework.*; import junit.textui.*; /** Test ISqrt. */ public class TestISqrt extends TestCase { /** Run the tests. */ public static void main(String[] args) { TestRunner.run(suite()); } /** Test isqrt. */ public void testIsqrt() { // line 26 below assertEquals(0, ISqrt.isqrt(0)); assertEquals(1, ISqrt.isqrt(1)); assertEquals(1, ISqrt.isqrt(2)); assertEquals(1, ISqrt.isqrt(3)); assertEquals(2, ISqrt.isqrt(4)); assertEquals(2, ISqrt.isqrt(7)); assertEquals(3, ISqrt.isqrt(9)); assertEquals(10, ISqrt.isqrt(100)); } /** Returns the test suite for this test class. */ public static Test suite() { return new TestSuite(TestISqrt.class); } } ------------------------------------------ ------------------------------------------ COMPILATION AND OUTPUT (formatted) $ javac ISqrt.java TestISqrt.java $ java TestISqrt .F Time: 0.03 There was 1 failure: 1) testIsqrt(TestISqrt) junit.framework.AssertionFailedError: expected:<0> but was:<1> at TestISqrt.testIsqrt (TestISqrt.java:26) at sun.reflect.NativeMeth... at sun.reflect.NativeMeth... at sun.reflect.Delegating... at TestISqrt.main (TestISqrt.java:15) FAILURES!!! Tests run: 1, Failures: 1, Errors: 0 ------------------------------------------ Is this better? Why? 3. exercise for students ------------------------------------------ FOR YOU TO DO Write a JUnit test class for testing public class ForYou { /** Return the minimum of x and y. */ public static int min(int x, int y) { ... } } === By filling in the following: === import junit.framework.*; import junit.textui.*; /** Test ForYou. */ public class TestForYou extends TestCase { /** Test min. */ public void testMin() { } /* ... rest as before ... */ } ------------------------------------------ D. using the JUnit framework 1. definitions a. test case, success and failure ------------------------------------------ SOME TERMINOLOGY def: A *test case* for a method M is a pair (o, args) where o is not null and M can be sent to o, args is a tuple of arguments that can be passed to M def: A test case, (o, args), for M *succeeds* iff it o.M(args) behaves as expected. def: A test case, (o, args), for M *fails* iff it does not behave as expected. ------------------------------------------ Why should o not be null? If M has a bug that is revealed by a test case, does that test case for M succeed or fail? b. test code terms ------------------------------------------ PARTS OF TEST CODE def: The *test fixture* is the set of variables used in testing. def: The *test driver* is the class that runs the tests def: The *test oracle* for a test case is the code that decides success or failure for that test case. ------------------------------------------ What in the code we saw so far was the test driver? The oracle? What difference is there between JUnit testing and non JUnit testing in what we saw before? 2. basic usage ------------------------------------------ BASIC USAGE OF JUNIT (1) To test a type T: 1. Write a class like: import junit.framework.*; import junit.textui.*; /** Test of class T. */ public class TestT extends TestCase { /** Run the tests. */ public static void main(String[] args) { TestRunner.run(suite()); } /** Returns the test suite for this test class. */ public static Test suite() { return new TestSuite(TestT.class); } } ------------------------------------------ ------------------------------------------ BASIC USAGE OF JUNIT (2) 2. Compile T.java and TestT.java $ javac T.java TestT.java 3. Run the JUnit graphical user interface on TestT $ java junit.swingui.TestRunner TestT OR Run the text interface (good from makefiles) $ java TestT 4. Look at the failures and errors ------------------------------------------ 3. extended example from StickSync ------------------------------------------ package sticksync; import java.io.BufferedWriter; import java.io.File; import java.io.FileWriter; import java.io.IOException; import java.io.Writer; import java.util.Date; import junit.framework.TestCase; /** * This is a JUnit test case for the Location class. * @author Hridesh Rajan */ public class TestLocation extends TestCase { private File[] testFiles; private Location[] testLocs; public TestLocation(String name) { super(name); } /** Run the tests. */ public static void main(String[] args) { junit.textui.TestRunner.run(suite()); } /** Returns the test suite for this test class. */ public static junit.framework.Test suite() { return new junit.framework.TestSuite(TestLocation.class); } public void testConstructors() { for (int i=0; i < testFiles.length; i++) { Location l1 = new Location(testFiles[i].getPath()); assertEquals(l1.getPath(),testLocs[i].getPath()); } } public void testReadable() { for (int i=0; i < testLocs.length; i++) { assertTrue(testLocs[i].readable()); } } public void testWriteable() { for (int i=0; i < testLocs.length; i++) { assertTrue(testLocs[i].writeable()); testFiles[i].setReadOnly(); assertTrue(!testLocs[i].writeable()); } } public void testCopyTo() throws IOException { String orig0 = testLocs[0].contentsAsString(); String orig1 = testLocs[1].contentsAsString(); assertTrue(!orig0.equals(orig1)); Date d = testLocs[0].copyTo(testLocs[1]); assertEquals(orig0, testLocs[1].contentsAsString()); assertEquals(d,testLocs[0].modTime()); assertEquals(d,testLocs[1].modTime()); } public void testAbsorbChangesFrom() throws IOException { // !E2! This test assumes merge will fail boolean result = testLocs[0].absorbChangesFrom(testLocs[1]); assertTrue(!result); assertEquals(testLocs[1].contentsAsString(), testLocs[0].getNearbyLocation().contentsAsString()); } protected void setUp() throws Exception { testFiles = configureTestFiles(2); testLocs = new Location[testFiles.length]; for (int i=0; i but was: at ...TestLocation.testCopyTo( TestLocation.java:76) at sun.reflect.NativeMethod... at sun.reflect.NativeMethod... at sun.reflect.Delegating... at ...TestLocation.main( TestLocation.java:38) FAILURES!!! Tests run: 5, Failures: 1, Errors: 0 ------------------------------------------ ------------------------------------------ class Location { /* ... */ /** Copies this location's file to the given location's file. */ public Date copyTo(Location loc) throws IOException { InputStream source = new BufferedInputStream( new FileInputStream(getFile())); OutputStream dest = new BufferedOutputStream( new FileOutputStream( loc.getFile())); byte[] buf = new byte[256]; int count = 0; while ((count = source.read(buf)) != -1) { dest.write(buf,0,count); } source.close(); dest.close(); getFile().setLastModified( loc.modTime().getTime()); return new Date(); } } ------------------------------------------ E. other aspects of JUnit 1. naming conventions ------------------------------------------ NAMING CONVENTIONS Test methods start with "test" e.g., testCopyTo, testIsqrt Test classes start with "Test" e.g., TestLocation ------------------------------------------ 2. test suites ------------------------------------------ TEST SUITES Organize tests into a larger test. Help with automation of testing. See the JUnit documentation for details. ------------------------------------------ II. Design by Contract with JML A. motivation 1. design by contract ------------------------------------------ DESIGN BY CONTRACT A way of recording: - details of method responsibilities - avoiding constantly checking arguments - assigning blame across interfaces Example: //@ requires y >= 0; //@ ensures (* \result is root of y *); Contracts in software: Obligations of client - passes positive integer Rights of client - gets square root approximation Rights of implementor - assumes argument is positive Obligations of implementor - must compute and return square root ------------------------------------------ ------------------------------------------ PRE AND POSTCONDITIONS def: a method's *precondition* says what must be true to call it. Example: //@ requires y >= 0; def: a method's *normal postcondition* says what is true when it returns (without throwing an exception) Example: //@ ensures \result * \result == y; def: an *exceptional postcondition* says what is true when a method throws an exception. /*@ signals (IllegalArgumentException e) @ e != null && y < 0; @*/ ------------------------------------------ Did you see these in fully dressed use cases? Which is the obligation of the client? Of the implementor? Which is the rights of the client? Of the implementor? ------------------------------------------ SAYING WHAT EXCEPTIONS MAY BE THROWN The signals_only clause says what excpetions may be thrown. //@ signals_only IllegalArgumentException; meaning: /*@ signals (Exception e) @ e instanceof IllegalArgumentException; @*/ default: the list in the method's throws clause. ------------------------------------------ So what does this mean for methods that don't list anything in their throws clause? ------------------------------------------ RELATIONAL MODEL OF METHODS Think of a method as a relation: Inputs <--> Outputs precondition describes the domain postcondition describes the relationships ------------------------------------------ 2. documentation ------------------------------------------ CONTRACTS AS DOCUMENTATION For each method say: - what it requires (if anything) - what it ensures Example in JML: //@ requires y >= 0; /*@ ensures -y <= \result @ && \result <= y @ && \result * \result <= y @ && y < (Math.abs(\result) + 1) @ * (Math.abs(\result) + 1); @*/ public static int isqrt(int y) { ... } Contracts are: - more abstract than code - not necessarily constructive - often machine checkable (in JML) and so can help with debugging - machine checkable contracts can always be up-to-date In summary, good documentation ------------------------------------------ How is constructing a square root different than the above spec? ------------------------------------------ ABSTRACTION BY SPECIFICATION A contract can be satisfied in many ways: E.g., for square root: - linear search - binary search - Newton's method ... These will have varying non-functional properties - efficiency - memory usage So a contract abstracts from all these implementations ==> can change implementations later ------------------------------------------ 3. blame assignment, modularity ------------------------------------------ MORE ADVANTAGES OF CONTRACTS Blame assignment: Who is to blame if: - precondition doesn't hold? - postcondition doesn't hold? Avoids inefficient defensive checks: /*@ requires a != null @ && (* a is sorted *); @*/ int binarySearch(Thing [] a, Thing x) { ... } ------------------------------------------ ------------------------------------------ MODULARITY OF REASONING Typical OO code: ... source.close(); dest.close(); getFile() .setLastModified( loc.modTime().getTime()); return modTime(); How to understand this code? - read the code for all methods? - read the contracts for all methods? ------------------------------------------ What if some methods are recursive? What if there is polymorphism? ------------------------------------------ RULES FOR REASONING Client code - must work for every implementation that satisifies contract - can thus only use the contract (not the code!) - must establish precondition - gets to assume postcondition //@ assert 9 >= 0; int res = ISqrt.isqrt(9); //@ assert res * res == 9; Implementation code: - must satisfy contract - gets to assume precondition - must establish postcondition - but can do anything permitted by it ------------------------------------------ ------------------------------------------ CONTRACTS AND INTENT Code makes a poor contract, because can't separate: - what is intended (contract) - what is an implementation decision E.g., if floating point square root routine gives an approximation good to 3 decimal places, can that be changed in the next release? By contrast, contracts: - allow vendors to specify intent - allow vendors freedom to change details - tell clients what they can count on ------------------------------------------ What kinds of changes might vendors want to make that don't break existing contracts? B. what is JML? ------------------------------------------ JML Stands for "Java Modeling Language" Design by contract for Java Available from www.jmlspecs.org Uses Java 1.4.1 ------------------------------------------ C. annotations ------------------------------------------ ANNOTATIONS JML specifications are contained in annotations, which are comments like: //@ ... or /*@ ... @ ... @*/ At-signs (@) on the beginnings of lines are ignored within annotations ------------------------------------------ What's the advantage of using annotations? D. simple examples 1. informal specs (organize specifications) ------------------------------------------ INFORMAL DESCRIPTIONS An informal description looks like: (* some text describing a property *) It is treated as a boolean by JML. Allows: - escape from formality - organization of English as contracts Example: public class ISqrt { //@ requires (* y is positive *); /*@ ensures (* \result is an @ int approximation to @ the square root of y *) @ && \result >= 0; @*/ public static int isqrt(int y) { ... } } ------------------------------------------ ------------------------------------------ FOR YOU TO DO Write informal pre and postconditions for the other operations of this type: public class Person { private String name; private int weight; /*@ also @ ensures \result != null @ && (* \result is a display @ form of this person *); @*/ public String toString() { return "Person(\"" + name + "\"," + weight + ")"; } public int getWeight() { return weight; } public void addKgs(int kgs) { if (kgs >= 0) { weight += kgs; } else throw new IllegalArgumentException(); } } public Person(String n) { name = n; weight = 0; } } ------------------------------------------ 2. formalization Are the informal specifications longer than the code sometimes? ------------------------------------------ WRITING FORMAL SPECIFICATIONS IN JML Formal assertions are written as Java expressions, but: - Cannot have side effects - no use of =, ++, -- - can only call "pure" methods - Can use some extensions to Java: Syntax meaning ===================================== \result result of method call a ==> b a implies b b <== a b is implied by a a <==> b a iff b \old(E) value of E in pre-state ------------------------------------------ ------------------------------------------ public class Person { private /*@ spec_public non_null @*/ String name; private /*@ spec_public @*/ int weight; /*@ public invariant !name.equals("") @ && weight >= 0; @*/ //@ also //@ ensures \result != null; public String toString() { return "Person(\"" + name + "\"," + weight + ")"; } /*@ ensures \result == weight; @*/ public /*@ pure @*/ int getWeight() { return weight; } /*@ ensures kgs >= 0 @ && weight @ == \old(weight + kgs); @ signals (IllegalArgumentException e) @ kgs < 0; @*/ public void addKgs(int kgs) { if (kgs >= 0) { weight += kgs; } else throw new IllegalArgumentException(); } } /*@ requires n != null @ && !n.equals(""); @ ensures n.equals(name) @ && weight == 0; @*/ public Person(String n) { name = n; weight = 0; } } ------------------------------------------ ------------------------------------------ MEANING OF ENSURES AND SIGNALS |-----------| |------------| | | | | [ Normal | [ Exceptional| | (Return) | | (Throws) | [ O | [ O | | | | | | | | | [ | [ | | | | | |-----------| |------------| /------------/ /------------/ / ensures / /signals(...)/ / kgs >= 0 ... / kgs < 0; / /------------- /------------/ ------------------------------------------ 3. invariants ------------------------------------------ INVARIANTS Are properties that are always true of an object's state, when control is not inside the object's methods /*@ public invariant !name.equals("") @ && weight >= 0; @*/ Allow you to define: - acceptable states of an object - "consistency" of an object's state ------------------------------------------ 4. exercise for students ------------------------------------------ FOR YOU TO DO Formally specify the method (in Person) public void changeName(String newName) { name = newName; } Hint: watch out for the invariant! ------------------------------------------ E. basic tool usage 1. overview of tools ------------------------------------------ TOOLS FOR JML Runtime assertion checking compiler: jmlc Java interpreter with RAC support: jmlrac JUnit test oracle generator: jmlunit JUnit test runner with RAC support: jml-junit Documentation (HTML) generator: jmldoc Type checker: jml ------------------------------------------ 2. jmlc ------------------------------------------ The Runtime Assertion Checking Compiler jmlc Usage: $ jmlc Person.java produces Person.class $ jmlc -Q *.java produces *.class, quietly $ jmlc -d ../bin Person.java produces ../bin/Person.class ------------------------------------------ ------------------------------------------ RUNNING CODE COMPILED WITH JMLC Must have JML's jmlruntime.jar in Java's boot class path Automatic if you use script jmlrac $ jmlrac PersonMain ------------------------------------------ ------------------------------------------ A MAIN PROGRAM public class PersonMain { public static void main(String [] argv) { System.out.println(new Person(null)); System.out.println(new Person("")); } } ------------------------------------------ ------------------------------------------ EXAMPLE (formatted) $ jmlc -Q PersonMain.java Person.java $ jmlrac PersonMain Exception in thread "main" org.jmlspecs.jmlrac.runtime .JMLEntryPreconditionError: by method Person.Person regarding specifications at Person.java:34:22 when 'n' is null at Person.checkPre$$init$$Person( Person.java:984) at Person.(Person.java:45) at PersonMain.main(PersonMain.java:8) ------------------------------------------ Any questions about the tools? F. more examples, from StickSync ------------------------------------------ STICKSYNC EXAMPLES (MORE REALISTIC CONTRACTS) class Location { /** The pathname of a file. */ private /*@ spec_public non_null @*/ String path; /** Caches the associated file. * Access only via getFile(). * This is null if no File is cached. * @see getFile() */ private transient File fileCache; /** Returns the associated file */ //@ ensures \result != null; private File getFile() { if (fileCache == null) { fileCache = new File(path); } return fileCache; } /** Initializes this Location. * @param fullpath the full pathname * to the file. */ //@ requires fullpath != null; //@ assignable path; //@ ensures path.equals(fullpath); public Location(String fullpath) { path = fullpath; } /** Is the file readable? */ /*@ ensures \result @ == getFile().canRead(); @*/ public /*@ pure @*/ boolean readable() { return getFile().canRead(); } /** Copies this location's file * to the given location's file. */ //@ requires loc != null; //@ ensures \result != null; /*@ ensures \result.equals(modTime()) @ && loc.modTime().equals(modTime()); @*/ public Date copyTo(Location loc) throws IOException { ... } } ------------------------------------------ Can you read these? To what extent do these specifications add to the javadocs? G. other things in JML 1. model fields and represents clauses ------------------------------------------ MODEL FIELDS, REPRESENTS CLAUSES What if you want to change a spec_public field's name? private /*@ spec_public non_null @*/ String path; to private /*@ non_null @*/ String fullPath; For specifications: - need to keep the old name public - but don't want two Strings So use a model variable: //@ public model non_null String path; and a represents clause //@ private represents path <- fullPath; ------------------------------------------ ------------------------------------------ MODEL VARIABLES Are specification-only variables. Like domain-level constructs. Given value only by represents clauses: path abstract, model ^ | represented by | fullPath concrete (real) ------------------------------------------ 2. web site for more info III. JML and JUnit based unit testing A. motivation 1. problem of writing test oracle code ------------------------------------------ WRITING TEST ORACLES Lots of test cases ==> don't want to write oracle for each Duplicates documentation, specifications ==> so hard to maintain Easy to not make them abstract ==> so depends on representation ------------------------------------------ B. idea ------------------------------------------ IDEA JML specifications can be checked at runtime. If the specifications are complete, then they can serve as test oracles: - precondition descibes what test cases are acceptable for testing a method - postcondition describes expected behavior So: 1. Compile code with jmlc 2. Test driver monitors assertion violations a. Entry precondition violations ==> reject test data b. Other violations ==> test fails (code has a bug) ------------------------------------------ What happens if the precondition isn't specified? What happens if the postcondtion isn't specified? C. simple example 1. using jmlunit tool ------------------------------------------ USING JMLC AND JMLUNIT TO DO TESTING Compile code with jmlc $ jmlc -U ISqrt.java produces ISqrt.class Generate JUnit test driver and test case files $ jmlunit ISqrt.java produces ISqrt_JML_Test.java and ISqrt_JML_TestData.java ------------------------------------------ ------------------------------------------ WHAT'S IN THE _JML_Test*.java FILES // This file was generated by jmlunit ... import junit.framework.*; import org.jmlspecs.jmlunit.*; import org.jmlspecs.jmlunit.strategies.*; public class ISqrt_JML_Test extends ISqrt_JML_TestData { /* ... */ /** Run the tests. */ public static void main(String[] args) { JMLTestRunner.run(suite()); } /** Return the test suite for * this test class. */ public static junit.framework.Test suite() { ISqrt_JML_Test testobj = new ISqrt_JML_Test("ISqrt_JML_Test"); TestSuite testsuite = testobj.overallTestSuite(); // Add instances of Test // found by the reflection mechanism. testsuite.addTestSuite (ISqrt_JML_Test.class); testobj.addTestSuiteForEachMethod (testsuite); return testsuite; } ------------------------------------------ ------------------------------------------ /** Test to see if the code * was compiled with jmlc. */ public void test$IsRACCompiled() { /* ... */ } /** Add tests for the isqrt method * to the overall test suite. */ private void addTestSuiteFor$TestIsqrt (TestSuite overallTestSuite$) { TestSuite methodTests$ = this.emptyTestSuiteFor("isqrt"); try { IntIterator vint$1$iter = this.vintIter("isqrt", 0); this.check_has_data (vint$1$iter, "this.vintIter(\"isqrt\", 0)"); while (!vint$1$iter.atEnd()) { final int y = vint$1$iter.getInt(); methodTests$.addTest (new TestIsqrt(y)); vint$1$iter.advance(); } } catch (TestSuiteFullException e$) { // methodTests$ doesn't want more } overallTestSuite$.addTest (methodTests$); } } ------------------------------------------ 2. adding test data ------------------------------------------ ADDING TEST DATA // Edit test data into ISqrt_JML_TestData import junit.framework.*; public abstract class ISqrt_JML_TestData extends TestCase { /* ... */ /** Return the overall test suite * for accumulating tests. */ public TestSuite overallTestSuite() { return new TestSuite ("Overall tests for ISqrt"); } /** Return an empty test suite for * accumulating tests for the * named method. */ public TestSuite emptyTestSuiteFor (String methodName) { return new TestSuite(methodName); } /** Return a new, freshly allocated * indefinite iterator that * produces test data of type * int * for testing the method named by * methodName in a loop that encloses * loopsThisSurrounds many other loops. */ protected IntIterator vintIter (String methodName, int loopsThisSurrounds) { return vintStrategy.intIterator(); } /** The strategy for generating test * data of type int. */ private IntStrategyType vintStrategy = new IntStrategy() { protected int[] addData() { return new int[] { 8, 9, 10, -5, -17, 20, 24, 99, 100, 101, }; } }; } ------------------------------------------ What test data would you use for testing Person's addKgs method? 3. running the test, output ------------------------------------------ RUNNING THE TESTS Adjust the CLASSPATH (on Linux, one line delete from \ to just before next /) $ CLASSPATH=.:\ /opt/junit/junit.jar:\ /opt/JML/bin/jmlruntime.jar:\ /opt/JML/bin/jmljunitruntime.jar $ export CLASSPATH Compile the files: $ jmlc -U ISqrt.java $ javac -g ISqrt_JML_Test*.java Then run the tests either by: $ jml-junit ISqrt_JML_TestData OR $ jmlrac ISqrt_JML_Test ------------------------------------------ ------------------------------------------ OUTPUT (formatted) ..F...F..F...F.F.F..F Time: 0.056 There were 7 failures: 1) isqrt(ISqrt_JML_Test$TestIsqrt) junit.framework.AssertionFailedError: Method 'isqrt' applied to Receiver class ISqrt Argument y: 0 Caused by: org.jmlspecs.jmlrac.runtime .JMLNormalPostconditionError: by method ISqrt.isqrt regarding specifications at File "ISqrt.java", line 4, character 22 when 'y' is 0 '\result' is 1 at ISqrt.checkPost$isqrt$I... at ISqrt.isqrt(ISqrt.java:... at ISqrt_JML_Test$TestIsqr... at ISqrt_JML_Test$OneTest.... at ISqrt_JML_Test$OneTest.... at org.jmlspecs.jmlunit.JM... at org.jmlspecs.jmlunit.JM... at ISqrt_JML_Test.main(ISq... 2) isqrt(ISqrt_JML_Test$TestIsqrt) junit.framework.AssertionFailedError: Method 'isqrt' applied to Receiver class ISqrt Argument y: 8 Caused by: org.jmlspecs.jmlrac.runtime .JMLNormalPostconditionError: by method ISqrt.isqrt regarding specifications at File "ISqrt.java", line 4, character 22 when 'y' is 8 '\result' is 3 at ISqrt.checkPost$isqrt$ISqrt... at ISqrt.isqrt(ISqrt.java:445)... ... FAILURES!!! Tests run: 14, Failures: 7, Errors: 0 JML test runs: 11/14 (meaningful/total) ------------------------------------------ D. conclusions ------------------------------------------ SUMMARY Unit Testing - should be done in layers - should be systematic, automated ==> repeatable, efficient (in human time) - JUnit helps do that for Java programs Design by Contract - documents assumptions and obligations - Specifications more abstract than code - Specifications that are checked: - will be up to date - can help in isolating bugs (blame assignment) JML/JUnit combination: - Use specifications as test oracles - Limitation: how complete the specification is. ------------------------------------------