Testing

Candidates should be able to:

• describe syntax errors and logic errors which may occur while developing a program
• understand and identify syntax and logic errors
• select and justify test data for a program, stating the expected outcome of each test.

---

What are syntax errors?

Syntax errors are errors that occur when instructions do not follow the rules or the grammar of the programming language. The instruction cannot, therefore, be executed and an error message will usually be generated. Examples of syntax errors include:

• Spelling errors with keywords
  i.e.

   

  PENCOLOUR = "#00FF00"

  instead of

  PENCOLOR = "#00FF00"

• Spelling errors with variables
  i.e.

   

  max_value = 100, 

  PRINT maxvalue

• Missing or extra brackets or separators in an instruction.
  i.e.

   

  IF(answer="True",,"Correct","Wrong"))

• Missing arguments in an instruction.
  i.e.

   

  DRAWRECTANGLE(50,50)

  should be

  DRAWRECTANGLE(50,50,200,400)

• Not closing iteration loops correctly.
  i.e. missing the ENDWHILE from a WHILE / ENDWHILE loop.
• Not putting double quotes around text so the program assumes the text is a variable.
  i.e.

   

  PRINT Hello World

---

What are logic errors?

Logic errors do not stop a program from running or producing error messages but will result in a program not running in the way it was expected to. Examples of logic errors include:

• Errors in a mathematical formula.
  i.e.

   

  result = (total + step) * 12

  will sometimes produce a different result to

  result = total + (step * 12)

  or

  result = total + step * 12

• Sequential instructions in the wrong order.
  i.e.

   

  count = count + 1
  PRINT count

  VS

  PRINT count
  count = count + 1

• The wrong conditions being set in iteration loops so the loop does not repeat the expected number of times.
• The wrong conditions being set in conditional instructions.
  i.e.

   

  IF A > 100

  rather than

  IF A >= 100

  or using ELSE instead of ELSEIF.

• Referring to variables that have not been set correctly.
  i.e.

   

  count = count + step

  when step has not previously been assigned a value.

• Type mismatch errors.
  i.e. trying to add number variables but having the program treat them as text:

   

  num1 = 9, num2 = 9, PRINT num1 + num2

  gives 99 rather than 18.

---

What is test data?

Testing is carried out once syntax and logic errors have been corrected and a program appears to be running correctly. Testing should cover as many types of input/processing/output data and program operation as possible. It is important to try and ‘break’ the program, not simply to show that it works is the right values are input.

Typical test input data can include:

• Normal data – data that should be accepted
• Borderline/extreme data – data that tests the limit of what should be accepted, i.e. just above, just below and actually on the limit
• Erroneous data – data that should be rejected

A test plan is made up of the test data, the justification for the test and the expected test outcome. A test plan is often in the form of a table such as the example below:

Test data	Validation rule being tested	Test type	Expected result	Actual result
Mrs	Title field member list check (Mr, Mrs, Miss, Ms, Dr)	Normal	Data accepted
Mz	Title field member list check (Mr, Mrs, Miss, Ms, Dr)	Erroneous	Data rejected with a suitable error message
10	Title field member list check (Mr, Mrs, Miss, Ms, Dr)	Erroneous	Data rejected with error message
10	Staff discount range check (between 1 and 20)	Normal	Data accepted
-2	Staff discount range check (between 1 and 20)	Erroneous	Data rejected with a suitable error message
20	Staff discount range check (between 1 and 20)	Borderline	Data accepted
20.01	Staff discount range check (between 0 and 20)	Borderline	Data rejected with a suitable error message
19.99	Staff discount range check (between 1 and 20)	Borderline	Data accepted
Ten	Staff discount data type check (between 1 and 20)	Erroneous	Data rejected with a suitable error message

Further Readings:

• Syntax (programming languages)
• Logic error