SAHPA Computer Based Test  Specifications

  prepared by Ulf Arndt, ulfa at, November 2002


  to replace the handwritten test approach
  less admin in marking the tests
  ensure equal level of marking
  ensure traceability, audit trace, authentication
  meet the quality control requirements expected by CAA and AeCSA

 Way of Operation

  Pilot applies to SAHPA for a test.

   SAHPA would arrange for a test session to take place at a certain venue under
   the supervision of a SAHPA representative in the area of the test applicant.

  SAHPA office/test coordinator will prepare an individual test package and
   email it to the SAHPA representative for the area

  Local SAHPA representative would arrange to have one PC or
    in the case of multiple applicants multiple PC's
    loaded with the code to run the test at a venue.

   The code will be available from SAHPA for free.
   The code is packaged either inside of each test or will be
   available from the SAHPA website for download.

  SAHPA will email the SAHPA representative
  a unique test question package file for each applicant.

  The test package will be a self extracting zip file containing
  -    information for who this package is
     The file would contain the name and id info for the pilot.
  -  And a set of random picked test questions from a range of topics determined
       by the level of rating that the pilot applies for.
   -  And gif picture files related to the questions.
   - and optional the code to run the test

 Each test for a person would be in a separate directory.
  directory name would relate to the person

  SAHPA representative would unzip the package and startup the test code
   and point the code to use the specific directory containing the test
   for a certain person.
  The code will test if this is the first attempt, only one attempt allowed.

  Pilot would arrive at the venue, could be at a school, a club meeting,
   a competition,..

 Still to solve is the problem of authentication make sure the correct person took the test..
  Pilot will have to bring along 2? methods of ID, eg drivers licence and ID book,
   or passport or company badge with a picure. SAHPA will need some paper proof
   to show that this was really the person who did the test.
  Copies of the id material along with a printed out score to be  signed by SAHPA rep
   and sent back to SAHPA office.

  Once arrived and seated in front of a PC the person will then
   read the explanantion how the test works and then start the test.
   A timer will start.
    The test will have a certain time to answer all the questions.
   Each question has a certain time allocated.
    Lenght of the test is determined by the sum of times for all the questions.

  The test is in multiple choice, with a various amount ( up to 9 or 10 ) answers.
   Multiple answers can be correct.
   Each answer can have a positive, zero, or negative score.

   Expect around 100 - 200 questions. In case the questions and answers are public,
    the test has to be tight since one can expect to person to know most of the answers.
  July 2002 SAHPA committee opted not to disclose any questions , answers and scores .

   Options are to keep the test restricted  and point the test takers to a curiculum
   or have the questions open  to the public and a curiculum
   or have the questions and answers without any indication of the scores
   or have everything out and make the test very time constraint and tight

 Once the person reckons that they answered all the questions to the best of
  their knowledge, or reached the maximum time limit,
  the test gets scored.

  A certain percentage of the total score is required to pass.

    SAHPA will need back for audit purposes the results of the test, along
   with the questions and answers given. The test program creates a results file.
   This results file is then emailed back to he SAHPA office.

  Along with some statistic info of what questions got answered correct, which wrong.

  Results will be encrypted to avoid being tampered with before getting sent back to SAHPA.

  A printer would be required to give the person a printout of the score.
  As proof that he did the test, in case there is some system problems and the results
  never make it back to SAHPA.
  Also the result sheet will provide some feedback for the pilot where he failed and where he scored.

 System specs

 Maximum of 10 answers per question.

Unlimited amount of questions in a xml file which is used for creating a test for a pilot.

  Maximum of 250 questions in a test stf file to be processed by the TakeTest VB module.
   This is a VB restriction, which only allows maximum 64k for an object.
  Expected test size will be around 120 questions.

 The Java web based version uses XML DOM elements for tests  and has no limit on the amount of questions.
 To be compatible with the VB code version and share the same XML files, the limits from the VB side shold be adhered to.


 Function Specs for Creating a Test Software Component



 Function Specs for Taking a Test Software Component