1 #
   2 # This file and its contents are supplied under the terms of the
   3 # Common Development and Distribution License ("CDDL"), version 1.0.
   4 # You may only use this file in accordance with the terms of version
   5 # 1.0 of the CDDL.
   6 #
   7 # A full copy of the text of the CDDL should have accompanied this
   8 # source.  A copy of the CDDL is also available via the Internet at
   9 # http://www.illumos.org/license/CDDL.
  10 #
  11 
  12 #
  13 # Copyright (c) 2013 by Delphix. All rights reserved.
  14 #
  15 
  16 illumos Testing README
  17 
  18 1. A Brief History of usr/src/test
  19 2. How to Run These Tests
  20 3. How to Develop New Tests
  21 4. Porting Tests from Other Frameworks
  22 
  23 --------------------------------------------------------------------------------
  24 
  25 1. A Brief History of usr/src/test
  26 
  27 The tests here come in two varieties - tests written from scratch, and tests
  28 that have been ported from the Solaris Test Collection. Not all of the STC
  29 tests have been ported, and a forked repository of those that have been made
  30 publicly available may currently be found here:
  31 
  32         https://bitbucket.org/illumos/illumos-stc/
  33 
  34 Regardless of origin, all of these tests are executed using the run(1) script
  35 described in the next section.
  36 
  37 2. How to Run These Tests
  38 
  39 Currently, all the test suites under usr/src/test provide a wrapper script
  40 around run(1). These wrappers allow environment variables to be set up which
  41 may (for example) allow the script to specify which disks may be used by a test
  42 suite, and which must be preserved. Additionally, the wrappers allow options to
  43 run(1) to be passed through so that a user may specify a custom configuration
  44 file for a test suite. For specifics on the options available in the framework
  45 itself, please see the run(1) manpage.
  46 
  47 3. How to Develop New Tests
  48 
  49 New tests should mimic the directory layout of existing tests to the degree
  50 possible. This includes the following directories:
  51 
  52 cmd - Any support binaries or scripts used by the tests in this package.
  53 doc - READMEs or other support documentation to be delivered with the package.
  54 runfiles -  Configuration files that dictate how the tests are run.
  55 tests - The tests themselves (see below).
  56 
  57 The tests you create will be run, and given a PASS or FAIL status in accordance
  58 with the exit value returned by the test. A test may also be marked SKIPPED in
  59 the event that a prerequisite test is marked FAIL, or marked KILLED in the
  60 event the test times out. Note that there is no way to force a test to be
  61 marked SKIPPED; this is intentional. If a test must be skipped due to
  62 insufficient resources for example, then a wrapper script should be provided
  63 that chooses or creates an appropriate configuration file. The goal of every
  64 run is that every test is marked PASS.
  65 
  66 4. Porting Tests from Other Frameworks
  67 
  68 STF (Solaris Test Framework)
  69 
  70 Porting tests from this framework is relatively straightforward. For the most
  71 part the tests can be arranged in the new directory structure and added to the
  72 configuration file. The template for a configuration file can easily be created
  73 using the -w option to run(1). There are a few other changes that may be
  74 required:
  75 
  76 Some STF tests consume the values of variables from the user's environment.
  77 These variables must be set before beginning the test run either manually, or
  78 via a wrapper script.
  79 
  80 Groups of tests in STF can automatically source .cfg files specified in the
  81 'stf_description' file that exists in an STF directory. Ported tests that
  82 require any of these variables must manually source the .cfg file instead.
  83 
  84 The configuration file of a newly ported test must specify the user the test
  85 should run as, along with the timeout value. In STF, these are also specified
  86 in the 'stf_description' file.