Testing OpenAF: how to add and run tests

For testing your own scripts and workflows without the source checkout, start with Test your automation. Documentation maintainers can run the example and coverage checks.

1. Purpose

This document explains how to create JavaScript test files and orchestrator YAML configurations to test OpenAF functions and modules, and how to run them locally and in CI. It is aimed at contributors who want to add new tests or understand how the existing test suite works. After reading this guide you will be able to write a focused test, hook it into the orchestrator, and verify results locally before opening a pull request.

2. Where tests live

All test artefacts live under the tests/ directory:

Path Description
tests/autoTestAll.yaml Main test orchestrator — entry point for ojob
tests/autoTestAll.allJobs.yaml Includes all area-specific YAML orchestrators
tests/autoTestAll.<Area>.yaml Area-specific orchestrator (one per functional area)
tests/autoTestAll.<Area>.js Area-specific JS test implementations
tests/autoTestAll.results.json Written by the orchestrator after a run; contains pass/fail counts
tests/autoTestAllResults.xml JUnit XML output (also generated by the orchestrator)
tests/autoTestAll.md Human-readable Markdown test report (generated)
tests/*.svg Badge images generated from test results

The shell helper oafTest.sh at the repository root installs the built JAR into _oaf/, runs the full suite, and exits non-zero if any test fails. CI invokes this script via Maven (see CI and Maven integration).

3. Test framework basics

OpenAF ships a built-in test utility at ow.test. Load it with ow.loadTest() at the top of any test file.

Core API 

ow.loadTest();

// Assert equality (throws on mismatch and marks the test as failed)
ow.test.assert(actual, expected, "Descriptive failure message");

// Control output
ow.test.setOutput(true);            // print pass/fail per assertion (default: true)
ow.test.setShowStackTrace(true);    // include Java stack traces on failure

// Profiling
ow.test.profile("operationName", function() {
    someExpensiveOperation();
});
var avg = ow.test.getProfileAvg("operationName");  // average time in ms
var max = ow.test.getProfileMax("operationName");  // max time in ms

// Counters (read after tests complete)
ow.test.getCountTest();     // total tests registered
ow.test.getCountPass();     // tests that passed
ow.test.getCountFail();     // tests that failed
ow.test.getCountAssert();   // total assert() calls executed
ow.test.reset();            // reset all counters

See also: docs/openaf.md — search for “Testing Framework” for the full API reference.

Minimal test file example 

// Copyright 2024 Your Name

(function() {
    exports.testMyModuleBasic = function() {
        // ow.test is already loaded by the framework — call assert directly
        ow.loadFormat();   // load any OpenAF module you need for the test

        var result = ow.format.toHumanSize(1024);
        ow.test.assert(result, "1 KB", "toHumanSize should format 1024 bytes as 1 KB");
    };

    exports.testMyModuleEdgeCase = function() {
        ow.loadFormat();
        var result = ow.format.toHumanSize(0);
        ow.test.assert(result, "0 B", "toHumanSize should return 0 B for zero input");
    };
})();

Key points:

  • Wrap everything in a self-executing (function() { ... })() so variables do not leak into the global scope.
  • Export each test as exports.testXxx = function() { ... }.
  • Each export function must be independent (no shared mutable state between tests).
  • ow.loadTest() is called once by the framework (autoTestAll.js) before any tests run — do not repeat it inside individual test functions.
  • Load any OpenAF modules your test needs (e.g. ow.loadFormat(), ow.loadObj(), plugin("ZIP")) at the top of the test function itself.

4. Naming conventions and structure

File names

Convention Example Use
autoTestAll.<Area>.js autoTestAll.ZIP.js Grouped tests for a functional area
autoTestAll.<Area>.yaml autoTestAll.ZIP.yaml Orchestrator for the same area

Smaller per-feature files are acceptable as long as the YAML orchestrator is updated to include them (see step-by-step guide).

Export names

Export functions must start with test followed by a descriptive name in CamelCase:

exports.testZIP                 ✓
exports.testZIPStream           ✓
exports.testMyModuleBasic       ✓
exports.myHelperFunction        ✗  (not discovered as a test)

How tests are discovered

The orchestrator YAML for each area explicitly lists every test function to run. There is no automatic discovery by naming convention — you must add an entry to the YAML for each new export. See the orchestrator YAML section and the step-by-step guide for details.

Reference examples in the repo:

5. Orchestrator YAML example

Each functional area needs a paired YAML file. The structure mirrors tests/autoTestAll.ZIP.yaml:

# Copyright 2024 Your Name

include:
   - oJobTest.yaml      # provides the "oJob Test" job template

jobs:
   # Initialise: load the JS test module once
   - name: MyArea::Init
     exec: |
       args.tests = require("autoTestAll.MyArea.js");

   # One job per exported test function
   - name: MyArea::Basic functionality
     from: MyArea::Init
     to  : oJob Test           # wraps the function in pass/fail reporting
     exec: args.func = args.tests.testMyModuleBasic;

   - name: MyArea::Edge case
     from: MyArea::Init
     to  : oJob Test
     exec: args.func = args.tests.testMyModuleEdgeCase;

todo:
   - MyArea::Init
   - MyArea::Basic functionality
   - MyArea::Edge case

Then register the new file in tests/autoTestAll.allJobs.yaml:

include:
- oJobTest.yaml
# ... existing entries ...
- autoTestAll.MyArea.yaml    # ← add this line

How the main orchestrator works

tests/autoTestAll.yaml:

  1. Runs all jobs defined in autoTestAll.allJobs.yaml (which includes every area YAML).
  2. After all tests complete, the Results job calls ow.test.getCountAssert() and writes autoTestAll.results.json.
  3. The JUnit results job writes autoTestAllResults.xml and regenerates autoTestAll.md (the Markdown report).

Run the full suite with:

cd tests
ojob autoTestAll.yaml
# or, if ojob is not on PATH:
java -jar ../openaf.jar --ojob -e autoTestAll.yaml

6. Run tests locally

Prerequisites

  • Java JDK ≥ 24 for building (JRE 21+ for running tests against an already-built JAR).
  • A built openaf.jar at the repository root (see BUILD.md).

Build first 

# Using a locally installed OpenAF:
ojob build.yaml

# Using the bootstrap installation in _oaf/:
_oaf/ojob build.yaml

Run the full test suite 

cd tests

# Option A — if ojob is on PATH (system OpenAF):
ojob autoTestAll.yaml

# Option B — using the JAR you just built:
java -jar ../openaf.jar --ojob -e autoTestAll.yaml

After the run, inspect the results:

cat autoTestAll.results.json
# {"pass":194,"fail":0,"count":194,"asserts":615}

Run focused object-pool tests

For a focused object-pool check against the built JAR, run from the repository root:

java -jar openaf.jar -c 'ow.loadTest(); var tests = require("tests/autoTestAll.Obj.js"); Object.keys(tests).filter(function(name) { return name.indexOf("testObjPool") == 0; }).forEach(function(name) { tests[name](); print("PASS " + name); });'

These tests cover resource reuse, maximum capacity, bounded batch growth, factory failures, and stopping pools with or without a close callback. They also run through the main orchestrator.

Using oafTest.sh (mirrors CI exactly)

From the repository root:

sh oafTest.sh

This script:

  1. Creates a fresh _oaf/ directory and installs openaf.jar there.
  2. Changes to tests/ and runs ../_oaf/ojob autoTestAll.yaml.
  3. Reads autoTestAll.results.json and exits with code 1 if fail > 0.

Run a single area file

To iterate quickly on one area, pass only that area’s YAML:

cd tests
java -jar ../openaf.jar --ojob -e autoTestAll.ZIP.yaml

7. CI and Maven integration

pom.xml contains an exec-maven-plugin execution that runs oafTest.sh automatically during the Maven test phase:

<execution>
  <id>oaf-test</id>
  <phase>test</phase>
  <goals><goal>exec</goal></goals>
  <configuration>
    <executable>sh</executable>
    <workingDirectory>.</workingDirectory>
    <arguments>
      <argument>oafTest.sh</argument>
    </arguments>
  </configuration>
</execution>

This means mvn test (or any CI pipeline that runs the Maven lifecycle) will automatically build the JAR, install it, and run the full OpenAF test suite. CI integrations that do not use Maven can invoke the same script directly:

sh oafTest.sh

The script exits non-zero on test failures, which causes the CI pipeline to fail.

8. How to add a focused test (step-by-step)

Step 1 — Identify the area and create the JS file

Decide which functional area your test belongs to (e.g., ZIP, IO, Format). Either add your test functions to the existing tests/autoTestAll.<Area>.js file, or create a new pair:

# Example: adding tests for a new "Cache" area
touch tests/autoTestAll.Cache.js
touch tests/autoTestAll.Cache.yaml

Step 2 — Write the test function

Edit tests/autoTestAll.Cache.js:

// Copyright 2024 Your Name

(function() {
    exports.testCacheSetGet = function() {
        ow.loadObj();   // load the module under test

        // Exercise the function under test
        var result = ow.obj.pool.AF(5).get();   // hypothetical API
        ow.test.assert(isDef(result), true, "Pool get should return a defined value");

        // Clean up any artefacts (files, sockets, etc.)
        // io.rm("some-temp-file.tmp");
    };
})();

Tips:

  • Use ow.test.assert(actual, expected, message) for every assertion.
  • Do not call ow.loadTest() — the framework already loaded it before running your tests.
  • Clean up temporary files with io.rm("path") at the end of the function.
  • Avoid hard-coded absolute paths; use relative paths inside tests/.
  • Avoid tests that require external network services unless absolutely necessary; mock or skip with a clear comment when network is unavailable.

Step 3 — Create the YAML orchestrator

Edit tests/autoTestAll.Cache.yaml:

# Copyright 2024 Your Name

include:
   - oJobTest.yaml

jobs:
   - name: Cache::Init
     exec: |
       args.tests = require("autoTestAll.Cache.js");

   - name: Cache::Set and get
     from: Cache::Init
     to  : oJob Test
     exec: args.func = args.tests.testCacheSetGet;

todo:
   - Cache::Init
   - Cache::Set and get

Step 4 — Register the new file

Add - autoTestAll.Cache.yaml to the include block in tests/autoTestAll.allJobs.yaml.

Step 5 — Run locally and review results 

cd tests
java -jar ../openaf.jar --ojob -e autoTestAll.Cache.yaml   # run your area only
java -jar ../openaf.jar --ojob -e autoTestAll.yaml          # run the full suite
cat autoTestAll.results.json

The orchestrator regenerates tests/autoTestAll.md (the Markdown report) and any badge SVG files automatically after each run. Commit the updated autoTestAll.md and *.svg files if their contents change.

9. Minimal test file template (copy-paste ready) 

// Copyright 2024 Your Name
// Tests for <module or function description>

(function() {

    exports.testMyFuncBasic = function() {
        // Load any OpenAF module you need
        // ow.loadFormat();

        // Arrange
        var input    = "hello";
        var expected = "HELLO";

        // Act
        var result = myModule.myFunc(input);

        // Assert
        ow.test.assert(result, expected, "myFunc should uppercase the input");
    };

    exports.testMyFuncNull = function() {
        var result = myModule.myFunc(null);
        ow.test.assert(result, null, "myFunc should return null for null input");
    };

    exports.testMyFuncWithCleanup = function() {
        var tmpFile = "autoTestAll.myFunc.tmp";
        try {
            myModule.myFuncWriteFile(tmpFile, "data");
            ow.test.assert(io.fileExists(tmpFile), true, "Output file should exist");
        } finally {
            io.rm(tmpFile);
        }
    };

})();

Note: ow.loadTest() is not needed here — it is called once by the orchestrator framework before your tests run.

Companion YAML (autoTestAll.MyModule.yaml):

# Copyright 2024 Your Name

include:
   - oJobTest.yaml

jobs:
   - name: MyModule::Init
     exec: |
       args.tests = require("autoTestAll.MyModule.js");

   - name: MyModule::Basic
     from: MyModule::Init
     to  : oJob Test
     exec: args.func = args.tests.testMyFuncBasic;

   - name: MyModule::Null input
     from: MyModule::Init
     to  : oJob Test
     exec: args.func = args.tests.testMyFuncNull;

   - name: MyModule::Cleanup
     from: MyModule::Init
     to  : oJob Test
     exec: args.func = args.tests.testMyFuncWithCleanup;

todo:
   - MyModule::Init
   - MyModule::Basic
   - MyModule::Null input
   - MyModule::Cleanup

10. Troubleshooting / common pitfalls

Tests do not appear in the results

  • Check that the YAML job entry uses to: oJob Test and sets args.func.
  • Verify the new YAML is listed in autoTestAll.allJobs.yaml.
  • Confirm the export name starts with test (case-sensitive).

A test fails with a permissions error

  • Ensure temporary files are written inside tests/ or /tmp/; the process may not have write access to other directories.
  • On Windows, file locking may cause teardown failures — use try / finally blocks for cleanup.

Tests that need network access

  • Guard with a connectivity check and skip gracefully when the network is unavailable:

    exports.testMyHTTP = function() {
    try {
        $rest().get("http://example.com");
    } catch(e) {
        log("Skipping network test: " + e.message);
        return;
    }
    // ... assertions ...
};

  • Set reasonable timeouts with $rest({ timeout: 5000 }).get(...).

Viewing detailed results 

# Pretty-print the JSON results
cat tests/autoTestAll.results.json

# Full Markdown report
cat tests/autoTestAll.md

# JUnit XML (for import into CI dashboards)
cat tests/autoTestAllResults.xml

Running a single test file for debugging 

cd tests
java -jar ../openaf.jar --ojob -e autoTestAll.MyArea.yaml

This runs only the jobs defined in that one YAML, making iteration fast without waiting for the entire suite.

The oafTest.sh script is not found

Run sh oafTest.sh from the repository root, not from inside tests/. The script expects openaf.jar and tests/ to be at the same level as itself.

Resource Description
tests/ All test JS files and YAML orchestrators
tests/autoTestAll.yaml Main test orchestrator
tests/autoTestAll.allJobs.yaml Master include list for all area YAMLs
tests/autoTestAll.md Generated test results report
tests/autoTestAll.ZIP.js Example area-specific JS test file
tests/autoTestAll.ZIP.yaml Example area-specific YAML orchestrator
oafTest.sh Shell helper used by CI and Maven
pom.xml Maven build — exec-maven-plugin runs oafTest.sh in the test phase
BUILD.md Full build instructions including test commands
AGENTS.md Testing guidelines for contributors (naming, commands)
docs/openaf.md ow.test API reference — search for “Testing Framework”
README.md Top-level docs with “Testing a build” section
This site uses Just the Docs, a documentation theme for Jekyll.