<br />

Before running test cases, make sure all test devices (Golden and DUT) are
attached to the Windows machine with the installed GRL Test Harness and Harness
Automation Tool.

> [!WARNING]
> **Warning:** For all manual test cases, you MUST reset the DUT manually with the `factoryreset` CLI command. This ensures all network information stored from the prior test case is erased.

## Using the Harness Automation Tool

All test cases are run from the `tools/harness-automation` directory. There are
two methods:

- `start.sh` --- Using Bash, which is bundled with Git for Windows
- `start.bat` --- From the Windows command line (cmd)

To run all test cases:
Bash

    ./start.sh

Windows cmd

    ./start.bat

To run individual test cases (names are case-sensitive):
Bash

    ./start.sh Router_5_1_1 Leader_5_1_1

Windows cmd

    ./start.bat Router_5_1_1 Leader_5_1_1

## Using the GRL Test Harness

After the test bed is configured, batches of test cases are run based on the
selected role of the DUT. The **Test Selection** section lists all available
test cases for the selected role under **Test Plan**:
[![OT Certification Test Harness](https://openthread.io/static/certification/images/ot-cert-harness-tests.png)](https://openthread.io/static/certification/images/ot-cert-harness-tests_2x.png)

Select test cases as desired, then click the green play button to execute.

> [!NOTE]
> **Note:** For detailed instructions on how to run test cases in the Test Harness, see the Test Harness Quick Start guide included with the software.

## Manual test operations

The Test Harness prompts the user for manual input during some test cases.

### Network credentials

For test cases in Sections 5, 6, 7, and 9 of the Test Plan, network credentials
must be configured out-of-band. For example:

Parameter configuration during device bring-up:

    panid 0xface
    Done
    channel 20
    Done
    networkname GRL
    Done
    extpanid 000db80000000000
    Done

> [!NOTE]
> **Note:** The Mesh-Local Prefix is automatically configured as `fd00:0db8::/64` once the `extpanid` is configured.

"DUT as Leader" configuration, prior to bringing up the interface:

    dataset activetimestamp 1
    Done
    dataset commit active
    Done

"DUT as End Device" configuration:

    mode rsn
    Done

"DUT as SED" configuration, where the `pollperiod` (in milliseconds) is set to
the default `SedPollingRate` in the Test Harness:

    mode s
    Done
    pollperiod 5000
    Done

Bring up the Thread interface and stack:

    ifconfig up
    Done
    thread start
    Done

### Commissioner role setup

For test cases in Section 8 of the Test Plan, configure the Commissioner role.

Start the commissioner after bringing up the Thread stack:

    commissioner start
    Done

Add steering data:

    commissioner joiner add eui64 THREADJPAKETEST 200

The `eui64` is the factory-assigned IEEE EUI-64 identifier, obtained via the
`eui64` CLI command. `THREADJPAKETEST` is the Joiner Credential, which
is used to generate the Pre-Shared Key for the Device (PSKd). The Joiner is
removed from steering data after the configured timeout.

<br />

> [!NOTE]
> **Note:** The Joiner Credential is a device-specific string of all uppercase alphanumeric characters (0-9 and A-Y, excluding I, O, Q and Z for readability), with a length between 6 and 32 characters.

<br />

We recommend using `500` as the timeout value during the 8.2.x Commissioner
test cases.

Start the Joiner role after bringing up the Thread interface, using the Joiner
Credential default of `THREADJPAKETEST`:

    joiner start THREADJPAKETEST

Start the Joiner role after bringing up the Thread interface, using the specified provisioning URL:

    joiner start THREADJPAKETEST www.threadgroup.org

### Other input

The Test Harness may prompt for other manual operations, and provide values
(such as an IPv6 address) for use in those operations. Use the following
OpenThread CLI commands if prompted for:

| Prompt | CLI command |
|---|---|
| Extended address | `extaddr` |
| [IPv6 addresses](https://openthread.io/guides/thread-primer/ipv6-addressing#unicast_address_types) | `ipaddr` |
| Joiner factory MAC address | `eui64` |
| Set the Commissioner provisioning URL | `commissioner provisioningurl www.threadgroup.org` |
| Factory reset | `factoryreset` |
| Get all TLVs | `dataset mgmtgetcommand active address ipv6-address` |
| Get Network ChannelMask (0x35), MeshLocal ULA prefix (0x07), Network Name (0x03) | `dataset mgmtgetcommand active address ipv6-address binary 350703` |
| Get Network Channel (0x00), MeshLocal ULA prefix (0x07), Network Name (0x03), Scan Duration (0x38), Energy List (0x39) | `dataset mgmtgetcommand active address ipv6-address binary 0007033839` |

For a list of all CLI commands, see the
[OpenThread CLI Reference](https://github.com/openthread/openthread/tree/master/src/cli/README.md).

## Certification tips

### Enable device selection for the mixed test bed

> [!NOTE]
> **Note:** This feature requires you have the requisite hardware physically on hand. See the Thread Certification Test Plan for more information.

During Thread Certification testing, mixed test beds featuring reference stacks
from ARM, NXP, Silicon Labs, and OpenThread are used. Test bed topologies are
defined in the `C:\GRL\Thread1.1\Thread_Harness\TestScripts\TopologyConfig.txt`
file.

To enable device selection in order to perform a test case with the same exact
reference platforms as the Thread Group's ATL would, set the
`EnableDeviceSelection` parameter in the
`C:\GRL\Thread1.1\Config\Configuration.ini` file to `True`.

```
EnableDeviceSelection = True
```

### RF shielding

> [!NOTE]
> **Note:** These tips are for manual certification only and are not needed if you are automating certification tests.

For test cases 9.2.9 (Leader, Router) and 9.2.10 (Router, ED1, SED1), special
care is needed to ensure the correct devices are RF-shielded. Assuming all
devices are configured in the roles as each test requires:

1. Start the Test Harness with only the NXP Sniffer attached to the Windows machine. This avoids device auto discovery by the Test Harness.
2. Plug in the test devices one by one and note their corresponding serial port.
3. Place one of the following groups of devices in the RF shield box:
   - For test 9.2.9:
     - Leader and Commissioner
     - Router1, Router2, and Sniffer
   - For test 9.2.10:
     - Leader and Commissioner
     - Router, ED, SED, and Sniffer
4. On the **Configure Test Bed** page of the Test Harness, configure each device for the following sort order in the **Test Bed** section, depending on the test case:

   | Final device sort order for RF shielding test cases ||
   |---|---|
   | **9.2.9** | 1. Router2 2. Commissioner 3. Router1 4. Leader |
   | **9.2.10** | 1. SED 2. ED 3. Router1 4. Commissioner 5. Leader |

   1. Clear the **Test Bed** section of any existing devices.
   2. Drag the appropriate devices for the specific test case topology from the **Supported Hardware** section on the left to the **Test Bed** section on the right. Make sure to drag them in the *reverse order* of the final sort order. For example, for test case 9.2.9, drag the Leader first, then Router1 second, and so on. If you drag the wrong device, clear the devices in the **Test Bed** section and start again.
   3. In the **Test Bed** section, fill in the serial number or IP information for each device in the final sort order. For example, for test case 9.2.9, after dragging, the first device is Router2, the second is Commissioner, and so on.

      > [!CAUTION]
      > The final sort order is important, as it's based on the role assignment sequence in each test case. The test case will fail if the devices are not configured in the **Test Bed** section in the correct order.

   4. Select the **Enable Auto DUT Device Selection** checkbox below the **Supported Hardware** list.
   5. Select the **Set as DUT** radio button for the DUT.
   6. Select **Connect All** to initiate device discovery.
5. Execute the test case (9.2.9 or 9.2.10). Shield and unshield each group of devices as prompted by the Test Harness.