To automate certification tests, additional setup is required. All installation and configuration steps below occur on the Windows machine running the GRL Test Harness software. Download and install the latest [GRL Thread Test Harness](http://graniteriverlabs.com/thread/) before continuing. Once installed, there are two automation methods for the device under test (DUT): 1. [OpenThread Harness Automation Tool](https://openthread.io/certification/automation-setup#openthread_harness_automation_tool) 2. [Test Harness Auto DUT mode](https://openthread.io/certification/automation-setup#test_harness_auto_dut_mode) ## OpenThread Harness Automation Tool ### Install 1. Install the following software: - Python 2.7 - Git for Windows (Bash support is highly recommended) 2. Clone the OpenThread repository to get the automation scripts: git clone https://github.com/openthread/openthread 3. Install Python requirements: cd tools/harness-automation pip install -r requirements.txt 4. Install both [Google Chrome](https://www.google.com/chrome/) and [ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver/). ### Configure 1. Create a configuration file for the automation tool: cp autothreadharness/settings_sample.py autothreadharness/settings.py 2. Update that new configuration file according to the instructions in the file. 3. Add the following parameter to the Test Harness configuration file, found at `C:\GRL\Thread1.1\Config\Configuration.ini`: ``` BrowserAutoNavigate = False ``` ### Mixed testbed support The Harness Automation Tool also supports mixed testbeds, where the reference devices used in each test case are a mixed set of devices rather than all the same device. For example, instead of using 32 TI CC2538 boards running OpenThread for all test cases, you could mix and match between all four Thread reference devices for each test case as desired. To use mixed testbeds, additional configuration is necessary: 1. Add the following additional parameter to the Test Harness configuration file, found at `C:\GRL\Thread1.1\Config\Configuration.ini`: ``` EnableDeviceSelection = True ``` 2. [Download the topology configuration file from the Thread Group](https://threadgroup.atlassian.net/wiki/spaces/THREADTEST/overview?mode=global). Add this file as `TopologyConfig.txt` to `C:\GRL\Thread1.1\Thread_Harness\TestScripts`. This file details which reference device to use for each role in each test case. The topology configuration file provided by the Thread Group can be modified for other mixed bed configurations, but for official certification the original configuration must be used. Valid values to use for reference devices in the topology configuration file are: | Value in topology configuration file | Test Harness reference device | |---|---| | `ARM` | ARM: NXP FRDM-K64F with Firefly 6LoWPAN Shield | | `NXP` | NXP(Freescale): USB-KW24D512 Dongles | | `OpenThread` | OpenThread: TI CC2538DK | | `SiLabs` | Silabs: EM358x USB Stick | Each test case entry in the topology configuration file must be on a separate line and formatted in `role:device` pairs as follows: ``` 5.3.10-Leader:NXP,Router_1:OpenThread,BorderRouter:OpenThread,MED_1:ARM ``` > [!NOTE] > **Note:** To use a mixed testbed with the Harness Automation Tool, all Thread Certification Test Cases must be present in the topology configuration file and must list all roles required for each test case. ## Test Harness Auto DUT mode The GRL Test Harness software provides an advanced auto discovery feature (Auto DUT) which accelerates the validation process for Thread reference stack vendors. OpenThread provides an example [Thread Host Controller Interface (THCI)](https://github.com/openthread/openthread/blob/main/tools/harness-thci) that allows the Test Harness to control your component or product as if it were one of the reference platforms. According to the serial port characteristics of your specific platform, your DUT could either: - [act as the existing OpenThread reference platform](https://openthread.io/certification/automation-setup#acting_as_the_existing_openthread_reference_platform) or - [act as a new reference platform](https://openthread.io/certification/automation-setup#acting_as_a_new_reference_platform) ### Acting as the existing OpenThread reference platform If the serial port of the DUT functions under settings of 115200 8-N-1, and the device responds correctly after every physical device reset (for example, to confirm the device's valid connection to the Windows machine), then the platform can use the OpenThread THCI tool. This tool allows the device to act as the OpenThread reference platform during testing. > [!NOTE] > **Note:** Refer to the vendor's device-specific documentation on how to confirm a valid connection setup after device reset. 1. Close the GRL Test Harness, if it is open. 2. Attach the device to the Windows machine. 3. Find the device serial port hardware identifier, using the Python pySerial module: 1. [Install pySerial](https://pythonhosted.org/pyserial/pyserial.html) on the Windows machine. 2. Use Python on the Windows command line to enumerate all hardware identifiers (VID and PID) for devices connected to the machine. In this output, one device is connected with an identifier of VID=1366 and PID=1015: python -m serial.tools.list_ports -v COM10 desc: USB Serial Port (COM10) hwid: USB\VID_1366+PID_1015+MI_00 3. If you know the COM port already, you can use a different command. For example, if the COM port is `COM10`: python -c "import serial.tools.list_ports;print [ports[2] for ports in serial.tools.list_ports.comports() if ports[0] == 'COM10']" ['USB\VID_1366+PID_1015+MI_00'] 4. Update the Golden Device list: 1. Open `C:\GRL\Thread1.1\Config\Configuration.ini`. 2. Update the OpenThread group in the `GoldenDeviceHardwareIds` array with the VID and PID of the device(s): ``` GoldenDeviceHardwareIds = { 'NXP': ['VID:PID=1FC9:0300','VID:PID=15A2:0300','VID:PID=1366:1015'], 'SiLabs': 'VID:PID=10C4:8A5E', 'ARM': 'VID:PID=0D28:0204', 'OpenThread':['VID:PID=10C4:EA60','VID:PID=1366:1015']} ``` 5. [Enable Auto DUT mode](https://openthread.io/certification/automation-setup#enable_auto_dut_mode). ### Acting as a new reference platform If the serial port of the DUT does not function under the proper serial port settings, or the device does not respond correctly after a physical device reset (outputs unreadable code within 3-6 seconds of reset), then OpenThread THCI can be customized to treat the device as a new platform in the Test Harness. For example, to customize THCI using the [Nordic Semiconductor nRF52840](https://openthread.io/platforms/nrf52840) platform: 1. Close the GRL Test Harness, if it is open. 2. Modify [`/tools/harness-thci/OpenThread.py`](https://github.com/openthread/openthread/blob/main/tools/harness-thci/OpenThread.py) based on the UART characteristics of the target device. Modifications may differ between target devices. In the case of the Nordic nRF52840 platform: 1. Rename the `OpenThread.py` file to `nRF52840.py`. 2. Change the first three occurrences of "OpenThread" to "nRF52840": >> Device : nRF52840 THCI >> Class : nRF52840 class nRF52840(IThci): 3. Change the serial port parameters: ``` def _connect(self): print 'My port is %s' % self.port if self.port.startswith('COM'): self.handle = serial.Serial(self.port, 115200, timeout=0) time.sleep(1) self.handle.write('\r\n') time.sleep(0.1) self._is_net = False elif ':' in self.port: host, port = self.port.split(':') self.handle = socket.create_connection((host, port)) self.handle.setblocking(0) self._is_net = True else: raise Exception('Unknown port schema') self.UIStatusMsg = self.getVersionNumber() ``` 4. Change the sleep time to eliminate the impact of unreadable code output after device reset: ``` def powerDown(self): """power down the Thread device""" print '%s call powerDown' % self.port self._sendline('reset') self.isPowerDown = True time.sleep(8) # New addition def reboot(self): """reset and rejoin to Thread Network without any timeout Returns: True: successful to reset and rejoin the Thread Network False: fail to reset and rejoin the Thread Network """ print '%s call reboot' % self.port try: self._sendline('reset') self.isPowerDown = True time.sleep(8) # Updated from 3 to 8 def reset(self): """factory reset""" print '%s call reset' % self.port try: self._sendline('factoryreset') time.sleep(8) # Updated from 3 to 8 self._read() def resetAndRejoin(self, timeout): """reset and join back Thread Network with a given timeout delay Args: timeout: a timeout interval before rejoin Thread Network Returns: True: successful to reset and rejoin Thread Network False: fail to reset and rejoin the Thread Network """ print '%s call resetAndRejoin' % self.port print timeout try: self._sendline('reset') self.isPowerDown = True time.sleep(timeout) if timeout < 8: # Sleep a bit longer if the timeout is short time.sleep(8 - timeout) ``` 3. Copy the modified `nRF52840.py` file to `C:\GRL\Thread1.1\Thread_Harness\THCI` 4. Add the new platform information to the Test Harness: 1. Create an icon (in png or jpg format) to make it easier to differentiate your platform and copy it to `C:\GRL\Thread1.1\Web\images`. 2. Update `C:\GRL\Thread1.1\Web\data\deviceInputFields.xml` with a new `DEVICE` section, where the `thumbnail` parameter is the icon file: ot;text&quo < >forParam="SerialBaudRate" validation="baud-rate" hint="eg: 115200"115200 /ITEM /DEVICE 5. Attach the device to the Windows machine. 6. Find the device serial port hardware identifier, using the Python pySerial module: 1. [Install pySerial](https://pythonhosted.org/pyserial/pyserial.html) on the Windows machine. 2. Use Python on the Windows command line to enumerate all hardware identifiers (VID and PID) for devices connected to the machine. In this output, one device is connected with an identifier of VID=1366 and PID=1015: python -m serial.tools.list_ports -v COM10 desc: USB Serial Port (COM10) hwid: USB\VID_1366+PID_1015+MI_00 3. If you know the COM port already, you can use a different command. For example, if the COM port is `COM10`: python -c "import serial.tools.list_ports;print [ports[2] for ports in serial.tools.list_ports.comports() if ports[0] == 'COM10']" ['USB\VID_1366+PID_1015+MI_00'] 7. Update the Golden Device list: 1. Open `C:\GRL\Thread1.1\Config\Configuration.ini`. 2. Add a new platform group in the `GoldenDeviceHardwareIds` array with the VID and PID of the device: ``` GoldenDeviceHardwareIds = { 'NXP': ['VID:PID=1FC9:0300','VID:PID=15A2:0300','VID:PID=1366:1015'], 'SiLabs': 'VID:PID=10C4:8A5E', 'ARM': 'VID:PID=0D28:0204', 'OpenThread':'VID:PID=10C4:EA60', 'nRF52840': 'VID:PID=1366:1015'} ``` 8. [Enable Auto DUT mode](https://openthread.io/certification/automation-setup#enable_auto_dut_mode). ### Enable Auto DUT mode After one of the two above configuration options are complete: 1. Open the GRL Test Harness, the device/port now appears as a new reference device. 2. Select the **Enable Auto DUT Device Selection** checkbox below the Supported Hardware list. 3. Select the **Set as DUT** radio button under the target device/port to set the device as the DUT. [![OT Certification Auto DUT](https://openthread.io/static/certification/images/ot-cert-harness-auto-dut.png)](https://openthread.io/static/certification/images/ot-cert-harness-auto-dut_2x.png)