How To ~ Configure Two AC-MXNET-CBOX-HAs for High Availability Mode

How To: Configure Two AC-MXNET-CBOX-HAs for High Availability Mode

What is High Availability?

High Availability is a new feature introduced in AC-MXNET-CBOX-HA firmware V4.25 (download link here), or V5.02 for AC-MXNET-10G-CBOX-HA products, That provides the ability to link two AC-MXNET-CBOX-HAs together in a “Primary & Backup” configuration. One Control Box will act as the Primary device, overseeing all control (internal and external from third party controllers) and endpoint monitoring applications. The other Control Box acts as a Backup device, continuously pinging the Primary CBOX for its current status, while copying the Primary designated CBOX's configuration file every minute. If the Backup CBOX is unable to ping the Primary CBOX, or the Primary CBOX reports the inability to communicate across one of multiple factors, the Backup CBOX will overtake and act as the Primary CBOX.

The Primary and Backup CBOX utilize what's called a Virtual IP Address - or VIP for short. This Virtual IP Address is designated by the user and acts as the main communication point for outside devices, such as devices accessing Mentor, or a third party control system communicating to the API server on port 24. The Primary CBOX hosts all services on the Virtual IP address, and if a "Fail-Over State Event" (Backup CBOX overtakes and acts as the Primary CBOX due to a service interruption from the previous Primary CBOX) occurs, the Backup CBOX will assume control and host its services on the designated Virtual IP address, thus massively mitigating service interruptions at a job site. 

There are two methods High Availability can be integrated: Via the physical HA Port located on any AC-MXNET-CBOX-HA, or via IP (most common method). 
The physical HA port can be linked using a typical RJ-45 cable.
NOTICE: If using the HA RS-232 port, the HA MCU CBOX version must be V0.08 or higher, please see the CBOX-HA Firmware page for more information. 
Linking two CBOX-HAs via IP require what is called a judge device to be designated by the user. A judge device is a device that exists on the network both CBOX-HAs communicate on (such as a "control" network) and is always expected to be online, such as a router, network switch, etc. Both CBOX-HAs will continuously ping each other as well as the judge device IP address to ensure stable communication is occurring.

A fail-over event may occur from the following types of service interruptions:
  1. The Primary CBOX reports a service interruption on the AV Network interface.
  2. The Primary CBOX fails to ping the designated Judge device IP address.
  3. The HA port communication link is severed (if no judge device has been designated)
  4. The Backup CBOX is no longer able to communicate to the Primary CBOX in general, such as when the Primary CBOX suffers a power failure. 
Note: If a fail-over event occurs and the initial Primary CBOX resumes operation, it will check the Backup CBOX's status; if the Backup CBOX has assumed the Primary role, the initial Primary CBOX will switch its role to Backup to avoid communication interference.

Configuring Two AC-MXNET-CBOX-HAs for High Availability Mode

Start by installing both AC-MXNET-CBOX-HAs to the same Control(MENTOR) and AV network following standard installation practices, and log in to the web GUI for both CBOXes. 
Note: link the HA port to both CBOX-HAs using any RJ-45 cable during this step, if desired (HA Port link is not required for High Availability functionality).
On CBOX V4.24, a new section within the System Utilities (homepage of the CBOX) page has been added to the bottom right of the page:

Configuring the Primary CBOX

To begin configuring the CBOXes for High Availability mode, navigate to the desired Primary CBOX IP Address/System Utilities Page. 
Note: It's always recommended to configure the Primary CBOX first, before configuring the Backup CBOX, or the Backup CBOX may unexpectedly change its state to Primary during the configuration process.
Once at the Primary CBOX's System Utilities page, click on the Configure button within the High Availability section.

After clicking the Configure button a new pop-up window will appear, as shown below:

The current (and default) CBOX Mode is Alone, or standalone. Normal CBOX behavior occurs in this state. 
  1. Click on the dropdown menu and select Primary.

  2. Enter in the Control/MENTOR IP address of the Backup CBOX within the designated field. For this example, 192.168.1.78 will be used.

  3. Enter in the desired Virtual IP Address (VIP) within the designated field. For this example, 192.168.3.123 will be used.
    1. A ? is present that can be highlighted with a mouse for a brief description of its functionality.
    2. The Virtual IP Address is manually designated by the user, and can be any unused IP address on the same subnet as the CBOX Control/MENTOR IP address.
    3. Reminder: This is the IP Address that will be utilized to integrate third party control systems to the CBOX.

  4. Enter in the desired Judge IP Address within the designated field. In this example, 192.168.3.201 will be used, which is the MGMT port of an AC-MXNET-SW24.
    1. A ? is present that can be highlighted with a mouse for a brief description of its functionality.
    2. Reminder: The Judge IP Address should be a device that is always expected to be online on the network, such as a network switch, control processor, router, etc.
      1. Ensure that the Judge IP Address can handle continuous pings from the CBOX.

  5. Once all of the appropriate fields mentioned above are populated, and the CBOX mode is chosen, click SAVE.

  6. After clicking Save, toggle the Enable High Availability option to ON (green, shifted to the right).
    1. Choose this option last during configuration applications. 
    2. A ? is present that can be highlighted with a mouse for a brief description of its functionality.

  7. The Primary CBOX is now configured and the pop-up can be closed by clicking the X on the bottom right. The High Availability section will have changed, indicating configuration was successful.


    note: the status buttons showing red is expected behavior as the Backup CBOX is yet to be configured.

Configuring the Backup CBOX

Configuring the Backup CBOX is configured in an extremely similar fashion to configuring the Primary CBOX in the previous section. Start by navigating to the IP address of the desired Backup CBOX, and click on the Configure button within the High Availability section on the System Utilities page.
As with the Primary CBOX, the Backup CBOX will be initially set to the Alone (standalone) CBOX mode. Click on the dropdown menu and choose Backup.
The next steps are essentially identical to configuration of the Primary CBOX, except the IP address for the Primary CBOX will be entered (it is important that the Virtual IP and Judge IP Addresses are identical on the Primary & Backup CBOX, or High Availability won't function as expected).
  1. Enter in the Control/MENTOR IP address of the Primary CBOX within the designated field. For this example, 192.168.1.222 will be used.

  2. Enter in the desired Virtual IP Address (VIP) within the designated field. For this example, 192.168.3.123 will be used.
    1. A ? is present that can be highlighted with a mouse for a brief description of its functionality.
    2. The Virtual IP Address is manually designated by the user, and can be any unused IP address on the same subnet as the CBOX Control/MENTOR IP address.
    3. Reminder: This is the IP Address that will be utilized to integrate third party control systems to the CBOX.

  3. Enter in the desired Judge IP Address within the designated field. In this example, 192.168.3.201 will be used, which is the MGMT port of an AC-MXNET-SW24.
    1. A ? is present that can be highlighted with a mouse for a brief description of its functionality.
    2. Reminder: The Judge IP Address should be a device that is always expected to be online on the network, such as a network switch, control processor, router, etc.
      1. Ensure that the Judge IP Address can handle continuous pings from the CBOX.

  4. Once all of the appropriate fields mentioned above are populated, and the CBOX mode is chosen, click SAVE.

  5. After clicking Save, toggle the Enable High Availability option to ON (green, shifted to the right).
    1. Choose this option last during configuration applications. 
    2. A ? is present that can be highlighted with a mouse for a brief description of its functionality.

  6. The Backup CBOX is now configured and the pop-up can be closed by clicking the X on the bottom right. The High Availability section will have changed, indicating configuration was successful.
Navigating to both the Primary CBOX and Backup CBOXes' System Utilities pages will now show green status lights for the status indicators, within the High Availability section, indicating that configuration was successful and communication has been established between the Primary and Backup CBOX. If the status indicator lights are showing green on one CBOX (such as the Primary), hard refresh the webpage.



Utilizing the HA Configured CBOX Pairs

Utilizing CBOX Web GUI and API Services

Now that two CBOX-HAs have been configured to operate in High Availability mode, it is not recommended to use the main IP address associated with either CBOX. Instead, the Virtual IP Address should be utilized. The designated VIP can be used in an identical manner to using a standalone CBOXes' main IP address.
  1. Entering in the VIP into a web browser will enable access to the web GUI for the Primary-Designated CBOX. 
  2. Accessing Port 24 on the VIP will enable access to the API Server hosted on the Primary-Designated CBOX.
  3. If the Primary-Designated CBOX suffers a service interruption based on the factors described in the first section, the Backup-Designated CBOX will take over the assigned VIP and act as the Primary-Designated CBOX going forward. 
The web GUI/API server of both the Primary CBOX and Backup CBOX can still be accessed by using the main IP address associated with the respective unit. 
Note: It is expected behavior to be unable to control/route devices when using the web GUI/API server of a CBOX configured in Backup operational mode. The Backup CBOX is programmed to only listen, and not actively control devices. That is the purpose of the Primary-Designated CBOX. 

Updating Firmware with High Availability Enabled

When applying firmware updates on an MXnet system, for updates to endpoints, the primary CBOX (therefore the designated VIP) must be used, as the Backup CBOX will be unable to apply firmware updates to endpoints.

For applying firmware updates to the CBOX itself, it is recommended to switch the roles of the Primary and Backup CBOXes to alone (standalone), apply the CBOX firmware update to the two CBOXes, then re-configure both CBOXes in High Availability mode.

Determining if a Fail-Over State Event Occurred

As explained in the first section, a Fail-Over State Event is the state change that occurs when the Backup CBOX overtakes and acts as the Primary CBOX due to a service interruption occurring from the previous Primary CBOX.
When a fail-over event occurs, there is a warning message that will populate within the High Availability section in the System Utilities page, as shown below:


This same message will also populate on the API server on Port 24 every hour, allowing third party control systems to parse the message and raise a flag to notify the user, if programmed to do so:
{"source":"hamsg","info":"WARNING! HA Failover State Occurred at 4-9-2025 11:14:51AM!","code":0}

High Availability API Explanation (For Programmers)

High Availability can be integrated with third party control systems via various programming methods executed by the programmer. 
The easiest method to parse the current status of the High Availability service is to execute the API command "config get mbstatus". This command will output all essential parameters and their respective values. Example output below (executed on port 24):
{"cmd":"config get mbstatus","info":{"Target_Device":{"mcumode":"2","avlink":"1","devicemode":"2","ip":"192.168.1.78","judgelink":"1"},"Local_Device":{"mcumode":"1","mcuver":"0.08","mbactive":"1","mblink":"1","judgelink":"1","ip":"192.168.1.222","mbmsg":"","avlink":"1","judgeip":"192.168.1.1","halink":"1","devicemode":"1"}},"code":0}

Human-Readable Output:
{
  Local_Device = {
    avlink = "1",
    devicemode = "1",
    halink = "1",
    ip = "192.168.1.222",
    judgeip = "192.168.1.1",
    judgelink = "1",
    mbactive = "1",
    mblink = "1",
    mbmsg = "",
    mcumode = "1",
    mcuver = "0.08"
  },
  Target_Device = {
    avlink = "1",
    devicemode = "2",
    ip = "192.168.1.78",
    judgelink = "1",
    mcumode = "2"
  }
}

Local_Device == The device where the "config get mbstatus" command was executed on. Will be the Primary CBOX-HA if the command is executed on the VIP. 
Target_Device == The device designated as either the Primary or Backup CBOX. Will be the Backup CBOX-HA if the command is executed on the VIP. 
avlink == Link Status on the A/V Network Port. 1 == Active, 0 == Disconnected. 2 == Inactive / Standalone Mode.
devicemode == Operational Status of the CBOX. 0 == alone (standalone). 1 == Primary. 2 == Backup. Can be modified using the "config set devicemode {0,1,2}" API command.
halink == Link Status on the HA Port. 1 == Active. 0 == Disconnected.
ip == The Main(control) IP address of the Local or Target Device.
judgeip == The designated Judge IP address. Can be modified using the "config set judgeip {IPV4 Address}" API command. Must be the same on both the Primary and Backup CBOX-HA.
judgelink == Indicates whether or not a successful link to the Judge IP is established. 1 == Active/Connected. 0 == Connection Lost. 2 == HA Inactive / Standalone Mode.
mbactive == The parameter that determines whether or not High Availability is currently active and functioning. 0 == Off. 1 == On. Can be modified using the "config set mbactive {0.1}" API Command.
mblink == Indicates whether or not a successful link to the Primary/Backup CBOX is established. 1 == Active/Connected. 2 == HA is Inactive/Disconnected. 0 == Link has been lost. 
mbmsg == The parameter that contains the Warning message that populates during a failover event. Empty if no fail-over event has occurred. 
mcumode == Current MCU operational mode, used solely for HA Port link. The mcumode parameter must be different between the Primary and Backup CBOX-HA. Can be modified using the "config set mcumode {1,2}" API command. 
mcuver == Current HA MCU firmware. HA RS-232 Port must be on V0.08 or higher in order to function. 

    • Related Articles

    • AC-MXNET-10G-CBOX-HA ~ V5.02 ~ 4/15/2025

      Note: Please update your CBOX before upgrading your endpoints. If your 10G CBOX is reporting firmware within the V3.XX range, please visit this article here to download the proper firmware. AC-MXNET-10G-CBOX-HA Latest Firmware Download Link: ...

    • AC-MXNET-CBOX ~ V2.73 ~ 03/14/2025

      NOTICE: If your CBOX is reading firmware as a version of V4.XX, please navigate to this page here: https://support.avproedge.com/portal/en/kb/articles/ac-mxnet-cbox-b-firmware Note: Please update the CBOX first, before attempting updates on the ...

    • AC-MXNET-CBOX-HA ~ V4.25 ~ 04/10/2025

      NOTE: If your CBOX is reading firmware with a version number of V2.XX, please navigate to this page here: https://support.avproedge.com/portal/en/kb/articles/ac-mxnet-cbox-firmware Please update the CBOX first, before attempting updates on the ...

    • The MXNet Quick Connect Guide

      THE MXNET 1G ECOSYSTEM QUICK CONNECT GUIDE The MXNet Ecosystem is an AV-over-IP platform of products that use a traditional networking infrastructure to route video and audio signals throughout large scale systems with unlimited numbers of sources ...

    • MXNet Network Switches - Basic Operation Configuration Guide

      This article provides a comprehensive overview of basic diagnostic and configuration commands for MXNet network switches, including instructions for making the necessary physical connections. The switches covered in this article are operating on ...