As RPA platform administrator, you can remove nodes from a Control Room cluster setup when you want to replace or update them for enhanced performance.

Prerequisites

Ensure the following:

  • Follow the decommissioning sequence of the nodes

    Whenever nodes are removed from the cluster, ensure that the decommissioning process is performed sequentially, handling one node at a time.

    Note:
    • When removing or downgrading nodes within a cluster, ensure that the resulting cluster always retains an odd number of active nodes. This practice helps maintain optimal quorum and cluster stability.

      For example, in a 7‑node cluster, if nodes need to be removed or downgraded, you must remove 2 nodes, leaving 5 nodes active. This ensures that the cluster continues to operate with an odd node count, which is essential for maintaining quorum.

    • For data to be completely available, it must have at least n/2 + 1 nodes up. It is recommended to have at least 3 nodes for High Availability (HA) to avoid split brain scenario which can result in data loss. For a three node cluster, at least two nodes must be available.
    • When removing a node from the cluster, always verify the placement of primary and replica shards for each index. Avoid deleting any node that contains both the primary and replica shards of the same index, as this can cause that index to move into a RED status.
  • Verify that all the primary nodes and databases have backups.
  • Ensure that you have administrator or root privileges on the Control Room servers.
Note: Starting from Automation 360 v.28 release (Build 17593), the Elasticsearch service is replaced with OpenSearch service as the underlying technology for core product capabilities such as audit, Bot Insight, and so on.

Perform the following steps to remove three nodes (N1, N2, and N3) from a cluster that has six nodes (N1, N2, N3, N4, N5, and N6).

Procedure

  1. Verify the primary node.
    Use the following OpenSearch API through a browser or API tool to identify the primary node:

    https://172.31.18.37/_cat/nodes

  2. Log in to the server as an administrator and stop all the Control Room services by running the following command:
    services.msc
    Log in to one of the original nodes N2 or N3 and not the primary node N1.
  3. Similarly, stop all the Control Room services for the primary node N1.
  4. Before removing the nodes, verify the health of the cluster by using the following OpenSearch API through a browser or API tool:
    From a browser, navigate to the IP address of any node in the cluster by using the OpenSearch port 47599 and use the _cat/nodes API.
    https://172.31.18.37/_cat/nodes
    Note: The primary node is indicated by an asterisk character (*).
  5. Edit the cluster.properties file located at C:\Program Files\Automation Anywhere\Automation 360\config.
  6. Remove the IP addresses for the original three nodes from the cluster.properties attribute.
    Note: Perform this action for all the nodes in the cluster. When you remove the IP addresses of the original three nodes, do not change their sequence.
  7. Edit the elasticsearch.yml file at C:\Program Files\Automation Anywhere\Automation 360\elasticsearch\config.
  8. Remove the IP addresses for the original three nodes from the discovery.zen.ping.unicast.hosts attribute.
    Note: The discovery.zen.ping.unicast.hosts attribute must contain the IP addresses only for the new nodes and in the same sequence within the file on each node.
  9. To verify the new primary node, use the following OpenSearch API through a browser or API tool:
    https://172.31.18.37/_cat/nodes
  10. Update the IP address of the new primary node in the cluster.initial_master_nodes attribute.
  11. Run the following command to start the services on each node:
    services.msc
    You must start the services on the primary node at the end.
  12. To verify the health of the cluster, use the OpenSearch API through a browser or API tool.
    1. From a browser, navigate to the IP address of any node in the cluster by using the OpenSearch port 47599 and use the _cat/nodes API.
      https://172.31.30.120:47599/_cat/nodes
      Note: The primary node is indicated by the asterisk character (*).
    2. From a browser, navigate to the IP address of any node in the cluster by using the OpenSearch port 47599 and use the _cat/health?pretty API.
      https://172.31.30.120:47599/_cat/health?pretty
    The replication status turns green when the cluster is fully synchronized. You must wait until the replication status turns green. The replication time varies depending on the amount of data in the clusters.