Process - Reconfigure OMS Node 1.1

1 Process

The process Process - Update OMS Node is a rather heavy weight process, since it completely deletes and reinstalls IOM. In case smaller configuration changes, like changing Java options or logging configuration, etc., need to be applied, reconfigure_oms_node provides a more lightweight process. The drawback of the process reconfigure_oms_node is that it does not cover all configuration changes. For details about the applicable changes in each OMS process, see Reference - Ansible4IOM Variables - OMS. Since reconfiguration does not make any changes to the database content, the process might run without forcing a downtime of the whole system, if the process is not executed for all nodes at the same time.

Description

The process reconfigure_oms_node is the right choice to update the project code and configuration. The process executes all hooks, which should be used by projects and it redeploys all artifacts. If you are using the process reconfigure_oms_node to update project code and configuration, you should execute the process on all servers in parallel (forcing a downtime) to avoid inconsistency between project code and project specific data-structures in the database.

Example: reconfigure oms nodes
ANSIBLE_LIBRARY=<path to Ansible4IOM>/modules/ \
ANSIBLE_ROLES_PATH=<path to Ansible4IOM>/roles/ \
ansible-playbook -i <path to inventory file>/inventory <path to Ansible4IOM>/processes/reconfigure_oms_node.yml

1.1 Steps

The process consists of the following steps:

  1. ftpd_cleanupservice
    1. Disable/stop pure-ftpd service in order to avoid successful health-checks during reconfiguration process.
  2. oms_cleanupservice
    1. Stop/disable IOM service.
  3. oms_reconfig_for_detach_from_cluster (backend server of groups oms_ha_node and oms_azure_node only)
    1. Increment setting of port-offset in installation.properties by one.
    2. Set thread count to 0 in quartz-cluster.properties.
  4. (java_install)
  5. oms_pgclient_config
    1. adds a .psqlrc file at OMS-users home directory to avoid autocommit for interactive sessions
  6. oms_installation_properties
    1. Update installation.properties file with values found in the inventory.
  7. oms_cluster_properties
    1. Update cluster.properties file with values found the in inventory.
  8. oms_service
    1. Setup and start OMS service directly (not by watchdog).
  9. oms_configuration
    1. Execute pre_oms_configuration_hook.
    2. Apply system.std.*.properties.
    3. Apply JMS load balancing.
    4. Apply cluster.properties.
    5. Execute post_oms_configuration_hook.
  10. oms_deploy
    1. Execute pre_oms_deploy_hook.
    2. Redeploy all artifacts and wait for Wildfly to get ready again.
    3. Execute post_oms_deploy_hook.
  11. oms_reconfig_for_attach_to_cluster (backend server of groups oms_ha_node and oms_azure_node only)
    1. Restore original port-offset in installation.properties.
    2. Restore original thread count in quartz-cluster.properties.
  12. oms_watchdog
    1. Update watchdog.properties.
    2. Stop OMS service.
    3. Update systemd unit file of watchdog service.
    4. Reload, enable, start watchdog service.
  13. ftpd_service
    1. Execute pre_ftpd_start_hook
    2. Start/enable FTPd-service.

1.2 Background Information

Reconfiguring the process is intended to change simple configuration settings without reinstallation of IOM. This limits the settings to be changed. Reference - Ansible4IOM Variables gives an overview about possible changes. Please see also Process - Setup OMS Node, to get a general overview about configuration of IOM installations.

1.2.1 Reconfiguration Process - Avoid Side Effects of Parallel Execution of Backend Servers

During the reconfigure process, the load balancer must not forward any requests to the IOM application server currently reconfigured. Since the load balancer uses health-check requests sent to the application servers to determine the application servers to use, the goal can be reached by marking the according node as unhealthy. This is realized by stopping FTPd-service during the reconfiguration process.

During the reconfiguration process, backend servers are not under control of the watchdog. During this time, the currently reconfigured backend server might run in parallel to other backend servers. It has to be ensured that the currently reconfigured backend server must not execute jobs and must not receive and handle JMS messages.

Technically, this behavior is realized by the roles oms_reconfig_for_attach_to_cluster and oms_reconfig_for_detach_from_cluster. When detached from the cluster, the backend server will not run any jobs and will not receive JMS messages. Even if it is running in parallel to other backend servers, the detached backend server will not influence the cluster at all.

2 Examples

2.1 Use Different Java Options for Backend and Frontend Servers

The variable JBOSS_JAVA_OPTS, defined in roles/oms_config/defaults, is a hash. The variable cannot be set in the inventory file directly. The inventory file supports setting of simple variables only. To set complex variables, use files located at directory group_vars. Depending on the installation type that is used, a file within group_vars has to be created. It must be named exactly as the server group assignment: use (oms_single_nodeoms_ha_nodeoms_azure_node).

The following example shows the necessary settings to change Java options of an Azure Cloud installation of IOM. Frontend and backend server will use different configurations.

group_vars/oms_azure_node
JBOSS_JAVA_OPTS: {
  backend: "-Xms512M -Xmx4096M -XX:+UseG1GC -XX:ParallelGCThreads=8 -XX:+UseAdaptiveSizePolicy -XX:TargetSurvivorRatio=40 -XX:+PrintGCDetails -XX:+PrintGCDateStamps",
  frontend: "-Xms512M -Xmx2048M",
}

Now, the process reconfigure_oms_node can be executed. To avoid a downtime of the system, both HA-nodes of the IOM system are updated after each other.

Example: setup of OMS node
ANSIBLE_LIBRARY=<path to Ansible4IOM>/modules/ \
ANSIBLE_ROLES_PATH=<path to Ansible4IOM>/roles/ \
ansible-playbook -i <path to inventory file>/inventory <path to Ansible4IOM>/processes/reconfigure_oms_node.yml --limit iomha1
 
ANSIBLE_LIBRARY=<path to Ansible4IOM>/modules/ \
ANSIBLE_ROLES_PATH=<path to Ansible4IOM>/roles/ \
ansible-playbook -i <path to inventory file>/inventory <path to Ansible4IOM>/processes/reconfigure_oms_node.yml --limit iomha2

2.2 Change is.oms.db.hostlist in cluster.properties

When adding a new PostgreSQL HA-node, all IOM application servers have to be reconfigured, to enable failover to the new node. Therefore the list of DB-hosts in property is.oms.db.hostlist has to be extended and cluster.properties have to be loaded into application servers.

Edit the inventory file and add the new server to variable is_oms_db_hostlist (see roles/oms_config/defaults).

inventory file
...
[all:vars]

is_oms_db_hostlist="10.10.10.1:5432,10.10.10.2:5432,10.10.10.3:5432"
...

Now the process reconfigure_oms_node can be executed. To avoid a downtime of the system, both HA-nodes of the IOM system are updated after each other.

Example: setup of OMS node
ANSIBLE_LIBRARY=<path to Ansible4IOM>/modules/ \
ANSIBLE_ROLES_PATH=<path to Ansible4IOM>/roles/ \
ansible-playbook -i <path to inventory file>/inventory <path to Ansible4IOM>/processes/reconfigure_oms_node.yml --limit iomha1
 
ANSIBLE_LIBRARY=<path to Ansible4IOM>/modules/ \
ANSIBLE_ROLES_PATH=<path to Ansible4IOM>/roles/ \
ansible-playbook -i <path to inventory file>/inventory <path to Ansible4IOM>/processes/reconfigure_oms_node.yml --limit iomha2

2.3 Change Log-Level of IOM Backend Servers to ERROR

The following property defines the default setting of the BAKERY file-handler:

  • /subsystem=logging/periodic-rotating-file-handler=BAKERY: level="WARN"

The property is part of the system.std.backend.properties file. Ansible4IOM does not reflect any property of system.std.*.properties in own variables. Hence, Ansible4IOM does not support an update of system.std.*.properties directly. Instead, custom Ansible code has to be placed to pre_oms_configuration_hook in order to make changes to system.std.*.properties. Also see Process - Setup OMS Node.

Depending on the scope of the hook (project or installation), the file pre_oms_configuration_hook.yml has to be stored in the directory global_hooks or installation_hooks. Also see Concept - Ansible4IOM Server Configuration Management 1.0 - 1.1.

It is important to use the variables OMS_SERVER_TYPE, OMS_ETC[OMS_SERVER_TYPE] and OMS_USER (see roles/oms_config/defaults) for a robust implementation of the hook. This way, the custom configuration will still work, even if the installation of IOM will be customized far more.

pre_oms_configuration_hook.yml (valid for IOM >= v2.2.0.0)
- name: update Log-Level of BAKERY file-handler in system.std.backend.properties
  lineinfile:
    dest: "{{OMS_ETC[OMS_SERVER_TYPE]}}/system.std.{{OMS_SERVER_TYPE}}.properties"
    regexp: '^[ \t]*/subsystem=logging/periodic-rotating-file-handler=BAKERY:'
    line: '/subsystem=logging/periodic-rotating-file-handler=BAKERY: level="ERROR"'
    state: present
  when:
    - OMS_SERVER_TYPE == "backend"
  become: true
  become_user: "{{OMS_USER}}"

Now the process reconfigure_oms_node can be executed. To avoid a downtime of the system, both HA-nodes of the IOM system are updated after each other.

Example: setup of OMS node
ANSIBLE_LIBRARY=<path to Ansible4IOM>/modules/ \
ANSIBLE_ROLES_PATH=<path to Ansible4IOM>/roles/ \
ansible-playbook -i <path to inventory file>/inventory <path to Ansible4IOM>/processes/reconfigure_oms_node.yml --limit iomha1
 
ANSIBLE_LIBRARY=<path to Ansible4IOM>/modules/ \
ANSIBLE_ROLES_PATH=<path to Ansible4IOM>/roles/ \
ansible-playbook -i <path to inventory file>/inventory <path to Ansible4IOM>/processes/reconfigure_oms_node.yml --limit iomha2


Disclaimer

The information provided in the Knowledge Base may not be applicable to all systems and situations. Intershop Communications will not be liable to any party for any direct or indirect damages resulting from the use of the Customer Support section of the Intershop Corporate Web site, including, without limitation, any lost profits, business interruption, loss of programs or other data on your information handling system.

Customer Support
Knowledge Base
Product Resources
Support Tickets