Event Manager

Overview

The event manager feature facilitates the automatic execution of a particular action item based on the event (operator log messages) that occurred in a device. This feature is configured by command line interface (CLI) and NetConf.

The following are the three parameters in the event manager feature:

  • Event: It is a trigger where event manager functionality starts. Once the syslog message with the details mentioned in the event occurs, an action is triggered. Some sample events are as follows:
    • IFMGR_IF_DOWN
    • IFMGR_IF_UP
    • STP_SET_PORT_STATE
    • STP_IPC_COMMUNICATION_FAIL
  • Action: Once an event has occurred, an action is triggered if there is a match of the event ID in the database. An action is executed by the execution of a Python script consisting of executable OcNOS commands and configurations.

    The sample action script is as follows:

    Copy
    import sys,os,time
    import subprocess
     
    #MACROS#
    ########
    TIME = 1
     
    #VARIABLES#
    ##############
    cmd_db_lock = "cmlsh -e 'configure terminal force "+str(TIME)+"'"
    cli_commands ="cmlsh -e 'configure terminal' -e 'interface xe11' -e 'shutdown' -e 'commit' -e'end'"
     
    if __name__ == '__main__':
    #if name == 'main':
      #Force user out of config mode after X seconds
      os.system (cmd_db_lock)
      #Wait X seconds before running clis
      time.sleep(TIME)
     
      os.system(cli_commands)
  • Policy: It maps the action with an event.

Feature Characteristics

  • The feature creates a database of event IDs and the corresponding actions as configured through CLI. When an event occurs, the event is matched in the database with the existing event ID, severity, and log pattern. If the event matches with the existing event in the database, it triggers a corresponding action automatically. If there is no match with the database, then no action is taken.
  • Configurable parameters for an event are event ID, severity, and log pattern, which are matched with the incoming log. In order to be unique, the recommendation is to have all these parameters configured for an event. Configuring the event ID is mandatory, while severity and pattern are optional. No manual configuration of severity applies the default severity of all (0-6).
  • Duplicate event configuration with the same value for event ID, severity, and log pattern as an existing event with a new event name is not allowed and displays an error.
  • The feature facilitates the configuration of one action for multiple events.
  • Place the action script file in the path /usr/local/etc. A warning message is displayed if the script file is not in the path, but the configuration is accepted.
  • The execution count or the trigger count per policy is stored and maintained. When a policy is cleared, the event and the action associated with the policy get cleared. When an action is associated with multiple policies, the action associated with the cleared policy is removed, and the same action associated with other policies remains.
  • This feature consumes a certain amount of CPU performance because it matches the logs recorded by the system with every configured event. Hence, a maximum number of 50 events, actions, and policies is configurable.
  • The command line shell (cmlsh) uses a locking mechanism. Follow the recommendation when a user or script file gets into the configure mode:
    • Disable the event manager feature while executing manual configuration in the system. This prevents the Python script from interfering with the manual configuration. After executing the manual configuration, enable the event manager feature.
    • There is a possibility of multiple Python scripts executing simultaneously. In order to sequence the configure mode execution, the Python script has the logic to wait for 45 seconds in the configure mode. This prevents the Python script from exiting without executing the commands if another script is still in configure mode.
    • If the script fails to execute, the event manager does not record such failures.

Validation checks

  • When the feature is neither enabled nor disabled, the event, action, or policy configuration displays an error.
  • The event manager displays an error if an event is edited when associated with a policy.
  • The event manager exercises priority-based selection of policies for any incoming logs. When there are more actions associated with the same event with different event IDs, severity levels, and pattern, the priority sequence is as follows:
    1. Matches the incoming log against a policy that has an event configured with all the parameters, which are event ID, severity, and pattern string.
    2. Matches the incoming log against a policy that has an event configured with only event ID and severity. 
    3. Matches the incoming log against a policy that has an event configured only with the event ID.  
Example 1

For the following configuration, when actual log “2020 Jan 03 08:46:56.455 : MH2 : NSM : CRITI : [IFMGR_IF_UP_2]: Interface xe3 changed state to up“ is received, event-manager execute action a2 (file2) than action a1 as this configuration matches the best.

Copy
#event-manager event e1 IFMGR_IF_UP severity 2
#event-manager event e2 IFMGR_IF_UP severity 2 pattern “Interface xe3” 
#event-manager action a1 script file1
#event-manager action a2 script file2
#event-manager policy p1 event e1 action a1
#event-manager policy p2 event e2 action a2
Example 2

For the following configuration, when actual log “2020 Jan 03 08:46:56.455 : MH2 : NSM : CRITI : [IFMGR_IF_UP_2]: Interface xe3 changed state to up“ is received, event-manager executes either action a1 (file1) or action a2 based on whichever gets hit first during database search. The recommendation is not to mix the same event configuration with a pattern and without a pattern for the same event ID.

Copy
#event-manager event e1 IFMGR_IF_UP severity 2 pattern “Interface ” 
#event-manager event e2 IFMGR_IF_UP severity 2 pattern “Interface xe3”
#event-manager action a1 script file1
#event-manager action a2 script file2
#event-manager policy p1 event e1 action a1
#event-manager policy p2 event e2 action a2
The solution supports the validation of event-id against configurable event-ids. It displays an error if the entered event-id is not supported.

Benefits

The event manager feature allows the execution of an automatic action when a failure or any other priority error occurs.