1. Cutsforth Support
  2. InsightCM™ Software
  3. InsightCM™ 3.10.0 User Manual

Troubleshooting

Follow

Logging Important Event Details

A tracepoint is a flag that, when enabled, instructs the server to log messages about when particular events occur. For example, the Storage.MajorOps tracepoint flags events related to file I/O. Review a dynamic list of tracepoints that update as they become available.

  1. Click the Navigational menu » Utilities » Trace Logger.

  2. (Optional) Double-click a tracepoint in the table to enable or disable a specific tracepoint.

  3. (Optional) Create an exception that inverts the status of a tracepoint for a specific device.

    Note

    Exceptions work differently depending on whether the tracepoint is enabled or disabled.

    • Enabled tracepoints - An exception prevents the server from generating messages for the exception item.

    • Disabled tracepoints - An exception allows the server to generate messages for only the exception item.

    1. Select a tracepoint whose name begins with Device in the table.

      Note

      Exceptions are valid only for tracepoints whose names begin with Device.

    2. Click the Edit button.

    3. Click Add and enter the name of the device you want to create an exception for in the text field that appears.

      Note

      Exceptions are case-sensitive.

  4. Download logs of a tracepoint or server-crash information as text files.

    1. Click the Action menu.

    2. Select Download Trace Log or Download Crash Log.

Exceptions for Device-Related Tracepoints

Exceptions are useful for storing tracepoint information from a select number of devices instead of all devices.

For example, if you only care about a specific device that is experiencing issues rather than every device in the system, you might disable the Device.SystemEvent.Level1 tracepoint. Additionally, you might create an exception for the one device you are interested in tracking so that the server generates messages for any events that device might have. Exceptions are case-sensitive, so when you add an exception, enter the exact device name.

Notable Tracepoints

Please note that this is not a complete list of all available tracepoints.

Tracepoint Name

Tracepoint Description

Use Case

Authentication.Srptracing

Reports about operations related to authenticating device credentials.

You are bringing devices online and troubleshooting connection issues.

Provisioning.StateChange

Each time a device tries to connect, reports specific issues related to the device status, such as the software configuration or application being out of date.

Transceiver.Connections

Reports when devices connect, disconnect, or become authorized to connect to the InsightCM Server.

Device.FileManager.SendData

Reports when the device sends data events to the server.

See when a device sends data to the server.

DataStorage.MajorOps

Reports general operations for storing data events.

Monitor the server receiving data events from devices.

PIHistorian.TagWritten

Reports whenever InsightCM Server writes a value to the PI software.

You want to monitor activity by historian software.
tracepointEnabled tracepointsDisabled tracepoints

Resolving and Clearing an Invalid Configuration

Enable your device to collect data again by changing the collection settings and conditions for an asset. When unresolved errors, such as reaching CPU and/or memory capacity, occur and persist in the device application, the device can enter a state of Invalid Configuration. This state works like an idle state where the application does not respond to any internal or external requests except requests to restart.

Complete one of the following steps to resolve and then clear the invalid configuration state of a device.

  1. Click the Configuration pull-down and select Devices.

  2. Double-click the device with the Config Status of Invalid Configuration.

  3. Identify which equipment asset sensors are mapped to the device.

  4. Click the Configuration button to get to the Asset Configuration page.

  5. Select the equipment mapped to the device with an Invalid Configuration.

  6. In the equipment's configuration panel, select the Operating States tab.

  7. In Data Set Collection Settings, configure the file length so that it is lower than its current setting.

    Note

    You may also free up memory or CPU by lowering the Sample Rate in the device's configuration page and/or removing integrated features from sensor asset properties.

  8. Navigate to the Device Configuration page in the web application.

  9. Select the device that still has a Config Status of Invalid Configuration and select the Action menu.

  10. Select Clear Invalid Configuration to make the device responsive to requests again.

If your device's application re-enters an Invalid Configuration state due to issues unrelated to memory or CPU, contact technical support.

Troubleshooting Deployment Issues

Complete the following steps to troubleshoot deployment issues.

  1. Expand the navigation menu and select Utilities » Deployment Status.

  2. Refer to the Action Status column for the failed entry to find the System Configuration API error code associated with that type of failure.

Troubleshooting Devices That Do Not Come Online

Bring your device(s) online by applying troubleshooting steps for one of several possible causes.

Note

Wait several minutes for devices to download configurations, update an application, and/or reboot before troubleshooting.

Status on Device Dashboard

Possible Cause

Troubleshooting Steps

Disabled

The device was manually disabled via the Action menu item on the Device Configuration page.

On the Device Configuration page, select the device and then select Action menu » Enable Device.

Offline

The device is in the process of rebooting.

Wait several minutes to see if the status changes.

The device does not have connection information.

  1. Select the device and then select Action menu » Connection » Test Connection.

  2. If the device responds to the ping with a message other than "All tests passed", complete the following steps.

    1. Wait a few minutes before repeating the test. If the same message recurs, continue to the next step.

    2. Verify the IP address is correct.

    3. With the offline device selected, select Connection » Reset Connection from the Action menu.

    4. If the device fails to reset, physically reboot the device using the Reset button on the controller.

The device is not powered on or not connected to network.

  • Verify the physical connections to power and the network.

  • Watch the USER1 LED on the device. This LED blinks in one-second increments to indicate normal operation.

  • Physically reboot the device using the Reset button on the controller.

Unconfigured

The device IP address is not configured and has never tried to connect to the InsightCM Server.

  1. Double-click the device to modify its configuration.

  2. Select the Hardware tab and then select Edit Hardware.

  3. Update the IP Address for the device and click OK.

Unauthorized

The device does not have credentials that match what the InsightCM Server expects.

  1. Select the device and then select Action menu » Connection » Test Connection.

  2. If the device responds to the ping with a message other than "All tests passed", complete the following steps.

    1. Wait several minutes before repeating the test. If the same message recurs, continue to the next step.

    2. Verify that the device IP address in the web application matches the device's current IP address.

    3. With the offline device selected, select Connection » Reset Connection from the Action menu.

    4. If the device fails to reset, physically reboot the device using the Reset button on the controller.

UpdatingFirmware

The device is in the process of downloading firmware and rebooting.

  1. Wait several minutes after the last action you performed in the web application before refreshing it to see if the status changes.

  2. Change the device type if the incorrect value is in the Device Type column by selecting Action menu » Change Device Type.

UpdatingConfiguration

You saved the device configuration, or changes to the server triggered the InsightCM Server to update the configuration on the device.

Wait at least five minutes after the last action you perform in the web application to see if the status changes.

Logging Information about the Connection Process

Enable the following tracepoints on the System page >Trace Logger tab to log events that occur specifically during the provisioning process.

Note

Refer to the help documentation for the Logging Important Event Details topic for more information about enabling tracepoints and accessing the logs they produce.

Tracepoint Name

Tracepoint Description

Provisioning.StateChange

Reports changes in the status of a device, such as when the software configuration or application is out of date.

Device.SystemEvent.Level1

Reports about device operations, such as acquisition triggers that fire and their cause.

Note

This point is not available until the first device connects to the InsightCM Server.

Authentication.Srptracing

Reports about operations related to authenticating device credentials.

Transceiver.Connections

Reports when devices connect, disconnect, or become authorized to connect to the InsightCM Server.

Troubleshooting the Connection between the Server and a Device

Determine why the server and a device are not connecting and establish/restore the connection.

The Test Connection dialog box checks common issues that prevent a device from connecting to the InsightCM Server and coming online.

  1. Check for common issues by testing the connection between the server and the device.

    1. Select the pull-down by Configuration » Devices.

    2. Select Action menu » Connection » Test Connection.

  2. (Optional) If the test results in All tests passed, you may need to deploy a supported application to the device before it will come online.

    Note

    The device may be formatted correctly, but not yet running a supported application.

    1. Select the pull-down by Configuration » Devices.

    2. Click the Software tab and select the device.

    3. Click Update Application.

Troubleshooting Email Delivery

Debug possible reasons email delivery fails using tracepoints.

Refer to the Setting Up Email Alarm Notifications topic for steps to configure settings for an SMTP server.

If the InsightCM Server fails to send alarm notification emails, complete the following steps to log information about the failure.

  1. Click the Navigation menu (noloc_env_navmenu.png ) and select Utilities >> Trace Logger.

  2. Double-click the SMTP.Emails tracepoint to enable it.

  3. Click the Navigation menu (noloc_env_navmenu.png ) and select Options.

  4. In the tree on the left, select SMTP.

  5. Select Test Mode.

    Note

    Test Mode prevents the server from sending emails, even when you run commands from the InsightCM console. Enable the SMTP.Emails tracepoint to view messages that the server writes to the trace log.

  6. Trigger an alarm that sends an email notification.

    The server attempts to send an email to that address group and logs information about the attempt in the trace log.

  7. Return to the Trace Logger tab and select the Action menu (noloc_env_action.png) and Download Trace Log.

  8. Open the log file with a text editor and search for the tracepoints labeled SMTP.Emails for email delivery information.

Error Values for Integrated Features

Symptoms of Issue

  • Features appear in the web application with incorrect values (by default, -1), which indicates an error occurred.

  • The features are for a measurement type derived by single or double integration. For example, features calculated from velocity data that you derive from acceleration data via single integration.

Possible Cause

The time-domain data from which the InsightCM Server calculates the features might not be of a sufficient duration. To calculate features from integrated data, the InsightCM Server requires enough time-domain data to allow for filter settling time.

Common Causes of This Issue

As mentioned previously, this issue occurs only when single or double integration is enabled for a channel. Also, the issue often occurs in data acquired as part of a run-up or coast-down stream because those acquisitions can be very short. For example, consider a device configured to perform stream acquisitions with a length of 10 revolutions. For fast-moving equipment, the revolutions might occur in a very short amount of time, meaning the resulting waveforms do not contain enough data for InsightCM Server to calculate feature values.

This issue can still occur even if you ensure that acquisitions always last at least one second. For example, consider an accelerometer channel with single integration enabled in order to perform velocity measurements. If features calculated from the acceleration data are valid but velocity features have error values, the cause might be that the velocity time-domain data is too short. This occurs because of the way the InsightCM Server filters integrated time-domain data.

How Filtering Affects the Duration of Integrated Data

To account for integration settling time, InsightCM discards the first n seconds of the integrated time-domain data before calculating features from it, where the following equation determines the value of n.

n = (1 / Integration Cutoff) * x

As the following table explains, the value of Integration Cutoff and x vary according to whether the InsightCM Server performs single or double integration.

Type of Integration

Source of Integration Cutoff Value 1

Value of x Constant

Single

Single Integration Cutoff property

3

Double

Double Integration Cutoff property

10

1Set these properties for each channel on the Device Configuration page»Channels tab»Properties tab.

Working Around This Issue

To avoid situations where integrated time-domain data is too short, set the acquisition duration to be a number of seconds such that the waveform is at least one second longer than the integration filter settling time.

Also, if you change the value of the Single Integration Cutoff or Double Integration Cutoff property for a channel, ensure the new value does not cause the integration settling time to become too long.

Replacing a Device in InsightCM

Bring replacement devices online in InsightCM.

Ensure that you have another device to replace your original device.

Complete the following steps to replace a device in the web application as you do so in person.

  1. Click the Configuration pull-down and select Devices.

  2. Click the Action menu and select Disable Device.

    Note

    Disabling the device prevents it from attempting to communicate with the InsightCM while you are replacing it.

  3. Power off the old device.

  4. Set the IP address of the new device to be the same as the device you are replacing.

    This allows the new device to connect to InsightCM and come online when it powers on.

  5. Mount the new device in place of the old device, connecting to the network, sensors, and power.

  6. In the web application, on the Device Configuration page, click the Action menu and select Connection » Test Connection.

    This prompts the device to connect to InsightCM.

  7. Click the Action menu » Connection » Reset Connection to send a new connection information file to the device.

    The connection file includes new credentials so that the device can come online.

  8. Check the device status on the Device Configuration page to ensure it comes online.

  9. If the model of the new device is different from the old module, update the device configuration.

Testing Sensors to Validate Hardware

Test sensor asset connections in the Test Panel page.

Task

Action

Viewing feature values for every asset on a device.

Browse to any device-related page, such as the Devices page or a Device Dashboard page, and then select Test Panel from the View menu.

Click the Feature Chart tab and select an asset from the pull-down menu in the top-left of the tab.

Configuring the feature trends tab

Browse to any device-related page, such as the Devices page or a Device Dashboard page, and then select Test Panel from the View menu.

Click the Feature Trend tab.

Expand the Select Data Group pull-down menu and select a data group.

Click Select Trends and use the resulting dialog box to manage what trends appear in the graph.

(Optional) Click the Set Scale button, remove the checkmark from Auto-scale, and enter the minimum and maximum axis values.

Displaying the trend of feature values

Browse to any device-related page, such as the Devices page or a Device Dashboard page, and then select Test Panel from the View menu.

Click the Feature Trend tab and select a data group to view feature values for in the pull-down menu in the top-left of the tab.

Viewing live domain data

Browse to any device-related page for an EMSA device, such as the Devices page or a Device Dashboard page, and then select Test Panel from the View menu.

Click the Time Domain tab and configure the options in the left side of the tab.

Note

Viewing live data on the Time Domain tab pauses the current spectrum sweep. The sweep resumes when you pause the live data view again.

Recording live domain data

Browse to any device-related page for an EMSA device, such as the Devices page or a Device Dashboard page, and then select Test Panel from the View menu.

Click the Audio tab and configure the options in the left side of the tab.

Note

Recording live time domain data on the Audio tab pauses the current spectrum sweep. The sweep resumes when the recording finishes.
tracepointEnabled tracepointsDisabled tracepoints
Was this article helpful?
0 out of 0 found this helpful

Articles in this section