ServiceNow auditing (history) feature for a table and troubleshoot related issues

ServiceNow auditing (history) feature:

ServiceNow auditing (history) feature for a table can be enabled using below steps:

1.       Navigate to System Definition > Dictionary.

2.      Select the table to audit. For example, alm_asset.

3.      Select the dictionary entry for the table. The table name always has an empty column name and the type of collection.

4.      Check the Audit check box.

5.      Click Update.

It should be noted that, by default, the system does not audit records from system tables. To audit a system table, add it to the list of tables in the glide.ui.audit_deleted_tables property list.

Once these steps are performed, ideally ServiceNow should start auditing field value changes in corresponding table. One can view the historical data for that table using below options.         

• Users can view History detail in Calendar format for that table by right click on the form and selecting History à Calendar
• Users can view Record History in List format for that table by right click on the form and selecting History àList
• Audit History section can be added at the bottom of form (for that table) by Configure à Related Lists and then moving Audit History to Selected list.

Note: In Fuji version, Configure option is selected on right click on the form. Prior to Fuji version, we use Personalize option

In some cases, audit feature doesn’t work as expected. At high level, it appears as if audit feature is not working. It may also appear that ServiceNow audit feature is working, perhaps some functionality is either restricting the audited data from saving and/or wiping out audit history data.

In midst of all these doubts, you need to be very clear that whether audit feature is not working for any specific table or all tables. If it is not working for any specific table, there can be multiple root cause for that behavior. It might take some reasonable amount of time and effort to identify the exact reason behind the auditing failure for that table.

Before deep dive into the issue, first please note below information from ServiceNow Wiki documentation:

·         ServiceNow tracks incident, change, and problem history in the sys_audit table.

·         Enabling auditing tracks the creation, update, and deletion of audited records.

·         In addition, auditing activates the History context menu entry.

·         It is not possible to audit individual fields without auditing the entire table.

·         It is possible, however, to hide certain fields from the audit using a dictionary attribute.

·         Auditing certain system tables that receive a large amount of traffic, such as Workflow Contexts [wf_context], can impact performance and is not recommended.

Auditing feature tracks only following record changes.

·         The Unique Record Identifier (sys_id) of the record that changed

·         The field that changed

·         The new field value

·         The old field value

·         The number of times this record and field have been updated

·         The date and time when the change occurred

·         The user who made the change

·         The reason for the change (if any reason is associated with the change)

·         The record's internal checkpoint ID (if the record has multiple versions)

Some updates are not audited despite enabling auditing on a table. Auditing excludes the following information:

·         Any updates made by an upgrade.

·         Any updates made through import sets.

·         Any records in parent or child tables.

·         Any field with the no_audit dictionary attribute.

·         Any system tables not listed in the glide.ui.audit_deleted_tables property list.

·         Any field that begins with the sys_ prefix (system fields) except the sys_class_name and sys_domain_id columns.

·         Any time an inactivity monitor touches a record (this prevents you seeing possibly hundreds of updates listed against an incident, with the noise drowning out the useful data).

If your case is not addressed by above details from ServiceNow Wiki documentation, you can explore further. To save your time, you can initially check if below statements are used in ServiceNow scripts related to the target table. One or more custom scripts using below statement might be preventing the sys_audit table from being updated for that table.

current.autoSysFields(false);

current.setWorkflow(false);

ServiceNow Wiki documentation explains,

autoSysFields:

public void autoSysFields(boolean e)

Enables or disables the update to the fields sys_updated_by, sys_updated_on, sys_mod_count, sys_created_by, and sys_created_on. This is often used for manually updating field values on a record while leaving historical information unchanged.

Note: Use caution if you use this method. When you use this method the sys_mod_count field will not be incremented, and other sys_ fields will not be updated. This can break functionality including, but not limited to, the Activity Formatter, History Sets, and Metrics.

Parameters:

e - Boolean variable that if false disables updates to sys_updated_by, sys_updated_on, sys_mod_count, sys_created_by, and sys_created_on.

setWorkflow:

public void setWorkflow(boolean e)

Enables or disables the running of business rules that might normally be triggered by subsequent actions. If the e parameter is set to false, an insert/update will not be audited. Auditing only happens when the parameter is set to true for a GlideRecord operation.

Parameters:

e - Boolean variable that if true (default) enables business rules, and if false to disables them.

The idea here is to examine your ServiceNow instance's Script Includes, UI Actions, and business rules for autoSysFields(false) or setWorkflow(false) calls related to that table. You can set active = false one by one and retest, or set them all to active = false and retest. That should help you to isolate the cause of the issue.

For example, to identify the business rules at fault, you can enable debugging trace of business rule. You can follow below steps:

1.       Navigate to Debug and click

·         Debug Business Rule (Details)

·         Debug Log

·         Debug SQL (Detailed).

2.      Update a record (from that table) by clicking 'Save' (Right-mouse-click header and click Save, if necessary), so that you stay on the same page.

3.      Examine the trace the bottom of the form. Note that

=== means a business rule was skipped because the condition did not evaluate to true

==> entering the business rule

<== exiting the business rule

In case you have already identified those script lines where autoSysFields(false) or setWorkflow(false)  are called, examine those lines. Whether those statements have been used purposely to achieve any desired business functionality? There can be also a possibility of developers overlook while using these statements. No matter what is the reason, autoSysFields(false) or setWorkflow(false) statements need to be used carefully and avoided, if not needed. Else it will override the audit feature on the related table. In addition to this, it might also impact metrics definition and instance data for related table and fields.

You need to identify and fix the custom scripts that are turning off the sys_audit updates on that table. For testing purpose, you can comment those lines and see if audit feature works on the required table by doing some record updates. In case, it still not work and you are sure that audit feature need to work on that table in all scenarios, you can follow a workaround as an alternate solution. This will be a brute-force sort of fix. You can create a business rule on that table for “update” and “insert” action with “onbefore” event. Include following statements in that business rule and keep its “order” to maximum in relation to other business rule for that table.

current.autoSysFields(true);

current.setWorkflow(true);

This will ensure to record the sys_audit table entries. If the problem still persists, it will be advisable to involve and consult ServiceNow team.