Auditing by manually testing the input and output of a computer system

Portions of this article, including many of the definitions and terminology, have been sourced and summarized from ISACA.org and course materials published by ISACA.

What are application controls?

Application controls are transactions and data relating to each computer-based application system and are specific to each application. The objectives of application controls, which may be manual or programmed, are to ensure the completeness and accuracy of the records and the validity of the entries made therein.

Application controls are controls over the input, processing and output functions. This includes several top-level items: 

• Ensure the input data is complete, accurate and valid
• Ensure the internal processing produces the expected results
• Ensure the processing accomplishes the desired tasks
• Ensure output reports are protected from disclosure

Upon closer inspection this includes:

• Edit tests
• Control totals/batch balancing
• Reconciliation of accounts
• Exception handling

Both automated controls and manual procedures should be used to ensure proper coverage. These controls help ensure data accuracy, completeness, validity, verifiability and consistency, and thus ensures the confidentiality, integrity and availability of the application and its associated data.

Understanding application risk: The CIA triad

An application is a computer-based system that processes data for a specific business purpose. Here are some examples: 

• General ledger
• Fixed assets
• Inventory control
• Sales
• Manufacturing resource planning (MRP)
• Distribution requirements planning (DRP) 
• Human resources
• And everyone’s favorite — payroll

Business applications have the same three basic risks as any other system handling data: confidentiality, integrity and availability (CIA).

Confidentiality relates to a data breach or a release of data in violation of legal regulations, such as the Federal Privacy Act, FERPA or HIPAA.

Integrity focuses on data that can be relied upon for accuracy and availability and is available when needed.

Application input controls

When we talk about input controls for applications we must look at several items: 

• Input authorization
• Batch controls and balancing
• Error reporting and handling
• Batch integrity in online or database systems

Application input authorization

Authorization of input means the data has been properly authorized to be input into the application system. Be aware of things like signatures on batch forms, online access controls, unique passwords, workstation identification and source documents.  

Batch controls and balancing

With batch controls and balancing, we might look at the total monetary amount, total items, total documents and hash totals. Some of the batch balancing might be input manually; we want to make sure the manual totals are in agreement with the computer totals.  

Error reporting and batch integrity

In error reporting and handling, we want to look for controls that determine what happens to a batch that has an error: do we reject only the transaction or the whole batch? Do we hold the batch in suspense pending correction, or do we just process the batch and flag the error? Some of the input control techniques include things like a transaction log, reconciliation of data, documentation, error correction procedures, anticipating, transmittal log and cancellation of source documents.

Application processing controls

There are three things to focus on with processing controls:

• Data validation and editing procedures
• Processing controls
• Data file control procedures

Data validation and editing procedures

For data validation, think SQL injection, and now you have a picture of just one of the many data validation edits. Data validation is meant to identify data errors, incomplete or missing data and inconsistencies among related data items. Editing procedures are preventive controls designed to keep bad data out of your database. ISACA lists several data validation edits and controls:

• Sequence check
• Limit check
• Range check
• Validity check
• Reasonableness check
• Table lookups
• Existence check
• Key verification
• Check digit
• Completeness check
• Duplicate check
• Logical relationship check

Processing controls

Processing controls are there to ensure that the incoming data is processed according to established rules for how particular data is to be processed through the application. Some of these processing controls include run-to-run totals, limit checks, and reasonableness verification of calculated amounts.

Data file control procedures

In data file control procedures we can ask, “Are you sure the master file was updated correctly?” We can respond, “We made a before image copy of the database, then ran the update and then ran an after image copy. We then compared the two images and the update performed as expected.” You will also run into other types of data file controls:

• Parity checking
• Transaction logs
• Version usage
• File updating and maintenance authorization

Application output controls

In output controls, the biggest concern is if the information distributed went to the appropriate recipient. As an auditor, you will need to find out:

• Where the sensitive report was printed?
• Was distribution controlled? 
• How long are the sensitive reports retained?
• Are they stored in a protected environment? 
• Are they protected from disclosure, or, are they confidential? 

Online transaction and database controls

The online world of transactions and databases is another and slightly different challenge for applications. Since databases consist of many tables, all interrelated, the updating is not just a single table, but several. Think commit and rollback, failure during midstream, a need to recover. To do that we first write the transaction to a transaction log file, and then we start updating all the different tables. Once all tables are updated successfully (atomicity), we set a flag in the transaction log to say that a particular transaction has been successfully applied.  

How long do we keep the transaction log file and where should it be backed up? These questions can best be answered by looking at the business impact analysis for the business process, finding the supporting applications, finding the recovery point objective (RPO) and recovery time objective (RTO). For example, if you look at the RPO and find that the business process owner has indicated a zero-tolerance for data loss, you can be assured that transaction logging will be taking place and that transaction logging will most likely be mirrored to a hot site. As an IT auditor, it is your responsibility to determine if the application controls in place satisfy the requirements of the RPO and RTO in the business impact analysis.

Other application controls

A few other areas of concern for application control are how changes to data are normally controlled. Often they are through the application. However, this needs to be checked. Application access control mechanisms, and built-in application controls, normally prevent unauthorized access to data. These controls can be circumvented by direct access to data.  For this reason, direct access to data (write, change or delete access) should be restricted and monitored.

There are a variety of ways to test an application. My favorite is to write test data and then run it through the production system. To accomplish this, you will need to ensure the existence of an integrated test facility (ITF). Don’t forget the Software Development Life Cycle (SDLC) in our discussion. When should you begin testing an application? As an auditor, you will want to make sure that you begin your testing of the application as soon as individual units are finished, which you can call pre-integration testing.

There are five different online auditing techniques for online applications:

• Systems control audit review file and embedded audit modules (SCARF/EAM)
• Snapshots
• Audit hooks
• Integrated test facility
• Continuous and intermittent simulation (CIS)

You would use SCARF/ERM when the complexity is very high and regular processing cannot be interrupted. An ITF would be used when the complexity is high and it is not beneficial to use test data. Snapshots give you an audit trail like taking a lot of snapshots and placing them end to end to get a movie.  CIS is for medium complexity when you have transactions meeting certain criteria, which need to be examined. And audit hooks are for those low complexity tasks when you only need to look at selected transactions or processes.

Applications are here to stay. Some large (SAP, PeopleSoft) and some small (QuickBooks). There will always be applications and there should always be auditors to check that the controls are in place to ensure CIA.

Sources

ISACA, http://www.isaca.org/