Veterans Affairs banner with U.S. FlagVeterans Affairs banner with U.S. Flag

Office of Budget

Fiscal Year 2005 Performance and Accountability Report
Published November 15, 2005

Back to Table of Contents

Assessment of Data Quality

VA's ability to accomplish its mission is dependent on the quality of its data. Each day, VA employees use data to make decisions that affect America's veterans. In delivering medical care, processing benefits, and providing burial services, data accuracy and reliability are paramount.

Each program office has initiated specific actions to improve data quality to better support business planning and day-to-day decisionmaking. In addition, the Office of the Inspector General (OIG) has conducted audits to determine the accuracy of our data. We consider OIG reviews to be independent and objective. The following discussion describes in detail the actions each VA administration has taken to improve its data quality.

Veterans Health Administration

VHA has focused on data reliability, accuracy, and consistency for the past several years. The principles of data quality are integral to VHA's efforts to provide excellence in health care. In 2001 the Under Secretary for Health commissioned a high-level, cross-cutting task force on data quality and standardization whose membership includes the chief officer from VHA's Office of Quality and Performance, the Assistant Deputy Under Secretary for Health, and officials from the Chief Network Office and the Office of Information. This task force has focused on strategic planning to provide consistent definitions of clinical and business data for more effective clinical and organizational decision support. The members seek collaboration with other parties including DoD, Indian Health Service, private sector health care providers, and standards organizations.

VHA's commitment to quality data was confirmed by the results of an OIG audit of the validity of data collection of the quality measures that VHA tracks - Clinical Practice Guidelines Index and Prevention Index II. The report acknowledged a high degree of accuracy. The OIG made no recommendations. VHA continuously monitors data accuracy to ensure these high standards are maintained.

VHA has long been recognized as a leader in documenting credentials and privileges of VA health care professionals. In 2001 VHA implemented a new electronic data bank, VetPro. This database dramatically improved VHA's ability to ensure timely and appropriate credentialing of health care professionals. VetPro promotes and demonstrates to other federal and private agencies the value of a secure, easily accessible, valid data bank of health professionals' credentials. In 2004 VHA and DoD launched a study into the merits of integrating DoD's system for credentialing and privileging, Centralized Credentials and Quality Assurance System, with VHA VetPro. The study resulted in recommendations favoring continued collaboration with a goal of accomplishing future integration.

VetPro improves the process of credentialing and privileging by:

  • Establishing a secure, accessible, valid electronic database.
  • Ensuring appropriate credentials for clinical roles of practitioners.
  • Allowing verification of practitioners' track records.

The VHA Data Consortium addresses organizational issues and basic data quality assumptions. The consortium works collaboratively to improve information reliability and customer access for the purposes of quality measurement, planning, policy analyses, and financial management. The ongoing initiatives and strategies address data quality infrastructure, training and education, personnel issues, policy guidance, and data systems.

The VHA data quality coordinator and data quality workgroups provide guidance on data quality policies and practices. Several initiatives support the integrity and data quality of coding including:

  • Development of strategies and standard approaches to help field staff understand the data content and meaning of specific data elements in VHA databases.
  • Participation in VHA's data standardization activities that involve the standardization of VHA's clinical and administrative data in support of critical activities including VA's Health Data Repository program and the Clinical and Health Data Repository data sharing and interoperability project (a collaborative effort between VA and DoD).
  • Development of coding resources for field facilities, including negotiating the purchase of QuadraMed products to support coding and billing. QuadraMed is providing its Veterans Health Information Systems and Technology Architecture (VistA)-integrated encoder and bill scrubbing software products and training to all VA medical centers. The use of these products is mandatory at all VA sites. The software products and services enable the hospitals to more efficiently manage their revenue cycle.
  • Completion of VistA software revisions to accommodate the requirements of the Health Insurance Portability and Accountability Act for use of code sets involving healthcare claims.

To support the need for guidance in medical coding, VHA established the Health Information Management (HIM) Coding Council, comprised of credentialed expert coders with support from VHA HIM Central Office staff to provide research and response to coding questions within 24 hours. The council has completed an update to the national coding handbook, which provides expert guidance to field facilities. Additional initiatives include:

  • "Close Encounters" and "Data Quality Highlights" newsletters for field staff guidance and information.
  • Ongoing, periodic training programs on such topics as national standard code set updates and refresher training in specific areas such as orthopedic coding.
  • Standardization of electronic encounter forms including documentation templates.

The Patient Financial Services System (PFSS) project is the pilot implementation of a commercial billing and accounts receivable system in VHA. This project is designed to incorporate business process improvements and commercial information systems that are proven in the private sector. The project will introduce commercial business practices and technology into VA through a VISN pilot project comprised of VA best practices and commercial best practices. The objectives of the pilot are to implement a commercial product and study a) the effects on collections, b) improvements to the business process, and c) information systems in a single test environment. Ultimately, the long-term strategy is to develop a scalable solution, which includes both a commercial solution and VA applications that can be implemented in all networks.

VHA completed the implementation of a national Master Patient Index (MPI). The MPI provides the ability to view patient clinical and administrative data from various VA medical facilities via the remote data view functionality within the Computerized Patient Record System. The MPI provides the mechanism for linking patient information from multiple clinical, administrative, and financial records across VHA health care facilities, enabling an enterprise-wide view of individual and aggregate patient information. The Federal Health Information Exchange (FHIE) effort utilizes the MPI via a secure framework in order to share patient data from DoD for separated servicemembers from active duty, reserves, and the National Guard. This project facilitates the sharing of clinical information including medications, discharge summaries, and laboratory data with our providers as those active duty members transition to VA care.

VHA is examining its current health information processing environment to plan how to best implement improvements over the next 5 years. As part of this process, VHA is assessing:

  • What a high-performance automated health system needs to provide.
  • What the ideal health and information system would look like.
  • What the advantages and disadvantages of our current system are.
  • How best to use a phased approach for moving from the current to the ideal environment.

Currently VHA is enhancing the VistA platform by completing the Decision Support System and implementing VistA Imaging. Given funding availability, mid/long-term efforts will include development of a comprehensive health database that will be timely and universally accessible across the full continuum of care settings. This platform will provide the basis for enhancements to eligibility/enrollment processing packages leading to attainment of One VA goals, the reengineering of the VistA Scheduling Package, and enhancements/improvements to the billing and fee basis systems.

VHA established a data standardization program to implement a common language for all VHA providers and facilities. The program enables sharing of commensurate data among VHA, DoD, and other health care providers. The availability of commensurate data will increase patient safety by ensuring that all clinical decisions are based on the patient's complete medical record; reduce costs and minimize the likelihood that duplicate tests and procedures will be performed; and improve data quality, aggregation, and reporting by ensuring the consistent interpretation of data across all VHA facilities.

VHA's HealtheVet-VistA project is focused on replacing the existing VistA legacy health care information system by rehosting, enhancing, and/or reengineering current health information applications on a modern robust technology platform. This effort will enrich the functionality currently available, benefiting veterans, clinical care providers, and the general public by expanding the availability and use of health care information. When fully implemented, HealtheVet-VistA will provide veterans access to their personal health record through the MyHealtheVet component and make these data available to the veterans' health care providers, enabling the veterans and health care providers to access and share the health record, access trusted health information, and access key supportive services including prescription drugs and appointments. HealtheVet-VistA will provide the transition to a veteran-centered health care system that will establish longitudinal electronic health records and track veteran visit history including their problems, orders, results, and treatments, and documentation across all visits enabling VA clinical care providers to have immediate access to critical information regardless of which facility the veteran visited.

Veterans Benefits Administration

VBA continues to focus on data reliability and validity in all facets of its operations from claims processing to FTE hiring patterns. Whether data are collected and housed in legacy systems or a data warehouse environment, the output must be accurate and consistent to be effective. Managing the accuracy of these data requires an ongoing commitment to data quality methods and strategies across all business lines. In 2005 VBA again invested resources in support of this commitment.

The Office of Performance Analysis and Integrity (PA&I) reports directly to the Under Secretary for Benefits. PA&I assesses data for completeness, validity, consistency, timeliness, accuracy, and appropriateness of use as indicators. These data are extracted from VBA's systems of record (e.g., Benefits Delivery Network) and are imported into an enterprise data warehouse. All reports are developed using business rules provided by the respective VBA business lines.

Prior to release, each report is subject to a validation process to ensure accuracy and adherence to the business rules. Specific data validation reviews are conducted throughout the year, and data anomalies are routinely investigated and brought to resolution. VBA's ongoing efforts to maintain data accuracy include reviews of definitions and the associated data related to those definitions. Below are several of the projects and approaches used by the business lines and PA&I as part of VBA's data quality practices.

  • VBA continues to use an online application, which allows all field offices to download timely and consistent information useful to the operations of that office. The data warehouse integrates the ability to convert large quantities of select information into a spreadsheet format for further analysis.
  • The Gulf War Veteran Information System affords trend data on population growth for policy purposes, including those dealing with post-traumatic stress disorder and amyotrophic lateral sclerosis.
  • The Inventory Management System (IMS) allows employees, coaches, and Veterans Service Center managers to be proactive in workload management through timely and accurate access to integrated information. In order to continually improve IMS, VBA regularly reviews the system for accuracy. One recent review focused on the Evidence Receipt Time. VBA is in the process of modifying the system to gather the necessary data to accurately reflect this processing cycle time.
  • The Fiduciary-Beneficiary System (FBS) provides Fiduciary program personnel and their managers with a database and diary system for the records of incompetent beneficiaries. It also generates field examination requests and accounting due letters as well as maintains workload and timeliness data. Through a series of standard listings and reports, as well as specialized query requests to the database, it allows for systematic workload and inventory management. FBS can generate monthly random samples of claims for local review, and the completed work products for the prior month are used to select cases for national review. This random sample approach allows managers and field staff to review claims systematically, saving both time and resources. A review of the methodology used in calculating the completed and pending cases in this system determined that all data and reports were complete and valid.
  • Corporate WINRS is a comprehensive case management system used to maintain complete case histories, generate forms and letters, control payments, and assist in scheduling and tracking appointments for the Vocational Rehabilitation and Employment (VR&E) program. VR&E Intranet reports are continuously refined for regional offices and Central Office to monitor and track this workload data. These reports and other data received from Central Office are released to the regional offices and provide a mechanism to validate the information for accuracy and discrepancies.
  • The Insurance Payment System ensures all manual transactions that result in disbursement (e.g., death award, loan, cash surrender) and all changes to bank data used for direct deposit are second-party verified by an independent staff. This system maintains daily counts of receipts and disbursements by the Insurance fund. Each year random system payments are sampled for accuracy and quarterly reports are reviewed to resolve questionable conditions, such as payments to two veterans at the same address.
  • Since the mid-1990's, VBA has developed a comprehensive program of customer satisfaction surveys for all of its major business lines. Surveys provide feedback on all aspects of the compensation and pension claims process, education benefits, VA home loans, transactions related to insurance policyholders, and the VR&E program. These surveys produce statistically valid performance data at the national and local regional office levels. The surveys are professionally designed to measure all aspects of the business process as experienced by the veteran or family member. Through extensive use of focus groups, cognitive labs, piloting, and pre-testing, the surveys are thoroughly tested and modified, and continue to be improved. These annual mail surveys follow the industry standard for pre-notification and follow-up reminders, resulting in high response rates. Capturing these comparable data within each business line facilitates trend analyses. PA&I conducts special analyses showing key drivers of customer satisfaction and comparisons of performance among regional offices to continue the focus on service improvements.

In addition, PA&I conducts workload and performance reviews on a regular basis. This information is reported at the Deputy Secretary's monthly performance reviews where data are discussed for accuracy and consistency.

National Cemetery Administration

Experience and recent historical data show that about 80 percent of those interred in national cemeteries resided within 75 miles of the cemetery at the time of death. From this experience, NCA considers eligible veterans to have reasonable access if a burial option (whether for casketed or cremated remains) is available within 75 miles of the veteran's place of residence. NCA determines the percent of veterans served by existing national and state veterans cemeteries within a reasonable distance of their residence by analyzing census data on the veteran population. Arlington National Cemetery, operated by the Department of the Army, and Andrew Johnson National Cemetery and Andersonville National Cemetery, operated by the Department of the Interior, are included in this analysis. For 2001 and 2002, actual performance was based on the VetPop2000 model using updated 1990 census data. Since 2003, actual performance and the target levels of performance have been based on a revised VetPop2000 model using 2000 census data. Projected openings of new national or state veterans cemeteries and changes in the service delivery status of existing cemeteries are also considered in determining the veteran population served. (Multiple counts of the same veteran population are avoided in cases of service-area overlap.) In 1999 the OIG performed an audit assessing the accuracy of the data used for this measure. Audit results showed that NCA personnel generally made sound decisions and accurate calculations in determining the percent of veterans served by a burial option. Data were revalidated in the 2002 report entitled Volume 1: Future Burial Needs, prepared by an independent contractor as required by the Veterans Millennium Health Care and Benefits Act, P.L. 106-117.

NCA collects data monthly on the timeliness of marking graves through field station input to the Burial Operations Support System. After reviewing the data for general conformance with previous report periods, headquarters staff validates any irregularities through contact with the reporting station.

Since 2001 NCA has used an annual nationwide mail survey to measure the quality of service provided by national cemeteries as well as the appearance of national cemeteries. The survey provides statistically valid performance information at the national and regional (Memorial Service Network) levels and at the cemetery level for cemeteries having at least 400 interments per year. The survey collects data annually from family members and funeral directors who recently received services from a national cemetery. To ensure sensitivity to the grieving process, NCA allows a minimum of 3 months after an interment before including a respondent in the sample population. VA headquarters staff oversees the data collection process and provides an annual report at the national level.

NCA has established an Organizational Assessment and Improvement Program to identify and prioritize improvement opportunities and to enhance program accountability by providing managers and staff at all levels with one NCA "scorecard." As part of the program, assessment teams conduct site visits to all national cemeteries on a rotating basis to validate performance reporting.

Office of Inspector General (OIG) Performance Audits

The OIG made an assessment of the Department's data quality in the Major Management Challenges section of this report.