Program evaluations play an important role in VA’s strategic planning process. Specifically, program evaluations are used to assess, develop, and/or update program outcomes, goals, and objectives and to compare actual program results with established goals. Program evaluations assess and possibly contribute to the revision of general goals and objectives included in the Department’s Strategic Plan. The VA Strategic Plan will include a section describing how program evaluations impact the Department’s goals and objectives, along with a schedule for future evaluations. Outcome measures identified or enhanced during the conduct of program evaluations are included in annual performance plans and will be used to continually refine the Strategic Plan. VA’s goal is to re-evaluate programs on a five to seven year cycle.

Program evaluations assess:

• the extent to which program outcome goals are being met and the extent to which current performance affects program outcomes;

• the interrelationships between VA programs and other Federal programs to determine how well these programs complement one another;

• the needs and requirements of veterans and their dependents in the future to ensure the nature and scope of future benefits and services are aligned with the changing needs and expectations of veterans and their dependents; and

• the adequacy of outcome measures in determining the extent to which the programs are achieving intended purposes and outcomes.

In addition, program evaluations can fill existing data gaps, particularly relating to outcome information that can only be obtained from veterans and beneficiaries. These studies also provide an opportunity to objectively and independently analyze VA programs and yield information useful in developing policy positions. Proposals for future benefit packages and improvement in existing programs evolve from the process of evaluating programs.

Consistent with legislative intent and 38 CFR §1.15, the Office of Planning and Analysis, an organizational entity not responsible for program administration, is responsible for the operational aspects of program evaluation providing an unbiased, third-party perspective. Within VA, most program evaluations are conducted through contracts, which further enhances third-party objectivity. In all cases, the evaluations are managed using a team approach that includes program officials. For each evaluation, an evaluation team develops the statement of work and oversees the execution of the contract. Pre-evaluation planning and post-evaluation discussion of results and related recommendations involve our major stakeholders including OMB, Congressional staff, veteran service organizations, and the VA National Partnership Council.

The evaluation of VA’s education program will be completed in FY 2000. An evaluation of VA’s programs for survivors of veterans who die of service-connected disabilities and who die on active duty will also be completed in FY 2000, and an evaluation of VA’s cardiac care program will be completed in FY 2001.

Strategic Goal: Restore the capability of disabled veterans to the greatest extent possible and improve their quality of life and that of their families.

Benefits for Survivors of Veterans with Service-Connected Disabilities (started in FY 1999 and scheduled for completion in FY 2000) -- This evaluation makes a determination of how effectively the Dependency and Indemnity Compensation (DIC) program as well as the Veterans Group Life Insurance, Servicemembers’ Group Life Insurance, Service Disabled Veterans Insurance, and Veterans Mortgage Life Insurance programs are meeting their statutory intent. The study also identifies assistance available to survivors of severely disabled veterans from other VA and non-VA federal benefit and insurance programs and assesses how well their transition needs are met.

Prosthetics and Sensory Aids Program (PSAS) (scheduled to start in FY 2000) -- The mission of PSAS is to assist physically disabled veterans to achieve maximum functionality by serving as case managers and providing prosthetic equipment, sensory aids and assistive devices. A management study evaluated the efficiency, productivity and effectiveness of the program. The program evaluation will determine whether or not PSAS is meeting its intended outcomes.

Vocational Rehabilitation and Employment Programs (scheduled to start in FY 2002) -- This program serves 50,000 disabled veterans to overcome employment handicaps by providing vocational and educational counseling services, training, and education, and employment services. The effectiveness of the program in meeting its outcome goals will be evaluated.

Environmental Medicine Programs, e.g., Agent Orange, Ionizing Radiation, Gulf War (scheduled to start in FY 2001) -- Physical exams and medical treatment are provided to veterans who have been exposed to Agent Orange, ionizing radiation, or who served in the Gulf War. The program will be evaluated for effectiveness and adequacy.

Strategic Goal: Ensure a smooth transition for veterans from active military service to civilian life.

Education Programs (started in FY 1999) -- The Montgomery GI Bill, the Montgomery GI Bill for Selected Reserves, and the Dependents Educational Assistance programs supported more than 415,000 persons in training in FY 1997. These programs were evaluated to determine the effectiveness of the programs in meeting their intended outcomes. Results were viewed in context of the changing nature and delivery modes of education and training in the early 21^st Century.

Strategic Goal: Honor and serve veterans in life and memorialize them in death for their sacrifices on behalf of the Nation.

Burial Benefits and Services (begun in FY 2000) – Burial programs will be evaluated and this evaluation will meet the requirements of the Millennium Act (PL 106-117).

Disability Compensation for Veterans Programs (scheduled to start in FY 2002) -- In FY 1997, compensation benefits were paid to more than 2.2 million veterans. This program evaluation will assess the degree to which VA is meeting intended program outcomes as well as the adequacy of outcome measures.

Non-Service Connected Pension for Veterans and Survivors (scheduled to be begin in FY 2000) – This evaluation will assess the extent to which the pension program is meeting the needs of veterans and their survivors.

Strategic Goal: Contribute to the public health, socio-economic well being and history of the Nation.

Cardiac Care Programs (scheduled for completion in January 2001) -- This program evaluation will assess the attainment of program outcomes; how VA’s level of services compares to the private sector; the effect on outcomes of shifting to outpatient care; how variances in procedures affect outcomes; how level of services vary across VISNs; and how VA’s coronary artery bypass graft and catheterization rates compare with the private sector, among others.

Mental Health – Psychosocial Programs, e.g., Homeless (includes VBA), Substance Abuse, Post Traumatic Stress Disorder, Readjustment Counseling (scheduled to start in FY 2001) -- These programs, at all levels of the continuum of care, address patients with mental health needs, as well as outreach, coordination of care, primary care, inpatient treatment, and counseling. These programs will be evaluated to see how well they are achieving their outcomes.

VHA’s Northeast Program Evaluation Center will conduct studies of the following programs that are relevant to performance goals and measures contained in this plan: Health Care for Homeless Veterans (HCHV), Domiciliary Care for Homeless Veterans (DCHV), Specialized Post Traumatic Stress Disorder (PTSD), Intensive Psychiatric Community Care (IPCC), Compensated Work Therapy (CWT), VA Mental Health Report Card (performance of general psychiatry and substance abuse programs), and HUD-VA Supported Housing (HUD-VASH) Program.

The purpose of the HCHV study is to monitor outreach, case management, contract residential treatment and transitional housing services provided to homeless veterans by the HCHV program. Appropriate use of program resources is monitored by monthly reports of program staffing and by reports of expenditures for contract residential treatment. Program activities are monitored by documentation of intake assessments and outpatient encounters. Veteran outcomes (addressing program effectiveness) are measured at the time of discharge from contract residential treatment.

The purpose of the DCHV study is to determine whether the program is reaching the intended target population (homeless, severely ill veterans), providing appropriate services, as well as whether the program is effective in improving the health status and quality of life for these veterans.

PTSD among veterans is of particular relevance to VA’s mission because it is a chronic disorder that causes substantial psychological suffering and social dysfunction resulting directly from their military service. In response to the need to treat this disorder, VA has established a national network of programs that specialize in the treatment of PTSD. The Under Secretary for Health has mandated that the costs, accessibility and effectiveness of these programs be monitored and evaluated on a continuous basis.

The purpose of the IPCC study is to monitor the clinical and cost effectiveness of VA intensive community-based case management services for veterans with serious mental illness. National program performance monitors address client characteristics, team structure and fidelity, service delivery and utilization, and clinical outcomes.

The CWT study will determine whether the program is achieving the major goal of maximizing a veteran’s level of functioning and preparing him or her for successful re-entry into the community. The study will provide a description of the status and needs of veterans in CWT, and will assure program accountability while identifying ways to improve the program.

The National Mental Health Program Performance Monitoring System Report, an annual VA Mental Health Report Card, has been generated in each of the past five years to comprehensively document population coverage, the quality and costs of inpatient and outpatient mental health care, overall economic performance and patient satisfaction.

The purpose of the HUD-VASH) study is to monitor intensive case management and permanent housing services provided to homeless veterans by the HUD-VASH program. Currently, about 1,500 veterans are permanently housed with support of the program, which includes a Section 8 rental assistance voucher provided by HUD. Appropriate use of program resources is monitored by monthly reports of program staffing and voucher usage. Program activities are monitored by documentation of case management activities during establishment of housing and by outpatient encounters. Veteran outcomes are measured on a long-term basis (for up to five years) through case manager reports and veteran reports of satisfaction with the program. The evaluation includes a four site randomized control trial of rental assistance and intensive case management.