FY 2006 Performance Budget Justification

Appendix C. Project Performance Review

ARC's project performance measurement program was designed to accomplish two primary objectives. The first is compliance with the GPRA in measuring the outputs and outcomes of ARC projects. The second objective is creation of a process that allowed for both feedback from grantees and analysis of funded projects in an effort to help improve programming.

Measuring project performance involves three components:

Project data collection and analysis through use of a management information system;
Site visits to validate actual outcomes of a sample of projects; and
Independent project evaluations.

These three components together support GPRA reporting and compliance, as well as help ARC glean "lessons learned" from previously funded grants. By structuring the program in this manner, ARC has made the GPRA a management tool and a valuable resource in determining program effectiveness.

Performance results for each of ARC's goal areas across recent years are presented below.

ARC has a limited number of specific annual performance measures that demonstrate progress toward achieving long-term program targets. The following are examples of these measures:

Intermediate Term Measure: Jobs created or Retained—2 Year Review. Two years after a project is funded ARC staff makes a field validation visit to 10-15% of the total number of projects funded in order to obtain information on the intermediate results for the selected projects. ARC obtains information from field visits regarding the percentage of the performance target achieved for infrastructure jobs created or retained within 2 Years as a result of ARC investments (sample of completed projects). Recent results are noted below.

*Year*	*Target*	*Actual*	*2 Yr Intermediate Results-% of Target*
1998	747	803	107% Exceeded Goal
1999	4,288	4,421	103% Exceeded Goal
2000	1,097	985	90% of Goal Achieved
2001	2,003	2,015	101% Exceeded Goal
2002	1,336	1,452	109% Exceeded Goal
2003	Results to be validated in FY2005

Long Term Measure: Jobs Created or Retained—5- Year Review. ARC contracts with outside consultants to evaluate the overall performance of a major investment category every five plus years. The Jobs created or retained investment category was evaluated by The Brandow Company in 2000. Another multi-year evaluation will be conducted in 2005-2006. The results for the 1990-1997 sample set of projects are reported below.

*1990-1997*	*Target*	*Results*	*% of Target*
Water/sewer projects	15,354	20,031	130% Exceeded Goal
Industrial park projects	5,829	5,716	98% of Goal Achieved
Business Incubators	550	2,358	428% Exceeded Goal
Access Road projects	2,174	3,459	159% Exceeded Goal

Intermediate Term Measure: Households Served with Water/Sewer -2 Year Review. Two years after a project is funded ARC staff makes a field validation visit to 10-15% of the total number of projects funded in order to obtain information on the intermediate results for the selected projects. ARC obtains information regarding the percentage of the performance target achieved for households served with water/sewer within 2 years as a result of ARC investments plus other funds (sample of completed projects). Recent results are noted below.

*Year*	*Target*	*Actual*	*2 Yr Intermediate Results- % of Target*
1998	4,887	5,566	107% Exceeded Goal
1999	974	872	90% of Goal Achieved
2000	504	576	114% Exceeded Goal
2001	7,859	8,395	107% Exceeded Goal
2002	1,667	1,649	99% of Goal Achieved
2003	Results to be validated in FY2005

Long Term Measure: Households served with Water/Sewer –5 Year Review. Households served with water and sewer within a 5 plus year period (sample of completed projects) as a result of ARC and other investments were evaluated by The Brandow Company in 2000. Another multi-year evaluation will be conducted in 2005-2006. The results for 1990-1997 are reported below.

*1990-1997*	*Target*	*Results*	*% of Target*
New Households served	1,928	4,553	236% Exceeded Goal
Existing Households served	12,136	13,148	108% Exceeded Goal
New Businesses served	48	191	398% Exceeded Goal
Retained Businesses served	447	443	99% of Goal Achieved

Intermediate Term Measure: Students and Trainees—2 Year Review. Two years after a project is funded ARC staff makes a field validation visit to 10-15% of the total number of projects funded in order to obtain information on the intermediate results for the selected projects. ARC obtains information from field visits regarding the percentage of the performance target achieved for education projects within 2 Years as a result of ARC investments (sample of completed projects). Recent results for ARC investments that document participating students are shown below. Student improvements are evaluated in the five-plus multi-year study.

Participating Students:

*Year*	*Target*	*Actual %*	*Results Achieved*
1998	3,998	5,088	127% Exceeded Goal
1999	4,547	4,498	99% of Goal Achieved
2000	5,097	5,114	100% Exceeded Goal
2001	2,497	2,343	94% of Goal Achieved
2002	7,370	6,627	90% of Goal Achieved
2003	Results to be validated in FY2003

Participating Trainees

*Year*	*Target*	*Actual*	*% Results Achieved*
1998	645	918	142% Exceeded Goal
1999	937	819	87% of Goal Achieved
2000	1,050	836	80% of Goal Achieved
2001	3,566	6,166	173% Exceeded Goal
2002	1,667	1,330	80% of Goal Achieved
2003	Results to be validated in FY2005

Long-term Measure: Students and Trainees: 5-Year Review. Over a five plus year period ARC requests an outside consultant to review the overall performance of a major investment category. The vocational education and workforce training activities of the ARC were evaluated by Westat in 2000. Another multi-year evaluation will be conducted in 2006-2008. The results for the 1995-1999 sample set of projects are reported below. The ARC has a target of achieving 70% of its objectives in seven activities categories.

*1995-1999*	*Percent Achieved*
Services provided to individuals	65% Not met
Services provided to Communities	83% Exceeded
Barrier reduction	80% Exceeded
Skill attainment	83% Exceeded
Degree attainment	57% Not met
Impact employment status	86% Exceeded
Impact Community Viability	70% Met

Long Term Measure: Health Care Professionals Placed in the Region—annual results. ARC assists local communities with the placement of physicians and other health care professionals in health professional shortage areas (HPSAs). Recent results of ARC activities are noted below.

*Year*	*Target*	*Actual*	*Results-% of Target*
2002	100	96	96% of Goal Achieved
2003	60	79	132% Exceeded Goal
2004	50	43	86% of Goal Achieved

Long Term Measure: Miles of Appalachian Development Highway Opened to Traffic---annual results. The current target of opening 25 miles of highway per year was set assuming at least $500 million available for ADHS funding each year. It should be noted that opening a highway involves a 4 to 8 year period of investment in design, right-of-way purchase and construction. For the period 2000-2004 an annual average of 26 miles was opened to traffic.

*Year*	*Target*	*Results*	*% of Target*
2004	25	5	20%

Long Term Measure: Infrastructure investments: private and other —1990-1997.

Private sector Investments and non-ARC investments are determined through an evaluation conducted by an outside contractor for a sample of ARC Infrastructure Projects. The 1990-1997 evaluation was conducted by The Brandow Company and the results are reported below. Another long term measure evaluation will be conducted in FY2005-2006.

Business Infrastructure and Job Creation/Retention. The target ARC to private sector investment ratio for business infrastructure projects is 1 to 15.

1990-1997	ARC Funding	Private Funding Target	Actual Private Investment
	$14,599,922	$175,199,060	$2,195,350,00 Exceeded Target

Residential Infrastructure and Households served. The target ARC to public sector long-term investment ratio is 1 to 3 for residential infrastructure.

*1990-1997*	**ARC Funding Target**	*Public Sector Investment*	*Actual Public Investment*
	$3,861,575	$11,584,725	$10,210,531 88% of Goal Achieved

Overview---All Infrastructure Project Types. The target ARC to non-ARC long term funding ratio is 1 to 10 for all ARC residential projects in distressed areas. This ratio was established in the FY 2005 Performance Budget Justification.

*1990-1997*	*ARC Funding Target*	*Private Investment*	*Actual Private Investment*
	$32,433,047	$324,330,470	$3,088,642,000 Exceeded Target

Annual Measure: Each year ARC targets a 1 to 4 initial investment ratio for ARC funds to initial private sector investments in all ARC Projects.

Year	*ARC Investment*	*Target Initial Private Investment*	*Initial Private Investment*
2001	$54,037,027	$216,148,100	$289,355,460 Exceeded Target
2002	$65,527,968	$262,111,870	$114,911,100 44% of Target
2003	$65,956,352	$263,825,400	$331,965,400 Exceeded Target
2004	$66,310,422	$265,241,688	$490,247,346 Exceeded Target

Table of Contents | Report in PDF (2.2 MB)