An Overview: Oregon Shines II and Oregon Benchmarks
Richard D. Young
Introduction
The State of Oregon has the most enduring and continuous key indicator system in the United States. Established in 1988 as a statewide economic planning initiative, Oregon Shines resulted initially in the publication of an economic strategy for Oregonians,[1] a strategy intended to span the following two decades. In 1989, as a counterpart to Oregon Shines, the state legislature passed legislation establishing the Oregon Benchmarks, a comprehensive indicator system to gauge the progress of the strategic vision laid out in the first Oregon Shines report. Over the past 17 years, Oregon Shines (now Oregon Shines II) and Oregon Benchmarks have evolved into a comprehensive strategic plan and set of select indicators or benchmarks that have been acknowledged and adapted, to varying degrees, by dozens of state and local governments across America.
In this brief paper, the Oregon system will be examined in summary fashion—its background, its processes, and lessons learned (i.e., “implications” for South Carolina). Additionally, a corollary work of Oregon Benchmarks—Competitive Index 2005—dealing with various state rankings will be reviewed. The aims of this paper are thus twofold: 1) to provide a summary of the Oregon system, and 2) to review those “lessons learned” from the Oregon Benchmarks experience that might be useful or transferable to other comprehensive key indicator systems.
Background
Oregon Shines and Oregon Benchmarks are overseen and governed by the Oregon Progress Board which was established in legislation in 1989. Today, the board consists of 12 members and is chaired by the governor. The remaining members consist of leading citizens (9) appointed by the governor and select legislative members (2).
Initially, the Progress Board was located in the governor’s office and consisted of nine “volunteer members.” The board met regularly and was supported by a small staff. The board was charged to develop a set of benchmarks for legislative approval and to report bi-annually on progress towards these benchmarks. With considerable citizen and expert input, the Progress Board initially established a list of 158 benchmarks.[2] Later, baseline data were developed in 1990; a set of benchmarks received official legislative approval in 1991; and, the first progress report was published and disseminated in 1993.
Notably, the 1993 report produced by the Progress Board was used by the governor to help establish budget priorities for his bi-annual budget for the fiscal years 1995-97. However in 1994, a newly dominated Republican state legislature clashed with then Democratic governor, John Kitzhaber, and in 1995, the authorizing legislation for the Progress Board and Oregon Benchmarks was effectively “sunset.” Nevertheless, the governor re-established the board and benchmarking process by executive order. From 1995 through 1997, the Progress Board and its staff addressed the concerns of the Oregon legislature and made numerous refinements to the benchmarking process. In early 1997, Oregon Shines II was established and the benchmarks were reduced and fine-tuned to 92. With these improvements, confidence in the Progress Board and its work was restored and legislation re-authorized a new, improved Oregon Progress Board in the late spring of 1997.[3]
In 2001, additional legislation moved the Progress Board to Oregon’s Department of Administration Services[4] and required the linkage of the benchmarks to state agency performance measures. The legislation also added legislative membership to the Progress Board.
In 2002, during a special session of the legislature on state appropriations, the Oregon legislature cut board funding to help contend with a substantial revenue shortfall. However, the governor funded the Progress Board with executive branch set-aside funding. In 2003, most state appropriations were restored by the legislature to the board.
The Oregon Progress Board and Oregon Benchmarks continue to perfect its system and functioning, with emphasis on serving as a compatible adjunct performance measurement mechanism used by state agencies and departments. Of late, the board has assisted the state’s Department of Administration Services in reviewing 87 state agencies in how their performance measures link to the current 90 Oregon Benchmarks.
The Process
The Oregon Shines II and Oregon Benchmarks are together an integrated system of 90 indicators that that assess the progress of Oregon with regard to three sweeping goals related to economic, social and environmental issues. The Oregon Progress Board is again the principal organization that provides governance and oversight for this integrated system. The 12-member board, chaired by the governor, is assisted by a staff of professionals who gather data, monitor progress, and prepare reports. The most recent comprehensive report—that measures the progress of strategic goals via 90 benchmarks—is entitled, Is Oregon Making Progress? The 2003 Benchmark Performance Report to the Legislative Assembly.[5]
To understand Oregon’s integrated process, it is first important to point out Oregon’s strategic vision or what is known as Oregon Shines II. Oregon’s strategic vision is capsulated in a terse, broad statement; namely, “A prosperous Oregon that excels in all spheres.” In turn, this strategic vision or Oregon Shines II consists of three far-reaching goals—1) quality jobs for all Oregonians, 2) engaged, caring and safe communities, and 3) healthy sustainable surroundings. According to the Progress Board,
Oregon Shines II is based on the assumption that the social and economic well being of Oregonians depends on the inter-connectedness of quality jobs, a sustainable environment, and caring communities or what is called the “Circle of Prosperity.”[6]
Figure 1. Circle of Prosperity
Source: Oregon Progress Board
(2003, March). The 2003 benchmark
performance report. Salem, OR: Author, p. 5.
The 90 benchmarks are of course designed to measure the progress of achieving the vision and goals of Oregon Shines II. In this way, the 90 benchmarks represent the best indices available, linked to reliable and accepted sources approved by the Progress Board, to assess progress “that Oregon excels in all spheres of life.” Examples of the benchmarks include:
▪ (Economy) BK #1. Percent of Oregonians employed outside the Willamette Valley and the Portland tri-county area.
▪ (Education) BK#18. Percent of children entering school ready to learn.
▪ (Civic Engagement) BK# 30. Percent of Oregon adults who volunteer time to civic, community or nonprofit activities in the last 12 months.
▪ (Social Support) BK# 39. Pregnancy rate per 1,000 females: a.) ages 10-14 and b.) ages 15-17
▪ (Public Safety) BK#61. Overall reported crimes per 1,000 Oregonians: a.) person crimes, b.) property crimes, and c.) behavior crimes.
▪ (Community Development) BK#72. Percent of roads in fair or better condition: a.) state and b.) county.
▪ (Environment) BK#75. Percent of time the air is healthy to breathe for all Oregonians.[7]
Further, it should be noted that the 90 benchmarks are divided among seven topical areas. These areas include 1) the economy [benchmarks1-17], 2) education [benchmarks 18-29], 3) civic engagement [benchmarks 30-38], 4) social support [benchmarks 39-60], 5) public safety [benchmarks 61-67], 6) community development [benchmarks 68-74], and 7) environment [benchmarks 75-90]. Figure 2 illustrates the relationship between the strategic vision and goals of Oregon Shines II and the seven areas of Oregon Benchmarks.
Figure 2. Oregon’s Integrated Indicator System
Vision, Goals, and Major Benchmark Areas
Additionally, each of the seven major benchmark areas is further divided into sub areas, a total of 26 (Figure 3). For example, the area “economy” has five sub areas: 1) business vitality [benchmarks 1-6], 2) economic capacity [benchmarks 7-8], 3) business costs [benchmarks 9-10], 4) income [benchmarks 11-15], and 5) international [benchmarks 16-17]. The remaining six major benchmark areas have a varying number of sub areas: education has three; civic engagement has four; social support has three; public safety has two; community development has three; and environment has five (See Figure 3).[8]
Figure 3. Benchmark Areas and Sub Areas
In the latest benchmarks progress report, The 2003 Benchmark Performance Report, the Progress Board chose not to assign a grade, “A” through “F”, as to the accomplishment of each major benchmark area, sub area, and individual benchmark. Instead, the board, using expert staff interpretation of trend lines, used the terms “Yes” or “No” to indicate a clear and unambiguous trend. Other terms used include “Yes, but” or “No, but” to indicate trends that that must be qualified. Also, the term “Mixed” is used to indicate a trend that is too difficult to call. Figure 4 illustrates this use of terminology.[9]
Figure 4. Example of a Benchmark(s) Assessment
Goal 1: Quality Jobs for all Oregonians
Major Area—Economy: Is Oregon Making Progress? Answer: Qualified yes,
overall.
Sub Area—Business Vitality: No, but.
In business vitality, Oregon improved its national ranking for trading outside its borders, maintained a top tier position for job creation and, despite recent losses, added over 300,000 jobs in the past ten years. On the down side, Oregon has lost ground in the professional services sector, diversification and the urban-rural job split.
BK#1 Employment Dispersion: No. The ten year trend for employment outside the Willamette
valley is worsening.
BK#2 Traded Sector Strength: Yes. Oregon's national rank for trading outside the state
improved from 40th in 1994 to 33rd in 2000.
BK#3 New Companies: No, but. Oregon is consistently highly ranked but has lost some
ground in recent years.
BK#4 Net Job Growth: Overall no. After creating hundreds of thousand of jobs in the 1990s,
Oregon lost over 23,000 jobs from 2000-2001.
a. Urban: No. Oregon
lost over 6,600 urban jobs in 2001. The ten year trend is worsening.
b. Rural: No. Oregon
lost over 4,300 rural jobs in 2001. The ten year trend is worsening.
BK#5 Professional Services: No. Oregon remains a net importer of financial,
insurance, engineering, business and legal services.
BK#6 Economic Diversification: No. Oregon's national rank in economic diversification was
35th in 2000. In 1990 it was 28th.
Source: Oregon Progress Board
(2003, March). The 2003 benchmark
performance report. Salem, OR: Author, pp. 9-11.
This new evaluation system of designating positive or negative progress, with or without qualification, as to trends and accompanying explanatory narration appears to be generally well-accepted and successful. According to the Progress Board, this acceptance extends beyond legislators and policy makers at the state level.
In addition to providing valuable context for policy makers in Oregon, this report is a valuable tool outside of state government. Non-profit organizations use it for planning and grant writing. Local governments use it for planning and policy development. Several counties have local versions of the Oregon Progress Board to strategically plan and track key indicators in their regions. Foundations use benchmark data to help inform their funding decisions.[10]
The Competitive Index 2005
Of significance, a corollary report related directly to many of the Oregon Benchmarks was published by the Progress Board in collaboration with the Oregon Business Council[11] in the fall of 2004 and was entitled, Competitive Index 2005: Indicators of Oregon’s Global Economic Competitiveness. This report essentially consists of 54 tables that “suggest how well Oregon is doing in giving its businesses a competitive edge.”[12] These 54 tables compare Oregon with all other states (incl., the U.S. average, if appropriate) as to seven topical areas: 1) general economic well being, 2) traded sector cluster health [groups of allied industries such as electronics, forest products, etc.], 3) pioneering innovation [R &D, venture capital investment, etc.], 4) people [population, education, etc.], 5) place [cost of living, crime, environment, etc.], 6) productivity [cost of doing business, wages, etc.], and public finance [tax collections, tax burden, etc.]. See Figure 5 for index of all 54 tables.
Figure 5. Index of 54 State Rankings Tables
Source: Oregon Progress Board and Oregon Business Council (2004, Fall). Competitive index 2005: indicators of Oregon’s global economic competitiveness. Salem, OR: Author, Table of Contents.
The importance of these 54 tables is of course that Oregon’s Progress Board and the Oregon Business Council have, using an established formal benchmarking system, put into context Oregon’s relative ranking among other states to assess or compare performance, status, or data—ultimately for purposes of evaluating economic standing. For instance, Table 1 “Personal and Household Income” ranks states as to 2003 per capita income and median household income (3-year average for 2001-2003). This is one table among four other tables that pertains to the “overall economic well being” of states, most importantly, “how Oregon is achieving quality jobs for all Oregonians.” Additionally, it is notable that the data source of Table 1, as with the remaining tables, is acceptable to both the Oregon Progress Board and the Oregon Business Council. Hence, there is no disagreement among policy makers and others as to the accuracy and reliability of the data.[13]
Figure 6. Table 1: Personal and Household Income
Source: Oregon Progress Board
and Oregon Business Council (2004, Fall). Competitive
index 2005: indicators of Oregon’s global economic competitiveness. Salem,
OR: Author, p. 1.
Lessons Learned
As discussed by this author in an earlier paper entitled An Overview of the GAO Report on Key National Indicators, policy makers, and others are looking for trusted data—a comprehensive key indicator system(s)—as to how political jurisdictions (a community, a state, or the U.S.) are doing comparatively speaking in the areas associated with economic, social, and cultural issues. While there are many sources of sound and reliable data, states and localities in particular—with the exception of Oregon, Minnesota, and a hand full of other political jurisdictions—do not have accepted or formal comprehensive indicator systems intact. Rather, nearly all states and localities pick and choose randomly comparative data or indicators that are sometimes unverified, biased, or simply unsound. This is obviously problematic and presents difficulties for accuracy and dependability in understanding data and information or utilizing them for decision making purposes.
Oregon’s Benchmarks offers some lessons for establishing and maintaining an accurate, reliable and, most importantly, an “accepted” comprehensive system of indicators to ascertain or measure the relative or absolute position of a state related to the economy, health, public safety, education, the environment, and so on. The following narrative briefly speaks to these lessons learned.
Clearly identify the aims and audience for the establishment of an indicator system. It is vitally important, based on the Oregon Benchmarks experience, to identify clearly the aims of an indicator system. In the case of Oregon, the aims are to measure the progress of an articulated strategic vision for the state and the accomplishment of three key goals. The audience is specifically the governor, the legislature, and selected stakeholders such as leaders in business, education and environmental affairs. Stakeholders also include both state and local individuals or organizations.
Establish a formal entity for governance and oversight responsibilities. Based on Oregon’s Progress Board, establishment of a board or commission (committee) to govern and oversee an indicator system has several advantages. First, a mix of government elected officials (e.g., the governor and select legislators) and leading private individuals ensure on-going commitment to the significance and utility of a system of indicators. This in turn allows for greater interest and acceptance among system users. Second, a board or commission provides valuable oversight of the system of indicators which allows for formal policy and/or process changes to perfect the indicator system independent of staff or other influences that may bring into questions those of legitimacy, pertinence, or efficacy.
Acquire or utilize a highly professional or expert staff. Oregon has found that it is extremely important to have a competent and specialized staff to support the collection and analysis of data relating to its benchmarks (indicators). Additionally, it is highly beneficial to have a top staff person (a director, manager, or supervisor) who is recognized for their expertise and professionalism, particularly with regard to inter-personal relations and communication skills.
Be prepared to invest sufficient funding for an indicator system. The Oregon Benchmark system has a history of struggling to meet its funding needs. The governor and legislature should both be fully committed to adequately funding an indicator system that clearly meets or satisfies data and information needs for critical decision making purposes. Since Oregon Benchmarks’ funding source is strictly state funds, it may be advisable for other state systems to pursue private or foundation funding sources in addition to state appropriations.
Ensure independence and accountability. As stated earlier, legislation authorizing Oregon Benchmarks was sunset in 1995 due to perceptions that the system “represented a partisan and ideological agenda.”[14] It was restored only after changes to the Progress Board membership and pro-active measures were taken to improve indicator credibility among users, particularly legislators. A proper and respected indicator system should establish procedures and processes that ensure independence and transparency. Any bias or skewing of data will inevitably cause mistrust and bring condemnation.
Identify indicators that will maximize understanding and decision making. Only indicators that are helpful, informative and understandable should be used. At one time, there were over 270 Oregon Benchmarks. The sheer number of indicators caused confusion and misunderstanding, and were basically ineffective for decision making purposes. After years of careful observation and refinement, Oregon narrowed its benchmarks to 90, ostensibly to fit—manageably—within seven major areas, which have been universally accepted as both “meaningful and otherwise helpful.”
Regularly evaluate the indicator system and its effectiveness. The Oregon Progress Board and its staff have come to realize that an indicator system, like Oregon Benchmarks, is something which is dynamic and, as such, requires continuous adjustments or improvements. For example, in its 2003 performance report, discussed previously in this paper, the board modified 17 benchmarks and replaced two relative to its earlier bi-annual report published in 2001.[15] Also, as recently as November 2004, the Progress Board sanctioned an independent report to evaluate the statutorily mandated linking of agency performance measures to the Oregon Benchmarks.[16] Thus, as Oregon has learned, it is necessary to evaluate and update an indicator system—on a regular basis—in order to maintain its effectiveness.
Link the indicator system to state and local government performance. In 2001, Oregon’s Legislative Assembly passed a law requiring the “Progress Board, in consultation with the Legislative Fiscal Office, the Office of the Secretary of State, and the Department of Administrative Services, to develop performance measure guidelines for state agencies.”[17] The chief purpose of this mandate was to link agency performance directly to the Oregon Benchmarks. The linkage or applicability would ensure that state government, and its local extensions and counterparts, would be targeting its various agency missions, goals and objectives to achieving pre-established benchmarks. Hence, the value of Oregon Shines II and its three major goals would be tied ultimately to agency activities. As such, states desiring to establish indicator systems may want to identify linkages with the aims and performance measurement of state and local government agencies and programs.
Conclusion
In this paper, the Oregon Benchmark system has been reviewed briefly as to its history, processes, and lessons learned. With beginnings in the late 1980s, it has evolved into a comprehensive key indicator system that has been recognized for its merits and, in some cases, has been emulated by state and local governments. Of importance, its relatively long-standing existence and evolution have provided many insights or lessons that have and continue to prove to be beneficial to governments attempting to replicate or adapt its processes. This paper has identified eight such lessons. In sum, they are again:
▪ Clearly identify the aims and audience for the establishment of an indicator system.
▪ Establish a formal entity for governance and oversight responsibilities.
▪ Acquire or utilize a highly professional or expert staff.
▪ Be prepared to invest sufficient funding for an indicator system.
▪ Ensure independence and accountability.
▪ Identify indicators that will maximize understanding and decision making.
▪ Regularly evaluate the indicator system and its effectiveness.
▪ Link the indicator system to state and local government performance.
Appendix:
Index of Oregon Benchmarks
|
ECONOMY |
CIVIC ENGAGEMENT |
PUBLIC SAFETY |
|
Business Vitality |
Participation |
Crime |
|
1. Percent of
Oregonians employed outside the
Willamette Valley and the Portland
tri-county Area. |
30. Percent of Oregon adults who
volunteer time to civic, community or nonprofit activities in the last twelve
months. |
61. Overall reported crimes per
1,000 Oregonians: a. person crimes b. property crimes c. behavior crimes |
|
2. Oregon's national
rank in traded sector strength.
|
31. Oregon's national rank in voter
turnout for presidential elections. |
62. Juvenile arrests per 1,000
juvenile Oregonians per year: a. person crimes b. property crimes |
|
3. Oregon's national
rank in new companies. |
32. Percent of Oregonians who feel
they are a part of their community. |
63. Percent of students who carry
weapons. |
|
4. Net job growth: a. urban counties b. rural counties |
Taxes |
64. Percent of paroled adult
offenders convicted of a new felony within
three years of initial release. |
|
5. Oregon's
concentration in professional services
relative to the U.S. concentration
in professional services. |
33. Percent of Oregonians who
understand the Oregon tax system and where tax money is spent. |
65. Percent of juveniles with a new
criminal referral to a county juvenile department within 12 months of the
initial criminal offense. |
|
6. Oregon's national
rank in economic
diversification. |
34. National ranking for state and
local taxes and charges as a percent of
personal income: a. Taxes b. Public Sector Performance |
66. Percent of counties that have
completed a strategic cooperative policing agreement. |
|
Economic Capacity |
Public Sector Performance |
Emergency Preparedness |
|
7. Research &
development expenditures as a
percent of gross state product: a. industry
(public/private) b. academia |
35. Governing magazine's ranking of
public management quality. |
67. Percent of Oregon counties and communities prepared for natural
and technological emergencies or
disasters: a. With hazard data and risk
reduction (natural only) b. With response and recovery
capabilities (for 2001 all counties, Portland,
Beaverton, Gresham and Salem only) |
|
8. Oregon's national
rank in venture capital
investments. |
36. State general obligation bond
rating (Standard and Poor's). |
COMMUNITY DEVELOPMENT |
|
Business Costs |
Culture |
Growth Management |
|
9. Oregon's national
rank in the cost of doing business:
a. labor costs b. energy costs c. tax costs |
37. Oregon's national rank in per
capita state arts funding. |
68. Hours of travel delay per
capita per year in urbanized areas: a. Portland metro b. all other |
|
10. Percent of permits
issued within the target time
period or less: a. air contaminant
discharge b. wastewater discharge |
38. Percent of Oregonians served by
a public library which meets minimum service criteria 86%. |
69. Percent of Oregonians served by
public drinking water systems that meet health-based standards. |
|
Income |
SOCIAL SUPPORT |
Infrastructure |
|
11. Per capita personal income as a
percent of the U.S. per capita income a. metropolitan as a percent of
metropolitan U.S. b. non-metropolitan as a percent of non-metropolitan U.S. |
Health |
70. Percent of Oregonians who
commute during peak hours by means other than a
single occupancy vehicle. |
|
12. Average annual payroll per
covered worker (all industries,1995 dollars): a. urban b. rural |
39. Pregnancy rate per 1,000
females: a. ages 10-14 b. ages 15-17 |
71. Vehicle miles traveled per
capita in Oregon metropolitan areas for local, non-commercial trips. |
|
13. Comparison of average incomes
of top 5th families to lowest 5th families: a. ratio b. national rank (1st = smallest gap) |
40. Percent of babies whose mothers received prenatal care beginning in
the first trimester. |
72. Percent of roads in fair or
better condition: a. state b. county |
|
14. Percent of covered Oregon workers with earnings of 150% or more of the poverty level for a family of four. |
41. Infant mortality rate per
1,000. |
Housing |
|
15. Oregon unemployment rate as a percent of U.S. unemployment
rate. |
42. Percent of two-year-olds who
are adequately immunized. |
73. Percent of households that are
owner occupied. |
|
International |
43. Number of 1st time positive HIV
test results among Oregonians age 13 and older. |
74. Percent of Oregon households
below median income spending more than 30% of their income on housing
(including utilities): a. renters b. owners |
|
16. Exports to non-primary partners as a percentage of total exports. |
44. Percent of Oregonians 18 and
older who do not smoke cigarettes. |
ENVIRONMENT |
|
17. Percent of Oregonians who speak a language in addition to English. |
45. Premature Death: Years of life
lost before age 70 (rate per 1,000). |
Air |
|
EDUCATION |
46. Percent of adults whose
self-perceived health status is very good or excellent. |
75. Percent of time that the air is
healthy to breathe for all Oregonians. |
|
K-12 |
47. Percent of families with
incomes below the state median income for whom child care is affordable. |
76. Carbon dioxide emissions as a percentage of 1990 emissions. |
|
18. Percent of children entering
school ready-to learn. |
48. Number of child care slots available
for every 100 children under age. |
Water |
|
19. Percent of 3rd graders who
achieve established skill levels: a. reading b. math |
Protection |
77. Wetland acreage change per
year: a. freshwater b. estuarine |
|
20. Percent of 8th graders who
achieve established skill levels: a. reading b. math |
49. Percent of 8th grade students
who report using: a. alcohol in the previous month b. illicit drugs in the previous
month c. cigarettes in the previous month
|
78. Percent of monitored stream
sites with: a. significantly increasing trends in water quality b. significantly decreasing trends
in water quality c. water quality in good to excellent condition |
|
21. Percent of high school
graduates who attain a Certificate of Mastery. |
50. Number of children, per 1,000
persons under 18, who are: a. neglected/abused b. at a substantial risk of … |