Developing and implementing customer-oriented service delivery systems has been a major shift in the service delivery paradigms that many government departments and non-profit agencies have operated under for many years....

As government and non-profit agencies have become more concerned about the effectiveness of their programs and policies, the importance of conducting formal program evaluation (process and product, and impact) has steadily increased. Often using a variety of qualitative and quantitative research methods with multiple sources of information, Bosma Research International has been a leader in developing new measures of impact and process effectiveness in human service, human resource, organizational performance, and educational settings. Throughout our history, we have designed and implemented studies that have examined:

• the effectiveness and impact of international cooperative educational agreements
• the effectiveness of ABE, vocational education, and literacy programs.
• the outcomes of college, corporate, and government training programs.
• programs serving special-needs populations that include vision-impairment, substance abuse, convicts, domestic violence, etc.
• the content of K-12 mathematics and science textbooks and teaching practices.
• the quality and effectiveness of undergraduate and graduate programs.
• the effectiveness and impact of mandated change programs.
• programs focused on K-12 school reform and transformation.
• the impact of school change processes on the quality of education.
• outcome-based education and alternate forms/methods of assessment.
• the cost of implementing inter-agency early childhood programs.
• the effectiveness of children's mental health service systems.

Many government departments and non-profit agencies have struggled to transform bureaucratic, process-oriented systems. Simply put, developing and implementing customer-based service delivery systems has been a major shift in the service delivery paradigms that many government departments and non-profit agencies have operated under for many years. Our experience includes a wide variety of studies and experience:

• program evaluation
• customer satisfaction
• employee satisfaction and productivity
• quality measurement
• climate/culture assessment
• organizational performance

Some of the organizations we have worked for include:

• US Committee for the United Nations Population Fund
• Fund for the Improvement of Postsecondary Education, U.S. Department of Education
• Library of Congress, National Library Services
• Institute for Museum and Library Services
• General Services Administration
• US Navy, Department of Defense, Crane Division, Surface Warfare Center Division
• National Science Foundation
• U.S. Department of Education
• Minnesota Departments of Education, Human Services, and Public Safety
• Minneapolis Urban League
• Minneapolis Public Schools
• Indiana Department of Education
• Missouri Department of Education
• Special education cooperatives
• Community action agencies
• K-12 school districts

Experience with Educational Change and Improvement Projects

Bosma Research International has had over 10 years experience in designing and conducting evaluations across a wide variety of state and federal programs and projects. While some of these projects have examined the effectiveness and impact of state-mandated change programs, others have reviewed other types of programs focused on K-12 school reform, systemic change, and/or transformation.

On an implementation level, these projects have examined issues related to:

• the different phases and processes that school systems go through when undertaking significant systemic change - noting the differences between state/federally-mandated change and grass roots change efforts.
• factors promoting and/or hindering change efforts in school systems.
• changing/shifting community and school demographics.
• schools' readiness and capacity to deal with change.

On an outcome level, these projects have examined:

• community reactions to and satisfaction with various types of change efforts.
• the effectiveness and impact of specific types of outcome- and/or performance-based learning activities at various stages of implementation.
• the effects of change efforts on student learning, staff, and communities.
• the impact of school change processes on the quality of education.
• outcome-based education and alternate forms/methods of assessment.

BRI has also designed evaluation models (quality indicators and outcome measures) for evaluating schools engaged in change processes as well as systems capable of identifying the impact of change-related programs on student learning and achievement.

Experience with Human Service and Children's Mental Health Programs

Bosma Research International has designed and conducted a number of research and evaluation projects that have documented state and local interagency implementation issues and the impact of implementing coordinated interagency service delivery systems on the quality of children's mental health. The results of various evaluation/research projects have been incorporated into a variety of State planning and decision-making activities.

We also conducted an exhaustive cost-analysis and needs assessment jointly for the Minnesota Departments of Education, Human Services, Health, and the Governor's Interagency Coordinating Council on Early Childhood Intervention in order to determine the types and costs of early intervention services currently available as well as the additional types of services needed to fully implement the provisions of Individuals with Disabilities Education Act, Part H.

Experience with ABE and Literacy Programs

BRI professionals have also developed quality indicators and outcome measures for evaluating the effectiveness of ABE programs. The model illustrated how "effectiveness" ought to be defined in the context of ABE programming and provided recommendations for improving the overall effectiveness of state ABE programs.

In another study, BRI assess the value and cost-effectiveness of state ABE monies dispensed to private non-profit agencies. This project represented one of the first efforts to address a nation-wide need to make state ABE funds available to non-profit agencies.

Experience with Self-Sufficiency and Welfare-Reform Initiatives

As part of a five-year project, Bosma Research International was involved with an intensive, non-traditional, self-sufficiency program built within the structure of a community action agency's Head Start Program. The project extensively documented a wide number of social, economic, systems, and personal barriers that families encounter in their efforts to become self-sufficient and free of government support systems. Funded largely by McKnight's Aid to Families in Poverty Initiative, an integral component of the project was to develop a replicable model for aiding situational and generational impoverished families in poverty to achieve self-sufficiency. The project was named one of 37 citationists by the 1992 President's Annual Points of Light Awards for the program's effort toward solving the serious social problems in their communities.

Experience with Special Needs Populations

Vision Impairment Programs - In a project for the Library of Congress, National Library Services for the Blind and Physically Handicapped, Bosma Research International evaluated the effectiveness of the Braille Magazine Program in order to determine the extent to which the program was meeting the needs of subscribers. Survey materials were distributed to subscribers in Braille, large print, and on audio cassette. Bosma Research International was also involved with developing an empirically-based assessment instrument (Braille Assessment Inventory) which is used in schools to determine whether a child should begin or continue to receive Braille instruction.

Special Education Programs - Many of our evaluations in the special education arena have examined the effectiveness of the educational and therapeutic models used in different types of day treatment settings. These project have measured the impact of different therapeutic approaches on students and their families, as well as the impact of the programs on the total system of care.

Non-Majority Populations - Throughout our history, Bosma Research International has subsidized programs designed to improve the academic achievement and performance of non-majority students in the areas of science, mathematics, and computer literacy. We have also advocated for case management initiatives and reform efforts undertaken to streamline the delivery of social services for families in poverty attempting to achieve self-sufficiency.

Minority Education - The purpose of another project was to improve the quality of teaching and instruction for fifth and sixth grade minority students in a major metropolitan K-12 school system. In particular, the project was designed to improve the academic achievement and performance of minority students in the areas of science, mathematics, and computer literacy; to sensitize regular K-12 teachers to differences in learning styles and other psychological, sociological, and economic issues which influence the teaching of minority students; and to inform parents of minority students of career opportunities in science and technology as well as the educational background and requisites needed for entry into these fields. Our role in the project was to evaluate the effectiveness of the ongoing conferences being used to convey the content of the project as well as the effectiveness and impact of the project overall.

Experience with Communications and Public Relations Projects

Bosma Research International has managed projects designed to identify the attitudes and behaviors of the public towards specific issues and/or referendums under consideration. The results of these project have been used to identify community support for/against specific actions, isolate the core issues of key constituent groups, and to develop public education campaigns and information programs.

Experience with Quality Measurement Projects

We have also conducted a number of quality measurement, climate assessment, and employee satisfaction studies in government and non-profit settings. Some of these studies have also evaluated the effectiveness and impact of training programs on organizational performance, and the efficiency of operational and service delivery systems.