Plan: Outcomes

Preparing Outcomes

Think about the program goal you developed in the previous step. What kinds of knowledge or skills will students need to have when they graduate in order to achieve that goal? That body of knowledge and that skill set are your program learning outcomes.

Program outcomes are the intended learning outcomes of an academic program. They are the answers to the question, “What should program graduates know and be able to do?”

Many people find that developing learning outcomes for assessment plans is the most difficult and time consuming part of the process. If you adhere to the guidelines provided below, it will become much easier with practice, and you will avoid many problems with the subsequent steps. The time you invest now will save time later and will ensure that you are able to collect high-quality assessment data.

The learning outcomes you develop at this stage will provide a foundation for all your assessment work. The time and attention you give this activity will pay off later.

Program faculty should identify at least two, and no more than five, program outcomes during a single assessment cycle. These few outcomes on which you will focus during a single cycle are not intended to represent the full array of potential outcomes for the program. They will serve as a limited snapshot of student learning within an academic year, and you will have ample opportunity to study other outcomes in future years.

Programs with more than five important outcomes, particularly those specified by an external accrediting body, should divide those into subsets for consideration across two or more assessment cycles. There are many possible ways to approach this.

Follow a predetermined rotation that includes a new subset of all program outcomes each year
Identify a small number of core outcomes that will be included every year, and rotate others each year
Identify a small number of core outcomes that will be included every year, and identify other outcomes that are focused on recent programmatic changes that should be evaluated; Identify a subset of outcomes, repeating those that are not met, and replacing those that are met with new outcomes
Group the outcomes into categories such as content knowledge, application, research, communication, clinical skill, or others. Select one outcome from each category during each cycle.

Guidelines for Writing Effective Program Outcomes

Write outcome statements that flow directly from, and support, the program goals. Think about the program goals you wrote previously. The connection to the goals should be evident in your program outcomes.
Write outcome statements that relate directly to the academic discipline and reflect the knowledge and skills students should acquire through both general education and discipline-specific courses. When possible, connect aspects of general education (e.g., critical thinking, quantitative reasoning, oral and written communication skills, and global awareness) to the curricular content of the major. You may consider writing or critical thinking to be very important for graduates of your program. If so, think about how you expect students to demonstrate those skills within the context of your academic discipline.
Write outcome statements that are observable and measurable. Focus on observable behaviors rather than what students think, understand, appreciate, etc. We cannot measure what students know or understand, but we can measure how they demonstrate evidence of knowledge and understanding. Avoid outcome statements that say, “Students will know …,” or “Students will understand ….” When you’re tempted to use these, think about what students who know or understand can DO with that knowledge or understanding. Consider the example below. (Include information on the levels of Bloom’s revised taxonomy)
Write outcome statements that focus on knowledge and skills graduates should possess rather than curriculum design, department resources, faculty characteristics, or instructional methods. Rather than saying that students will learn, students will increase understanding, students will acquire knowledge, etc., express outcomes in terms of what students will be able to do. Additionally, internal departmental outcomes are not assessed for this type of skill focused program assessment; your team should consider separately any issues of resources and dynamics that might improve functionality in the program.
For programs that have specialized accreditation or certification, write outcome statements that take those assessment expectations into consideration. Some specialized accreditation organizations focus on curriculum design or other inputs rather than student outcomes. For those, you may want to write an outcome statement that addresses an input-based standard from the perspective of student-based outcomes. Some specialized accreditors provide specific learning outcomes that institutions must measure. Although the language and format of those mandated outcomes may not adhere to our guidelines, you should use the specific language provided by the specialized accreditation agency. The only time you may need to restate an external standard would be to focus on the student, if the standard is focused more on resources or program operations. Additionally, please make it clear when an outcome comes directly from an accreditor when designing your assessment plan.

While using a courses critical to students’ success in a program, important knowledge and skills can be evaluated using rubrics with analytic scoring. Through the use of rubric scoring at this detailed level, program faculty can separately evaluate students’ ability to knowledge and skills as they relate to specific program outcome. Such a rubric will permit faculty to give feedback (and grades) for each of the separate components. This same approach can be used at the course level and aggregated to program-wide levels easily. We will see later that this approach can also yield rich assessment information that can be used to identify specific program strengths and weaknesses, and guide continuous improvement efforts, and measure this development over time. (Note: 3.8 was chosen in the examples above because it is > 75% of a 1-5 scale).
Write outcome statements that do not combine multiple outcomes in a single statement. Avoid the temptation to bundle everything you value about your program into a lengthy outcome statement. Stay focused on clear and simple outcomes that will yield high quality information. There are times when an outcome must be rather complex in order to capture the complexity of a particular program. We sometimes speak of such outcomes as being so “interwoven” that to separate the elements into separate outcomes would somehow diminish the richness of the assessment. When evaluating your outcome statements, be careful not to lump multiple elements into a single statement unless you truly have a complex outcome for a complex program.
One of the best ways to resolve problems with an outcome statement that consists of multiple outcomes is to collapse them into a single outcome. A very common example is an outcome that refers to program graduates’ ability to “design and conduct research studies, and communicate the results of their research both orally and in writing.” While using the same project or student artifact, each of these skills can be evaluated with separate rubric elements.Through the use of a structured rubric, program faculty can then separately evaluate students’ ability to design a study, collect data, analyze data, interpret results, write research reports, and communicate their findings to others as they relate to the specific outcome. Such a rubric will permit faculty to give feedback (and grades) for each of the separate components. This same approach can be used for any individual or group written or performance projects that can be assigned to students. We will see later that this approach can also yield rich assessment information that can be used to identify specific strengths and weaknesses in your students’ abilities.
Write outcome statements that are short and concise. Longer statements tend to be vague or include multiple outcomes.
Write your outcome statements in the form of “Students of the _____ program will be able to _____;” or “Students of the _____ program will be prepared to _____.” This format will help you to avoid many of the problems described in the preceding paragraphs.

Don’t be afraid to consider outcomes that may seem too vague or too difficult to measure. If you have an idea about an outcome that you consider important to your program but doesn’t seem to fit these guidelines, contact the assessment team. We may be able to help you identify an appropriate measure for your outcome or to revise it into something more easily measured. There are high-quality ways in which you can measure critical thinking, creative thinking, ethical reasoning, and other important skills you may value, but are hesitant to use.

The choice of program outcomes for your assessment plans should always be guided by the program mission and long term goals for your graduates. Program faculty may identify program outcomes that seem appropriate for both an undergraduate program and a graduate program in the department, yet believe that it is important to differentiate between the two levels of study.

Consider the following options.

Include as many skill areas as possible in each assessment plan and strive to include all American Association of Colleges and University’s Intellectual and Practical Skills are addressed over the term of each program.

Critical Thinking
Creative Thinking
Language & Literature (Reading)
Information Literacy
Collaboration/Teamwork
Written Communication
Verbal Communication
Quantitative Literacy
Inquiry & Analysis
Problem Solving

Whenever possible, also include American Association of Colleges and University’s Personal & Social Responsibility Skills, and Integrative and Applied Learning Skill.

Civic Engagement—Local & Global
Intercultural Knowledge and Competence
Ethnical Reasoning
Foundations and Skills for Lifelong Learning
Global Learning
Integrative Learning

Select different cognitive levels for undergraduate and graduate program outcomes that express your expectations of students. For example, you might define an undergraduate program outcome at Bloom’s comprehension level and a graduate program outcome at the evaluation level. A table of verbs for the cognitive domain levels of Bloom’s revised taxonomy is shown later in this section.

Bloom's Revised Taxonomy

Bloom's Revised Taxonomy is a hierarchical system of ordering thinking skills from lower to higher, with the higher levels including all of the cognitive skills from the lower levels. The table below contains samples of verbs associated with student response at the different levels of Bloom’s Taxonomy.

Furthermore, when planning to use data from previous assessment reports, review the information available to be certain the information available matches your assessment description and meets your needs.

Link to Sample Assessment Report: https://canvas.asu.edu/courses/81507/files/53312913?module_item_id=9254596

Assessment Handbook

To assist units in the assessment planning process, we created a handbook: Effective Assessment Planning, Reporting, and Decision Making. Please refer to this handbook as you create your assessment plans and reports.

Assessment Portal

The following link will open the UOEEE Assessment Portal where all assessment plan development and reporting activities take place.