Learning Outcomes: From the Big Picture to the Classroom

For their 2016 Annual Project, the CARLI Instruction Committee decided to provide an outline and accompanying materials for the Learning Outcomes webinar and the Curriculum Mapping IACRL PreConference workshop. The information that follows was compiled by the Committee to provide information for instruction librarians who were unable to attend the events, or for those who would like to refer to the ideas shared during the events about implementing and assessing the Framework.

This outline is intended to assist librarians viewing the recorded webinar.

Learning Outcomes: From the Big Picture to the Classroom
Webinar presented by Debra Gilchrist, Pierce College, Lakewood Washington

December 10, 2015

Click image above to view recording of webinar.

Introduction to Outline

We’ve become accustomed to talking about assessment and assessment of student learning. However, assessment is the measure or test. Let’s rather focus on the outcome, on the student, on what we want to see, and what students are learning.

Philosophy of Outcomes Assessment

Outcomes assessment is foundational to our teaching. They help assist in designing our library sessions. Sometimes our carefully planned library sessions are done so from the teacher’s perspective. Outcomes and assessment help us look through the student lens. They give us a guide to what we want to teach and set the stage for assessing that outcome. The foundational question when drafting an outcome is “What do you want the student to be able to do?”

Metaphor of the Puzzle
We often look at the products (i.e. bibliography) of student work, but we also need to look at the individual elements that make up that composition. The pieces of the puzzle that comprise the puzzle as a whole. You are encouraged to look at the process students use to get to the final design. Process (that critical thinking element) and product are equally important. For example:

  • When students had an opportunity to make a choice about what direction to take with their research, why did they choose resource A instead of resource B?
  • What criteria did they use to make that decision to choose that resource?
  • How did they decide to stop their search and determine they had sufficient information?

5 Questions to Instructional Design

Questions that will get at the instructional design process and help you take theory into practice. Since outcomes set the stage for design, it is where we need to begin.

  1. Outcome: What do you want the student to be able to do?
    • Inspired by the institutional/library mission, values, goals, strategic plans, curriculum or gen ed / information literacy definition / information literacy program goals
  2. Content: What does the student need to know to do this well?
    • This is the content or curriculum that you are going to work with. What are you deciding to incorporate to help the student get there?
  3. Pedagogy: What’s the activity that will enhance the learning?
    • For example, lecture? Hands-on work? What are the things that you are going to do in designing the experience for students?
  4. Assignment: How will the student demonstrate the learning?
    • This is the assessment or the assignment. What is the opportunity you are going to give the student so that you can see the learning?
  5. Criteria: How will you know the student has done this well?
    • What criteria will you use to determine what is a great answer and what is an answer that doesn’t quite get there?

Definitions to Use to Frame Outcomes Assessment Work

  • “Assessment is the ongoing process aimed at understanding and improving student learning” – Tom Angelo, AAHE Bulletin, 1996.
  • Knowing WHAT you are doing, WHY you are doing it, what students are LEARNING as a result, and CHANGING because of the information -- Deb Gilchrist

Outcomes are the foundation. They are our guides. They are the agreed upon elements of our curriculum.

2 Foundational Rules

  1. Work backwards. Think about the end product first, the ideal information literate student. What do we want them to be able to do as a result of the teaching? How do we get to that point?
  2. The work is about the student. This is not about what we teach, but what they will learn.

Checklist for Good Learning Outcomes

  • Measurable or “judgeable”
  • Set the stage for learning that is clear to the student, faculty, and librarian
  • Integrated, Developmental, Transferable
  • Integrated: Information literacy instruction needs to be integrated within the session, within the course, within the program, within the degree, not something the library owns by itself
  • Developmental: Asking different things of first year students as opposed to 2nd, 3rd, 4th year or grads students
  • Transferable: Students understand that it is not something that they are doing just for today or for this course, but for lifelong learning
  • Relates to institutional definitions and documents
  • Inspired by something that already exists for the institution or the library, i.e. definition of information literacy within the library or for general education
  • Matches the level (course, 50 minute session, program, etc.)
  • Use variety of levels of Bloom Taxonomy
  • “In order to” gets to the uniqueness of the learning – they are “balanced” statements
  • Describes what the students will do

Formula for Writing Outcomes

Verb or Action Phrase   +   IN ORDER TO =   Great Outcomes

  • All outcomes begin with a verb or action phrase -- an intentional or strong verb
    • For example: identifies multiple perspectives; distinguish between general and special databases; analyzes information
  • In order to: the why statement. Sets up a good assessment. Helps you to think about what action you want to see from students.

Examples of outcomes using the formula:

  • Brainstorm topic-relevant vocabulary in order to search databases with maximum flexibility and effectiveness.
  • Distinguish between general and specialized databases in order to select the best database for the topic and level of specialization.
  • Utilize knowledge of the inequalities of information and information power in order to strategically select where to search for sources.

Another example:

  • Develop student learning outcomes
  • Design measureable assessments of student learning

But it is much more powerful to talk about what it is you want students to do and why

  • Develop measurable outcomes, clear criteria, and valid assessment tools in order to impact student learning and improve teaching

Balance the verb and the why statements

Balance the “verb phrase” and the “in order to phrase” to capture the uniqueness of the verb phrase. Or find the unique reason WHY you want the student to DO the verb phrase.

Example of Balancing:

  • Describe criteria for evaluating sources > In order to assess the quality of information
  • Determine bias and perspective > In order to gauge the author’s audience, point of view, and what might be missing from the discussion

Writing Outcomes: 5 Things to Remember (or In Order to avoid Pitfalls)

  1. Balance the verb and the why statements.
    • NO: Evaluate websites IN ORDER TO search databases
    • YES: Evaluate websites IN ORDER TO distinguish quality from unreliable online information
  2. Avoid using broad phrases. Be descriptive and focused.
    • NO: Search periodical databases in order to retrieve good information
    • YES: Describe criteria for evaluating sources in order to assess the quality of information
  3. Avoid multiple verbs. Separate out the outcomes if you have three or more verbs.
    • NO: Define, identify, and formulate vocabulary in order to conduct successful online searches
  4. Transferability: Write outcomes, not just for the class assignment, but for what students will doing beyond the class.
    • NO: Find 2 scholarly articles in order to write a 10 page paper in psychology
    • YES: Distinguish between popular and scholarly literature in order to match quality and validity of information to the type of inquiry
  5. Avoid “understand.” Outcomes go beyond “understanding” to get at what students will be doing. You cannot see understanding.
    • NO: Understand the increasingly social nature of the information ecosystem where authorities actively connect with one another and sources develop over time (Framework)
    • YES: Describe the information ecosystem in order to assess how experts collaborate informally and formally to develop a network of information on a subject

Practice Applying the Checklist

For a 100 level Business course, the instructor asks you to emphasize use of scholarly articles.

  • Outcome: Differentiate between popular and scholarly articles in order to use them in the right setting

What’s wrong with this outcome?

  • In order to phrase is way too broad, unclear: what does “right” mean?
  • Also, try using a different level of Bloom’s Taxonomy. For example:
    • Students apply their ability to distinguish between the two types of sources
    • Categorize according to satisfaction of need
    • Identify the difference between popular and scholarly articles

A 100 level Business course assignment: Research a company you might consider employment with, including size, structure, earnings, philosophy, history, and competition. Include information that indicates how the company sees itself, and how others (on the outside) perceive the company.
How would you write the outcome? Examples:

  • Compare impressions of a company to evidence in order to determine acceptability for potential employment
  • Evaluate the perspective/point of view in a given source in order to determine how that viewpoint effects ones’ understanding of the topic

Example: All Inclusive Outcome

Introduce the concepts of information literacy to the student population in order that they will improve their ability in writing research papers in all classes which require them, experience greater academic success, will be more likely to persist in completing degrees, will view the library as an environment as helpful in meeting academic needs, and will experience the mission of the College providing education to all who may profit from it.

  • Needs to be broken down into individual concepts

Writing Outcomes in Context

Remember to think about the context for the outcome and at what level you want to write them.

  • General Education: Information Literacy & Critical thinking
    • Library Definition
      • Course
        • Instruction Session

Critical Thinking Competency Learning Outcomes Example
Students will:

  • Define the concepts of critical thinking, logic, and argument;
  • Assess the function of clarity in arguments;
  • Compare and contrast the purposes of language in persuasive statements;
  • Evaluate different types of inductive and deductive arguments:
  • Distinguish fallacies from good arguments; and
  • Apply critical reasoning concepts in order to evaluate issues of contemporary importance

Degree Level Example: Pierce College
The information competent student acquires and applies information in order to impact change, inform perspective, make decisions, and frame context.

  • Values inquiry and information needs in order to continually engage in learning
  • Applies a repertoire of creative and flexible information seeking strategies in order to navigate the unfamiliar, take action, or solve a problem.
  • Identifies appropriate sources in order to access relevant information.
  • Notice the broad nature of these outcomes that thread through courses and classes  – what faculty will turn to when preparing classes

Program Level Example: Criminal Justice, Pierce College
How the department applied the broader degree outcomes (above) to their list of program outcomes.

  • Seek, use, and be informed by information to understand and to decrease problems and crime in society and keep current in the field as a professional [Intended Outcome(s)]
  • Ability to go beyond one’s own opinion and construct an argument with meaningful points on multiple aspects of a topic [Skills & Strategies]

The Content of Outcomes

So many sophisticated to think about and approach information literacy. This is not just about picking a verb, but thinking about the content of what you are trying to frame for students.

For example, three ways in which students could be asked to search for Information:

  • Utilize search terms, Boolean operators, and database limiters in order to focus a search
    • Skills
  • Utilize citation chaining in order to determine what’s missing from a bibliography and develop next steps in a search.
    • Context & connectivity
  • Analyze biases in search algorithms in order to seek out information beyond the readily available and determine the power relationships in info availability.
    • Transformative thinking

Another example, for evaluating information:

  • Apply the CRAP TEST in order to evaluate information for appropriate use
    • Skills
  • Critically read a work in order to evaluate the claims, methods, and strength of evidence
    • Context & connectivity
  • Determine the power behind the standards of evidence in order to consider who is heard and not heard in evidence
    • Transformative thinking

Documents from our professional organizations can be used as inspiration for outcomes content

  • Information Literacy Competency Standards for Higher Education
  • Framework for Information Literacy for Higher Education

Additional Resources/Examples Noted