Annual Program Assessment Report
Academic Year Assessed: 2022/2023
College: College of Agriculture
Department: Animal and Range Sciences
Submitted by: Carl Yeoman

Download a pdf of this information.

Program(s) Assessed:
Indicate all majors, minors, certificates and/or options that are included in this assessment:

Majors/Minors/Certificate Options
BS - Natural Resources and Rangeland Ecology Rangeland Ecology and Management
  Wildlife Habitat Ecology and Management
Minor Natural Resources & Rangeland Ecology  

 

AnnualAssessment Process(CHECK OFFLIST)

  1. Data are collected as defined by Assessment Plan: YES 
  2. Population or unbiased samples of collected assignments are scored by at least two faculty members using scoring rubrics to ensure inter-rater reliability. YES
  3. Areas where the acceptable performance threshold has not been met are highlighted. YES
  4. Assessment scores were presented at a program/unit faculty  YES
  5. The faculty reviewed the assessment results, and responded accordingly (Check all appropriate lines)

Gather additional data to verify or refute the result:

Identify potential curriculum changes to try to address the problem:

Change the acceptable performance threshold, reassess. Choose a different assignment to assess the outcome.  Faculty may reconsider thresholds.

Evaluate the rubric to assure outcomes meet student skill level. Use Bloom’s Taxonomy to consider stronger learning outcomes. Choose a different assignment to assess the outcome.

OTHER:

Does your report demonstrate changes made because of previous assessment results (closing the loop)? YES           

  1. Assessment Plan, Schedule and Data

        a. Multi-year assessment schedule for assessing all program learning outcomes:

PROGRAM LEARNING OUTCOME
Our graduates will:

2021-2022

2022-2023

2023-2024

2024-2025

2026-2027

1.     Demonstrate the ability to develop sustainable management and habitat restoration plans by synthesizing and applying knowledge of rangeland and wildlife ecology, soils, and vegetation. [Knowledge]

 

 

 

 

X

 

2.     Critically review and evaluate information to make decisions regarding the management of renewable resources in order to achieve conservation and management goals. [Critical

Thinking]

 

 

 

 

 

X

3.     Demonstrate effective written and oral communication skills and facilitate communication within collaborative environments. [Communication and

Collaboration]

 

X

 

 

 

 

4.     Use scientific principles to formulate questions, explore solutions, and problem solve in their chosen profession. [Problem Solving]

 

 

X

 

 

 

5.     Apply ethical standards to manage natural resources. [Ethics]

 

 

X

 

 

 

b. Threshold values and data sources for assessing program learning outcomes:

 

Threshold Values

PROGRAM LEARNING OUTCOMES

Threshold Value

Data Source

1.     Demonstrate the ability to develop sustainable management and habitat restoration plans by management and habitat restoration plans by synthesizing and applying knowledge of rangeland and wildlife ecology, soils, and vegetation. [Knowledge]

The threshold value for this outcome is for ≥ 80% of assessed students to score ≥ 2 on a 1-3 scoring rubric

Randomly selected student writing assignments

2.     Critically review and evaluate information to make decisions regarding the management of renewable resources in order to achieve conservation and management goals. [Critical Thinking]

The threshold value for this outcome is for ≥ 80% of assessed students to score ≥ 2 on a 1-3 scoring rubric

Randomly selected student writing assignments

3.     Demonstrate effective written and oral communication skills and facilitate communication within collaborative environments. [Communication and Collaboration]

The threshold value for this outcome is for ≥ 80% of assessed students to score ≥ 2 on a 1-3 scoring rubric

Evaluators attend student oral presentation and randomly selected writing assignments

4.     Use scientific principles to formulate questions, explore solutions, and problem solve in their chosen profession. [Problem Solving]

The threshold value for this outcome is for ≥80% of assessed students to score ≥ 2 on a 1-3 scoring rubric

Randomly selected student individual or group assignments

5.     Apply ethical standards to manage natural resources. [Ethics]

The threshold value for this outcome is for ≥ 80% of assessed students to score ≥ 77% on the online test

Randomly selected student individual or group assignments

 

2.   What Was Done

a) Was the completed assessment consistent with the plan provided?  YES

If no, please explain why the plan was altered. N/A

b) Please provide a rubric that demonstrates how your data was evaluated.

We used the Natural Resources and Rangeland Ecology Rubric for Learning Outcomes 1, 2, 3, and 4 (see below) to assess Learning Outcome 3: Communication and Collaboration.

 

Department of Animal and Range Sciences
Natural Resources and Rangeland Ecology BS

Rubric for Learning Outcomes 1 – 4

[Knowledge, Critical Thinking, Communication and Collaboration, and Problem Solving] Scoring: 1 = not acceptable; 2 = acceptable; 3 = exceeds acceptable

Evaluators may assign one of 5 possible scores: 1.0, 1.5, 2.0, 2.5, or 3.0

 

 

Cognitive Domain

Scoring Criteria

 

Score

1

2

3

Investigate and Research

Little inquiry; limited knowledge shown

Explores topic with curiosity; adequate knowledge from variety of sources displayed

Knowledge base displays scope, thoroughness, and quality

 

Examine and Identify the Problem/Question

Does not identify or summarize the problem/question accurately, if at all

The main question is identified and clearly stated

The main question and subsidiary, embedded or implicit aspects of a question are identified and clearly stated

 

Analyze and Synthesize: Identifies and evaluates the quality of supporting data/evidence; detects connections and patterns

No supporting data or evidence is utilized; separates into few parts; detects few connections or patterns

Evidence is used but not carefully examined; source(s) of evidence are not questioned for accuracy, precision, relevance and completeness; facts and opinions are stated but not clearly distinguished from value judgments

Evidence is identified and carefully examined for accuracy, precision, relevance, and completeness; facts and opinions are stated and clearly distinguished; combines facts and ideas to create new knowledge that is

comprehensive and significant

 

Construct and Interpret: Identifies and evaluates the conclusions, implications, and consequences; develops ideas

Combines few facts and ideas; needs more development; conclusions, implications, and consequences are not provided

Accurately identifies conclusions, implications and consequences with a brief evaluative summary; uses perspectives and insights to explain relationships; states own position on the

question

Accurately identifies conclusions, implications, and consequences with a well-developed explanation; provides an objective reflection of own assertions

 

 

TOTAL:

 

MEAN:

 

 

     3. How Data Were Collected

     a) How were data collected? (Please include method of collection and sample size).

Oral presentations were assessed in WILD 420 (Range and Wildlife Policy and Planning) and were used to evaluate the Three Tenants of the Learning Objective: 1) Formulate questions, 2) Explore Solutions, and 3) Problem solve. Oral presentations were given by groups of students, three to four in size. Two groups presented at the same time and where one groups supported and the other opposed an issue that was both current and relevant to management of natural resource in Montana and the greater US. The course assignment is included at the end of this report.

     b) Explain the assessment process, and who participated in the analysis of the

The Problem-solving learning outcome for NREE states that students will use “scientific principles to formulate questions, explore solutions, and problem solve in their chosen profession”.

Ten NRRE students were enrolled in WILD 420 during Spring Semester 2023 and all ten students were evaluated by two range faculty, Dr. Jared Beaver and Dr. Hayes Goosey. Students were evaluated first by observing their presentations and how each anticipated and addressed the three tenants as part of the oral PowerPoints. Second, the evaluators listened to how NREE students answered questions from the general audience and evaluators also asked pointed questions with the intention of flustering students to present a real-life, problem-solving situation.

Students were scored on a 1-3 scale, after which each evaluator’s scores were averaged by student and across student on the criteria of addressing and responding to real life problem solving. Student scores are summarized in Table 1.

4. What Was Learned

Based on the analysis of the data, and compared to the threshold values provided, what was learned from the assessment?

NRRE students exceed the departmental threshold on each of the three objective tenants. The threshold value for this outcome is for ≥80% of assessed students to score ≥ 2 on a 1-3 scoring rubric. Ten of ten (100%) of students reached that mark with across student means equaling:

  1. Formulates questions: 2.35
  2. Explores solutions: 2.25
  3. Problems solves: 2.13

While the student performance means across all ten students exceeded the 2.0 bench mark, students 6, 9, and 10 did receive three separate marks of 1.5, which is below acceptable (Table 1). Two range faculty evaluating ten NREE students on three criteria gives us 30 means scores. Of these 30, 11 scores (36.67%) were acceptable (2.0), eighteen scores (60%) were above acceptable (> 2.0), and one score (3.33%) was below acceptable (< 2.0) (Table 1).

Table 1. Student evaluation scores assessed in WILD 420.

Evaluator 1                                     Evaluator 2                                         Mean

 

Student

Formulate Questions

Explore Solutions

Problem Solve

Formulate Questions

Explore Solutions

Problem Solve

Formulate Questions

Explore Solutions

Problem Solve

1

2.50

2.00

2.00

2.50

2.00

2.00

2.50

2.00

2.00

2

3.00

2.50

2.50

3.00

3.00

2.50

3.00

2.75

2.50

3

2.00

2.00

2.50

2.00

2.50

2.50

2.00

2.25

2.50

4

2.00

2.00

2.00

2.50

2.00

2.00

2.25

2.00

2.00

5

3.00

2.50

2.50

2.50

2.50

2.00

2.75

2.50

2.25

6

2.00

1.50

2.00

2.50

2.00

2.00

2.25

1.75

2.00

7

2.50

2.50

2.00

2.00

2.50

2.00

2.25

2.50

2.00

8

2.00

2.50

2.00

2.50

3.00

2.50

2.25

2.75

2.25

9

2.00

2.00

2.00

2.00

2.00

1.50

2.00

2.00

1.75

10

2.00

1.50

2.00

2.50

2.50

2.00

2.25

2.00

2.00

Mean

2.30

2.10

2.15

2.40

2.40

2.10

2.35

2.25

2.13

 

     a) Areas of strength

  • Majority (96.67%) of marks are acceptable or above (> 0).
  • Majority (60%) of marks are above acceptable (> 0)
  • Student marks scored the highest in formulating questions and many started their presentations with questions they intended to answer during the talk.
  • Well-cited talks, good use of scientific
  • All students were well prepared, arrived on time, and dressed
  • Presentation delivery was well suited to the room and audience
  • Overall, students scored above average at synthesizing the issue information in to a digestible packet for the audience.

    b) Areas that need improvement

  • Would like to see means for Explore Solutions and Problem Solve
    • The real essence of a natural resource degree is to provide realistic and workable solutions to those in need.
  • Some students were very dependent on their note cards
  • Some students limited eye contact with the audience and moved to the back of the group to hide during question period

5. How We Responded

     a) Describe how “What Was Learned” was communicated to the department or program faculty. Was       there a forum for faculty to provide feedback and recommendations?

In anticipation of our August 2023 Faculty Retreat, a first draft of this report was provided to the Range Program Faculty. At the faculty retreat, the range faculty discussed the report thoroughly. The range faculty was pleased with the report; however, there was one faculty member who expressed concerns about if the course we evaluated, Wild 420, was the best choice to assess the ‘Problem Solving’ learning outcome of the NREE program. Beyond that, the faculty appreciated the assessment and agreed to continue to work toward maintaining a high standard of excellence in our departmental curriculum and to improve areas as necessary.

     b) Based on the faculty responses, will there be any curricular or assessment changes (such as plans for measurable improvements or realignment of learning outcomes)? Answer: NO

If yes, when will these changes be implemented? N/A

Please include which outcome is targeted, and how changes will be measured for improvement. If other criteria are used to recommend program changes (such as exit surveys, or employer satisfaction surveys) please explain how the responses are driving department, or program decisions.

     c) When will the changes be next assessed?

Learning outcome assessed: ‘Use scientific principles to formulate questions, explore solutions, and problem-solve in their chosen profession’. Next assessed in 2028-2029.

6. Closing the Loop

Based on assessment from previous years, can you demonstrate program level changes that have led to outcome improvements?

In general, scores improved from 81% of students meeting the minimum requirements in spring of 2018 to 97% meeting the minimum in 2023. This indicates that efforts put forth by the MSU NREE faculty may be increasing students' problem-solving skills. It is worth noting, that in 2023 a one to three point scoring system was used compared to a one to five point system in 2018. The narrowed point system of 2023 may have artificially inflated the percentage of passing scores, compared to the broader score range in 2018, so contrasting these scores may not be valid. In general, the range faculty are committed to continuing to incorporate assessment programs in the curriculum and using these to deploy collaborative learning and cross- curriculum tactics during the next 5 year period.

Appendix

Class Assignment: WILD 420

REQUIREMENTS AND FORMAT FOR DEBATES/PRESENTATIONS

In your future careers as land and wildlife managers, you will be called upon regularly to write agency/operation positions and plans, and to present the results of your work to your bosses (e.g., the Commission), colleagues, and the public. To develop those skills, each student is required to participate in a team debate (or give a presentation if approved by the instructor), and develop a position paper that critically evaluates a contemporary policy issue. Debate presentations and term papers will be scheduled for the end of the semester. However, they require much preparation, so do not fall into the ‘I’ll do it later’ trap. You will want to begin your background research and contacting sources for information early in the semester. Failure to give a presentation or submit a term paper will result in a failing grade for the course.

You must inform the instructor of your debate/presentation topic NO LATER THAN THURSDAY, FEBRUARY 3, 2022. Choice of topic and whether presentations will be permitted in lieu of debate are subject to approval by the instructor.

Debates

Each debate involves 4 students with 2 persons advocating each side of a controversy in wildlife or land conservation / management. Each student in the debate will represent the viewpoint of a particular agency or private organization involved in the controversy. Debate teams must meet with the instructor as a group following selection of an issue to discuss the topic and decide who will represent appropriate positions. Once the issue and positions are chosen, they cannot be changed without permission of the instructor.

Each individual will prepare a Position Paper which is due on the date of presentation. In the paper, you must make it clear why the agency or group position you represent took the position it did on the issue. Your position must present relevant biological, sociological, and economic data that are supported with citations of pertinent laws, regulations, policies, scientific studies, and other authoritative sources. [Wikipedia and your cousin’s blog is not authoritative]. Your paper must include a bibliography that includes all cited sources. See Paper Guidelines below. In some cases, there may be experts or agency representatives you will want to contact for information; before you impose on their valuable time, do you your homework so you can ask intelligent questions. The instructor may be able to assist you in identifying additional sources/experts. You are encouraged to work together with your team to locate materials and discuss the issues, but remember, it is a debate so there is an element of competition in your presentation. Your performance will be based on your performance, and not that of your colleagues.

Debates will begin with 3 minute opening statements by each participant. The opening statement should describe and support your organization’s position in the issue. The opening statement should be your own work, not a reading of a position statement prepared by the organization you represent. After opening statements, each individual will have 3 minutes to respond. This is a rebuttal period, not an opportunity to continue or rehash your opening statement. You can prepare your rebuttal in advance by anticipating the points that the other side will try to make. A second rebuttal period will be available to address your opponents’ counterpoints. Finally, you

will present a 2 minute closing statement that summarizes your position. After closing statements, the class, guests, and instructors will be free to question you on the issues presented.

Performance in debates will be evaluated on the basis of preparation, familiarity with the issue, ability to clearly articulate your position, command of supporting information including laws, regulations, and policies and success in convincing the audience of your point of view. Emotional appeals will not receive high marks; be prepared to present rational arguments supported by evidence. Remember, you do not necessarily have to ‘believe’ in a position in order to successfully advocate it in a debate. In fact, taking a position counter to her beliefs puts a debater in a better position to anticipate and prepare for counter- arguments.

Presentations

As an alternative to participating in a debate, you may be permitted to prepare a 20-minute oral presentation for the class. The presentation may be a summary of an important wildlife or land management issue (not the subject of a debate), or it may deal with an organization that is particular interest to you and relevant to the course. Expect a 5-10 minute Q&A period to follow your presentation. Whether or not a presentation will be permitted will depend on class size and the number of class periods available for debates and presentations.

If a presentation presents an issue, it and the Paper must summarize all relevant positions on the issue, the agencies or organizations involved, the history of the conflict, pertinent laws, regulations, and policies, options for resolution, and a bibliography. If the presentation addresses agency or organization, the paper must describe the organization, its funding, history and mandate, current programs and issues, and pertinent legislation and regulations.

DEBATE / PRESENTATION EVALUATION CRITERIA (100 points)

GENERAL (25 points)

  • Professionalism and poise (5 points)
  • Speech mechanics (5 points)
  • Effective use of visual aids (5 points)
  • Evidence of preparation (5 points)
  • Effective use of time (5 points)

OPENING STATEMENT (3 minutes; 20 points)

  • Identify organization
  • Clearly articulate position statement
  • Arguments supporting position REBUTTAL[s] (2 minutes; 20 points)
  • Evidence of command of supporting information
  • Convincing arguments and counter-arguments
  • Lack of emotional basis for position and arguments CLOSING STATEMENT (2 minutes; 20 points)
  • Includes strongest points
  • Credible and rational

Q & A (Time variable; 15 points)

  • Well-reasoned and supported answers

PAPER GUIDELINES

General

The paper will be graded on its substance, style and organization, and grammar/spelling. Proper style and good grammar are required to receive better than a B grade. To develop a good paper, you must review the literature; be sure to distinguish between fact and opinion. A safe bet is the peer-reviewed literature. Do not plagiarize by not citing the ideas and work of others. I fail all plagiarists. Give your sources full credit, but do provide your own comments. You can certainly disagree with someone’s opinion, but you should explain why with support from the literature. A quality paper acknowledges the positon of opposing groups, but counters such points with well-researched and presented counter- arguments.

Length/Format

The paper should be a minimum of 6-8 pages long plus literature cited (limit your papers to 15 pages total, excluding any figures or tables). The paper must be typewritten, double-spaced using 10-12 font with 1 inch margins. Use a header (your name) and page numbering but suppress both on the first page. A title page is optional…but a nice touch.

Style/Organization

Format your paper according to the standards of the Journal of Wildlife Management or those of the journal Rangeland Ecology and Management. Both journals have “Guidelines for Authors” available on their website. [Hint: If you are not using a bibliography software like EndNote – you are missing out; EndNote and instruction in its use is available to you in the library and I highly suggest you use it]. Each part of a well-organized paper serves a purpose. I suggest the following organization:

Introduction. – This section sets the stage for the paper. It should contain a concise review of the literature related to your position, a brief history of the issue, a description of the group you represent, and a clear statement of your position.

Position Description/Justification. – This is where you present the facts of the case that support your point of view and position. You may identify the opposition and their arguments, but you counter with facts and evidence that supports your position. Your arguments should include biological, ecological, economic, and social issues and themes in a dispassionate manner. This section should have second and sometimes even third-order subheadings.

Conclusion. – Clearly state why your position is stronger than your opponents. This is your last chance to sway opinion to your point of view, so you should concisely summarize your key points and strongest arguments. This section may also present a “call to action”.

Literature Cited. – Style must follow JWM or REM. All sources cited in the text should be in the literature cited section and formatted to journal standards. Information obtained from web searches can also be cited, if formatted correctly. See Author Guidelines and examples of journal articles. You should review published books, journal articles, newspapers, literature available from the group you represent, and other information. Excessive reliance on any single source is not acceptable and will result in a grade no higher than a B. Citations of only un-vetted sources (most of the internet) will result in a grade no higher than a C.

PAPER SCORE SHEET

CONTENT (60 points total)

60-56          Paper clearly identifies the issues debated and reveals a strong grasp of the pros and cons of the issues. It clearly identifies stakeholders involved, their motives, and preferred outcomes. The discussion blends analysis and reflection with good examples and support from the literature.

55-46          Paper does a good job of identifying issues and their pros and cons, but not as effectively as the top-ranked papers.

45-40          Paper covers the topic, but the discussion is partial, general, or imprecise.

39-30          Paper fails to deal with the topic in a comprehensive way. Statements are not well supported by the literature with specific or persuasive evidence. Choice of literature is weak.

29-20          Paper is superficial and contains many inaccuracies. It reflects less than expected effort and little real understanding of the topic. Literature cited section does not meet requirements.

STYLE (40 points total)

40-36           Paper demonstrates effective command of sentence structure, diction, and organization. The writer displays obvious effort in creating a well-written document. Virtually no grammar or spelling mistakes. Follows formatting guidelines.

35-30           Paper is well written in appropriate style and ideas are clearly presented, but not as well as the top-ranked papers. Few grammar, spelling, and formatting mistakes.

29-20           The writing is adequate, but demonstrates inconsistent control over elements of composition. Attempt to organize the contents is obvious, but not fully realized or effective.

19-10           Paper conveys the author’s ideas, but reveals weak control over diction, syntax, and organization. Several spelling and grammar mistakes.

9-0                This paper is poorly written and reveals a lack of effort suitable for a university-level course. Weak grammar, spelling, and/or organization.