2021 Natural Resources and Rangeland Ecology Program Assessment Report

Natural Resources and Rangeland Ecology

Click to download the Natural Resources and Rangeland Ecology Assessment Report

Annual Program Assessment Report
Academic Year Assessed: 2020/2021
College: College of Agriculture
Department: Animal and Range Sciences
Submitted by: Carl Yeoman

Program(s) Assessed: Range Science
Indicate all majors, minors, certificates and/or options that are included in this assessment:

Annual Assessment Process (CHECK OFF LIST)

1. Data are collected as defined by Assessment Plan

Answer: YES

2. Population or unbiased samples of collected assignments are scored by at least two faculty members using scoring rubrics to ensure inter-rater reliability.

Answer: YES 

3. Areas where the acceptable performance threshold has not been met are highlighted.

Answer: YES

4. Assessment scores were presented at a program/unit faculty meeting.

Answer: YES

5. The faculty reviewed the assessment results, and responded accordingly (Check all appropriate line)

Gather additional data to verify or refute the result. 
Identify potential curriculum changes to try to address the problem
Change the acceptable performance threshold, reassess 
Choose a different assignment to assess the outcome 
Faculty may reconsider thresholds
Evaluate the rubric to assure outcomes meet student skill level 
Use Bloom’s Taxonomy to consider stronger learning outcomes
Choose a different assignment to assess the outcome

Answers: None Checked

OTHER:

6. Does your report demonstrate changes made because of previous assessment results (closing the loop)?

Answer: NO

1. Assessment Plan, Schedule and Data Source

a. Please provide a multi-year assessment schedule that will show when all program learning outcomes will be assessed, and by what criteria (data). (You may use the table provided, or you may delete and use a different format).

*Data sources can be items such as randomly selected student essays or projects, specifically designed exam questions, student presentations or performances, or a final paper. Do not use course evaluations or surveys as primary sources for data collection.

b. What are your threshold values for which you demonstrate student achievement? (Example
provided in the table should be deleted before submission)

2. What Was Done

a) Was the completed assessment consistent with the plan provided?

Answer: YES

If no, please explain why the plan was altered.

Answer: N/A

b) Please provide a rubric that demonstrates how your data was evaluated.

The Rubric for the Assessment of: Critical Thinking (Learning outcome 2) was used in evaluating these assignments (see below).

Learning Outcome 2. Critical Thinking

Department of Animal & Range Sciences
Natural Resource and Range Land Ecology B.S.

Rubric for the Assessment of: Critical Thinking
1 = not acceptable; 2 = acceptable; 3 = exceeds acceptable

3. How Data Were Collected

a) How were data collected? (Please include method of collection and sample size).

Student papers from Wild 420 (Range and Wildlife Policy and Planning) were used in the evaluation of this learning outcome. The class assignment is included below. There were 10 NRRE students in the class and all 10 students had their work evaluated against the rubric. The 10 student papers were evaluated twice, once by Dr. Jeff Mosely and once by Dr. Craig Carr. Papers were read and evaluated against the four indicators of subject content knowledge and each indicator given a score out of 3. The scores reported (Table 1) for each student and indicator are an average score from the two evaluations.

b) Explain the assessment process, and who participated in the analysis of the data.

The average student score was an average value of the four indicator scores while the indicator average score is the average score of the 10 students. The threshold value for this learning outcome is 80% of students scoring above two in their average score.

The results of our evaluation are presented in table 1. All students evaluated had a score of two or better meeting our desired threshold. The average score across all students was 2.35. All students scored two or better for indicator 2; 90% of students scored two or better for indicators one and three; and 80% scored two or better for indicator four. Students scored the highest on indicator 2 and the lowest on indicator 3.

Table 1.: Student evaluation scores.

Class Assignment:

WILD 420
REQUIREMENTS AND FORMAT FOR DEBATES/PRESENTATIONS

In your future careers as land and wildlife managers, you will be called upon regularly to write agency/operation positions and plans, and to present the results of your work to your bosses (e.g., the Commission), colleagues, and the public. To develop those skills, each student is required to participate in a team debate (or give a presentation if approved by the instructor), and develop a position paper that critically evaluates a contemporary policy issue. Debate presentations and term papers will be scheduled for the end of the semester. However, they require much preparation, so do not fall into the ‘I’ll do it later’ trap. You will want to begin your background research and contacting sources for information early in the semester. Failure to give a presentation or submit a term paper will result in a failing grade for the course.

You must inform the instructor of your debate/presentation topic NO LATER THAN THURSDAY, FEBRUARY 4, 2021. Choice of topic and whether presentations will be permitted in lieu of debate are subject to approval by the instructor.

Debates

Each debate involves 4 students with 2 persons advocating each side of a controversy in wildlife or land conservation / management. Each student in the debate will represent the viewpoint of a particular agency or private organization involved in the controversy. Debate teams must meet with the instructor as a group following selection of an issue to discuss the topic and decide who will represent appropriate positions. Once the issue and positions are chosen, they cannot be changed without permission of the instructor.

Each individual will prepare a Position Paper which is due on the date of presentation. In the paper, you must make it clear why the agency or group position you represent took the position it did on the issue. Your position must present relevant biological, sociological, and economic data that are supported with citations of pertinent laws, regulations, policies, scientific studies, and other authoritative sources. [Wikipedia and your cousin’s blog is not authoritative]. Your paper must include a bibliography that includes all cited sources. See Paper Guidelines below. In some cases, there may be experts or agency representatives you will want to contact for information; before you impose on their valuable time, do you your homework so you can ask intelligent questions. The instructor may be able to assist you in identifying additional sources/experts. You are encouraged to work together with your team to locate materials and discuss the issues, but remember, it is a debate so there is an element of competition in your presentation. Your performance will be based on your performance, and not that of your
colleagues.

Debates will begin with 3 minute opening statements by each participant. The opening statement should describe and support your organization’s position in the issue. The opening statement should be your own work, not a reading of a position statement prepared by the organization you represent. After opening statements, each individual will have 3 minutes to respond. This is a rebuttal period, not an opportunity to continue or rehash your opening statement. You can prepare your rebuttal in advance by anticipating the points that the other side will try to make. A second rebuttal period will be available to address your opponents’ counterpoints. Finally, you
will present a 2 minute closing statement that summarizes your position. After closing statements, the class, guests, and instructors will be free to question you on the issues presented.

Performance in debates will be evaluated on the basis of preparation, familiarity with the issue, ability to clearly articulate your position, command of supporting information including laws, regulations, and policies and success in convincing the audience of your point of view. Emotional appeals will not receive high marks; be prepared to present rational arguments supported by evidence. Remember, you do not necessarily have to ‘believe’ in a position in order to successfully advocate it in a debate. In fact, taking a position counter to her beliefs puts a debater in a better position to anticipate and prepare for counter-arguments.

Presentations

As an alternative to participating in a debate, you may be permitted to prepare a 20-minute oral presentation for the class. The presentation may be a summary of an important wildlife or land management issue (not the subject of a debate), or it may deal with an organization that is particular interest to you and relevant to the course. Expect a 5-10 minute Q&A period to follow your presentation. Whether or not a presentation will be permitted will depend on class size and the number of class periods available for debates and presentations.

If a presentation presents an issue, it and the Paper must summarize all relevant positions on the issue, the agencies or organizations involved, the history of the conflict, pertinent laws, regulations, and policies, options for resolution, and a bibliography. If the presentation addresses agency or organization, the paper must describe the organization, its funding, history and mandate, current programs and issues, and pertinent legislation and regulations.

DEBATE / PRESENTATION EVALUATION CRITERIA (100 points)

GENERAL (25 points)

• Professionalism and poise (5 points)
• Speech mechanics (5 points)
• Effective use of visual aids (5 points)
• Evidence of preparation (5 points)
• Effective use of time (5 points)

OPENING STATEMENT (3 minutes; 20 points)

• Identify organization
• Clearly articulate position statement
• Arguments supporting position

REBUTTAL[s] (2 minutes; 20 points)

• Evidence of command of supporting information
• Convincing arguments and counter-arguments
• Lack of emotional basis for position and arguments

CLOSING STATEMENT (2 minutes; 20 points)

• Includes strongest points
• Credible and rational

Q & A (Time variable; 15 points)

• Well-reasoned and supported answers

PAPER GUIDELINES

General

The paper will be graded on its substance, style and organization, and grammar/spelling. Proper style and good grammar are required to receive better than a B grade. To develop a good paper, you must review the literature; be sure to distinguish between fact and opinion. A safe bet is the peer-reviewed literature. Do not plagiarize by not citing the ideas and work of others. I fail all plagiarists. Give your sources full credit, but do provide your own comments. You can certainly disagree with someone’s opinion, but you should explain why with support from the literature. A quality paper acknowledges the positon of opposing groups, but counters such points with well-researched and presented counter-arguments.

Length/Format

The paper should be a minimum of 6-8 pages long plus literature cited (limit your papers to 15 pages total, excluding any figures or tables). The paper must be typewritten, double-spaced using 10-12 font with 1 inch margins. Use a header (your name) and page numbering but suppress both on the first page. A title page is optional…but a nice touch.

Style/Organization

Format your paper according to the standards of the Journal of Wildlife Management or those of the journal Rangeland Ecology and Management. Both journals have “Guidelines for Authors” available on their website. [Hint: If you are not using a bibliography software like EndNote – you are missing out; EndNote and instruction in its use is available to you in the library and I highly suggest you use it]. Each part of a well-organized paper serves a purpose. I suggest the following organization:

 Introduction. – This section sets the stage for the paper. It should contain a concise review of the literature related to your position, a brief history of the issue, a description of the group you represent, and a clear statement of your position.

Position Description/Justification. – This is where you present the facts of the case that support your point of view and position. You may identify the opposition and their arguments, but you counter with facts and evidence that supports your position. Your arguments should include biological, ecological, economic, and social issues and themes in a dispassionate manner. This section should have second and sometimes even third-order subheadings.

Conclusion. – Clearly state why your position is stronger than your opponents. This is your last chance to sway opinion to your point of view, so you should concisely summarize your key points and strongest arguments. This section may also present a “call to action”.

Literature Cited. – Style must follow JWM or REM. All sources cited in the text should be in the literature cited section and formatted to journal standards. Information obtained from web searches can also be cited, if formatted correctly. See Author Guidelines and examples of journal articles. You should review published books, journal articles, newspapers, literature available from the group you represent, and other information. Excessive reliance on any single source is not acceptable and will result in a grade no higher than a B. Citations of only un-vetted sources (most of the internet) will result in a grade no higher than a C.

PAPER SCORE SHEET

CONTENT (60 points total)

60-56         Paper clearly identifies the issues debated and reveals a strong grasp of the pros and cons of the issues.                     It clearly identifies stakeholders involved, their motives, and preferred outcomes. The discussion blends                     analysis and reflection with good examples and support from the literature.

55-46         Paper does a good job of identifying issues and their pros and cons, but not as effectively as the top                           ranked papers.

45-40         Paper covers the topic, but the discussion is partial, general, or imprecise.

39-30         Paper fails to deal with the topic in a comprehensive way. Statements are not well supported by the                              literature with specific or persuasive evidence. Choice of literature is weak.

29-20         Paper is superficial and contains many inaccuracies. It reflects less than expected effort and little real                          understanding of the topic. Literature cited section does not meet requirements.

STYLE (40 points total)

40-36          Paper demonstrates effective command of sentence structure, diction, and organization. The writer                            displays obvious effort in creating a well-written document. Virtually no grammar or spelling mistakes.                        Follows formatting guidelines.

35-30           Paper is well written in appropriate style and ideas are clearly presented, but not as well as the top                             ranked papers. Few grammar, spelling, and formatting mistakes.

29-20           The writing is adequate, but demonstrates inconsistent control over elements of composition.                                        Attempt to organize the contents is obvious, but not fully realized or effective.      

19-10           Paper conveys the author’s ideas, but reveals weak control over diction, syntax, and organization.                                 Several spelling and grammar mistakes.

9-0               This paper is poorly written and reveals a lack of effort suitable for a universitylevel course. Weak                                 grammar, spelling, and/or organization.

4. What Was Learned

Based on the analysis of the data, and compared to the threshold values provided, what was learned
from the assessment?

a) Areas of strength

Students demonstrated a strong understanding of contemporary issues in natural resource
management. This assignment was designed to be a critical evaluation of an existing management issue
in Montana and the students appeared to be engaged in the issue. The assignment prompted the
students to consider multiple perspectives of an issue and support or refute these perspectives using
scientific literature. For the most part students clearly presented their perspective on the issue and
provided scientific support for their stance, however there was some variability in the rigor with which
the supporting information was sought out and presented. This evaluation showed that NRRE students
seem generally well versed in contemporary issues in natural resources management and well prepared
to investigate and seek out scientific support for their stance on these issues.

b) Areas that need improvement

Several students’ papers lacked depth in evaluating alternative perspectives. Although most students
could identify the issue and articulate their stance, a more in-depth evaluation of opposing viewpoints
would have benefited these papers and the critical thinking activity (Indicators 3 and 4). This critique is
interesting because the assignment prompted students to critically evaluate multiple perspectives of an
issue however the lower scoring papers lacked effort in developing a robust and scientifically supported
argument. Recognizing and developing a well thought out and supported idea is an area that our
students could improve upon.

The students evaluated varied in their writing skills and as mentioned in previous NRRE program
assessments, a poorly written paper can mask the knowledge possessed by any student. Although
writing is not the learning objective evaluated in this assessment, it is a skill that should continue to be
cultivated in our students so they can convincingly convey their knowledge and provide sound and
defensible evaluation of natural resource issues.

5. How We Responded

a) Describe how “What Was Learned” was communicated to the department, or program
faculty. Was there a forum for faculty to provide feedback and recommendations?

A copy of this report was provided to the Range Program Faculty and, along with the 2019/2020
assessment, these findings were discussed at an August 2021 Faculty Retreat. The Range Faculty was
generally pleased with the results of this assessment, however they discussed the need to promote
critical thinking as a developmental process where students are exposed to critical thinking activities
throughout their academic careers. Faculty agreed to place greater emphasis on critical thinking in
classes.

b) Based on the faculty responses, will there any curricular or assessment changes (such as plans for
measurable improvements, or realignment of learning outcomes)?

Answer: NO

If yes, when will these changes be implemented?

Please include which outcome is targeted, and how changes will be measured for improvement. If
other criteria is used to recommend program changes (such as exit surveys, or employer satisfaction
surveys) please explain how the responses are driving department, or program decisions.

c) When will the changes be next assessed?

The learning outcome “Critical Thinking” will be assessed again in 2022-2023.

6. Closing the Loop

a) Based on assessment from previous years, can you demonstrate program level changes that
have led to outcome improvements?

No. Based on this assessment, our outcome scores improved slightly since the last assessment in
2017. The 2017 evaluation had an average score of 2.25 and 75% of the students met the threshold
of a score of two. The recommendations from the last evaluation of this learning objective were
geared toward the development of better writing skills for NRRE students. This remains an issue
and the suggestions brought forth in 2017 remain valid. The 2017 assessment also identified
inadequate student response to the assignment as an issue and the evaluators in the current review
also observed this in the lower scoring papers.

That the reviewers in this assessment found that some students lacked robustness in their critical
thinking activities suggests a need to provide more opportunities for students to develop critical
thinking skills and to understand the effort required to critically evaluate natural resources issues.
Incorporating more critical thinking exercises in courses within the NRRE curriculum will aide in
providing our students with the opportunity to cultivate this skill set.