Portfolio Assessments: A One–Way Ticket to Better Evaluations
Abstract
Using a portfolio as a course assessment tool yields a twofold benefit. First, it can enhance the active learning components within the course. A portfolio encourages active student involvement, offers a mixture of opportunities for student success, and promotes critical thinking, creativity, and development of informed judgment. Second, it improves student scoring on course evaluations. Comparative results provide demonstrable evidence that student–driven evaluations are higher using the portfolio approach than not. Opportunities abound to employ portfolio assessment across disciplines.
Introduction
Evaluation of teaching effectiveness is an integral part of the review process for faculty at universities and colleges. Salary, reappointment, promotion, and tenure decisions are contingent upon the outcomes of the evaluation process. Performance indicators used to document teaching effectiveness include peer, self– and student evaluations. Peers are apt to be kind in their evaluations. An upward bias tends to prevail in self–evaluations. But students, the primary benefactors of our instructional efforts, are in a position to candidly assess the successes and shortcomings of our classroom performance. As such, it behooves a faculty member to develop a product that balances professional integrity with an eye toward student success.
Evidence gathered by educational psychologists and instructional specialists contends that students have a higher commitment to learning when actively engaged and that active learning promotes positive student attitudes. Using a portfolio as a basis for assessment is one vehicle to promote active learning. It can be designed to employ a variety of assessment techniques offering a mix of opportunities for student success. Progressive writing assignments make students shoulder more of the burden for their education. Assessment moves up Bloom’s Taxonomy by embedding opportunities for critical thinking, creativity, and developing informed judgment on contemporary social issues.
Active learning benefits not only the student but also the instructor. When students feel more successful in a course, this sense of success translates into higher scores on evaluation instruments. Comparative results provide demonstrable evidence that student–driven evaluations are higher using the portfolio approach than not.
Example of a Portfolio
One example of a portfolio that is used in an introductory economics course is composed of five basic elements created by students over the course of the semester: 1) a reaction paper, 2) position papers, 3) a haiku, 4) an economic journal, 5) a reflective essay.
1. Reaction Paper: The reaction paper is the first element of the collection. The goal is to facilitate learning both the introductory concepts and the rhetoric of economics and to encourage students to connect economics to a real world personal experience. The paper is meant to be open–ended, and informal, yet, creative. Topics or principles drawn from the first section of course content are eligible for use in the reaction paper.
2. Position Papers: Position papers provide structured learning opportunities for students to apply the power of economics to understand and to solve contemporary socioeconomic problems. In fact, the position paper is the goal of the class and serves as a substitute for a formal exam over eighty percent of the covered material. Position papers provide an opportunity to assess student learning at cognitive levels higher in Bloom’s Taxonomy (Knowledge > Comprehension > Application > Analysis > Synthesis > Evaluation) than mere knowledge and comprehension.
3. Haiku: The haiku is a lesson in brevity. Written after the final position paper is completed, the purpose of the haiku is to encourage creative expression in the affective realm of learning.
4. Economic Journal: The journal promotes economic literacy by requiring students to tune into current news events and media presentations outside the classroom. Economic concepts and principles taught during the final section of course material are fair game for journal entries.
5. Reflective Essay: The reflective essay is the final feature of the portfolio. This is the instructor’s figurative apple. The essay’s objective is to encourage students to reflect on their experience. Students are asked to consider how their personal thinking about economics, their opinions and values on contemporary economic issues, and their reading, writing, and research skills have been affected by the introductory course.
6. Portfolio Collection: Putting It All Together: The finished product is bound in a three ring binder or other bound format. The components include: 1) a cover page, 2) a table of contents, 3) a reflective essay, 4) a reaction paper, 5) a haiku, 6) a revised position papers, and 7) an economic journal.
Methodology
As a first experience course, an introduction to economics uses the power of the discipline to explain the nature of, cause of, and solution to contemporary issues facing American society. Students are exposed to basic terminology and concepts crossing both microeconomics and macroeconomics. The course pedagogy is a mix of lecture and group discussion.
Students are encouraged to begin thinking early in terms of where, when, and how they have encountered economics in real life. These experiences become the substance for the reaction paper. Any introductory concept from the role of choice and opportunity cost to the mechanics of the model of supply and demand are eligible topics. Students take ownership of the vocabulary at this stage. Elzinga (2001) asserts that giving students this ownership inspires a lasting appreciation of the subject because the terminology serves as a memory peg. The due date for the reaction paper coincides with the first examination. If a reaction paper misses the mark, the student is encouraged to rewrite and resubmit.
Approximately eighty percent of the introductory course is spent examining contemporary socioeconomic issues, including an analysis of the priorities established in the federal budget, the magnitude of the national debt, the solvency of the Social Security program, implications for the U. S. income distribution, and the economics of crime, pollution, and discrimination. Four position papers are written which serve as substitutes for taking a formal exam.
The lecture mixed with discussion introduces each new socioeconomic issue. A consistent presentation format is employed to help students logically organize information. Interest in an issue is, first, peaked by soliciting student perceptions and establishing common ground for future discussion. Next, a review of current and historical statistical evidence gleaned from authoritative sources is presented. The purpose is to highlight the height, width, depth, and breadth of the issue. Functional literacy in the use and interpretation of statistical data is encouraged. Occasionally, students are asked to track down information on the Web. During these Web assignments, students hone Internet surfing and computer literacy skills initially by locating pre–specified URLs and downloading and printing requested information. In time students engage in independent searches. An analysis of the role of economics and an introduction to relevant terms and principles follow. Finally, students offer possible solutions and discuss potential positive and negative implications. Throughout this stage, note taking is required and rewarded. Carrier (1983) concludes that note taking is a legitimate component of the educational experience. Additionally, Becker (1979: 1362) suggests that “[p]eriodically collecting and reviewing students’ classroom notebooks is an assessment device that encourages students to take the classroom activity seriously and provides information on what students are attending to in class.” Note taking activities develop the ability to distill the essential ideas from minor points.
The concluding activity is the preparation of a position paper. Students respond to a well–structured question on the issue. Critical thinking skills are fostered as a student progresses from paper one through paper four. In general, each paper requires a restatement of the socioeconomic problem; a presentation of empirical evidence either affirming or refuting the issue at hand; a clarification of definitions, if any; an explanation of the role of economics; and, finally, an evaluation of proposed public policy choices.
Students submit their papers twice. The first submission results in a grade. Students turn in not only their response to the question but also all notes taken plus any Web assignments. The second submission of only the revised response occurs two class days later electronically. The class papers are converted to one composite PDF file and posted to the Internet, thereby expanding the audience well beyond the classroom.
The haiku, a seventeen syllable Japanese poem, is a lesson in brevity and a creative expression of affective learning. The haiku is prepared within a week of completing the fourth position paper. Students are encouraged to draw on one of the socioeconomic issues in writing the haiku.
The last segment of the course provides insight on the operation of the macro economy. Students are encouraged to tune in to the world beyond their front door. News events, media presentations, and personal discussions related to the macro economy are used to prepare dated entries for the economic journal.
The reflective essay, the final element of the portfolio collection, is collected on the day of the final examination. This essay provides an opportunity for students to reflect and assess a semester’s efforts. The instructor has an opportunity to evaluate student growth beyond their grades. Several prompting questions are suggested:
• How has my thinking changed about economics in general, in the level of appreciation for the subject matter, and in the creative thinking and problem solving capacities?
• How does the subject matter of this course relate to my overall learning, my ability to read and analyze news events, my ability to see the whole picture as well as the parts, my future course work or life?
• How has the cou rse material or course activities changed the way that I think, my opinions or values concerning contemporary social issues, and my tolerance of opinions or perspectives different from my own?
• How have my reading, writing, and research skills been affected as a result of my work in this course?
Comparison of the Spring and Fall 2001 Courses
The same instructor taught introduction to economics during both the spring and fall semester of 2001. The fundamental aspects of the course remained consistent. Both standardized course goals and learning outcomes were unchanged. Economics of Social Issues and a companion reader, The Economics of Public Issues, continued to be used following an unchanged course outline. Assessment continued to be geared toward meeting the 39 pre–identified student–learning objectives. Students wrote a total of four position papers in both the spring and fall semesters. Course evaluations were conducted using the university’s instrument as well as a more detailed instructor instrument.
Two key differences between the courses involved the added burden placed on students in the fall class and the increased variety of opportunities for students to experience success. The portfolio approach to assessment required each student to prepare a reaction paper, haiku, an economics journal, and a reflection paper in addition to writing the two formal exams and four position papers. The posting of position papers to the Internet increased the viewing audience during the fall semester. At the conclusion of the fall course, each student possessed a polished and bound collection of work as evidence of a semester’s efforts.
Evaluations
Formal student evaluation of teaching effectiveness at the university is handled using an 8–statement (Woods) instrument with rankings that range from 7—excellent to 1—very poor. A second instructor developed instrument (Non–Woods’) poses 48 evaluative statements with rankings running from a high of 5 to a low of 1. Comparative results provide demonstrable evidence that student–driven evaluations are higher using the portfolio approach to assessment in the introduction to economics course than not. Table 1 (p. 126) reports the results from the university evaluation form. The eight criteria are identified in the first column. Respective scores from the two semesters are recorded in columns two and three. The percentage improvement in scores using the portfolio pedagogy is reflected in the last column. Student Accomplishment, that is, the degree to which the economics class helped the student increase in knowledge, skills, and competencies in comparison with other classes taken by the student improved by 35.4%. The Total Experience score (including teaching, content, readings, tests, outside activities, and homework) increased 33.6% over the spring semester.
The instructor–developed instrument echoes similar positive results. Most striking, though, is the percentage gain in student self–evaluation, 31.8%, in Table 2 (p. 126). This is the first time that the score for student self–evaluation exceeded that of the instructor evaluation. One advantage of the instructor–developed instrument is its attention to detail.
The first twenty statements relate to instructor performance; the eighteen following statements focus on the course, and the remaining statements are directed at student self–evaluation1. Notable among the course evaluation items are #32, #35, #28 and #36. The students disagree that the course should be taught in some other way (–26.5%) and that the course material was too difficult (–23.1%). The scores reflect approval of the portfolio approach in #28 and #36. Students agree that the course was one of the best ever taken (31.1%) and should be required of all students (18.0%). Students acknowledge the effect of an increased workload (#27) by reporting that the amount of required work was not as appropriate (–2.7%). Gains were made in several student self–evaluation areas. Significant among these are #47, #45, #48, #41 and #46. Students report a 28.1% gain in
| Table 1: Woods’ Form | |||
| Econ 200 | Spring 01 | Fall 01 | |
| Sample Size | #18 | #23 | % Change |
| Scholarship | 5.28 | 6.4 | 0.229 |
| Organization/Clarity | 4.89 | 6.4 | 0.285 |
| Interaction w/ Group | 4.94 | 6.3 | 0.267 |
| Interaction w/ Individuals | 4.72 | 6.2 | 0.312 |
| Enthusiasm | 5.39 | 6.4 | 0.227 |
| The Instructor | 4.89 | 6.5 | 0.281 |
| Total Experience | 4.67 | 6.2 | 0.336 |
| Student Accomplishment | 4.56 | 6.2 | 0.354 |
| Average | 4.92 | 6.3 | 0.28 |
| Table 2: Non–Woods’ Form | |||
| Econ 200 | Spring 01 | Fall 01 | |
| Sample Size | #19 | #22 | % Change |
| Instructor Evaluation | 4.41 | 4.61 | 0.045 |
| Course Evaluation | 3.44 | 3.64 | 0.058 |
| Self–Evaluation | 3.58 | 4.72 | 0.318 |
| Average | 3.87 | 4.15 | 0.072 |
greater tolerance for opinions different than their own, an increased ability to see the whole picture as well as the parts (22.8%), a greater appreciation for the subject matter (22.2%), an ability to apply course material to new situations (20.7%), and the development of an informed concern for contemporary social issues (20.4%).
Conclusion
Empirical evidence supports the notion that students have a higher commitment to learning when actively engaged. The portfolio approach to assessment encourages student involvement. Salemi (2002) identified several rationales that explain the success of active learning pedagogies. Student reaction to using a portfolio approach in the introductory economics course highlights the impact of these rationales.
1) Students reach a deeper understanding of concepts by developing creative thinking and problem–solving skills:
This economics course has taught me to analyze situations first, before I come to my conclusion. I used to be the kind of person who would go with my first reaction rather than think about the situation and what the effect might be. I learned to think as an economist rather than an individual. The idea is to increase social well–being rather than trying to benefit myself. I learned that what is right for me would not necessarily help society as a whole. (M. Woodburn, F01)
I have, also, found that through the writing of the position papers, I was offered a chance to voice my opinion in a way that I may not have ordinarily had. I was able to actually think like an economist and offer a solution to some of the problems that our country faces. If not for this class, I may have never had that opportunity, nor would I have done it on my own. The papers also helped me to understand the objectives much better by relating the issue to real life situations, like social security. Talking about it in class and taking notes is one way of understanding, but to actually write it down in an organized fashion and be forced to make sense of it, really helps to get the information to stick in your memory. (J. Fleming, F01)
2) Mutual respect in the student–instructor relationship is fostered:
It seems to me, through all the research papers we have done, you gave us a chance to express ourselves as individuals, and it let you know what we have learned from your lectures and from the readings. I found out that there is no way you could have written a position paper well if you did not read the chapters it covered. As a teacher you probably know that there are students who do not like to read text material and just study from their notes. I could even be one of these, but you did a good job with the structure of your class. And you have definitely expanded my mind with information that I did not know was imaginable. Maybe, you even learned a bit about us in return. (K. Wysenski, F01)
3) Differences among student learning and assessment styles are capitalized.
Using a variety of pedagogical and assessment techniques increases the probability of student success. Lectures highlight foundation material. Discussions build on the realm of possibilities. Variety in assignments enables students to balance an area of weakness with one of strength. Progressive writing assignments push students from the passive role of audience to the active role of writer, thinker, and communicator. The position papers and haiku provide opportunities for critical thinking, creativity, and developing informed judgment on issues of real concern:
When I entered class on the first day, I did not know what to expect and after going through what was expected of us I was not sure I was going to like it. I did not like the fact that we had to write all those papers. It seemed like so much at the time and I did not think I was going to be able to do it. But what I did not realize was that those papers were like our test. And by the end I was not at all upset about writing the papers. I actually came to enjoy them because they helped me understand the information much more than a test ever would. The reason I think they helped me so much was that I had to actually think about the topic before writing and had to connect it to something that was going on in the economy. (B. Sabo, F01)
4) A positive attitude toward economics is developed:
I have a new–found likeness for economics. I actually read the business page now! I am aware of what the numbers mean and directly or indirectly how they affect my family and me. I have even chosen to take another economics class next semester. Trust me, this fact alone is astonishing as my interest in economics was nonexistent when I started. This class has definitely opened my eyes in more than one way. And I will try to show my children that they need to be a little more aware of how everything around them is affected in one way or another by economics. I don’t want them to go through twenty–eight years of their life, like I did, being oblivious to that fact. (A. Cammalleri, F01)
5) Tolerance for different opinions and perceptions are encouraged through peer–to–peer exchanges:
In class there were many different discussions. We discussed things from crime to social security to pollution to poverty and much more. Throughout those discussions there were many different views. Some people had the same view as I did, that made me feel that my view was right. Many people, also, had views there were different than mine. I had to think about those views and usually I would decide that both views could be correct. Before this class I thought that when dealing with economics there was one right answer and a lot of wrong answers. I now understand that there are many right answers and many wrong answers. By considering other peoples’ views, I feel that I have become a better person, as well as, a better economist. (R. Boreman, F01)
6) Practitioners are challenged:
To be a successful practitioner of the portfolio approach requires a commitment of time that greatly exceeds the requirements of a chalk–’n– talk pedagogy. This portfolio collection approach requires a commitment of time to search for the latest news, the most current statistical evidence, a commitment of in–class time to allow opportunity for verbal expressions from students, a commitment to read pages upon pages of response papers, and to fairly offer constructive suggestions for revision, and the task of rereading those same papers. Yet the commitment is not without rewards. Student growth is evidenced in the testimonials offered in the reflection papers. Acknowledgments from students who have used their newly acquired economic knowledge to win scholarships and awards is a source of personal satisfaction. And the improvement in student scoring on course evaluations reflects a high level of satisfaction with this approach. Portfolio assessments, employable across disciplines, appear to offer a one–way ticket to better course evaluations.
Endnotes
1A copy of this instrument can be obtained by contacting the author.
References
Becker, William E. 1997. “Teaching Economics to Undergraduates.” Journal of Economic Literature 35(3): 1347–1373.
Carrier, C. A. 1983. “Notetaking Research: Implications for the Classroom.” Journal of Instructional Development 6: 19–25.
Elzinga, Kenneth G. 2001. “Fifteen Theses on Classroom Teaching.” Southern Economic Journal 68(2): 249–257.
Miller, Roger LeRoy, Daniel K. Benjamin and Douglass C. North. 2001. The Economics of Public Issues. Boston, MA: Addison Welsey Longman, Inc.
Salemi, Michael K. 2002. “An Illustrated Case for Active Learning.” Southern Economic Journal 68(3): 721–731.
Sharp, Ansel M., Charles A. Register and Paul W. Grimes. 2002. Economics of Social Issues. New York, N.Y.: McGraw–Hill/Irwin Co.
Biography
Kay E. Strong is an assistant professor of economics and applied statistics at Bowling Green State University—Firelands. Strong can be reached by e– mail at kstrong@bgnet.bgsu.edu.