PsycholARTSical: Psyched about the arts/Assessing and Evaluating Creativity

Assessing and Evaluating Creativity

Introduction

Preamble

One of the key tasks for teachers of all subjects is to be able to assess and evaluate student performance. Various disciplines employ a range of assessment and evaluation techniques, which can be tailored to meet specific student needs and goals. Although it can sometimes be a challenging task for all teachers, it is especially so within disciplines where subjective/qualitative assessment is required – the arts.

Assessment and evaluation are uniquely elusive when it comes to subjects like Visual Art, Music and Drama. In addition to measuring knowledge and understanding, thinking, communication, application, effort and technical skill, teachers are given the difficult task of gauging a student’s creativity.

Creativity is understood to be a highly individualized quality – it can materialize in an infinite number of ways that rarely fit into a given set of criteria. In fact, the premise of creativity rests on the synthesis of new and unconventional ideas. Creativity is also something very personal – it often represents an emotional investment by the student and, therefore, requires sensitivity on the part of the teacher. These challenges raise an important question: how might teachers go about assessing and evaluating creativity in student work?

What does it mean to be creative?

According to dictionary.com, the adjective creative is defined as having the power or ability to create. This definition appears to focus on someone’s ability to produce rather than the quality of the work produced. This view of pluralism is sometimes criticized in artistic circles for perpetuating the trend to make things for the sake of making things. An example might be an artist who produces vast volumes of work, but spends little time contemplating the significance of each piece. This person is productive but is he or she creative? Later in this book, we will be looking more closely at the role of self-reflection and self-assessment in creative processes.

Creativity – the ability to transcend traditional ideas, rules, patterns, relationships and to create meaningful new ideas, forms, methods, interpretations, etc.; originality, progressiveness, or imagination. (dictionary.com)

The initiative to transcend conventions and create meaningful new ideas engages the students’ higher order thinking skills that deal with Application and Synthesis as per Bloom’s Taxonomy. This indicates that creativity operates at a fairly high level of cognition. It also means that in order to function at that level, students must build upon the more fundamental cognitive skills such as Knowledge, Comprehension and, in turn, Analysis.

So, for example, a student of classical music will learn notation first (knowledge). He or she will then gain an understanding of musical theory (comprehension). Finally, the student will practice the ability to examine the qualities of existing pieces of music (analysis) before applying all of the above to compose an original piece (application and synthesis).

The way we assess or evaluate higher order thinking is important because there is a certain risk involved for students. In synthesizing new ideas, students open themselves up to peer criticism and possible embarrassment. For this reason alone, many learners choose to “play it safe” and conform to what the majority of their peers are producing. Therefore, it is crucial to consider the effectiveness of the different methods of measuring creativity.

Assessment vs. Evaluation

For the sake of further discussion, it is important to distinguish between assessment and evaluation. While the two functions are commonly paired, they differ considerably in purpose and approach.

Assessment is generally understood to be formative. In other words, assessment takes place while learning is happening. It is an ongoing process that is intended to improve learning by allowing the teacher to respond to student needs as they become apparent. This approach is associated with holistic/constructivist methodology because it provides a “safe to fail” environment that encourages self-expression and creative risk-taking.

Evaluation is summative – it is normally done after a unit of learning has been completed. Evaluation focuses on the formal grading of student work and, therefore, it is numerical and final. It is often associated with competition, performance and risk. While grades can be a powerful extrinsic motivator, they do little to encourage students to try new things because students want to minimize the above risk.

Are both assessment and evaluation appropriate methods for gauging creativity? Teachers, students and parents would agree that creativity should be rewarded. While assessment tends to offer more flexibility in working with subjective qualities, evaluation appears more readily applicable to the appraisal of technical skill and effort levels.

--Artursedov (talk) 20:41, 22 April 2008 (UTC)

Peer Assessment and Professional Arts Juries

This report uses the "Peer Assessment" model as prescribed by professional arts funding agencies such as The Canada Council for the Arts, the Ontario Arts Council, and the Toronto Arts Council to address the issue of how creativity may be evaluated in a contemporary high-school arts classroom. For the purpose of this address, my principal source is The Canada Council for the Arts. In brief, there are advantages and disadvantages to this model, however, its' effective use in the classroom makes specific demands on the evaluated which may, or may not, be compatible with the learning objectives of the teacher.

The following discussion on Arts Juries is broken into four parts:

1) How They Work;

2) When They Work Well;

3) When They Don't Work;

4) Peer Assessment in the Classroom.

1) How They Work

Juries are selected from creative practitioners within the discipline being judged, (dance, theatre, music, visual art), and are—within each discipline—specific to certain job descriptions like, performer, writer, director, producer, administrator. Juries must also be balanced in terms of gender, cultural background, and political/religious/aesthetic affiliation. Most juries need four to six people. Canada Council (CC) Jurors are paid well relative to a freelancer’s wage.

Juries are selected by an executive board, who also produces the event: handling communications; travel and lodging for out of town jurors, delivery of all print materials (a substantial amount of paper is used), and catering.

The executive also administers the adjudication, handling applications, to printing, to overseeing the 2 to 3 day process by providing structure to the meetings, and accelerating the process of elimination, which is what the whole thing is about. The criterion of adjudication is determined by the jury, based on vague administrative guidelines.

Essential parts of the application:

• a clear and justifiable mission statement
• an accessible track record: media documentation, press reviews, endorsements and testimonial
• an application is judged by comparing the proposition with the company’s mission statement and track record

2) When They Work Well

In brief, when the process works well, you get the funding you asked for, then execute the responsibilities you’ve outlined in your grant application. Although it is hard to gauge what any one successful candidate may learn from the process, there are a number of qualities successful applicants seem to have in common. By looking at a few of these it is possible to infer a learning curve for the evaluated over time.

organizational: Good grant writing requires detailed record-keeping and the maintenance of healthy professional relationships. Because most grant applications require a press file—print reviews, features, interviews and transcripts from broadcast media—the artist has to have these at hand. Also, because most applications require a sponsor, or professional endorsement of some sort, successful artists must maintain relationships with respected, often senior, members of their artistic community. Having a friend is one thing. Having a friend who is a respected member of your artistic community who is willing to both, risk his or her artistic reputation before the CC, and, take the time necessary to review your application then write an endorsement which speaks to the details of your proposal, is quite another thing.

metacognative: Successful, grant-getting, artists must be rigorously self-aware of not only their body of work, but also, of their creative process because, for most artists funded by the Canada Council (CC), their initial successful grant application is the first substantial step in the creation of a new work. Also, because most grants are awarded to fund a specific phase of artistic development in the creation of new work, the artist is required to be detailed in imagining their artistic process, securing the personnel and means of production, budgeting for expenses, and determining the length of time necessary to fulfill their artistic obligations for the money awarded. The end result is that a successful artist knows who they are, what they have done, what they plan to do, how they work, how they have failed and succeeded in the past, exactly how they hope to achieve their next task, who they can call on for help, and how much time they will need to accomplish their proposed short-term goals.

responsibility: Accountability is the key to successful grant-getting. Artists have to make good on their previously funded promises, and continue to develop according to their own often-stated prescriptions for success.

3) When They Don’t Work

There are significant downsides to Peer Assessment as practiced by the CC. Here are a few.

emerging artists: Because so much of the grant application is dedicated to assessing an artists’ creative ‘track record’ it can be difficult for the new artist to state their future claim against their past record. This aspect of the process requires new artists to seek respected artistic mentors for ongoing support, and legitimate media outlets for recognition.

results are final: Lessons for unlucky applicants can only be achieved accumulatively over several failed attempts, and a synthesis of imperfect feedback. Usually a simple sentence of anything like criticism appears in the rejection letter. A follow up appeal might yield something along the lines of, “…the jury didn’t feel that the application was strong enough to justify a new direction in terms of your mandate.”

subjectivity: All judgments in any creative process, no matter how well structured, are subjective and fallible. At the extreme end of this paradigm is something called ‘low-balling’ whereby one low score from any juror on any round of evaluation can sink a proposal. This tactic is often the preserve of the grudge-bearing juror who has failed to disclose their bias to the rest of the jury. This is rare, although every unsuccessful candidate will undoubtedly claim to have ‘an enemy’ on the jury…and sometimes they are right. What works against the tendency toward spurious judgment is the peer process itself. The process is intense, and usually lasts two or three full days. Also, any one juror may well find his or herself in the company of four or five substantially qualified, and well-connected, artistic peers who are more than capable of arguing aesthetic principles, and justifying their choices in a thoroughly rational manner. There is nothing whimsical about this process. Those who treat their role as an adjudicator lightly (or strictly emotionally) will not be asked back, will embarrass themselves in front of influential members of their community, and, will jeopardize their standing with the country’s most powerful arts funding body.

4) Peer Assessment in the Classroom

The virtues engendered by successful grant-getting artists are compatible with the goals of arts education to a limited extent, namely, the virtues of self-awareness, metacognition, demonstrable discipline in terms of process, organization, and accountability. In the classroom however, it is difficult to establish a deep sense of accountability because the goals for success in the classroom are not as well defined for the students as they are in the professional arts where all candidates strive for just one goal: to get the money. If all students were driven to achieve but one goal—good marks for instance—then it might be easier for the teacher to police against favouritism, and harsh judgments, but this is not the case. Many students are all too willing to risk getting a good mark for the sake of peer-pressure, status-achieving behaviour, or short-term popularity.

The task then, for a teacher who wishes to implement this type of assessment structure, is to start early in the academic year, work closely with the jurors for the first few cycles, and then be willing to use the process as many times as is necessary for the students to see the repercussions of their actions as jurors and prospective candidates. The learning curve really takes off once students can weigh the cost of a previous harsh judgment when it is their turn to be judged.

If well-done, this type of assessment can foster academic skills for students in three significant areas: metacognative; critical/analytical; intra-personal accountability. It does need to be scaffolded though, and the likelihood of success is much greater with mature students in a studio/conservatory type of environment where students learn together in close-quarters over the course of more than one year.

Jameso'reilly (talk) 19:48, 24 April 2008 (UTC)

Self-Assessment & Evaluation

Definition: "The evaluation or judgement of ‘the worth’ of one’s performance and the identification of one’s strength and weaknesses with a view to improving one’s learning outcomes” (Klenowski, quoted in Ross 2).

Three Types of Self Assessment (Rose 4-6)

1. Co-created rubrics for assignments: Students and teacher discuss and list what the evaluative criteria or goals of an assignment should be as well as defining the levels of success for those criteria.

2. Short answer questionnaire: Students are asked to respond to questions about their performance. For example:

- What was good about your performance today? Name three things and explain why.

- What needs improvement? Give 3 ideas.

- Explain some exercises that would help you make those improvements.

3. Students rate themselves in specific areas of their performance and conduct. For example:

- On scale of 1-5, rate yourself on the following criteria:

i. I came to class having practiced all of my parts.

ii. I come to class on time and promptly assemble my instrument.

iii. I can hear when I am playing well.

Why Teacher’s Use Self-Assessment (Ross 2)

1. It gives students the opportunity to contribute to the criteria on which their work will be evaluated increases their engagement with the work.

2. It contributes to variety in assessment methods, which aids in maintaining student interest and attention.

3. It provides information that might otherwise not be as easily determined:

- Students are given a forum to assess and comment on the degree of effort they put into an assignment.

- Students often bring unthought-of criterion to attention that a teacher might have otherwise overlooked.

4. Some argue that students learn more when they share the responsibility for assessment and that increased attention and engagement lead to increased learning.

Is Self-Assessment Reliable? (Ross 3)

Research shows that self-assessment strategies have been largely consistent, especially with older students (i.e. Grades 7 and up) who might better understand the criteria and how to apply them. Self Assessment strategies have been more effective with shorter assessment periods. Longer assessment periods would suggest larger assignments with a greater number of criterion that are harder for students to remember and regulate.

Does Self-Assessment Provide Valid Evidence of Student Performance?(Ross 3)

There are some discrepancies in this area:

1. Student ratings are often higher than teacher ratings when the assessments contribute to a grade, as the temptation to inflate their own grades seems to skew student judgement.

2. There tends to be more consistency in student ratings when students are informed that their ratings will be compared to the teacher’s (i.e. using the same rubric) and that they will have to explain any large discrepancies between their ratings and those of the teacher.

3. Students may confuse their ability to meet the criterion of an assignment with the effort that they put forth to meet that criterion. In light of what students might regard as a great degree of effort that was put toward an assignment, they might be more inclined to inflate their ratings to reflect their efforts.

- As problematic as this might seem, it does give the teacher the opportunity to identify and act upon student difficulties that might otherwise go unidentified.

Strengths of Self-Assessment (Ross 7)

1. Students have a better understanding of their goals through their active role in creating the dimensions of a rubric. Thus, there is often a greater degree of success in attaining those goals.

2. Students have commented that self-assessment was a fairer form of assessment because it allowed them to incorporate criterion that would not otherwise be included (such as student effort or extra-curricular time commitments).

3. Students can communicate information to the teacher (i.e. goals and rationale) that they would otherwise not be able to.

- Self-assessment gives students information that they can use to improve their own performance on an assignment by encouraging students to focus on attaining specific criteria and tracking their own improvement in those areas.

Weaknesses of Self-Assesment (Ross 7-8)

1. There is a concern that students will lower academic standards by inflating their ratings on their assessment of their own work.

2. Some students might not want to participate in the process, feeling that it is not their responsibility to carry out what they regard as the “teacher’s” job.

3. Such an attitude might decrease student motivation and commitment to their work. Nonetheless, this does not apply as much to uses of self-assessment in the formative assessment of student work that would be taking place throughout the progress of a curriculum unit. In conjunction, self-assessment and teacher feedback can be powerful tools in helping students succeed with their work.

4. It takes a lot of class time construct rubrics with students. It can often take up to three periods to work out the details of a student created rubric. Revised drafts often need to be presented to and reworked with the class over the period of a few days.

Four-Stages Involved in Making Self-Assessment More Effective (Ross 8-9)

1. The teacher should not surrender control when creating rubrics. The teacher should use the process as a means to develop students’ deeper understanding of expectations in a manner that will allow them to relate to those expectations in their own language.

2. The teacher must explain how to apply the criteria of a rubric by providing exemplars as well as modelling the desired expectations for the class.

3. The teacher should promptly provide comments and feedback for a student’s self-assessment in an effort to clarify the expectations and criteria within a rubric. Comparisons between a student’s self-assessment and the teacher’s (or a peer’s) assessment of the same work using the same criteria may be use to resolve discrepancies and discourage inflated ratings.

4. The teacher should assist their students in using self-assessment data/ratings in a manner that would improve their performance with future assignments. Providing students with recording forms to track performance over time will help them establish links between their immediate goals and their long-term goals.

--Maurosavo (talk) 03:40, 23 April 2008 (UTC)

Case Study: A Model for Self-Assessment and Evaluation

In music class, students are expected to improve their technique and sound when it comes to playing an instrument. Many teachers still enforce the standard practice of a playing test, which is typically done in front of one’s peers on a specific day. But how can you truly assess someone of his or her growth when performance anxiety may play a huge role in the end result? Some teachers have found that this rigid process is very ineffective in terms of progress assessment; although it may only require one day to ‘get through everyone’, it often becomes an unfair judgment.

Technology today is far more innovative than when I was in high school, even though that was only five years ago. The following process is one that I plan to implement into my music classroom, and although I cannot take credit for the idea, I hope that other music teachers, like me will put this strategy into effect.

A student enters the classroom knowing that today is the day that he will be tested on his trumpet. He is a grade nine student who has only just begun playing and instrument for the first time in September. It is now late in October, and he has been practicing hard both at home and in the morning ensemble classes. He is not scared, because he knows that there is no pre-judgment, as he has never been tested before. Yesterday, the teacher assured the class that they would not have to play in front of one another, but rather only her, in one of the practice rooms as the rest of the class has the period to work in sectionals on the two pieces of repertoire that they have been learning. She then explains that she will be recording each of their playing tests as a way of assessing their progress throughout the year.

The teacher begins the class by leading a warm up so that everyone is in top-notch form. Students will then take turns taking their test in the practice room as per a sign up schedule that was posted in the class last week. Once the student enters the practice room, he or she is allowed to set up the music stand in which ever way is most comfortable for them, along with deciding whether they want to sit or stand. From there, the teacher records the playing test while making a few notes. Once the student has finished, her or she has the option to record again. This is important especially for the first test as many students suffer from performance anxiety.

A few days later, when all the playing tests are done, the teacher has each student listen to their playing test, assess what they have just heard, and only then does she explains to them their rubric markings. The teacher may need to play the playing test a second time in order to accurately explain her assessment, and this is a good idea as it allows the student to have both a visual and aural assessment. The teacher highlights the appropriate sections of a rubric in one distinct colour, gives a photocopy to the student and stores the original in a specific file for each student. When the next playing test comes around, the teacher will use this original rubric for each student, but will highlight the appropriate sections in a different colour. This process will allow students to visually see their own progress as well as areas in which they can improve.

This is a very strong way of assessing creativity as it includes both teacher and self-assessment. It also allows the students to build on their learning, and to see how far they have come since the beginning of the school year. Teachers should allow the students to compare their progress by listening to past recordings throughout the year, as this is also a great tool for motivating students to learn. It is especially important that students listen to their first playing test following the last test of the year. What a great way for students to see the actual result with documented proof.

--Winchell (talk) 00:20, 26 April 2008 (UTC)

A Unit Based on 'Creativity'

From Personal Experience...

During second practicum, I decided that I wanted to further explore various approaches to assessing creativity within the classroom setting after having done a lot of research and engaging in great conversations with my seminar group. With permission from my associate, I designed a ‘Self-Reflection’ unit for the grade nine vocal music classes that allowed them to explore their creative side while learning basic skills on how to analyze music. I tailored this unit towards the current interests of the grade nines as I used a variety of music samples from those artists who they listen to on a daily basis. I used weekly listening quizzes as a diagnostic assessment for what the students needed specific lessons geared towards and I fabricated my lessons to fill in the gaps in their learning.

The summative activity for this unit was the ‘Self-Reflection’ project (or presentation), for which I allowed the students the freedom to choose how they wanted to create the visual aspect, but at the same time, I guided them through the assignment with a clear outline of specific requirements that needed to be included in some manner. The various dates for this project were clearly outlined on a calendar in the classroom, so that the students would always know what to expect during each class. A total of one hour of class time was given for this project over a span of two weeks (excluding other accompanying activities), which allowed them to collaborate with one another in order to format their project. During this time I divided my time equitably between the class in order to give guidance or support in any possible means that I could.

Just as there always seems to be one student who has trouble staying on task, so was the case in both of the grade nine classes. In order to ensure success from all the students, I had the students write a ‘progress report’, which was their opportunity to tell me, in writing, what they were creating for their project, what they had done thus far, and most importantly what supports I could give them (if needed). This activity confirmed many of my initial observations, which highlighted that the vast majority of the students in both classes were making good progress; however a few students did not fit into this category and I quickly turned my attention towards them. A couple of the students had been off ill and needed some assistance as to how they might get started so late in the game, whereas other students were fundamentally unsure of their project. I addressed each matter individually and offered extra help before and after school, which was an offer that I had extended to all students. I myself created my own project to share with the students, which further excited some, or gave direction to those in need as the students became more self-directed.

The summative project was due the week following March Break as I did not want any of the students to feel pressure to do work over the break. I was thrilled with the projects that came in as they were all very individual and contained a great deal of creativity and imagination. I received projects ranging from abstract drawings to music videos, dance presentations to original piano compositions. I evaluated each project based on four rubrics, (*) which included assessment strategies both formal and informal:

• 1) Theory
• 2) Communication (the project/presentation itself)
• 3) Creation* (the process)
• 4) Thinking and Inquiry (written reflection)

Marking these projects was very difficult even after creating a rubric with the students however, I decided to mark based on the rubric for everyone first, and then assigned four separate grades based on the rubric and strands of assessment and then gave consistent equitable grades. I put a lot of thought and consideration into these grades and provided each student with detailed comments regarding their areas of strengths and personal connections to their project.

Reflection

The response that I received from this unit was overwhelmingly positive and not only did I receive praise from my associate (as this unit “encouraged students to explore other avenues of art”), but I received a phone call from one of the grade nine parents who was very thankful for my “allowing [of her] daughter to explore such creativity and share her gifts (for the first time) with an audience (the class)”. This particular student had written an original piano composition based on the song Winter by Tori Amos, and not only was it an outstanding piece, but she performed it with extreme grace, passion and confidence.

My experience with this unit was very rewarding, full of many unforgettable moments that I will always treasure. I believe that each person has a gift to share and this project allowed me to uncover many hidden talents within the various arts disciplines. It is projects like these that students will never forget as you take them ‘outside the box’ from the traditional ways of thinking. The students have learned a new appreciation for songs they relate to on a personal level and I learned the value of assessment and feedback as seeing the students read over my comments was a very rewarding experience. --Winchell (talk) 03:58, 26 April 2008 (UTC)

Bibliography

Markus, Janet. "Student Assessment and Evaluation in Studio Art." Dissertation, University of Toronto. 2002.

Rose, Leslie Stewart. “Self-Assessment Workshop.” workshop booklet. 2004

Ross, John A. “The Reliability, Validity, and Utility of Self-Assessment.” Practical Assessment, Research & Evaluation. November 2006, 1-13.