Thursday, September 13, 2012

Writing level summary statements and cut-off points

So, now you have an idea about the mechanics of setting up a set of rubrics (one rubric consists of: 1 Indicator + the quality criteria - hence we call our matrix a set of rubrics).  In this lesson, we will be going further into the process of developing the rubrics and finishing them off so we can analyse the results.
**If you have any trouble with the rubric terminology, just go back to the information I've posted on the website for week 5.

Remember that we need to find out what the students' achievement level is so that we can identify where they are at (i.e. what they can demonstrate they can DO, SAY, MAKE and/or WRITE in the assessment) and then predict where they need to go next (i.e. identify the ZPD, which is where we need to help them develop).  For example, if you are assessing a child's reading and they get 90% or above, they are usually ready to go up to the next level (i.e. they have developed and their ZPD has shifted to a higher level of complexity).  If the child does not yet achieve the requisite results to go up the level (i.e. they get 65%) their ZPD remains at the level they are currently working on and we need to be targeting the reading skills that are inherent at that level.

For today...

We already know about writing the rubric.  Today we will briefly go through the process again and we will add two components: 1) the level summary, and 2) cut-off points.

I've developed another set of rubrics for an example today.  This set of rubrics was designed to assess a vocal performance.  Because the aspect of singing that I am testing is based around skills development I have used Dreyfus' Model of Skills Acquisition as my framework.

Rigid adherence to rules; unable to distinguish between what is important what is not so important
Advanced Beginner
Requires guidelines based on aspects of what they are learning – they may miss details
Achieves what is considered to be standard and can perform routine procedures
Looks for most important things that need to be focused on and ignores or sidelines irrelevant information, deviations from the rules are perceived
No need to rely on rules because they are so familiar with them.  The skills are now second nature

Have a good look at this set of rubrics.  The capabilities (the broad concepts that you are assessing) are at the bottom.  They are split up into indicators, which are then broken down into various levels of sophistication for each skill called quality criteria.

3.3 Articulates the lyrics according to the  context of the composition
4.4 Extracts meaning from the composition according to context, notation and knowledge of other extrinsic influences on the composition.
5.3 Contextualizes emotions in the song and conveys the emotions through voice, and facial expression.
6.3 Evokes emotions in the listeners that oscillate  with the moods, interpretations and character contexts of the composition.
1.3 Consistent placement  of voice creating a warm  and rich quality across the breadth of the vocal range.
2.3 Pitch accuracy and flexibility is supported by diaphragmatic control.
4.3 Conveys the context of the piece using notation and  lyrics as a guide.
5.2 Uses the lyrics and notation to guide the light and shade of the performance
6.2 Arouses emotions in the listeners that are congruent with the intentions of the composition.
1.2 Consistent  vocal quality that spans the breadth of the vocal range
3.2 Articulates  consonants and rounded vowel sounds.
4.2 Emphasizes key lyrics according to context and notation
6.1 Conveys emotions according to lyrics and notation
Advanced Beginner
2.2 Accuracy of pitch varies depending on volume and height/depth of the notes.
4.1 Interprets the song based on an understanding of the lyrics.
5.1 Uses the context of the lyrics to emphasize meaning-laden words and phrases.
1.1 Breath control varies changing the quality of sound.
2.1 Accuracy of pitch varies throughout the song.
3.1 Clarity of articulation varies throughout the performance.
1. Vocal quality
3. Articulation of lyrics
4. Interpretation of the context
5. Expression
6. Pathos
Cut- points
Technical skills

Notice also that there are a number of blank boxes?  Even though Dreyfus has 5 levels of skill, I have made sure that I stick to the rule that there should be NO MORE than 4 levels of quality criteria for each indicator (remember that too many quality criteria make the results less reliable because the results tend to overlap or blur together).


Our cut-points will determine which level a child falls into and they are a very easy way to score and analyse the results.  Remember that each of the quality criteria is equal to 1 point (check week 5's lecture for the rules for rubric writing).  What we need to do now is figure out what levels our students are at.  To do this we look across the matrix.

At the bottom of the matrix, we have a novice level in which there are 3 quality criteria.  Each of these criteria will get 1 point.  This means that someone who falls within the Novice category will score between 1 and 3.

The next level up is the Advanced Beginner category.  Again, each of the quality criteria are worth 1 point, but we start counting where the Novice level left off.  So, students in the Novice level will score between 4 to 6 points.

At the Competent level, there are 4 quality criteria, so we start counting from 7 (because the previous level ended at 6).  A student who is in the competent range will score between 7 and 10.

And so on...

Level Statements:

But what do these cut points mean??  In order to give the scoring system some meaning, we summarise the skills that are being developed at each of the levels (this makes it REALLY EASY when it comes to report writing!!).  For example:

Level 5: At this level, the student is learning to interpret and communicate the underlying story and emotions within a composition.  S/he is learning to use the voice to evoke emotions and imagination in listeners congruent with the story that is being told through the composition.
Level 4: At this level, the student is learning to refine diaphragmatic control across the full vocal range to support pitch and tone, and to develop flexibility.  S/he is learning to use the lyrics and notation to inform and convey an understanding of the contexts within the composition.
Level 3:  At this level, the student is learning to maintain a consistent quality of sound across the vocal range.  S/he is learning to enunciate and emphasize lyrics in order to communicate the emotions of the context and the intentions of the composer. 
Advanced Beginner
Level 2:  At this level the student is learning to refine their use of muscles to control volume and pitch.   S/he is learning to understand the relationship between the lyrics and the feelings that the song represents.  S/he is learning to enunciate consonant and vowel sounds to convey meaning
Level 1:  At this level the student is learning to use the diaphragm to breathe.  He/she is learning to use the muscles of the vocal chords, face, mouth, throat, chest, back and abdomen to control the quality of vocal production and pitch.

Notice that as the levels get more sophisticated, the skills that are being focused on become more complex and sophisticated.  Level 1 (Novice) is all about the mechanics of voice production; Level 3 (Competent) is about refining the mechanics and introducing ways to communicate emotions, stories and contexts; Level 5 (Expert) has mastered the mechanics (i.e. they are second nature) and is now focusing on impacting the emotions of the listeners.

Writing the level summaries helps us to get a picture of what the students need to work on and provides areas of focus for our teaching.  Not all students perform consistently (i.e. they might show more proficiency in a couple of indicators, but are lower in others).  However, the level statements are intended to give generic information about where they are at.  We then need to look at the individual students' rubrics for the areas that the students are inconsistent in to determine how or what we need to teach to those students specifically.  We will go through this in the next post (and we will also go over it in your next face-to-face class).

**For your second assignment, you only need to develop the rubrics up to the level statements.

Discuss what you are doing for your second assignment:

  1. What framework(s) are you intending to use?
  2. What are the domains you are wanting to work within?
  3. What types of capabilities will you focus on?
  4. What do you want the students to be able to do, say, make and/or write?


  1. 1. Without having an in-depth look at the second assignment yet, I believe I will be working with Bloom's.
    2. Our project is quite broad so until I take a closer look at the assignment we could be looking at a crazy amount of domains: Communication, History, Civics and Citizenship, Mathematics, Social interaction, English, ICT.
    3. Until I narrow this down its quite hard to say what capabilities I'll be focusing on. But Communication will most likely look at presentation skills, Civics and Citizenship might require students to explain the basics of how a town council is formed etc.
    4. Still working out which assessment to actually address for Assignment 2, but it will essentially look like this: Do: Participate in the running of a theoretical town, Say: Hold town meetings and address issues, Make: a scale model of the town, Write: letters to the council, mayor.

  2. 1. I am pretty sure I will use Bloom's as a framework for my assignment as I have a bit of an understanding of his theory and I like the way it works. I have also PD where Blooms was used so that will be helpful.
    2.The domains I will work within are history, literacy, civic and citezenship and the arts I think although I am still working through that but am hopeful I can integrate it all effectively.
    3. I will focus on critical thinking and problem solving, cultural understanding, ethical treatment, aspects that relate closely with dealing with social issues. I really want students to be able to appreciate the Indigenous culture and look at what can be learnt from it.
    4. Students will produce artworks, written work, make judgments on aspects of indigenous culture, Discuss and debate aspects of culture. Compare and contrast families, differences, cultures.

  3. 1. I'm also thinking of using blooms. To be honest it's the only one we've really covered in detail in class and it also seems to make a lot of sense so i'll probably be going with that.
    2. Because of the nature of our unit it will involve a number of domains such as Civics and Citizenship, Maths, English, and a number of others but these would be the main ones.
    3. Communication will be a huge focus within this unit because of the amount of group work involved and because all the students will be relying on each other in order to complete their own individual work. Problem solving will be another focus because as the teacher i will be intentionally messing with the students progress in order to see how they problem solve to get back on track.
    4. Ultimately at the end of the unit i want to see that my students can physically work together and produce plans and strategies for dealing with problems. I also want to see that they can write reports on what needs to be done and also what they have done once it's all finished.

  4. What framework(s) are you intending to use? I will be using Bloom's Taxonomy for my matrix
    What are the domains you are wanting to work within? Listening and reiterating discussed information, Logical Thought, English skills
    What types of capabilities will you focus on? Planning, Introduction, Ability to construct points for and against, Persuasive Conclusion, Literacy
    What do you want the students to be able to do, say, make and/or write? Write a persuasive piece on a chosen topic about Farm animals

  5. 1. Working with Bloom's taxonomy for the matrix
    2. For the domains, I'll be looking at Literacy, Numeracy, Creative thinking, personal & social capabilities and intercultural understanding
    3. The capabilities to focus on will be presentation, research and illustration
    4. I want to see the students working together as a group, being able to research about the food of another country, to create a market stall with that food researched and work in an environment where they are buying and selling with one another

  6. 1. I will defiantly be working with Bloom's taxonomy for the matrix
    2. At the moment the domains I will be looking at are: Literacy , Science, Critical and Creative thinking and ICT
    3. The capabilities I will be focusing on are Basic Planet information, Presentation and Resourcing information
    4. I want the students to work in their own, researching about their planet using ICT and Books and construct a report on their planet

  7. I am intending to use Blooms.
    Year 6 History and interpersonal relationships.
    To be able to report of facts; identify challenges; express emotions; and empathise with others.
    To use a number of media to develop a personal journal about an immigration experience.

    By the way, I like the way the matrix stretches our own understanding of the subject and ensures that we ourselves are 'expert' enough the identify all that is involved.

  8. What framework(s) are you intending to use?
    I will be using Bloom's taxonomy
    What are the domains you are wanting to work within?
    History- content knowledge, Literacy: narrative writing, ICT: recording of a podcast.
    What types of capabilities will you focus on?
    I'm focusing on
    1. Rough draft and recording of podcast
    2.Introducing the convict
    3.Explanation of journey on the First Fleet
    4.Explanation of arrival and life in Australia
    5. Narrative writing
    What do you want the students to be able to do, say, make and/or write?
    Some of my indicators include rough draft is completed and checked before recording, convict is named, convicts crime is described, the trial and sentence are mentioned, dates of arrival, name of ship, observations of Australia on arrival, explanation of life in Australia and structure of podcast is in narrative form.

  9. Using Bloom's
    Looking at domains Interpersonal development, English, ICT & communication and Civics & citizenship.
    Capabilities include working in teams, cultural awareness, social relationships, writing, speaking & listening, communication, creating and multiculturalism
    they need to be able to be seen to be working as a team, producing a written poster, a verbal group presentation that outlines a made up or existing australian event with researched cultural influences (both indigenous and overseas).

  10. I am not certain yet but I think I will choose to use Blooms Taxonomy.
    My main domain focus is Literacy which includes research and presentation skills and group work.

  11. 1. Framework: Bloom’s Taxonomy – Level 1 – Remember and Understand, Level 2 – Apply and Analyse, Level 3 – Evaluate and Create
    2. Domain: History, Literacy (Narrative) and ICT (creating a podcast and online research)
    3. (a)Capabilities: Teamwork, Life of the Convict and Effective presentation of the project/podcast
    3. (b)Students to focus on: working as a team, researching the life and experiences of a chosen convict and putting themselves in his/her shoes; learning ways to use technology in learning and enhance their knowledge and skills in ICT
    4. Do: Research a convict; Say: Present information in Narrative form; Make: A podcast; Write: A script for the podcast

  12. What framework(s) are you intending to use? Bloom's Revised Taxonomy
    What are the domains you are wanting to work within? History, Literacy, ICT.
    What types of capabilities will you focus on? Working Together, Historical Questions and Research, Analysis and Use of Sources, Perspectives and Interpretations, Explanation and Communication, Chronology and Terms of Concepts, Understanding the roles that people play in local and national history.
    What do you want the students to be able to do, say, make and/or write? Make a timeline that follows the settlement of the local community using key information: including different perspectives from local and national persons/events of historical influence, considering the influence and importance of Aboriginal and Torres Strait Islander peoples, a range of historical texts and sources, historical terms and comparison of their community with the broader development of Australia.
    DO: Research settlement of local community. SAY: present this information in a timeline.

  13. This comment has been removed by the author.

  14. Hi all,
    My topic is on Food Debate, which I think is a very challenging subject.
    In any case, I have already chosen it, and must give it a go....

    Framework: Bloom’s Taxonomy.

    Domain: Communication; Design, Creativity and Technology; Speaking and Listening: Literacy; School-based Curriculum consideration: Respect of other culture and individuality.

    Capabilities: Communication Skill, Creativity Skill, Debating Skill, Presentation Skill, Team work.

    Students’ ability: Research, Present, Debate, Support Debate with examples, Using prop and technology, Use appropriate body language for communication.

  15. For my assessment I have used a combination of theories. I have also focused on testing student’s understanding and application of the subject.
    I have assessed students according to their knowledge, comprehension, application, analysis, synthesis and evaluation.
    The assessment task is focused on different learning styles, and ability groups. Creativity, communication skills, inter personal skills are other criteria’s tested
    The students must be able to:
    Do: Research, find information about mountains, work in groups
    Say: Present their findings to the class
    Make: A project based on Mountains of the world.
    Write: Facts about mountains

  16. Having rubrics for each student is so helpful in planning effective lessons as well as assessment. Having a variety of levels allows students the space of increasing their performance levels. It's encouraging to know that blank boxes are ok in that they don't have to be filled for the sake of being filled especially when it's not relevant.

    I used Dreyfus' Model of Skills Acquisition for my framework. The Domains I focused on were Literacy, Speaking and Listening and History. I wanted the students to engage in tasks of comparing their own culture with Aboriginal culture, participate in group work of presenting an aspect of Indigenous Culture and engaging in self reflection.

  17. What framework(s) are you intending to use?
    Now understanding a little more as to how to set out a rubric i believe i would use Bloom the next time i am to design one.

    What are the domains you are wanting to work within?
    The history with also a section focus on numeracy

    What types of capabilities will you focus on?
    The next time i really need to set better skill focus and what is needed for them to achieve the level or beyond their level.

  18. Now that I understand also a little more clearly I would use Blooms or Dreyfus Model in designing of a rubric.
    The domain I am working in for this assignment is Literacy

    The Capabilities: that I was focusing on for the formative assessment were Students are required to write an email to a favourite Olympic athlete. This moderated writing task will identify whether each student is able to conform to writing conventions, appropriate letter formatting and basic word processing skills.

    The indicators : Within the email the students need to introduce themselves and express why they are writing to the athlete of their choice. Students should write about themselves, their hobbies and interests, and ask the athlete for some advice how one could achieve success in following ones passions. Students need to include questions about the life and background of their particular athlete

  19. What framework(s) are you intending to use?
    Blooms’ Level 1 (Remember & Understand), 2 (Apply & Analyse), 3 (Evaluate & Create).

    What are the domains you are intending to work within?
    The humanities (history), interpersonal development (working in teams), and ICT (ICT for creating).

    What types of capabilities will you focus on?
    The Humanities:
    Introducing the aviator, the aviator’s accomplishments, the impact of the aviator’s accomplishments on progress of aviation, working in teams

    Interpersonal Development:
    Working in teams.

    Use of book creator application on ipads, recording of digital information, and producing an iBook

    What do you want the students to be able to do, say, make and/or write?
    Do: research and report on chosen aviator according to framework.
    Say: present presentations.
    Make: work in teams to create ICT projects.
    Write: record their findings, observations and analysis in journals and in written work.

  20. Topic - Olympics

    1. My intended framework will be Bloom’s Revised Taxomomy

    2. The domains I will work within are Interpersonal Development, English/ Thinking Processes, Communication, Information and Computer Technology and Design Creativity and Technology.

    3. The capabilities I will focus on are working in collaboration, research techniques, introduction of the students chosen athlete, explanation of the athletes career, construction of an informative text from research, oral presentation of research, word processing, digital image inclusion and design of a game.

    4. I want students to write an informative document about their chosen athlete, orally present (say) their research to the class, design and make a game based on their chosen athletes sport (make) and work collaboratively with a partner to complete the assignment (do).

  21. What framework(s) are you intending to use?
    I would be using Bloom's taxonomy for my matrix.
    What are the domains you are wanting to work within?
    I would like to work on Literacy, numeracy and Integration studies.
    What types of capabilities will you focus on?
    I would focus on Communication skills, Creative thinking, and Analytical skills
    What do you want the students to be able to do, say, make and/or write?
    I would like the students to know the importance of clear communication skills and communicate effectively. Encourage students to be creative and tasks would be given to promote analytical skills.

    It is interesting how charts and tables are used to indicate students ZPDs.
    I hardly use Excel and I must confess that I have not worked with charts and tables for quite a while. However, I believe I should be able to eventually comprehend these.

  22. I was thinking along the path of Bloom's taxonomy.
    2. The domains I aimed at were Science, Aboriginal studies, Literacy and Critical and Creative thinking and history.
    3. The capabilities I will be focusing on are Astronomy history focusing on Aboriginal history and astronomy.
    4. I want the students to work in their own, researching about Aboriginal astronomy looking at the dream times and how important is was for them and for our western astronomy and how the western world use the same or similar information.

  23. 1. Framework:
    Bloom's Revised Taxonomy

    2. Domains:
    The Humanities - History, ICT, English.

    3. Capabilities:
    Collaboration, Understanding of historical images, Research, Ability to write a short story about a historical context (using a computer), Class presentation

    4. Do, say, make and/or write?

    Organise a timeline in chronological order (Do)
    Assign roles in the group and decide on how the project will be presented (Say)
    Add own pictures to timeline, colour in the pictures and write a short story (Make/Write)
    Present the finished project in class (Say)

  24. 1. Framework:
    Bloom's Revised Taxonomy

    2. Domains:
    Content: Flight
    Disciplines: Literacy, Numeracy, Arts
    Physical, Personal and Social learning: Interpersonal development, Personal Learning
    Interdisciplinary: Communication, Design, Creativity and Technology, Thinking process, ICT

    3. Capabilities:
    symmetry between natural and mechanical flight, using varied texts, planning, data collection, appropriateness of art forms, teamwork, personal goals, logical ideas and feedback, plan production designs, explore the natural world, classify and access resources by using ICT

    4. Do, say, make and/or write?

    Reflective Journal and journal entries
    classwork, design of flight machines, e.g. plane, helicopter, etc, data recording in tables, pictures, drawings, etc

  25. What framework(s) are you intending to use?
    Deciding on Using Blooms Taxonomy

    What are the domains you are wanting to work within?
    Covered quite a few domains. Interpersonal development, mathematics, communication, design creativity and technology, thinking process.

    What types of capabilities will you focus on?
    Teamwork, data collection, presentation, model design and production, use of resources.

    What do you want the students to be able to do, say, make and/or write?
    Make a model of a flying machine and do a group presentation to the class about their learning and design process.

  26. Upload for Week 6 ‘Writing Level Summary Statements’
    I used ‘Revised Bloom’s Taxonomy’ as the framework as my focus was more on assessing student’s cognitive abilities rather than solely the skills they had attained. The Domains tested are ‘interpersonal communication’, ‘English’, ‘Mathematics’, ‘Healthy & Phys. Ed’, and ‘Presentation Skills’.
    The capabilities I am seeking to examine are student’s ability to work as part of a team as well as researching, analysing and evaluating information. The topical context in which I examine these cognitive abilities is the area of ‘Healthy Living’; I expect to find much evidence of student learning in this unit regarding Healthy Diet and the consequences of healthy and unhealthy eating, as well as active or sedentary lifestyles through the preparation and presentation of their project in addition to my primary assessment goal.

  27. 1. The framework I used for Assignment 2 was Bloom’s Revised Taxonomy. I have to say though I am not entirely convinced about Bloom’s because it assumes that creativity is the highest level to gain for, whereas I wonder if it is different for each student.
    2. The domains I looked at in the assessments included Literacy- communication and Presentation skills, ICT-writing tables, graphs and Inter Personal skills including teamwork.
    3.The capabilities I looked at in the ICTassignment included accurate data collection, using colour in medal tallies, correctly setting out and displaying information in graphs. In the presentation about the Olympic Athletes I included capabilities such as Introduction of the athlete, the identification of the attributes of the athletes, and the ability to accurately record and report the information.
    4. In this assignment there were many different activities which came under the ‘Do, Say, Make, Write’ including giving a presentation about the Olympics (using a variety of mediums such as a poster, cartoon, booklet or journal), writing up brainstorming ideas, and making different graphs on the computer connected to the medal tallies.

  28. This comment has been removed by the author.

  29. What framework(s) are you intending to use? Bloom’s Taxonomy
    Level 1 - Remember & Understand: Emergent,
    Level 2 - Apply & Analyse: Consolidating,
    Level 3 - Evaluate & Create: Advanced

    What are the domains you are intending to work within? What types of capabilities will you focus on?
    Interpersonal Development- Working in teams & Presentation
    Humanities-Geospatial, Geographic knowledge and understanding,
    Mathematics- Statistics - They interpret and compare data displays,
    Health and Physical Education- Movement, motor skills and physical activity

    What do you want the students to be able to do, say, make and/or write?
    -Student to be able to work successfully in a team, negotiating difficulties and attain timely completion of tasks.
    -Student to be able to hold audience engagement using different strategies of props and aids. -Student to be able to reflect on their own and others' learning and evaluate and comment on their tasks outcomes, presentation, and collaborative efforts.
    -Student to able to identify locations, and compare and contrast different characteristics of places and explore the interactions of people with the environment.
    -Student to be able to analyse and interpret bar graphs.
    -Student to be able to reflect on their participation in physical activities and consider all the related safety principles associated with the activities.

  30. 1. What framework(s) are you intending to use?
    2. What are the domains you are wanting to work within?
    I struggled placing the domains in my rubric, with none present.
    3. What types of capabilities will you focus on?
    4. What do you want the students to be able to do, say, make and/or write?
    They are required to complete a range of mathematical statistic, historical planet facts, scientific properties of the planet and the presentation format (PowerPoint or poster)

  31. What framework(s) are you intending to use?
    I will use the Bloom's Taxonomy for my second assignment.

    What are the domains you are wanting to work within?
    I want to work within the cognitive, affective and psychomotor skills of Blooms.

    What types of capabilities will you focus on?
    Students need to be able to work in a team, listen to their team members, contribute to discussion, research work and compilation of knowledge to present a creative oral presentation.

    What do you want the students to be able to do, say, make and/or write?
    Be able to work in a group of 3 to present a creative oral presentation on the chosen farm animal based on the rubrics criteria.

  32. What framework(s) are you intending to use?
    Bloom's Taxonomy to detremine student's diverse learnin bilities.
    What are the domains you are wanting to work within?

    The domins :English, science,ICT, Interpersonal development
    What types of capabilities will you focus on?
    students to be able to work cooperatively within their groups, create outstanding quality of work through presentation.
    What do you want the students to be able to do, say, make and/or write?

    An oral presentation and everyone contributd equally.

  33. In all honesty I have no clue, the plan is Bloom but I have issues with his and I need to work on my domains as I really have no clue what to do with them. Can I get back to you at a later date?

    Bloom's Taxonomy because of its simplicity compared to the rest. Gaining confidence with this one first will help me grapple and gain competency with the others.

    Civics and Citizenship, Literacy, Mathematics, and ICT

    "Communication" - required within the group work which characterizes much of unit activities.
    "Problem solving" - addressing constant issues arising, including the final introduced one 'Big Brother' style (earthquake)!

    - Communication and conflict resolution
    - Creation of town blueprints, models and mini presentations
    - Present and argue ideas and final assessment piece
    - Presentation speeches, letters to mayor, feedback to peers