Archive | Juliol 2008

Collabowriting & Self- and Peer-Assessment

To work in teams and being evaluated by your peers are two practices with an enormous potential and relevance for individual motivation as well as for being able to contribute value to an organisation. Both practices are being used in e-learning and the following paragraphs will introduce some approaches and cases. Stanford considered collaborative writing (here refered to as collabowriting) so important they ran the Collaborate project in order “to encourage institutional change at the level of the department, college, university, and profession–change that will foster the development of collaborative writing and research.

Why consider collaborative writing assignments?

Kate Kiefer from Colorado State University’s suggests several reasons:

  • Collaborative groups draw upon the strengths of all their members. Although one student may be stronger in critical thinking skills, another may excel in organizing. By working in groups, students learn from each other while they complete assigned tasks.
  • More and more workplace activities involve project teams. Giving students opportunities to work collaboratively on academic projects can help prepare them for the advantages and pitfalls of collaborative work on the job.
  • Students working in collaborative groups can take advantage of group members for built-in peer review as they complete writing projects.
  • Not least important, collaborative writing assignments usually entail much less grading time for the instructor.

How to Collabowrite?

People have been writing texts together for a very long time, but the computers, the internet and wiki technology in particular allows for a more efficient practice Most collabowriting today is probably done by sending documents back and forward by email (and if you endulge in this practice please take two minutes to learn how to Track Changes), wikis are really much more efficient because of version-control, online availability and much more technical reasons. But wikis are also the weapon of choice because they allow for a more transparent practice when it comes to showing who contributed how much and what to the writing process. However collaborating is a very complex effort that has to be learned and trained. Hence we recommend some work on the subject:

Bruce Speck has done very interesting research in the field of collaborative writing. His book
Facilitating Students’ Collaborative Writing (2002) is an academic publication that introduces the topic and discusses the whole range of collaborative writing opportunities that can be offered, ways to form groups, and approaches to integrate technology. A final topic is grading, which is particularly difficult for collaborative writing projects.

Also
Connery & Vohs @ UC Davis have developed a comprehensive guide to online collabowriting, and they find: “when students are asked to list problems they have encountered in previous project groups, the most commonly cited difficulties can be traced to a lack of leadership and coordination: defining/assigning tasks; enforcing participation, scheduling meetings, setting and agreeing upon group goals”. They suggest to assign the roles (Initiator, Opinion&Information seeker&giver, Clarifier, Elaborator, Summarizer) to the different team members.

Virginia Montecino and Ashley Williams from the New College have developed a guide particularly targeted at supporting students in their collaborwriting projects. Especially the section on collaborative strategies as well as the section on collaborative roles are useful.

Last but not least an example from the UPC. Professor Marc Alier (who is also responsible for the development of the new Moodle Wiki) is using wiki based collabowriting for several iterations of the History of Informatics course. He allowed students to organize their team work themselves, which allows for a much more intense learning curve. The fact that his class is semi-precential and that the students have it much more easy to meet to coordinate efforts makes this approach less attractive for pure e-learning.

What Tools can be used for Collabowriting and How to get started @ UOC

The most common tool for collabowriting is a wiki. By far the most widely used wiki is MediaWiki but there are good alternatives such as SocialText or PBwiki. Kolabora has compiled a pretty useful list of collaborative writing tools, which covers many non-wiki services available outside of an educational institutions virtual campus. Most of today’s universities offer a wiki as part of their online learning environments (to request and explore the use of a wiki @ UOC please write to Pepe Mora @ jmora@uoc.edu)

As raised in the Speck’s book, evaluation of collaborative work is particularly challenging. One of the best methods to “crowdsource” the problem is to have students engage in self- & peer-evaluation, which is covered in the following section:

Self- & Peer Assessment

Even though self- and peer-assessment are around for many many years, no standard has developed for e-learning environments to integrate the practice. Nevertheless, there has been quite some interest in the subject resulting in extensive publishing () and some Learning Management Systems (LMS) such as Blackboard have a self- and peer-evaluation feature (see video) integrated. However most solutions are rather rough homegrown (iPeer or peerScholar) or more slick and advanced comercial add-ons such as Turn-It-In. But let’s start off with two hands-on examples:

At the University of Florida the online MBA uses a very simple word-doc based peer-review system in a course designed to train students in “acquiring and practicing traditional business writing skills”. This is how it works: “Students actually download the report from the discussion board, mark it up with their comments, and upload it back to the board…..only faculty can see the actual names of the reviewers…the blind answer format thus enforces original critiquing…The professor grades the original writing and grades the efforts of the peer reviewers.” The report that “students indicate their enthusiasm for this kind of activity, and the ensuing class discussions related to these peer evaluation activities reflect the students’ increased engagement with each other and with the course material” (Fisher, 2003)

Here is another story of a Professor from the University of Toronto who had the custom system peerScholar developed in which the students review and evaluate their peers answers in exams: “There is no way I could mark the written work of even 100 students, let alone 1500…The original reason for seriously considering a peer-to-peer evaluation process was financial. We cannot afford to pay a large team of TAs to mark written answers for large classes…[But there are substantial benefits as well:] peerScholar gives you feedback in a matter of days, while the papers are still potentially on your mind….With the peer-to-peer evaluation, you think about good writing both when you write your answers AND when you are evaluating the answers of others. Moreover, because the answers are not your own, you will be better able to look at them critically and think about what they are doing well, and what they are not doing so well. In doing so you may actually be learning more about good versus poor writing that you do when you write your own answers. Thus, when you are evaluating others, you are also learning yourself…..[The system has some build in functionality to assess whether the peer is working fairly:] By examining how long they actually spend on marking, the variability of the marks they give,and the correlation of the marks they give with the marks that others give, we can detect “dud-like” behaviour and, when it is detected, the person will be warned and the marks they gave not included” (Joordens, 2006)

Exploring the Practice

A) Self-Evaluation

Overall Learning Experience: When students apply for the class have them define their personal learning goals (depending on the subject you might want to consider having them refine their goals after the introductory part of the course). Hence when the class is over they should review their original objectives and comment on how the class allowed them to learn what they wanted (and how they performed).

Learning-Journal: Have students continually write and reflect about their learning experiences. A private individual blog is commonly used as tool to have students record interesting insights and reflect upon their learning in general.

These practices also produces very relevant information for the improvement of the course content and teaching method.

B) Evaluation of Team Dynamics

Have teams assess their own dynamics and performance. See e.g. fig. 1 http://www.educ.dab.uts.edu.au/darrall/sparksite/Screen2.html for an example for “teacher-like” grading. Another solution would be to have the professor grade the overall performance of the team in form of reached points, while the team members themselves distribute the points (Winchester-Seeto, 2002). E.g. the team is assigned a percentage of the maximum score (= number of team members x 100) which the team then decides to divide amongst themselves. A robotics course at the University of Calgery uses the first version of the Team Dynamics Evaluation with a particular twist. All students have to hand in a 1/3 page evaluation of each of their team mates and while doing the peer-assessment only contributes 1% to the overall course grade, failing to hand in the team mate evaluation results in failing the course (Kermer 2008)

C) Peer Evaluation

A discussion in an online forum can be seen as a constant qualitative peer evaluation, and many online environments have the functionality to leave comments about content items. These tools can be used as qualitative non-explicit evaluations from peers. A more quantitative and explicit peer evaluation technique that is spread widely is the star rating system, allowing peers to grade the quality of a contribution e.g. from 1 to 5 stars. In order to deploy these tools pedagogically their use has to be explained and they have to be made mandatory. One interesting use of peer-evaluation can be observed on www.plastic.com (or slashdot.org) where (expert)users on the one hand peer-review & evaluate the submitted articles, but on the other hand all readers evaluate the quality of contributions to the discussion and thereby allow the most valid arguments to prevail.

Peer evaluation

Anuncis

Micro-Blogging

Named after blogging, micro blogging – or micro media – has evolved into the Next Big Thing within the internet scene. Micro blogging means posting extremely short snippets of information – just like in blogs, the most recent post is displayed at the top of the page, the older ones are displayed below.

While there are a number of micro blogging services, the two most notable are Pownce and – much bigger! – Twitter. (For our purposes, most examples and use cases will Twitter-related if not noted otherwise.)


So first up: What is Twitter?

Twitter is a micro blogging service where users post information snippets with a maximum length of 140 characters. (Yes, characters, not words.) This equals the length of a SMS message (160 characters) minus 20 characters to enter text commands. Twitter can be accessed from the web, through a client software or from practically any mobile device like mobile phones, PDAs or iPhones. Imagine micro blogging as the missing link between instant messaging (for example ICQ, MSN, AIM, Yahoo Messenger etc.) and blogging (Fig. 2). Users are prompted to simply answer the question “What are you doing?”

Figure 2: (Tom Barrett)


Quick facts about Twitter
Website:http://twitter.com/
What is it: A micro blogging service
What does it do: Allow to send brief messages of 140 characters max (from any device to any other device, or to the WWW)
Users: Roughly 1 million in April 2008 (estimated, figures not officially disclosed)
Who uses it: A mixed crowd, with lively communities of internet and IT consultants, designers, and educators
Company:Obvious, San Francisco, USA
Founder: Evan Williams, founder of Blogger.com


Why would that be useful? Why would anyone want to read what I’m doing right now?

Excellent question, and also the main point most users (including the authors of this newsletter) cannot understand when they first look at Twitter. Twitter allows – once the user’s network has reached a certain critical mass – to gradually build, foster and extend relationships; to evaluate, judge, and build personal reputation; to stay connected with a large social network through ambient intimacy, i.e. getting an intimate impression of other users daily lives through “passive” communication. Says social media consultant Laura Fitton: “For a contrived, weird and techy way to communicate, Twitter’s “passive conversation” fosters very natural, gradual relationship-building.”

 

This brief video by Common Craft Show gives a great overview.

 

Strengths Problems
  • Great way to extend your social network
  • Profit from exchange with others in the field
  • Peer-filter provides relevant information
  • Very engaging
  • Communication may seem superficial and irrelevant in the beginning, and it can indeed be.
  • Due to its “live” nature, Twitter has quite a potential to distract.
  • Information overflow.


What are the challenges when introducing micro-blogging to the classroom?

There is a number of issues you should be aware of when introducing Twitter to the classroom:

  • Twitter is distracting
    One of the main problems with Twitter is that the constant stream of information and conversations can be quite distracting. It takes time to get comfortable with Twitter and to use it efficiently. Also, some teachers won’t be comfortable with the idea of a backchannel where their students can exchange notes and chatter during class.
  • Using Twitter via mobile phone costs money
    If your students use Twitter via their mobile phones, they might be charged for their text messages. Consider using a computer-based Twitter client instead.
  • The Internet doesn’t forget
    Just like blog conversations, Twitter messages get archived and syndicated. This also means that everything posted on Twitter could – potentially – stay available on the web forever. Keep that in mind when posting.
  • Twitter isn’t for everybody
    Some love it, some hate it: Twitter is one of those tools that not everybody is comfortable with. Some won’t like the way information flows by, some will have concerns about privacy implications (please keep in mind that you can protect your posts so only your friends can see them). Both are valid points. Forcing students to use Twitter wouldn’t lead to good results. Experiment together with your students to find out what works for you and them.

How can micro-blogging be useful for education?

  • Exchange ideas with other educators
    With a quickly growing community of educators on Twitter, you can easily exchange ideas with other educators. No matter if you are looking for ideas, a solution, literature recommendations or teaching materials – one of your Twitter colleagues might just have the answer or know where to look for the right solution. It’s as easy as watercooler talk.
  • Follow professionals
    “Students can follow someone else who is on Twitter, who interests them. For example if they are thinking about journalism they should follow NewMediaJim who works for NBC and Tweets about being on Airforce One, covering the Middle East etc. This is a rare inside, “real-time” view into journalism,” says University of Texas’ Professor David Perry. He presents more useful usecases for Twitter on academHack.
  • Get instant feedback
    Through Twitter, you can stay in touch with your colleagues and students – and thus get instant feedback. Looking for an informed opinion about your curriculum? Ask your colleagues in your own or another school. But you can also use Twitter as a backchannel to an ongoing presentation. Even while presenting yourself, your students might share interesting ideas or feedback through Twitter. Admittedly, watching a Twitter stream while teaching a class that might need quite some practice, and it certainly isn’t the right thing for everybody.
  • Foster exchange of ideas with researchers around the world
    Through Twitter, it’s easy to follow loosely what other research teams are focusing on. Since communication via Twitter is so light-weight and non-intrusive, it’s much easier to follow discussions and status updates from other researchers than one-on-one communication. Thus, opportunities for collaboration as well as synergy effects are more easily discovered and exploited.
  • Coordinate spontaneous events
    You are thinking about calling a spontaneous meeting, or just noticed there’s something interesting on TV? Just twitter it and your students, colleagues or co-researchers know what you’re thinking about. This is just the first step into the conversation.
  • Stay in touch with your students
    Twitter allows for a very easy, simple way to keep students updated on homework, assignments, events and notifications without knowing their mobile phone numbers. Students can control the communication, so it’s very non-intrusive. Students can also easily get feedback from their educators or each other, and they can set up spontaneous meetings or study groups on the go.


A perspective: Teaching With Twitter

Introducing Twitter to the classroom was “…the single thing that changed the classroom dynamics more than anything I’ve ever done teaching,” says David Parry, a professor at the University of Texas, Dallas. “The more we can reach out and use the communication tools students are used to using, the more we can show the students that education actually does matter and that what goes on inside the walls of this institution matters for the rest of what’s going on in their lives.” Watch a brief introductory video with David Perry on Chronicle.com (2:30min).

Recommended resources: