I've just attended an excellent Open University conference on "Emergent Technologies." The conference was a nice mix of the practical in the present, evaluating the technologies we currently use to support distance learners, and the visionary, imagining the shape of higher education in a few years in which anything from MOOCs to gaming-as-learning might be in play.

One interesting session was that run by Derek Jones, who lectures in architecture design. As part of a talk on the role of social media in education, he described a new system called Open Design Studio. Essentially this is a Flickr-like repository in which art and design students can deposit files and photos of their creations or sketches and models leading up to the finished product, with other students able (and indeed expected as part of the module and website design) to comment on other students' work. The thinking behind ODS is that design students are assessed not so much on the artefact that they produce at the end of a project, as the processes of creative inspiration, research and then critical reflection that go into its making. Collaboration is encouraged as being intrinsic to the process of reflection.

As is often the case when educational technologies are showcased, I immediately wondered how this might apply to my own discipline which is text oriented. In this case, though, the fact that I can see only barriers rather than opportunities is perhaps the more interesting cause for reflection.

It seems from where I stand as a front-line teacher that there is a fairly fundamental division between so-called creative subjects and so-called critical ones, even though these are often two sides of the same coin. Creative subjects (design, arts, creative writing) actively encourage peer review, collaboration, development through discussion. These are regularly built into module learning outcomes and assessment methods. Yet in critical subjects - such as literature - we remain inherently resistant to the notion that collaboration can play a part in assessment. We believe that when we assess our students, we need to judge them only on the basis of their final output, typically an essay or exam, not on the basis of the process that has led up to that output.

Some of the OU's rules on the modules on which I teach are symptomatic of this. For example, students are forbidden from posting any materials relating to assignments on forums, or from uploading their own work, such as assessed essays. The ostensible reason is that to do so might infringe copyright, and the rights of tutors whose marks and comments may be attached to assessed work; there is also a concern about plagiarism; finally, in critical subjects we need to avoid giving the impression that there is a single best answer or approach to a question, such that writing essays becomes merely a question of ticking the boxes and doing certain things like including x number of quotations (as it is increasingly at A-Level).

All of these are very valid concerns. However, I wonder whether the fears outweigh the benefits of a more liberal approach, and are preventing us from exploring alternative modes of assessment which might better inculcate good academic practice. For example, students often ask me how to write an effective essay introduction. In answer to this, I could share examples of work that students have agreed to let me use anonymously. However, coming from me this would give the impression that there is a single model that I personally have in mind, such that if students do this for me, then they will get better marks; the risk is different tutors who they encounter at a later stage in their education may have subtly different perceptions, reflected in the samples they hold up as good practice. It would thus be better for students to be able to post their own introductions to past essays on forums, to discuss them, peer review them, pick them apart independently. This would better teach the conceptual basis for an introduction, rather than my particular idealised manifestation of one. It would enable students to reflect on what they are doing right or wrong with their own individual style, rather than expecting them to conform to some idealised a priori template.

Or to give a different example, on my OU courses we run active and highly beneficial forums for students. However, when we prepare discussion questions these must always circle carefully around a particular forthcoming assessment. If an assessment relates to a short story, for example, we have to focus on a different text, though often we take students through comparable motions of analysis. There is of course a valid pedagogy behind this, in that we want students to be able to demonstrate that they can think critically and independently. Yet especially at level 1, these skills are not always intuitively there; they need to be drawn out. On many occasions, I have encountered a group of new university students who thrive in the discursive and dynamic environment of a forum, who between them tease out and articulate issues and concepts relating to a text under discussion. However, place those same students individually before the static environment of their word processor, and what emerges in the essay does not represent the critical work that in principle they are capable of. At level 1, as we try to encourage and build the confidence of students, why not make that forum activity part of the assessment? As well as judging them on the basis of their finished essay, what about also judging them on the basis of their contributions to discussions in preparation for that work?

In critical subjects - such as literature - we are resistant to the notion that collaboration can play a part in assessment, or in working towards an assessment, or in fact that it can be the assessment. We presume that when we assess our students, we need to judge them on the basis of their final output, not on the basis of the process that has led up to that output. I'm picking on the OU because as a distance-learning university where technology is prominent, the symptom is most obvious. However, in some shape or form the issue has been there in all four institutions I have taught at. The issue is a disciplinary one, not an institutional one.

Here's one final example. In the distance learning course on Modernism that I wrote for SIM University, I included a graded discussion board topic in which students were asked to debate the modernist characteristics of a piece of unattributed prose. There was also an independently written essay on a different text, and the obligatory exam at the end. I encouraged students to do the discursive bit. I demanded that they create polished artefacts in the form of end of year essays. But my course design failed to join the two aspects together, so that discussion and collaboration was not part of the mainstream process of creation of the final essay, even if it may have been a tributary that influenced their skills and thoughts further downstream.

Where, then, might we go from here? Well it seems to me - again an uneducated hunch - that we are not going to go anywhere: we will be pushed first. On Facebook and other social media collaborative discussions about assessments, sharing of essays, comparisons of each others' work and tutor feedback are already taking place. Most institutions have a policy against this; most are losing the battle.

We need then to beat the enemy by bringing it into our camp. Especially at level 1 we need to teach students that our online spaces are safe, reflective and above all genuinely useful environments in a way noisy social media is not. Trolls lurk in the public sphere; our job as tutors (just as in the physical classroom) is to create a safe zone to keep the trolls out, to steer students to learn from peer criticism in a positive and respectful way. We can guide discussions about essays and other assessment topics in a way that allows them to collaborate but without overstepping into plagiarism. We can suggest to students that the link to Wikipedia or Gradesaver that they have just posted might not be such a great resource after all. We can enhance participation and a feeling of group identity by bringing all our students under one roof, where they will happily reside if they know that their involvement on a forum or other collaborative medium is being assessed.

At present, and increasingly, students are outsourcing the process of preparation for essays to other platforms, although they continue to present us with the finished product, whose origins (such as work pulled from Wikipedia) may or may not be apparent. We need to get our students to do both under our purview. Assessing the process, as well as the artefact, is a way to achieve this.

Labels: assessment, essay marking, Open University, pedagogy, University Life

In teaching students in a subject like English Literature at university level, one of the hardest challenges is to encourage them not to fixate on marks. At A-Level, students get given a fairly tick-the-boxes marks schema; if they do certain things right, they will get awarded a certain number of marks. This is why marks of 90% at A-Level English are not uncommon, whereas they would be once-in-a-lifetime beasts at university.
Understandably, such students arrive at university with uncertain expectations, and often struggle to know what is required of them to produce a good university-standard essay. They will almost certainly be aware of our marks schemes, but as I commented before on this blog, it is sometimes inevitable that the marker resorts to intuition rather than exact standards:

Marking criteria in a subject such as English are notoriously problematic. Whilst the rubric has obviously to be carefully considered, how is one to judge the difference between "well focused work" (65% to 69%) and "relevant work" (60% to 64%)? The mark schemes can only be taken up to a point, from where intuition takes over, the sense of a First as opposed to Two-One class work; this indefinable difference leaves high Two-One students seeing through a notorious glass barrier between a 69% and 70%.

Especially at the high Two-One end, students new to university think that there must be a certain additional number of boxes they can tick to get those extra percentage points to tip them over the 70%. And even beyond this level, I have heard students want to know the qualitative difference between a 72% and a 74% on two consecutive essays.

In this environment, giving effective and workable feedback to students is sometimes difficult. Students are used to thinking about marks in a rationalistic, even computational, way: input x and get a grade of y. At university, we want them to work on the principle of the writer, to be able independently to reflect on the quality of their own work and thought, and to be able to work according to an academic standard whilst retaining a sense of individuality in their responses to literary texts. Thus I can never say to students that if they do a particular something next time, this will guarantee a specific grade of improvement, though it may help towards it. Nor can I say that the difference between a 72% and a 74% is definable according to certain criteria; different essays may vary by two percentage points for a host of unspecifiable reasons.

To my mind, what should matter most to a university student is not their quantitative mark, but my qualitative assessment of how they could improve. One of the benefits of the traditional university at which I do some of my teaching is that we still also have one-on-one consultations with our students, to explain the finer points of their essays (though I have little doubt that in the brave new Higher Education world of market efficiency, these will soon be scrapped).

In these sessions, my marking strategy is always to conceal the actual mark from the student until towards the end, after I’ve had a chance to discuss their work in a general sense. Some get agitated, and if they start to tip over into anxious (or even floods of tears: not unknown) I do end up telling them their mark to settle them down. I’ve generally found this approach works well, allowing me to focus on areas that could be developed, without inviting the potential apathy that the essay was still a decent grade. Some students are perfectly satisfied with a 2:1, and I fear that the moment they are told they can get this, they would be less interested in the things I can tell them to do to aspire - with a bit more effort and directedness - to a First.

However, in my latest round of essay returns one student confronted me outright on this policy, having seen right through it. "What I really hate about your marking sessions," he said (tongue slightly in cheek) "is that you always tell me lots of things I could have done differently, but then end up saying it was actually all right." This is, in essence, absolutely true. But the comment has caused me to reflect on my own practice. My principle has been to avoid marks fixation by stating the grade only at the end. But maybe, once my strategy becomes transparent to students, this has the opposite effect: they know that I will get to their mark eventually, so they simply wait patiently but disconnectedly until I finally get around to what they have come along to hear.

I would be interested to hear what other teachers do, if they have similar one-to-one feedback sessions. Do you announce the mark at the beginning? Or do you wait until the end of a session to get most out of it? Which elicits the best response from students in the immediate setting of the teaching session, and which do you think will elicit the best response over the longer-term in encouraging development?

Labels: essay marking, University Life

This end of the academic term, students and teachers alike are faced with a cluster of deadlines. No matter how carefully one has planned, the writing or marking of assignments seems to lead to a last-minute rush before Christmas. Even so, I was still surprised to read a recent tongue-in-cheek article on Guardian Education, written by a university lecturer of all people, that offers some tips on how to pull the infamous "all nighter" to get those last essays done. I was, though, sadly unsurprised by the comments on the post, many of which seem machoistically to advocate the idea of doing things at the last minute.

As a marker, I do not automatically worry about an essay written late at night (and believe me students, it only takes a glance at File Properties to figure that one out). I appreciate that some people genuinely do work better under pressure. Some students dare to do such unproductive, degree-distracting pursuits as charity fundraising, drama, arts, sports coaching - all of which are more likely to get them jobs than a standard 2:1 degree alone. Contrary to popular belief, many students do intense degrees with a full-time burden of lectures and assessment. And some students - not least those I teach with the Open University - have to juggle part- or even full-time jobs to fund their education.


However, for every well-intentioned student who is forced to work into the early hours of the morning through no fault of their own, there are far more for whom, I suspect, this is not only a bad habit but a required rite of the university experience. I do worry about a university culture where the ability to do an essay at the last minute is a sign of bravado or "working the system," as was implied by many of the posts on the Guardian article, or as I detect in my work with mainstream university students (by contrast, I know full well quite how hard my Open University students work, even if they too are, by necessity, forced to work late into the night).

Doing an all-nighter may focus one's mind to look at internet resources and hash together notes with great efficiency (which may indeed be useful for the harassed office environment), but this does not necessarily lead to a student developing a thorough and cultivated knowledge of a subject (and those three years before students step through those office doors are the only time when this precious opportunity will be available).

In my experience those students who plan their work well ahead may not automatically get better grades than the all-nighters, but they do perform better than they personally might if they were not so enthusiastic and capable of planning their work around other commitments; conversely, I mark many all-nighter essays which I know full well do not represent a student's potential, even if I still award them a decent grade. Whilst all-nighter essays are usually coherent, focused and well-written, they often lack the attention to detail, careful proofing, and editing needed for the first class marks - marks which I am certain more of my students could, but don't, achieve.

I sense that students are increasingly swayed by an anti-"geek" attitude, such that those who work hard are frowned upon by those who can pull things out of the bag at the last minute to get equivalent marks. Indeed, I have first-hand experience of good students who find it genuinely difficult to cope with the campus atmosphere where it is only the end that matters, not how hard one works to get there. From the point of view of a diligent student, it can be intensely demotivating if they "only" receive the same mark as someone who can churn work out at the last minute (even if, in the long run, the former student will likely turn out to be better educated).

The idea of doing assignments just to "get enough marks" is precisely the sort of utilitarian principle that underpins Browne: the sole measure of the value of a degree is how much money you can earn at the end of it (or the mark you get on graduation).

As I wrote in my post on the implications of Browne for teaching and learning, when the student becomes a consumer of their education this fundamentally changes the idea of what that education is about, namely an end product rather than a means. As a teacher, I know that the best thing for my students is to motivate them sufficiently so that they actively want to do their subject, rather than suffering its interminable assignment demands just to get the degree at the end of it. But this may become increasingly difficult to achieve in an education marketplace. Browne suggested that student-consumers should be required to sign a contract, which include "commitments on attending a minimum number of classes or completing a minimum number of assessments per term." Why should students who are paying for their degrees have to do any work at all? After all, you don't buy a happy-meal in McDonalds, only to be told you have to cook it yourself. Thus I can see that the culture of all-nighter bravado which, let us admit, has always been there in university life, is going to get more prevalent, as students seek - or are even encouraged - to do as little as possible to get the reward, the degree result, they have effectively purchased.

Labels: all nighter, Browne review, essay marking, University Life

No, the above title is not a reference to myself, ill-tempered though I may have been as I was buried under a pile of exam papers this past fortnight. Rather, it is a reference to a peculiar quirk of my marks' outcomes this year. When I marked last year, I noted in my post on An Examiner's Perspective that it is tempting to try to mark predictively according to the neat bell curve, that sees a few marks in the 2:2 range, more in the first, then the majority towards the upper middle. Although we may instinctively want to rail against The Mismeasure of Man by this omnipresent graph, like it or not that does seem to be the way in which students fall, even if one tries to mark without statistics in the back of one's mind.

Except this year, and illustrating the dangers of presumptuously assuming the bell curve will always appear, my marks seem strangely to have fallen more like a double-humped dromedary. They have taken on a hilly appearance, with a lump of marks around the high 2:2 or low 2:1 range, then another lump around the high 2:1 or low First range, with a body of marks missing in the middle. It is hard to know quite how to account for this phenomenon. Were it that all my marks were uniformly higher or lower than expected, it might be reasoned that I was marking unfairly or too leniently. But with them pushed to two poles, it is unlikely that I was alternately over-zealous and over-exuberant.

The only thing that can feasibly account for it is a quirk in the year group. It is hard to conceive how a year group can differ substantially from year to year at university level, such that a whole group of student consistently underperforms or performs very highly. Certainly, in the closed and fashion-pressured environment of schools, a particularly hard-working or lazy group of esteemed peers can conceivably pull an entire year along with them up or down a scale. We have all heard teachers complain about horrible year groups, and celebrate brilliant ones. However (and maybe I am being naïve here), my belief is that at university students are largely independent learners, and come with more independence to pursue their self-set aims. Whilst one is pushed into attending school, one opts to go to university, and is less likely to be subject to the pressure of a group of peers not to do as much work as one might independently want to. So why so much variance between the able and the less able, or the hard workers and the less hard workers, as apparently testified by my marks?

One other potential explanation presents itself. This is that A-Levels are decreasingly valuable as preparation for university study. Although virtually every student will be coming to my university with three As, that letter encapsulates a range of abilities, rather than the minority elite as it once did. On the other hand, being a well-established and top-ten university, it will attract people who have genuine talent and ability (as well as, more likely than not, a background in private or grammar-school education). These students might be expected to perform very highly indeed. But there might also be a tranche of students who are less capable performers, who have still got in on the back of a three A grades. (Lest we be too pessimistic, it must be acknowledged that "less capable" here still means very good indeed. Even the low 2:1 essays that fell into the first of my humps testify to very good writers and literature students.) This split in my marks might be the first indications of the inability of A-levels to discriminate between the genuinely excellent, and the straightforwardly good, candidates.

Of course, the most likely possibility is that I am simply reading too much into a limited range of data. Almost all the essays I marked fell within the 60 to 70 range, meaning that there are only a few percentage points between what I would call a "low" grade and a "high" one. Having marked less than 100 scripts (which, let me tell you, is still a damn lot of essays!), this could just be a statistical blip - and one that gives the lie to the predictive value of the bell curve, except with very large numbers of students indeed.

Labels: bell curve, essay marking, exams, University Life

I am currently marking my way through 70 exam scripts, for a couple of the introductory English Literature modules at my university. This blog post is the confession of this examiner, perhaps a bit risky if you find out who I really am, but nevertheless I hope worth making public as a way of demystifying a process between the end of the exam and the publication of marks that students do not often see or even understand.

If I remember my own, not-too-distant days correctly, students might want to imagine that examiners treat their scripts as sacred objects. A student has attended numerous lectures and read numerous books over the year, poured over revision notes long into the night, and then spent a few hours hunched over a desk in some dismal hall, frantically trying to pour out knowledge in the hopes that that brief exam will do justice to all the hours of work put in over the previous year. With this much invested on a few sheets of paper, surely examiners deal with them reverently, in a darkened room, with the white paper subject to the glare of an anglepoise lamp, as the examiner interrogates and teases that script to give up its worthy marks?

The reality is somewhat different. Naturally, I look after exams with the utmost care, and mark them as conscientiously as I can. However, there are certain unavoidable practicalities of marking, and of human psychology, that mitigate against any such pure, religious process described above.

The big practical issue is time. With a large number of scripts to be marked in a brief period, it is simply not possible to spend hours on each one. It would be nice if I could read an essay carefully, and then go for a walk, take a shower, and massage my temples as I try to weigh up whether to give it 66 percent or a 67. But that does not - it cannot - happen. Even marking a qualitative, essay-based subject like English, having read an essay I tend to place my mark quickly and instinctively. At my university, we work from very detailed guidelines that explain the characteristics that should be present in an essay for it to merit a First, 2:1, 2:2 or lower, with each band sub-divided into two, for example, a high 2:1 (65 to 69 percent) or a low 2:1 (60 to 64 percent). It is very rare that I ponder deeply what percentage to give an essay. Essays usually fall easily into a band, and the pressure of having perhaps a week to mark 50 scripts leaves me little time to deliberate at length whether it needs a 64 or a 63.

People often grumble that an essay-based exam cannot be marked as objectively and as fairly as something like mathematics, with a right or wrong answer. Certainly the personality of the marker may have an effect on a percentage point here or there. But on the whole it is always surprising from my examiner's perspective how easily papers drop into one of these assigned bands. The moral for university students, then, is not to lose sleep over percentages. It is the band that says everything about what sort of student you are, even if you are frustratingly just on the borderline. In many ways, a 69 percent is the most horrible mark an examiner has to give - and in the last few days I have been heard shouting at papers, because I was frustrated that a good student was not quite there, and could see that with a little nudge and feedback the student could go on to improve in subsequent essays. But my 69s are below that glass ceiling not because a few tiny details were overlooked by me, the examiner, not because I was tired, or because my football team had just lost, but because it read, argued, reasoned, discussed, evidenced in ways which said 2:1.

With this caveat about the band being everything, I will admit to some of the other factors that an examiner faces that may well lead to small variations in marks.

Imagine this scenario. I have just read two First-class essays. The third essay I mark is going to have to do something impressive not to look weaker in comparison (for those of a mathematical bent, this is an effect called regression to the mean). Perhaps I will dock it a few more marks than I might have done if marking it in isolation, because it compares worse against the previous efforts. But in the alternative scenario, marked after two solid but not particularly remarkable 2:1 essays, perhaps suddenly essay three looks better than that localised average. I know that I must be guilty, at times, of marking relative to other essays, rather than against the single standard of the mark scheme.

Luckily, there are a few ways to negate this effect. One of the most hotly debated is that fad of the 1990s, the bell curve. Perhaps I get a run of three weak essays before lunch, and then suddenly give three Firsts after lunch. Is it that I am in a better mood after my break? Is it that I have remembered those three earlier, average essays, so that those that come later are bound to look more positively in their light? I do get anxious when runs of unusually high or low results happen - as they have done this year - and that is why I find the bell curve a useful check. I may perceive that my marks are being affected by local circumstances, but taking a larger sample of my marks, I can see that they have fallen out in a normal distribution. Usually, there is a statistically good range, with a smattering of 2:2s and Firsts, and the majority bunching around the mid 2:1.



The reason that the bell curve, or normal distribtion, comes in for debate is that it is tempting to mark for the curve, rather than to construct the curve on the basis of marks. Out of ten essays I have given three 66s. Better make the next one a 59 or 71 just to smooth out the graph. This is a real risk for the individual examiner, whilst institutionally it may be tempting to adjust marks across the board to create a smooth curve with its apex at the point the university suspects most candidates should be at. In my institution, with most students coming with excellent A-levels, we would expect more high 2:1s and Firsts than another institution with a lower achieving intake, so our marks tend to have a peak around the high 60s.

Now I do not know - or have reason to believe - that my own institution does any sort of retrospective adjustment to bump our averages higher than the national baseline for English Literature degrees, but if they did the problem would be clear. Just as I get funny moments marking when there have been no Firsts for ages then three come along at once, an institution could quite feasibly have consecutive year groups which seem to achieve comparable marks, until one year is comprised of an unusually bright or slightly less well-performing group. By shoving that bell curve to fit expectations based on previous experience, the institution is engaging in a sort of social engineering, making results fit students, rather than the other way around, so that the unusually bright or underperforming group is down or upgraded unfairly. This is precisely the sort of complaint about "grade inflation" long levelled at A-Levels and GCSEs, and increasingly at universities. But as an examiner, I can sympathise with the faith in statistics and the normal distribution, because it offers subjects like English an objective foundation for marking, helping to cancel out those personal factors that do come into play, no matter how hard one tries to contain them.

The bell curve aside, students need to remember that the mark they get is not dependent on the individual examiner because other, less controversial, controls are there to restrict the impact any one examiner can have. I have admitted that time, my mood, marking an essay relative to previous results, the effect of statistics, all can affect what percentage an essay achieves, even though I would hope that these would not affect which broader band an exam falls into. But once they leave my hands, exams are filtered through layers of double-marking, moderation by other examiners from within the institution, oversight by external examiners outside of the university, anonymous exam codes, board meetings, appeals procedures, publicly displayed marks so that it is possible to see how each year's exams compare to previous ones and, finally, individual students can request copies of their exam papers and examiner's comments under the Data Protection Act. These controls too ensure that, when the best-willed examiner gets a mark an entire band out, it should be an isolated incident.

However, this last control - allowing students to see and hence to interrogate their own papers - is also controversial. My own university does not exactly make public the fact that students have a legal right to see their scripts after they have been marked. Personally, I think this right should become an expectation among students, who are still often fearful of approaching departments with what seem like trivial requests. The National Union of Students has a policy that feedback should be provided on exams, and have issued stickers for students to put on exam papers stating that "Exam Feedback Helps Me Learn." From an examiner's perspective, although in many cases it is not possible to indicate specific places where students might improve (again, partly because time pressure makes it impossible to write detailed comments), there are many papers about which I do note specific stylistic issues that could be quite easily addressed. Making these comments, though, seems like shouting into the wind, if students are never going to get the opportunity to see them. Having gone to the effort to mark a script as an examiner, why not at least allow students to get as much from your work as possible?

Besides the adminstrative burden, the reason universities are reluctant to provide exam feedback is, I suspect, from a fear of litigation or of students picking examiners up on every point to gain even more marks. Even if the fear of litigation is a little hyperbolic, the idea of student's challenging their papers may affect the exam process unduly. Those students prepared to go through the technical process of questioning their results may end up with better marks than those who are mostly concerned with studying their subject for the pleasure of it, and who are not so end-focused, and who simply accept the results given to them and look to the following year. In a system where exams are always open to challenge, results might become partly determined by a student's ability to work the system, rather than their ability in any given subject. On the other hand, is this issue not precisely the problem with exams overall, that not only are they testing knowledge but they are also testing one's ability to sit exams and to have good "exam technique" in the first place? Allowing students to interrogate and receive feedback on their own marks at the end of the process only mirrors the effect that happens in that artificial period called "exam season" at the start of it. At this time of year students who may have done less work all year sit down to cram and prepare model answers just to pass the three essay questions on an exam, whilst students who have conscientiously studied broadly throughout the year continue in their model approach to their subject in a way that does not always help them to focus on the specialised nature of an exam. As an examiner, I usually have a pretty good hunch which students have prepared to pass a few questions on the exam, and which have enjoyed studying their course as a whole, but it is a very difficult thing to prove, and it is not possible to adjust marks based on a hunch.

From my examiner's perspective, then, encouraging students to seek the written feedback from their exams would be a positive step, because it would add a qualitative report to the process, allowing those students who have worked well throughout the year even if not reflected in the pure exam percentage to seek guidance on how to improve. These sorts of students are more likely to incorporate these comments into their more holistic approach to the subject (such as their desire to write well), than those who simply aim to pass the exam as a technical challenge, and so hopefully some sort of levelling might be achieved.

If you are a student reading this post, then, I hope you feel some sense of schadenfreude. If you have been sat there feeling fed up about the fact that you have to work through exams which seem a disproportionate measure compared to the way you have worked throughout the year, it is worth knowing that this examiner at least feels the same way about marking the exams. It may be slightly disturbing that I have drawn attention to the human frailties of the marking process, but on the other hand I hope too students appreciate firstly that it is bands, not single percentages, that are the most important indicator of ability, and secondly appreciate the lengths institutions go to in order to mitigate against any widespread effect marks can be consistently misjudged, even though probably every examiner misplaces a percentage point here or there, and even occasionally gets a band wrong.

The trouble is, exams remain the most efficient system we have for testing even qualitative subjects like English. The good news is that even though there may be candidates who can work the exam system to their unrepresentative benefit, and even though examiners of essay-based subjects may be unable, as ordinary human beings, to mark every essay to its perfectly deserved percentage, on the whole, the system, tumultuous though it is during the early days of Summer, does work.

Labels: English Literature, essay marking, exams, University Life

I have not posted anything here for a while. This is not due to my unwillingness to comment on Tony Blair's retirement, the climate change bill, the bad science of Panorama's Wi-Fi "investigation," or the hilarious science of the newly opened Creation Museum in Kentucky. Rather, it's got much to do with the large pile of exam papers that have been sitting on my desk for the past couple of weeks. This time last year, I commented on the postgraduate perspective on undergraduate exams, lamenting the communal hush brought by exams upon the lively university activity as well as remembering that they provide for one of the great British moments of the communal moan.

This year, that general moan has felt a little more prosaic, as I have heard it not amongst the students but amongst staff who have to mark the papers. Merrily responding to an email asking if I would like to mark some exams this year, my smile dropped as I was landed with 70 plus scripts. Marking these has been a frustrating experience, time consuming, often repetitive, but - conscious of the responsibility of marking summative as opposed to formative work - I have had to focus closely on the task.

Marking criteria in a subject such as English are notoriously problemmatic. Whilst the rubric has obviously been carefully considered, how is one to judge the difference between "well focused work" (65% to 69%) and "relevant work" (60% to 64%)? The mark schemes can only be taken up to a point, from where intuition takes over, the sense of a First as opposed to Two-One class work; this indefinable difference leaves high Two-One students seeing through a notorious glass barrier between a 69% and 70%. I have some sympathy with the government's plans to standardise degree classifications, which at the moment are not comparable across different universities or subjects, making it very difficult for employers (who may not be aware of the divide between a First and a Two-One, or of the difference between the University of Polytechnic and the University of Redbrick) to compare candidates.

And yet, having covered so many scripts, the glass barrier seems to me to be a valid one, and there is a qualitative difference between top and good work, one which cannot accurately be reflected in the quantitative difference of a single percentage point. Further, marking by a combination of rubric and experience does appear to work, at least according to the systems of double marking, moderation, external examining and the distribution curves against which we are judged. My grades passed their moderation, though with some slight modification in precise percentages in the first category, and the tally chart of grades I have been keeping has turned out to form the tell-tale bell of the normal distribution, centred around the high two-one.

More positively from a personal point of view has been the opportunity to get the sense of a year group, and a year's work (something I can't obtain by teaching a few tutorials a year to a few groups). As script after script pursues similar lines of argument, and presents comparable pieces of evidence, and similar historical, social and philosophical understanding, I realise that teaching does actually work: lectures have been attended, information has been absorbed, knowledge gained. Even when formal teaching comprises the minor part (about six hours) of the undergraduate week, it has a huge impact on a student's cumulative education.

However, the recognition of this leaves me frustrated that a further opportunity to educate students is not being pursued. The greatest frustration of marking has been my inability to follow up those marks with individual advice about how they might be improved. A student (not one of mine) last week remarked that she had never attended any of the one-to-one essay handback consultations with her tutors. I remarked that, regardless of whether they wanted to go or not, it is slightly unfair on the lecturer not to attend, since if they are anything like me, the greatest satisfaction is filling an essay with red pen, but then being able to tell the student precisely which aspects of their work were really positive, and how they can build on them. Seeing their subsequent essays, in which they have adopted this advice, gives a massive boost to the teacher. Teaching the really bright students, those who come with a unique and advanced writing style anyway, is rewarding, but I'm not sure how much "teaching" they actually benefit from; teaching those with potential not yet fulfilled, and bringing it out through contact with them, is by far the best aspect of the job.

Yet it is an aspect for which there is not as yet a replicable system in relation to the end of year exams. How frustrating it is to mark a paper in which one answer attains a good First, whilst the other two answers are solid Two-Ones! I am confident that, because the students don't see the breakdown of their marks, that Two-One candidate might go away from the board on which results are published believing themselves to be sitting comfortably in that latter category, whereas I know that, if they were told that they were capable of the very highest work, and shown the evidence of this on the papers, then the prophetic fallacy might kick in. At the bottom of our marking forms is a reminder that under new data protection laws, students can ask to see the forms, but since I doubt many are pro-active, we should make it available to them from the beginning. Our feedback may only take the form of a couple of sentences, and clearly there are not the resources to have face-to-face meetings with students, but to see that First on the page, nestling there amongst the expected results, would be, one hopes, a significant incentive for the second and third years, when marks count towards their final degree classification.

There is a great danger that the First word is something only whispered to a select few, adding to the mystique of the glass barrier. I know this was something I encountered, and even by the end of my degree I was still unsure precisely what constituted top work, and even whether I deserved it. This really is a culture of (a word often used wrongly) elitism, because every student coming to the top universities with good A-level grades should be capable of striving for the ultimate result, though for a number of reasons they may not reach it (the formal degree is only a part of a university education). So today, with almost every student, I bring the word into public discourse, saying openly which parts are First Class responses, demanding that if they get a Two-One on their early essays they should be aiming to achive the grade above by the end. Inexperienced (and possibly naive), I cannot know what impact this actually has. But it's a shame to put so much effort into marking, only to have students discouraged from making efforts for top marks.

Labels: essay marking, exams, Postgraduate Diary

One of my favourite tasks of the academic year took place last week, when I handed back my first-year students' first essays. At my university, we are lucky enough still to have a system whereby each student has an individual fifteen minute slot with their tutor, in which the tutor returns the marked essay and explains and discusses its positive and negative points. Obviously, this is a massively intensive use of teaching resources, compared to tutorials (one teacher to eight students) or lectures (one tutor to several hundred). It is also massively useful, both for the tutors and for the students.

In the case of the latter, who are coming to our university with straight A-grades, it can often be shocking suddenly to find that they have gone from getting many ticks on their work from a school teacher who thinks the world of them, to having scrawls of red corrective pen applied condemnatorily by a tutor who has known them for a couple of hours. Shockingly, it is possible for an A-level English Literature student to get 100%; and to have met our standard entry criteria, all our students will have achieved higher than 80%. So to get their first essays back with a mark of 60% (a mere 60%!) can be quite a shock to the system. I know it was to me. With the handback sessions, however, we get to alleviate these concerns, to assure them that a 2.1 is perfectly normal for their stage of work: after all they are three years from becoming graduates, and three months from being school pupils. When put in context in the handback session, that corrective red pen is less a condemnation of where they are at, than a prompt to look at areas in which they need to improve, if they are to realise their potential and achieve a First.

For my part, the handbacks are beneficial because I get to have an individual meeting with my students, in which I can find out how work really is going (cutting through the mumbled happiness that comes across in a tutorial group at 9.00 on a Monday morning), I learn a little bit about their background and other interests, and I get to talk to them as individuals. It is after the first round of essay handbacks that I finally start to remember student's names, and put names to faces, which has a beneficial effect on tutorials, preventing me from seeming like some anonymous voice of divine wisdom. And, without wanting to sound too arrogant, when a student comes in feeling nervous about how they have performed, and goes out knowing precisely what they did well and what they need to work on to do better in their second essays, I feel like I am making a real difference to them, intellectually and emotionally.

But if this is the high point of my postgraduate life, one of the lows must be getting negative feedback on one's own work. It is one thing for an A-Level student let down by a weak exam system to come to university unable to write grammatical sentences; it is another for a PhD student to suffer the same humiliating corrections to their style. Luckily, grammar is not one of my weak points, and although I occasionally write an over-long sentence with too many embedded clauses, this is a mark more of failed ambition than of limited capability. However, my supervisor is a fast reader, but a close marker, and any misspellings or errors will not escape her red pen, just as I hope none of my students escape mine (there is something faintly hubristic about the experience of marking).

But, in case the reader thinks I am getting a little vain, I must cut myself down to size. Although happily now I am able to write to a technically high standard, it was not always the case. Although most of the essays on The Pequod were written in the last couple of years, a few of the essays were written as undergraduate assignments (the bottom seven on the Essays page). That there are so few is partly because my computer crashed in my second year, so I lost quite a lot of work, and partly because I only put on those essays which were actually any good, both in my opinion and in the eyes of those who marked them. The exception is my essay on "The Representation of Memory in Time's Arrow and Shame." Written in my second year, I got a low 2.1 for this essay, and I was pretty upset, because I had thought I was writing in a very advanced way. Contrary to my principles, I decided to put this one online to spite my tutor, rather than for grander principles of public education.

But my vindictiveness has come back to haunt me. Through my Statcounter, I discovered that an English Instructor at the University of West Georgia has given her students a link to my essay in their reading assignments, and asked them to critique it in class. Through Google I discovered some of their presentations and responses were also published online. In the same response my tutor might conceivably have had, one of these complained that:

the author [sic] thoughts were confusing. He made random points that were not valuable to his argument. We believe the intended reaction was to inform the reader of the importance of time and narration in the story. Yes because we were too confused to understand the point author was trying to get across.

How dearly I would love a handback session with these students! Although, actually, on re-reading the essay, their points are valid. Whilst some of the other responses were more complimentary, I have to admit that the argument does appear highly convoluted, with structural weaknesses both at the level of sentences (winding and long-winded) and of the essay as a whole. Finding my original essay plan, at 6000 words I expect I probably had too much information, and a too-passionate desire to disseminate all my knowledge, rather than succinctly presenting a briefer but more coherent argument (a comment that could have been lifted from one of several of the handback forms I distributed to my own students last week).

Whereas when I received my mark at the time I was a little horrified (and clearly still am, given my drive to publish it online when I launched The Pequod in 2004), I can now look back laconically. I can appreciate the flaws in the essay, safe in the knowledge that, in my second year of a PhD, I would not write in such a way today. And I can laugh at the irony that, a week after I return work to my students, here is an English postgraduate in the United States, probably the same age as myself, giving her undergraduate students my old undergraduate essay to read, and then publishing their comments on my work online. Oh, what incestuous circles we English literature students weave and wander in.

Labels: essay marking, Postgraduate Diary

I remember being horrified by the amount of red pen scrawled across the first university essay I handed in, correcting basic grammar and sentence construction. If the technicalities of my writing were not up to scratch when I started university, I could blame my teachers and (probably more significant) a flimsy A-level system which allowed me to score high marks without being able to write grammatical English. Seeing red, both literally and figuratively, I was shocked into action, and I am happy to say that by the end of my first year my essays were being commented on more for their content than for their syntax.

Having taught last year, and having hovered and plunged my red pen above and into numerous essays, regular as a sewing needle stiching the holes in English usage, I was clearly not alone in being unable to construct an essay without splitting my infinitives or, worse, leaving a comma hanging mid-sentence when there should have been a full stop. I only hope my red pen electrified my first year students into corrective action, as it did to me.

With term now underway, in a couple of weeks I can expect to meet my new tutorial groups. So it is by a happy coincidence that our local, parochial Parish News dropped through our letterbox the other day, with the following helpful advice that I can give to my students when I first meet them:

I have a spelling checker,
It came with my PC.
It plane lee marks four my revue
Miss steaks aye can knot sea.

Eye ran this poem threw it,
Your sure reel glad two no.
Its vary polished in it's weigh.
My checker tolled me sew.

You know you have an English degree when you are confident in telling your Word grammar checker that those squiggly-underlined sentences really are perfectly constructed, and that contrary to the spelling advice you really are practising (not practicing) good English in your essays.

Labels: essay marking, Postgraduate Diary