Author Archives: Jenny

Put away your h-index Professor

As a childless, single woman over forty, opines Professor Jordan Peterson, “You’re going to be one lonesome, isolated, miserable creature“. Don’t hold back now, Professor. Why not add meaningless, pointless and desperate? But, thank you for your “evidence-based” assessment. Not that I asked for your opinion, you just appeared on my YouTube feed and I clicked on you. But, silly me, I clicked on you. It is the modern equivalent of asking for your opinion. Like striking up a conversation with a man you’ve never met before who happens to be sharing the same bus shelter. Except that my limited experience of strange men in bus shelters is that our conversations are safely confined to the arrival and departure times of buses and the weather.

I suppose when one has an h-index as big as yours you might feel emboldened. Perhaps you even feel entitled to be an authority on the subject of women (an h-index, for those outside the academic club, is the equivalent of currency – a highly valued measure calculated from how much you’ve published and how many people have cited your work). Who would know better about the lives of women than a male academic, with a towering h-index? I know. I’m revealing my h-index envy here.

And one needs to tread carefully. Not long ago, the tweeting mob rounded on one person with a smaller h-index. Ira Wells had the temerity to question Peterson’s academic work. But who was he? How much had he published? Who had cited him? The message from the mob, and Peterson himself, was clear. Only some have authority to question. And be under no illusion, the mob exists in academia as it does elsewhere. Outsiders without the necessary credentials (in this case a humanities scholar questioning a scientist), interlopers who fail to observe the crustimoney proseedcakes (Pooh Bear, 1926) are summarily dealt with and despatched. But let me not get diverted by yet another interminable discussion about academic metrics. Suffice to say, the necessary credentials are nothing more than the currency that the academic community has agreed to value. While that agreement is far from universal, it is the agreement that currently prevails.

And, the voice of the mob, it seems, is baked into our genes. With a single click of the mouse, that voice is embodied and articulated through the esteemed personage of Peterson. Have I just uttered a heresy? Peterson is a scientist, a respected scientist. Hell! All these scientists, not long ago, marched for science. We need science. And I agree, we do. But we need a science that is built upon humanity, not an inflated h-index. We need a science that values diversity, kindness and care.  Not just for its subjects but for its workers and for its audience. And part of that care, involves not only choosing our words carefully but thinking through the purpose and the implications of uttering them in the first place.

Imagine, if you can Professor, a woman of advancing years, who is single and childless and who, for whatever reason at some moment, is struggling with her place in the world, with the meaning of her existence. And, I suspect most of us have those moments at some time. She clicks on your exposition and as if her own self-talk, her own self-doubt, informed by the prevailing norms of the wider mob of humanity, does not cause her sufficient grief, you march into her space. With the clatter of your academic jack-boots, your polished articulation, you proclaim, with all the authority of science behind you, that she is, effectively, finished. Do you really think that helps? And what is your message to young women? Conform to the norm or be damned? For the only thing your science has discovered is the prevailing norm. I have to tell you, this authoritative scientific finding comes as no news to our subject. She has soaked in these norms for years and understands them better than you think.

I feel fortunate to live in a country where the person who holds our highest political office is (often) barefoot and certainly pregnant. She is smart, savvy and humane. I don’t think she has an h-index at all. Your clear advice to her, should she make the mistake of clicking here, is on no account to try to do the job of Prime Minister. However, it turns out for her and for us as a country, that the very fact that she is doing what she is, is shaping our new normal. For now, at least for a time, there is hope of a solution to the paradox of normality and it involves the continuous hard labour of embracing difference, diversity and normality. As Laurie Lee observed in Cider with Rosie:

We were as hateful and cruel as most primitives but our inborn hatred of freaks and outcasts was tempered by meeting them daily.

So, now I must choose my own words carefully. From where I sit, far from tempering the cruelty of the mob, all that your “science” and use of social media is doing, is magnifying the pain of those who live outside the norm and about whose lives you are arrogant and clueless. Worse, it seems  you profit from your misogyny. What makes you think you have any right or authority to speak about what is right for women in general, for young women or for any woman? Put away your h-index Professor. Put it away.

PS: Clover is no post-anything scholar. The only post she engages with is a fence post but she is most definitely out standing in her field.

 

 

 

 

 

Personalised learning or a flight of fancy?

From the time when I was a very young girl until probably my late twenties, I recall the intense irritation I felt when (usually) older men, used to say to me something along the lines of, ‘Smile, it might not happen!’. Setting aside the horribly gendered implications (I was a child in the 1960s when little girls were supposed to smile), I have the sort of mouth that in repose tips downwards. This conveys the demeanour of one who is unhappy, discontented, or sour. I’m none of those things on a permanent basis. As an older woman now (and thus largely invisible) people comment less on my facial expression but if they do it is invariably to express surprise that the picture they see on my face is not reliably consistent with the words they hear uttered from my mouth.

I know I am by no means unique in the south-pointing mouth department. A more notable example is Helen Clark, former Prime Minister of New Zealand. While being female, capable and staunch were probably more than sufficient to invoke much of the vitriol and bigotry she has been subjected to, I’ve often wondered whether being in possession of a mouth that naturally turns down at the edges has exaggerated the effect. A single anatomical feature ‘confirming’ the worst generalisations about her character.

And this bias, bothersome as it is, is at least a human bias. More troubling, is the palpable excitement and enthusiasm in educational circles for affective computing, or emotion artificial intelligence (AI). Here, such bias can be compiled into and rapidly promulgated by machines. Affective computing proposes to identify, interpret and predict human affect, a major component of which is emotion. A range of data are used for this enterprise. Automated detection of facial expression is an important one but also the analysis of speech and language and physiological measurements such as pulse rate, blood flow and skin resistance.

As the owner of a face that is regularly misinterpreted by others, I have more than a few misgivings about building algorithms that rely, in whole or in part, on identifying facial features associated with specific emotions. And setting aside the issue of troublesome facial features, how confident can any of us be in differentiating emotions? How to distinguish anger from fear? Melancholy from sorrow? Emotions manifest in subtle ways and can change in a moment.

“Orlando would fall into one of his moods of melancholy; the site of the old woman hobbling over the ice might be the cause of it, or nothing […] For the philosopher is right who says that nothing thicker than a knife’s blade separates happiness from melancholy.” — Virginia Woolf

Speech and language can of course be used to gauge emotion. We use language in this way all the time – and we regularly get it wrong even with context and a range of cues, such as tone of speech or body language, to draw on. When taken out of context or incomplete it may be impossible to discern the emotional state of the speaker. And, when language becomes data, context is the first casualty. If it is hard enough for people to correctly discern emotion or, more broadly affective state, machines working from impoverished context will fare no better and certainly not to within the tolerance of the philosopher’s knife blade.

If the use of facial features and linguistic analysis to automatically detect emotion is a bit wobbly the foundation for physiological measurement is blancmange. Heart rate and electrical activity of the heart are easy to measure and ideal for telling us about the state of the human cardiovascular system. For example, an irregularly irregular pulse and no P wave on an ECG usefully suggest atrial fibrillation in a patient. They are useful measures precisely because we can explain, in detail, the relationship between the cardiac cycle, by which blood is circulated through the body and lungs, the quality of the pulse felt and changes in electrical activity in the heart. By contrast, readily accessible physiological measures are only general indicators of emotional state and may not be causally related to emotional state at all. A  sympathetic fight or flight response can occur in wide range of contexts. A pedagogenic fear of exams and anxiety that a boyfriend’s mother is coming to visit may induce similar biometrics but their aetiology is quite different and requires different interventions – assuming making an intervention is the point of taking the measures in the first place – simple biometrics are not going to distinguish the two. Setting aside potential pathologies, a regular, rapid pulse and raised blood pressure in a student could equally be a reflection of plain old physiology. Maybe the student came to class after a party or perhaps they were just running late.

The American Psychological Association’s assessment of polygraph (‘lie detector’) tests makes a similar argument. Polygraphs employ physiological tests to infer whether a subject is responding truthfully to statements. Their use is contested and in most countries, they are not admissible in court. If we cannot be confident that simple physiological measures or biometrics can accurately and reliably indicate a binary motive, to speak the truth or fabricate, why would we imagine they could distinguish complex, mercurial, multi-valued emotions?

Yet already, in a few schools and classrooms, biosensors are starting to appear. From video cameras for facial expression detection, to heart rate or blood flow monitors and EEG skull caps. Data harvested from these devices, together with interaction data from apps used as part of coursework, is monitored and analysed with the aim to enhance learning through influencing a student’s affective state. Except these devices don’t monitor emotion or affect – they provide data from which we rightly or wrongly infer the presence of an emotion. Nor do they provide actionable insights to teachers or to students – in the wild of the classroom they cannot hope to identify the source of any particular emotion with confidence. What they can do is help to shape behaviour.

The inimitable Audrey Watters’ latest piece on educational technology and the new behaviourism cuts to the quick of this issue:

[Proponents of affective educational technologies] will guide us – algorithmically, of course – to “good” academics and “good” thoughts and “good” feelings and “good” behavior, defining and designing, of course, what “good” looks like.

Her cautionary article points out that emotional and social learning, developing things like grit and a growth mindset, are being rapidly turned into technology products designed to change behaviour and to change it on a very personal level.

Personalised learning, understanding individual needs and responding appropriately and in a timely manner, has been on the AI in education agenda all the way back to Pressey and Skinner and their teaching machines. It has surfaced periodically since in the form of the Keller Plan, intelligent tutoring and adaptive testing but has always been dogged by its behaviourist roots – until now.

The dominant discourse in education, which traces back to Dewey, eschews the worst of behaviourism’s excess. It holds that formalised teaching and learning should be student-centred, inquiry-led, community supported and attend to the whole student. Yet, how better to attend to the whole student than to marshal biosensors, big data and machine learning to meet unique individual needs, including social and emotional needs. The result is what Ben Williamson has delightfully called Big Dewey, which combines progressivism with big data and analytics and undergirds the drive towards personalised education.

But if we don’t understand the cause of an emotion how can we effect change to address it? We cannot. All we can do is effect change in the thing we are measuring – we can slow our pulse, modify our EEG and of course, smile at the teacher.

And as Watters’ convincingly argues, wittingly or otherwise, measurement, control, vested interest and commerce are at the heart of this project. The most alarming signs of what may lie just over the horizon include monitored classrooms, DIY neurofeedback kits and reports like this one from the Potomac Institute. But Big Dewey has issued his rallying cry and the pundits and politicos are taking note.

Perhaps 2017 has been the year for parody – for Big Dewey is surely a parody of both education and science – a mawkish, flatulent figure, resplendent in a silicon suit and topped with a jaunty skull cap, flapping his arms and willing his students to fly.

 

Clover’s triplets present their facial features. Photo: Nick Beckwith

 

 

 

 

 

 

Is it safe to pee on the street?

The University of Otago plans to install 60 CCTV cameras in North Dunedin at a cost of 1.27 million trumpeted across the NZ media on Friday May 12th. The ODT headline, ‘Cameras to keep our students safe‘ creates the clear impression that Otago students are not safe, yet nothing I could find in the ODT article, or any others from Stuff, NZ Herald or even Critic presented a clear rationale for what seems like a gross intrusion on individual privacy, and not just of students. If the CCTV proposal was confined to campus that would be one thing, but, it is not. A large chunk of Dunedin to the north of Frederick street and south of George and Warrender streets is slated for surveillance. But this is public property, not University property. The University may be a large and influential corporate citizen in our town but that does not give it carte blanche to intrude on the privacy of the rest of us.

I love living in Dunedin. It is an increasingly diverse community. It feels safe, egalitarian and fair-minded. Friends are never far away, the landscape is stunning and I can always find a car park – um – other than close to the University. I have lived here happily for more than 30 years.  One of the reasons I enjoy living here is because Dunedin offers many of the benefits of a larger centre yet, thus far, has largely avoided many of the downsides of cosmopolitan living, including wall-to-wall surveillance. And yes, I know, we already have 14 cameras in the Octagon. However, the Octagon is squarely within the zone where most reported crimes in Dunedin actually occur.

So, where is the evidence of such an egregious surge in criminal activity that warrants the intrusion of 60 cameras on all our lives in Dunedin North? The Dunedin City Council website notes that there has actually been a 5% reduction in reported crime in the city in recent years. Statistics NZ crime data for 2015 mapped by Andy Fyers shows that there were three times as many assaults, sexual assaults and robberies to the south of Frederick Street than there were to the North and the number of incidents in North Dunedin in 2015 is slightly below the national average. So, on the basis of published reported crime data it is hard to see a strong case for intensive surveillance in Dunedin North, even assuming that CCTV surveillance would actually solve this problem.

Nevertheless, those of us who live and work in Dunedin are well aware of the glass on Castle Street, the drunken, antisocial and sexist behaviour, and the couch burning…especially around certain times of the University and Polytechnic year. Of course, not all students engage in such antisocial behaviour  – but some do. Some self-entitled students seem to think that feral behaviour is a right of passage. A telling comment from one little noddy, quoted in the ODT and the NZ Herald, pretty much sums up this attitude:

“[A] Castle St resident … and a group of his friends said they would be supportive of the plan as long as the university did not use the cameras to catch out students for minor offences, such as urinating in public.”

Apropos of which, for an interesting case study on just how antisocial culture manifests in the Dunedin student quarter, Mava Enoka writing in The Wireless reported on her encounter with some other Castle Street residents:

“Before I could even catch his name, the boy started to gyrate against me as I stood in the middle of the lounge. The Castle Street boys roared with laughter.

The guy, who’d apparently been drinking for nine hours, went in for a kiss.

“Can someone please come get him off me!?” I said. I was trying to process why a complete stranger would greet another person with sexual assault.”

A culture change is required – students do not routinely act out in airports, in shopping malls or in other public spaces. Why should they feel entitled to behave badly in North Dunedin? Surveillance systems may be useful for institutions to demonstrate that they are doing something or they may help police to identify culprits but as the article in Critic points out there is little evidence that public CCTV systems actually keep anyone safe – they would have made no difference at all to the encounter described above.

An analysis of 44 evaluation studies of CCTV in public spaces, predominantly in the UK, by Welsh and Farrington in 2009 found an overall 16% decrease in crime in experimental areas (CCTV) when compared to control areas (no CCTV). However, the authors note that much of this decrease was related to the effectiveness of surveillance in car parks. According to the authors,

‘Schemes in most other public settings had small and nonsignificant effects on crime’ (p. 716)

William Webster, writing in Surveillance & Society warns of ‘surveillance creep’ and cautions that when CCTV systems become normalised, the original purpose of the system may evolve and take on new functions without further discussion or debate. What other purpose might the proposed CCTV network be put to? Checking student attendance? Monitoring staff movements? Tracking or profiling ‘non-students’ (see ODT article)? If the University’s plan proceeds, critical consideration of these issues will be important and needs to form part of the wider discussion with the people of Dunedin.

With more than $1 million to spend and all the intellectual resource that a research-intensive university has at its disposal, one might hope that our local Critic and Conscience of Society could manage a more thoughtful and convincing response to behavioural issues than wholesale surveillance.  If the University is concerned to manage perceptions it can do this without impinging on the privacy of Dunedin citizens. If the University wants to keep its students safe from crime, the best advice, based on available evidence, would seem to be to advise students to travel no further south than Frederick Street.

Naysayer rising

Never before have so many people been so well educated. Two of my grandparents left school without qualifications. Neither of my parents attended university. My siblings and I all had the benefit of a tertiary education. This is a common and familiar pattern especially in western democracies. But what are we learning?

One of the strong arguments for the value of education and in particular higher education, has always been to encourage questioning and critical thinking. Yet, as higher education responds to the imperatives of the global marketplace – increasing student enrolments, retention and progression, and demand for research outputs – loftily lumped together as research and teaching ‘excellence’, are we thinking anymore critically than our forebears? I suspect not.

Grandma Mac was my father’s mother. She left school around the age of 12 and went into service. She lost her own mother as a young girl and was brought up by a German baker and his wife in London. For this reason, despite her limited schooling, she could understand and speak a little German. I have no idea how her adoptive family were treated during the war. She was an accomplished cook and baker and was of the English wartime generation that knew how to make do. I still remember the giblet pie and her homemade lemonade. Grandma Willcox was my mother’s mother. She completed school, trained as a stenographer and went to work for the Indian railways. She drove an ambulance and helped the men coming off the Burma Railroad at the end of the war. Born and raised half a world apart, both my grandmothers would have known and understood suffering, intolerance and refugees fleeing horror and war. How could they not? However, the rampant individualism and consumerism of today would shock them both.

But alongside globalisation, growth, consumerism and the increasingly inequitable distribution of wealth and power, something else is happening. An old word is starting to reappear: naysayer. Now, of course language and language use changes, and it must, but this old word has been bothering me. Naysayer, as far as I can tell, means today much the same thing that it meant in the 1700s when it first appeared: a person who expresses doubt or denies or opposes something. But its use, estimated from Google’s ngram viewer, is increasing. And its plural use even more so. Naysayers it seems are everywhere. And if there is one thing you don’t want to be called today, it is a naysayer.


The plural use of naysayer casts those who naysay or question anything into an amorphous group of negative, disaffected, malcontents and misanthropes. And, we are everywhere, except perhaps in positions of authority, power or wealth. A simple question or call for explanation, whether in a public setting, in government, business, voluntary or even, paradoxically, in a university environment is quickly scotched with a charge of naysaying or being too negative. Simply taking care to question thoughtfully is insufficient. One must always be positive…

How did this happen? Has all our education equipped us only to take our place in a brave new world of noddies and simpering acolytes? For an example, look no further than the people surrounding Trump and worse, those highly educated world leaders who find themselves tongue-tied at the worst of his excesses.

At the risk of being labelled an archetypal naysayer, I’d recommend keeping a weather-eye out for this thin-lipped little word and be prepared to give it a good slap whenever you come across it.

Postscript: For a foundational reading on education, ideology and power see Michael Apple’s Ideology and Curriculum. To experience a native naysayer look no further than the Goat.

 

 

Changing paddocks and hitting the road

Clover has been fairly quiet for the last few months and mostly that is on account of her keeper being wedged down a rabbit hole without the benefit of keys or shrinking potions. No amount of irritable hoof-tapping from Clover could redeem the situation. I simply had to sit it out until the grip of circumstance released me and I popped out, fractionally slimmer, into the light and sweetness of the paddock.

As luck would have it, it’s a lovely new paddock and I am now happily engaged as a Research Associate with the Centre for Learning and Research in Higher Education at the University of Auckland. In October, along with my colleagues on our Ako Aotearoa funded project, Building an Evidence-Base for Teaching and Learning Design Using Learning Analytics Data, we’ll be heading round New Zealand for a series of workshops, seminars and conversations about learning analytics. We are visiting most of the main centres: Auckland, Hamilton, Wellington, Christchurch and ending in Dunedin. We’re looking forward to lots of good discussion, debate and opportunities to share practice.

My focus will be a workshop designed to introduce teachers to convenient and simple approaches to analysing large volumes of student generated text; specifically, responses to short-answer questions. To this end, I’ve created a Jupyter notebook that demonstrates an analysis pipeline. It begins with text responses and ends with a wordtree. Wordtrees were first described by Martin Wattenberg and Fernanda B. Viégas and are a lovely way to represent a keyword in context (KWIC). KWIC or concordances, are very familiar to linguists and language teachers but arguably less familiar to educators in general. They can be incredibly useful if you are trying to understand how people are using a particular word or phrase in a given context. As an example, here’s a wordtree with a focus on the modal verb need as used by tertiary teachers responding to a question about ethics and learning analytics.

wordtree

Participants who are handy with coding may find the notebook useful but there are lots of less technical options for achieving the same end and this is where the workshop will be firmly focussed. To me, the exciting thing is that text analysis has the potential to take us well beyond the click-counting, engagement proxies and predictive modelling that is the preoccupation of much learning analytics research. Analysing student responses along with the language that we use as teachers gets to the heart of teaching and learning conversations – and to the heart of student learning.

Find out more about the Learning Analytics Roadshow

 

 

 

 

 

 

ACODE70 in Orange

Until ACODE70, I had never heard of Orange much less thought that I might visit it. Orange is around 3.5 hours from Sydney by car, over the Blue Mountains and part-way between Bathurst and Dubbo. By train or bus it takes longer to get there but you see more of the scenery which, especially around the Blue Mountains, is spectacular. Stepping off the Australia-wide coach after dark to a deserted railway station and empty streets it felt like I had arrived in Brainerd, Minnesota (only without the snow). Marge Gunderson would not have been out of place and young men were shedding rubber from their tires on Peisley Street (pronounced peas-lee) –  a surreal start to what turned out to be a thoroughly engaging visit to the Charles Sturt, Orange Campus.

Charles Sturt is a regional university with multiple campuses in regional centres and a focus on distance education. As such, you might imagine more than a passing institutional interest in the potential of educational technologies. And so it was – in spades. The VC, Professor Andrew Vann, alighted on our topic of analytics and adaptive learning and teaching with ease. His opening address set the tone and left us in no doubt about the rapidly changing context of higher education, the challenges facing the sector and the many challenges and opportunities for those of us working within the educational technology field. He then removed his tie and joined us for a day of rapid-fire presentations and extended discussions; an impressive example of leadership in action.

Michael Sankey (USQ) kicked off the presentations by describing how USQ is sharing data across their various systems. A key lesson, both from Michael’s presentation and from the discussions that followed was the importance of working out what questions academic staff want the data to answer. Assuming you have all the data in the world and manage to make it accessible, you still have to figure out what you want to do with it. Perhaps because funding decisions and system-building follow institutional imperatives, pedagogical considerations often seem to just get tacked on as an afterthought. Linda Corrin (Uni Melb) introduced the OLT-funded LOOP project. One of the project outputs is a web-based analytics tool for visualising learner data and ‘completing the loop’ for teachers on the impact of pedagogical design decisions. It was interesting, especially  following Michael’s talk, that findings from Linda’s work so far suggest that academics struggle to work out what questions they want answered and that requests to include specific information or data are still at a fairly basic level.

Three presentations along the theme of staff development for learning analytics followed morning tea. Sheila McCarthy talked about several iterations of Learning Analytics at Griffith and the ongoing challenge of supporting staff to make effective use of the data and systems that are available. Garry Allan outlined the experience at RMIT and emphasised the importance of informing and engaging staff with learning analytics initiatives to avoid generating a culture of fear; who is measuring what and for what purpose? Cathy Gunn from the University of Auckland introduced our Ako Aotearoa-funded project through which we are creating a series of scenarios to support staff development. The scenarios are built from a range of case-studies and are designed to be used as templates to help staff to find solutions to their own teaching and learning design questions.

A shift in focus to engaging stakeholders followed for the next round of presentations and discussion. Deb West (CDU) made the link between the Scholarship and Teaching and Learning (SoTL) and the potential of learning analytics to help us to get to know our students and inform teaching practice. Danny Liu gave a lively presentation on getting alongside teaching staff to create bespoke developments at Sydney Uni and Macquarie to support student engagement and provide accessible visualisations for staff.

After a well-deserved lunch, four presentations burrowed into adaptive learning and personalised learning. Barney Dalgarno (CSU) reminded us that adaptive learning systems have a long history and pointed to new potential. For example, using data which describe student solution paths to suggest hints (See Barnes and Stamper, 2008). Lucy Webster gave specific examples of improvements in teaching and learning in haematology and histopathology at CSU through using virtual microscopy, Smart Sparrow and learning analytics. Lucy also noted just how time-consuming creating adaptive resources can be. Simon Welsh followed with a higher-level view of Learning Analytics at Charles Sturt. Finally, I brought up the rear with a change of tack to text analytics and a practical example from the health sciences to illustrate how teachers can use student writing to gain a window on understanding and adapt their teaching on-the-fly.

What did I take away from the day? Many things: confirmation that all institutions are wrestling with how to leverage the quantities of data we collect to extract meaning and value for teaching and learning; that institutions that seem most advanced in terms of learning analytics have positions dedicated to the task (data wranglers, learning/educational designers and so on); that managing the many tensions between institutional drivers and on-the-ground imperatives for teachers/learners is far from a self-evident enterprise; that black-box commercial systems do not solve all the problems no matter what claims are made; that creating a supportive culture for ground-up initiatives is essential; that scaling-up initiatives that are shown to work is key; that none of this is easy.

I think the last word from Stanley Frielick during the closing plenary captured the tension evident in most, if not all, of the discussion during the day: grafting analytics, which come out of agile, contemporary commercial enterprise, onto archaic institutions like universities is far from straightforward, if not completely flawed. Only one place to go after that – the pub!

Hats-off to Philip Uys and helpers who did a fabulous job organising everything, looking after us and hosting a very pleasant evening at the Union Bank in Orange.

Overall, a long way from my film-noir first impressions. Charles Sturt Uni and Orange are well worth a second visit.

IMG_1071

Orange Railway Station

From across the ditch

It’s been great this week to have Dr Danny Liu from Macquarie University visit us here at Otago. On Monday, Danny gave a talk, to a packed seminar room, on three projects he’s been involved with at Macquarie University and the University of Sydney: The Moodle Engagement Analytics Plugin (MEAP), the Student Relationship Engagement System (SRES) (for which Danny was a finalist in the 2015 ascilite learning analytics awards) and the Maquarie Open Analytics Toolkit (MOAT). Slides from Danny’s talk are available on slideshare.

For each system, Danny gave us a clear picture of the teaching and learning issues and motivation behind the developments, an insight into system evaluation, and reflections on student and staff experiences. Danny’s approach to development, firmly embedded in the local teaching and learning context but aligned with institutional and IT infrastructure imperatives, resonated strongly with us. There were lots of questions from the floor and some good discussions following the talk.

Several Otago teaching staff expressed an interest in working with us, here in the Higher Education Development Centre, to pilot a local instantiation of the SRES later this year. We plan to include this nascent trans-Tasman venture as a case-study in our Ako Aotearoa funded project, Building an evidence-base for teaching and learning design using learning analytics dataThis is a national project in partnership with the University of Auckland, Massey University and The Open Polytechnic of NZ.

We’ve also been sharing with Danny some of our bespoke teaching and learning software including Otago inFORM for teaching and course evaluations, UniTube, our media file sharing platform and Pictation for annotating images. We’re keen to explore and develop opportunities to share these and outputs from our current work with other Universities and Polytechnics in NZ and Australia.

Overall, a really worthwhile three days and we’ll look forward to continuing a productive collaboration – thanks Danny!

 

Women and academic promotion

As anyone, anywhere, who has ever applied for an academic promotion will tell you, promotion applications are onerous. Evidence is gathered, documents are written, forms are filled, boxes are ticked, support is sought and a hefty wad of material is signed off and dispatched with fingers crossed and a sigh of relief. Then, you wait. And wait. Months, rather than weeks, later a letter with the outcome arrives. Confidence and a firm resolve are required for sure.

There is a popular belief, and research which supports the idea, that women are less confident than men; they are certainly less likely to apply for promotion to senior levels. Yet, I have never had a sense at all that my female colleagues are a bunch of retiring violets – quite the reverse. What I think many women struggle with, I certainly do, is the valuing of certain attributes which go along with promotion at the expense of others which do not. There is a fabulous talk by journalist, Reni Eddo-Lodge who in just 3.5 minutes deconstructs the argument that women should be seeking equality by asking who we are seeking equality with and under what conditions. This is a very different kettle of fish to lacking confidence.

In any event, research also shows  that confident women can fare badly compared to confident men (see for example, Victoria Brescoll’s work on gender and talking and gender and anger). It seems that displaying just the right type and amount of confidence is a wee trick, especially for women.

Arguably, flawed process can also have an impact on promotion chances. A good example, is the issue of feedback. Usually, you can get feedback from promotions committees but you often have to ask for it. There is a real irony here. When tens of thousands of academic research hours and thousands of journal pages are devoted to the necessity of and benefits accruing from good feedback to our students, feedback to staff that would provide insight, assist performance and improve chances of future success is provided only on request… And, just imagine if your application has been unsuccessful: you’ve turned yourself inside out to apply, you get turned down and the last thing you feel like doing is going cap-in-hand to get feedback. There is a clear power imbalance and that may well play out as a reluctance to try again.

More pedestrian issues are the length of time it takes to process applications and the poundage of paperwork. The quantity of material to plough through will hardly focus the minds of any committee and worse, much of this is still handled in hard-copy. We’ve known for decades that human judgement is subject to bias (Tversky and Kahneman, 1974). Standardising criteria, as in the case of promotions, can help reduce this. Nevertheless, when judgement is required to assess whether criteria are met based on evidence supplied, bias will creep back in. And, hunting for evidence in mounds of paper seems unlikely to reduce bias. While any promotion applicant is well advised to clearly address the criteria, institutions moving to online and streamlined systems could also help, not least by making it easier to verify evidence. Many university processes have benefited from going online. For example, research at Otago University has shown that improving the teaching evaluations system may improve teacher engagement with evaluations in addition to reducing processing time (Moskal et al, 2015) . Imagine a similar effect on academic engagement with the promotion process.

If we are serious about improving the gender balance and encouraging diversity in Universities, we need to address process issues and inequities. Better still, let’s address the structural issues while we’re at it and critically examine what we value and reward in higher education (see an earlier post). If what is valued by institutions aligns with what is valued by more women, we might start to see more women applying for promotion and ultimately achieve gender equity across all levels of academia. The same holds for diversity; if we value diversity in our workforce, let our institutional values reflect diverse values.

And the caprine perspective? Clover, Chicory and their respective broods have little patience with structural inequities. They are called fences and are to be jumped over or pushed through and always when you least expect it.

JEM-NZ2016--0762

Chicory contemplates structural inequity

Life and death

Sir Ken Robinson once famously said in a TED talk, “university professors… look upon their body as a form of transport for their heads…it’s a way of getting their head to meetings.” Underlying some of the best humour is often a certain truth. Much of academic life involves thinking, writing, reading, arguing and abstracting away from the real world. And, I do believe (and worry) that the very distance we claim from the physical in order to be objective or reflective (or whatever epistomological adjective rocks your boat) means we can readily lose touch with the practical realities of life’s central conundrums.

A wee while ago I was fortunate to have some very supportive neighbours and friends over to help me despatch a goat. This is a necessary corollary of living a rural lifestyle; sometimes you have to deal with rural things including taking the life of another animal. Not something that comes easily to me at all. I struggle to kill anything including so-called pests like flies, aphids, possums and rabbits, let alone goats. Nonetheless, this goat had to go.

Clover had abandoned him at birth. Undaunted and with supreme human arrogance, I rescued him and enlisting the aid of friends and colleagues, hand-reared him much to Clover’s disgust. I probably should have followed her lead and let nature take its course. When small he was biddable and endearing but he was a cryptorchid and desexing failed. He was big, strong, impossible to contain and when in the rut, was familiar to a hazardous degree. Clover herself snorted and spat if he came anywhere near her. Several attempts at re-homing had failed; the most promising lasting only a few days before he was returned with apologies. I couldn’t bear the thought that he might end up solitary, tethered or endure worse if I just gave him away. So, after much soul searching, I decided he needed to serve a useful purpose; as food.

One minute he was standing in the paddock chewing on some blue gum and wondering why more people than usual had gathered, the next he was gone. One neighbour, expertly skinned and gutted the goat and the rest of us buried the remains. We shared a beer and a few reflections about the business of turning life into food then the carcass was carefully wrapped and stowed in the back of my car for delivery to the local butcher. Two weeks later, the animal was transformed into sausages and cuts of meat.

It was strange knowing the source of this food but also a comfort. The goat had a pretty good life. He had company, shelter and plenty to eat and drink. He was never caged, tortured, processed, tagged, fed drugs or crammed together with other anxious and frightened animals and trucked to his fate. His life ended as it had begun, in a paddock with a view of the hills and the sea.

So, how does this back-to-nature, grounding experience relate to higher education? I guess it is about the conundrum of choice and that so often you can weigh evidence and evaluate options but there is no ground truth, no scale of measurement. As academics whose deliberations are so often theoretical, there may never be a requirement for us to confront the real life outcomes of choices that we debate in the course of our research or for that matter in our teaching. I can rationalise the choice I made about the goat and have done exactly that in this blog post. But was it a good choice? Was it the right choice? Was it the best choice? Who knows. It was the choice I made and I had to make it because of an earlier choice that I made to intervene in the little animal’s life.

Does higher education equip us to deal with the endless succession of choices that we face throughout our lives? I don’t know but I do know that the sausages tasted very good indeed.

Straw men, academic artisans and instruments of torture

If the title of this post sounds like the title of a Bruegel painting that is because sometimes I feel like I am living inside a Bruegel painting. Reading a comment posted in response to an article in the Times Higher Education caused this unpleasant feeling to repeat on me, even while enjoying my Christmas break.

The article by Chris Havergal was reporting on a new study by Angela Brew, David Boud, Karin Crawford and Lisa Lucas, Absent research: academic artisans in the research university. The study suggests that academics who are not necessarily high performing researchers nevertheless play an important role in the functioning of the university.

I had never thought of academics who devote a lot of time and energy to what one might call academic citizenship as artisans but that is a good label for them. People who recognise talent, skill, passion and commitment in others and who use their skill, experience and wisdom to labour, nurture, shape, carve, create….not a functional work of art but something far more extraordinary: people working together towards a shared goal.

The academic trinity of research, teaching and service fails to capture the artisanal trait. The dominant discourse privileges individual research excellence but, it seems to me, marginalises the art, craft, creativity and teamwork required to foster productive academic environments.

So, it was with a heavy heart that I read the only comment posted in response to the article:

This is straw manism … this all seems like an exercise in making non-research ‘academics’ feel better about themselves. 

The full comment is here and you can decide for yourself about the merits of their argument. Here are my reflections:

  • I’m not sure just who or what the straw man is in this instance. The study sought to describe and illuminate the role played by “those people who have not developed a recognised or ‘accepted’ research profile for research assessment purposes“. Such people exist – they are not made of straw, nor, on the basis of the study findings, are the roles they play inconsequential. The straw man, if he is there at all, seems to have been raised by the person who posted the comment.
  • There are some fairly compelling reasons to ensure people do feel good about themselves and their role in the workplace. For example, how hard would it be to perform at a high level if you felt undervalued and invisible? Are you likely to operate effectively as part of a team if you feel that what you do is worthless?
  • I assume the scare quotes around academics are intended to emphasise that a non-research academic is an oxymoron. Yet, there do exist teaching-only academic positions in many institutions including my own, so that can’t be right. Or, maybe this is a cunning case of Grice’s conversational implicature and we all just know that non-research academics aren’t real academics …..

But, what I found really disturbing was that I could almost hear the closing words from this comment echoed closer to home:

This is the game we’re in, we need to play it (counting publications); Stop whinging, start publishing; If you’re not likely to make at least a B in the next PBRF round your job could be at risk … 

There is no shortage of research which describes, evaluates and contests the impact of national research performance assessments (e.g. PBRF in NZ, ERA in Australia, REF in the UK). The problem is not that there is not enough research. The problem is that in practice, the detail seems to be largely ignored by the institutions who claim to prize research. Funny that.

Personally, I think some form of accounting for institutional effectiveness is important. The humble taxpayer funds our work and deserves reassurance that their hard earned cash is not gurgling down the toilet. So, I have far less issue with the external measures per se than I do with internal measures of individual performance and gaming that have spawned as a result. Assessment, including research performance assessment, can have unintended consequences. Professor Jonathon Boston’s comments relating to the design of PBRF reported in McGilvray, 2014 are apposite:

“We simply failed to fully realize the implications of the Privacy Act and Official Information Act,” … “If I had known we would end up with a regime in which individuals had their scores reported to them, and that other people could potentially know what they were, I would not have supported it. 

Where university departments operate as largely independent fiefdoms the ground is fertile for manipulation. All performance indicators are not equal, decisions about what to count are subjective and within departmental and institutional cloisters, these decisions are almost impossible to contest. Furthermore, it is far from easy to gather data relating to academics’ experiences of research performance management. Comments made by Edgar and Geare, 2013 regarding the ticklish business of collecting data for their study into factors affecting research performance at NZ universities are telling in this regard.

In the hands of thoughtful, even-handed and artisanal Heads of Department there may be little to fear from internal performance measures. In the hands of bumbling but well-meaning or distant and distracted HoDs, there could be cause for concern. In the hands of self-serving, manage-up and boot-down career-climbers, alarms should sound: through a solitary interpretive act, something as simple as a publication count can be transformed into an instrument of torture.

The only outcomes of torture are fear, defensiveness, toxicity and despair. On every level, institutional, departmental and individual, this serves no one well.

So, good on Brew et al. for their study. I hope that it does get noticed and I really hope that it informs thoughtful and critical action. Blindly counting publications is the road to hell.

The Blind Leading the Blind by Pieter Bruegel the Elder, 1568

(Note: Clover made herself scarce at the mention of Bruegel. Stories involving goats during Bruegel’s time seldom ended well. I tried to explain that depictions of medieval goat owners were far from flattering either but Clover was already well beyond the reach of reason and had fled to the far paddock.)