Teaching advanced research and scholarship skills to postgraduate social science students

1. Teaching advanced research and scholarship skills to postgraduate social science students Paper presented at the 6th Leeds Learning and Teaching Conference, January 8th 2009 Linda Evans, School of Education

2. What do we mean by ‘advanced’ research and scholarship skills? • ‘advanced’ is a relative term • ‘skills that research students and, indeed, some academics often find difficult and challenging’ (Evans, 2002) • The basis of my evaluation: • my teaching/supervision experience • my experience of reviewing submitted bids and journal articles - and reading published ones! Evans, L. (2002)Reflective Practice in Educational Research: developing advanced skills, London, Continuum

3. What specific advanced skills? • Categorisation/classification • Conceptual clarity and definitional precision • Formulating theory/theoretical perspectives and propositions • theoretical models • Critical analysis • We ask for it, but seldom teach it.

4. A MA student’s attempt at critical analysis: an uncorrected extract from a draft essay Evan’s comment that ‘amongst teachers who respected it and responded well to it, ‘extended’ professionality in headteachers was an effective motivator.’ (Evans, 1998, p130) struck a particular chord with me because I am experiencing a greater sense of motivation since changing schools in September. My new headteacher constantly encourages ongoing reflection on individual practice with a view to ensuring that each child reaches their potential. I am experiencing a much greater sense of motivation as her professionality is very much in tune with my own. Although my last headteacher at my previous school was more of an extended professional than his predecessor, he was unable to filter out his extended professionality to the rest of the school due to his poor delegation skills: he wasted valuable time dealing with things others could easily have managed. At my new school, from the leadership team to the heads of department, the headteacher’s professionality is clear. As a consequence the extended professionals at my new school are more motivated than those with a similar degree of professionality at my previous school.

5. ‘Teaching’ critical analysis:the rationale • Students generally have little or no conception of what is meant by ‘critical analysis’. • They often underestimate the depth and degree of originality that we are, ideally, hoping to see in their work. • They consider themselves to be incorporating critical analysis when, in fact, they are merely offering superficial comments or observations. • They usually are able to identify critical analysis when shown work that includes examples of it. • They naturally find it extremely challenging and difficult to do. • They derive a tremendous sense of achievement from having managed to do it well.

6. What do we mean by ‘critical analysis’? • questioning what you read in books and articles • finding flaws in what you read in books and articles • identifying contradictions in arguments and/or theoretical perspectives • thinking of ways in which arguments and/or theoretical perspectives may be enhanced or developed further • explaining intelligently why you approve of, or support, a particular argument or theoretical perspective • explaining intelligently why you disapprove of, or have reservations about, a particular argument or theoretical perspective • originality of thought – your own ideas • developing sound, rational argument and/or discussion

7. A teaching tool: identifying examples of critical analysis • On a one-to-one basis, using the student’s own draft essays as a resource. In group teaching: • Using extracts from past students’ essays • Using extracts from journal articles • Using extracts from my own work, to allow me to explain my thinking and thought processes.

8. The Literature Review Purpose of the literature review: • to provide background and contextual information against which to set your argument • to fuel your argument • The purpose is not to show the tutor how much you’ve read!

9. Reviewing the Material • The literature is a tool. • Use it to support your argument. • Use it to provide evidence of perspectives that you can argue against. • Use conflicting views and contradictions to develop discussion. • Draw on a reasonably wide range of authors and texts. • Paraphrased references are as good as – and often better than – quotes.

10. Using the Literature as a Tool Do not simply present information found in the literature. • Comment on it thoughtfully and intelligently. • Indicate whether you agree with it: • to what extent; • why; • with what reservations or qualifications. • Argue against it. • Support it – but give your reasons for doing so. • Suggest modifications to it - but give your reasons for doing so. • Pick holes in it. • Compare it with your personal experience. • Compare it with conflicting views/evidence/information. • Analyse it critically.

11. Riddell, S. (1989) Exploiting the Exploited? The Ethics of Feminist Educational Research, in R.G Burgess (ed.) The Ethics of Educational Research, Lewes, Falmer, pp. 77-99 After talking to one mother and father for nearly an hour and a half, I was trying to bring the interview to a close with what I thought was a fairly innocuous question. I asked: ‘Are you generally happy with the education Stephen’s getting?’ Mr Gammage, a policeman, replied: ‘Well apart from this bloody irresponsible strike that’s been going on…’ Instead of bringing the interview to a calm conclusion, this resulted in a half hour’s denunciation of the entire teaching profession. Analyzing how I responded to male attempts to establish power in the interview, I found that I generally backed off from conflict, and often found it difficult to probe for more detailed answers when what they said was very brief. Although I did not probe the women’s responses either, they certainly offered me a far more intimate view of themselves. Of the three fathers whom I interviewed by themselves, one blocked my questions completely and gave yes/no responses to everything, and another answered very briefly. The headmasters whom I interviewed also provided proficient examples of school politics in operation, spending a very long time avoiding my questions.

12. From: Evans, L. (2002) Reflective Practice in Educational Research: developing advanced skills, London, Continuum: (Chapter 7: Telling it as it Is) As a woman – and a feminist – I feel able to question with impunity that may well be denied a man the researcher’s conclusion that these reported incidents demonstrate ‘male attempts to establish power in the interview’. My immediate response to reading this and similar passages in Riddell’s paper was to categorise them as evidence of partisanship and of the author’s having a feminist chip on her shoulder. Certainly, that the researcher’s experiences reflect male dominance – or attempts to dominate - is one possible interpretation, but it is by no means the only interpretation. Ascribing motives to others - particularly in the absence of their own accounts and explanations for them – is an extremely inexact science that, at best, may be described as conjecture and, at worst, as arrogant assumption. It is highly susceptible to misinterpretation that leads to the generation of spurious conclusions. We are treading on thin ice if we subsume our unsubstantiated interpretations of the motives behind the nature of their responses within our data. This involves stepping over the line - albeit a thin one - that separates the manner in which data are collected and provided and the data themselves; it blurs the distinction between process and product. Based on the evidence available in her paper, and bearing in mind that, like any paper, this is unable to provide us with the full picture, I formulated the impression that the author is too ready to ascribe sexist or stereotypical gender-related motives to her research subjects. Her male interviewees seem unable to put a foot right: if they are loquacious and responsive she evidently interprets their responses as domineering, and their reticence is evidently interpreted as attempts to sabotage the interview through non-cooperation. This reflects what I interpret as her blinkered viewpoint, and in a later section I examine ways in which she might have analysed and presented her data in a more balanced way.

13. From: Evans, L. (2002) Reflective Practice in Educational Research: developing advanced skills, London, Continuum: (Chapter 7: Telling it as it Is) Riddell’s analysis contrasts sharply with Luttrell’s (2000, pp.513-514) description of how what was effectively the removal of her blinkers allowed her to see and re-analyse her research data in a new light: Identifying my reluctance to deal with strong emotions and mixed feelings about mothers – what could be called counter-transference in my fieldwork relationships – marked a major breakthrough in my research process. I again returned to all the interview material, and through this recursive process I discovered a range of maternal images and mixed feelings that the women had expressed, but that I had minimized in my analysis of the links between structure, culture and agency.

14. From: Evans, L. (2002) Reflective Practice in Educational Research: developing advanced skills, London, Continuum: (Chapter 7: Telling it as it Is) Riddell’s interpretation of her male interviewees’ behaviour would be justifiable if she had applied appropriate controls to, or at least consideration of, other variables together with incorporation of consideration of evidence that supported or conflicted with her interpretation before systematically assessing rival explanations. She presents no evidence of having adopted such a rigorous approach to her analysis.

15. From: Evans, L. (2002) Reflective Practice in Educational Research: developing advanced skills, London, Continuum: (Chapter 7: Telling it as it Is) Yet I have to acknowledge another side to the arguments I have presented within my criticism of Riddell’s analysis. In his outline explanation of consensus theory Bridges (1999, p. 606) warns: Philosophers have tended to be sceptical of consensus theory as an account of what it means for a belief to be true… Researchers ought perhaps to share especially in this scepticism of consensus, since it is, arguably, their particular function as intellectual citizens to challenge the easy and self-perpetuating consensus that society creates for itself. At the very least they have to observe that what any group of people believe or agree to be true may rest on, for example, unexamined tradition, the hegemony of a dominant class, the suppression or self-censorship of dissenting opinion or collective hysteria – all the things that intellectuals and researchers have traditionally been expected to subvert. Applying this consideration to my criticism of Riddell’s work, I am forced to accept the possibility that, in presenting my interpretation of her analysis as the reflection of a blinkered viewpoint, and my own as reflecting the more consensual form of truth, I may have fallen into the trap, to which Bridges refers, of perpetuating a consensus that fails to question - because it fails to see – the hegemony of one or more specific societal groups. In a sense, moreover, I seem to be trapped in a ‘Catch 22’-type situation; by contending that Riddell’s viewpoint is blinkered does not my own viewpoint then risk becoming so?

16. Morale, Job Satisfaction and Motivation:Research Questions • What factors influence morale? • What factors influence job satisfaction? • What factors influence motivation? • Activity: examine the raw research data presented as example quotes, in order to find answers to the above research questions.

17. Conceptual clarity and definitional precision • I do not tell students in advance that this is my teaching focus. • Students better appreciate the need for it and the potential difficulties created by its absence if they discover these for themselves. • The substantive focus of the teaching activity is, where possible, appropriate to the content of the module they are following.

18. Conceptual clarity • adds precision and rigour • facilitates construct validity • is a tool for achieving critical analysis

19. Construct validity • ‘There’s glory for you’, said Humpty Dumpty. • ‘I don’t know what you mean by “glory”,’ Alice said. • Humpty Dumpty smiled contemptuously. ‘Of course you don’t – till I tell you. I meant “there’s a nice knock-down argument for you”.’ • ‘But “glory” doesn’t mean “a nice knock-down argument”,’ Alice objected. • ‘When I use a word’, Humpty Dumpty said in a rather scornful tone, ‘it means just what I choose it to mean – neither more or less.’ (From: Through the looking Glass, by Lewis Carroll)

20. Construct validity • involves consensual acceptance and understanding of specific terms • is threatened when researchers and research subjects do not share the same interpretation and understanding of key constructs • is threatened when two parties are communicating at cross purposes • can impact upon the whole process of data collection, data analysis, and dissemination

21. Developing your capacity for pursuing construct validity The process has four elements: • identification of key concepts for clarification • conceptualisation and construct clarification • communicating and matching constructs • conceptual clarity and dissemination

22. Identification of key concepts for clarification • Conceptualisation is the process whereby we develop concepts. • It is the process of attaching meaning to words. • It is also the product of having attached meaning to words: • your conceptualisation of ‘milk’, for example, is the meaning of milk that you have formulated

23. Identification of key concepts for clarification • Identify what you consider to be the key concepts within your research. • Decide which of these need clarifying: • Which concepts are potentially ambiguous? • Which concepts are susceptible to being interpreted differently by different people? • The more nebulous the concept, the more likely it is to need clarifying. • Concepts of tangible objects are less susceptible to multiple interpretation than are intangible concepts. • a study of the causes of student disaffection

24. Conceptualisation & construct clarification • Examine your selected key concepts: • Decide how you interpret or define them. • What precisely do I mean by …? • It is not enough simply to ‘know’ what you mean by something – you must be able to communicate that meaning. • Formulate conceptual interpretations or definitions of your key concepts. • Aim to achieve definitional precision.

25. Achieving definitional precision What is a definition? • stipulative definitions • stipulate, in precise and unambiguous words, what you mean when you use a particular word • ostensive definitions • where a word is defined by pointing to the objects to which it exclusively refers • e.g. in defining “education” one might point to particular activities, or to what goes on in particular schools

26. Recognising stipulative definitions Which of these is a stipulative definition of a chair? • A chair is a piece of furniture intended for sitting on. • A chair is a seat. • A chair has four legs, a seat and a back and is sometimes padded. It is usually made of wood or plastic but other materials can be used. • A chair is intended to seat people. • A chair is a piece of furniture that people may sit on.

27. Recognising stipulative definitions • Stipulative definitions should explain precisely and unambiguously what something is. • They should not simply describe something. • A stipulative definition should be exclusive –applicable only to what it is that is being defined. • A stipulative definition should encompass explanation of all of the features that are integral to what is being defined. • It should incorporate consideration of the necessary conditions for whatever is being defined.

28. Stipulative definition of a chair • A chair is a piece of furniture intended for sitting on. • A chair is a seat. • A chair has four legs, a seat and a back and is sometimes padded. It is usually made of wood or plastic but other materials can be used. • A chair is intended to seat people. • A chair is a piece of furniture that people may sit on. • A chair is a piece of furniture with a back and is intended to seat one person at a time.

29. Formulating Stipulative Definitions • You do not necessarily need to formulate your own definition of something. • There’s no point in re-inventing the wheel. • Examine the range of definitions available in the literature. • Adopting an existing definition is perfectly acceptable. • But you should be able to explain why you choose to adopt it – what you like about it • Adapting an existing definition is perfectly acceptable. • But you should be able to explain why you choose to adapt it – what you like and don’t like about it; why it doesn’t quite meet your needs without being altered. • Knowing how to formulate a definition from scratch will help you to adapt an existing one.

30. Formulating Stipulative Definitions • A stipulative definition should encompass explanation of all of the features that are integral to what is being defined. • Start by identifying the necessary conditions for or properties of whatever it is that you wish to define. • First list as many conditions or properties as you consider relevant. • Then narrow down the list to necessary conditions or properties.

31. What do we mean by ‘necessary conditions’? • Necessary conditions are all those components that constitute what the ‘thing’ is. • They are components that, if missing, would change the nature of the ‘thing’.

32. Listing necessary conditions or properties:the example of a chair • Ask yourself, for something to be called a chair, what does it need to have? • Initially list as many things as spring to mind. • In the case of every thing on your list ask yourself, do all chairs have this? • Cross off anything that prompts you to answer, ‘no’. • Try to think of an example of a chair that has something that you haven’t listed. • Ask yourself, does it need to have this in order to be called a chair? • If the answer is ‘yes’ add that thing to your list.

33. Properties of a chair/necessary conditions for something to be a chair • Initial list might include: • has a seat • is a piece of furniture • has a padded seat • has four legs • is meant for sitting on

34. Refining the list • In the case of everything on the list ask yourself, do all chairs have this? • has a seat • is a piece of furniture • has a padded seat • has four legs • is meant for sitting on

35. Refining the list • Reduce the list to include only the properties that apply to all chairs: • has a seat • is a piece of furniture • is meant for sitting on • Try to think of an example of a chair that has something that you haven’t listed. • Ensure that the property that you think of is common to all chairs. • Add it to your list.

36. Necessary conditions or properties: some examples • Necessary conditions for something to be a chair • categorised as furniture • designed as a seat • intended to seat one person at a time • a back • a seat part • legs or a pedestal • Incorporate everything in the list into a stipulative definition.

37. Punishment - necessary conditions Flew (philosopher): Punishment must be: • unpleasant • for an offence or a supposed offence • of an offender or of a supposed offender • carried out intentionally • carried out by those with authority to do so. Formulate a stipulative definition of punishment.

38. Stipulative definition of punishment • Punishment is an action directed at an offender or supposed offender by one or more persons with the authority to carry out the action, with the deliberate intention of causing unpleasantness to the offender or supposed offender, in response to an offence or supposed offence committed by the targeted offender or supposed offender. • Punishment is a deliberate action carried out by someone in authority to carry it out on, and causing unpleasantness to, an offender or a supposed offender for an offence or a supposed offence.

39. Formulating stipulative definitions • List the necessary conditions or properties. • Decide what category of phenomena is the concept that you are defining • It is a/the … • Arrange all of the necessary conditions or properties into a statement that defines the concept, beginning with reference to the category of phenomena. • Pay particular attention to the meaning of every single word. • Ensure that the phrases within the definition convey the precise sense that you intend them to convey. • Test your definition by actively seeking examples of the concept being described that are not covered by the definition. • Modify your definition until you can find no more examples of the concept being described that are not covered by the definition.

40. Achieving conceptual clarity • The key point is that you must communicate to others what you mean when you refer to a certain term that is susceptible to misinterpretation. • You may do this be presenting a definition or simply by presenting your interpretation of a term. • Conceptual interpretations are as acceptable as stipulative definitions. • Stipulative definitions are intellectually challenging to formulate • rewarding • sense of achievement • Stipulative definitions may constitute a form of critical analysis and may earn you extra marks … • … but only if you formulate (or adapt) them effectively.

41. Definitions and Conceptual Interpretations What’s the difference between the two? • Stipulative definitions: • stipulate precisely what something is; • are clear and unambiguous; • relate exclusively to what is being defined. • Conceptual interpretations: • typically describe, rather than define; • may includeinformation that supplements what would beincluded in a stipulative definition; • tell us what something looks like, rather than what it is; • are what many people would categorise as ostensive definitions.

42. The difference between stipulative definitions and conceptual interpretations Morale • morale is: a state of mind encompassing all of the feelings determined by the individual’s anticipation of the extent of satisfaction of those needs which s/he perceives as significantly affecting his/her total work situation. • morale has levels; it ranges from high to low and these levels may fluctuate as circumstances and situations affect them. Morale in a work context is about how people feel about their work. Morale is future-oriented, so it relates to people’s feeling about their work in the future – whether they anticipate being happy or unhappy at work… (from: Evans, L. (2002) Reflective Practice in Educational Research, London, Continum, p. 66)

43. The difference between stipulative definitions and conceptual interpretations Definition or interpretation? Professional development consists of all natural learning experiences and those conscious and planned activities which are intended to be of direct or indirect benefit to the individual, group or school and which contribute, through these, to the quality of education in the classroom. It is the process by which, alone and with others, teachers review, renew and extend their commitment as change agents to the moral purposes of teaching; and by which they acquire and develop critically the knowledge, skills, planning and practice with children, young people and colleagues through each phase of their teaching lives. From: Day, C. (1999) Developing Teachers: the challenges of lifelong learning, London, Falmer, p. 4)

44. Definition or Interpretation? Conceptual interpretation: Professional development consists of all natural learning experiences and those conscious and planned activities which are intended to be of direct or indirect benefit to the individual, group or school and which contribute, through these, to the quality of education in the classroom. Stipulative definition: It is the process by which, alone and with others, teachers review, renew and extend their commitment as change agents to the moral purposes of teaching; and by which they acquire and develop critically the knowledge, skills, planning and practice with children, young people and colleagues through each phase of their teaching lives.

45. How to criticise a definition formulated by someone else • List what you consider to be the necessary conditions for, or properties of, the thing being defined. • Examine the definition against this list. • Does the definition cover everything? • if not, is it the definition or your list that’s deficient ? • Does the definition incorporate reference to conditions or properties that are not on your list? • if so, is it the definition or your list that’s deficient ? • Formulate a critique. • Explain why you consider the definition to be deficient.

46. Activity • Formulate a stipulative definition of a concept that is key to your work. • Don’t tell anyone what concept you are defining. • In small groups, each person is to read out her/his definition, but without the name of the concept being defined. • The rest of the group must work out what concept is being defined.

47. An alternative activity • Prepare two sets of cards: • with the name of a concept on the card • with the stipulative definition of the concept on the card – but not the name of the concept • Students to work in groups, sorting the cards into pairs, matching the concepts with their definitions.

48. Categorisation and coding • The process of grouping, sorting and classifying: of formulating and identifying categories and placing phenomena into these categories. • Codes categorise information at different levels. • Categorisation is probably the most difficult research/scholarship skill • An invaluable analytical tool • allows you to analyse your own findings • allows you to criticise other people’s analyses and categorisations • Useful for essay/dissertation writing: • subheadings • The difficulty lies in formulating categories that: • don’t overlap • are on the same coding ‘level’ (as each other).

49. From: Evans, L. (2002) Reflective Practice in Educational Research: developing advanced skills, London, Continuum: (Chapter 8: Tools of the Trade: developing coding and categorisation skills) What makes effective categorisation so difficult are the two requirements that categories be both exhaustive and exclusive. Gillham (2000a, p. 60) explains: ‘(A) requirement for the derivation of categories is that they should be exhaustive. But another requirement is that they should be exclusive, i.e. that the kind of statements that go into one category clearly belong there and couldn’t really go anywhere else’.

50. well-organised trustworthiness and honesty approachable positive attitudes interpersonal skills strong leadership qualities abilities to make decisions innovative knowledge of the staff and children knowledge of teaching and learning knowledge of the school vision supportive high expectations encouragement of team work a leading professional lead by setting an example development of the staff An example from a student’s work characteristics of effective school leadership: