Sunday 8 December 2013

A school, finally (maybe)

Good news: I think I finally found a school that could be my research site, though I have yet to get the nod from my supervisor.

Bad news: Apparently getting permission from MOE and ethics clearance from Lancaster could take 3 months or more. My aim is to have my confirmation panel while I'm in Lancaster in July, but I might not be able to collect enough data for that. Which means I'll have to do it via Skype (confirmation is supposed to happen within 36 months of starting part-time study, and I don't really want to fly down again in December). Not ideal. Ugh.

Regardless, I hope this school meets with L's approval, and I can get the paperwork started before 2014 begins.

In the meantime, I'm finishing up preparations for my new ePortfolio course by working on my own eportfolios (a general career one and a PhD one) and also the course Google Site. Reflecting on the courses I've taught since leaving NIE as a full-timer, it seems that only this one will be somewhat related to my research interests. I will probably get the chance to run assessment workshops for the school I end up collaborating with, but I wonder if I'll ever get to actually teach a course on it? Theoretically this is a good niche to specialise in (everyone says so), but I remain doubtful there's a market for it in Singapore. Will I find my alt-ac career here? Another 'why am I doing this again' moment.

Friday 29 November 2013

Still alive (and sane)

Not sure how I did it, but I miraculously survived the semester. I survived 16.5h of teaching a week and an insane amount of marking, a course development project, and 3 assignment deadlines (>12.5k words in total). It was very hard going, but I think it helped me to 'level up' in terms of productivity. Certainly I had to exercise very strong self-discipline because missing one deadline was potentially disastrous for everything else. I'm a born procrastinator so it's a particular achievement not to have missed any (important) deadlines. It looks like my workload next semester could be similar or worse, but at least without the assignment deadlines. 

It does make me worry that without externally imposed deadlines, I may not be very productive with regard to my PhD. Many of my assignments helped me to get my reading and thinking done, for which I'm quite grateful. With this latest batch of essays, my ideas have achieved a new clarity, though I wouldn't dare say they've finished 'baking' (when do they ever?)

I'm on a lot firmer ground with regard to methodology now. If nothing else, I think the methodology modules have been very good for me. Methodology is interesting and important to me, and I want my study to be solid in that aspect. I also think I've now got a deeper understanding of assessment issues and digital literacies (which was my last paper). With practice theory, I've also found I think the 'missing link' between assessment practices and literacy practices. It was very fuzzy and unconvincing before. When I jumped on practice theory (Schatzki, Reckwitz and later Shove, Pantzar & Watson), I thought that it was a better fit for talking about assessment practices though maybe not so for literacy practices, since that already has a strong tradition going. Lo and behold, I see Reckwitz popping up in Lankshear & Knobel's New Literacies (though new to this 3rd edition I think). Which just goes to show my knowledge of the lit isn't what it should be yet, though I am making new connections as I progress. 

As much as I'd learnt from writing my assignments, finishing my coursework does allow me to read more specifically for my PhD rather than to fulfil assignment requirements. With my last 2 5k word papers, I learnt how to use Citavi. It is exactly what I wanted Mendeley et al. to be, in that it not only manages my references, but also helps me to organise, make sense of and synthesise literature. I could have done with Citavi with previous lit heavy papers, like the Qual one on Rasch.

Now that I have some free time, I have some things to accomplish before the new semester starts:
  1. Catch up on reading, not only the lit that's directly related to my PhD, but also the peripheral stuff that could help with my writing and career. Practice theory is at the top of my lit list.
  2. Focus on getting access to a school. This is actually quite urgent but I've put it on the backburner this semester and am trying hard not to panic now.
  3. Develop my own ePortfolio as a means of reflecting on my progress (which this blog is also meant to do). I think it would help me personally as well as model ePortfolio keeping to my Reflection & ePortfolio students in the new semester.
I'm going to try to blog more frequently, if only to update on my progress.

Friday 4 October 2013

Pragmatic epistemology, possibly

Bored and unmotivated this morning after sleeping in. (Yesterday was my killer 8h Thursday, so I felt justified.) Things to do today include marking (I spread it out as thinly as I can while still meeting deadlines) and working on my Qual paper on case study. 

While thinking of ways to procrastinate, I suddenly remembered this blog. I'm sure it only counts as semi-procrastination! 

So while working on my Classroom Research paper I recalled L's suggestion of Pragmatism, because I was (still am) troubled by this theoretical framework thing. I'm reading up on Pragmatism, after watching a series of YouTube videos (yay YT!) on the basics of the philosophy and deciding that it might be a good fit. It doesn't seem to conflict with the approach I described in my previous post. Using Crotty's (1998) framework for theoretical frameworks (LOL), maybe I could put it this way:

Epistemology: Pragmatism
Theoretical perspective: Social practice
Methodology: Ethnographic case study
Methods: Participant observation, interviews, focus group discussions, document analysis, questionnaire (maybe)

But Pragmatism is of course more complex than as few YT videos can explain. I have to delve into the scary rabbit hole of philosophy, while I wrangle that case study paper.

Friday 27 September 2013

Post-post-pre-confirmation thoughts

I really should have blogged after the July residential since that was a pretty major milestone -- my pre-confirmation panel. The panel threw me off-balance (to say the least) and I was pretty lost for at least a few days but by the time I left Lancaster I did have some sense of the new direction I was going to move in. 

While in London I just wanted to have fun (which I did) but from the moment I landed in Singapore it has been work, work, work. I didn't work much last semester because semesters here clash with the Jan residential, so I've been trying to make up for it this semester. As a result I probably took on too many jobs -- I'm currently teaching at NIE, NUS, SIM University and NAFA, in addition to some course development work and miscellaneous workshops. This has also made it hard for me to chase up schools for access; I'm hoping to do that at the end of the year which is school vacation in Singapore.

I worked out an overly ambitious schedule, according to which I should have started the 2nd of the 3 assignments due this semester. Which I haven't of course. I count myself lucky to be able to finish the first (and the shortest at 2.5k words) before the 30 September deadline. In the course of writing this paper, I've had some time to think about my research. In my post-panel report, I wrote that I was looking into practice theory instead, specifically Reckwitz's (2002). This is something which I found and put aside previously, and I think now it could work for me. I'm not sure, especially after the panel, if the literacies practices paradigm really works for me, and Lave and Wenger's CoP isn't quite it either. 

While at the Lancaster campus bookshop, I came across Shove, Pantzar and Watson’s (2012) book "The Dynamics of Social Practice: Everyday life and how it changes" and found it very interesting. Prof Elizabeth Shove is from the Sociology department at Lancaster, and the authors have adapted Reckwitz's framework, which is more complex, into something more streamlined and usable. They analyse practices based on the three elements of materials, competences and meanings. I like that they approach this with the aim of changing practices, not just describing them.

My methodological approach as it stands now is primarily case study, with an ethnographic perspective. I've read up further on ethnography and participant observation for the paper, and feel better prepared to explain how I'm taking up the ethnographic perspective without actually doing ethnography. In exploring ethnography, I discovered that there is a strong mixed methods tradition, which I wasn't aware of before. I guess this further explains the considerable overlap between case study and ethnography. This is useful to know if I end up administering a questionnaire. 

I know many people find coursework a drag, but I'm really grateful for it, especially at this point. I think a solid methodological foundation is important for research. Not to say that I believe that epistemologies and paradigms are unshakeable; I tried that and I think that view doesn't really work for language assessment. Maybe pragmatism really is my epistemology. 

I also think I've got a more precise notion of the research gap I'm trying to fill now. Not only is there insufficient research into the classroom-based, social and digital aspects of language assessment, but what there is currently also tends to focus on assessment tools and frameworks, which on their own don't seem to have much of an impact on assessment reform. So I'm arguing that this is why there's a need to look at things from a social practice viewpoint, which has been done in general ed assessment but not in language assessment.

This feels like a move forward for me, but that's what I feel every time there's a change in my thinking, so it might just be an illusion :P

Friday 12 April 2013

To-do list, as of 12 April 2013


  1. Flesh out HK preso for pre-confirmation: 'A few days prior to the meeting of the panel, students should provide the supervisor and other member of the panel with a written document briefly outlining their work and progress in the last year (including planned Research Questions, data, data collection and data analysis) and pointing out any issues which they consider have impeded their progress (3-4 pages).' 
  2. Finish reading Understanding digital literacies: A practical introduction.
  3. Sort out literature (spreadsheet, Zotero) -- what's the best way?
  4. Find at least 2 schools -- follow up on possible candidates.
  5. Design draft questionnaire for students (for MOE application).
  6. Work on ethics and permissions docs.

Misc stuff to follow up, as of 12 April 2013

From CR forum exchange with D:
Could the conventional concept of a construct could also be challenged if it assumes a theoretical perspective of 'practice' rather than 'skill'?

Case studies with ethnographic perspective? According to U on DL forum: 'Call it taking an ethnographic perspective - in the sense that you seek the emic perspective, look at things in context/as situated, so you share something with ethnography, but doing it in a more condensed and probably less 'deep' way. Lots of people these days speak about their research being ethnographic, but that doesn't mean it is ethnography in the sense anthropologists define it. Often what they do is qualitative research or case study research.'

Or multi-sited ethnography? U: 'The issue is that the research questions might lead you into different directions, requiring you to look in different places, hence multi-sited. These directions might not be physical or virtual places, but rather issues or questions - hence more abstract.'

HK presentation (Feb 2013)

On the last lesson of the Classroom Language Assessment module, I presented my latest research plan to the class. L says it can be developed into research plan for pre-upgrade in July.


Notes:

BACKGROUND 
Hot topics in educational research but not so much in language assessment
Issues are LINKED
Issues central to lifelong learning (self-assessment, SDL, learner autonomy) and to preparing us for this age of exponential changes

RATIONALE 
Increasing importance of ICT in teaching & learning
Singapore MOE's 3rd Masterplan for ICT in Education:
'...enrich and transform the learning environments of our students and equip them with the critical competencies and dispositions to succeed in a knowledge economy.'
'Strengthening integration of ICT into Curriculum, Assessment and Pedagogy'

Future Schools testbeds for innovation

New syllabus acknowledges multimodal literacies, e.g. Writing and Representing, not just writing.
Doc mentions 'visual elements', 'multimodal texts', 'experiment with the use of sound, imagery and language', 'use a range of technologies'.

Importance of 21st C skills, e.g. various digital literacies (c.f. critical competencies & dispositions)
Singapore is a founding member of ATC21S (Assessment & Teaching of 21st C Skills)

How it's being assessed and WHAT is being assessed (i.e. the construct) important

METHODOLOGY 
Literacies & assessment as social situated practices - must be considered along other social & cultural practices
Literacies (multiple) not as skills but as activities
Skills view does not capture the diversity and complexity of people’s reading and writing activities in everyday life
Think about all the kinds of writing you do everyday, esp tech-mediated

Assessment practices also not neutral, but also shape and are shaped by society & culture

"A ‘practice’ (Praktik) is a routinized type of behaviour which consists of several elements, interconnected to one other: forms of bodily activities, forms of mental activities, ‘things’ and their use, a background know-ledge in the form of understanding, know-how, states of emotion and motivational knowledge." Reckwitz (2002)

Notes from Feb 2013 HK meetings

New RQs foregrounding e-assessment:

RQ1. What role does ICT play in the assessment practices of EL teachers in Singapore secondary schools?
RQ2. How do they balance formative and summative e-assessment in the classroom?
RQ3. What are the factors that enable and inhibit their use of formative e-assessment?

To be considered:
  • Refocus on digital writing instead (because e-assessment difficult to rationalise for research).
  • Rationalise digital writing research from literacy practices perspective, 21st c edtech development in last 5 years
  • Literacies and assessment as social practices, not skills
  • Challenges to construct of writing.
  • Instead of including formative/summative in RQ, refer to them simply as practices.
  • Grounded Theory -- Lincoln and Guba (1984)?
  • Theory of Planned Behaviour TPB -- Azjen (social psychologist)
  • Innovation Theory as an orientating theory (?) to start with
  • Paul Gruba on e-assessment and blended learning

Notes from Jan 2013 Residential

My RQs at this date:

RQ1. What are the assessment practices of EL teachers in Singapore secondary schools actively engaged in integrating ICT in teaching and learning?

RQ2. How do they balance formative and summative assessment in the classroom?

RQ3. What are the factors that enable and inhabit their use of formative assessment?

RQ4. What role does ICT play in their (formative) assessment practices?

New focus on e-assessment (mostly writing) requires definition:

  • MOE?
  • Research literature?
  • What about e-portfolios?

Consider infrastructure and resources as factors.

Consider literature on curriculum reform, particularly those studies which are concerned with situational or environmental analysis (essentially, those studies that look at factors which enable and inhibit). Also innovation theory in ELT (see Markee's book).

Notes from July 2012 Residential

July 2012 Residential

Meeting 1

Explore the ethnographic approach in language assessment (e.g. Hill & McNamara, 2012)
Consider what sort of participant schools would suit
Read up on 
  • Case studies (see Yin)
  • Situated Literacies ed. Barton
  • Assessment literacy
  • Teachers' understanding of formative assessment
  • Teacher cognition (Borg, 2003)
  • Definitions of formative assessment

Meeting 2

Fulcher's focus on PRINCIPLES is important and would apply to informal assessment
Should decide what to foreground and background in study
Could use formative vs summative CBA as a lens through which to examine factors which inhibit or enable formative assessment (e.g. learner autonomy, exam-driven culture, assessment literacy)
Follow up on:
  • Classroom-based Language Assessment 
  • Kathryn Hill's research
  • 2013 Language Testing special issue on Assessment Literacy

Meeting 3

Tentative RQs:
RQ1. What are the assessment practices of EL teachers in Singapore?
RQ2. How do they balance formative and summative assessment in the classroom?
RQ3. What are the factors that enable and inhibit their use of formative assessment?
Should not bring factors I'm interested into the RQs (e.g. washback), but choose one as a focus RQ, e.g. assessment literacy, because it can make a considerable contribution to PD?

Framework for CBA: TIERNEY 2006Changing practices: influences in classroom assessment
Model missing resources and materials (ready made)
Note mediating sources - variables

Follow up:
  • Black and Wiliam on formative assessment
  • Van lier
  • Focus on formative - precise end point (narrow to writing? Digital writing?) in order to include all influences
  • Use model to analyse data?
  • Ethnographic methods for classroom research

My original proposal

B. Your Research

B1. What is your proposed area of research?
Language testing. Specifically the testing of non-traditional digital writing skills, as defined by researchers in the field of multiliteracies (New London Group, 1996). The context is secondary school English language education in Singapore.

B2. Within this area, what would you like to investigate? Please include at least one possible research question here.
  1. How can a given construct of non-traditional digital writing skills be operationalised in terms of assessment criteria and rating scales? 
  2. In the operational test, to what extent do the individual criteria distinguish between test-takers who score different grades? 
  3. Are raters experienced in the assessment of traditional writing skills able to use the new criteria consistently and reliably? 
  4. Does the use of such a test result in desired washback; that is, improvements in the instruction of non-traditional writing skills? 

B3. What are your reasons for choosing this proposed research project? (These might be theoretical, professional or both.)
The nature of literacy has changed with the advent of affordable computing devices and higher internet speeds. These new forms of literacy are fast becoming essential in modern society. However, mainstream education continues to lag behind research in this area. This is despite the strong push by education authorities in many countries, such as Singapore, to integrate ICT into their literacy programmes.

In Singapore’s assessment-driven education system, this problem could be attributed to the traditional assessment practices still dominant here. Crucially, the high-stakes Singapore-Cambridge General Certificate of Education (Ordinary Level) Examination for English Language, taken by secondary school students in their final year, remain very much a traditional language test. Therefore, assessment practices would have to take the lead in changing the status quo.

However, research in the assessment of multiliteracies has been limited to date. My research aims to fill the gap in a way that is immediately applicable in the context of Singapore schools. This is in line with Singapore’s role as one of the five founding countries of the Assessment and Teaching of 21st Century Skills project (ATC21S).

B4. Do you have ideas about the sort of data you might need for your research?
  1. Test-takers’ scripts from a new test of non-traditional digital writing. 
  2. Double-rated scripts by teacher-raters. 
  3. Think-aloud protocol (concurrent or retrospective) data from teacher-raters. 

B5. How might you collect this data? Do you already have access to it? (N.B. Do not list here books you will read or have read.)
The participants will be teachers and pupils from a secondary school in Singapore who are also participating in a study that aims to identify similarities between the skills involved in traditional pen-and-paper writing and non-traditional digital writing.

Why this blog

For a long time now I've considered putting down thoughts, etc. related to my PhD studies in some way, in a kind of journal. As usual I've procrastinated and procrastinated.

Let's hope I can keep this up!