Conducting SoTL of an online art history course: using discourse analysis of discussion boards
By AHTR editor Aly Meloche
For those of us who are just beginning to teach online, the concept of conducting scholarship of teaching and learning in addition to all of the other new responsibilities must sound about as much fun as running a virtual meeting while trying to homeschool new math. But these are important times for art history pedagogy, and teaching and learning has changed rapidly. Many of us feel like we’re thrashing in an uncontrollable tempest, seeking out the edges of our own abilities to not just cope, but to learn. And in this learning there are lessons — lessons that when shared could provide a timely lifeline that another is seeking. But again, who has time for that?
The purpose of this post is to introduce one type of SoTL that is possible without necessarily adding a lot of additional work on the part of the instructor/researcher. Discourse analysis is a qualitative research method that allows researchers to track many different elements such as identity exploration, agentic action, social development, and content mastery and (here’s the good part!) data can be collected from online courses using discussion boards.
Many of us already use discussion boards, sometimes one for every week. In my own online art history class I have used discussion boards to track and assess student’s understanding of certain “big picture” concepts that I hope they take away from our content material, such as textbook readings, recorded lectures, or other things. Here’s a sample of two prompts that I use in my 16th century in Northern Europe module:
Example 1
- What was the impact of the Reformation on art in Northern Europe? Discuss artistic style, patronage, subject matter, and/or the overall effect on the art world.
- How does the art produced in 16th century Northern Europe compare or contrast to what was going on in Italy? Include specific examples.
In these examples, I’m looking to see certain themes that I know were presented in the content material parroted back to me. But I also look for areas where students come to their own conclusions. Part of the rubric offers points for originality, which discourages students from repeating a concept that was mentioned by one of their peers. Students are also encouraged to form a dialogue by commenting in a way that builds off of the conclusions of their peers.
Sometimes I use discussion boards as a more open forum in which students are encouraged to reflect on something that stood out to them for a certain reason.
Example 2:
- This week we covered many different artists and styles and topics from the history of Mid to Late 19th Century art in Europe and the Americas, but we didn’t cover everything. What is something that you wish had been part of our course materials? Or, if you are not very familiar with the Late 19th century, what is something that we touched on briefly that you would enjoy further discussing?
In these discussion boards there is a greater emphasis placed on the comments that students leave on their peers and the dialogue that they create.
I’m sharing these two examples as typical types of discussion boards. In example 1 I am looking for content mastery, in example 2 there is less of an emphasis on “getting it right” and more on facilitating an online community. I’m also looking to honor students’ prior knowledge, which allows them to bridge what they know to what they are learning.[1] Both of these types of discussion boards can be collected as part of a SoTL project.
Whether you are an experienced online instructor, or you’ve just recently made the switch out of necessity, you’re probably using discussion boards similar to one or both of the examples I shared. In fact, yours are probably much better!
Now it’s time for the even better news: Discussion board data is easy to collect because it is already digital (no scanning or transcribing necessary) and can be exported or copy/pasted into a word processing document.
And the better-er news? Most SoTL projects that involve the collection of student data which they are already doing for class are likely IRB exempt [2]. This means that the committee reviewing your research plan should push it through pretty quickly and, once you are approved, you will likely only need verbal/written consent. No lengthy consent forms, YAY! As part of your IRB description you should make the following things clear: 1. the data being collected is part of a normal class assignment, 2. students are able to have their data not included, if they wish, without grade penalization, 3. you will protect student’s identity using pseudonyms and redacting identifiable information in any publications.
If that didn’t scare you away, let’s do a very brief introduction to three of the types of SoTL you can conduct using online discussion boards.
Assessment of content knowledge:
Let’s imagine you have tried a new way to deliver course content, like a new video lecture or OER, and you want to see if students are still learning the material adequately. Or maybe you want to know what part of the material stood out to them. You can assess the efficacy of new course content by highlighting instances that this content is referenced and qualifying the level of understanding that the student demonstrates. Here is a sample codebook that specifically assesses how the students refer to course material in an online discussion post:
| Qualifier | Insufficient | Developing | Adequate | Expert |
| Description | Discussion presents mainly personal opinions. No connections are made to relevant course material (insert specific material here). | Discussion post presents minimal connections to relevant course material (insert specific material here). Connections that are made are somewhat unclear at times and/or lacking specific reference (pg #, time, etc). | Discussion post presents some direct connections to relevant course materials (insert specific material here). Many of the connections are clearly stated with occasional specific reference. | Discussion post is full of direct connections to relevant course material (insert specific material here). Course material is clearly stated and documented with specific references. |
| Sample student text: | Insert examples of insufficient text here |
Data like this can even be quantified and tracked over time. For example, if you introduce a new content material, you might ask if the quality of references to course material improves by the end of the semester, once students get used to a new way of learning.
Social network analysis:
Let’s say one of your stated goals for your online course is to develop a community of learners. A worthy and achievable goal for those of us who wish to leverage the way that social interactions leads to strong cognitive impressions.[3] Social network analysis is the study of interpersonal interactions and network ties. It has recently been utilized to understand online interactions to understand peer to peer interactions and the development of mentoring relationships.[4] This can be conducted in discussion boards of either type (example) listed above, but evidence of social network connection may be more organic in example 2. Social network analysis can be used to answer research questions such as:
- Who is reaching out to whom with discussion responses that answer questions or offer advice?
- To whom do students turn for support?
- To what extent is information travelling to all the participants?
The following is an example of what a network analysis from an online forum can look like. This one uses color to designate type of student, size to designate participation frequency, and arrows to show unidirectional or bi-directional communication.
Viewing the networks like this allow us to make conclusions about the way that students are interacting. There are 10 students in this class, 4 of which are art/design students. Visual analysis of this diagram concludes that, for the most part, the non art/design students are commenting on only one person, typically another non art/design major. There are numerous free software that allow faculty to upload data from their discussion boards and create visualizations.[5]
Critical discourse analysis:
I have experience using Gee’s critical discourse analysis as a framework to inform my research questions and coding[6] and I can recommend it for use in online discussion boards in art history courses. His framework presents 7 easy to understand language building elements that you can code and analyze for themes in your discussion board data.
| Building task | Question(s) | Sample student text |
| Significance | “How is this piece of language being used to make certain things significant or not and in what ways?” (Gee, p. 17). | Insert examples of significance here |
| Practices | “What practice (activity) or practice(s) (activities) is this piece of language being used to enact?” (p. 18). | |
| Identities | “What identity or identities is this piece of language being used to enact…? What identity or identities is this piece of language attributing to others and how does this help the speaker or writer enact his or her own identity?” (p. 18). | |
| Relationships | “What sort of relationship or relationships is this piece of language seeking to enact with others?” (p. 19). | |
| Politics | “What perspective on social goods is this piece of language communicating?” (p. 19). | |
| Connections | “How does this piece of language connect or disconnect things?” (p. 19). | |
| Sign systems and knowledge | “How does this piece of language privilege or disprivilege specific sign systems or different ways of knowing and believing or claims to knowledge and belief?” (p. 20). |
What is lovely is that you can select to use only a few (or even just one) of his language building tasks in your research, depending on your research question. Say, for example, you want to explore student development of identities as art historians. You can use Gee’s question to begin looking for examples of language that students use to assert their identity, and also examples in which they establish their identity by contrasting with identities they reject. Here is an example of a student establishing a new identity after participating in a class activity and reflecting on the experience.
One could conclude that the pedagogical activity, combined with self-reflection, led to a new realization of the self. This would be a strong case (one student) but maybe this trend repeats for several participants, cool!
Summary of key points:
- You can conduct SoTL using data from the discussion boards that you’re already using in your online art history course.
- IRB review will probably determine that your research is exempt, so long as you are conscientious of your students’ data privacy.
- A sample of the types of research that you can conduct using online discussion boards:
- Demonstration of mastery of the course content
- Building of a social network
- A deep dive into the way that students form and express opinions, identity, agency.
[1] This is a teaching and learning concept that Lev Vygotsky calls the “zone of proximal development.” (1980) Mind in society: The development of higher psychological processes.
[2] According to the IRB new policies all research that hopes to be shared or published is subject to IRB review, even assignments collected as part of a course.
[3] Charles Brunner discusses this in his 1998 book, The Culture of Education.
[4] Scott, J., & Carrington, P. J. (2011). The SAGE handbook of social network analysis. SAGE publications.
[5] While I do not personally endorse any particular software, I have heard good things about NodeXL and UCInet.
[6] Gee, J. (2011). Discourse analysis: What makes it critical? In R. Rogers (Ed.), An Introduction to critical discourse analysis in education (2nd Ed.) (pp. 23-45). New York: Routledge.