OER Case Study No.6
From OER Commons Wiki
Prepared by: The Institute for the Study of Knowledge Management in Education, Fall 2008
ISKME: 
Download this case study
Introduction
Launched in 2002 by WGBH, the non-commercial public media service, located in Boston, Massachusetts, Teachers’ Domain is an online repository of multimedia open educational resources for use in classrooms and for professional development. As part of its effort to increase the availability of freely accessible resources that can be downloaded, shared, and remixed by teachers and learners, WGBH has developed content from public media archives into high quality, open educational resources for Teachers’ Domain.
The aim of this case study has been to examine WGBH and Teachers’ Domain’s successes and challenges involved in developing an open content model. Specifically, this case study seeks to shed light on the process of shifting public media archival content from a proprietary licensing model to an open educational resource model, and to explore the impact of these activities, specifically in terms of how teachers are using and working with the newly developed Teachers’ Domain resources within the context of teaching and learning.
The Teachers’ Domain case study is part of a larger initiative led by the Institute for the Study of Knowledge Management in Education, which involves case studies of six open educational resources (OER) projects that seek to increase understanding of issues that are common across them—such as engagement of online communities, volunteer recruitment, and licensing decisions—and thereby create possibilities for synergistic knowledge-sharing and field-building.
Methodology
This study draws upon a participatory case study methodology similar to David Fetterman’s empowerment model. Fetterman (2005) defines empowerment research as that which helps organizations and communities assess and improve their practices by instilling mechanisms that allow them to align their theory of action (what they think they should be doing) with their theory of use (what they are actually doing). It empowers organizations to build the capacity to assess their practices and to develop insights that may inform those practices in light of project goals. Empowerment research entails collaboration with the case study organization to develop research and data collection tools that can be used to reinforce, test and modify internal knowledge. According to this method, the external researcher’s role becomes that of a critical friend and facilitator as opposed to an expert purveyor of knowledge. In consequence, the insights and experiences of importance to the organization acquire value and weight as the proper perspective from which to cultivate knowledge to inform practices and continue improvement within the organization (Petrides & Loesch-Griffin, 2005).
Consistent with the participatory research model, the Teachers’ Domain case study was an iterative and collaborative process. The case study work began with three initial phone meetings between the ISKME research team and the WGBH/Teachers’ Domain project leaders. The purpose of these conversations was to provide context for the study, to shed light on the project’s past and current challenges and opportunities, to determine the research questions to be addressed by the case study, and how to best answer those research questions. A review of internal documents was also conducted as a way to inform the development of the case study research plan, and to provide further insight on the initiative’s goals, strategies, challenges and opportunities.
The specific questions addressed by the Teachers’ Domain case study included the process, successes and challenges of transitioning from proprietary to open content; how often and why users are visiting Teachers’ Domain; how users engage around the Teachers’ Domain site and its resources; and what factors helped or hindered engagement and use. In answering these questions, five interviews were first conducted by the ISKME research team with members of Teachers’ Domain core leadership team and with one of its content development partners (a public broadcasting station). The interview protocols for these interviews were developed collaboratively by Teachers’ Domain and ISKME, and sought to understand the practices, successes and challenges around the move toward open content.
An online survey of Teachers’ Domain users was then conducted to understand use and user engagement of the resources. The user survey consisted of both multiple choice and open-ended questions, specifically addressing how often users visit Teachers’ Domain, their specific use behaviors (i.e., where they click, what they do), their incentives and disincentives to use, and whether and how they use and modify Teachers’ Domain resources in the classroom. The survey protocol was developed collaboratively by ISKME and the WGBH/Teachers’ Domain project, and was distributed via email to 19,369 Teachers’ Domain registered users. A link to the survey was also posted on the Teachers’ Domain site. In total, 515 users responded to the survey. The majority of respondents (76 percent, or 323 respondents) self-identified as K-12 teachers; however, professional development coaches, administrators, and researchers responded to the survey as well (also self-identified).
In analyzing the survey data, basic descriptive statistics were obtained for all survey items to look at overreaching use behaviors, experiences and patterns. Independent chi-square analyses were conducted to determine whether there were significant differences with regard to use behaviors between different groups of users. More specifically, the chi-square analysis sought to reveal how active users potentially differ from inactive users in terms of how they draw on and use Teachers’ Domain’s tools and resources.
Subsequently, a user interview protocol was developed by ISKME, with input provided by the Teachers’ Domain management team. Informed by the preliminary analysis of the survey data, the interview questions were designed to gain a qualitative understanding of the users’ experiences with Teachers’ Domain and with open educational resources more generally. An email invitation was sent by ISKME to eleven survey respondents who indicated their willingness to participate in the follow up interviews, but only two of the eleven responded to the invitation. The two interviews were conducted by telephone and both lasted approximately 30 minutes.
Findings
In creating open content for Teachers’ Domain, WGBH worked with its own collection of public broadcasting materials and provided guidance to other public broadcasting stations who also sought to make the transition from proprietary to open content. Once converted from proprietary to open, the materials were made available on the Teachers’ Domain site. The site was then redesigned to accommodate the new categories of use—from download, to share, to remix—for teachers and learners. The findings below discuss the process of transitioning toward open, as well as the engagement of users around the newly opened resources.
Transitioning from Proprietary to Open Content
WGBH began the process of shifting resources from proprietary to open content by analyzing and categorizing each resource and its elements[1] by the type and extent of rights clearances necessary to shift to an open licensing model. Specifically, the project worked closely with legal staff to determine the appropriate rights profile for the resources based on the resources’ existing licenses. Each element within a resource was assigned one of four categories. These included:
- Level 0 – Not available for open access. The user can view the media resource only on the Teachers’ Domain website
- Level 1 – Download allowed. The user can download the media resource to a local computer or device
- Level 2 – Download and reuse allowed. The user can embed the entire media resource in a non-commercial educational presentation or email it to a colleague, as long as its use remains non-commercial and educational, the entire asset is used, and the source is acknowledged
- Level 3 – Download, reuse and remix allowed. The user has the right to re-edit any of the individual elements of the media resource to create a derivative work. The derivative work may only be used for non-commercial educational purposes and the source must be acknowledged
Once the individual elements were categorized, the resources that they comprised were also assigned a level. The level assigned to a resource was determined by its most “restrictive” element. Thus, if a resource was comprised of, e.g., three elements, with one being Level 1, and the other elements being Level 2, the resource would be categorized as a Level 1 resource.
The project also assessed technological characteristics of each resource to determine its suitability for open educational purposes. Specifically, the technological assessment evaluated whether the content was, e.g., downloadable and remixable. The resources were then examined from a pedagogical perspective to assure that they would meet the needs of teachers, specifically by considering existing gaps perceived by teachers in terms of access to content.
Once the legal, pedagogical and technical assessments were complete, the project identified those resources that were potentially eligible to migrate to higher level of use (e.g., from Level 1 to Level 2). The rights clearances work for a resource sometimes involved multiple rights holders, and it was not uncommon to pursue rights clearances through negotiations that included not only actors unions and talent guilds but also rights-holders for such elements as building sites and music. In seeking rights clearances, Teachers’ Domain contacted rights-holders to obtain permissions and clearances. Throughout this process, it became evident that many rights-holders were resistant to the concept of open. In particular, three key concerns surfaced:
- Integrity of materials and control: A prevailing concern among some rights holders was that if a portion of content were to be highlighted out of context, then that content would be perceived as having been altered or having lost its original integrity. Television producers were anxious that, for instance, interviewees in their documentaries might be less willing to share their thoughts and ideas openly if they thought that the material produced would later be modified and potentially taken out of context.
- Attribution: Many rights-holders were concerned that they might not receive the recognition or credit due to them as original author or rights-holder.
- Commercial viability: For owners of commercially licensed content, actors and others who appear in the videos, fears of loss or diminution of the financial value of their work posed an obstacle. At the very least, the business benefits of open appeared as unknown.
However, the project discerned that many producers and rights holders were already comfortable with allowing materials to be used “for educational purposes only.”[2] Yet, because no such license existed, WGBH developed its own licensing structure based on the four levels of use listed above (Levels 0, 1, 2, and 3). The licenses stipulated that the materials are open yet must be used according to the designated category, and for educational purposes only. In short, Teachers’ Domain found that in order to address the concerns of rights-holders, both the messaging and buy-in needed to stipulate conditions of use including that the content was for educational purposes only.
Once the rights clearance work was complete, the production phase of the work began. If rights clearances had not been obtained for a particular segment or element of the content, it was sometimes necessary to reshoot that segment. Production work also entailed adjustments to the content such as segmenting the resources into smaller, stand-alone chunks to better accommodate an interactive audience’s ability to mix and remix. The production work also entailed the development of metadata for the content (abstracts, keywords, etc.) as well as supplemental materials such as discussion questions and lesson plans. Other necessary production tasks included fixing broken links and improving the graphics as needed. In terms of final editing, the project, established an iterative editing process in small batches, rather than whole collections, as a way to better facilitate workflow. Here, experts carefully reviewed each resource for educational appropriateness as well as for factual, grammatical, or typographical errors.
On the whole, interviews with the WGBH/Teachers’ Domain project participants revealed that that some types of content were easier than others to convert to open. For example, content created by local rather than national stations was easier to secure rights for, as there were fewer layers of complexity (for example, there were fewer large name, or famous individuals to secure rights from, and many local productions do not involve talent guilds). Additionally, the project learned that had the content been produced with an eye toward open up front, the transition would have been easier in the end. Thus, while the project learned valuable lessons in the process of transitioning proprietary materials to open content, the project going forward plans to encourage the creation of open materials from the outset, by working more broadly in educating talent guilds, producers, and others up-front.
Engaging Users Around the Content
Teachers’ Domain aims to provide free resources to its users, and to inspire them to develop resources, collaborate with other users and share ideas. As such, they offer several tools to support these activities. First, users are able to create their personal folders where they can save and organize favorite resources, sort their collections, and write notes specific to each resource (“My Folders”). Teachers’ Domain users are also able to download resources to their own computer or compatible viewing device; share resources with their students and colleagues (through “My Groups”); and edit and remix resources. Thus, they can embed video or audio into their presentations, give compilations of resources to students and create derivative resources. Teachers’ Domain users can also customize their searches by specifying the grade range they teach, the state standards they use, and the accessibility features they need (through “My profile” function).
Through analysis and presentation of data from surveys and interviews with Teachers’ Domain users, the sections below illuminate how often and in what ways Teachers’ Domain users visit the site and draw upon its resources and tools, as well as the incentives and disincentives for doing so.
How Often Users Visit Teachers’ Domain
Survey respondents indicated how often they visit the Teachers’ Domain site, and in this sense self identified as active (daily or weekly visits), intermediate (1-2 times per month), infrequent (1-3 times in the prior six months) or rare users (have not visited the site in the past 6 months). As revealed in Table 1 below, 18 percent of the survey respondents (92) identified as active users. Thirty percent (153) identified themselves as intermediate users, with 28 percent (145) identifying as infrequent users, and 24 percent (125) as rare users.
Table 1. Breakdown of Survey Participants by Frequency of Use
| Frequency of Use
(Multiple choice question; respondents select one answer) | Percent of Respondents (N)
(Total respondents=515) |
|---|---|
| Active (daily or weekly) | 18% (92) |
| Intermediate (monthly) | 30% (153) |
| Infrequent (2-3 times in the past 6 months) | 28% (145) |
| Rare (have not visited in the last six months) | 24% (125) |
Why Users Come to Teachers’ Domain
As illustrated in Table 2 below, the most common reasons that users visit Teachers’ Domain were to get ideas for supplementing existing lessons or for new lessons. Each of these categories was represented by 66 percent of the survey respondents (305 to supplement existing lessons, and 306 to get ideas for new lessons). Thirty-seven percent of survey respondents (170) reported using the site to improve their teaching methods or enhance their professional development, and 33 percent (154) to stay current in a subject or topic area. The least common reported purpose was 5 percent (23) who reported using Teachers’ Domain to connect to other professionals, indicating that the resources, rather than the communication tools, may likely serve as the biggest draw in bringing users to the site.
Table 2. Reasons for Visiting Teachers' Domain
| Reason for visiting
(Multiple choice question; respondents select all that apply) | Percent of Respondents (N)
(Total respondents=464) |
|---|---|
| To supplement my existing lessons or coursework | 66% (306) |
| To get ideas for new lessons | 66% (305) |
| To improve my teaching methods or enhance
my professional development | 37% (170) |
| To stay current in a subject or topic area | 33% (154) |
| To learn about a new topic | 31% (143) |
| To connect with teachers or learners who
have similar interests | 5% (23) |
| Other | 10% (45) |
Table 2 also shows that 10 percent (45) of survey respondents cited “other” as a reason for visiting Teachers’ Domain. Survey respondents were allowed to list specific reasons when choosing “other” as a response, and these included finding training materials, previewing resources for students, and checking what is new on the site.
In examining the reasons for visiting Teachers’ Domain by the frequency of use (active, intermediate, infrequent, rare), statistically significant results surfaced for three of the reasons: to supplement existing lessons, get ideas for new lessons, and to stay current in a subject or topic area. Specifically, the chi-square analysis revealed that active and intermediate users were more likely than infrequent or rare users to visit the site to supplement existing lessons (χ2=40.94, p<.000); to get ideas for new lessons (χ2=19.68, p<.000); and to stay current in a subject or topic area (χ2=16.24, p<.001).
How Users Engage with Teachers’ Domain Resources and Tools
Table 3 below provides a breakdown of the ways that users are engaging with Teachers’ Domain resources. The most common type of engagement with the resources was viewing them, which was reported by 65 percent (283) of survey respondents. Using the resources “as-is” in their original format was reported by 40 percent (174). The table also shows that 28 percent (121) reported remixing, and 23 percent (102) editing, 18 percent (78) of the survey respondents reported sharing resources via email. In terms of the way that users share the resources, the data indicate that they were much more likely to share via users’ personal email than through the mechanisms offered on the site, such as “My folders” or “My Groups”.
Table 3. Types of Engagement with Teachers’ Domain Resources
| Type of Engagement
(Multiple choice question; respondents select all that apply) | Percent of Respondents (N)
(Total respondents=434) |
|---|---|
| View them by clicking on the “view” button | 65% (283) |
| Use them “as is” by, e.g., embedding them in a presentation
or showing them to my students | 40% (174) |
| Download them to my computer or handheld device | 39% (171) |
| Save them to “My Folders” | 30% (128) |
| Remix them (or add parts of them to my own
or other educational materials) | 28% (121) |
| Edit them on my own computer after downloading | 23% (102) |
| Share them with others via email | 18% (78) |
| Share them with others via “My Folders” or “My Groups” | 6% (27) |
| None of the Above | 8% (35) |
| Other | 5% (21) |
When examined by frequency of use, chi-square analyses indicated three statistically significant differences in the above types of engagement. Specifically, active and intermediate users were more likely than infrequent or rare users to download resources to their computers or handheld devices (χ2=26.45, p<.000), were more likely to share resources with others (χ2=16.10, p< .001), and were more likely to use the resources by embedding them in a presentation or showing them to students (χ2=40.70, p < .000).
In addition, the survey revealed that 55 percent (234) of survey respondents used Teachers’ Domain resources in their classrooms. The most often-cited classroom use involved supplementing lectures with audio and video clips. Use of resources in the classroom was found to decline in relation with reported frequency of use: 89 percent (76) of active users indicated of having used Teachers’ Domain resources in the classroom, compared to 67 percent (95) of intermediate users, 41 percent (52) of infrequent users and 14 percent (10) of rare users (χ2=107.05, p<.000).
Incentives and Disincentives to Use
As noted above, survey data indicate that the majority of Teachers’ Domain users visit the site to get ideas for new lessons and to supplement their existing course work and lessons. The interviews shed additional light on why users visit Teachers’ Domain and use its resources. Specifically, interview respondents indicated that the high quality, ease of use and comprehensibility of the resources were important incentives to use. One teacher indicated frequent use of video clips targeted to primary or secondary students in her college-level courses because of the accessibility and ease of use of the clips. Another teacher indicated that the video clips filled a gap in existing content and resource needs—for example, she used videos that demonstrated laboratory experiments because she did not have access to a student science lab at her teaching site. Interestingly, both teachers indicated that the high quality of the resources made editing and remixing them unnecessary.
As revealed in Table 4 below, survey data also revealed perceived disincentives to using Teachers’ Domain resources. Specifically, when asked about the challenges they faced in using the site’s tools and resources, survey respondents indicated several issues. Thirty-four percent (151) reported overcoming technology hurdles in finding, viewing, using, sharing or creating resources. Twenty-five percent (110) reported not knowing whether they had permission to use, change, or modify the resources; and 20 percent (92) of the survey respondents report the challenge of understanding the organization and layout of the site.
Table 4. Challenges in Using Teachers' Domain Resources'
| Type of Challenge
(Multiple choice question; respondents select all that apply) | Percent of Respondents (N)
(Total respondents=448) |
|---|---|
| Overcoming technology hurdles in finding, viewing, downloading,
sharing, or using the resources | 34% (151) |
| Knowing whether I have permission to use, change, or modify
resources (copyright and use permissions) | 25% (110) |
| Understanding the organization and layout of the site,
so that I can easily navigate through it | 20% (92) |
| Knowing how to use the resources in the classroom | 18% (79) |
| None of the Above | 22% (98) |
| Not Applicable | 10% (43) |
| Other | 14% (61) |
Specifically with regard to the 25 percent of users who viewed copyright and use permissions as a challenge to use, the analysis further revealed—through an additional survey question about licensing—that approximately 23 percent (97) of survey respondents felt they lacked knowledge of the distinction between the four categories of licenses (View Only; Download; Download and Share; and Download, Share and Remix). However, more than one-half of the respondents, 56 percent, (237) indicated having above average knowledge of the distinction, with 33 percent (139) rating themselves as “fully knowledgeable” of the differences between the licenses. On the other hand, 80 percent of survey respondents (335) reported that the licensing categories were helpful in determining whether to use a resource, versus 20 percent (83 respondents) who indicated that the licensing categories were not helpful. Open ended comment fields on survey questions about the licenses indicated that while some respondents identified the distinctions a cumbersome in terms of helping them navigate the resources, many others stated that the transparency of the license distinction facilitated their ability to determine which resources to focus on when visiting the site.
Table 4 also shows that 14 percent of survey respondents (61) cited “other” challenges to using Teachers’ Domain. The specific reasons listed when choosing “other” as a response included the lack of computers, internet connection, or the necessary software or access rights within their schools or institutions. The interviews with Teachers’ Domain users underscored these findings, as interview participants indicated that their hesitancy specifically to remix Teachers’ Domain content stemmed from lack of necessary tools as well as the knowledge necessary to do so.
In examining these challenges for visiting Teachers’ Domain by the frequency of use (active, intermediate, infrequent, rare), the chi-square analysis revealed only one statistically significant difference: that active users were less likely than intermediate, infrequent or rare users to indicate the organization and layout of the site as a challenge to Teachers’ Domain use (χ2=9.13, p<.028).
Discussion and Conclusions
As illustrated by the Teachers’ Domain case study, converting proprietary resources into open content presents several complex challenges. To meet these challenges successfully, WGBH and Teachers’ Domain underscored the importance of preliminary assessments—legal, pedagogical, and technical—to assure consideration of all aspects and layers of proprietary materials. As preliminary assessments prepare the groundwork for the licensing and production work necessary to realize the conversion to open content, assessments also help to assure maximum quality in the final resource. Because of the costs involved in converting materials to open, Teachers’ Domain plans to work more broadly in educating talent guilds, producers, and others up-front, to encourage the creation of materials with an eye toward open from the outset.
The Teachers’ Domain case study further revealed that projects seeking to transform proprietary content into open content may encounter hesitancy among rights-holders. As revealed by the case study, these reasons for this hesitancy may include: fear of loss of control of content, anxiety regarding loss of attribution, and worry that the commercial value of their work may be diluted and lost. As the project worked to create shared understanding around the concept of open, its assessment of such issues helped to inform messaging around the meaning of open, and to evolve strategies for creating and maintaining buy-in. A revealed, Teachers’ Domain found that in order to address the concerns of rights-holders, both the messaging and buy-in needed to stipulate conditions of use including that the content was for educational purposes only.
The findings from the case study further highlighted opportunities for Teachers’ Domain to support use and user engagement around its resources, its tools, and the site overall. First, given that the most commonly cited reasons for visiting Teachers’ Domain were to get ideas for new lessons and to supplement existing coursework, the continuous addition and enrichment of resources on behalf of Teachers’ Domain becomes central.
Additionally, given that active Teachers’ Domain users were more likely to share resources with others, embed them in their own content, and use them in the classroom, finding ways to support those who may want to engage in these more “advanced” activities, but who may lack the knowledge or supports to do so becomes central. Importantly, these supports should potentially move beyond mitigating design and technology hurdles related to the organization and layout of the Teachers’ Domain site. In light of the finding that teachers do not necessarily have the institutional and technology supports at their local institutions for OER use, mechanisms to facilitate OER engagement in the context of (and as a supplement to) teachers’ local teaching and learning environments also becomes important. These could include, for those unfamiliar with licensing and the categories of use permissions for the resources, supporting knowledge sharing and training around copyright issues more generally, and the implications of the different licensing options. They could also include knowledge sharing and training more generally on OER and its direct use in the classroom.
On the whole, the findings suggest that moving toward increasingly active use and enhanced user engagement around Teachers’ Domain’s resources and tools necessitates further exploration into existing use behaviors. Beyond conducting additional surveys and interviews directly with users, this also potentially includes user log file data collection and analysis as a way to track and understand how users navigate through the Teachers’ Domain site, what they view on the site, and what specific actions they take. For OER projects more generally, ongoing research on user behaviors, experiences and perceptions can be a challenging and resource-intense process; however, by assessing and building data collection mechanisms and research questions into organizational practices, knowledge and learnings can be cultivated to inform how users are best supported, as well as to inform continuous improvement for the projects overall.
Acknowledgements
This case study report was written by Lisa Petrides and Cynthia Jimes, with support from Anastasia Karaglani, Clare Mortensen and Rene Keep. The case study initiative was funded by the Shuttleworth Foundation, the International Development Research Centre, The William and Flora Hewlett Foundation, and Curriki. The initiative was facilitated with support from iCommons.
References
Fetterman, D. (2005). "A Window into the Heart and Soul of Empowerment Evaluation: Looking Through the Lens of Empowerment Evaluation Principals." In Empowerment Evaluation Principles in Practice. Ed. David M. Fetterman and Abraham Wandersman. New York: The Gilford Press.
Petrides, L. and Loesch-Griffin, D. (2005). “Strengthening Evaluation for Continuous Improvement Through Knowledge Management.” Working Paper, Institute for the Study of Knowledge Management in Education, Half Moon Bay, CA.
Notes
[1] Each resource is composed of smaller pieces, which are here referred to as elements. For example, a video clip or segment could contain ten or more shorter video shots. It could also contain interviews with individuals, a narrator’s voice, music, etc. Each of these is an element that has rights associated with it. [2] Teachers’ Domain defines educational purposes in terms of use “for the primary purpose of learning or teaching in both formal and informal educational settings, use by educators or students in classrooms, libraries, schools and at home, and/or the presentation of materials to an individual or a group for the purpose of learning or teaching.”
Links to Other Case Study Project Resources
OER Case Study Project
Individual Case Study Reports
- Building Open Educational Resources from the Ground Up: South Africa’s Free High School Science Texts
- Curriki: Facilitating Use and User Engagement Around Open Educational Resources
- CurriculumNet: Creating Freely Available Curriculum Materials to Meet Uganda’s Growing Student Population
- Mission 2007 Training Commons: Developing a Living Curriculum for Telecentre Workers in India
- Stanford Encyclopedia of Philosophy: Toward a Sustainable Funding Model for Open Access
- WGBH’s Teachers’ Domain: Producing Open Materials and Engaging Users
Case Study White Paper
The case study white paper is a cross-case comparison of the individual case studies listed above.
Case Study Framework
This case study framework seeks to assist OER projects in tracking, analyzing and sharing key developments in the creation, use and reuse of open educational resources. Based on the case studies of each of the six projects above, the framework provides research tools, reflective questions, and examples that OER projects can draw upon in conducing their own case studies.
