Practical Guide: Conduct and Assess Teacher/Learner Trials

The aim of teacher and learner trials to collect valuable feedback somewhat early on from the end users of open content, which can be used to inform the revision and further development of the content. The trials ideally take place in the actual teaching and learning environment, where future users can test or pilot recently developed teaching and learning materials to help determine, e.g., if the content meets students’ educational levels and varied learning styles, and overall, whether it is relevant, usable and adaptable to local teaching and learning needs. This guide aims to help OER projects organize their own teacher and learner trials of open content.

1. Identify and solicit participating schools. Word of mouth and networking are good ways to identify people at locations who are willing to participate in your trials. For example, you could approach schools in which you know a teacher or principal, or schools that you or your project colleagues attended as a student. Be prepared to explain the purpose of the trial and your expectations for the participating teachers and learners, as well as potentially needing to request permission from the parents or other key stakeholders.

Also when choosing and soliciting schools, you may want to consider the advantages of selecting those that offer diversity in terms of demographics and aptitude within or across school populations—so as to ensure that your content is usable and adaptable for diverse sets of users. Of course, the more settings that participate the more feedback you will obtain; however, this must be weighed against the available resources for conducting the trials, the timeline, and the requirements and needs of the trial participants.

2. Match content offering to content needs. In order to ensure that your open content can be directly incorporated into the classroom setting for the trial period, it may be necessary to survey teaching needs before content is delivered. What content areas will teachers be focusing on during the trial period? What are their needs and wishes for content exercises and activities? By answering these questions, you can better tailor your existing content and potentially develop new content to match needs. Pre-trial questionnaires to assess teacher and learner needs accompanied by face-to-face meetings with teachers are ways to ensure that content is well matched.

3. Pilot the content. Depending on the context, provide the teachers with electronic or printed versions of the content. Some situations may call for training or guidance in using the content for the first time. The amount of time for each pilot often depends upon the amount of content the teacher is given, as well as upon the teacher’s own schedule. Talk with the teachers to determine a suitable schedule that meets your project’s needs as well as the needs of the teacher.

4. Develop instruments for post-trial feedback. In order to assess the feedback on the content, you will need to develop an instrument(s) to understand teacher and learner perceptions. Questionnaires, interviews, and focus groups are common methods of collecting this type of data. In a questionnaire, for example, students and teachers could be asked pointed questions that get at their perceptions about the level of difficulty, layout, or the language of the content. Teachers could also be asked questions to get at how well specific topics were covered in the texts, the layout and appearance of the texts, etc.

5. Asses post-trial data. After the pilot data have been collected, the analysis phase begins. Collected data needs to be organized, summarized and reported in some way. This can be a time consuming process, so you might consider soliciting either paid or volunteer help through from students, researchers, etc.

6. Conduct follow-up workshops. You can decide to have a follow up meeting with teachers (or other trial participants) delve deeper into the results of the questionnaires, and to ask clarifying questions. In these workshops, the trial participants could be invited to discuss and expand upon their post-trial questionnaire responses in person. The workshop can include a presentation of the main findings from the questionnaires, with questions throughout to invite feedback and further discussion. Such workshops offer a good way to validate findings, enable the trial participants to see the consequences of their participation, and to provide a means of building and strengthening relationships with the participants.