Initial Publication Date: April 19, 2012

Your Module/Course Development Timeline


Overview of the Materials Development Timeline

Each module/course development team will follow a series of steps and checkpoints designed to 1) align the materials development process with the overall InTeGrate project goals and timeline and 2) allow for timely interaction among the materials development teams, assessment team, InTeGrate leadership, and SERC staff. After your proposal has been accepted and your contract is initiated, there are three major phases of developing, testing, and revising materials. After these phases are complete your, materials will be published on the InTeGrate website and will be publicly available for use.

Phase 1: Materials in Development (as a team)
Each team's authors work together to create detailed descriptions of materials for faculty use and student versions of the materials when necessary. The Materials Development and Refinement Rubric describes the types of InTeGrate materials the project is creating. The team leader (Tim Bralower, Anne Egger, David Gosselin, David McConnell, John Taber) and the team's assessment consultant provide guidance to the team to help them understand and meet the elements of the rubric. At the end of Phase 1 the materials will need to pass through the rubric, demonstrating that they have all of the desired attributes. In Phase 2, student data will be collected, so authors must prepare by obtaining IRB approval from their institution.

Phase 2: Classroom Pilot (individually)
During the classroom pilot phase, module/course authors take their newly developed curriculum and try it out in their own classroom. Authors collect data that supports the evaluation of the team's materials and of the projects overall goals. Learn more about the design of the assessment and evaluation. Piloting allows authors to understand and document the learning that is taking place in the classroom. This helps ensure that the desired level of learning is being achieved and demonstrates pieces of the curriculum that may need revisions. Since each team member individually pilots the materials, there will be multiple sets of data to inform the material revisions. The classroom pilots also contribute to the evaluation of the larger InTeGrate project effort by gathering data that is common to all InTeGrate classroom pilots. Read more about the assessment data being collected and how it relates to your module/course and the overarching project on the Assessment Data page.

Phase 3: Post-Pilot Materials Revision (as a team)
The materials development team uses data collected during their classroom pilot along with assessment team feedback from the rubric review to guide the team toward meaningful revisions. Authors create individual case studies based on experiences and reflections from implementing the materials, and finalize the information that supports other faculty in using the materials. By the end of the process, the teaching materials and case studies will become publicly-available web pages that other faculty may access and implement in their own classes.

The show/hide links in each section outline the complete steps involved in passing the different phases of the materials development process and are separated by project roles. We encourage you to familiarize yourself with your own responsibilities as well as those of others.

Materials Team Application Accepted and Contract Initiated

To begin materials development, teams will need to do the following: confirm the title of the course/module, provide a short description of the module/course that defines its scope and its central pedagogic features, describe how the materials the team develops address the guiding principles of InTeGrate teaching materials, provide contact information for team members, and indicate the details about the course in which each team member will test the materials. This information will be used to establish the contract for each team member to ensure payment of the stipend. Each team member should understand the terms and conditions of the contract prior to the completion of this step.

Much, if not all, of this information will be collected at the face-to-face new author orientation meeting. If the team does not have a representative present at the new authors face-to-face meeting, you can submit the information using the materials set-up form.

Once this information is received (either at the meeting or via the form), the webteam liaison will build your team's initial webspace and email list, your contract will be drafted, and Stuart Birnbaum will assign the team an assessment consultant. While the team is waiting for this to take place, they can read the Information for New Authors and supporting pages to familiarize themselves with the authoring process.

Phase 1: Materials In Development

As the team develops materials, they will reach various checkpoints where feedback will be given from the overall team leader (e.g. David McConnell, Anne Egger, David Gosselin, Tim Bralower, John Taber), the team's assessment team consultant, and the team's webteam liaison The purpose of the checkpoints is to ensure that the team has adequate input and support to develop successful materials as described by the InTeGrate Materials Development and Refinement Rubric and is moving toward successfully passing rubric based review at Checkpoint 4. By checking the work against the rubric several times throughout development, there is a much smaller chance of having to make major revisions once the module/course is complete.

Checkpoint 1: Goals, Outline, Summative Assessment, and a Plan for Development

The purpose of this checkpoint is to review the initial design and scope of the materials, as well as the timeline and communication plan. This checkpoint insures that the team has a common vision and plan for creating the materials and that all support people understand and can meet the timeline.

The materials development team will have succinctly summarized the module/course content and how the module/course will address the guiding principles as part of the contract set up process. The next step in writing the module or course is for each team to develop module/course level goals, an overall outline of the materials, and a draft of their summative assessment (note: the summary and guiding principles were covered prior to this step, but may need to be modified as materials are further developed). This information should be recorded on the Checkpoint 1 Workpage in the team's course/module pages (note: teams led by McConnell and Egger that initiated work in May 2013 will complete this information within their module/course pages rather than in the Checkpoint 1 Workpage) . This work will help the materials team, team leader, assessment team consultant, and webteam liaison all have a better understanding of the scope of the work being done and its alignment with the rubric.

At this point, the materials development team should also establish a timeline, ways to communicate with one another, and strategies for managing their collective progress. When the team's material is ready for review, the team should inform the team leader via the team's email list (listed on the Materials in Progress page).

Timing for this checkpoint:

  • This checkpoint should be completed after your contract is in place.
  • It will take your assessment team member and team leader approximately 1-2 weeks to review your work.

Checkpoint 2: Breadth, Depth, and Assessment

The purpose of this checkpoint is to provide comprehensive feedback on the structure of the materials, the pedagogic design of the materials and assessments, and the alignment of assessments, materials and goals. Checkpoint 2 is designed to occur at the earliest point in the materials development process, when it is possible to provide detailed and comprehensive feedback. The goal is to spot any developing problems early enough to prevent unnecessary work.

In order to be able to evaluate the team's progress towards meeting the design rubric, the team leader and assessment consultant will be looking for both breadth and depth in the module or course by checkpoint 2, including the assessments. Specifically, the team's web pages should include:

  • a complete outline that shows the breadth of the course or module (overview page), including activity descriptions, pedagogic strategies, planned assessment strategies, and descriptions of any materials that will be developed for students (detailed in unit/module pages).
  • at least one complete activity (for a module) or module (for a course) that includes faculty materials, student materials, and formative and summative assessments with rubrics to display the depth of your materials (upload these in the course materials- work with your webteam consultant if you have questions, and refer to the Teaching Materials Format page)
  • complete example(s) of a summative assessment(s) that will address the module- or course-level goals (linked from the assessment page)

Once these pieces are complete, the team should communicate with the team leader via the team's email list to initiate the review of the materials. The team leader will review and provide feedback for revisions and pass them to the assessment consultant when the material is ready for review. The assessment consultant will score the materials using the rubric and provide constructive feedback.

Timing for this checkpoint:

  • This checkpoint should occur about half-way through the materials development process, an absolute minimum of 3-4 weeks prior to checkpoint 3, 4-6 weeks prior to materials review, and 11-15 weeks prior to the earliest pilot test.
  • It will take your assessment team member and team leader approximately 1-2 weeks to score your work.

Checkpoint 3: Materials Nearly Complete (75-90%)

The purpose of this checkpoint is to provide feedback on any final elements that are likely to prevent the materials from passing the rubric and finalize the timeline for the review process.

The materials development team continues working to create materials. At this checkpoint, some portions of the materials may be complete and others may be mostly complete, however, no individual pieces should be missing entirely. Your webspace should include full descriptions of activities, pedagogic approaches, and all assessments with needed rubrics.

Once materials are nearly complete, the team communicates with the team leader via the team's email list to initiate another review of the materials and to set the date when Checkpoint 4 (Materials Review) will occur.

Timing for this checkpoint:

  • This checkpoint should be completed 1 to 2 weeks prior to the materials review (Checkpoint 4) deadline, and a absolute minimum of 7-9 weeks prior to the earliest classroom pilot date.
  • It will take your assessment team member and team leader approximately 1-2 weeks to review your work.

Checkpoint 4: Materials Review and Revisions

This checkpoint is actually an interval of time over which the materials to be reviewed are finalized on the website, the review is conducted, and revisions necessary to pass the rubric are completed.

The materials development team enters the checkpoint when the materials for review are fully available in the module/course pages. Student worksheets are uploaded as needed, activity descriptions are completed, and the overview of the materials on the module/course overview page are sufficient for another instructor to understand how the parts of the module or course build upon one another to create a cohesive module or course. Once complete, the materials are reviewed by the team lead. The team may be asked to make revisions or additions by the team lead. When the team lead has determined that the materials are ready for review, he/she initates the review by contacting the assessment team. The materials are reviewed using the InTeGrate Curriculum Development and Refinement Rubric by three members of the assessment team: the assessment consultant and two other assessment team members

Timing for this checkpoint:

  • This checkpoint should be completed a minimum of 7 to 9 weeks prior to testing- which allows time for reviews and revisions. This is only enough time if there are no major issues with the module/course.
  • It will take the team leader and assessment team at least 2 weeks to review your work (from the scheduled date).

Materials Team Receives first payment upon completing Checkpoint 4

Phase 2: Classroom Pilot

During Phase 2 you will teach with your materials and collect data that your team will use to improve the materials. These data are also used to evaluate the project as a whole. Learn more about how these data will help you and the project.

Pilot testing steps occur for each individual author (not as a team). Please note that there are data collection steps that need to be addressed prior to the start of the semester in which you plan to pilot the module/course in your class (see the "Data Check A: Before your course begins" section below in the Outline of Piloting Data Checks). Depending on timing, this may occur after Checkpoint 3 or 4. Your webteam liaison can help you determine when you should complete these steps. Read more about Collecting Data in your Classroom.

You will receive the results of the assessment prior to the second face-to-face meeting of your materials development team. These results will include

  • a summary of the GLE pre/post questions, two common essay questions and attitudinal data for your module or course situated in the results to date for the project as a whole.
  • an independent scoring of the student responses to the summative assessments by multiple members of the assessment team.

Phase 3: Post-Pilot Materials Revision and Review

During this phase of the materials development process, the team will revise the materials based on the information gathered during the pilot testing including the experiences of the team in teaching with the materials and the assessment results. Authors will complete the materials that support other faculty in using the materials, including the instructor testimonials which describe the adaptation of the materials for each author's class. At the end of phase 3 the materials will be published on the project website.

Checkpoint 5: Revision Plan

This work begins with the development of a revision plan by the team, typically, at the second face-to-face meeting of your team. This revision plan is based on all three classroom pilot tests, and is informed by the data collected, reflections, case studies, and an intimate knowledge of what worked/didn't work with the module or course. Comparing and contrasting challenges and successes with team members will help development teams create a strong coordinated revision plan. Each team will need to define changes as either:

  1. Global: changes to be applied to the module/course as a whole.
  2. Individual: changes that may help individual teaching scenarios.
Authors will work together to develop the plan, including a timeline and assignment of specific responsibilities.

Checkpoint 6: Individual Instructor Stories and Coordinated Revisions Completed

The goal of this checkpoint is to revise the materials and prepare them for final review. As is the case with journal reviews, the authors will need to prepare a letter indicating how they addressed the concerns raised by the assessment team.

Authors revise materials in keeping with their plan. In addition, they attend to the final edits and formatting described below in the Materials Checklist. They obtain iterative feedback from their team lead as they work. The team leader oversees editing to ensure materials from different teams share a common voice and are fully completed.

Materials Checklist

  • All necessary student materials are available on-line and consistent with the InTeGrate student materials format.
  • All necessary instructor materials are available on-line and consistent with the InTeGrate teaching materials format.
  • Writing style and tone are appropriate for the material. Teaching materials should follow best practices for web writing and present an accessible and compelling case for their use. Materials directed at students should have a voice and presentation appropriate for use in a broad range of classroom.
  • Spelling and grammar are correct.
  • Navigation within the web pages follows InTeGrate conventions and is complete and consistent.
  • All links are functional.
  • Appropriate attribution is given and sources cited throughout Copyright information.
  • All teacher-only materials are identified and communicated to the team webteam member so they can be placed in a protected space (only accessible to instructors).
  • Provide team's webteam member with three examples of the kind of image the team would like to use on their front page.
  • All downloadable materials:
    • are provided formats appropriate for their intended use.
    • appropriately cite their sources.
  • Figures and images have captions including attribution and references as appropriate.
  • All included materials (including photos), both online and downloadable, have accurate licensing information and can be legally redistributed by InTeGrate.
  • Individual instructor studies are finished.

When the authors feel the materials are finished, they provide a letter to their team leader summarizing the changes they've made in response to the initial assessment team feedback. The letter should detail rationale for both changes made *and* areas where they didn't make the requested changes, similar to a letter in response to peer review of a scientific paper. Specifically the letter will address:

  • the editing suggestions from the initial Materials Development Rubric
  • assessment team feedback based on in-class testing data
  • issues and challenges that were called out in the piloting instructors' post-instruction reflection

The Team Leader notifies Krista Herbstrith when this letter is ready. Krista initiates the 3 parallel elements of the final review: technical review, science review and copyediting. These results in feedback to the team which they address to reach checkpoint 7.


Checkpoint 7: Final Assessment and Content Review and Final Revisions

If necessary, feedback from the three reviews (science review, technical review, copyediting review) is sent to the team leader, who works with the module/course team to ensure appropriate changes are made. Stuart Birnbaum apprises Cathy Manduca of final completion of the instructional materials. The team leader apprises Cathy Manduca of final completion of the instructor materials.

Materials published on InTeGrate site and become freely available for use

Second payment is made when materials are published.