Distributed Item Review: Administrator User Guide (Technical Report No. 1603)

Irvin, P. S. (2016). Distributed item review: Administrator user guide. (Technical Report No. 1603). Eugene, OR: Behavioral Research and Teaching, University of Oregon. The Distributed Item Review (DIR; Behavioral Research and Teaching, 2013) is a secure and flexible, web-based system designed to present test items to expert reviewers across a broad geographic area for evaluation of important dimensions of quality (e.g., alignment with standards, bias, sensitivity, and student accessibility). The DIR is comprised of essential features that allow system administrators to design and conduct customized review studies, in which test items are bulk uploaded, paired with target academic content/performance standards and images, and then organized and assigned for review by selected experts (e.g., content and field experts, education leaders and practitioners, education researchers, and test and curricula developers). Administrators can choose from predefined item review questions, or design new review questions pertinent to their study.     TechRpt_1603