OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT,DEPARTMENT OF EDUCATION

34 CFR Ch. VII (7-1-00 Edition) Off. of Educ. Research and Improvement, Education CHAPTER VII—OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT, DEPARTMENT OF EDUCATION Part Page 700 Standards for the conduct and evaluation of activities carried out by the Office of Educational Research and Improvement (OERI)—Evaluation of applications for grants and cooperative agreements and proposals for contracts 903 701 Standards for conduct and evaluation of activities carried out by the Office of Educational Research and Improvement (OERI)—Designation of exemplary and promising programs 909 702 Standards for conduct and evaluation of activities carried out by the Office of Educational Research and Improvement (OERI)—Evaluation of the performance of recipients of grants, cooperative agreements, and contracts 911 Pt. 700 PART 700—STANDARDS FOR THE CONDUCT AND EVALUATION OF ACTIVITIES CARRIED OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT (OERI)—EVALUATION OF APPLICATIONS FOR GRANTS AND COOPERATIVE AGREEMENTS AND PROPOSALS FOR CONTRACTS Subpart A—General Sec. 700.1 What is the purpose of these standards? 700.2 What activities must be governed by these standards? 700.3 What additional activities may be governed by these standards? 700.4 What definitions apply? 700.5 What are the processes of open competition? Subpart B—Selection of Peer Reviewers 700.10 When is the peer review process used? 700.11 Who may serve as peer reviewers? 700.12 What constitutes a conflict of interest for grants and cooperative agreements? 700.13 What constitutes a conflict of interest for contracts? Subpart C—The Peer Review Process 700.20 How many peer reviewers will be used? 700.21 How are applications for grants and cooperative agreements evaluated? 700.22 How are proposals for contracts evaluated? Subpart D—Evaluation Criteria 700.30 What evaluation criteria are used for grants and cooperative agreements? 700.31 What additional evaluation criteria shall be used for grants and cooperative agreements? 700.32 What evaluation criteria shall be used for contracts? Subpart E—Selection for Award 700.40 How are grant and cooperative agreement applications selected for award? 700.41 How are contract proposals selected for award? Authority:

20 U.S.C. 6011(i).

Source:

60 FR 47810, Sept. 14, 1995, unless otherwise noted.

Subpart A—General

§ 700.1 What is the purpose of these standards?

(a) The standards in this part implement section 912(i) of the Educational Research, Development, Dissemination, and Improvement Act of 1994.

(b) These standards are intended to ensure that activities carried out by the Office of Educational Research and Improvement (the Office) meet the highest standards of professional excellence.

(Authority: 20 U.S.C. 6011(i)(1))

§ 700.2 What activities must be governed by these standards?

(a) The standards in this part are binding on all activities carried out by the Office using funds appropriated under section 912(m) of the Educational Research, Development, Dissemination, and Improvement Act of 1994.

(b) Activities carried out with funds appropriated under section 912(m) of the Act include activities carried out by the following entities or programs:

(1) The National Research Institutes.

(2) The Office of Reform Assistance and Dissemination.

(3) The Educational Resources Information Center Clearinghouses.

(4) The Regional Educational Laboratories.

(5) The Teacher Research Dissemination Demonstration Program.

(6) The Goals 2000 Community Partnerships Program.

(7) The National Educational Research Policy and Priorities Board.

(Authority: 20 U.S.C. 6011(i)(1))

§ 700.3 What additional activities may be governed by these standards?

(a) The Secretary may elect to apply the standards in this part to activities carried out by the Department using funds appropriated under an authority other than section 912(m) of the Act.

(b)(1) If the Secretary elects to apply these standards to a competition for new grant or cooperative agreement awards, the Secretary announces, in a notice published in the Federal Register, the extent to which these standards are applicable to the competition.

(2) If the Secretary elects to apply these standards to a solicitation for a contract award, the Secretary announces in the request for proposals the extent to which these standards are applicable to the solicitation.

(Authority: 20 U.S.C. 6011(i)

§ 700.4 What definitions apply?

(a) Definitions in the Educational Research, Development, Dissemination, and Improvement Act of 1994. The following terms used in this part are defined in 20 U.S.C. 6011(l):

Development

Dissemination

Educational Research

Office

National Research Institute

Technical Assistance

(b) Definitions in Education Department General Administrative Regulations. The following terms used in this part are defined in 34 CFR 77.1:

Applicant

Application

Award

Department

Grant

Project

Secretary

(c) Definitions in the Federal Acquisition Regulation. The following terms used in this part are defined in 48 CFR chapter 1:

Contracting Officer

Employee of an Agency

Proposal

Solicitation

(d) Other definitions. The following definitions also apply to this part:

Act means the Educational Research, Development, Dissemination, and Improvement Act of 1994 (Title IX of Pub. L. 103-227, 108 Stat. 212).

EDAR means the Education Department Acquisition Regulation, 48 CFR chapter 34.

EDGAR means the Education Department General Administrative Regulations, 34 CFR parts 74, 75, 77, 79, 80, 81, 82, 85 and 86. FAR means the Federal Acquisition Regulation, 48 CFR chapter 1.

(Authority: 20 U.S.C. 6011)

§ 700.5 What are the processes of open competition?

The Secretary uses a process of open competition in awarding or entering into all grants, cooperative agreements, and contracts governed by these standards. The processes of open competition are the following:

(a) For all new awards for grants and cooperative agreements, the Secretary will make awards pursuant to the provisions of EDGAR with the exception of the provisions in 34 CFR 75.100(c)(5), 75.200(b)(3), (b)(5), 75.210, and 75.217(b)(1), (b)(2), (c), and (d); and

(b) For contracts, the Department will conduct acquisitions pursuant to this part in accordance with the requirements of the Competition in Contracting Act, 41 U.S.C. 253, and the FAR.

(Authority: 20 U.S.C. 6011(i)(2); 41 U.S.C. 253)

Subpart B—Selection of Peer Reviewers

§ 700.10 When is the peer review process used?

The Secretary uses a peer review process—

(a) To review and evaluate all applications for grants and cooperative agreements and proposals for those contracts that exceed $100,000;

(b) To review and designate exemplary and promising programs in accordance with section 941(d) of the Act; and

(c) To evaluate and assess the performance of all recipients of grants from and cooperative agreements and contracts with the Office.

(Authority: 20 U.S.C. 6011(i)(2)(B))

§ 700.11 Who may serve as peer reviewers?

(a) An individual may serve as a peer reviewer for purposes of reviewing and evaluating applications for new awards for grants and cooperative agreements and contract proposals if the individual—

(1) Possesses the following qualifications:

(i) Demonstrated expertise, including training and experience, in the subject area of the competition.

(ii) In-depth knowledge of policy or practice in the field of education.

(iii) In-depth knowledge of theoretical perspectives or methodological approaches in the subject area of the competition; and

(2) Does not have a conflict of interest, as determined in accordance with § 700.12.

(b) For each competition for new awards for grants and cooperative agreements—

(i) Department staff may not serve as peer reviewers except in exceptional circumstances as determined by the Secretary; and

(ii) The majority of reviewers may be persons not employed by the Federal Government.

(2) For each review of an unsolicited grant or cooperative agreement application—

(i) Department employees may assist the Secretary in making an initial determination under 34 CFR 75.222(b); and

(ii) Department employees may not serve as peer reviewers in accordance with 34 CFR 75.222(c).

(c) To the extent feasible, the Secretary selects peer reviewers for each competition who represent a broad range of perspectives.

(Authority: 20 U.S.C. 6011(i)(2)(B))

§ 700.12 What constitutes a conflict of interest for grants and cooperative agreements?

(a) Peer reviewers for grants and cooperative agreements are considered employees of the Department for the purposes of conflicts of interest analysis.

(b) As employees of the Department, peer reviewers are subject to the provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's policies used to implement those provisions.

(Authority: 20 U.S.C. 6011(i)(2)(B))

§ 700.13 What constitutes a conflict of interest for contracts.

(a) Peer reviewers for contract proposals are considered employees of the Department in accordance with FAR, 48 CFR 3.104-4(h)(2).

(b) As employees of the Department, peer reviewers are subject to the provisions of the FAR, 48 CFR part 3 Improper Business Practices and Personal Conflict of Interest.

(Authority: 41 U.S.C. 423)

Subpart C—The Peer Review Process

§ 700.20 How many peer reviewers will be used?

(a) Each application for a grant or cooperative agreement award must be reviewed and evaluated by at least three peer reviewers except—

(1) For those grant and cooperative agreement awards under $50,000, fewer than three peer reviewers may be used if the Secretary determines that adequate peer review can be obtained using fewer reviewers; and

(2) For those grant and cooperative agreement awards of more than $1,000,000, at least five reviewers must be used.

(b) Each contract proposal must be read by at least three reviewers unless the contracting officer determines that an adequate peer review can be obtained by using fewer reviewers.

(c) Before releasing contract proposals to peer reviewers outside the Federal Government, the contracting officer shall comply with FAR, 48 CFR 15.413-2(f).

(Authority: 20 U.S.C. 6011(i)(2)(B))

§ 700.21 How are applications for grants and cooperative agreements evaluated?

(a) Each peer reviewer must be given a number of applications to evaluate.

(b) Each peer reviewer shall—

(1) Independently evaluate each application;

(2) Evaluate and rate each application based on the reviewer's assessment of the quality of the application according to the evaluation criteria and the weights assigned to those criteria; and

(3) Support the rating for each application with concise written comments based on the reviewer's analysis of the strengths and weaknesses of the application with respect to each of the applicable evaluation criteria.

(c)(1) Except as provided in paragraph (c)(2) of this section, after each peer reviewer has evaluated and rated each application independently, those reviewers who evaluated a common set of applications are convened to discuss the strengths and weaknesses of those applications. Each reviewer may then independently reevaluate and re-rate an application with appropriate changes made to the written comments.

(2) Reviewers are not convened to discuss an unsolicited application unless the Secretary determines that discussion of the application's strengths and weaknesses is necessary.

(d) Following discussion and any reevaluation and re-rating, reviewers shall independently place each application in one of three categories, either “highly recommended for funding,” “recommended for funding” or “not recommended for funding.”

(e) After the peer reviewers have evaluated, rated, and made funding recommendations regarding the applications, the Secretary prepares a rank order of the applications based solely on the peer reviewers’ ratings.

(Authority: 20 U.S.C. 6011(i)(2)(C))

§ 700.22 How are proposals for contracts evaluated?

(a) Each peer reviewer must be given a number of technical proposals to evaluate.

(b) Each peer reviewer shall—

(1) Independently evaluate each technical proposal;

(2) Evaluate and rate each proposal based on the reviewer's assessment of the quality of the proposal according to the technical evaluation criteria and the importance or weight assigned to those criteria; and

(3) Support the rating for each proposal with concise written comments based on the reviewer's analysis of the strengths and weaknesses of the proposal with respect to each of the applicable technical evaluation criteria.

(c) After each peer reviewer has evaluated each proposal independently, those reviewers who evaluated a common set of proposals may be convened to discuss the strengths and weaknesses of those proposals. Each reviewer may then independently reevaluate and re-rate a proposal with appropriate changes made to the written comments.

(d) Following discussion and any reevaluation and re-rating, reviewers shall rank proposals and advise the contracting officer of each proposal's acceptability for contract award as “acceptable,” “capable of being made acceptable without major modifications,” or “unacceptable.” Reviewers may also submit technical questions to be asked of the offeror regarding the proposal.

(Authority: 20 U.S.C. 6011(i)(2)(C))

Subpart D—Evaluation Criteria

§ 700.30 What evaluation criteria are used for grants and cooperative agreements?

(a) Except as provided in paragraph (d) of this section, the Secretary announces the applicable evaluation criteria for each competition and the assigned weights in a notice published in the Federal Register or in the application package.

(b) In determining the evaluation criteria to be used in each grant and cooperative agreement competition, the Secretary selects from among the evaluation criteria in paragraph (e) of this section and may select from among the specific factors listed under each criterion.

(d) In determining the evaluation criteria to be used for unsolicited applications, the Secretary selects from among the evaluation criteria in paragraph (e) of this section, and may select from among the specific factors listed under each criterion, the criteria which are most appropriate to evaluate the activities proposed in the application.

(e) The Secretary establishes the following evaluation criteria:

(1) National significance. (i) The Secretary considers the national significance of the proposed project.

(ii) In determining the national significance of the proposed project, the Secretary may consider one or more of the following factors:

(A) The importance of the problem or issue to be addressed.

(B) The potential contribution of the project to increased knowledge or understanding of educational problems, issues, or effective strategies.

(D) The potential for generalizing from project findings or results.

(E) The potential contribution of the project to the development and advancement of theory and knowledge in the field of study.

(F) Whether the project involves the development or demonstration of creative or innovative strategies that build on, or are alternatives to, existing strategies.

(G) The nature of the products (such as information, materials, processes, or techniques) likely to result from the project and the potential for their effective use in a variety of other settings.

(H) The extent and quality of plans for disseminating results in ways that will allow others to use the information.

(2) Quality of the project design. (i) The Secretary considers the quality of the design of the proposed project.

(ii) In determining the quality of the design of the proposed project, the Secretary may consider one or more of the following factors:

(A) Whether the goals, objectives, and outcomes to be achieved by the project are clearly specified and measurable.

(B) Whether there is a conceptual framework underlying the proposed activities and the quality of that framework.

(C) Whether the proposed activities constitute a coherent, sustained program of research and development in the field, including a substantial addition to an ongoing line of inquiry.

(D) Whether a specific research design has been proposed, and the quality and appropriateness of that design, including the scientific rigor of the studies involved.

(E) The extent to which the research design includes a thorough, high-quality review of the relevant literature, a high-quality plan for research activities, and the use of appropriate theoretical and methodological tools, including those of a variety of disciplines, where appropriate.

(F) The quality of the demonstration design and procedures for documenting project activities and results.

(G) The extent to which development efforts include iterative testing of products and adequate quality controls.

(H) The likelihood that the design of the project will successfully address the intended, demonstrated educational need or needs.

(I) How well and innovatively the project addresses statutory purposes, requirements, and any priority or priorities announced for the program.

(J) The quality of the plan for evaluating the functioning and impact of the project, including the objectivity of the evaluation and the extent to which the methods of evaluation are appropriate to the goals, objectives, and outcomes of the project.

(3) Quality and potential contributions of personnel. (i) The Secretary considers the quality and potential contributions of personnel for the proposed project.

(ii) In determining the quality and potential contributions of personnel for the proposed project, the Secretary may consider one or more of the following factors:

(A) The qualifications, including training and experience, of the project director or principal investigator.

(B) The qualifications, including training and experience, of key project personnel.

(4) Adequacy of resources. (i) The Secretary considers the adequacy of resources for the proposed project.

(ii) In determining the adequacy of resources for the proposed project, the Secretary may consider one or more of the following factors:

(A) The adequacy of support from the lead applicant organization.

(B) The relevance and commitment of each partner in the project to the implementation and success of the project.

(D) Whether the costs are reasonable in relation to the objectives, design, and potential significance of the project.

(E) The potential for continued support of the project after Federal funding ends.

(5) Quality of the management plan.

(i) The Secretary considers the quality of the management plan of the proposed project.

(ii) In determining the quality of the management plan of a proposed project, the Secretary may consider one or more of the following factors:

(A) The adequacy of the management plan to achieve the objectives of the project, including the specification of staff responsibility, timelines, and benchmarks for accomplishing project tasks.

(B) The adequacy of plans for ensuring high-quality products and services.

(D) Whether time commitments of the project director or principal investigator and other key personnel are appropriate and adequate to meet project objectives.

(E) How the applicant will ensure that a diversity of perspectives are brought to bear in the operation of the project, including those of parents and teachers, where appropriate.

(F) How the applicant will ensure that persons who are otherwise eligible to participate in the project are selected without regard to race, color, national origin, gender, age, or disability.

(G) The adequacy of plans for widespread dissemination of project results and products in ways that will assist others to use the information.

(Approved by the Office of Management and Budget under control number 1850-0723) (Authority: 20 U.S.C. 6011(i)(2)(D)(ii))

§ 700.31 What additional evaluation criteria shall be used for grants and cooperative agreements?

In addition to the evaluation criteria established in § 700.30(e), the Secretary uses criteria or factors specified in the applicable program statute to evaluate applications for grants and cooperative agreements.

(Authority: 20 U.S.C. 6011(i)(2)(D)(ii))

§ 700.32 What evaluation criteria shall be used for contracts?

(a) The evaluation criteria to be considered in the technical evaluation of contract proposals are contained in the FAR at 48 CFR 15.605. The evaluation criteria that apply to an acquisition and the relative importance of those factors are within the broad discretion of agency acquisition officials.

(b) At a minimum, the evaluation criteria to be considered must include cost or price and quality. Evaluation factors related to quality are called technical evaluation criteria.

(1) Technical excellence.

(2) Management capability.

(3) Personnel qualifications.

(4) Prior experience.

(5) Past performance.

(6) Schedule compliance.

(Authority: 20 U.S.C. 6011(i)(2)(D)(ii))

Subpart E—Selection for Award

§ 700.40 How are grant and cooperative agreement applications selected for award?

(a) The Secretary determines the order in which applications will be selected for grants and cooperative agreement awards. The Secretary considers the following in making these determinations:

(1) An applicant's ranking.

(2) Recommendations of the peer reviewers with regard to funding or not funding.

(3) Information concerning an applicant's performance and use of funds under a previous Federal award.

(4) Amount of funds available for the competition.

(5) Any other information relevant to a priority or other statutory or regulatory requirement applicable to the selection of applications for new awards.

(b) In the case of unsolicited applications, the Secretary uses the procedures in EDGAR (34 CFR 75.222(d) and (e)).

(Authority: 20 U.S.C. 6022(i)(2)(D)(i))

§ 700.41 How are contract proposals selected for award?

Following evaluation of the proposals, the contracting officer shall select for award the offeror whose proposal is most advantageous to the Government considering cost or price and the other factors included in the solicitation.

(Authority: 20 U.S.C. 6011(i)(2)(D)(i))

Pt. 701 PART 701—STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES CARRIED OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT (OERI)—DESIGNATION OF EXEMPLARY AND PROMISING PROGRAMS Subpart A—General Sec. 701.1 What is the purpose of these standards? 701.2 What definitions apply? 701.3 Who is eligible to submit an educational program for review? 701.4 What must a program sponsor submit for review? 701.5 What are the procedures for submitting an educational program for review by an expert panel? Subpart B—Selection of Panel Members 701.10 How are panels established? 701.11 How is the membership of expert panels determined? Subpart C—The Expert Panel Review Process 701.20 How does an expert panel evaluate programs? 701.21 What is the difference between an exemplary and a promising program? 701.22 What criteria are used to evaluate programs for exemplary or promising designation? Authority:

20 U.S.C. 6011(i), unless otherwise noted.

Source:

62 FR 61430, Nov. 17, 1997, unless otherwise note.

Subpart A—General

§ 701.1 What is the purpose of these standards?

(a) The standards in this part implement section 941(d) of the Educational Research, Development, Dissemination, and Improvement Act of 1994.

(b) These standards are intended to provide quality assurance that educational programs designated by the U.S. Department of Education as either exemplary or promising have met criteria that will allow educators, professional organizations, and others to use these programs with confidence.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d))

§ 701.2 What definitions apply?

The following definitions apply to this part:

Assistant Secretary means the Assistant Secretary for the Office of Educational Research and Improvement.

Educational programs mean educational policies, research findings, practices, and products.

Program sponsor means a party submitting an educational program for designation by the Secretary as either promising or exemplary.

Secretary means the Secretary of the Department of Education or an official or employee of the Department acting for the Secretary under a delegation of authority.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d))

§ 701.3 Who is eligible to submit an educational program for review?

Any public or private agency, organization or institution, or an individual may submit an educational program for review.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d))

§ 701.4 What must a program sponsor submit for review?

(a) To have an educational program considered for designation as exemplary or promising, a sponsor must submit to the Secretary a description of the program, program materials, and a discussion of the program that is responsive to the criteria in § 701.22.

(b) Information submitted must include, to the extent relevant to the particular program,—

(1) A program abstract of 250 words or less;

(2) A description of the salient features of the program;

(3) A description of the program's philosophy and history;

(4) Site information, including demographics;

(5) A description of evaluation results;

(6) Funding and staffing information;

(7) Specific materials relevant to content and methods, as appropriate; and

(8) Organization name, address, telephone and fax numbers, e-mail address (if available), and contact person.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d)) Effective Date Note:

At 62 FR 61430, Nov. 17, 1997, § 701.4 was added, effective July 1, 1998. This section contains information collection and recordkeeping requirements that will not become effective until approval has been given by the Office of Management and Budget.

§ 701.5 What are the procedures for submitting an educational program for review by an expert panel?

(a) An applicant seeking the exemplary or promising designation for its educational program may submit its program at any time for consideration to the Assistant Secretary, who will assign the submitted program to the appropriate expert panel for review.

(b) The Assistant Secretary will periodically establish and announce in the Federal Register specific topic areas of high priority. Sponsors of educational programs in these areas will be invited to submit their programs for consideration.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d))

Subpart B—Selection of Panel Members

§ 701.10 How are panels established?

The Assistant Secretary selects individuals, based on their areas of expertise, to serve on expert panels in specific topic areas for the purpose of reviewing and evaluating educational programs and recommending, to the Secretary, those programs that should be designated as exemplary or promising.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d))

§ 701.11 How is the membership of expert panels determined?

(a) For the review of each program or group of programs, the Assistant Secretary establishes an expert panel. The membership of the expert panels will represent both the community of practice and the community of research.

(b) In establishing the membership of each expert panel, the Assistant Secretary—

(1) Selects individuals who have in-depth knowledge of the subject area or content of the program or group of programs to be evaluated;

(2) Selects at least one current teacher, principal, or other school-based or community-based professional;

(3) Selects at least one individual with expertise in evaluating educational programs;

(4) Ensures that no more than one-third of the panel members are employees of the Federal Government; and

(5) Ensures that each panel member does not have a conflict of interest, as determined in accordance with paragraph (c) of this section, with respect to any educational program the panel member is asked to review.

(c) Panel members are considered employees of the U.S. Department for the purposes of conflicts of interest analysis and are subject to the provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's policies used to implement those provisions.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d))

Subpart C—The Expert Panel Review Process

§ 701.20 How does an expert panel evaluate programs?

(a) Each panel member shall—

(1) Independently review each program based on the criteria in § 701.22;

(2) Provide written comments based on an analysis of the strengths and weaknesses of the program according to the criteria;

(3) Participate in site visits or other verification activities, if appropriate; and

(4) Participate in a meeting of the expert panel, if appropriate, to discuss the reviews.

(b) A panel may not eliminate an educational program from consideration based solely on the fact that the program does not have one specific type of supporting data, such as test scores.

(c) Each expert panel shall make a recommendation to the Secretary as to whether the program is exemplary, promising, or neither.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d))

§ 701.21 What is the difference between an exemplary and a promising program?

(a) In determining whether an educational program should be recommended as exemplary or promising, the panel shall consider—

(1) Whether, based on empirical data, the program is effective and should be designated as exemplary; or

(2) Whether there is sufficient evidence to demonstrate that the program shows promise for improving student achievement and should be designated as promising.

(b) The Secretary relies upon the judgment and expertise of peer reviewers, as established in § 701.11, to determine the nature and extent of evidence required to distinguish between promising and exemplary programs and to apply the four criteria established in § 701.22, and their own individual factors under each criterion in making this determination.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d))

§ 701.22 What criteria are used to evaluate programs for exemplary or promising designation?

The Secretary establishes the following evaluation criteria for expert panels to use in determining whether an educational program should be recommended as exemplary, promising, or neither:

(a) Evidence of success.

(b) Quality of the program.

(d) Replicability.

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and (E), 6041(d))

Pt. 702 PART 702—STANDARDS FOR CONDUCT AND EVALUATION OF ACTIVITIES CARRIED OUT BY THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT (OERI)—EVALUATION OF THE PERFORMANCE OF RECIPIENTS OF GRANTS, COOPERATIVE AGREEMENTS, AND CONTRACTS Subpart A—General Sec. 702.1 What is the purpose of these standards? 702.2 What activities must be evaluated by these standards? 702.3 What additional activities may be evaluated by these standards? 702.4 When is performance assessed under these standards? 702.5 What definitions apply? Subpart B—Selection of Peer Review Panels 702.10 What are the characteristics of peer reviewers? 702.11 What constitutes a conflict of interest for grants and cooperative agreements? 702.12 What constitutes a conflict of interest for contracts? 702.13 How are peer reviewers selected for panels? Subpart C—The Evaluation Process 702.21 How does a peer review panel evaluate the performance of a recipient? 702.22 What information does a peer review panel consider for an interim assessment? 702.23 What information does a peer review panel consider for a final assessment? 702.24 What evaluation criteria must be used for performance assessments? Authority:

20 U.S.C. 6011(i), unless otherwise noted.

Source:

63 FR 57573, Oct. 27, 1998, unless otherwise noted.

Subpart A—General

§ 702.1 What is the purpose of these standards?

(a) The standards in this part implement section 912(i) of the Educational Research, Development, Dissemination, and Improvement Act of 1994 (the Act).

(b) These standards establish criteria and a peer review process to provide recipients of OERI grants, cooperative agreements and contract awards with assessments of their projects.

(1) The purpose of the assessments is to provide feedback to recipients to improve the quality of funded activities and to provide information to OERI as it determines if a recipient of a multi-year award merits continuation funding.

(2) The criteria and peer review process are intended to address the statutory requirement that the research, development, and dissemination activities carried out by the recipients of grants from and contracts and cooperative agreements with the Office of Educational Research and Improvement (OERI) meet the highest standards of professional excellence.

(Authority: 20 U.S.C. 6011(i)(2)(F))

§ 702.2 What activities must be evaluated by these standards?

These standards apply to activities carried out by OERI using funds appropriated under section 912(m) of the Act including activities carried out by the following entities or programs:

(a) The National Education Research Institutes.

(b) The Office of Reform Assistance and Dissemination.

(d) The Regional Educational Laboratories.

(e) The Teacher Research Dissemination Demonstration Program.

(f) The Goals 2000 Community Partnerships Program.

(g) The National Educational Research Policy and Priorities Board.

(Authority: 20 U.S.C. 6011(i)(1))

§ 702.3 What additional activities may be evaluated by these standards?

The Secretary may apply these standards to other activities funded by the Department, as appropriate.

(Authority: 20 U.S. C 6011 (i)(1))

§ 702.4 When is performance assessed under these standards?

(a) The Secretary will assess the performance of recipients of OERI grants, contracts, and cooperative agreements subject to these standards during and at the conclusion of their period of performance.

(b) The Department requires at least one interim assessment by a peer review panel for all awards.

(c) The Assistant Secretary will approve and require more than one interim assessment when an award is identified, either by the initial interim review or by Department of Education staff monitoring the award, as having difficulty in achieving project objectives.

(d) A final assessment by a peer review panel is required for all awards.

(e) As used in this part—

(1) Interim assessment is any assessment conducted during a recipient's period of performance.

(2) Final assessment is one conducted at the conclusion of a recipient's period of performance.

(Authority: 20 U.S.C. 6011(i)(2)(F))

§ 702.5 What definitions apply?

(a) Definitions in the Educational Research, Development, Dissemination, and Improvement Act of 1994. The following terms used in this part are defined in 20 U.S.C. 6011(l)(1):

Development Dissemination Educational Research

(b) Definitions in the Education Department General Administrative Regulations. The following terms used in this part are defined in 34 CFR 77.1:

Application Award Department Grant Project Secretary

(c) Definitions in the Federal Acquisition Regulation. The following term used in this part is defined in 48 CFR Chapter 1: Contract Proposal.

(Authority: 20 U.S.C. 6011(i)(2)(F)

Subpart B—Selection of Peer Review Panels

§ 702.10 What are the characteristics of peer reviewers?

(a) The Assistant Secretary selects each peer reviewer. Each peer reviewer must have the necessary knowledge and expertise in the area of the project being reviewed to evaluate the performance of a recipient. This experience may include—

(1) Expert knowledge of subject matter in the area of the activities to be reviewed;

(2) Expert knowledge of theory or methods or both in the area of the activities to be reviewed;

(3) Practical experience in the area of the activities or type of institution or both to be reviewed;

(4) Knowledge of a broad range of education policies and practices;

(5) Experience in managing complex organizations; or

(6) Expertise and experience in evaluation theory and practice.

(b) Each peer reviewer must be free of conflict of interest, as determined in accordance with § 702.11 or § 702.12.

(c) The Assistant Secretary may solicit nominations for peer reviewers from professional associations, nationally recognized experts, and other sources.

(d) OERI and other Department staff who possess the qualifications in paragraphs (a) and (b) of this section may serve as peer reviewers only in exceptional circumstances as determined by the Assistant Secretary.

(Authority: 20 U.S.C. 6011(i)(2)(B))

§ 702.11 What constitutes a conflict of interest for grants and cooperative agreements?

A peer reviewer assessing the performance of the recipient of a grant from or cooperative agreement with OERI is considered an employee of the Department for the purposes of conflict of interest analysis. As an employee of the Department, the peer reviewer is subject to the provisions of 18 U.S.C. 208, 5 CFR 2635.502, and the Department's policies used to implement those provisions.

(Authority: 20 U.S.C. 6011(i)(2)(B))

§ 702.12 What constitutes a conflict of interest for contracts?

A peer reviewer assessing the performance of the recipient of a contract with OERI is considered an employee of the Department in accordance with the Federal Acquisition Regulation (FAR), 48 CFR 3.104-4(h)(2). As an employee of the Department, the peer reviewer is subject to the provisions of the FAR, 48 CFR Part 3, Improper Business Practices and Personal Conflict of Interest.

(Authority: 41 U.S.C. 423)

§ 702.13 How are peer reviewers selected for panels?

(a) The Assistant Secretary assigns peer reviewers to panels that conduct the performance assessments.

(b) The Assistant Secretary may establish panels by category of recipient, such as a panel to review the performance of all Regional Educational Laboratories. Each recipient is evaluated individually by reviewers who have been assigned to this type of panel.

(c) In establishing panels, the Assistant Secretary, to the greatest extent feasible, selects peer reviewers for each evaluation who represent a broad range of perspectives.

(Authority: 20 U.S.C. 6011(i)(2)(B))

Subpart C—The Evaluation Process

§ 702.21 How does a peer review panel evaluate the performance of a recipient?

(a) In each evaluation, a peer review panel—

(1) Considers relevant information about the recipient's performance, as described in §§ 702.22 and 702.23; and

(2) Makes judgments about the recipient's performance, using the criteria in § 702.24.

(b) Each peer reviewer prepares a report based on the reviewer's assessment of the quality of the project according to the evaluation criteria.

(c) After each peer reviewer has evaluated each project independently, the panel may be convened to discuss the strengths and weaknesses of the project. Each reviewer may then independently re-evaluate each project with appropriate changes made to the written report.

(d) The report of the interim assessment must include any recommendations the peer reviewer may have for improving the recipient's performance.

(e) The report of the final assessment must contain each peer reviewer's evaluative summary of the recipient's performance, from the beginning of the contract, grant, or cooperative agreement to its conclusion.

(Authority: 20 U.S.C. 6011(i)(2)(F))

§ 702.22 What information does a peer review panel consider for an interim assessment?

(a) Sources of information for the interim assessment must include—

(1) The original request for proposals or grant announcement and the contract proposal or grant application;

(2) Documentation of any changes in the work described in the contract, grant, or cooperative agreement, including reasons for the changes;

(3) Any progress reports delivered to the Department or made available to the public by the recipient;

(4) Examples of products delivered to the Department or made available to the public by the recipient;

(5) Any relevant reports written by OERI staff, including reports of site visits by OERI staff;

(6) Any performance evaluations conducted under the FAR or the Education Department General Administrative Regulations (34 CFR Part 75).

(7) Any relevant information provided by the recipient in response to Government Performance and Results Act (GPRA) (Pub. L. 103-62) requirements; and

(8) Any reports from program evaluations commissioned by the Department.

(b) Sources of information for the interim assessment may also include—

(1) A self-assessment, prepared by the recipient, addressing the criteria in § 702.24;

(2) One or more site visits by the peer review panel;

(3) One or more oral or written presentations to the panel by the recipient describing its performance; or

(4) Other information about the recipient's performance.

(Approved by the Office of Management and Budget under control number 1850-0746) (Authority: 20 U.S.C. 6011(i)(2)(F))

§ 702.23 What information does a peer review panel consider for a final assessment?

(a) Sources of information for the final assessment must include—

(1) The original request for proposals or application notice and the contract proposal or grant application, together with documentation of any changes in the work described in the proposal or application, including reasons for the changes;

(2) If consistent with the recipient's contract, grant, or cooperative agreement with OERI, a written report or oral presentation or both by the recipient summarizing its activities and accomplishments;

(3) Any relevant information provided by the recipient in response to Government Performance and Results Act (GPRA) (Pub. L. 103-62) requirements;

(4) Any reports from program evaluations commissioned by the Department; and,

(5) Any relevant information provided by the interim assessment.

(b) The final assessment may also include other sources of information, such as one or more of those listed in § 702.22.

(Approved by the Office of Management and Budget under control number 1850-0746) (Authority: 20 U.S.C. 6011(i)(2)(F))

§ 702.24 What evaluation criteria must be used for performance assessments?

(a) Peer reviewers (and those recipients who conduct self-evaluations) shall use the criteria in paragraph (b) of this section to assess performance and, in case of interim assessments, to identify areas in which the performance of recipients may need improvement.

(b) The following evaluation criteria are to guide the assessment process undertaken by peer reviewers. The peer reviewers determine the extent to which recipients meet these criteria:

(1) Implementation and management. (i) Peer reviewers shall consider the degree to which the recipient has fully executed its program of work. In doing so, peer reviewers shall consider evidence on the extent to which the recipient completes the work described in the approved application or contract, including any approved modifications, in the time period proposed and in an efficient manner.

(ii) In examining the degree of implementation, peer reviewers may also consider evidence on the extent to which—

(A) The recipient implements and utilizes a quality assurance system for its products or services or both; and

(B) The recipient conducts self-assessment or self-evaluation activities, including periodically seeking out independent critiques and evaluations of its work, and uses the results to improve performance.

(2) Quality. (i) Peer reviewers shall consider the degree to which the recipient's work approaches or attains professional excellence. In determining quality, peer reviewers shall consider evidence on the extent to which—

(A) The recipient utilizes processes, methods, and techniques appropriate to achieve the goals and objectives for the program of work in the approved application; and

(B) The recipient applies appropriate processes, methods, and techniques in a manner consistent with the highest standards of the profession.

(ii) In determining quality, peer reviewers may also consider the extent to which the recipient conducts a coherent, sustained program of work informed by relevant research.

(3) Utility. (i) In determining the utility of the recipient's products or services or both, peer reviewers shall consider evidence on the extent to which the recipient's work (including information, materials, processes, techniques, or activities) is effectively used by and is useful to its customers in appropriate settings.

(ii) In determining utility, peer reviewers may also consider the extent to which the recipient has received national recognition; e.g., articles in refereed journals and presentations at professional conferences.

(4) Outcomes and impact. (i) Peer reviewers shall consider the results of the recipient's work. In examining outcomes and impact, peer reviewers shall consider evidence on the extent to which—

(A) The recipient meets the needs of its customers; and

(B) The recipient's work contributes to the increased knowledge or understanding of educational problems, issues, or effective strategies.

(ii) In examining outcomes and impact, peer reviewers may also consider the extent to which recipients address issues of national significance through its products or services or both.

(c) For National Research and Development Centers, peer reviewers also shall consider evidence on the extent to which recipients meet the following criteria:

(1) Quality. (i) The recipient uses a well-conceptualized framework and sound theoretical and methodological tools in conducting professionally rigorous studies; and

(ii) The recipient conducts work of sufficient size, scope, and duration to produce sound guidance for improvement efforts and future research.

(2) Utility. The recipient documents, reports, and disseminates its work in ways to facilitate the effective use of its work in appropriately targeted settings.

(3) Outcomes and impact. (i) The recipient's work contributes to the development and advancement of theory in the field of study, including its priority area; and

(ii) The recipient addresses issues of national significance through its products or services or both.

(d) For the Regional Educational Laboratories, peer reviewers also shall consider evidence on the extent to which recipients meet the following criteria:

(1) Quality. (i) The recipient utilizes a well-conceptualized framework and sound theoretical and methodological tools in conducting professionally rigorous studies;

(ii) The recipient conducts work of sufficient size, scope, and duration to produce sound guidance for improvement efforts; and

(iii) The recipient's products are well tested and based on sound research.

(2) Utility. The recipient documents, reports, and disseminates its work in ways to facilitate its effective use in appropriately targeted settings, particularly in school improvement efforts of States and localities.

(3) Outcomes and impact. (i) The recipient assists States and localities to implement comprehensive school improvement strategies through the provision of research-based information (including well-tested models and strategies), materials and assistance; and

(ii) The recipient's work results in widespread access to information regarding research and best practices, particularly within its region.

(e) For Field-Initiated Studies, peer reviewers also shall consider evidence on the extent to which recipients meet the following criteria:

(1) Implementation and management. The recipient's work responds to the goals, objectives and mission of the National Institute from which it is funded.

(2) Quality. The recipient utilizes a well-conceptualized framework and sound theoretical and methodological tools in conducting professionally rigorous studies.

(3) Utility. The recipient documents, reports, and disseminates its work in ways to facilitate its effective use in appropriately targeted settings.

(4) Outcomes and impact. (i) The recipient's work contributes to the development and advancement of theory and knowledge in the field of study; and

(ii) The recipient addresses issues of national significance through its products.

(f) For the ERIC Clearinghouses, peer reviewers also shall consider evidence on the extent to which recipients meet the following criteria:

(1) Quality. The recipient applies an integrated approach to acquiring and disseminating significant and high-quality educational literature and materials to maintain and enhance the ERIC database.

(2) Utility. The recipient contributes to the development of the ERIC database as a source of literature and materials that reflects trends and issues within its scope.

(3) Outcomes and impact. (i) The recipient meets the informational and educational needs of its customers through dissemination and outreach approaches and the development of an array of print and non-print materials; and

(ii) The recipient provides national leadership on the use of current computer, networking, and information technology.

(Approved by the Office of Management and Budget under control number 1850-0746) (Authority: 20 U.S.C. 6011(i)(2)(F))