[Federal Register Volume 77, Number 241 (Friday, December 14, 2012)]
[Proposed Rules]
[Pages 74392-74407]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2012-29897]
========================================================================
Proposed Rules
Federal Register
________________________________________________________________________
This section of the FEDERAL REGISTER contains notices to the public of
the proposed issuance of rules and regulations. The purpose of these
notices is to give interested persons an opportunity to participate in
the rule making prior to the adoption of the final rules.
========================================================================
Federal Register / Vol. 77, No. 241 / Friday, December 14, 2012 /
Proposed Rules
[[Page 74392]]
DEPARTMENT OF EDUCATION
34 CFR Parts 75 and 77
RIN 1890-AA14
[Docket ID ED-2012-OII-0026]
Direct Grant Programs and Definitions That Apply to Department
Regulations
AGENCY: Department of Education.
ACTION: Notice of proposed rulemaking.
-----------------------------------------------------------------------
SUMMARY: The Secretary proposes to amend the regulations in 34 CFR
parts 75 and 77 of the Education Department General Administrative
Regulations (EDGAR) in order to improve the Department's ability to
promote projects supported by evidence; evaluate the performance of
discretionary grant programs and grantee projects; review grant
applications using selection factors that promote reform objectives
related to project evaluation, sustainability, productivity, and
capacity to scale; and reduce burden on grantees in selecting
implementation sites, implementation partners, or evaluation service
providers for their proposed projects. These proposed changes would
allow the Department to be more effective and efficient when selecting
discretionary grantees, provide higher-quality data to Congress and the
public, and better focus applicants on the particular goals and
objectives of the programs to which they apply for grants.
DATES: We must receive your comments on or before February 12, 2013.
ADDRESSES: Submit your comments through the Federal eRulemaking Portal
or via postal mail, commercial delivery, or hand delivery. We will not
accept comments by fax or by email. Please submit your comments only
one time, in order to ensure that we do not receive duplicate copies.
In addition, please include the Docket ID at the top of your comments.
Federal eRulemaking Portal: Go to http://www.regulations.gov to submit your comments electronically. Information
on using Regulations.gov, including instructions for accessing agency
documents, submitting comments, and viewing the docket, is available on
the site under ``How To Use This Site.''
Postal Mail, Commercial Delivery, or Hand Delivery. If you
mail or deliver your comments about these proposed regulations, address
them to Margo Anderson, U.S. Department of Education, 400 Maryland
Avenue SW., Room 4W313, Washington, DC 20202-5900.
Privacy Note: The Department's policy for comments received from
members of the public is to make these submissions available for
public viewing in their entirety on the Federal eRulemaking Portal
at www.regulations.gov. Therefore, commenters should be careful to
include in their comments only information that they wish to make
publicly available.
FOR FURTHER INFORMATION CONTACT: Erin McHugh, U.S. Department of
Education, 400 Maryland Avenue SW., Room 4W319, Washington, DC 20202.
Telephone: (202) 401-1304 or by email: erin.mchugh@ed.gov.
If you use a telecommunications device for the deaf (TDD) or text
telephone (TTY), call the Federal Relay Service (FRS) toll free at 1-
800-877-8339.
SUPPLEMENTARY INFORMATION:
Invitation to Comment on Proposed Regulations
We invite you to submit comments regarding these proposed
regulations. To ensure that your comments have maximum effect in
developing the final regulations, we urge you to identify clearly the
specific section or sections of the proposed regulations that each of
your comments addresses and to arrange your comments in the same order
as the proposed regulations.
We invite you to assist us in complying with the specific
requirements of Executive Order 12866 and 13563 and their overall
requirement of reducing regulatory burden that might result from these
proposed regulations. Please let us know of any ways we could reduce
potential costs or increase potential benefits while preserving the
effective and efficient administration of the Department's programs and
activities.
Because Executive Order 12866 and the presidential memorandum on
``Plain Language in Government Writing'' require each agency to write
regulations that are easy to understand, we invite you to comment on
how to make these proposed regulations easier to understand, including
answers to questions such as the following:
Are the requirements in the proposed regulations clearly
stated?
Do the proposed regulations contain technical terms or
other wording that interferes with their clarity?
Does the format of the proposed regulations (grouping and
order of sections, use of headings, paragraphing, etc.) aid or reduce
their clarity?
Would the proposed regulations be easier to understand if
we divided them into more (but shorter) sections? (A ``section'' is
preceded by the symbol ``Sec. '' and a numbered heading; for example,
Sec. 75.210 General selection criteria.
Could the description of the proposed regulations in this
preamble be more helpful in making the proposed regulations easier to
understand? If so, how?
What else could we do to make the proposed regulations
easier to understand?
Retrospective Review of EDGAR
On January 21, 2011, President Obama issued Executive Order 13563,
``Improving Regulation and Regulatory Review'' (76 FR 3821). The order
requires all Federal agencies to ``consider how best to promote
retrospective analysis of rules that may be outmoded, ineffective,
insufficient, or excessively burdensome, and to modify, streamline,
expand, or repeal them in accordance with what has been learned.''
Accordingly, on August 22, 2011, the Department issued its Plan for
Retrospective Analysis of Existing Regulations. (See ed.gov/policy/gen/reg/retrospective-analysis/index.html).
Our plan identified a number of regulatory initiatives for
retrospective review and analysis. One of those initiatives, already
begun in 2010, was a review of the Department's discretionary grants
process. Part of that initiative was a close retrospective review of
the Education Department General Administrative Regulations (EDGAR),
which govern discretionary grantmaking and administration.
As part of this retrospective review of EDGAR, we identified key
provisions that required substantive changes to improve transparency
and the efficiency and effectiveness of our grant-making
[[Page 74393]]
functions. These included our regulations on establishing and
collecting data on measures of grantee performance, the selection
criteria that peer reviewers use to evaluate applications, and the
procedures grantees must use to select research sites and evaluators.
This notice is the result of the Department's regulatory review of
those provisions.
On May 10, 2012, President Obama issued Executive Order 13610,
``Identifying and Reducing Regulatory Burdens.'' (77 FR 28469). Among
other things and as part of their retrospective review, this order
requires Federal agencies to invite ``public suggestions about
regulations in need of retrospective review and about appropriate
modifications to such regulations.''
Therefore, in addition to your comments on the specific regulations
proposed in this notice, we seek input on other regulations within
EDGAR that may be in need of modification and amendments to those
regulations that you would suggest. We are particularly interested in
your feedback on the following questions:
Are the regulations achieving their intended outcomes,
e.g., do they establish a fair and equitable process for selecting
applications for funding while ensuring transparency in the selection
process and enhancing accountability for funding decisions?
Have changes in the economy or other external factors led
to an increase or decrease in costs imposed on applicants for, and
recipients of, discretionary grants?
Are any of the regulations outmoded, unnecessary, or out
of date?
Do the regulations cause confusion or create other
questions? If so, how could we amend the regulations to address this
problem?
What do relevant data show about the effectiveness and
benefits of the regulations in comparison to their costs?
Although the Department may or may not respond to comments that we
receive on the retrospective review of these other provisions of EDGAR,
we will use that feedback to further inform and plan our retrospective
review efforts.
During and after the comment period, you may inspect all public
comments about these proposed regulations by accessing Regulations.gov.
You may also inspect the comments, in person, in Room 4W335, 400
Maryland Avenue SW., Washington, DC, between the hours of 8:30 a.m. and
4:00 p.m., Washington, DC time, Monday through Friday of each week
except Federal holidays. Please contact the person listed under FOR
FURTHER INFORMATION CONTACT.
Assistance to Individuals With Disabilities in Reviewing the Rulemaking
Record
On request, we will provide an appropriate accommodation or
auxiliary aid to an individual with a disability who needs assistance
to review the comments or other documents in the public rulemaking
record for these proposed regulations. If you want to schedule an
appointment for this type of aid, please contact the person listed
under FOR FURTHER INFORMATION CONTACT.
Summary of Proposed Changes
In this notice, the Secretary proposes amendments that would:
1. Allow the Secretary, in an application notice for a competition,
to establish performance measurement requirements;
2. Revise requirements regarding project evaluations submitted to
the Department by grantees;
3. Authorize applicants to use simplified procurement procedures to
select implementation sites and procure services from implementation
and evaluation service providers, but only if the site or service
provider is named in the grant application;
4. Allow the Secretary, through an announcement in the Federal
Register, to authorize grantees under particular programs to award
subgrants to directly carry out programmatic activities. The
subgrantees and programmatic activities must be identified and
described in the grantees' applications;
5. Add one new selection criterion and revise two existing criteria
that the Department could choose to use to evaluate applications. The
new criterion would be used to assess the extent to which a proposed
project could be brought to scale. We would add five new factors to the
criterion ``Quality of the Project Evaluation'' that could be used to
assess how well a proposed project evaluation would produce evidence
about the project's effectiveness. Finally, we would revise one factor
and add five new factors to the criterion ``Quality of the Project
Design'';
6. Authorize program offices to consider the effectiveness of
proposed projects under a new priority that could be used as either an
absolute, competitive preference, or invitational priority; and
7. Allow the Secretary to fund data collection periods after the
end of the substantive work of a project so that project outcomes could
be assessed using data from the entire project period.
As discussed in more detail later in this notice, the proposed
changes would strengthen the Secretary's authority to: (a) Evaluate
grantee performance; (b) provide applicants and grantees with greater
flexibility in selecting implementation sites, implementation partners,
and evaluation service providers; (c) allow the Secretary to authorize
subgrants for particular programs; (d) improve the targeting of
selection criteria and factors so that applicants are better informed
and able to focus their application narratives on specific program
objectives; and (e) allow consideration of the strength of evidence
supporting the proposed project when evaluating grant applications.
Significant Proposed Regulations
We group major issues according to subject, with appropriate
sections of the proposed regulations referenced in parentheses. We
discuss other substantive issues under the sections of the proposed
regulations to which they pertain.
Generally, we do not address proposed regulatory provisions that
are technical or otherwise minor in effect.
I. Performance Measurement
Background
Congress passed the Government Performance and Results Act of 1993
(GPRA) (Pub. L. 103-62) in order to hold Federal agencies accountable
for achieving program results. Under GPRA, agencies are required to
report to Congress on the effectiveness of the programs they
administer, based on performance measures established for those
programs.
The purposes of GPRA are to improve Federal program effectiveness
and accountability to the public by: Focusing on results, service
quality, and customer satisfaction; giving Federal program managers
information about program results and service quality; and providing
objective information to Congress and the public on the relative
effectiveness and efficiency of Federal programs and spending. The GPRA
Modernization Act of 2010 (Pub. L. 111-352) supports additional
improvements in Federal agencies' performance planning and reporting.
Federal agencies are required to make their strategic and annual plans
publicly available and post quarterly updates via a central,
Government-wide Web site. The goal of the GPRA Modernization Act is to
improve the use of data in
[[Page 74394]]
policy, budget, and management decision-making.
GPRA requires Federal agencies to establish performance measures
and targets for programs they administer and to report annually to the
Office of Management and Budget (OMB) on the extent to which those
programs are meeting their targets. For discretionary (non-formula)
grant programs, the Department establishes performance measures to
address the extent to which the program as a whole is effective in
achieving its goals through the projects it funds. However, we have
found that grantees' performance data do not consistently correspond to
overall program performance measures because grantees typically only
report on and measure data related to project-specific outcomes.
The Secretary therefore proposes the following amendments to
improve the Department's ability to collect reliable, valid, and
meaningful data for evaluating the outcomes of Department programs and
the performance of individual grantees.
Proposed Regulatory Changes
34 CFR Part 75
Section 75.110 Information Regarding Performance Measurement
Current Regulations: None.
Proposed Regulations: Proposed Sec. 75.110 would allow the
Secretary to establish performance measurement requirements in an
application notice for a competition. These requirements could include
performance measures, baseline data, performance targets, and
performance data. This proposed section would also allow the Secretary
to establish in an application notice a requirement that applicants
propose performance measures for their projects, as well as the
baseline data and performance targets for each proposed measure.
Reasons: To improve the likelihood that grantees collect and report
data that effectively measure the outcomes of each grant, the proposed
amendments would allow the Secretary to require discretionary grant
applicants to include program-level and project-specific performance
measures, baseline data, and targets in their applications. Requiring
this information improves the ability of the Department to measure
program effectiveness under GPRA performance measures, clarifies that
grantees will be required to report on their project-specific
performance measures, and stresses that the extent to which grantees
meet performance targets will be considered in making continuation
grants.
II. Procurement and Subgrant Processes for Entities Named in
Applications
Proposed Regulatory Changes
34 CFR Part 75
Background:
From our experience, many applicants find it useful to describe
elements of their proposed evaluations in their applications, including
implementation sites or the provider that would conduct the project
evaluation should the proposed project be funded. This information is
often an important factor in the Department's peer review of
discretionary grant applications, particularly in instances when the
quality of the project evaluation is a selection criterion.
The Department's procurement regulations in 34 CFR 74.43 and 34 CFR
80.36(c) provide that a grantee must conduct its procurement
transactions in a manner that provides, to the maximum extent
practical, full and open competition. This requirement is intended to
ensure that grantees consider contractor performance objectively and
offer an opportunity for providers to compete for the contract. While
the Department values full and open competition, the Department also
recognizes that this requirement presents challenges for applicants
whose applications would be strengthened by including details about the
implementation sites and the evaluation service provider. The Secretary
proposes to reduce this burden by simplifying the procurement process
used to select implementation sites, implementation partners, and
evaluation service providers.
Section 75.135 Competition Exception for Implementation Sites,
Implementation Partners, or Evaluation Service Providers
Current Regulations: There is no current Sec. 75.135. The
Department's procurement regulations in 34 CFR 74.43 and 34 CFR
80.36(c) provide that a grantee must conduct its procurement
transactions in a manner that provides full and open competition. In
many cases, grantees must use formal competition procedures to select
contractors. Under these current provisions, an applicant for a grant
requiring an evaluation would need, in many cases, to conduct a formal
bidding process to select implementation sites, implementation
partners, or evaluation service providers before submitting its
application to the Department or following award of the grant. These
types of procurement requirements can be very costly and time consuming
at a time when the applicant cannot be sure it will be selected for a
grant. Because the selection of implementation sites or partners and
evaluation service providers is often an important factor in designing
a project and submitting a high-quality application, we propose an
exception to the Department's procurement regulations for entities
named in a grant application.
Proposed Regulations: The Secretary proposes to add a new Sec.
75.135 that would exempt certain applicants from the full competitive
contracting requirements in 34 CFR 74.43 and 80.36(c). Specifically, an
applicant for a grant that must be conducted at multiple sites or that
requires an external evaluation would not be required to comply with
the applicable formal competition requirements in 34 CFR 74.43 and
80.36(c) when entering into a contract if--
(1) The contract is with an entity that agrees to provide a site or
sites where the applicant would conduct the project activities under
the grant or the contract is with the evaluation service provider that
would conduct the project evaluation;
(2) The implementation sites, implementation partners, or
evaluation service providers are identified in the application for the
grant; and
(3) The implementation sites, implementation partners, or
evaluation service providers are included in the application in order
to meet a regulatory, statutory, or priority requirement related to the
competition.
A successful applicant would need to certify that any employee,
officer, or agent participating in the selection, award, or
administration of a contract is free of any real or apparent conflict
of interest.
In the case of a contract for a provider to conduct the project
evaluation, the proposed amendment would permit the applicant or
grantee to use the informal competition requirements for small
purchases that are currently applicable only to governments under 34
CFR 80.36(d)(1), regardless of whether the applicant or grantee is a
government entity and regardless of whether the purchase meets the
small purchase threshold.
During the course of the grant, a successful applicant would be
required to obtain the Department's permission to change any
implementation site, implementation partner, or evaluation service
provider that the applicant specified in the application and selected
under proposed Sec. 75.135. The exception also would not relieve an
applicant of the obligation to conduct an informal review of evaluation
service providers
[[Page 74395]]
in order to determine the best available provider or from its
obligations under the Department's other procurement requirements.
A successful applicant that does not meet the three criteria above
would not be exempt from complying with the applicable formal
competition requirements in 34 CFR 74.43 and 80.36(c) when entering
into a contract. For example, an applicant that does not identify its
implementation sites, implementation partners, or evaluation service
provider in its application would be required to comply with the
applicable formal competition requirements in 34 CFR 74.43 and
80.36(c).
Reasons: This proposed new section addresses a difficulty many
applicants face when selecting implementation sites, partners, and
evaluation service providers prior to submitting their applications.
Requiring grantees to use formal competitive procedures to select
implementation sites and partners could significantly diminish both the
ability of many applicants to compete for grants and the quality of
project evaluations. For example, without this proposed regulation, a
successful applicant would be limited in its ability to select
implementation sites that include specific populations that it proposed
to serve through the project or to work with the evaluation service
provider that assisted in designing the applicant's evaluation plan.
Formal competition requirements also inhibit the ability of many
applicants to select evaluators who would work with the applicants to
design project evaluation plans. Some of the best evaluations of
projects may be conducted by evaluation service providers that are
involved in the initial design of a project. Such work generally takes
place during the development of an application, before the applicant
knows whether it will receive a grant. Thus, requiring an applicant to
hold a formal competition involving sealed bids or competitive
proposals in order to select an evaluation service provider (either
before or after it receives a grant) can have major negative
consequences. For example, an evaluation service provider would be
excluded from the competition to select the project evaluator under the
procurement requirements in parts 74 and 80 if it helped prepare an
application and helped the applicant set up the standards used to
select an evaluation service provider or contractor (see 34 CFR 74.42,
74.43, and 80.36). High-quality evaluation of a project funded by the
Department may be hindered if an evaluation service provider that
designed the evaluation strategy for an application is excluded from
the evaluation procurement competition for that project. Given the
uncertainty of the competitive process, the Secretary also believes
that applicants should not be required to use formal competition
procedures to select an evaluation service provider at the time they
prepare their applications.
While the Secretary proposes to remove the competition requirement
for selecting sites and implementation partners and thus permit
applicants and grantees to use informal procedures instead, the
Secretary would continue to require all applicants to comply with the
other procurement requirements in parts CFR 74 and 80, including the
requirements for cost price analysis, standards of conduct, conflicts
of interest, and the prohibition of contingent payment for services.
Additionally, the proposed amendment does not supersede any State laws
regarding procurement.
Finally, based on the other procurement requirements in CFR parts
74 and 80, these exceptions would not relieve an applicant of its
responsibility to document that it made genuine efforts to select the
best implementation sites, implementation partners, or evaluation
service providers for the project, considering qualifications,
capabilities, availability, price, and other important factors.
Sec. 75.708 Prohibition on Subgrants.
Current Regulations: Section 75.708(a) prohibits grantees from
awarding subgrants unless specifically authorized by statute.
Proposed Regulations: The Secretary proposes to revise the
prohibition on subgrants in Sec. 75.708(a) to allow subgrants when
authorized by statute or as provided for by a new Sec. 75.708(b).
Under this proposed new Sec. 75.708(b), the Secretary could, through
an announcement in the Federal Register, authorize subgrants when
necessary to meet the purposes of a particular program. In addition,
the Federal Register announcement would identify the types of entities
(e.g., State or local educational agencies, institutions of higher
education, or non-profit organizations) that could receive subgrants
under the program.
We would add Sec. 75.708(c) to provide that subgrants, if
authorized under Sec. 75.708(b), could be awarded to entities
identified in a grantee's application. The subgrant must be used to
directly carry out activities described in the application.
We would add a new Sec. 75.708(d), which would establish
requirements grantees would have to follow in awarding subgrants
authorized under Sec. 75.708(b). We would re-designate the current
Sec. 75.708(b) as Sec. 75.708(e).
Reasons: The revision of Sec. 75.708(a) is necessary to provide
grantees with flexibility to work with partners or other entities to
carry out project activities. The prohibition on subgrants, in
conjunction with the requirement on full and open competition for
procurement transactions in 34 CFR 74.43 and 34 CFR 80.36(c), unduly
restricts grantees from working with partners or other entities
identified in their applications as being directly responsible for
carrying out project-related activities.
In order to ensure appropriate subgranting by Department grantees,
our proposed revision authorizes subgrants only when approved by the
Secretary for a particular program and only to the types of entities
(e.g., State or local educational agencies, institutions of higher
education, or non-profit organizations) designated by the Secretary. In
addition the proposed revision would limit the entities that may
receive subgrants to those that: (1) Are identified in a grantee's
application, or (2) are competitively selected using the grantee's
procedures for selecting subgrants and, (3) will use the subgrant
directly to carry out project activities described in the grantee's
application. In all cases where a grantee is working with an
organization or entity that is not identified in its application, not
selected through a competitive process, or not an organization or
entity directly responsible for carrying out an activity or activities
described in the grantee's application, the grantee would be required
to follow the procurement procedures set out in 34 CFR Parts 74 and 80.
Additionally, the grantee--as the fiscal agent--would remain
responsible to the Department for the proper use of all grant funds,
including those subgranted to another entity.
In addition, we would add a new Sec. 75.708(d) requiring grantees
to ensure that: (1) Subgrants are awarded on the basis of an approved
budget that is consistent with the grantee's approved application and
all applicable Federal statutory, regulatory, and other requirements;
(2) subgrants include all conditions required by Federal law; and (3)
subgrantees are aware of requirements imposed upon them by Federal law,
including the Federal anti-discrimination laws enforced by the
Department.
This revision provides grantees, in programs and to entities
designated by the Secretary, with the flexibility to award subgrants in
specific
[[Page 74396]]
circumstances where necessary to ensure proper implementation of an
approved project without diminishing accountability for Federal funds
or project outcomes.
III. Selection Criteria
Background
The regulations in subpart D of 34 CFR part 75 set forth the
general requirements that govern the Department's selection of grantees
for direct grant awards. For those direct grant programs that make
discretionary grant awards, the Secretary uses selection criteria to
evaluate applications submitted under those programs. The regulations
specify certain selection criteria from which the Secretary may choose
(general EDGAR criteria). They allow the Secretary to use program-
specific selection criteria and the general EDGAR selection criteria,
as well as to develop other criteria based on the statutory provisions
for the funding program. However, some program regulations currently do
not provide that the Secretary may use program-specific selection
criteria in conjunction with EDGAR and statutory criteria. The
regulations also describe how the Secretary determines which criteria
and which factors within those criteria are used in a particular
competition and how the Secretary may weight the criteria and factors.
As we have managed competitions under the general regulations
governing selection criteria, we have found that some of the
regulations on the selection of grantees do not provide the Department
the discretion it needs, absent a lengthy rulemaking process, to
conduct grant competitions closely aligned with Department,
legislative, and program objectives and priorities that can change from
year to year in response to new and unanticipated circumstances. These
proposed regulations, therefore, would provide the Department
additional flexibility to establish criteria based on program
regulations, in addition to the current authority to base criteria on
statutory provisions. The proposed regulations would also specifically
authorize program offices to establish additional selection criteria in
Sec. 75.210 based on statutory and regulatory provisions.
These proposed regulations would also add new selection factors
under the ``Quality of Project Design'' criterion on organizational and
programmatic sustainability and organizational productivity. The
proposed regulations would also add to the ``Quality of the Project
Evaluation'' criterion five new selection factors on the types of
evidence the evaluation designs would produce on the performance and
implementation of the project. Finally, the proposed regulations would
establish a new criterion to evaluate the extent to which an applicant
proposes a project that could be brought to scale.
The addition of these selection factors would ensure that the
Department's discretionary grant programs would more effectively
promote the development and implementation of effective and sustainable
practices, and support adoption and implementation of necessary
reforms. These proposed regulations would not change the way the
Secretary uses the current and new selection criteria and factors. The
Secretary would continue to use those selection criteria and factors
that are consistent with the purpose of the program and permitted under
the applicable statutes and regulations.
Proposed Regulatory Changes
34 CFR Part 75
Section 75.209 Selection Criteria Based on Statutory or Regulatory
Provisions
Current Regulations: Current Sec. 75.209 provides that the
Secretary may evaluate applications by establishing selection criteria
based on the statutory provisions for the authorized program. These
provisions include, but are not limited to, those related to specific
statutory selection criteria, allowable activities, application content
requirements, and other pre-award and post-award conditions.
Proposed Regulations: We propose to revise Sec. 75.209 to allow
the Secretary to use selection criteria, the factors in program
regulations, and those based on program statute, along with the
selection criteria in Sec. 75.210 (often referred to as the EDGAR
selection criteria) to produce more focused selection criteria. Thus,
Sec. 75.209 would allow the Secretary to establish selection criteria,
and factors for considering those criteria, based on statutory or
regulatory provisions that apply to the authorized program, which may
include, but are not limited to, criteria and factors that reflect:
Criteria contained in the program statute or regulations;
Criteria in Sec. 75.210;
Allowable activities specified in the program statute or
regulations;
Application content requirements specified in the program
statute or regulations;
Program purposes, as described in the program statute or
regulations; or
Other pre-award and post-award conditions specified in the
program statute or regulations.
Reasons: The Secretary proposes amending this section so that the
Department can establish selection criteria based both on a program's
statute and regulations. Program regulations are used to help clarify
and fill in the gaps of more general statutory requirements and provide
further detail about authorized activities for a program.
Under this proposed amendment, the Secretary would be able to use
the more specific regulatory provisions to establish selection criteria
that are focused more closely on the intended outcomes of the
competition and, thereby, help applicants to structure their
applications so as to more accurately and concisely describe how they
will achieve those outcomes. In addition to providing for establishment
of criteria based on program regulations, this amendment would allow
the Secretary to use a combination of criteria from the program
statute, its established regulations, or the general selection criteria
in Sec. 75.210.
Sec. 75.210 General Selection Criteria
Current Regulations: Current Sec. 75.210 contains a list of eight
selection criteria: ``Need for Project'' in paragraph (a);
``Significance'' in paragraph (b); ``Quality of the Project Design'' in
paragraph (c); ``Quality of Project Services'' in paragraph (d);
``Quality of Project Personnel'' in paragraph (e); ``Adequacy of
Resources'' in paragraph (f); ``Quality of the Management Plan'' in
paragraph (g); and ``Quality of the Project Evaluation'' in paragraph
(h). Under each of these selection criteria, the Secretary may select
from a number of factors to focus each criterion.
Proposed Regulations: The Secretary proposes to revise the
introductory paragraph of Sec. 75.210, add selection factors to the
criteria in Sec. 75.210(c) and (h), and add a new criterion as
paragraph (i) to address the ability of an applicant to bring a project
to scale.
Introductory Text: We propose to amend the introductory paragraph
of Sec. 75.210 so that the Secretary may select factors that could be
considered under a criterion both from the factors listed under that
criterion and factors listed under other criteria. For example, the
proposed amendment would allow the Secretary to establish ``Quality of
the Project Design'' as a selection criterion and include selection
factors from ``Need for Project'' (Sec. 75.210(a)) or ``Significance''
(Sec. 75.210(b)) in the factors that will be considered under the
``Quality of the Project Design''
[[Page 74397]]
criterion. Currently, to use a single selection factor under the ``Need
for Project'' criterion, the Department, in most cases, would need to
include the ``Need for Project'' criterion, even if the factor in
question could be appropriately grouped with factors from another
selection criterion, such as ``Significance.''
Reasons: The purpose of this amendment is to provide the Secretary
with the flexibility to choose and combine selection factors
established in Sec. 75.210 under various selection criteria. This
would enable the Department to align the selection criteria and factors
with the goals and objectives of a particular discretionary grant
competition in a more coherent and effective fashion than is currently
permitted. Selection criteria and factors that are concise and are
aligned as closely as possible with the goals and objectives of a
particular grant competition would more effectively guide applicants in
preparing clearer and more focused applications that in turn can be
more effectively evaluated and rated by peer reviewers. The current
regulations, by contrast, do not allow this close focus. Including a
greater number of selection criteria in application notices, solely to
include particular selection factors, makes it more likely that
applications will be less focused and more difficult for peer reviewers
to accurately evaluate and score.
Current Regulations: Section 75.210(c) establishes the selection
criterion ``Quality of the Project Design.'' The Secretary may consider
one or more of the 23 factors listed under this criterion in
determining the quality of the project design, including the extent to
which the project design will build capacity that extends beyond the
project period and establish linkages to services provided by other
programs.
Proposed Regulations: The Secretary proposes to add new factors to
the criterion in paragraph (c), (xxiv and xxv) relating to the
sustainability of the proposed project after the end of the project
period.
Reasons: Adding these selection factors would help ensure that the
Department's discretionary grant programs effectively promote the
development and implementation of effective and sustainable practices
and support adoption and implementation of necessary reforms. By
promoting the development of a multi-year plan for incorporation into
the applicant's ongoing work, the proposed factors would better
encourage applicants to develop sustainability plans than do the
related selection factors in current Sec. 75.210(c). The proposed
factors also would allow the Secretary to consider a proposed project's
potential for sustainability over time, including the extent to which
the project has the support of various stakeholders and adequate
resources to continue the project after the grant period ends.
Proposed Regulations: The Secretary proposes to revise Sec.
75.210(c) to add a new selection factor (xxvi) regarding the extent to
which the proposed project will increase efficiency in the use of time,
staff, money, or other resources in order to improve results and
increase productivity.
Reasons: Current Sec. 75.210(c) does not include a factor that
promotes increased productivity. Considering the budget challenges that
State and local educational agencies, institutions of higher education,
non-profit organizations, and other entities working in education face
during economic downturns, and given the potential for new knowledge
and capabilities to improve efficiency, the Department believes that it
is appropriate to consider the potential for increasing productivity,
i.e., the extent to which a proposed project includes a strategy to
make more efficient use of time, money, and staff, when assessing an
application.
Proposed Regulations: The Secretary proposes to revise Sec.
75.210(c)(xvi) to read ``The extent to which the proposed project will
integrate with or build on similar or related efforts to improve
relevant outcomes, using existing funding streams from other programs
or policies supported by community, State, and Federal resources.''
In addition, the Secretary proposes to add a new selection factor
(xxvii) regarding the extent to which the proposed project will
integrate with or build on similar or related efforts in order to
improve relevant outcomes, using nonpublic funds or resources.
Reasons: Given the budget challenges facing State and local
educational agencies, institutions of higher education, and other
entities working in education, there is a need for strategies and
practices to improve relevant outcomes while controlling costs.
Moreover, ``silos'' within and between agencies at the local, State,
and Federal levels often impede program integration and result in less
efficient and effective efforts. The purpose of revising this selection
factor and adding a factor focused on nonpublic investments is to
improve levels of program integration, to facilitate shared agendas for
actions focused on common outcomes, and to leverage public and private
sector investments in education. In addition, the Department believes
that leveraging existing programs and policies that are supported by
other funds, including other Federal, State, local, or private funds,
increases the likelihood that selected projects will be sustained
beyond the grant period.
Proposed Regulations: The Secretary proposes to add two new
selection factors (xxviii and xxix) regarding the extent to which the
proposed project is supported by evidence of promise or strong theory.
Later in this notice, we propose adding definitions to Part 77,
including evidence of promise, strong theory, and other terms to ensure
consistent understanding of the selection factors we propose in this
notice.
Reasons: The Department recognizes that at the various stages of a
proposed project's development, different types of evidence are
available to assess the effectiveness of a project. The proposed
selection factors would permit the Secretary to use strength of
evidence as a selection factor in determining the projects the
Department will fund while maintaining the flexibility to consider a
wider variety of studies or data an applicant might present that is
appropriate to the goals of the project. The flexibility provided by
the proposed selection factor would be particularly beneficial for
innovative areas where strong or moderate evidence of effectiveness is
not yet available because it would allow the Secretary to consider
strength of evidence appropriate to a project's stage of development.
Current Regulations: Section 75.210(h) establishes the selection
criterion ``Quality of the Project Evaluation.'' The Secretary may
consider one or more of the seven factors listed under this criterion
in determining the quality of the project evaluation design, such as
the extent to which the project proposes feasible and appropriate
evaluation methods, uses objective performance measures, and permits
periodic assessment.
Proposed Regulations: The Secretary proposes to revise Sec.
75.210(h) to add five new selection factors. Two of the new selection
factors address the extent to which the methods of evaluation will, if
well-implemented, produce evidence about the project's effectiveness
that would meet the What Works Clearinghouse Evidence Standards.\1\ The
other three proposed selection factors address the extent to which the
methods of evaluation will produce
[[Page 74398]]
evidence of promise about the grant-supported intervention, valid and
reliable performance data on relevant outcomes of the project, and the
extent to which the evaluation plan articulates key components as well
as measureable thresholds for acceptable implementation of the project.
---------------------------------------------------------------------------
\1\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
Reasons: Although current Sec. 75.210(h) includes selection
factors regarding proposed evaluation methods, it does not include a
selection factor that promotes use of the strongest possible study
designs for estimating a program's effect or a selection factor that
assesses the extent to which the proposed evaluation will articulate
information that can be used to assess whether the project was
implemented with fidelity.
Linking two of the proposed new selection factors to the What Works
Clearinghouse Evidence Standards \2\ reflects the predominant view
among research experts that the randomized controlled trial (also
referred to as an experimental design study) is the most rigorous and
defensible method for producing unbiased evidence of project
effectiveness. Random assignment of entities (students, teachers,
schools, or other units of analysis) to a treatment or control group is
the most effective way to eliminate plausible competing explanations
for observed differences between treated and non-treated individuals or
groups (i.e., the estimated treatment effect). Adding these selection
factors will allow the Secretary to consider the extent to which
applicants propose evaluations that will contribute to a strong body of
evidence on the effectiveness of the proposed project.
---------------------------------------------------------------------------
\2\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
Additionally, the other three proposed selection factors allow the
Secretary to consider evaluation methods that will produce data on a
project's evidence of promise, performance on relevant outcomes, and
fidelity of implementation. Each of these factors would improve the
Department's ability to assess evaluation plans for projects at various
stages of their development.
Current Regulations: None.
Proposed Regulations: The Secretary proposes to add selection
criterion Sec. 75.210(i), ``Strategy to Scale.'' ``Scale'' refers to
expanding the use or implementation of a proposed practice, strategy,
or program to provide services at a State, regional, or national level
while maintaining the demonstrated effectiveness of the approach. Under
the proposed new criterion, the Secretary would consider the
applicant's strategy to scale the proposed project. In determining the
applicant's strategy to scale the proposed project, the Secretary would
consider one or more factors, including the extent to which the
applicant's strategy to scale addresses a particular barrier or
barriers that prevented the applicant, in the past, from reaching the
level of scale that is proposed in the application; the applicant's
capacity (e.g., in terms of qualified personnel, financial resources,
or management capacity) to bring the proposed project to scale; and the
extent to which the applicant demonstrates there is unmet demand for
the proposed project that will enable the applicant to reach the level
of scale that is projected in the application. In addition, the
Secretary could consider the feasibility of replicating the project and
the mechanisms for broadly disseminating information on the project so
as to support further development or replication.
Reasons: It is important that successful best practices be shared
and implemented more broadly. The addition of this selection criterion
would allow the Secretary to consider the proposed scaling methodology
and the feasibility of successfully replicating the proposed project in
a variety of settings and with other populations. The proposed
selection criterion would allow the Department to consider whether
applicants have the potential to serve more groups in a variety of
settings, which would be important in estimating the likelihood of a
proposed project's success at scale and in considering applications for
activities that include broad sharing of best practices. Additionally,
Department programs could use the proposed criterion, in conjunction
with the proposed priority regarding evidence of effectiveness, to
encourage the field to focus its attention and resources on projects
that are effective.
IV. Evidence of Effectiveness
Background
To support effective projects and provide incentives to the field
for building an evidence base on the effectiveness of interventions,
the Secretary proposes a priority for projects that can cite and build
upon an existing base of strong or moderate evidence of effectiveness.
This priority would be a critical part of the Department's efforts to
fund and increase the use of programs with evidence of effectiveness.
Section 75.226 Consideration of Applications Supported by Strong or
Moderate Evidence of Effectiveness
Current Regulations: None.
Proposed Regulations: The Secretary proposes to establish
procedures for giving special consideration to applications supported
by strong or moderate evidence of effectiveness. Proposed Sec. 75.226
would establish that if the Secretary determines to give special
consideration to applications supported by strong or moderate evidence
of effectiveness for a particular grant competition, the Secretary
could either establish a separate competition or give a competitive
preference to applications supported by strong or moderate evidence of
effectiveness under the procedures in 34 CFR 75.105(c)(2).
Reasons: By expanding the number of Department programs awarding
grants to those projects supported by strong or moderate evidence of
effectiveness, the Department could better ensure that discretionary
grant funds are used to support effective interventions and activities.
V. Program Budgets
Background
So that the Department can learn as much as possible from
successful discretionary grants and its programs as a whole, we propose
amendments regarding budget periods. We would:
Establish that a project may receive an extension of the
funding period for the purpose of collecting, analyzing, and reporting
performance data;
Clarify that a multi-year data collection may be funded
through separate budget periods; and
Clarify that any information relevant to the grantee's
performance during the project period should be considered when
determining whether a grantee receives a continuation award.
Proposed Regulatory Changes
34 CFR Part 75
Section 75.250 Maximum Funding Period
Current Regulations: Current Sec. 75.250 is titled ``Project
period can be up to 60 months.'' This section provides that the
Secretary may approve a project period of up to 60 months, but it does
not specifically authorize funding grants for periods longer than 60
months. Other regulations in part 75 prohibit the use of Federal funds
for projects extending past 60 months. See current Sec. 75.261, which
addresses the circumstances under which a grantee may request a no-cost
extension of a project period.
[[Page 74399]]
Proposed Regulations: We propose to amend Sec. 75.250 to provide
that the Secretary may approve a data collection period of up to 72
months--if not inconsistent with any statutory limits on the grant
award period--after the end of the project period and provide funding
during this period for the sole purpose of collecting, analyzing, and
reporting data regarding project performance. During a data collection
period, a grantee could use the funds only for data collection,
analysis, and reporting purposes. Section 75.250(b)(2) would give the
Secretary discretion to notify applicants in the notice inviting
applications for a competition or later, after grantees have started
their projects of the intent to fund data collection periods.
Given these proposed changes, the Secretary also proposes to change
the title of this section to ``Maximum funding period.''
Reasons: It is the Department's experience that the effectiveness
of a project cannot always be determined on the date that the
substantive work of the project is complete. For example, a four-year
project designed to increase the ability of certain high school
students to successfully complete college may require data collection
for up to six years after the date the substantive work of the project
ends. With the discretion to approve a data collection period after the
end of a project period and offer continued funding for data
collection, the Department could ensure that performance data are
collected and are used to evaluate both the project and program
performance. The Secretary would expect to fund any data collection
period of a grant at a much lower level than the original substantive
work of the grant.
Section 75.251 The Budget Period
Current Regulations: Current Sec. 75.251 describes how the
Secretary may fund multi-year projects through separate budget periods,
generally of 12 months each.
Proposed Regulations: The Secretary proposes to add a new paragraph
(c) to this section to clarify that multi-year data collection periods
may be funded through budget periods in the same manner as project
periods are funded.
Reasons: We are proposing to revise Sec. 75.251 to correspond to
the proposed revisions to Sec. 75.250.
Section 75.253 Continuation of a Multi-Year Project After the First
Budget Period
Current Regulations: Under current Sec. 75.253(a), a grantee may
only receive a continuation award if the grantee has met certain
requirements, including the requirement that the grantee make
substantial progress toward the objectives of the grant. If a grantee
does not make substantial progress, it must obtain permission from the
Department to make changes to the project that would help the grantee
make substantial progress during the remainder of the project period.
Proposed Regulations: The Secretary proposes to amend Sec. 75.253
by adding a new paragraph (b) to clarify that in deciding whether to
make a continuation grant, the Secretary could consider any information
relevant to the grantee's performance during the project period. This
could include information relevant to the authorizing statute, a
criterion, a priority, or a performance measure, or any financial or
other requirement that applied to the selection of applications for new
awards. While this proposed standard for granting continuation awards
is implicit under the current regulations, the Secretary believes that
this standard should be explicit so that grantees have a clearer
understanding of how the Department decides to make a continuation
award.
In addition, we propose to amend paragraph (a)(2) so that in making
continuation awards, the Secretary could consider not only the extent
to which a grantee has made substantial progress in achieving the goals
and objectives of the project, but also whether a grantee met the
performance targets in the approved application, if the Secretary
established performance measurement requirements for the grant in the
application notice. If a grantee fails to meet these targets, proposed
paragraph (a)(2) would require the grantee to obtain the Secretary's
approval for changes to the project that enable the grantee to achieve
the project's goals, objectives, and performance targets, if any,
without changing the project scope or objectives. The Secretary would
retain the requirement in the current regulation that any such changes
may not increase the amount of funds obligated to the project by the
Department.
Reasons: Current Sec. 75.253 does not describe the standards used
to determine whether a grantee has made substantial progress on its
grant. Therefore, we propose these amendments to clarify the standards
that the Department considers when determining whether a grant will
receive a continuation award. The proposed amendments would establish
that the Secretary may also consider whether a grantee has met the
performance targets in its approved application when making
continuation awards.
Section 75.590 Evaluation by the Grantee
Current Regulation: Current Sec. 75.590 requires a grantee to
submit performance reports to the Department that evaluate at least
annually the grantee's progress in achieving the objectives in its
approved application, the effectiveness of the project in meeting the
purposes of the program, and the effect of the project on participants
being served by the project. This provision does not currently provide
any standards for evaluating the progress in achieving performance
targets.
Proposed Regulation: The Secretary proposes to revise Sec. 75.590
to add a new paragraph (a) to provide that if an application notice for
a competition requires applicants to describe how they would evaluate
their projects, any evaluation must meet the standards set in the
approved application for the project. The performance measurement data
collected by the grantee and used in the evaluation must meet the
performance measurement requirements in the approved application.
We also propose to designate the current regulatory text in Sec.
75.590 as new paragraph (b) and revise that text to conform to the
other changes we are proposing regarding performance measurement.
Specifically, we propose that if the application notice for a
competition did not require an applicant to submit an evaluation plan,
the grantee must provide information in its performance report to the
Department demonstrating (1) The progress made by the grantee in the
most recent budget period; (2) the effectiveness of the project; and
(3) the effect of the project on the participants served by the
project. If the application notice required applicants to propose how
they would meet performance requirements, the performance report would
also need to address the extent to which the grantee met the project's
performance targets and other performance measurement requirements for
the budget period addressed by the performance report.
Reasons: The proposed revisions to Sec. 75.590 strengthen the
Department's authority to monitor the quality of grantees' project
evaluations. Additionally, these revisions complement other proposed
regulations in this notice regarding performance measurement
requirements.
[[Page 74400]]
VI. Definitions
Background
These proposed regulations include references to terms that are not
currently defined in EDGAR. To ensure a common understanding of these
terms, we propose establishing the following definitions.
Proposed Regulatory Changes
34 CFR Part 77
Section 77.1 Definitions That Apply to All Department Programs
Current Regulations: Section 77.1(c) establishes definitions that
unless a statute or regulation provides otherwise, apply to parts 34
CFR 74 and 80.
Proposed Regulations: The Secretary proposes to incorporate the
definitions for the following terms into Sec. 77.1(c): ``ambitious,''
``baseline data,'' ``evidence of promise,'' ``large sample,'' ``logic
model,'' ``moderate evidence of effectiveness,'' ``multi-site sample,''
``national level,'' ``performance measure,'' ``performance target,''
``randomized controlled trial,'' ``regional level,'' ``relevant
outcome,'' ``quasi-experimental study,'' ``strong evidence of
effectiveness,'' and ``strong theory.''
Reasons: The Secretary proposes establishing these definitions to
ensure consistent understanding of the selection factors and priority
we propose in this notice.
Executive Orders 12866 and 13563
Regulatory Impact Analysis
Under Executive Order 12866, the Secretary must determine whether
this regulatory action is ``significant'' and, therefore, subject to
the requirements of the Executive order and subject to review by the
Office of Management and Budget (OMB). Section 3(f) of Executive Order
12866 defines a ``significant regulatory action'' as an action likely
to result in a rule that may--
(1) Have an annual effect on the economy of $100 million or more,
or adversely affect a sector of the economy, productivity, competition,
jobs, the environment, public health or safety, or State, local, or
tribal governments or communities in a material way (also referred to
as an ``economically significant'' rule);
(2) Create serious inconsistency or otherwise interfere with an
action taken or planned by another agency;
(3) Materially alter the budgetary impacts of entitlement grants,
user fees, or loan programs or the rights and obligations of recipients
thereof; or
(4) Raise novel legal or policy issues arising out of legal
mandates, the President's priorities, or the principles stated in the
Executive Order.
This proposed regulatory action is a significant regulatory action
subject to review by OMB under section 3(f) of Executive Order 12866.
We have also reviewed these regulations under Executive Order
13563, which supplements and explicitly reaffirms the principles,
structures, and definitions governing regulatory review established in
Executive Order 12866. To the extent permitted by law, Executive Order
13563 requires that an agency--
(1) Propose or adopt regulations only upon a reasoned determination
that their benefits justify their costs (recognizing that some benefits
and costs are difficult to quantify);
(2) Tailor its regulations to impose the least burden on society,
consistent with obtaining regulatory objectives and taking into
account--among other things and to the extent practicable--the costs of
cumulative regulations;
(3) In choosing among alternative regulatory approaches, select
those approaches that maximize net benefits (including potential
economic, environmental, public health and safety, and other
advantages; distributive impacts; and equity);
(4) To the extent feasible, specify performance objectives, rather
than the behavior or manner of compliance a regulated entity must
adopt; and
(5) Identify and assess available alternatives to direct
regulation, including economic incentives--such as user fees or
marketable permits--to encourage the desired behavior, or provide
information that enables the public to make choices.
Executive Order 13563 also requires an agency ``to use the best
available techniques to quantify anticipated present and future
benefits and costs as accurately as possible.'' The Office of
Information and Regulatory Affairs of OMB has emphasized that these
techniques may include ``identifying changing future compliance costs
that might result from technological innovation or anticipated
behavioral changes.''
We are issuing these regulations only on a reasoned determination
that their benefits justify their costs. In choosing among alternative
regulatory approaches, we selected those approaches that maximize net
benefits. Based on the analysis that follows, the Department believes
that these regulations are consistent with the principles in Executive
Order 13563.
We also have determined that this regulatory action would not
unduly interfere with State, local, and tribal governments in the
exercise of their governmental functions.
In accordance with both Executive orders, the Department has
assessed the potential costs and benefits of this regulatory action.
The potential costs associated with this regulatory action are those
resulting from statutory requirements and those we have determined as
necessary for administering the Department's programs and activities.
Summary of Potential Costs and Benefits
Under the proposed regulations, applicants would have to use
performance measures, baseline data, and performance targets
established by the Department or establish their own performance
measures, and determine baseline data performance targets for each
performance measure. Although these proposed regulations would
explicitly require such determinations and data collections, these
requirements are implicit under the current regulations and grantees
are already required to report on the extent to which they are meeting
performance targets under the performance report ED 524B, which is
approved under OMB control number 1894-0003. Therefore, we do not
expect an increase in reporting burden on grantees under the proposed
amendments.
The benefits of the proposed regulations would be that the
Department would have explicit authority to collect meaningful data
that we could use to assess the success of individual projects and
report to Congress and OMB about the success of Department programs in
achieving their legislative objectives. The ability to determine the
success of Department programs could help improve the effectiveness of
Department programs, without imposing additional costs on grantees or
other parties.
The proposed regulations would also permit the Department to
provide an exception for certain applicants from the full competitive
contracting requirements in 34 CFR 74.43 and 80.36(c) for a grant that
requires an external evaluation. Additionally, the proposed regulations
would remove the prohibition on subgrants and allow for subgrants to
any entity that is identified in a grantee's application and uses the
subgrant directly to carry out activities described in the application.
This action would reduce costs and increase benefits.
The benefits are that the proposed rule would remove a barrier for
these grantees to contracting with the same evaluator both in the grant
application
[[Page 74401]]
stage and after receiving a grant award (and similarly, to selecting
evaluation sites and implementation partners both pre-grant award and
post-award), and thereby potentially enhance the quality of these
projects. At the same time the proposed regulations would relieve
grantees of the costs of administering competitions without reducing
accountability or increasing the risk of improper use of or accounting
for grant expenditures.
Additionally, under the proposed regulations, the Department would
have greater flexibility in conducting grant competitions to use
selection criteria that (1) are closely aligned with program objectives
and priorities, and (2) promote reform objectives related to project
evaluation, sustainability, productivity, and capacity to scale. This
change would benefit applicants as well as the Department because it
allows the Secretary to establish selection criteria that are concise
and closely aligned with the goals and objectives of a particular grant
competition and are focused more closely and coherently on the intended
outcomes of the competition. The regulations would generate these
benefits without increasing the costs for applicants, grantees, or the
Department that already exist for creating and reviewing grant
applications.
Elsewhere in this section under the heading Paperwork Reduction Act
of 1995, we identify and explain burdens specifically associated with
information collection requirements.
Clarity of the Regulations
Executive Order 12866 and the Presidential memorandum on ``Plain
Language in Government Writing'' require each agency to write
regulations that are easy to understand.
The Secretary invites comments on how to make these proposed
regulations easier to understand, including answers to questions such
as the following:
Are the requirements in the proposed regulations clearly
stated?
Do the proposed regulations contain technical terms or
other wording that interferes with their clarity?
Does the format of the proposed regulations (grouping and
order of sections, use of headings, paragraphing, etc.) aid or reduce
their clarity?
Would the proposed regulations be easier to understand if
we divided them into more (but shorter) sections? (A ``section'' is
preceded by the symbol ``Sec. '' and a numbered heading; for example,
Sec. 75.210 General selection criteria.)
Could the description of the proposed regulations in the
SUPPLEMENTARY INFORMATION section of this preamble be more helpful in
making the proposed regulations easier to understand? If so, how?
What else could we do to make the proposed regulations
easier to understand?
To send any comments that concern how the Department could make
these proposed regulations easier to understand, see the instructions
in the ADDRESSES section of this preamble.
Regulatory Flexibility Act Certification
Paperwork Reduction Act of 1995
The Secretary certifies that these proposed regulations would not
have a significant economic impact on a substantial number of small
entities because the proposed regulations would affect only slightly
the overall burden on applicants and grantees, as explained in the
Paperwork Reduction Act of 1995 discussion in this SUPPLEMENTARY
INFORMATION.
Paperwork Reduction Act of 1995
As part of its continuing effort to reduce paperwork and respondent
burden, the Department conducts a preclearance consultation program to
provide the general public and Federal agencies with an opportunity to
comment on proposed and continuing collections of information in
accordance with the Paperwork Reduction Act of 1995 (PRA) (44 U.S.C.
3506(c)(2)(A)). This helps ensure that: The public understands the
Department's collection instructions, respondents can provide the
requested data in the desired format, reporting burden (time and
financial resources) is minimized, collection instruments are clearly
understood, and the Department can properly assess the impact of
collection requirements on respondents.
Section 75.110 contains an information collection requirement.
Under the PRA the Department has submitted a copy of this section to
OMB for its review.
A Federal agency may not conduct or sponsor a collection of
information unless OMB approves the collection under the PRA and the
corresponding information collection instrument displays a currently
valid OMB control number. Notwithstanding any other provision of law,
no person is required to comply with, or is subject to penalty for
failure to comply with, a collection of information if the collection
instrument does not display a currently valid OMB control number.
In the final regulations we will display the control number
assigned by OMB to any information collection requirement proposed in
this NPRM and adopted in the final regulations.
Collection of Information: The proposed regulations would affect
applicants and grantees of the Department's discretionary grant
programs, including State, local, and tribal governments and non-profit
organizations, such as institutions of higher education.
Applications: OMB has approved the Department's Generic Application
Package under OMB Control number 1894-0006, which applies to those
competitions that use the current EDGAR selection criteria in Sec.
75.210 and statutory criteria that have been developed under the EDGAR
procedures in Sec. 75.209.
Regarding the burden imposed by the Generic Application Package,
the Department proposes to add proposed Sec. 75.110 to the other
sections already identified as creating burden related to that package.
While Sec. 75.110 is new, it would not impose any new data collection
requirements for the Generic Application Package because performance
measurement burden for that package has already been calculated under
the selection criteria in Sec. 75.210. The amendments proposed in this
NPRM would not increase the existing paperwork burdens under the
Generic Application Package. The Secretary also proposes to cover the
burden associated with the EDGAR selection criteria from Sec. 75.209
and Sec. 75.210 under Sec. 75.200, which fully details the sources
that program offices can use to establish selection criteria under
EDGAR.
Each fiscal year, the Department receives over XX,000 applications
under competitions covered by the Generic Application Package.
Applicants that apply to programs that use the EDGAR criteria would be
affected by the proposed changes to the selection criteria that would
require applications to address evaluation and performance measurement
more specifically.
The Department already has selection criteria that ask applicants
to describe the evaluation plans for their projects; the burden
associated with the proposed regulations is currently covered under
Sec. 75.210(h). However, an applicant for a discretionary grant would
only have to respond once to provide the following information
regarding the project: The performance baselines; the performance
measures; the performance targets; and the methodology for collecting
performance data. Thus, we do not expect greater burden under these
proposed regulations and the Generic Application Package because that
burden is already covered under existing criteria. Instead, we expect
that
[[Page 74402]]
as a result of these proposed regulations, applicants would provide
greater clarity on the methodologies they would use to collect and
report data.
Because these proposed regulations would expand the number of
programs that could use proposed Sec. 75.209 to create criteria based
on statutory and regulatory requirements, there is a potential under
the proposed regulations that more program offices would use the EDGAR
process to establish criteria for their competitions. If more
competitions use the Generic Application Package, the overall hours of
burden under the Generic Application Package and OMB Control number
1894-0006 would grow. However, any ``new'' burden covered by the
Generic Application Package would result from fewer programs using
program-specific application packages, so the total burden covered by
program-specific application packages would be reduced in an amount
equivalent to the burden increase associated with the Generic
Application Package. If the amendments to the sections regarding the
selection criteria become final, we would work closely with OMB to
monitor the extent to which burden currently covered by separate
program-specific application packages would shift to the Generic
Application Package and request appropriate changes in the total burden
covered by the Generic Application Package.
The current Generic Application Package was approved by OMB based
on an estimate of 9,861 responses over three years and an estimate of
447,089 total hours required to prepare applications.
Performance reports: OMB has also approved the U.S. Department of
Education Grant Performance Report (ED 524B) under OMB Control number
1894-0003.
Over three years, the Department receives ED 524B performance
reports from approximately 5,900 discretionary grantees. A grantee
would have to respond on an annual basis to prepare performance reports
throughout the course of the project period, including any no-cost
extensions of the grant or funded data collection extensions, and
respond once to prepare a final performance and financial report. These
burdens have already been accounted for under the ED 524B.
The number of reports estimated annually under the ED 524B is 5,900
and the estimated reporting burden-hours for that report is 132,200. We
do not expect any change in burden under these proposed regulations.
However, there is some potential that more programs might be able to
use the ED 524B performance report as a result of more programs using
the EDGAR selection criteria. We will monitor that potential and work
with OMB to determine if the Department needs to revisit the total
burden covered by the ED 524B performance report.
Intergovernmental Review
These proposed regulations affect Direct Grant programs of the
Department that are subject to Executive Order 12372 and the
regulations in 34 CFR part 79. One of the objectives of the Executive
order is to foster an intergovernmental partnership and a strengthened
federalism. The Executive order relies on processes developed by State
and local governments for coordination and review of proposed Federal
financial assistance.
This document provides early notification of our specific plans and
actions for these programs.
Assessment of Educational Impact
In accordance with section 411 of the General Education Provisions
Act, 20 U.S.C. 1221e-4, the Secretary particularly requests comments on
whether these proposed regulations would require transmission of
information that any other agency or authority of the United States
gathers or makes available.
Accessible Format: Individuals with disabilities can obtain this
document in an accessible format (e.g., braille, large print,
audiotape, or compact disc) on request to the program contact person
listed under FOR FURTHER INFORMATION CONTACT.
Electronic Access to This Document: The official version of this
document is the document published in the Federal Register. Free
Internet access to the official edition of the Federal Register and the
Code of Federal Regulations is available via the Federal Digital System
at: www.gpo.gov/fdsys. At this site you can view this document, as well
as all other documents of this Department published in the Federal
Register, in text or Adobe Portable Document Format (PDF). To use PDF
you must have Adobe Acrobat Reader, which is available free at the
site.
You may also access documents of the Department published in the
Federal Register by using the article search feature at:
www.federalregister.gov. Specifically, through the advanced search
feature at this site, you can limit your search to documents published
by the Department.
List of Subjects
34 CFR Part 75
Accounting, Copyright, Education, Grant programs-education.
34 CFR Part 77
Education, Grant programs-education.
Dated: December 6, 2012.
Arne Duncan,
Secretary of Education.
For the reasons discussed in the preamble, the Secretary proposes
to amend parts 75 and 77 of title 34 of the Code of Federal Regulations
as follows:
PART 75--DIRECT GRANT PROGRAMS
1. The authority citation for part 75 continues to read as follows:
Authority: 20 U.S.C. 1221e-3 and 3474, unless otherwise noted.
2. Add a new Sec. 75.110 to read as follows:
Sec. 75.110 Information regarding performance measurement.
(a) The Secretary may establish in an application notice for a
competition one or more performance measurement requirements, including
requirements for performance measures, baseline data, or performance
targets, and a requirement that applicants propose in their
applications one or more of their own performance measures, baseline
data, or performance targets.
(b) If an application notice requires applicants to propose
project-specific performance measures, baseline data, or performance
targets, the application must include the following, as required by the
application notice:
(1) Performance measures. How each proposed performance measure
would accurately measure the performance of the project and how the
proposed performance measure would be consistent with the performance
measures established for the program funding the competition.
(2) Baseline data. (i) Why each proposed baseline is valid; or
(ii) If the applicant has determined that there are no established
baseline data for a particular performance measure, an explanation of
why there is no established baseline and of how and when, during the
project period, the applicant would establish a valid baseline for the
performance measure.
(3) Performance targets. Why each proposed performance target is
ambitious yet achievable compared to the baseline for the performance
measure and when, during the project period, the applicant would meet
the performance target.
[[Page 74403]]
(c) If the application notice establishes performance measurement
requirements, the applicant must also describe in the application--
(1)(i) The data collection and reporting methods the applicant
would use and why those methods are likely to yield reliable, valid,
and meaningful performance data; and
(ii) If the Secretary requires applicants to collect data after the
substantive work of a project is complete regarding the attainment of
certain performance targets, the data collection and reporting methods
the applicant would use during the post-performance period and why
those methods are likely to yield reliable, valid, and meaningful
performance data.
(2) The applicant's capacity to collect and report reliable, valid,
and meaningful performance data, as evidenced by high-quality data
collection, analysis, and reporting in other projects or research.
(Authority: 20 U.S.C. 1221e-3 and 3474)
3. Add a new undesignated center heading ``Competition Exceptions''
in subpart C immediately before the undesignated center heading ``State
Comment Procedures''.
4. Add a new Sec. 75.135 to subpart C under the undesignated
center heading ``Competition Exceptions'' to read as follows:
S75.135 Competition exception for proposed implementation sites,
implementation partners, or evaluation service providers.
(a) When entering into a contract with implementation sites or
partners, an applicant is not required to comply with the competition
requirements in 34 CFR 74.43 or 80.36(c), as applicable, if--
(1) The contract is with an entity that agrees to provide a site or
sites where the applicant would conduct the project activities under
the grant;
(2) The implementation sites or partner entities that the applicant
proposes to use are identified in the application for the grant; and
(3) The implementation sites or partner entities are included in
the application in order to meet a regulatory, statutory, or priority
requirement related to the competition.
(b) When entering into a contract for data collection, data
analysis, or evaluation services, an applicant may select a provider
using the informal, small-purchase procurement procedures in 34 CFR
80.36(d)(1), regardless of whether that applicant would otherwise be
subject to that part or whether the evaluation contract would meet the
standards for a small purchase order, if--
(1) The contract is with the data collection, data analysis, or
evaluation service provider that would conduct the project services;
(2) The evaluation service provider that the applicant proposes to
use is identified in the application for the grant; and
(3) The evaluation service provider is included in the application
in order to meet a statutory, regulatory, or priority requirement
related to the competition.
(c) If the grantee relied on the exceptions under paragraph (a) or
(b) of this section, the grantee must certify that any employee,
officer, or agent participating in the selection, award, or
administration of a contract is free of any real or apparent conflict
of interest.
(d) A grantee must obtain the Secretary's prior approval for any
change to an implementation site, implementation partner, or evaluation
service provider, if the grantee relied on the exceptions under
paragraph (a) or (b) of this section to select the entity or evaluator.
(e) The exceptions in paragraphs (a) and (b) of this section do not
extend to the other procurement requirements in 34 CFR part 74 and 34
CFR part 80 regarding contracting by grantees and subgrantees.
(Authority: 20 U.S.C. 1221e-3 and 3474)
5. Revise Sec. 75.209 to read as follows:
Sec. 75.209 Selection criteria based on statutory or regulatory
provisions.
The Secretary may establish selection criteria and factors based on
statutory or regulatory provisions that apply to the authorized
program, which may include, but are not limited to criteria and factors
that reflect--
(a) Criteria contained in the program statute or regulations;
(b) Criteria in Sec. 75.210;
(c) Allowable activities specified in the program statute or
regulations;
(d) Application content requirements specified in the program
statute or regulations;
(e) Program purposes, as described in the program statute or
regulations; or
(f) Other pre-award and post-award conditions specified in the
program statute or regulations.
(Authority: 20 U.S.C. 1221e-3 and 3474)
6. Amend Sec. 75.210 by:
A. Revising the introductory text.
B. Revising paragraph (c)(2)(xvi).
C. Adding paragraphs (c)(2)(xxiv) through (xxix).
D. Adding paragraphs (h)(2)(viii) through (xii).
And
E. Adding a new paragraph (i).
The revisions and additions read as follows.
Sec. 75.210 General selection criteria.
In determining the selection criteria to evaluate applications
submitted in a grant competition, the Secretary may select one or more
of the following criteria and may select from among the list of
optional factors under each criterion. The Secretary may define a
selection criterion by selecting one or more specific factors within a
criterion or assigning factors from one criterion to another criterion.
* * * * *
(c) * * *
(2) * * *
(xvi) The extent to which the proposed project will integrate with
or build on similar or related efforts to improve relevant outcomes (as
defined in 34 CFR 77.1(c)), using existing funding streams from other
programs or policies supported by community, State, and Federal
resources.
* * *
(xxiv) The extent to which the applicant demonstrates that it has
the resources to operate the project beyond the length of the grant,
including a multi-year financial and operating model and accompanying
plan; the demonstrated commitment of any partners; evidence of broad
support from stakeholders (e.g., State educational agencies, teachers'
unions) critical to the project's long-term success; or more than one
of these types of evidence.
(xxv) The potential and planning for the incorporation of project
purposes, activities, or benefits into the ongoing work of the
applicant beyond the end of the grant.
(xxvi) The extent to which the proposed project will increase
efficiency in the use of time, staff, money, or other resources in
order to improve results and increase productivity.
(xxvii) The extent to which the proposed project will integrate
with or build on similar or related efforts in order to improve
relevant outcomes (as defined in 34 CFR 77.1(c)), using nonpublic funds
or resources.
(xxviii) The extent to which the proposed project is supported by
evidence of promise (as defined in 34 CFR 77.1(c)).
(xxix) The extent to which the proposed project is supported by
strong theory (as defined in 34 CFR 77.1(c)).
* * * * *
(h) * * *
(2) * * *
(viii) The extent to which the methods of evaluation will, if well-
implemented,
[[Page 74404]]
produce evidence about the project's effectiveness that would meet the
What Works Clearinghouse Evidence Standards without reservations.\3\
---------------------------------------------------------------------------
\3\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
(ix) The extent to which the methods of evaluation will, if well-
implemented, produce evidence about the project's effectiveness that
would meet the What Works Clearinghouse Evidence Standards with
reservations.\4\
---------------------------------------------------------------------------
\4\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
(x) The extent to which the methods of evaluation will, if well-
implemented, produce evidence of promise (as defined in 34 CFR
77.1(c)).
(xi) The extent to which the methods of evaluation will provide
valid and reliable performance data on relevant outcomes.
(xii) The extent to which the evaluation plan clearly articulates
the key components, mediators, and outcomes of the grant-supported
intervention, as well as a measurable threshold for acceptable
implementation.
(i) Strategy to Scale
(1) The Secretary considers the applicant's strategy to scale the
proposed project.
(2) In determining the applicant's capacity to scale the proposed
project, the Secretary considers one or more of the following factors:
(i) The applicant's capacity (e.g., in terms of qualified
personnel, financial resources, or management capacity) to bring the
proposed project to scale on a national or regional level (as defined
in 34 CFR 77.1(c)) working directly, or through partners, during the
grant period.
(ii) The applicant's capacity (e.g., in terms of qualified
personnel, financial resources, or management capacity) to further
develop and bring to scale the proposed process, product, strategy, or
practice, or to work with others to ensure that the proposed process,
product, strategy, or practice can be further developed and brought to
scale, based on the findings of the proposed project.
(iii) The feasibility of successful replication of the proposed
project, if favorable results are obtained, in a variety of settings
and with a variety of populations.
(iv) The mechanisms the applicant will use to broadly disseminate
information on its project so as to support further development or
replication.
(v) The extent to which the applicant demonstrates there is unmet
demand for the process, product, strategy, or practice that will enable
the applicant to reach the level of scale that is proposed in the
application.
(vi) The extent to which the applicant identifies a specific
strategy or strategies that address a particular barrier or barriers
that prevented the applicant, in the past, from reaching the level of
scale that is proposed in the application.
7. Add Sec. 75.266 to subpart D to read as follows:
Sec. 75.266 What procedures does the Secretary use if the Secretary
decides to give special consideration to applications supported by
strong or moderate evidence of effectiveness?
(a) As used in this section, ``strong evidence of effectiveness''
is defined in 34 CFR 77.1(c);
(b) As used in this section, ``moderate evidence of effectiveness''
is defined in 34 CFR 77.1(c); and
(c) If the Secretary determines that special consideration of
applications supported by strong or moderate evidence of effectiveness
is appropriate, the Secretary may establish a separate competition
under the procedures in 34 CFR 75.105(c)(3), or provide competitive
preference under the procedures in 34 CFR 75.105(c)(2), for
applications supported by:
(1) Evidence of effectiveness that meets the conditions set out in
paragraph (a) of the definition of ``strong evidence of effectiveness''
in 34 CFR 77.1;
(2) Evidence of effectiveness that meets the conditions set out in
either paragraph (a) or (b) of the definition of ``strong evidence of
effectiveness'' in 34 CFR 77.1; or
(3) Evidence of effectiveness that meets the conditions set out in
the definition of ``moderate evidence of effectiveness.''
(Authority: 20 U.S.C. 1221e-3 and 3474)
8. Revise Sec. 75.250 to read as follows:
Sec. 75.250 Maximum funding period.
(a) The Secretary may approve a project period to fund the
substantive work of a grant and a data collection period to fund data
collection, analysis, and reporting related to a grant after the end of
the project period.
(b) The Secretary may approve a project period of up to 60 months
to perform the substantive work of a grant.
(1) The Secretary may approve a data collection period for a grant
for a period of up to 72 months after the end of the project period and
provide funding for the data collection period for the sole purpose of
collecting, analyzing, and reporting performance measurement data
regarding the project.
(2) The Secretary may inform applicants of the Secretary's intent
to approve data collection periods in the application notice published
for a competition or may decide to fund data collection periods after
grantees have started their project periods.
(Authority: 20 U.S.C. 1221e-3 and 3474)
9. Amend Sec. 75.251 by adding a new paragraph (c) to read as
follows:
Sec. 75.251 Budget Periods.
* * * * *
(c) If the Secretary funds a multi-year data collection period, the
Secretary may fund the data collection period through separate budget
periods and fund those budget periods in the same manner as those
periods are funded during the project period.
10. Amend Sec. 75.253 by--
A. Revising paragraph (a)(2).
B. Adding a new paragraph (a)(5).
C. Re-designating paragraphs (b) through (e) as paragraphs (c)
through (f).
D. Adding a new paragraph (b).
And
E. Revising newly re-designated paragraph (f).
The revisions and additions read as follows:
Sec. 75.253 Continuation of a multi-year project after the first
budget period.
(a) * * *
(2) The grantee has either--
(i) Made substantial progress in achieving--
(A) The goals and objectives of the project; and
(B) If the Secretary established performance measurement
requirements for the grant in the application notice, the performance
targets in the grantee's approved application; or
(ii) Obtained the Secretary's approval for changes to the project
that--
(A) Do not increase the amount of funds obligated to the project by
the Secretary; and
(B) Enable the grantee to achieve the goals and objectives of the
project and meet the performance targets of the project, if any,
without changing the scope or objectives of the project.
* * *
(5) The grantee has maintained financial and administrative
management systems that meet the requirements in 34 CFR 74.21 or 80.20,
as appropriate.
(b) In deciding whether a grantee has made substantial progress,
the Secretary
[[Page 74405]]
may consider any information relevant to the authorizing statute, a
criterion, a priority, or a performance measure, or to a financial or
other requirement that applies to the selection of applications for new
grants.
* * * * *
(f) Unless prohibited by the program statute or regulations, a
grantee that is in the final budget period of its project period may
seek continued assistance for the project as required under the
procedures for selecting new projects for grants.
11. Revise Sec. 75.590 to read as follows.
Sec. 75.590 Evaluation by the grantee.
(a) If the application notice for a competition required applicants
to describe how they would evaluate their projects, each grantee under
that competition must demonstrate to the Department that--
(1) The evaluation meets the standards of the evaluation in the
approved application for the project; and
(2) The performance measurement data collected by the grantee and
used in the evaluation meet the performance measurement requirements of
the approved application.
(b) If the application notice for a competition did not require
applicants to describe how they would evaluate their projects, each
grantee must provide information in its performance report
demonstrating--
(1) The progress made by the grantee in the most recent budget
period, including progress based on the performance measurement
requirements for the grant, if any;
(2) The effectiveness of the grant, including fulfilling the
performance measurement requirements of the approved application, if
any; and
(3) The effect of the project on the participants served by the
project, if any.
(Authority: 20 U.S.C. 1221e-3 and 3474)
12. Amend Sec. 75.708 by:
A. Revising paragraph (a).
B. Re-designating paragraph (b) as paragraph (e); and
C. Adding new paragraphs (b), (c) and (d).
The revision and additions read as follows.
Sec. 75.708 Subgrants.
(a) A grantee may not make a subgrant under a program covered by
this part unless authorized by statute or by paragraph (b) of this
section.
(b) The Secretary may, through an announcement in the Federal
Register, authorize subgrants when necessary to meet the purposes of a
program. In this announcement, the Secretary will--
(1) Designate the types of entities, e.g., State educational
agencies, local educational agencies, institutions of higher education,
and non-profit organizations, to which subgrants can be awarded; and
(2) Indicate whether subgrants can be made to entities identified
in an approved application or, without regard to whether the entity is
identified in an approved application, have to be selected through a
competitive process set out in subgranting procedures established by
the grantee.
(c) If authorized under paragraph (b) of this section, a subgrant
is allowed if it will be used by that entity to directly carry out
project activities described in that application.
(d) The grantee, in awarding subgrants under paragraph (b) of this
section, must--
(1) Ensure that subgrants are awarded on the basis of an approved
budget that is consistent with the grantee's approved application and
all applicable Federal statutory, regulatory, and other requirements;
(2) Ensure that every subgrant includes any conditions required by
Federal statute and executive orders and their implementing
regulations; and
(3) Ensure that subgrantees are aware of requirements imposed upon
them by Federal statute and regulation, including the Federal anti-
discrimination laws enforced by the Department; and
* * * * *
PART 77--DEFINITIONS THAT APPLY TO DEPARTMENT REGULATIONS
13. The authority citation for part 77 continues to read as
follows:
Authority: 20 U.S.C. 1221e-3 and 3474, unless otherwise noted.
14. Amend Sec. 77.1(c) by adding the following definitions in
alphabetical order:
Sec. 77.1 Definitions that apply to all Department programs.
* * * * *
(c) * * *
Ambitious means promoting continued, meaningful improvement for
program participants or for other individuals or entities affected by
the grant, or representing a significant advancement in the field of
education research, practices, or methodologies. When used to describe
a performance target, whether a performance target is ambitious depends
upon the context of the relevant performance measure and the baseline
for that measure.
* * * * *
Baseline means the starting point from which performance is
measured and targets are set.
* * * * *
Evidence of promise means there is empirical evidence to support
the theoretical linkage(s) between at least one critical component and
at least one relevant outcome presented in the logic model for the
proposed process, product, strategy, or practice. Specifically,
evidence of promise means the conditions in paragraphs (a) and (b) are
met:
(a) There is at least one study that is a--
(1) Correlational study with statistical controls for selection
bias;
(2) Quasi-experimental study that meets the What Works
Clearinghouse Evidence Standards with reservations; \1\ or
---------------------------------------------------------------------------
\1\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
(3) Randomized controlled trial that meets the What Works
Clearinghouse Evidence Standards with or without reservations.\2\
---------------------------------------------------------------------------
\2\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
(b) The study referenced in paragraph (a) found a statistically
significant or substantively important (defined as a difference of 0.25
standard deviations or larger), favorable association between at least
one critical component and one relevant outcome presented in the logic
model for the proposed process, product, strategy, or practice.
* * * * *
Large sample means an analytic sample of 350 or more students (or
other single analysis units) who were randomly assigned to a treatment
or control group or 50 or more groups (such as classrooms or schools)
that contain 10 or more students (or other single analysis units) and
that were randomly assigned to a treatment or control group.
* * * * *
Logic model (also referred to as theory of action) means a well-
specified conceptual framework that identifies key components of the
proposed process, product, strategy, or practice (i.e., the active
``ingredients'' that are hypothesized to be critical to achieving the
relevant outcomes) and describes the relationships among the key
[[Page 74406]]
components and outcomes, theoretically and operationally.
* * * * *
Moderate evidence of effectiveness means one of the following
conditions is met:
(a) There is at least one study of the effectiveness of the
process, product, strategy, or practice being proposed that meets the
What Works Clearinghouse Evidence Standards without reservations,\3\
found a statistically significant favorable impact on a relevant
outcome (with no statistically significant unfavorable impacts on that
outcome for relevant populations in the study or in other studies of
the intervention reviewed by and reported on by the What Works
Clearinghouse), and includes a sample that overlaps with the
populations or settings proposed to receive the process, product,
strategy, or practice.
---------------------------------------------------------------------------
\3\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
(b) There is at least one study of the effectiveness of the
process, product, strategy, or practice being proposed that meets the
What Works Clearinghouse Evidence Standards with reservations,\4\ found
a statistically significant favorable impact on a relevant outcome
(with no statistically significant unfavorable impacts on that outcome
for relevant populations in the study or in other studies of the
intervention reviewed by and reported on by the What Works
Clearinghouse), includes a sample that overlaps with the populations or
settings proposed to receive the process, product, strategy, or
practice, and includes a large sample and a multi-site sample (Note:
multiple studies can cumulatively meet the large and multi-site sample
requirements as long as each study meets the other requirements in this
paragraph).
---------------------------------------------------------------------------
\4\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
* * * * *
Multi-site sample means more than one site, where site can be
defined as an LEA, locality, or State.
* * * * *
National level describes the level of scope or effectiveness of a
process, product, strategy, or practice that is able to be effective in
a wide variety of communities, including rural and urban areas, as well
as with different groups (e.g., economically disadvantaged, racial and
ethnic groups, migrant populations, individuals with disabilities,
English learners, and individuals of each gender).
* * * * *
Performance measure means any quantitative indicator, statistic, or
metric used to gauge program or project performance.
* * * * *
Performance target means a level of performance that an applicant
would seek to meet during the course of a project or as a result of a
project.
* * * * *
Quasi-experimental design study means a study using a design that
attempts to approximate an experimental design by identifying a
comparison group that is similar to the treatment group in important
respects. These studies, depending on design and implementation, can
meet What Works Clearinghouse Evidence Standards with reservations \5\
(they cannot meet What Works Clearinghouse Evidence Standards without
reservations).
---------------------------------------------------------------------------
\5\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
* * * * *
Randomized controlled trial means a study that employs random
assignment of, for example, students, teachers, classrooms, schools, or
districts to receive the intervention being evaluated (the treatment
group) or not to receive the intervention (the control group). The
estimated effectiveness of the intervention is the difference between
the average outcome for the treatment group and for the control group.
These studies, depending on design and implementation, can meet What
Works Clearinghouse Evidence Standards without reservations.\6\
---------------------------------------------------------------------------
\6\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
* * * * *
Regional level describes the level of scope or effectiveness of a
process, product, strategy, or practice that is able to serve a variety
of communities within a State or multiple States, including rural and
urban areas, as well as with different groups (e.g., economically
disadvantaged, racial and ethnic groups, migrant populations,
individuals with disabilities, English learners, and individuals of
each gender). For an LEA-based project, to be considered a regional-
level project, a process, product, strategy, or practice must serve
students in more than one LEA, unless the process, product, strategy,
or practice is implemented in a State in which the State educational
agency is the sole educational agency for all schools.
* * * * *
Relevant outcome means the student outcome(s) (or the ultimate
outcome if not related to students) the proposed process, product,
strategy, or practice is designed to improve; consistent with the
specific goals of a program.
* * * * *
Strong evidence of effectiveness means one of the following
conditions is met:
(a) There is at least one study of the effectiveness of the
process, product, strategy, or practice being proposed that meets the
What Works Clearinghouse Evidence Standards without reservations,\7\
found a statistically significant favorable impact on a relevant
outcome (with no statistically significant unfavorable impacts on that
outcome for relevant populations in the study or in other studies of
the intervention reviewed by and reported on by the What Works
Clearinghouse), includes a sample that overlaps with the populations
and settings proposed to receive the process, product, strategy, or
practice, and includes a large sample and a multi-site sample (Note:
multiple studies can cumulatively meet the large and multi-site sample
requirements as long as each study meets the other requirements in this
paragraph).
---------------------------------------------------------------------------
\7\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
(b) There are at least two studies of the effectiveness of the
process, product, strategy, or practice being proposed, each of which:
Meets the What Works Clearinghouse Evidence Standards with
reservations,\8\ found a statistically significant favorable impact on
a relevant outcome (with no statistically significant unfavorable
impacts on that outcome for relevant populations in the studies or in
other studies of the intervention reviewed by and reported on by the
What Works Clearinghouse), includes a sample that overlaps with the
populations and settings proposed to receive the process, product,
strategy, or practice, and includes a large sample and a multi-site
sample.
---------------------------------------------------------------------------
\8\ See What Works Clearinghouse Procedures and Standards
Handbook (Version 2.1, September 2011), which can currently be found
at the following link: http://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.
---------------------------------------------------------------------------
* * * * *
[[Page 74407]]
Strong theory means a rationale for the proposed process, product,
strategy, or practice that includes a logic model.
[FR Doc. 2012-29897 Filed 12-13-12; 8:45 am]
BILLING CODE 4000-01-P