[Federal Register Volume 79, Number 33 (Wednesday, February 19, 2014)]
[Pages 9485-9488]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2014-03534]



Notice of Intent To Seek Approval To Establish an Information 
Collection System

AGENCY: National Science Foundation.

ACTION: Notice and request for comments.


SUMMARY: Under the Paperwork Reduction Act of 1995, Public Law 104-13 
(44 U.S.C. 3501 et seq.), and as part of its continuing effort to 
reduce paperwork and respondent burden, the National Science Foundation 
(NSF) is inviting the general public or other Federal agencies to 
comment on this proposed continuing information collection.
    Comments: Comments are invited on: (a) Whether the proposed 
collection of information is necessary for the proper performance of 
the functions of the Foundation, including whether the information will 
have practical utility; (b) the accuracy of the Foundation's estimate 
of the burden of the proposed collection of information; (c) ways to 
enhance the quality, utility, and clarity of the information to be 
collected; and (d) ways to minimize the burden of the collection of 
information on those who are to respond, including through the use of 
automated collection techniques or other forms of information 

DATES: Written comments on this notice must be received by April 21, 
2014, to be assured consideration. Comments received after that date 
will be considered to the extent practicable. Send comments to address 

FOR FURTHER INFORMATION CONTACT: Ms. Suzanne H. Plimpton, Reports 
Clearance Officer, National Science Foundation, 4201 Wilson Boulevard, 
Suite 1265, Arlington, Virginia 22230; telephone (703) 292-7556; or 
send email to splimpto@nsf.gov. Individuals who use a 
telecommunications device for the deaf (TDD) may call the Federal 
Information Relay Service (FIRS) at 1-800-877-8339, which is accessible 
24 hours a day, 7 days a week, 365 days a year (including federal 

    Title of Collection: Engineering Program Monitoring Data 
    OMB Number: 3145-NEW.
    Expiration Date of Approval: Not applicable.
    Type of Request: Intent to seek approval to establish an 
information collection for post-award output and outcome monitoring 
    Proposed Project:
    NSF provides nearly 20 percent of federal funding for basic 
research to academic institutions.\1\ Within NSF, the Directorate for 
Engineering (ENG) has primary responsibility for promoting the progress 
of engineering in the United States in order to enable the Nation's 
capacity to perform. Its investments in engineering research and 
education aim to build and strengthen a national capacity for 
innovation that can lead over time to the creation of new shared wealth 
and a better quality of life. Most NSF programs in engineering are 
funded through the Directorate for Engineering, which also sponsors the 
NSF's Industrial Innovation and Partnerships (IIP) Division. To these 
ends, ENG

[[Page 9486]]

provides support for research and implementation activities that may 
meet national needs. While scientists seek to discover what is not yet 
known, engineers apply fundamental science to design and develop new 
devices and engineered systems to solve societal problems. ENG also 
focuses on broadening participation in engineering research and 

    \1\ National Science Foundation. (2012). NSF at a glance. 
Retrieved from http://www.nsf.gov/about/glance.jsp.

    The Directorate for Engineering (ENG) requests of the Office of 
Management and Budget (OMB) a clearance that will allow NSF-ENG to 
improve the rigor of our surveys for evaluations and program 
monitoring, as well as to initiate new data collections to monitor the 
immediate, intermediate and long-term outcomes of our investments by 
periodically surveying the grantees and their students involved in the 
research. The clearance will allow any program in the Directorate for 
Engineering at NSF to rigorously develop, test, and implement survey 
instruments and methodologies.
    Some NSF-ENG programs regularly conduct a variety of data 
collection activities that include routine program monitoring, program 
evaluations, and education-related data collections from federally 
funded institutions of higher education. The primary objective of this 
clearance is to allow other programs in NSF-ENG to collect outcome and 
output data from grantees, their partners and students, which will 
enable the evaluation of the impact of its investments in engineering 
research over time. With that purpose, this clearance will allow us to 
use a bank of approved question items as needed as long as the 
resources consumed to do not exceed this request. The second related 
objective is to improve our questionnaires and/or data collection 
procedures through pilot tests and other survey methods used in these 
activities for different programs. Under this clearance a variety of 
surveys could be pre-tested, modified and used. The exact combination 
of questions from the question bank is currently unknown for each 
program, but it will be based on their respective logic models and 
program goals. Following standard OMB requirements, NSF will submit to 
OMB an individual request for each survey project it undertakes under 
this clearance. NSF will request OMB approval in advance and provide 
OMB with a copy of the questionnaire (if one is used) and materials 
describing the project.
    In doing so, this request seeks approval for multiple data 
collections that have similar elements and purposes and will provide 
essential information for program monitoring purposes through multiple 
possible methods of collection. Data collected by ENG program outcome 
monitoring systems will be used for program planning, management, 
evaluation, and audit purposes. Summaries of output and outcome 
monitoring data are used to respond to queries from Congress, the 
public, NSF's external merit reviewers who serve as advisors, including 
Committees of Visitors (COVs), and NSF's Office of the Inspector 
General. These data are needed for effective administration, program 
and project monitoring, evaluation, strategic reviews and for measuring 
attainment of NSF's program and strategic goals, as identified by the 
President's Accountable Government Initiative, the Government 
Performance and Results Act (GPRA) Modernization Act of 2010, and NSF's 
Strategic Plan.
    Outcome and output monitoring data represented in this collection 
is complementary to the data collected in the RPPR both with respect to 
type of questions and indicators (content) and timeliness of the 
collection. All questions asked are questions that are NOT included in 
the final or annual report and the intention is to ask them even beyond 
the period of performance on voluntary basis in order to capture 
impacts of the research that occur beyond the life of the award. 
Questionnaire items fall into the category of general items that could 
be used across programs as well as items of interest to a particular 
division. We are seeking to collect additional information from the 
grantees about the outcomes of their research that go above and beyond 
the standard reporting requirements used by the NSF and could span a 
period of up to 10 years after the award.
    The six (6) divisions or offices in NSF-ENG which oversee multiple 
programs are included in this request. They are designed to assist in 
management of specific programs, divisions, or multi-agency initiatives 
and to serve as data resources for current and future program 

              Program/office                       Type of program
Emerging Frontiers in Research and          Fundamental Research.
 Innovation (EFRI).
Engineering Education and Centers (EEC)...  Large research center's
                                             research (Implementation &
                                             Development) & Research and
Industrial Innovation and Partnerships      Translational Research.
Chemical, Bioengineering, Environmental,    Fundamental Research.
 and Transport Systems (CBET).
Civil, Mechanical, and Manufacturing        Fundamental Research.
 Innovation (CMMI).
Electrical, Communications, and Cyber       Fundamental Research.
 Systems (ECCS).

    ENG-funded projects could include research opportunities and 
mentoring for educators, scholars, and university students, as well as 
outreach programs that help stir the imagination of K-12 students, 
often with a focus on groups underrepresented in science and 
engineering. The surveys to be tested and implemented would be designed 
to assist in management of specific division programs, divisions, or 
multi-agency initiatives and to serve as data resources for current and 
future program evaluations.
    This data collection effort will enable program officers to 
longitudinally monitor outputs and outcomes given the unique goals and 
purpose of their programs. This is very important to enable appropriate 
and accurate evidence-based management of the programs and to determine 
whether or not the specific goals of the programs are being met.
    Grantees will be invited to submit this information on a periodic 
basis to support performance review and the management of ENG grants by 
ENG officers. Once the survey tool for a specific program is tested, 
ENG grantees will be invited to submit these indicators to NSF via data 
collection methods that include but are not limited to online surveys, 
interviews, focus groups, phone interviews, etc. These indicators are 
both quantitative and descriptive and may include, for example, the 
characteristics of project personnel and students; sources of 
complementary cash and in-kind support to the ENG project; 
characteristics of industrial and/or other sector participation; 
research activities; education activities; knowledge transfer 
activities; patents, licenses; publications; descriptions of 
significant advances and other outcomes of the ENG-funded effort.
    Use of the Information: The data collected will be used for NSF 
internal reports, historical data, program level studies and 
evaluations, and for securing future funding for the ENG program 
maintenance and growth. These data could be used for program

[[Page 9487]]

evaluation purposes if deemed necessary for a particular program. 
Evaluation designs could make use of metadata associated with the 
award, and other characteristics to identify a comparison group to 
evaluate the impact of the program funding and other interesting 
research questions. Different designs could be possible based on the 
research questions varying from program to program but the fact that 
NSF-ENG has already collected data on the outcomes of interest will 
result in substantial savings on the evaluation per se.
    Estimate of Burden:

                                                                                   Annual number
                        Collection title                             Number of     of responses/    Annual hour
                                                                    respondents     respondent        burden
Emerging Frontiers in Research and Innovation (EFRI)............              85            0.25           21.25
Civil, Mechanical, and Manufacturing Innovation (CMMI)..........            1300            0.25             325
Chemical, Bioengineering, Environmental, and Transport Systems              1750            0.25           437.5
Electrical, Communications, and Cyber Systems (ECCS)............            1000            0.25             250
Engineering Education and Centers (EEC).........................             100            0.25             100
Industrial Innovation and Partnerships (IIP)....................            1000               4            4000
    Total.......................................................           5,235  ..............        5,133.75

    Below is an example that shows how the hour burden was estimated 
for the monitoring system.
    The estimated average number of annual respondents is 5,235, with 
an estimated annual response burden of 5,133.75 hours. For post-award 
monitoring systems, most divisions expect to collect data at 1, 2, 5, 
and 10 years post-award, in order to have the best chance of capturing 
the more immediate outcomes expected by 1-2 years post-award, 
intermediate outcomes at 5 years post-award, and long-term outcomes/
impacts at 10 years post-award. These four (4) data collections spread 
over the span of 10 years; this averages to 0.25 data collections/year. 
For the IIP division, many awards are made in translational research, 
such that we might expect a shorter and more condensed timeline of 
outcomes and impacts. Thus, some programs may wish to collect data 
quarterly for the first two years of the award, and then once annually 
at 5 and 10 years post-award. The annual number of responses for the 
first 2 years post-award is included in this table.
    For life-of-award monitoring, the data collection burden to 
awardees will be limited to no more than 2 hours of the respondents' 
time in each instance.
    Respondents: The respondents are either PIs or program 
coordinators. One PI or program coordinator per award completes the 

Estimates of Annualized Cost to Respondents for the Hour Burdens

    The overall annualized cost to the respondents is estimated to be 
$214,635. The following table shows the annualized estimate of costs to 
PI/program coordinator respondents, who are generally university 
professors. This estimated hourly rate is based on a report from the 
American Association of University Professors, ``Annual Report on the 
Economic Status of the Profession, 2011-12,'' Academe, March-April 
2012, Survey Report Table 4. According to this report, the average 
salary of an associate professor across all types of doctoral-granting 
institutions (public, private-independent, religiously affiliated) was 
$86,319. When divided by the number of standard annual work hours 
(2,080), this calculates to approximately $41 per hour.

                                                     Number of     Burden hours   Average hourly     Estimated
                 Respondent type                    respondents   per respondent       rate         annual cost
PIs/Program Coordinators (EFRI, CBET, CMMI,                4,235            0.25             $41        $173,635
 ECCS, EEC).....................................
PIs/Program Coordinators (IIP Division).........           1,000               1              41          41,000
    Total.......................................           5,235  ..............  ..............         214,635

    Estimated Number of Responses per Report: Data collection for the 
collections involves all awardees in the programs involved. The table 
below shows the total universe and sample size for each of the 

 Respondent Universe and Sample Size of ENG Program Monitoring Clearance
                                            Universe of
            Collection title                respondents     Sample size
Emerging Frontiers in Research and                    85              85
 Innovation (EFRI)......................
Civil, Mechanical, and Manufacturing                1300            1300
 Innovation (CMMI)......................
Chemical, Bioengineering, Environmental,            1750            1750
 and Transport Systems (CBET)...........
Electrical, Communications, and Cyber               1000            1000
 Systems (ECCS).........................
Engineering Education and Centers (EEC).             100             100
Industrial Innovation and Partnerships              1000            1000

[[Page 9488]]

    Dated: February 12, 2014.
Suzanne H. Plimpton,
Reports Clearance Officer, National Science Foundation.
[FR Doc. 2014-03534 Filed 2-18-14; 8:45 am]