[Federal Register Volume 78, Number 227 (Monday, November 25, 2013)]
[Notices]
[Pages 70303-70305]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2013-28198]
[[Page 70303]]
-----------------------------------------------------------------------
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Centers for Disease Control and Prevention
Announcement of Requirements and Registration for the Predict the
Influenza Season Challenge
AGENCY: Centers for Disease Control and Prevention (CDC), Department of
Health and Human Services (HHS).
Award Approving Official: Thomas R. Frieden, MD, MPH, Director,
Centers for Disease Control and Prevention, and Administrator, Agency
for Toxic Substances and Disease Registry.
ACTION: Notice.
-----------------------------------------------------------------------
SUMMARY: The Centers for Disease Control and Prevention (CDC) located
within the Department of Health and Human Services (HHS) announces the
launch of the Predict the Influenza Season Challenge.
Each year annual epidemics of influenza occur in the United States.
However, variations in the timing and intensity of the season occur
each year. Early insights into the timing of the beginning, the peak,
and the intensity of the influenza season would be very useful in
planning vaccination campaigns, targeting resources and therefore
reducing costs associated with influenza prevention and control, and
communicating prevention messages to the public. Mathematical and
statistical models can be useful in predicting the timing and impact of
the influenza season, but no models published to date have successfully
predicted key influenza season milestones with sufficient accuracy.
The Influenza Division, National Center for Immunization and
Respiratory Diseases, CDC would like to invest in innovation through
the research and development of mathematical and statistical models
that use digital surveillance data (e.g. Twitter, internet search data,
web surveys, etc.) to predict the timing, peak, and intensity of the
upcoming influenza season. By hosting this challenge, the Influenza
Division is able to garner the technical innovation required to
accurately forecast the influenza season for less resources than would
be required through more traditional mechanisms. Furthermore, this
challenge will allow the Influenza Division to foster competition and
receive and evaluate multiple influenza season forecasts from a number
of scientists using a variety of different methodologies, which would
not be possible if a single entity were funded. Accurate influenza
forecasts have the potential to reduce long-term costs by more
efficiently utilizing resources that are available to track influenza
and implement control measures during the influenza season.
The challenge could be solved by applying any mathematical,
statistical, or other approach to predictive modeling. This challenge
will provide the Influenza Division with methods that advance the
science of prediction modeling, enhance the understanding of influenza
modeling and the use of digital data for influenza surveillance, and
improve the implementation of prevention and control measures for
seasonal influenza.
The historical national surveillance data that could be used to
enable training and correlation model development by competitors are
available at http://gis.cdc.gov/grasp/fluview/fluportaldashboard.html
and are updated every Friday at noon. The competitors' predictions and
methodology describing their models will comprise their initial
challenge entry. Eight subsequent bi-weekly submissions of predictions
based on the submitted methodology are also required. Predictions must
be national in scope but may also include HHS region predictions; all
predictions must be scaled to ILINet. Competitors' models will be
evaluated based on the methodology and how well the predictions match
the 2013-14 influenza season as measured by the U.S. Outpatient
Influenza-like Illness Surveillance Network (ILINet: http://www.cdc.gov/flu/weekly/overview.htm#Outpatient).
The competition will award a $75,000 prize and singular recognition
to the person or team that most closely predicts the influenza season.
DATES: Registration opens on November 23 and closes December 2, 2013.
Register by sending email to [email protected] and include your first
and last name, your address, and your phone number. Please see
``Registration Process for Participants'' below for additional details.
The Competition Submission Period is from December 1, 2013-March
27, 2014. The first submission of the prediction must be received by
December 2, 2013. Subsequent submissions will be required biweekly
until the close of the submission period and must be received by 11:59
p.m. Eastern on December 19, 2013; January 2, 2014; January 16, 2014;
January 30, 2014; February 13, 2014; February 27, 2014; March 13, 2014;
and March 27, 2014.
Judging will take place between March 28, 2014 and May 30, 2014.
Winners will be announced by June 20, 2014.
FOR FURTHER INFORMATION CONTACT: Matthew Biggerstaff, MPH, Centers for
Disease Control and Prevention, 1600 Clifton Rd NE MS A32; Atlanta, GA
30333, Phone: 404-639-3747, Email: [email protected].
SUPPLEMENTARY INFORMATION:
Subject of Challenge Competition: Entrants of the Predict the
Influenza Season Challenge are asked to predict the beginning, the
peak, and the intensity of the 2013-2014 influenza season at the
national level and at each or any Health and Human Services (HHS)
region level(s) in the United States by developing mathematical and
statistical models that utilize digital surveillance data (e.g. Twitter
data, mining internet search term data, internet-based surveys). If
there are questions about the eligibility of different data sources,
please contact Matthew Biggerstaff at 404-639-3747 or
[email protected].
Eligibility Rules for Participating in the Competition:
To be eligible to win a prize under this challenge, an individual
or entity--
(1) Must be at least 18 years old;
(2) Shall have registered to participate in the competition under
the rules promulgated by HHS/CDC;
(3) Shall have complied with all the requirements under this
section;
(4) In the case of a private entity, shall be incorporated in and
maintain a primary place of business in the United States, and in the
case of an individual, whether participating singly or in a group,
shall be a citizen or permanent resident of the United States; and
(5) May not be a Federal entity or Federal employee acting within
the scope of their employment.
(6) Shall not be an HHS employee working on their applications or
submissions during assigned duty hours.
(7) Shall not be an employee of or contractor at HHS/CDC.
(8) Federal grantees may not use Federal funds to develop COMPETES
Act challenge applications unless consistent with the purpose of their
grant award.
(9) Federal contractors may not use Federal funds from a contract
to develop COMPETES Act challenge applications or to fund efforts in
support of a COMPETES Act challenge submission.
An individual or entity shall not be deemed ineligible because the
individual or entity used Federal facilities or consulted with Federal
[[Page 70304]]
employees during a competition if the facilities and employees are made
available to all individuals and entities participating in the
competition on an equal basis.
By participating in this challenge, an individual or organization
agrees to assume any and all risks related to participating in the
challenge. Individuals or organizations also agree to waive claims
against the Federal Government and its related entities, except in the
case of willful misconduct, when participating in the challenge,
including claims for injury; death; damage; or loss of property, money,
or profits, and including those risks caused by negligence or other
causes.
By participating in this challenge, individuals or organizations
agree to protect the Federal Government against third party claims for
damages arising from or related to challenge activities.
Individuals or organizations are not required to hold liability
insurance related to participation in this challenge.
Registration Process for Participants: To register, please send an
email to [email protected] and include your first and last name, your
address, and your phone number. The subject line of the email
submission must be labeled ``Predict the Flu Challenge [Competitors
Name] [Date of Submission]''. Additionally, this email should include
the following statements: ``By participating in this competition, I
agree to assume any and all risks and waive claims against the Federal
Government and its related entities, except in the case of willful
misconduct, for any injury, death, damage, or loss of property,
revenue, or profits, whether direct, indirect, or consequential,
arising from my participation in this prize contest, whether the
injury, death, damage, or loss arises through negligence or otherwise.
Additionally, I agree to indemnify the Federal Government against third
party claims for damages arising from or related to competition
activities.'' Registration opens on November 23, 2013, and closes 11:59
p.m. EST on December 2, 2013.
The Competition Submission Period is from December 1, 2013-March
27, 2014. The first submission of the prediction must be received by
December 2, 2013 and include a narrative describing the methodology of
the prediction model and the results of that model (figures, tables, or
narratives) using digital surveillance data (e.g. Twitter data, mining
internet search term data, internet-based surveys) that predicts the
beginning, the peak, and the intensity of the 2013-2014 influenza
season at the national level in the United States. The methodology and
data source(s) submitted to CDC must match the actual methods utilized
by the competitors when making their influenza season predictions. In
addition to the national-level predictions, competitors may also submit
predictions of the beginning, the peak, and the intensity of the 2013-
2014 influenza seasons for any of the 10 HHS regions.
Subsequent submissions will be required biweekly until the close of
the submission period and must include the updated results of the
prediction model (figures, tables, or narratives) at the national level
in the United States using digital surveillance data; predictions for
any of the 10 HHS regions will also be accepted, and submitting
predictions for the 10 regions can potentially add to the final scores
competitors. Subsequent submissions that include the updated
predictions of the beginning, peak, and the intensity of the 2013-2014
influenza season must be received by 11:59 p.m. Eastern on December 19,
2013; January 2, 2014; January 16, 2014; January 30, 2014; February 13,
2014; February 27, 2014; March 13, 2014; and March 27, 2014. A total of
9 submissions must be received over the course of the contest to be
eligible for the contest prize; late submissions will not be
considered.
Contest Prize: For the contest, one $75,000 prize will be awarded.
The winning competitor will be recognized on the CDC influenza Web
page.
Payment of the Prize: Prizes awarded under this competition will be
paid by electronic funds transfer and may be subject to Federal income
taxes. HHS will comply with the Internal Revenue Service withholding
and reporting requirements, where applicable.
Basis Upon Which Winner Will Be Selected: A total of 9 submissions
must be received over the course of the contest to be eligible for
judging; late submissions will not be considered. The Competitors'
model methodology and nine biweekly predictions will comprise their
challenge entry. The selection of the winner for this challenge will be
based on an evaluation of the methodology used to make the prediction
and the accuracy of the prediction compared to the U.S. Outpatient
Influenza-like Illness Surveillance Network (ILINet: http://www.cdc.gov/flu/weekly/) at the national level and any of the included
10 HHS regions. National and HHS-level predictions must be scaled to
ILINet.
Submissions will be judged by a panel of reviewers that may include
CDC staff, public health officials, and/or academics from noncompeting
colleges or universities. Judges will score submissions on a scale of 0
to 150 using the six criteria identified below. Based on the average
score for each submission, the top submissions will be recommended to a
Lead Judge in CDC's Influenza Division. The Lead Judge will assess the
top submissions using the criteria below and select the first place
entry. Judging will take place between March 28, 2014 and May 30, 2014.
i. Methodology (25)
Are key materials to support the correct interpretation of
the predictions by the judges, such as concepts, sources, and methods,
provided as part of the submission?
Are the predictions scaled to ILINet? Are the data and
results presented clearly, allowing the judges who may not be experts
in mathematical modeling to evaluate the model and its outputs?
What are the data sources used to make the prediction? Is
the data source representative? If not, which groups are
underrepresented in the data? Are there any impacts caused by these
exclusions?
Is there a clear description of how the search terms were
selected (if relevant)?
Are there any serious accuracy or methodological problems
with the prediction approach?
Are measures provided that give an indication of how
certain contestants are about their predictions? (e.g. confidence
intervals, probability of the prediction occurring)?
ii. Predicting the Start of the Influenza Season (10)
Does the model accurately predict the start of the
influenza season at the national level and any of the included 10 HHS
regions? The start of the season will be defined as the week when the
percentage of visits reported through ILINet crosses the baseline value
for three consecutive weeks. ILINet baseline values for the United
States and the 10 HHS regions for the 2013-2014 influenza season are
available at http://www.cdc.gov/flu/weekly/overview.htm#Outpatient.
iii. Predicting the Peak Week of the Influenza Season (10)
Does the model accurately predict the peak week of ILINet
at the national level and any of the included 10 HHS regions? The peak
week will be defined as the surveillance week that the ILINet
percentage is the highest for the 2013-14 influenza season in the
United States and the 10 separate HHS regions.
[[Page 70305]]
iv. Predicting the Intensity of the Influenza Season (25)
Does the model predict the intensity of the influenza
season at the national and any of the included HHS regional levels? The
intensity will be defined as the number of weeks that ILINet remains
above baseline and the highest numeric value that the ILINet percentage
reaches in the United States and the 10 separate HHS regions.
v. Timeliness and Reliability of the Predictions (20)
How many weeks before each prediction milestone (the start
and the peak week) was the most accurate prediction made?
Did the prediction of each milestone vary widely between
the Competitors' submissions?
vi. Geography (10)
How comprehensively are the geographic regions of the
United States represented in the source data?
vii. Optional HHS Regional Predictions (Up to 50 Bonus Points)
HHS regional predictions will be evaluated separately
using the six criteria identified above. Each submitted regional
prediction will add between 0 and 5 points to the competitor's final
score, depending on the evaluation of the accuracy, timeliness, and
reliability of the prediction.
Additional Information: The historical national surveillance data
that could be used to enable training and correlation model development
by competitors are available at http://gis.cdc.gov/grasp/fluview/fluportaldashboard.html and are updated every Friday at noon.
The Competitors will not have to transfer their exclusive
intellectual property rights to the CDC. Instead, the Competitors will
grant to CDC non-exclusive license to practice their solutions.
Compliance With Rules and Contacting Contest Winners: Finalists and
the contest winners must comply with all terms and conditions of these
official rules, and winning is contingent upon fulfilling all
requirements herein. The initial finalists will be notified by email,
telephone, or mail after the date of the judging. Awards may be subject
to Federal income taxes, and the Department of Health and Human
Services will comply with the Internal Revenue Service withholding and
reporting requirements, where applicable.
Privacy: If contestants choose to provide the CDC with personal
information by registering or filling out the submission form through
the Challenge.gov Web site, that information is used to respond to
contestants in matters regarding their submission, announcements of
entrants, finalists, and winners of the contest. Information is not
collected for commercial marketing. Registering through the
Challenge.gov Web site is not required, however. Registrants may submit
an email to [email protected] as noted in ``Registration Process for
Participants.'' Winners are permitted to cite that they won this
contest.
General Conditions: The CDC reserves the right to cancel, suspend,
and/or modify the contest, or any part of it, for any reason, at CDC's
sole discretion.
Participation in this contest constitutes a contestants' full and
unconditional agreement to abide by the contest's official rules found
at www.Challenge.gov.
Authority: 15 U.S.C. 3719.
Dated: November 19, 2013.
Stacey Hoffman,
Acting Director, Division of Executive Secretariat, Centers for Disease
Control and Prevention.
[FR Doc. 2013-28198 Filed 11-22-13; 8:45 am]
BILLING CODE 4163-18-P