[Federal Register Volume 76, Number 9 (Thursday, January 13, 2011)]
[Pages 2390-2393]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2011-408]



Agency for Healthcare Research and Quality

Agency Information Collection Activities: Proposed Collection; 
Comment Request

AGENCY: Agency for Healthcare Research and Quality, HHS.

ACTION: Notice.


SUMMARY: This notice announces the intention of the Agency for 
Healthcare Research and Quality (AHRQ) to request that the Office of 
Management and Budget (OMB) approve the proposed information collection 
project: ``Connecting Primary Care Practices with Hard-to-Reach 
Adolescent Populations.'' In accordance with the Paperwork Reduction 
Act, 44 U.S.C. 3501-3520, AHRQ invites the public to comment on this 
proposed information collection.

DATES: Comments on this notice must be received by March 14, 2011.

ADDRESSES: Written comments should be submitted to: Doris Lefkowitz, 
Reports Clearance Officer, AHRQ, by e-mail at 
[email protected].
    Copies of the proposed collection plans, data collection 
instruments, and specific details on the estimated burden can be 
obtained from the AHRQ Reports Clearance Officer.

Clearance Officer, (301) 427-1477, or by e-mail at 
[email protected].


[[Page 2391]]

Proposed Project

Connecting Primary Care Practices With Hard-to-Reach Adolescent 

    The overall goal of this exploratory project is to improve the 
quality of adolescent health care. The project will address suboptimal 
adolescent care with respect to health risk behaviors, which can have 
serious health consequences. In particular, failure to address health 
risk behaviors among adolescents (e.g., smoking, substance abuse, poor 
diets, physical inactivity, and high-risk sexual behavior) contributes 
significantly to increased morbidity and mortality. Adolescents (11-17 
years of age) constitute 17% of the population of the U.S., but they 
are responsible for only 7% of medical office visits. As a result, 
primary care providers have relatively less opportunity to evaluate and 
counsel adolescents in their offices than most other patients. Even 
when adolescents receive routine health care, open communication with 
their health care providers may be problematic. A national survey found 
that the majority of adolescent boys and girls in the U.S. report at 
least 1 of 8 potential health risks, but most (63%) had not spoken to 
their doctor about any of these (Klein & Wilson, 2002). Improved 
engagement and communication between adolescents and their primary care 
providers could increase the likelihood that effective preventive 
services and health care are provided. It could also improve the 
efficiency of health care services for adolescents, in terms of 
appointments kept and adherence to recommended screening or treatment 
    Technological interventions to improve care may be particularly 
appropriate for adolescents, since they are typically the early 
adopters of new technology (Skinner, Biscope, Poland, & Goldberg, 
2003). Use of in-office electronic screeners before appointments has 
proven useful (Olson, Gaffney, Lee, & Star 2008; Salerno, 2008; Yi, 
Martyn, Salerno, & Darling-Fisher, 2009). Outside of the office, youth 
have increasingly turned to the internet for health-related 
information, and have also rapidly adopted mobile technology (Lenhart, 
Ling, Campbell, & Purcell. 2010) and social media (Lenhart, Purcell, 
Smith, & Zickuhr, 2010). Health plans (e.g., Kaiser Permanente) and 
practices (Hawn, 2009) have conducted early work in applying patient-
centered web and mobile technologies. These projects have included 
interventions to decrease patient no-show rates, increase the use of 
sunscreen, and engage adolescents in diabetes management. Much work 
remains to be done, however, in understanding how primary care 
practices can best embrace advances in communications and information 
technology to improve health outcomes for adolescent patients.
    This project has the following goals:
    (1) Explore the benefits of supplementing an electronic in-office 
pre-visit screener with a set of web technologies for adolescent 
outreach and engagement outside of office visits.
    a. The Rapid Assessment for Adolescent Preventive Services 
(copyright) (RAAPS), as described below, will be used for in-office 
pre-visit screening
    b. The web technologies will include (i) a web page for more static 
content such as information about practices and health-related 
commentary from practice clinicians and staff, (ii) a Facebook page for 
social interaction about health topics including topical content that 
will engage adolescents in conversations about general, not personal, 
health behaviors and encouraging youth to discuss these issues with 
their primary care practitioners at clinic visits, and (iii) a Twitter 
site that will allow youth to use mobile phones with text messaging to 
subscribe to Facebook posts.
    (2) Increase adolescent visits to primary care and identification 
of health risks during visits
    (3) Promote healthier behavior in four domains: (1) Diet, (2) 
physical activity, (3) substance abuse (smoking, alcohol, and use of 
other recreational drugs), and (4) sexual health
    (4) Develop a manual of best practices for these components in 
primary care.
    This study is being conducted by AHRQ through its contractor, State 
Network of Colorado Ambulatory Practices and Partners (SNOCAP-USA), a 
practice-based research network (PBRN) based at the University of 
Colorado Denver, pursuant to AHRQ's statutory authority to conduct and 
support research on healthcare and on systems for the delivery of such 
care, including activities with respect to the quality, effectiveness, 
efficiency, appropriateness and value of healthcare services and with 
respect to clinical practice, including primary care and practice-
oriented research. 42 U.S.C. 299a(a)(l) and (4).

Method of Collection

    This project will be conducted in four primary care practice sites 
that have a substantial number of adolescent patients. The following 
activities and data collections will be implemented:
    (1) RAAPS questionnaire. Practices will use the 21-item RAAPS 
questionnaire for in-office pre-visit screening. RAAPS was developed by 
the University of Michigan Regional Alliance for Healthy Schools to 
elicit information about risky adolescent behaviors that should be 
addressed, but often are missed, in primary care. It is available in 
both paper and online forms; the latter will be used in this project. 
The primary purpose of the RAAPS questionnaire is to improve clinical 
recognition of risky behaviors so that personal counseling may be 
    (2) Process measures for web technologies. For each of the web 
technologies used (the web page, Facebook page, and Twitter site), data 
on the number of unique visitors, the frequency of their visits, and 
their activities (e.g. whether they create a new post or ``like'' 
postings) will be obtained by the research team. These data will not 
include personally identifiable information (e.g. the user's username, 
birth date, IP address, etc.). OMB clearance is not required for this 
data collection.
    (3) Extraction of medical record data. Staff members at each 
practice will use their clinical information systems to extract medical 
record data for use by the research team. Data to be extracted consist 
of (a) Contact information for patients seen in the 18 months prior to 
the start date for implementation of RAAPS and the web technologies. 
This is the sample frame for the adolescent behavior and communication 
survey. These data will be used by the project staff to prepare the 
recruitment mailings. (b) Clinic notes for adolescents seen in the 12 
months prior to implementation start date and for adolescents seen in 
the 12 months following the implementation start date. Clinic notes 
will be made accessible either by pulling paper charts or printing 
notes from electronic medical records. The notes will be reviewed and 
abstracted by the research team to assess whether the intervention had 
the intended effect of increasing adolescent visits to primary care and 
the identification of potential health risks during visits.
    (4) Consent-assent form. This is used to obtain consent from the 
parent or guardian and assent from the adolescent to participate in the 
adolescent behavior and communication survey.
    (5) Adolescent behavior and communication survey. A questionnaire 
(by mail, with an online option) will be administered twice to 
adolescent patients for whom consent-assent has been obtained: Once at 
baseline and again six months after the intervention. The purpose of 
this survey is to measure the adolescent's level of comfort with 
discussing their health with their

[[Page 2392]]

clinician and their level of satisfaction with their medical care, and 
to see how this changes after the intervention.
    (6) Post-visit satisfaction survey. Practices will provide 
adolescents with a brief, post-card sized anonymous questionnaire at 
every office visit during the study period. The purpose is to assess 
the perceived utility of the RAAPS questionnaire, and whether the visit 
was related to the project's web technologies.
    (7) Adolescent focus groups. Eight adolescents (two from each 
practice) will provide feedback on the web page, Facebook, and Twitter 
pages. There will be one in-person group meeting preimplementation, 
followed by a series of 3 additional asynchronous group discussions 
conducted via the web at three-month intervals. These provide a process 
for user-centered design and refinement of the of web technologies.
    (8) Adolescent ``think-aloud'' sessions. These sessions, which will 
be conducted near the end of the study period, will involve a set of 
eight adolescent patients (two from each practice) that did not 
participate in the focus groups. Subjects will come to the practice for 
individual sessions in which they will be asked to say aloud what they 
are thinking about the Web technologies as they navigate them as they 
typically would. The purpose is to assess the perceived utility of the 
components of the Web, Facebook, and Twitter pages.
    (9) Clinician semi-structured interviews. At each site, individual 
interviews will be conducted with two clinicians (eight clinicians 
total). The purpose is to assess clinician perceptions of the effects 
of the RAAPS questionnaire and the Web technologies on the clinical 
encounter and the care they provide.
    (10) Administrator-staff semi-structured interviews. At each site, 
semi-structured interviews will be conducted with the practice manager 
and a front-desk staff member. The purpose is to assess the effect of 
the interventions on the check in process and other business processes.
    (11) Semi-structured interviews for the draft manual. The draft 
manual of best practices in primary care for adoption of Web and 
assessment technologies (such as the RAAPS questionnaire) developed by 
the research team will be sent to the practice manager and the practice 
director (lead clinician) of each site. Their feedback will be 
solicited by telephone roughly two weeks later. This ``member 
checking'' enhances the validity of the manual's conclusions and 
    The results from this exploratory project will be used to inform 
development of a manual to assist primary care practices in adopting 
interventions to improve the effectiveness of their outreach to and 
interactions with adolescent patients. In addition, information 
collected in the RAAPS questionnaire may be used by clinicians to 
improve clinical care.

Estimated Annual Respondent Burden

    Exhibit 1 shows the estimated annualized burden hours for the 
respondents' time to participate in this research. Among the 776 
adolescent patients across the 4 participating practices, 310 are 
expected to complete the RAAPS questionnaire, which takes about 15 
minutes to complete, at each office visit (on average there will be an 
estimated 1.25 office visits per patient). Practice staff members will 
perform the extraction of medical record data pre-implementation, and 
again post-implementation, for 50 patients. This task is estimated to 
require 4 hours per practice (slightly less than 5 minutes per patient 
    The consent-assent form for participation in the adolescent 
behavior and communication survey will be sent to the homes of all 
adolescents in the practice's panels. The estimated average time for 
reading and responding to the form is 15 minutes. The adolescent 
behavior and communication survey will be completed twice, pre- and 
post- intervention, by 233 adolescent patients and requires 15 minutes 
to complete. The post-visit satisfaction survey will be completed by 
each of the 310 participating adolescent patients after each office 
visit and will take 5 minutes to complete.
    A series of four focus groups will be held with 8 adolescent 
patients over the course of the study period with each session lasting 
about 1.5 hours. In addition to the focus groups one ``think-aloud'' 
session will be held with a group of 8 adolescent patients and will 
also take 1.5 hours.
    Feedback from the practice staff and the clinicians will be 
obtained through 3 different semi-structured interviews. Two staff 
members from each of the 4 practices will participate in these 
interviews. The clinician and administrator-staff semi-structured 
interviews will each last 30 minutes. Semi-structured interviews for 
the draft manual will require about one hour total (30 minutes to 
review the manual and 30 minutes to participate in the interview). The 
total annualized burden is estimated to be 548 hours.
    Exhibit 2 shows the estimated annualized cost burden associated 
with the respondents' time to participate in this research. The total 
annual cost burden is estimated to be $8,601.

                                  Exhibit 1--Estimated Annualized Burden Hours
                                                                      Number of
             Activity/data collection                  Number of    responses per    Hours per     Total burden
                                                      respondents     respondent      response         hours
RAAPS questionnaire...............................             310           1.25        15/60                97
Extraction of medical record data.................               4           2               4                32
Consent-assent form...............................             776           1           15/60               194
Adolescent behavior and communication survey......             233           2           15/60               117
Post-visit satisfaction survey....................             310           1.25         5/60                32
Adolescent focus groups...........................               8           4               1.5              48
Adolescent ``think-aloud'' sessions...............               8           1               1.5              12
Clinician semi-structured interviews..............               4           2           30/60                 4
Administrator-staff semi-structured interviews....               4           2           30/60                 4
Semi-structured interviews for the draft manual...               4           2               1                 8
    Total.........................................           1,661          na              na               548

[[Page 2393]]

                                   Exhibit 2--Estimated Annualized Cost Burden
                                                     Number of     Total burden   Average hourly    Total cost
            Activity/data collection                respondents        hours       wage rate \1\      burden
RAAPS questionnaire.............................             310              97       $9.01 \2\            $874
Extraction of medical record data...............               4              32       18.15 \3\             581
Consent-assent form.............................             776             194       22.11 \4\           4,289
Adolescent behavior and communication survey....             233             117        9.01 \2\           1,054
Post-visit satisfaction survey..................             310              32        9.01 \2\             288
Adolescent focus groups.........................               8              48        9.01 \2\             432
Adolescent ``think-aloud'' sessions.............               8              12        9.01 \2\             108
Clinician semi-structured interviews............               4               4       84.53 \5\             338
Administrator-staff semi-structured interviews..               4               4       29.63 \6\             119
Semi-structured interviews for the draft manual.               4               8       64.75 \7\             518
    Total.......................................           1,661             548              na           8,601
\1\ Mean hourly and wage costs for Colorado were derived from the Bureau of Labor and Statistics National
  Compensation Survey for May 2009 (http://www.bls.gov/oes/current/oes_co.htrn).
\2\ Hourly rate for an entry level worker (occupation code 3 5-0000) estimates the cost of time for adolescents,
  although many will not be employed.
\3\ Hourly rate for medical records and health information technician (29-2071).
\4\ Hourly rate for the mean for all occupations (00-0000) estimates the cost of time for the parent or guardian
  of the adolescent.
\5\ Average of hourly rates for a family medicine practitioner (29-1062) and a general internist (29-1063).
\6\ Average of (1) the hourly rate for a medical and health services manager (11-9111) and (2) the average of
  the hourly rates for a receptionist (43-4171) and a medical assistant (31-9092).
\7\ Average of (1) the hourly rate for a medical and health services manager (11-9110) and (2) the average of
  the hourly rates for a family medicine practitioner (29-1062) and a general internist (29-1063).

Estimated Annual Costs to the Federal Government

    Exhibit 3 shows the estimated total and annualized cost to the 
Federal Government for conducting this research. These estimates 
include the costs associated with the project such as the preparation 
of survey administration procedures, labor costs, administrative 
expenses, costs associated with copying, postage, and telephone 
expenses, data management and analysis, and preparation of final 
reports. The annualized and total costs are identical since the data 
collection period will last for one year. The total cost is estimated 
to be $436,524.

             Exhibit 3--Estimated Total and Annualized Cost
          Cost component                Total cost      Annualized cost
Project Development...............            $72,364            $72,364
Data Collection Activities........             48,904             48,904
Data Processing and Analysis......             73,937             73,937
Publication of Results............             21,890             21,890
Project Management................             75,733             75,733
Overhead..........................            143,696            143,696
    Total.........................            436,524            436,524

Request for Comments

    In accordance with the above-cited Paperwork Reduction Act 
legislation, comments on AHRQ's information collection are requested 
with regard to any of the following: (a) Whether the proposed 
collection of information is necessary for the proper performance of 
AHRQ healthcare research and healthcare information dissemination 
functions, including whether the information will have practical 
utility; (b) the accuracy of AHRQ' s estimate of burden (including 
hours and costs) of the proposed collection(s) of information; (c) ways 
to enhance the quality, utility, and clarity of the information to be 
collected; and (d) ways to minimize the burden of the collection of 
information upon the respondents, including the use of automated 
collection techniques or other forms of information technology.
    Comments submitted in response to this notice will be summarized 
and included in the Agency's subsequent request for OMB approval of the 
proposed information collection. All comments will become a matter of 
public record.

    Dated: January 4, 2011.
Carolyn M. Clancy,
[FR Doc. 2011-408 Filed 1-12-11; 8:45 am]