Routine, standardized collection of patient information using patient-reported measures or outcomes (PROs) offers an evidence-based solution. PROs are reports elicited directly from patients regarding their health status (Snyder et al., 2012), which can include symptoms, health behaviors, and other relevant context (e.g., living and social circumstances). In HIV care, same-day, on-site PRO collection using hand-held computer tablets with results available to providers during clinic visits in real time, has improved provider ability to detect and address depression/suicidal ideation, inadequate ART adherence, substance use, cognitive functioning, and symptoms. Integrating PROs into clinical care of patients with chronic conditions, such as cancer, rheumatoid arthritis, and HIV, has helped improve patient outcomes (Ruland et al., 2010; Cleeland et al., 2011; Basch et al., 2016, 2017), has proven useful to providers (Wolfe et al., 2003; Stover et al., 2015; Fredericksen et al., 2016), has shown to be both acceptable and useful to patients in helping them prioritize what to discuss (Wolfe et al., 2003; Stover et al., 2015; Fredericksen et al., 2016), and has improved both satisfaction with care (Wasson et al., 1999; Taenzer et al., 2000; Chen et al., 2013; Nelson et al., 2015) and patient-provider communication (Wagner et al., 1997; Taenzer et al., 2000; Brown et al., 2001; Detmar et al., 2002; Velikova et al., 2004) from the perspective of both parties. PROs have also positively impacted delivery of care, for example, in reducing emergency department utilization and hospitalization among cancer patients (Basch et al., 2016).  

Studies outside of HIV care, primarily from oncology, have shown providers find PROs useful, in areas such as identifying less-observable or infrequently-discussed symptoms, behaviors, and psychosocial issues, in addressing sensitive topics, and in helping focus the agenda for the visit (Espallargues et al., 2000; Ruland et al., 2010; Berry et al., 2011; Chen et al., 2013; Crane et al., 2017; Kjaer et al., 2018; Fredericksen, 2020). However, studies of provider perception of value in HIV care have been  limited  to  a  single  study  in  academic-based hospital and clinic settings (Fredericksen et al., 2016). In addition, there is a dearth of research regarding PRO implementation with respect to provider satisfaction. Provider perceptions of the utility and acceptability of tablet-based patient-reported outcomes (PRO) assessment and its integration into routine HIV care in two contrasting North American clinics, a hospital-based ambulatory clinic in Toronto, Canada, and a community-based clinic in rural Fort Pierce, FL, US were assessed.

Provider interviews were part of a broader evaluation project, the PROgress study, the goal of which was to understand the impact of integrating self-administered touch screen electronic PROs into clinical HIV care in two North American HIV care outpatient clinics: St. Michael’s Hospital (SMH), in Toronto, Canada, and Midway Specialty Care Center (MSCC) in Ft. Pierce, FL, US, between August 2018 and July 2020. A full description of the PROgress study is available at: https://progresshivcare.org/.

PROs were integrated into clinical care in each clinic. PWH self-administered a ~10 min PRO assessment of several clinical domains on-site immediately prior to their routine care visit. Study coordinators delivered paper-based feedback to providers immediately after completion which succinctly summarized the results, which providers reviewed just prior to meeting with the patient. Pager alerts informed providers in real-time when patients indicated frequent suicidal ideation and/or intimate partner violence in the PRO.

After PRO results had been routinely delivered to providers for at least 1 month, we conducted 1:1 semi-structured interviews with providers about integration into workflow and practice, and perceptions of usefulness to practice and patient care.

Sites

Study sites were selected based on (1) their expressed interest in exploring PRO implementation in their practice, (2) having a substantial number of providers to experience the intervention, (3) substantial patient caseloads per provider, and (4) patient demographic, clinical, and geographic diversity. SMH in Toronto, Ontario, is an urban outpatient hospital-based clinic serving ~1800 to 2000 PLWH; 70% are men, with a high proportion of men who have sex with men (MSM). MSCC in rural Ft. Pierce, FL, serves ~1500 patients, with a high rate of uninsured and impoverished patients, 60% of whom are women. More than half of MSCC patients are ethnic minorities.

PRO assessment

PRO measures were selected based on brevity, validity, and potential to inform provider decision-making during the visit, building on lessons learned from PRO integration in the Centers for AIDS Research Network of Integrated Clinical Systems cohort⁵. Each clinic’s leadership, which included site providers, were availed of common PRO measures and solicited for discussion of site-specific PRO needs. At both sites, the PRO assessment included: mental health [depression/suicidal ideation (PHQ-9) (Spitzer et al., 1999; Kroenke et al., 2001), anxiety (single item from HIV symptom index) (Justice et al., 2001)]; health behaviors (antiretroviral adherence  (Simoni  et  al.,  2006;  Lu  et  al., 2008), substance use  (ASSIST, AUDIT-C) (Bush et al., 1998, 2002; Bradley et al., 2003; Newcombe et al., 2005), nicotine use (Kiechl et al., 2002; Nance, 2017), sexual risk behavior (Fredericksen et al., 2018); circumstantial factors [housing status (Whitney, 2020), intimate partner violence (Fitzsimmons et al., 2019)], and other forms of screening [nutrition (Canadian Nutrition Screening Tool) (Laporte et al., 2015), attitudes toward medications (2 items from HATQOL) (Holmes and Shea, 1998), sexual orientation, gender identity]. For all measures, questions had multiple choice response options; the HATQOL used a Likert scale. An exception was a question which used a visual analogue scale for measuring self-reported antiretroviral adherence; for this, patients estimated the percentage of medication taken in the past month.

Measures unique to individual sites included a measure at SMH querying citizenship status, as well as a measure querying sex practices under the influence of illicit drugs. MSCC included a review of symptoms index. The number of questions included in the assessments ranged from 65 to 101 at MSCC and from 51 to 100 at SMH depending on skip logic, which was programmed to minimize patient response burden. A small number of questions were presented to the patient on the screen at a time with large touch-sensitive buttons next to each response option. After responding to the final question on the page, a new screen with subsequent questions automatically appears. Patients navigate forward or backward in the assessment using arrow icons at screen base, allowing them to skip a section without answering by using the forward button, or to change their response from an earlier screen. At the bottom of the screen, a progress bar indicates the proportion of the PRO assessment that has been completed. Staff may also view this bar from their own work stations, and remotely monitor patient progress.

Results delivery to providers

At each site, a designated project Research Coordinator delivered the PRO results to providers. The PRO results were shown on a 1-page physical print-out which listed health domains in order of clinical relevance/urgency that had been previously agreed upon by each clinic’s providers and clinic leadership. Health domains were listed on each side of the print-out.

Interview guide development

Two seasoned qualitative researchers with expertise in evaluating patient perceptions of PRO data collection in HIV care developed a semi-structured interview guide in conjunction with team members with expertise in health evaluation research and clinical HIV care. Areas of inquiry were informed by previous study findings with providers on this topic (Fredericksen et al., 2016), and included ease of integration into clinic flow and practice, perceived usefulness to their practice, impact on workflow and time, impact on patient-provider communication, and impact on quality of care.

Interviews

Providers on-site were interviewed after at least one month of having routinely received PRO results, in order to ensure they had time to integrate the new procedure into their practice. Interviews were up to 60 min in length. Providers were not remunerated for interviews. Qualitative data were collected by digital recorder, and transcribed by an external transcription agency (Verbal Ink). All study activity was approved by respective ethics boards; for MSCC, approval was gained through the University of Washington Institutional Review Board, and for SMH approval was gained through the Review of Ethics Board within that institution.

Analysis

Initial analysis was performed using Dedoose qualitative software, first coding within general pre-established thematic areas based on the interview questions. These codes included: perceived impact of PROs on patient care, integration into clinic flow and practice, impact on workflow and time, and usability of the PRO results form. Within each thematic area, using an “open-coding” process, two coders independently identified key concepts, reconvening to reconcile interpretive differences in coding schematics toward the goal of creating a unified coding scheme for data categorization. Within the theme of “impact on care”, we created the following codes:  (1) usefulness for prioritizing discussion topics with the patient; (2) usefulness for identifying topics prone to social desirability bias; (3) allows for deeper line of inquiry; (4) helps familiarize with patients; and (5) improves care. Additionally, we created themes within “integration into clinic flow and practice”, including (1) initial issues, (2) impact on provider workload, (3) usability of feedback form, and (4) general recommendations. Coders then used this schematic to code remaining data, and used a memo-ing process to summarize each provider quote in order to help reference the range of types of statements at a glance.

Of the providers interviewed (n=12), five MDs, one PA, two Pharm Ds, one nurse practitioner, one social worker, and two RNs (four at Midway, seven at SMH) were interviewed. Interview  excerpts are organized here first into two sections: (1) impact on care, and (2) integration into workflow.

Impact on care

Providers reported the use of PROs had a positive impact on care, particularly in terms of helping prioritize topics for discussion, reducing social desirability bias, legitimizing deeper lines of inquiry on sensitive topics, helping familiarize providers with patients and the scope of their needs, and improving identification of issues with subsequent increase in referrals. Responses in each of these areas are summarized:

Helps prioritize topics for discussion

Several providers commented on the usefulness of PROs for helping set the agenda for the visit:

“You have the whole picture of what needs to be addressed in that visit, what we are missing…[otherwise] you won’t be able to ask so many questions in such a short period of time. So to me, it’s a great tool” (Provider #1, SMH).

“We don't usually have that much time because our patient load is large…it's not possible to address all of those things [in the PRO], but the fact that it's all printed out for you, it's there, and wherever concerns are, it's highlighted so it can be dealt with” (Provider #2, SMH).

One provider noted the usefulness of the PRO in allowing them to skip topics that are not currently an issue for the patient:

“It’s a launching pad for discussion. So instead of re-asking all these questions, my usual drill of asking about medication adherence, tolerance, side effects and all that stuff…I have them look at the [PRO results summary] right away and talk about adherence, taking their meds…but if they said 100% [adherent], I wouldn’t go over that…you’re not spending a lot of times asking about things that are irrelevant, because they already asked them on here” (Provider #3, SMH).

Helped identify issues prone to social desirability bias

This was particularly true of depression and suicidal ideation:

“…A lot of things are being picked up that, even after [the patient] being in clinic that day, had not gotten picked up before…somebody’s depression score was rated very high…so clearly, things are not obviously being recognized with just our conversations” (Pharmacist #1, SMH).

“Before the PRO was happening, rarely, rarely would I engage in questions about mental health– only like in the most obvious [situations], like you look at someone and they're clearly distraught, maybe tearful and that sort of thing. But as everyone knows, I feel you can mask that stuff very easily. People do it all the time every day. And unless prompted or asked specifically around depression and suicidal ideation, people won't admit to it…there's also a big stigma and shame around mental health issues. These are literally people that I don't think I would have flagged…the best element of this tool is, if nothing else, it helps us engage in a topic or conversation that would be awkward otherwise” (Provider #4, SMH).

“I know I’ve learned more about the patients, 'cause I've been in this clinic a long time and I know a lot of the patients a long time. And to see that some have thought of suicide, I would never think that person has gone there. So it just goes to show patient may be coming for their visit every three or four months, and you really don't know exactly what's going on in their lives…so, [the PROs] helped me to know my patients better” (Provider #2, SMH).

It was also valued in the insight the PROs shed onto substance use:

“It offers opportunities to explore something that might have been glossed over a little bit. In some patients we're

seeing a lot of alcohol use and a lot of cannabis. It sometimes helps us, to have another way of asking… because sometimes when we ask people directly, you get different answers. We're finding people who are smoking more cannabis than I had realized, because it's easy [in person, for a patient] to say, "Not too much." And even then I push to try to get people to quantify it, but those sorts of questions are probably better asked on the tablet here” (Provider #5, SMH).

“A lot of it will confirm things that I already know. It does help me be more sensitive to issues that I wasn't aware of. Alternatively, if I have a patient who says to me they're clean and I see on the PRO that they're not, that gives me an opportunity to intervene and link them to some care that I wouldn't have offered otherwise” (Provider #1, Midway).

Providers reported that the use of PROs reduced their own social desirability bias, allowing them to feel more comfortable following up with questions that otherwise could be perceived by patients as too personal or asked ‘out of the blue’ without prior context:

“[The PRO] allows us to engage…., 'cause I can just say, "Hey, the reason I'm here talking to you right now is that I couldn't help noticing, when you were on that iPad, that you hit this button that said that you had sort of a lot of thoughts about self-harm. I'm just curious to know what's going on with you." That is easy. I can say, "This happened, which led to this question." Right? It's a very easy flow” (Provider #4, SMH).

“Sometimes you don’t want to ask. Like you know that their partner has died, and they have many other issues, and you don’t want to ask….‘oh, by the way are you having sex again?’ Now you know” (Provider #2, Midway).

One provider noted that the PROs likely reduced their own unintended bias, as well, challenging their own presumptions about patients:

“I think it was nice because sometimes I don't think to ask that particular question of that particular demographic, and to have that down in black and white across the board for everybody, it was interesting to see some answers that might have surprised me…I had a few patients that noted that affordable housing was difficult for them and it has never really been anything we approached, and the patient was always [adherent to ART and stable], so I never really questioned it. And to be able to say, "Hey, this patient has trouble paying their bills, their electric bill," and to be able to deploy my case management team to help link them to community resources was nice. And I was really shocked when I saw that, 'cause it's a  patient that always shows up, is always so nicely dressed and very put together. And to hear him say, "The money only stretches this far and then we have to struggle the rest of the month," it was an interesting perspective that I probably would have never seen without the [PRO]” (Provider #1, Midway).

Allows for deeper lines of inquiry

As an impartial data collection tool that reduces social desirability bias, some providers reported it allowed for deeper questioning on sensitive topics:

“If there's something that's flagged, then I'll ask them more specifically about it. If the screening question for alcohol use shows up, then I may probe a little harder than I would have. I try to remember to ask about alcohol use routinely anyway, but [having the PRO results available] might just cause me to go a little deeper. If there's a relationship issue, I'll ask more about that as well” (Provider #5, SMH).

“I look at [the PRO results] with them and I discuss it with them. It's…this third thing is in the room with you. Yesterday, I had this patient who has two partners and so, "How did that go? Are you back with your partner?" It creates more diverse questions by [reviewing the results] with the patient. "Why are you depressed? What's going on here? Is this new?" (Provider #3, Midway)

One provider noted that the use of PROs gives them more to discuss with their relatively stable patients:

“For me, the patient concerns are the most important part of their visit. The T-cell environment, all this stuff, is minor now. The [antiretroviral] medication is what it is. [The PRO] creates another discussion point for taking care of patients, apart from the routine. How many times can you tell a patient for ten years their T-cells and viral load counts are fine? You want to tell them about something else” (Provider #3, Midway).

Another provider found that PROs, in their ability to illustrate symptoms over multiple time points, helped open more evidence-based inquiry about changes over time, in this case, for depression:

“Some [patients’] depression scores would vary. Maybe somebody was a 9 (on the PHQ-9) and it was mild and then [at a later visit] was 15 and it was moderate. So I would be, "What's going on with life? How are your depression symptoms? Has anything happened that might make this worse?" That would be the biggest difference, that I noticed” (Provider #1, Midway).

Helps familiarize providers with patients and a fuller scope of their needs

Providers   reported   that   the   PROs   were   useful   for orienting themselves to the needs of patients with whom they were less familiar, such as patients new to care:

“Well, since I'm getting to know them, it gives me an idea of them…is this a patient that's engaging in risky sex? Should I have more of a conversation regarding that, as opposed to me just going to the room and addressing the HIV? Maybe we need to sit down and have a conversation about other sexually transmitted infections that they could get. And of course, [the PROs] help us address mental health that most patients don't want to talk about” (Provider #4, Midway).

This was also true of patients with whom providers had assumed they knew well:

“Honestly, I was surprised at how useful I was going to find [the PROs]…I figured my patients and I have a pretty good relationship…they’re gonna tell me all these things, [but I was] hearing things I hadn’t heard before. You’re almost glancing behind a curtain” (Provider #1, Midway).

Other members of the care team outside of the main provider found the information provided by PROs helped put their conversations with them into better context. For example, one pharmacist echoed other such staff in noting the PROs usefulness for helping them understand issues pertinent to ART adherence:

“Normally I would ask questions about pharmacy-related issues, like adherence, drug coverage, drug access, thinks like that...through the [PROs], these questions are already answered, and there's other social determinants of health that might – if someone scores high on depression or suicidality, these are issues that might be barriers to adherence to therapy, so they might be red flags for me” (Provider #6, SMH).

Provides better care/referrals/outcomes

Providers concurred that PROs improve patient care, most notably by identifying previously hidden needs and allowing them to address them:

“[Patients] now feel more safe that they have someone they could speak to if, in the future, that [partner violence] happens… since we've already had this conversation, they know that there's someone in the hospital they can talk to about it if it repeats” (Provider #4, SMH).

“[In the case of suicidal ideation], I have a discussion with the patient about it, and we unpack it just like we would unpack any other issue the patient brings up, and figure out what needs to be done, and give them a chance to talk…we might refer to a mental health provider, for example” (Provider #7, SMH).

Others noted benefits in terms of patients’ relationship with care:

“It makes patients feel as if they’re more involved in their care and that’s important” (Provider #3, Midway).

“I guess maybe the other related impact it has on patient care is…it signals to patients that we’re thinking holistically about their care” (Provider #7, SMH).

Integration into workflow

Integration of PROs into workflow of each clinic was characterized by an initial brief period of adjustment, typically prolonged over a few days.

“It's always frustrating, I think, working in a team dynamic that requires so many moving parts, but we all adapted. At the end, there was minimal complaint. We worked it out” (Provider #1, Midway).

“Overall thumbs up. However, there are certainly things that could have been better I suppose is one way of putting it, or one could also just say as expected there’s lots of things that needed to be fixed along the way” (Provider #7, SMH).

Initial issues included the need to purchase an additional Wi-Fi router in order for iPad tablet reception to extend to distal exam rooms, and the need for a dedicated printer exclusively for PRO printing so that delivery of PRO results were not held up by other print jobs. The most concerning pre-implementation issue for providers and staff, however, was the potential impact on clinic flow. 

“At first, it was very cumbersome. I'd have a 10:00 appointment that I wouldn't see until 10:20 because it took that long to finish” (Provider #1, Midway).

“If a patient is delayed because they're doing the [PRO], then it delays every other patient that comes after them. And some people are much faster on the tablet than other people” (Provider #5, SMH).

Differing speeds of completing PROs, relative to arrival time and provider readiness to see the patient, caused pressure:

“Ideally, it’d be nice if yes, the patient did [the PRO], the nursing goes in, then pharmacy, plus or minus social worker and then the physician to wrap everything up with all of that. But I think because everybody wants to get in and out – I think that gets all chopped up. And so I think that’s … the bottle neck is at the very beginning where they’re trying to get them to complete a [PRO] first. I don't know how to fix that unless you get the ball rolling earlier and   have   [PROs]   already  completed  and  everything completely done…there’s just that…urgency, of course, to get it in to see the patient” (Provider #8, SMH).

“In an ideal situation all these get done prior to myself as the main provider, a physician that walks into the room. That’s not always possible, and my practice is in the clinic when there is time, I’ll wait. But when there’s three or four other patients, when it’s busy, I can’t wait. I can’t hold everything up and twiddle my thumbs waiting for this thing to be done” (Provider #7, SMH).

Initial negative impact on clinic flow required protocol decision-making in both clinics in order to avoid such delays. This included starting patients on the PRO as soon as possible if they had checked in early, creating a cut-off time for stopping the PRO if patients were not done, and having staff remotely monitor speed of progress:

[Patients] usually have some wait time in the room…while they're waiting, they're able to do [the PROs]. It keeps them busy (Provider #6, SMH).

“Once we streamlined the process, and I kind of tracked the progress [of patients completing the PRO] on the website, it slowed things up a little bit less…if [after] 15 minutes and [the patient] wasn’t done, the research coordinator would end the session and print out and deliver the results], so that way we could move on…that was helpful” (Provider #1, Midway).

It also meant as a clinic proactively deciding which patients should not begin the PRO, such as not administering to patients who are late for their appointment, patients with known low literacy, patients who are extremely ill (e.g., nausea) and/or too disabled to self-administer a questionnaire. At times, the decision to administer the PRO was made on a case-by-case basis upon arrival, after having started the PRO.

“I think I pulled [a patient] off [of the PRO} because I thought the time would be better spent with me, and it was probably a suicidal ideation one. Where we already knew from the previous visit it was positive. Or, the patient had a fever that day or the patient complained of chest pain. I’m like, OK, no, there’s no PRO [for that patient] today. We need to move along” (Provider #2, Midway).

Other logistics required resolution during the integration phase, including whether and how other members of the care team outside of the main provider are alerted to particular results (e.g., mental health issues for the social worker, or adherence issues for the pharmacist). One clinician commented on the latter issue:

“I know the pharmacist will look at the drugs and tick off that he or  she  has seen them. The social worker may be

doing something different. I’m not sure what the nurses are doing. We’re hoping after this week when we’ve all thought about it and been forced to think about it, to meet and see how we could use this. Should everybody look at everything?...when I see the pharmacist has looked at the meds [adherence reporting] and they’re happy, I’m less concerned about that piece. I don’t suspect that everybody looks at everything, but I don’t think we’ve decided who’s looking at what and how they deal with the information other than the alerts we sorted out” (Provider #3, SMH).

This issue was easily resolved:

“So what we've decided is, whoever gets that information looks and then if you need to interact and let, like I said, a dietician know or social worker…I initial that I've seen it and then if there's anybody needs to be informed as well, I let them know” (Provider #1, SMH)

Both sites noted that having a dedicated staff member to act as a ‘champion’ in ensuring the integration goes smoothly was deemed essential.

“[The study coordinator] is very central to this, a very important part of this whole thing. I bet you those will be the growing pains once someone is not on site to help us, and it's going to be a little trickier, I think. Then we're gonna have to figure out more systems work around that” (Provider #4, SMH).

“I don't think it can run without a dedicated person...you can give it to [Medical Assistants} in time but you need to be trained…once it gets in, it's seamless” (Provider #3, Midway).

Impact on provider workload

Upon delivery of the PRO, providers reported that the impact on their time and workload was dependent on factors such as the level of familiarity with the patient, the number and depth of concerns identified the provider’s professional style, and the availability of referral resources in-house. In general, where discussions with patients regarding PROs had an additional time impact on workflow, this was regarded as a manageable impact; the identification of issues and comprehensiveness of care were a valuable and acceptable trade-off.

“PROs add time, and on it add time because you know, we now have to explore an issue that was not otherwise going to have been discussed necessarily during that particular visit” (Provider #6, SMH).

“We talk about more stuff. But at least it is helping with depression. It shortens [having to ask about] that one piece…[but]   it  elongates  the  appointments. I  mean,  it depends. If that PRO is pretty clean [not many symptoms or high risk behaviors] it doesn’t. But if we’re talking about three things because of the PRO, yeah. It elongates it [but] even though it’s frustrating to me, I’m like okay, I’ll get useful information” (Provider #2, Midway).

“I really feel prescreening helps and I feel it makes the visit go fast…when you have already found out something that you might not have focused on” (Provider #4, Midway).

“It ebbs and flows…you never know in what clinic how many people will have something that highlights something I need to see them about…so I mean, yeah, it's a slight impact, if any, in terms of workload for me, but it must be said that [depression, suicidal ideation, and partner violence] are 100 percent a priority anyway, so even if I was seeing other people and I was totally crushed in terms of time and that, it's like, whatever, I'll drop it all to see that person immediately. Right? It’s obviously a priority” (Provider #4, SMH).

Usability of PRO results form

Since providers interviewed helped design the PRO feedback form to their specifications prior to project launch, satisfaction with the form, and perceived usefulness as a clinical tool was uniformly high. However, providers had additional requests for modifications for ease of use. For SMH, a common request was to group domains together:

“The format needs to be restructured, I think for ease of readability. So I think…what we’ve proposed makes more sense…our plan is to group topics together going forward: the pharmacist is going to make sure he or she looks at the issues around adherence, and medication payment and things, all the time. [Each provider will] go to that one place [on the form]. The social worker will look at the housing and finances piece. The physician will look at the more medical pieces. It’s easier if they’re grouped together like that, and it’s more appealing and your eyes can follow it” (Provider #3, SMH).

Other requests concerned modification of how scoring and interpretation of results are presented:

“I think the areas that I have to pause and read through are the smoking section or the alcohol section, a little bit, because there's no real key as to what each score is. It just says at risk or not at risk. For the smoking, I have to see, yes, former smoker, then how many smokes per day. Are they currently smoking? It's broken down in a way that I have to read over” (Provider #1, Midway).

For the IPV question, one provider considered the pros and cons of raw scores vs. text-based answers on the feedback form (the IPV  item  displays results in the latter format):

“Scales 0 to 5. I always find that very interesting, 0, 1, no harm, medium harm, to grade of these, meaning how trapped do you feel. [But the current response format is] just open ended, "Yes, I feel trapped." What does that mean? But again, that also now gives you the opportunity to go deeper into those questions by just having a yes. There's two sides to that. If you give a grade for it, it's good for data analytics, but this [text interpretation] works well for providers” (Provider #3, Midway).

Provider recommendations: Looking ahead

Providers recommended several actions that would improve the usefulness of PROs in their practice. The most common of these recommendations was to have the PRO results auto-populate the electronic medical record. Providers also expressed a desire for input on an ongoing basis as to how frequently certain PRO measures are administered, with some interest in tailoring the PRO for individual patients or groups of patients. There was interest among individual providers on specific topics: for example, post-traumatic stress disorder, cardiovascular health behaviors, gastro-intestinal symptoms. Providers also suggested administering PROs in additional commonly-spoken languages within each clinic, including French (SMH), Haitian Creole (Midway), and Spanish (both sites). Finally, providers recommended that the clinic designate responsibility to a specific staff member to oversee the administration of PROs including collection and maintenance of tablets, management of patient flow, and ensuring results are delivered to providers.

Providers reported PROs allowed for more comprehensive identification of issues and concerns, and were particularly valued for identifying and addressing sensitive issues that would likely have been missed, particularly those prone to social desirability bias, such as depression/suicidality, substance use, sexual behavior, and intimate partner violence. In addition, PROs also helped prioritize areas to discuss during the visit, highlighting the most pertinent patient concerns and allowing clinicians to bypass areas of inquiry that had been reported as stable. Impact on time and workflow varied between providers; all providers interviewed found the impact on workflow increased, yet was manageable, echoing prior work which found minimal impact on visit length or consultation time (Fredericksen et al. 2020). Gaining additional information, particularly in the areas of suicidal ideation and partner violence, was regarded as a valuable  trade-off.  These  findings  echo similar   studies which found similar impacts on social desirability bias (Detmar et al., 2002; Velikova et al., 2004; Mark et al., 2008a; Fortner et al. 2008a; Fredericksen et al., 2016; Tufano et al. 2016; Sharma et al., 2016) agenda-focusing (Mark et al., 2008b; Johnson et al. 2008b; Fredericksen et al., 2016; Tufano et al., 2016) and overall comprehensiveness (Velikova et al., 2004; Booth et al. 2004).

Initial integration of PROs into each clinic yielded workflow challenges, such as the need to account for timing of patient arrival relative to their provider’s readiness to see the patient, and patient ability to complete and speed of completion of the PROs. In both clinics, these challenges were met with the creation of or adjustments to protocols, including identification of patients that should not complete PROs (such as those who are severely symptomatic, e.g., nausea), identifying whether an appropriate block of time exists for the patient to complete the PROs prior to seeing the provider, and enhancing real-time communication between varying provider types, such as pharmacists and social workers in addition to the regular provider, in order to ensure all parties had reviewed the results. These initial challenges were characterized as temporary and resolvable.

Providers cautioned that the feasibility of continued use of PROs in their clinics was dependent on the role of a coordinator to oversee and manage their administration, a role which was acknowledged could possibly be integrated into other positions, such as that of medical assistant, in order to ensure minimal disruption to flow. Providers also strongly urged the need for integration of PRO results into the electronic medical record in order to avoid duplication of data entry and to streamline use. Utility of the PRO results form was contingent on ease of interpretation of data and formatting of results into topic-specific blocks, such as medication adherence and satisfaction with medications. These qualifications align with previous studies in which providers also noted that adoption of PROs in practice was dependent on minimal disruption to clinic flow, manageable patient response burden, ease of provider access to data, quantity/ presentation of data, and a focus on directly actionable domains such as depression (Fredericksen et al., 2012; Chen et al., 2013; Fredericksen et al., 2016; Barr et al., 2020). In this project, provider involvement in selection of the domains and measures of interest, as well as provision of input on the format of reported results prior to integration, was likely critical to ensuring buy-in, as has been the case in prior work (Fredericksen et al., 2012). 

As a highly treatable chronic condition, HIV care in recent decades has shifted its focus to maintaining or improving patient quality of life over the life course (Murphy et al., 2001) with a population at increased risk for multiple co-morbidities (American Foundation for AIDS Research 2021). With the median age of this population increasing, the need has intensified for providers  to  quickly  and  effectively address several co-morbidities, concerns, and health behaviors, within the time constraints of a brief visit. The complex needs of PWH in the modern era warrant a broad assessment across multiple health-related domains, with attention to domains such as substance use and depression for which PWH are at higher risk, yet which are prone to social desirability bias. Such domains may be difficult for providers to elicit or for patients to express in this context, even among well-known patients with whom rapport is well-established, as demonstrated by the study. The findings suggest that PRO collection can act as a launchpad, rather than a replacement, for otherwise challenging discussions, and act as an expeditious means of assessing multiple patient needs, allowing for provision of care that is comprehensive, tailored to individuals, and patient-driven. 

Strengths

This is among the first of studies to assess both integration and utility of PROs among health care providers.

Limitations

Providers were interviewed at their place of employment, which may have introduced bias toward reporting favorably toward PROs.

Providers found PROs with results delivery prior to patient appointments both useful and acceptable for routine HIV care. Integration into care was characterized by the need to resolve issues of how PROs fit into clinic flow and was contingent on having a dedicated staff person. The value added by PROs to patient care in terms of addressing topics not otherwise likely to have been identified, particularly depression and suicidal ideation, offset additional burden on clinic flow and provider workload.

The authors have not declared any conflict of interests.

This study was funded by ViiV Healthcare PO# 3001390801. The authors are thankful to the patients, providers, and research staff from the two PROgress study clinics, St. Michael’s Hospital in Toronto, Ontario, Canada, and the Midway  Specialty  Care Center  in  Fort Pierce, Florida, USA. Additionally, they thank the PROgress study steering committee, comprising patients living with HIV, HIV care providers, and HIV care researchers, for their oversight and guidance.