mHealth system (ATOPE+) to support exercise prescription in breast cancer survivors: a reliability and validity, cross-sectional observational study (ATOPE study)
A cross-sectional observational study was conducted to test the reliability and validity of ATOPE+ with 22 BCS.
Participants
Potential participants were identified from the referrals received from the Surgical Unit of the Hospital Universitario Clínico San Cecilio in Granada, Spain, between February and August 2021. BCS were eligible if they had been diagnosed with breast cancer (stages I-III), had to have basic mobile phone capabilities, and had at least one year since the end of oncological treatment (hormonal treatment was not an exclusion criterion). In contrast, potential participants were excluded if they had not finished chemotherapy or radiotherapy at least one year before the study start date, had psychiatric or cognitive disorders that prevented from following the instructions of the protocol given, or did not have access to a smartphone.
Eligible women were asked to come to the CUIDATE group’s facilities. A member of the research group explained the assessment protocol and installed ATOPE+ on their mobile phones. They were asked to use ATOPE+ in the presence of a researcher to ensure correct assessment performance. They were also given the materials needed for remote assessment (i.e., ECG device, chest strap, questionnaires and assessments instructions).
Sample size
A sample size of 20 participants was estimated to be necessary to identify an intraclass correlation coefficient (ICC) of 0.8 between the mean square root differences of the standard deviation (LnRMSSD) assessed with the Polar H10 chest band and the ECG (Gold Standard), 90{f5ac61d6de3ce41dbc84aacfdb352f5c66627c6ee4a1c88b0642321258bd5462} power, and an alpha error of 0.527. Considering a potential 10{f5ac61d6de3ce41dbc84aacfdb352f5c66627c6ee4a1c88b0642321258bd5462} dropout rate, 22 BCS were recruited for the study.
Description of ATOPE+ and data collection
To complete the study, patients had to take measurements with ATOPE+ and their comparison instruments (Table 1): Holter monitor22, Perceived Recovery Status Scale23, Sleep Diary24, Emotional Distress Thermometer25, and Borg CR-10 Scale26, during four consecutive mornings, including one weekend day in order to be as precise to normal routine as possible. Patients were told to follow a normal sleep routine during the study. Once they finished the application protocol, they continued filling out the comparison questionnaires given in paper format and the sleep diary. An overview of the ATOPE+ mHealth system is shown in Fig. 1.
ATOPE+ mHealth system overview. Created with Biorender.com.
ATOPE+ was developed by the Biomedical (BIO-277) ‘CUIDATE’ research group and the Department of Computer Architecture and Technology, CITIC-UGR Research Centre, both from the University of Granada, Spain. The development of ATOPE+ is part of the ATOPE project28, registration number NCT03787966 ClinicalTrials.gov, December 2019.
The ATOPE+ mHealth system is composed of a cross-platform application (Android/iOS) and a centralized secure server. The application provides patients with an interface to record their HRV and to report their wellness through questionnaires. The centralized secure server enables data storage and processing, as well as the generation of tailored exercise prescription according to expert rules. The architecture and usability of ATOPE+ have previously been described16. The registration code of the system is 1710092555522.
Once the research team has installed the application on the participant’s phones and created their personal profiles, patients were ready to start using the application. In the main view, patients were able to read a quick tutorial of how to perform the assessment or start it. The measurement started once they pushed the “Start” button, so they had to be prepared before pushing the button. The assessment of the HRV was first. A notification with sound and vibration alerted the participant that this first step was completed, and the rest of the protocol continued.
Perceived recovery, sleep satisfaction and fatigue were assessed with horizontal continuous Likert scales from 0 to 10 with labels in the values at the extremes and a continuous slider included in ATOPE+. For emotional distress, the scale was positioned vertically. The final part of the assessment consisted of performing 10 repetitions of the “Sit To Stand Test” (STS) and assessing the fatigue perceived after the effort with a rating of perceived exertion scale from 0 to 10. After that, the evaluation was completed. The answers were sent to the server, and the participant received an automatic personalized message about their readiness for either a high-intensity session, a moderate-intensity session, or active recovery. More information about the intervention was published on a previous protocol28.
Comparison instruments
Autonomic balance
Autonomic balance was assessed with ATOPE+ and a Polar H10 chest strap (Polar H10, Polar Electro Oy, Kempele, Finland) connected through Bluetooth and was compared with an ECG (Norav Holter DL800, Braemar Inc, Eagan, EEUU) monitor, which is considered the gold standard. From a 7-min recording, the first and last minutes were cut off to achieve clear and precise interpretations of vagal tone with a 5-min signal, as recommended by the Task Force of the European Society of Cardiology and the North American Society for Pacing and Electrophysiology29. The time domain parameter rMSSD (the square root of the mean squared differences) was analysed.
For ATOPE+ , data were exported to a computer for analysis. As recommended by the Taskforce, all artifacts (ectopic beats, arrhythmic events, and noise effects) in the RR time series were corrected or removed to reduce the chances of substantial deformities that can occur in HRV analysis30. In the case of Holter monitor data, NH300 software (Norav, version 3.0, 2009, Norav Medical Ltd) was used to perform the spectral analysis by using Fast Fourier transform algorithms to remove noise from recordings. The sampling rate was 128 samples/second. The frequency filter was set from 0.05 to 60 Hz. Due to low sampling rate, the software itself applied an interpolation algorithm to improve R peak detection31.
After waking up and emptying their bladder, participants were instructed to moisten and place the chest band and the ECG monitor. Then, lying on their beds facing the ceiling, data recording was performed under the same terms of duration for both devices.
Perception of recovery
The Perception of Recovery Scale was used as a comparison to assess the perception of recovery. It is a subjective self-administered Likert-type scale with scores from 1 to 10 (Table 1) and with a sensitivity and specificity of 0.82 and 0.81, respectively23.
Sleep satisfaction
As a comparison instrument, the subsection of quality of sleep from the consensus sleep diary, a reliable tool for prospectively measuring quality of sleep24, was used. It is a self-reported method that includes quantitative and qualitative aspects related to each night of rest (Table 1). This method, compared to polysomnography, has a kappa coefficient of 0.8732.
Emotional distress
The Emotional Distress thermometer according to “The NCCN Clinical Practice Guidelines in Oncology” was used as a comparison to measure emotional distress. This tool consists of a Likert-type scale with values from 0 to 10, where 0 is “no emotional distress” and 10 constitutes “extreme emotional distress” (Table 1). In the Spanish oncology population, this thermometer has a sensitivity of 0.9 and a specificity of 0.6433.
Peripheral fatigue
The Borg-CR 10 scale was used as comparison for the evaluation of the perceived level of fatigue after physical exertion. After performing 10 repetitions at a rhythm of 40 beats per minute (marked by a metronome included in ATOPE+) of the STS, a test frequently used as a protocol to induce fatigue in the lower extremities, participants completed this questionnaire, which consists of scores from 0 to 10 (“Not at all” to “Very, very hard”, respectively) (Table 1). This scale has a reliability of 0.66 according to the kappa coefficient in the clinical population of women26.
Statistical analysis
A descriptive analysis was performed to summarize sociodemographic and clinical characteristics of participants. Continuous variables are expressed as the mean ± standard deviation, and categorical variables are expressed as numbers and percentages. The normal distribution of the variables was checked by means of the Shapiro-Wilks test. Data that did not follow a normal distribution were transformed into Ln(x) or Ln(x + 1) to enable parametric analysis. All analyses were carried out by a blinded researcher.
IBM SPSS version 24 was used for all analyses (IBM Statistical Program for Social Sciences SPSS Statistic, Corp., Armonk, New York). Bland–Altman analyses were carried out in order to properly establish agreement34 between ATOPE+ methods and Gold Standard methods by using Excel worksheets (Microsoft Excel version 16.55, Microsoft, Washington, EEUU). A 95{f5ac61d6de3ce41dbc84aacfdb352f5c66627c6ee4a1c88b0642321258bd5462} Confidence Interval (CI) was established, and significance was set at p < 0.05.
Reliability
For each outcome measure, concordance between comparison instruments and those included in ATOPE+ was calculated. Bearing in mind that Pearson correlation coefficients, paired t test, and Bland–Altman plots are methods for analysing agreement but not ideal in terms of reliability35, interdevice ICC were calculated to reflect relative reliability (Table 2). ICC scores were categorized as poor (< 0.5), moderate (0.5–0.75), good (0.75–0.90) and excellent (> 0.90)36. Weighted kappa was used for categorical variables. The suggested interpretation for agreement is as follows: ≤ 0 poor, 0.01–0.20 slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and 0.81–1 almost perfect37. Additionally, the standard error of measurement was calculated. These calculations identified within subject variation for each method, indicating the magnitude to which repeated measures changed for participants.
Validity
To determine the validity of ATOPE+, paired samples t tests were conducted comparing ATOPE+ measurements versus reliable measurements. Continuous variables were analyzed by Student’s t-test in the case of normal parametric variables, and nonparametric variables were analyzed with Wilcoxon test. Considering that they only reflect proportional relationships and can cause erroneous interpretation of measurements, to establish the agreement between the comparison instruments and ATOPE+ methods, Bland–Altman analyses were also carried out, which allowed us to see the difference between two clinical measurement devices against each method’s mean. To obtain further information, sleep satisfaction was treated as a continuous variable for this purpose. To establish interdevice agreement, Cohen’s d for effect size was used, with effect sizes categorised as follows: 0 to 0.19, trivial; 0.2 to 0.59, small; 0.6 to 1.19, moderate; 1.2 to 1.99, large; and > 2.0, very large38. The Wilcoxon rank test and effect size were calculated for ordinal variables.
Ethics approval and consent to participate
This study was approved by the ‘Ethics Committee of Biomedical Research of Granada’ (Granada, Spain) (0507-N-18, July 27, 2018). All participants received written and verbal information. Informed consent was obtained from all participants by signing a specific document for this purpose. All methods were carried out in accordance with the Declaration of Helsinki.