Score

Data acquisitions follow a well-defined protocol composed of regularly scheduled moments, where participants must acquire data in the study. Furthermore, each scheduled data acquisition may comprise the actual measurement of one or more variables, such as physiological (e.g., heart rate, BP, weight) or activity variables (e.g., physical activity, app interactions, etc.) and they may or may not be effectively implemented by the study participant. Additionally, some context information (e.g. the date) will be provided.

Let the consecutive data acquisitions in the remote monitoring protocol be scheduled for time instances i=1,…,n. Assume that each data acquisition is comprised of m variables (e.g., heart rate, BP, weight, etc.). Hence, let xi,j, j=1,…m, be the acquisition of variable j in time instance i.

The goal of the challenge is, given a window of n=12 consecutive scheduled data acquisitions, to predict the adherence during the forthcoming 3 scheduled data acquisitions. The adherence during the next 3 scheduled acquisitions is considered:

  • LOW if number of effective acquisitions is 0 or 1 during the period.
  • HIGH if it is 2 or 3 during the period.

In the context of this challenge, it is considered that a planned acquisition has been effectively implemented by a participant if at least one of the variables scheduled for measurement has been received.

The score of each prediction will be determined by the geometric mean of the achieved sensitivity (SE) and specificity (SP), i.e.,

Where:

      • TP: Predicted adherence is HIGH and ground truth adherence is HIGH
      • FP: Predicted adherence is HIGH and ground truth adherence is LOW
      • TN: Predicted adherence is LOW and ground truth adherence is LOW
      • FN: Predicted adherence is LOW and ground truth adherence is HIGH

The global adherence score will be determined by the average of the GE obtained for each prediction.

Example for the specific dataset provided in phase 1

For phase 1 the specific dataset was collected in Spain using the following protocol:

  • Type of periodic activities: brain training games, finger tapping, physical activity, mindfulness, digital phenotyping.
  • Frequency of acquisition of periodic activities: each activity should be acquired twice a week (as previously communicated to participants on 8th October and 25th October).

In the example shown, it is observed that in the last n=12 planned acquisitions, were effectively implemented the acquisitions in the instants (T-11, T-1 and T); for instant (T-10) no activity was implemented; other instants are not shown. The goal is to predict, in next n=3 planned acquisition instances, the level of the adherence (LOW or HIGH). In this example the level of adherence is HIGH.

You can also download the full explanation with a dataset example here: