3. Study Arms Testing

As a part of your pre-launch testing plan, you should ensure that your project's study arms and intervention are built as outlined in the protocol.  This involves enrolling test participants to each of the project arms, simulating what it will be like for a participant randomized to that arm. Use the attached testing template and checklist to test and confirm each part of the study. 

 Unsure how to build arm events? See Building: Project Arms and Events for guidance!

1. Enroll test participants into each of the arms
   
   
   

2. Pretend like you are a participant in your project- sync your device, answer bi directional prompts, complete surveys, etc. You should plan to do this multiple times, testing what should happen (e.g. I am eligible for the lottery winnings if I met my step goal yesterday), but also what shouldn't happen (e.g. WTH accepts 'yes' to a numeric text prompt). 
   
   
   

3. Goals of arm testing 

   1. Understand what it's like to be a participant in your project

   2. Identify bugs, misconfiguration, spelling mistakes, etc. prior to launching with real participants 

   3. Confirm that intervention is identical to what is outlined in the protocol 

Device events

1. Sync the device, open the pill bottle, step on the scale, etc.  Make sure that the data attached to the event by checking in Manage Data. 
   
   
   

   Open

   

   
   
   
   

2. You can click into the details view of the event to see more information about the data collected and the event time range (e.g. 4/1 12 pm - 4/2 11:59 am) and the completion time (e.g. 8:47 am). 

   1.  Completion time = when the participant completed the event by either A. stepping on the scale which triggered the event to close or B. at the end of the event period (11:59 am)
      
      
      

      Open

      

      
      
      
      

3. If there are compliance criteria built onto the event, be sure that WTH evaluated the data properly.  For example, if my step goal is 7000 steps/day and I walk 7327 steps, the event should be marked as compliant because I met my goal. Compliance can be quickly assessed by filtering for the participant and the event in Filter Encounters.  If the participant met their goal, there will be a green check mark in the Complied column.  
   
   
   

   
   
   
   

   1. You should also test that if the participant doesn't meet their targets, WTH does not mark them as compliant 
      
      
      

4. Feedback messages evaluating the device data should be correct

   1. For example, if I met my goal yesterday, my daily message should reflect that. 
      
      

Bi-directional SMS events 

1. If you are asking participants questions via text, you will need to test texting in different responses and edge case scenarios.   

   1. Example: Daily text- "How many fruits and vegetables did you eat yesterday? Send a response 0-10."

      1. 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 

      2. .5 

      3. 11 

      4. 2.5 

      5. Hello 

      6. Three 

   2. For an incorrect response, did I get the "we don't understand" auto reply? 

   3. For an incorrect response ("Three") and then a correct response (3), I get the proper feedback (Great job!)?
      
      
      

      
      
      
      

2. Confirm that the correct response attached to the event and any incorrect responses did not attach. 
   
   

Survey events

1. Practice filling out any surveys that the participant will receive (mid point, end of study) in the same manner that they will fill it out, whether via an emailed/texted SURVEY_LINK or on the frontend dashboard 
   
   
   

                 

   
   
   
   

2. Did you receive the proper feedback upon completing the survey?
   
   
   

   
   
   
   

3. If there is a financial incentive for completing the survey, was the participant credited? 
   
   
   

   
   
   
   

4. If the survey is an Office Visit Event (only filled out by the project team), ensure that the survey does not appear on the participant dashboard and only in the admin Events view in the participant's profile.
   
   
   

Survey reminder events 

1. If you have round up events sending reminders to participants for incomplete surveys, you should make sure that the reminder goes out at the correct interval (e.g. 3 days after the survey first opened)
   
   
   

2. The reminder should only get sent to participants who haven't completed the survey.
   
   
   

3. The reminders should cease once the survey is filled out. 

Syncing reminder events 

1. If you have round up events sending reminders to participants to sync their device, you should make sure that the reminder goes out at the correct interval (e.g. 3 days after last sync)
   
   
   

2. The reminder should only get sent to participants who haven't synced their device over the past 3 days.
   
   
   

3. The reminders should cease once the participant syncs their device.