Custom Data Source (Manual Upload): CSV File Import

Owned by Former user (Deleted)
Last updated: May 09, 2022 by Sarah.Milinski
5 min read

Data can be manually uploaded to W2H using the CSV File Import feature. This is useful for programs that may want to capture data from a source that is not integrated with W2H. This is a great alternative to a full data source integration which requires development resources and funding. CSV File Import is ofter used in smaller pilots, when project teams are establishing proof of concept or running a feasibility study. An example would be pulling appointment data from the electronic health record in order to schedule messaging around those appointment dates. After the pilot shows results, then a full integration can be considered.

Creating your template

To start, you’ll need to have a sample CSV file from the external data source. The file should include a column that matches to one of the following W2H profile fields: participant ID, short participant ID (excludes an prepended 1000-), username, mrn, or cell phone. In addition to these profile fields, there is also an option for a foreign participant id. That can be added to the participant’s record in W2H by navigating to the devices tab once the CSV data source is setup.

You can specify the type of data in the second row of the template. Data types include:

  • String

  • Number

  • Date

  • DateTime

  • Time

  • Boolean

  • Enum

  • String[] - An array of strings. Example of a value for an activities field of this type: "['swimming', 'running', 'biking']"

Example template

In this example, we will match on MRN. We are uploading appointment information. The timestamp field tells us when the data was extracted from the external data source.

 

Set up

  • Navigate to Manage Study > Data Sources > SMS/Other tab and click + New SMS / Other Data Source.

  • Name the data source and select Custom Data Source (Manual Upload) from the Type drop-down. 

  • Select the Participant match field. This is the field W2H will use to link the data to the participant. Skip the Participant field and the Date field and hit Save.

The Participant field is the column name in your CSV file that will match to the unique identifier in the Participant match field.

The Date field is the column in your CSV file that determines the timestamp for a row of data.

Both fields will have nothing in the dropdown until CSV configuration is complete.

Upon saving, you will return to the Data Sources page. A banner will appear indicating that your configuration is not yet complete.

Go to Manage Study from the main menu and you will see a new submenu item called CSV Configuration. Select that to continue setup.



Click Browse and find the template CSV that includes the expected column headers (row 1) and data types (row 2). The Data Source dropdown will default to the Data Source you just created. Click Upload CSV.

A modal will appear with a warning that uploading a new template will update the data source fields for the study. This is most relevant when making a change to the template AFTER launch. For initial setup this will not be the case. As such, click Ok.

 

 

Once you have uploaded the CSV Headers, you may need to wait up to 5 minutes for a job to run in W2H to make the fields appear in the database. This means there will be a lag between the current and next step.

Return Manage Study > Data Sources > SMS/Other tab. Click on the pencil button next to the data source you previously created.



Update the configuration with the Participant field and Date field. click Save.


Upload cadences and general best practices

The data from your custom data source will be driving some part of your digital intervention. You should work with the project team to identify an upload cadence based on the needs of the intervention. Here are a few scenarios:

  • If participants will receive a weekly feedback message on Fridays, you might consider starting all participants on a Monday and conducting uploads on Thursdays.

  • If participants will receive daily feedback messages, you will need to be sure someone is available to conduct daily uploads.

  • If you are using a date column from your data source to trigger actions and messaging, you will need to upload with enough days notice. For example, you might build an event block with reminder messages for an appoint to go out 2 days befor the appointment. You’ll want to make sure all dates in your file are at least 2 days in the future before conducting the upload.


Uploading data

When you are ready to upload data from a csv file, navigate to the Manage Participants and go to CSV Upload.

Click Browse and find the csv file on your computer that includes data from your external data source. Make sure the Data Source name matches the one you created. Click Upload CSV.

You will see a banner confirming a successful upload. If you upload a file that includes column headers that W2H is not expecting, the file will fail.

You can go to Manage Data and view the uploaded data immediately.


Leading Zeros

Some fields such as MRNs often contain leading zeros. This has been a pain point for us in the past. It does seem that when editing the CSV in excel you can set a custom column type with the number of expected characters (HUP MRNs might have 9 characters) and it will maintain (or even add in if needed) leading zeros.