Beams Consulting

The Michigan Institute forClinical Health Research is a non-profit organization to enable and enhance clinical and translational research at University of Michigan. Their program mission is to connect potential volunteers with researchers at University of Michigan in order to advance research. MICHR currently operates a website UMClinicalStudies.org, a site that has over 300 studies posted to the site and nearly 25,000registered users of the website.

Key Questions To Understand MICHR

• What is the background history of the client organization’s service?
• What are the pros and cons of the current service in scholarly points?
• What theories and practices has been successful working within this organization?

‍

Historical Background

In 2011, MICHR conducted a research to improve the recruitment web application, UMClinicalStudies.org, for recruiting and retaining volunteers in clinical trials. The researchers investigated the efficiency of system architecture and intuitiveness of the overall interaction and interface. To evaluate these aspects, the MICHR research team use qualitative research methodologies: For evaluating system structure efficiency, Generalized Transition Network (GTN), Comparative Evaluation,Heuristic Evaluation, Vocabulary Analysis had used. For assessing each user groups’ needs, Focus Groups, User Interviews, User Testing, Surveys, Personas and Scenarios had used. The research team extracted design improvement directions that are mainly focused on enhancement of user experience.

This recruiting web application is clearly improved in user interaction aspects because the performance and visual elements of the site, UMClinicalStudies.org, shows tangible improvements. However, the main focus of these improvements is solely on volunteers. In other words, the improvements for study team (recruiters) are not well implemented. MICHR research team acknowledges this fact, and mentioned the limitations of their research: insufficient coverage in recruiting methods and needs of clinical researchers are remained.

This paper, thus, will concentrate on seeking solutions clinical researchers’ needs by reviewing relevant scholarly literatures. Before I address potential solutions, assessing current system is essential. I will discuss the current solution with other relevant scholarly research review.

‍

Strong Points and Weak Points of the service

Strong Points

The improvements had made in three different aspects: system structure’s operational effectiveness, users’ needs monitoring, and successful implementation of new functions.

• System Efficiency: higher precision of matching performance and researchers and volunteers, and collective Bulletin Board improves search time and effort.
• Tracking Volunteers' needs: implemented complaints and unmet needs to current version; shortened reg. process, active 'Help' information provision.
• Successful Implementation: all requirements has implemented successfully.

‍

Weak points

These issues mostly come from limited beneficiaries: target user group was mainly volunteers group.

• Limited Beneficiaries from Current System: the benefit of the new update does not do much improvement for researchers.
• Insufficient Needs Finding for Research Groups: the update is mainly focused on volunteer side. Research Coordinators and Researchers need more managerial functions

‍

Research Objectives

• Supplement of Volunteer Pool: enhancing fundamental need of the system. Researchers use the platform to find the right volunteers. Enlarge and variate volunteer pool will increase the researchers active use of the system.
• Enhancing Management Options: more management tool and providing intuitive dashboard of on-going projects will enhance the usability for researchers.

‍

There was a specific process that was taken in order to analyze the Participant Recruitment Program’s main issues. First, our team assigned individual roles to each member in order to ensure that we stayed organized and structured. To better understand the problem, we set up an initial interview with our client. As previously mentioned, we decided as a team to focus our project on the study team interface of the recruitment tool due to the scope of the project. 

Various individuals with differing job roles were interviewed in order to analyze the client’s issues with greater depth. We interviewed a total of eight people in their workplace. These interviews lasted about 60 - 90 minutes and they were all recorded with permission from the interviewees. Two team members were present for every interview. One team member was the interviewer while the other was the notetaker. In some cases, two to three individuals were interviewed at a time. Each interview was followed by an interpretation session during which interview notes were analyzed in detail and questions were asked and clarified. 

The tool’s administrators, designers, developers, and users were interviewed in hopes of gaining an understanding of different perspectives related to the platform. Each individual provided feedback on features they felt were user friendly and features they felt could be improved. This was useful in assessing reasons for poor user retention rates. It also allowed us to formulate a suggested course of action for PRP to consider, in order to improve the recruitment tool. 

After conducting all of the interviews, the notes obtained were used to create an affinity wall. This wall contained important quotes from interviewees as well as observations made by the notetaker. These notes were grouped by conceptual theme and provided a visual representation of our client’s main challenges, which enabled us to propose specific recommendations. 

‍

In order to assess the problem, the client presented to us, we conducted interviews to find conceptual patterns of challenges experienced concerning the UMHealthResarch.org tool. Interviewees provided a great deal of understanding of the pending issues at hand. There were critical issues that were mentioned by multiple interviewees, and these findings seem to be the most important to address. In depth analysis allows for clear recommendations to be provided, and gives the PRP team a greater understanding of the challenges. 

Although we recognize several issues, we also identified successful efforts to improve user-friendliness of the site. For example, making the volunteer side mobile friendly and the ability to contact and manage volunteers directly through the platform prove to be highly user-friendly features. These unique features increase the usability of this platform compared to other platforms. Overall, users recognized that significant efforts are being made to improve the usability. However, there are certain features that do not seem to appropriately fit the needs of every user, especially from study teams’ perspectives. 

‍

1. Website Functionality 

The majority of our findings suggest that study teams are struggling with specific features on the UMHealthResearch.org website. The overall functionality of specific features on the website such as lack of filters that suit the needs of study teams, unclear labeling, and underdeveloped matching capabilities proves to be a major challenge for study teams. Therefore, ensuring that the tool is user BEAMS CONSULTING FINAL REPORT - DECEMBER 12, 2016 6 

friendly can bode well for the study teams and attract more people to the tool. Issues with functionality of the recruitment website can be divided into two subgroups, non-existent features and improper labeling/design of existing features

1. Non-Existent Filtering Feature 
   The most pressing functionality issue is the non-existent filtering feature that many study teams have suggested, would greatly save time spent on searching for specific volunteers. When a volunteer signs up for a particular study, the site sends an email notification to the study team that posted the specific study. The study team then screens the volunteers and adds suitable volunteers to a list of individuals who have signed up for the study. The problem is that if study teams are conducting multiple studies at any given time, volunteers are not separated by each study. The site groups all volunteers who expressed interest in any study on the same list, without providing any filtering functions to help navigate through and manage the list. Only once a study team member clicks on a particular individual can he/she see which studies that particular person wants to participate in. This issue is well known by the PRP team. As one interviewee mentioned, “We know that filtering is one of the major requests from study teams. The lack of filters makes it difficult for study teams to manage volunteer requests.” However, it can be difficult to address the needs of every study team. Study teams not only want to filter volunteers by study, but also by common characteristics based on information from their profile or their application. This kind of filtering would also assist in determining volunteers suitable for a study, different from the one they initially expressed interest in. 
2. Design Challenges for Volunteer List 
   ‍The second group of functionality issues relates to current design challenges. One instance of this issue is the recommended volunteer list. This feature matches volunteers with ongoing research projects on a continuous basis, and sends the list of successful matches to the study team. This tool is important because it helps increase the pool of potential volunteers for study teams. Although many study teams expressed interest in such a tool, they were unaware of the fact that it already existed. The current label for this feature was unclear to users who did not understand its purpose. Users feel that there is a lack of instructional aid regarding site usage, which could help clarify specific labels and features. 

Another instance of a design issue is encountered when a study team sets conflicting criteria for the type of volunteer wanted in a study. For example, if a study team is recruiting volunteers for a single study requiring two age groups and a volunteer then attempts to sign up for a study by indicating that they fall into one of the age groups, they are rejected from signing up for the study because the program algorithm does not allow the selection of only one age group. This type of error is not notified to the study team and they are unclear as to why they have not received any volunteers. 

‍

2. Communication 

While the functionality issue with the participant recruitment site was a major finding, there seems to be a disconnect both internally and externally about this tool and what pending issues need to be addressed. 

1. Lack of Formal Channel to Report Feedback 
   It became apparent throughout the interview process that when study teams have feedback to provide to the PRP, a formal channel to report this feedback was lacking. Study teams that have already established some sort of rapport with one of the PRP team members working on the website, feel comfortable reporting the challenges that they experienced with the tool. However, if this is not the case, many study teams leave the issues unreported. This is clear evidence of a breakdown in the PRP’s ability to communicate with external sources. When technical issues are experienced with the site, they go into a ticketing system to be sent to the designer of the application. Additionally, regarding external communication, changes made to the website are only announced when a major update occurs. As a result, study teams are often unaware of some functionality changes that could be potentially be useful to them. 
2. Breakdown of Internal Communication 
   Additionally, breakdown of communication internally within the PRP team prevents progress on the recruitment website. The team members of the PRP have regular meetings during which they discuss the status of the UMHealthReseach.org website. It appears that the team is aware, to some extent, of many of the issues that the site is currently facing. The client has even created their own affinity wall to help them sort out some of the problems that they feel need to be addressed. However, while observing the team members interact with the tool, it became apparent that not everyone had a clear understanding of how particular features of the site worked. This is clear evidence of internal communication issues. 

‍

3. Resources 

Another distinct finding is lack of resources. While the PRP team members are aware of many of the issues at hand, it is difficult to progress due to issues of time and funding. Many of the team members are currently working on several other projects and cannot always devote the time needed to improve this tool. In addition, MICHR is a non-profit organization that is funded by a grant and this makes it challenging at times to make advancements to the tool. For these reasons, MICHR cannot easily overcome the limitation of resources. It fundamentally precludes them from getting through the underlying problems. 

1. Other Projects
   PRP team members are constantly working on other projects which hinders progression of the recruitment platform. Multiple interviewees mentioned that it can be overwhelming at times to continuously switch between projects. However, they do feel that there is not enough advancement of this tool specifically. It is also difficult to ensure that the time that is available is used wisely because each team member has their own task list and they have different priorities on the pending issues.
2. Budget Limitation 
   As a non-profit organization, MICHR is run on a grant. This means that they have to stay within the grant and make sure that any changes or improvements are feasible within that budget. In addition, it makes it difficult to hire new team members who would focus primarily on this tool. Interviewees have mentioned that this is a challenge they recognize, may not have many solutions, but still hope there is a way to address. 

‍

The following recommendations are based on our findings, personal observations, and interactions with interviewees throughout the contextual inquiry process. Knowledge from background research of the organization is also taken into account in order to provide recommendations that can be realistically applied to assist the Participant Recruitment Program in finding solutions that address their problems. 

‍

1. Formal Evaluation 

The Participant Recruitment Program is operated by a relatively small team, which utilizes the skills of other MICHR teams such as a designer and programmers, to accomplish their goals. One of the inevitable limitations that prevents progress from occurring on the recruitment website is the fact that all of the individuals involved, are also focused on completing other projects. However, it is important that all team members have a thorough understanding of the tool as it stands today. Since each member of the team has a different responsibility with regards to the website, it is naturally expected that they will have the deepest understating of the features that they interact with the most. Before further developments are made to the website, it is important that all team members have a clear or complete understanding of the full capacity of the site, beyond the features that they primarily interact with. It is suggested that the team complete a formal evaluation of the site and document current features and capabilities of the tool as well as whether or not the features should be unaltered, updated, or removed. In order to improve the product, one must know their product. 

‍

2. Prioritization 

It is quite evident that limited time and monetary resources creates hurdles for the PRP in fixing their recruitment website. However, it is important to keep in mind that although there may not be available resources that will allow for all problems to be addressed at one time, it is probable that there are time and resources available to take on smaller portions of the projects in small increments. After performing the evaluation mentioned above of the website, the team should compile a detailed priority list. An effective form of such a document should include a list of the most important updates that need to be made, specific changes that need to be addressed, other individuals who need to be contacted (designer, programmers) and a tentative deadline for these changes. Based upon our findings and interviews with those who frequently interact with the study team side, the most common issue was the filtering capabilities of the site. Based on this observation, we would suggest that the first technical issue to be fixed should be the filtering capabilities. If anything, addressing this issue will save time for many of the users and make their experience more pleasant. 

Assessment of priorities also applies at the level of each individual team member. One must take a realistic look at their individual work priority list and determine where UMHealthResearch.org fits in. If the team members feel that this website truly has value in improving volunteer recruitment, all members must put in the time to actively participate in making improvements. Additionally, although there is a team lead for the program, it is not evidently clear of any one individual taking on the role of project lead. Determining a project lead who can take on the role of delegating tasks and maintaining a progress timeline for the project will help the team stay on track with any updates that are to be made on the site. 

‍

3. Using Available Resources 

One way that the issue of lack of time and resources can be addressed is reaching out to other groups and organizations that share connections with MICHR, in an attempt to share their resources. It would be worthwhile for the Participant Recruitment Program to reach out to the School of Information or the Computer Science department at the University of Michigan to inquire about obtaining faculty who would be willing to volunteer some of their time to assist the developers in fixing programming algorithms, and adding new features to the site. The PRP may even inquire about recruiting the manpower of students from the departments in exchange for their participation. Students could receive some sort of class credit. It is also important to remember that reaching out in this manner could build relationships and continue to spread the word about MICHR’s mission, and the goals of the Participant Recruitment Program. 

The PRP team could also consider launching a pilot program of an updated recruitment website. We suggest that those working on the volunteer recruitment website look at data to determine if there is a particular research department that uses the website the most. Alternatively, the PRP team could also reach out to various departments with groups that are conducting clinical research. It will also be worthwhile to look at the size of each department and the resources that they have available to them. Once a research department has been chosen, offer to that department the ability to be a part of a pilot program. In being a part of the pilot program, this department will be the model for the first round of filtering changes and updates. The assessment of the problem will be initially focused on criteria tailored to this specific group. In exchange for this, the chosen department would lend out any available resources such as extra programmers or extra designers to assist the recruitment program with these changes. 

‍

4. Alternative Feedback Collection 

The Participant Recruitment Program team should also continue obtaining user feedback in the long-term. It is important to continue fostering relationships with research departments across the university, and encourage those who use the site to provide feedback to a designated member of the team. An alternative method of gathering user feedback, which may be worth exploring, is the MouseTrack tool which monitors and visualizes mouse movements activity on any website (Arroyo et al., 486). The benefits of implementing such a tool are its ability to be massed deployed, allowing for large datasets to be collected. The users can use this tool in their natural work environment which can prevent disruption of workflow.

‍

5. Long-term 

Considering a long term goal, it was evident throughout the findings that one reservation that study teams had in utilizing the site was that they were not always able to smoothly navigate the website, and oftentimes valuable features of the site were overlooked because study teams were not aware of its functional capabilities. As the site begins to make satisfactory updates, detailed narrated video tutorials should be posted on the site to instruct users on how to properly take advantage of the all of the features of the application. 

It was previously mentioned that the recruitment tool had been made mobile friendly for the volunteer interface. Making a similar update for the study team interface of the tool would be useful for many study teams who may want to check the progress of their study recruitment from a handheld device. Additionally, quick links to a Frequently Asked Questions page and a Best Practices Guide on how to “game” the site in the most practical way would be beneficial. The execution of this recommendation reflects back to the suggestion of maintaining a complete understanding of the functions of the site. In order to instruct study teams on how to use the site in the most efficient manner possible, those who are constructing the help pages have to be sure to understand the ins and outs of the site 

‍

One of the goals of the Michigan Institute for Clinical and Health Research is to enhance clinical and translational research across the University of Michigan. The Participant Recruitment Program within this organization is helping to fulfill this goal by assisting study teams with the process of recruiting volunteers, to participate in research studies. The efforts of UMHealthResearch.org is appreciated by several study teams, however the experience each of these study teams have while using the site can be greatly improved. Throughout our process of contextual inquiry our team aimed to discover the reasoning behind poor retention of study teams in using the site. The main finding dealt with issues in the technical functionality of the website. Issues with this aspect of the recruitment tool seem to discourage users from using the site as a frequent aid for recruitment. Working to prioritize the pending issues and discussing these with not just the platform team members but leadership as well, can allow the right steps to be taken in advancing the recruitment tool. Through the interviews, it became evident that not PRP team members understood the site and that there is a feeling that progression of the site is moving rather slowly. To address these issues BEAMS Consulting suggests a thorough evaluation of the site in its current state to garner a clear understanding of the tool. Additionally, it is suggested that the PRP prioritize the updates that they wish to have made to the site, and focus on each prioritized goal one at a time. We truly believe that the recommendations provided in this report will assist the Participant Recruitment Program in creating a recruitment tool that will meet the needs of its users. 

‍