Programs to Recruit and Retain Staff Who Reflect the Community’s Cultural Diversity

Part I. Community Guide Update and Rationale for Intervention

1. Recent Evidence

Author & Year	Intervention Setting, Description, and Comparison Group(s)	Study Population Description and Sample Size	Effect Measure (Variables)	Results including Test Statistics and Significance	Follow-up Time
Salto, Riggs, Leon D., Casiano, & Leon M., 2014	Inland Empire Region of Southern California Apprentice-ship Bridge to College (ABC) 8-week summer research experience, including individual research mentors that guide research findings, interpretation and presentation Undergradua-te training program (UTP) to aide in underrepres-ented minority students experience in biomedical research to grow diversity in healthcare staff committed to addressing health disparities 40 hours/ week  No control group used	53 (n=53) underrepres-ented minority high school  students aiming to pursue biomedicine or become physicians	Pre/Post-Test Survey Intent to pursue a continuing educational degree in biomedicine or desire to become an MD	89% increase in students in both ABC and UTP intent for research to be a part of career intent (p<0.001) 39% increase in students intent to seek an MD/PhD  from pre- to post-test 64% increase in students intent for research to be a part of career intent (p<0.001)  4% increase in students intent to seek MHD/DrPH-Public Health Statistically significant results in self-assessed research skills and research self-efficacy	Follow-up data was measured retrospe- ctively for 16 years of the program’s existence, but no exact follow-up timeline was stated
Guerrero et al., 2014	UCLA, Los Angeles, CA Pathways for Students into Health Professions (PSHP) Program includes 4-unit course, summer field practicum, volunteer experience, and academic advising and counseling to recruit and train undergraduate students from under-represented minority groups to pursue maternal and child health (MCH) professions No control groups used	39 (n=39) undergraduate students from 7 different race/ ethnicities	Pre/Post-Test Survey Student’s ratings of received academic advising, post-college plans, and interest in seeking MCH career	PSHP resulted in a significant difference  on intended goals (p<0.05)  38% of the students anticipated  a future career in medicine Statistically significant increase in areas including Knowledge about MCH, Knowledge about cultural competence, and Knowledge on how MCH topics will influence their future career  Results show program can strengthen a diverse healthcare workforce	One year follow up survey
Alfred et al., 2010	San Diego, CA 8 week summer research training program  No control groups used	n=82 Undergraduate college students age 18-30, that are at least one of the following: Underrepresen- ted community member, low income, or neither parent holds degree from 4-year US college	Pre/Post- Test Survey Student’s continuation in science, retention in science until graduation, and continuation on to further education in science	93% persistence rate in science of program students after 1 year 56 students graduated as science majors  25 students continued on to graduate school  10 students continued on to medical school P values not given because UCSD did not publish actual numbers of comparison students in science majors (not able to determine significance)	One year follow up survey  Continued outreach and surveys throughout completion of college career

2. &   3.

In 2001, the Community Guide found that Programs to Recruit and Retain Staff who Reflect the Community’s Cultural Diversity, were not able to be recommended due to lack of comparison groups. After searches in the UGA library database, and the Web of Science Journal specifically, the above studies on the target intervention were selected. Based on the original Community Guide done in October 2001 research of Programs to Recruit and Retain Staff who Reflect the Community’s Cultural Diversity, combined with the research presented in the table above, the updated strategy recommendation remains at Insufficient Evidence. 

The recommendation remains unchanged due to lack of finding new studies that include a comparison group. For example, the Alfred et al., 2010 study was unable to determine a significance for their study because the university would not release the number of comparison students in science majors. 

In the study published in 2010 by Guerrero et al., 39 students were recruited as part of the Pathways for Students into Health Professions (PSHP) program. The program allows undergraduates of a minority demographic to learn about, experience first hand, and be mentored in the field of Maternal and Child Health (MCH), with hopes to recruit and diversify the medical staff population. The results tested were the student ratings about MCH in general and as a personal profession, and the sufficiency of academic and career guidance. To measure the results of the program, a Quasi Experimental Pre/Post Test Survey design was used.  With an 82% response rate, knowledge about maternal and child health showed a significant increase, but the minority population’s interest in pursuing and contributing to a career in the topic did not show significant between baseline and the follow up survey. Therefore, while the target goals of the particular study were somewhat positive (increase in overall knowledge of MCH as a career), the part of the study as it pertains to the Community Guide intervention (recruitment of minority students for MCH, or medical in general, career) is not accomplished when the gain in interest to MCH programs resulted statistically not significant. Lastly, because this study did not have a control group, it is not able to change the previously researched finding of Insufficient Evidence, and therefore agrees with it (Guerrero et al., 2010).

In the study published by Alfred et al., 82 students applied to the UCSD Cancer Center’s Continuing Umbrella of Research (CURE) Experiences program at the University of California San Diego (UCSD). In this program, students must be at least one of the following: Underrepresented community member, low income, or neither parent holds degree from 4-year US college. This aligns with the target population for the selected Community Guide Intervention. The program consists of 8 weeks of lab and classroom training and mentorship in the research labs of program mentors. The goal of the program is for a significant portion of minority students to continue on to graduate or health professional schools, also looked at as recruiting a diverse medical staff. Results of the CURE program showed 61% of participants continued on to post-undergraduate schooling. However, no control group was measured as part of this study. There the ethical dilemma of creating a true control group while other enrichment opportunities for control group participants exist and nullify the group (Alfred et al., 2010). Therefore, this study is not able to change the Insufficient Evidence ruling of the 2001 Community Guide findings. 

Lastly, the Salto, Riggs, D. Leon, Casiano, & M. Leon study done in Southern California resulted in similar evidence findings. This program is split into two parts, the Apprenticeship Bridge to College (ABC) summer research experience, and the Undergraduate Training Program (UTP). This program was designed to provide summer research experience to underrepresented, minority high-school students, providing individual research mentors that guide research findings, interpretation and presentation. The program was also designed to provide experience in biomedical research in order to grow diversity in healthcare staff committed to addressing health disparities. A total of 91 (n=91) participated. The statistically significant results from the finding included an 89% increase in students in both ABC and UTP intent for research to be a part of career intent (p<0.001), and 64% increase in students intent for research to be a part of career intent (p<0.001). This can be summarized as statistically significant results in self-assessed research skills and research self-efficacy (Salto, Riggs, D. Leon, Casiano, & M. Leon, 2015). Unfortunately, like the other studies, this research also does not include a control group, the measure in which disqualified the previous Community Guide findings from being anything except Insignificant. 

Overall, the studies chosen in the table do not counter any of the Community Guide findings prior to 2001 on Programs to Recruit and Retain Staff who Reflect the Community’s Cultural Diversity. In fact, all of the post 2001 studies provide statistically significant evidence to some kind of benefit in their programs to recruit the target population. However, the ethical dilemma of acquiring a control group remains unsolved. Therefore, the lack of control group for the studies stated above confirms the Community Guide findings of Insignificant Findings. 

Part II. Theoretical Framework/Model

4. Selected Theory

The Transtheoretical Model structures the intervention of recruiting and retaining undergraduate minority students at UGA from ethnically diverse backgrounds into medical school and the healthcare profession. This model flows from the Precontemplation phase, to the Contemplation Phase, to the Preparation Phase, to the Action Phase and finally to the Maintenance Phase. There is some debate on whether this model should contain a sixth phase of Termination, but for this intervention, termination would not be part of the end goal. It is possible for persons to regress into a previous stage, especially before establishing themselves in the maintenance phase. It is said that absolute maintenance is reached only after five years of in this phase.

Precontemplation:

In this phase, minority students are not aware of their need to represent their demographic of the community within healthcare, and therefore are not aware of their need to go to medical school. Even in the most homogenous communities, minority populations need to have their needs met in a culturally competent manner. In this stage, minority students most likely do not understand the need to provide culturally competent care or the process to providing it.

In order to change from Precontemplation to Contemplation, certain methods can be implemented to motivate change. Some methods for minority students to enter the thought of medical school might include raising awareness, changing attitudes and increasing perceived benefits. Consciousness raising might look like testimonies being given from culturally diverse members of the local community stating the reasons why they would like the students to represent their segment of the population.

Contemplation:

The identifying difference between the Precontemplation and the Contemplation stages is the minority student’s awareness of the need to provide culturally competent services by means of medical school enrollment. During this stage, current students may not yet agree with the need for diverse staffing, but the idea of it has been presented. Or the students might not feel obligated to fill that need personally. The importance of this stage comes from the minority students in vision casting the possible, positive impacts that providing a culturally representative healthcare system might create. This process could include an information booth, academic advising, and a summer immersion program.

Preparation:

During this stage, minority students have committed to a pre-medical major and are making actions to do so. Some actions might include learning about perceived benefits through continued academic advising and a summer immersion program. Actions as such would be in motion during the preparation phase.

Similar to the previous contemplation phase, self-reevaluation should occur in this preparation stage. Minority students must accept the need for more training and wish to better themselves and their community through filling the need. Motivation to continue in their personal growth as a professional is crucial during this preparation phase.

Action:

The action stage is where minority students are in the meat of their pre-med major and preparing to apply to medical school. This is the beginning of the behavioral change process, often where behavior relapse occurs. During this phase, reinforcement management, helping relationships, and counterconditioning might be needed to establish concrete dedication of choosing a medical profession. Here, the MCAT is taken as an action step towards commitment to medical school. Helping relationships might aide this process via social support, professional support (letters of recommendation), and physicians of the recruitment team who can lend advice.

Maintenance:

As the end goal, the Maintenance stage is often only recognized as achieved after five years of continuing the new behavior.  For the desired result, for minority students to become trained healthcare professionals, accurately representing their community, retention of five years seems appropriate and minimal. Reinforcement might encourage students to seek residencies where their demographic is present and in need of representation in medical care. Counterconditioning can be used in a similar practice held during the contemplation stage. This action can help prevent relapse into previous stages.

5. 

Part III. Logic Model, Causal and Intervention Hypotheses, and Intervention Strategies

6. Target population and Program Setting 

The target population for the intervention are the students of demographic minority within the incoming freshman, undergraduate class at the University of Georgia. This is the population needed to recruit and retain as a culturally competent healthcare staff to match the demographic of their community. The university is located in Athens, GA, a community with a demographic of 61% White, 26% Black, 10% Hispanic or Latino, and 4% Asian (Current Athens). Meanwhile, the University of Georgia is the pot from which future, community healthcare workers will come. The demographics of it’s population are as follows: 73.4% White, 7.6% Black, 4.7% Hispanic or Latino, and 9.3% Asian (University of Georgia). According to the Needs Assessment done by Saint Mary’s Hospital in Athens, being a part of a racial or ethnic minority is listed as a hardship of the large population living in poverty in the city. Also stated in the needs assessment for the county is the reality that Latinos are among the group specifically facing difficulty receiving health care (Community Health). Most importantly, the demographic of the health care staff in training associated with the University of Georgia does not represent the community. As of 2012, 64% of the enrolled Medical College of Georgia are listed as White, 10% are Black, 3% are Hispanic and 13% are Asian (Medical College of GA). Compared with the demographics of Athens, Clarke County, the Black and Hispanic or Latino populations of the community are the most underrepresented population by the current recruited and training health care providers. Therefore, the current health care staff is not culturally representative of the community and a program to recruit and retain a staff that is culturally representative of the community is needed. 

The setting of the program will take place on both the University of Georgia campus and the Georgia location of the Summer Medical and Dental Education Program (SMDEP). Specifically within UGA, the Tate Grand Ball room will host the orientation information fair, Franklin College department buildings will host the academic advising, and the location from which follow up emails will be collaborated and sent out. The SMDEP will take place at the program’s Georgia location.

7. Intervention Methods

The intervention methods used for the program include raising awareness for the need of minority students to fill medical care need through consciousness raising. The method of changed attitudes in profession is projected through self-liberation and environmental re-evaluation. Lastly, the method of increasing perceived benefits is used through movement of students from the contemplation to preparation stages of the transtheoretical model. In fact, all listed intervention methods are all aligned to the transtheoretical model as a structure of theoretical framework.

8. Intervention Strategies

Intervention Method	Alignment with Theory	Intervention Strategy
Raise Awareness	Consciousness Raising: This method aligns with the Transtheoretical model as it brings the minority students from the precontemplation phase to the contemplation phase. In the pre precontemplation phase, the students are not aware of their need to become physicians in order to provide healthcare that represents the demographics of the surrounding community. This education phase can be implemented into the program during the information fair.	Host a table at each UGA Freshman orientation, information fair with a recruitment team of an MCG rep, Academic Advisor, SMDEP rep, and a current physician  Follow up emails are sent about next steps towards a pre-med major, including info on summer medical experience
Change Attitudes	Environmental Reevaluation: This strategy allows physicians to reassess if and how their misrepresentation of the community in their staffing might affect their patient and their community. This method aligns with the transtheoretical model by helping minority students transition from precontemplation to contemplation by means of awareness. Before students can fully commit to administering culturally diverse care, they must first be aware of its importance and their personal affect on the environment (local community). Self-Liberation: This strategy recruits minority students to believe they can be a part of a change and empowers them to commit to the career path. This method aligns with the transtheoretical model as it pushes (future) healthcare providers from the preparation phase to the action phase.	Host a table at each UGA Freshman orientation, information fair with a recruitment team  Follow up emails are sent about next steps toward a pre-med major, including info on summer medical experience  Summer Medical and Dental Education Program (SMDEP) is attended and aids in the commitment/ self-liberation to a pre-med major
Perceived Benefits	This method aligns with the transtheoretical model as it brings minority students from the contemplation phase to the preparation phase. Brainstorming the perceived benefits for both the community and themselves allows the student to consider benefits and therefore be moved to prepare for such benefits after deciding those benefits are worth moving towards.	Host a table at each UGA Freshman orientation, information fair with a recruitment team  Follow up emails are sent about next steps toward a pre-med major, including info on summer medical experience  Summer Medical and Dental Education Program (SMDEP) is attended and aids in the commitment/ self-liberation to a pre-med major Academic Advising throughout undergrad career

9. Logic Model

Diversify the Medical College of Georgia: A Program to Recruit & Retain a Culturally Diverse, and Therefore Representative, Healthcare Staff

Inputs/Resources	Activities	Outputs	Short-term Outcomes	Intermediate Outcomes	Long-term Outcomes	Program Goal
UGA Orientation Office University of Georgia participation and facilities Table Brochures of MCG & SMDEP Email address for program iPad loaded with community testimony Incoming Freshman class at UGA’s information fair Parents of freshman, minority students at UGA MCG rotation physician   UGA academic advisor Medical College of Georgia representative  Summer Medical and Dental Education Program Representative Minority Student SMDEP fees	Host a table at each UGA Freshman orientation, information fair with a recruitment team  Follow up emails are sent about next steps to a pre-med major, including info on summer medical experience  Summer Medical and Dental Education Program (SMDEP) is attended and aids in the commitment/ self-liberation to a pre-med major Academic Advising throughout undergrad career	16 UGA orientations visited 575 email addresses received from minority students  575 follow up emails sent from recruitment team to minority students 40 minority students in 1 week summer program  8 Advisor-Minority Student meetings held over 4 years	Raise minority student awareness Change minority student attitude Increase minority student’s perceived benefits of pre-med major	Minority students Commit to Pre-med major	Minority students are accepted and attend medical school ______ Program Goals:  (*Unable to access and add this info to the next column) 5-10 years after start of intervention, minority community members receive culturally representati-ve care from medical staff  5-10 years after start of intervention, healthcare disparities are reduced by 5% among minority populations in Athens, GA

Intervention Hypothesis:

The intervention activity of having a Medical College of Georgia/ Pre-Medicine station at the University of Georgia’s freshman orientations raises awareness, changes attitudes, and increases perceived benefits of the minority students. 

The follow up information emails from the intervention team raises awareness, changes attitudes and increases perceived benefits of the minority students.

The Summer Medicine and Dental Education Program (SMDEP) changes attitudes and increases perceived benefits of the minority students.

The academic advisor’s oversight of the minority students throughout their undergraduate career increases their perceived benefits.

Causal Hypothesis:

The raised awareness of the minority students will increase the commitment to the pre-medical undergraduate track.

The follow up information emails to the minority students will increase the commitment to the pre-medical undergraduate track.

The Summer Medical and Dental Education Program including the minority students will increase the commitment to the pre-medical undergraduate track.

The academic advisor’s overview with the minority students will increase the commitment to the pre-medical undergraduate track.

11. Outcome Objectives

Goal 1: Increase awareness of need to recruit and retain culturally diverse populations in healthcare settings.

• Objective 1: 1 day after information fair implementation, 80% of engaged minority students will understand need for culturally representative healthcare by 60% from baseline, articulated via conversations with recruitment team at fair and confirmed by retrospective pre-test, and also post-test survey #1.
• Objective 2: 6 months after intervention implementation, 60% of the minority students who received follow up emails will articulate via post-intervention survey the need for culturally representative health care among a community.

Goal 2: Increase changed attitudes towards desiring culturally representative care of undergraduate, minority students at UGA to pre-medical majors.

• Objective 1: 3 months after intervention implementation, there will be a 10% increase from baseline in freshman, minority undergraduates committed to a pre-medical major at the University of Georgia.
• Objective 2: 1 year after intervention implementation, 60% of minority students will speak with their academic advisor on their baseline attitudes of culturally representative care versus their current attitudes. 

Goal 3: Increase perceived benefits for community and minority students in committing to a pre-medical undergraduate track

• Objective 1: 1 day after information fair implementation, 80% of minority students originally engaged will articulate via post-test survey #1 examples of benefits in medical career.
• Objective 2: 1 year after intervention implementation, 60% of minority students originally engaged will understand and articulate reasons for dedication to track in meetings with their academic advisor.

Part IV. Evaluation Design and Measures

12. Stakeholders

Stakeholder	Role in Intervention	Evaluation Questions from Stakeholder	Effect on Stakeholder of a Successful Program	Effect on Stakeholder of an Unsuccessful Program
The University of Georgia	Intervention setting as host for undergraduate orientation information fair.	–  What are the benefits for allowing MCG to recruit on our campus? – How will the enhanced connection with MCG benefit our organization in the long run? – How will this effort keep our investment in our students local after graduation?	Students become physicians and remain in Athens, contributing back into financial support of the university. Recruitment for diversity from both MCG and UGA combined will increase likelihood of a diverse workforce in healthcare and a continued strong, mutual-beneficiary relationship between the colleges. This might include the sharing of professors via guest lecture, or strengthened recruitment of a healthcare intrigued freshman class.	Connection between UGA and MCG will remain present, but extra benefits like sharing professors and outside recruitment of future freshman at UGA interested in healthcare will be lost. UGA will not think connection with MCG is important. UGA will continue to teach and recruit students of the majority demographic to the medical field.
The Medical College of Georgia	The primary beneficiary of the intervention by receiving a diverse medical school class. They will host a table at the information fair to begin recruitment of culturally diverse populations for their healthcare setting.	– Why do we need a culturally diverse demographic of medical students? – What are the best tools to persuade students to choose a medical career? – How do we best relate to culturally diverse families at the information fair? – What is the cost-benefit analysis of recruitment and retention? – How could a stronger connection with UGA benefit us as an institution? – Could a diverse incoming class help us get increased funding from grants and other donors? – Would a diversified student population make our program more competitive world-&-nation-wide?	A diversified incoming class of medical students Increased clout to advertise and recruit for the most qualified students of diverse demographics. Increased funding of diversity grants for students. Higher number of applications to program, and therefore higher capacity to recruit most qualified students. Higher recruitment rates of medical students from UGA. Benefits of stronger connections with UGA as a whole, including professor sharing and other in-kind benefits.	A continued, homogenous medical student class at the current rate of diversity. Their desire to expand diversity will not be present. Maintained or decreased ability to compete for incoming highest caliber students of diversity. Maintained or decreased connection and recruitment with UGA. Less funding from grants and scholarships received from lack of diversity
The ethnic minority populations in the community of Athens, GA (African American, Hispanic or Latino, and Asian)	The secondary beneficiaries of the intervention by receiving healthcare from culturally representative physicians and healthcare team	– Will the diversified MCG class benefit the Athens community specifically? – Will my specific demographic be accounted for in the new recruitment? – How can I be a part of the intervention? – How long until I will see the results of this intervention in my healthcare staff? – Will a diverse health care staff really benefit my health personally?	Receiving culturally competent healthcare Less readmission to hospitals and clinics due to reduced continuation of disease and illness. Increased overall wellness from no longer feeling stigmatized based on race and ethnicity and avoiding admission to healthcare settings. Population will receive culturally appropriate, patient centered care.	This population will remain untreated due to perceived stigmatization from the majority demographic healthcare providers. This population will continue to go mistreated and readmit to hospitals and/or clinics based on differences in cultural norms, practices, and influences. The overall health of this population in Athens will go underserved and be less than patient centered.
The culturally diverse, minority students among the freshman class at UGA (African American, Hispanic or Latino, and Asian)	These students will be recruited and retained by MCG to choose a medical career for their future. They will learn why they specifically are needed as medical staff in their community.	– Why should I choose to invest the time and finances in medical school? – Do I have an advantage in applying to medical school because I am from a minority demographic? – How can I pay for medical school? – Is the Summer Medical and Dental Education Program a good representation of medical school? – What are the skills developed to provide culturally competent healthcare, other than simply being of the minority demographic myself?	– Students take undergraduate courses that prepare them for medical school – Students remain in close contact and guidance from MCG staff due to the strengthened connection between the school and UGA. – Students check the Med-MAR (Medical Minority Applicant Registry) upon taking the MCAT – Students receive information from a number of medical schools requesting their application because they checked the Med-MAR box on the MCAT. – Students apply to and are admitted into the Medical College of Georgia (MCG). – Students attend MCG and end up practicing medicine in a community of which their ethnicity and race are represented	– Students continue on to complete undergraduate majors other than pre-medicine and contribute to dispersed career fields – Students do not take the MCAT – Students do not check the Med-MAR box on the MCAT – Students do not apply to and are not accepted to MCG – Students do not attend medical school

 

13. Outcome Evaluation Design & the Scientific Notation

This study is a Quasi-Experimental, One Group experiment. This study has no comparison group and a pre-test post-test design. This design is noted as follows: N  O1     X     O2     O3

This study design comes with many threats to internal validity. Some threats include selection bias, history, and statistical regression. Selection bias is a threat to internal validity because it could be an already natural direction and decision based on the development of student in college to choose a pre-medical degree. A retrospective pre-test survey will be given to those students who choose to speak with the intervention team and receive follow up emails by providing their email at the fair. The pre-test survey will ask their level of interest, based on a Likert scale, their level of interest in the medical school and why. One of the questions will be directed at their original, perceived need for culturally representative healthcare. This threat will also be attenuated with a post-test 6 months after the information fair of the intervention, asking their level of commitment to pre-med major based on the information they received. History is a threat to internal validity due to the length of intervention of this program. Because it follows undergraduates throughout their entire career at UGA, infinite possibilities could have occurred to cause a student of an ethnic minority to commit to a career in healthcare between their freshman orientation and application to medical school. An example might be a student in the target population had a family member experience discrimination based on their demographic at a doctor’s appointment and this would be the ultimate reason why they choose to go to medical school. This threat will be attenuated with a second Post-Test survey at the completion of the program (3-4 years from the start) distributed once students have been applied to medical school. The test will ask what the major motivations were for choosing the healthcare career path and their level of intention in serving in an area with their identified, ethnic demographic as a future physician. Statistical regression is also a threat due to the low percentages of minority demographic students that usually apply to medical school. Therefore, the specific target population could contribute to the extreme results of the intervention and with the number that apply and are accepted to medical school.

Short-term or Intermediate Outcome Variable	Scale, Questionnaire, Assessment Method	Brief Description of Instrument	Example item (for surveys, scales, or questionnaires)	Reliability and/or Validity Description
Increase awareness for need of culturally representative healthcare	Adaptation of VERB branding	This instrument is a mass media campaign, inherent in raising awareness of it’s designed topic. Included in this instrument is branding, the building of a relationship between people and a product. Attributes contribute to branding to help the consumer identify with the message they are receiving.	The first, post retrospective pre-test survey implemented will ask students, “What were your intended major and career plans before this orientation?” This is an example of a pre-campaign or ghost awareness. Depending on what the minority students say will determine the pre-campaign desire for a medical profession.	Reliability or validity of the intervention’s effectiveness can be determined through pre-campaign and ghost awareness assessments. These assessments determine, through interpersonal questions or survey questions, the amount of awareness for the need of culturally representative healthcare. These questions use discretion to ask minority students their level of awareness before the campaign/ intervention (Huhman, 2008).
Changed attitudes on need for and personal conviction towards pursuing a medical career	Adaptation of Attitudes Towards Social Issues in Medicine	This instrument consists of 63 items, and 7 sub scales. Some of the sub scales of interest to this study include doctor-patient relations, social desirability, and social factors.	This instrument will test attitudes in both retrospective pre-test and both post-tests on student attitudes towards pursuing a medical career. A Likert Scale question might include “On a scale of 1-5, how convicted are you in providing culturally appropriate doctor-patient care?”	ATSIM overall reliability resulted in a high (0.82-0.90), is analyzed via a two-factor (year of program x gender) analysis of variance (ANOVA). Also to test reliability is having the mean subscale scores analyzed by multivariate analysis of variance (MANOVA). P value was found to be (p<0.00001) in study (Woloschuk, 2004).
Increased perceived benefits for individual and community in commitment to medical major and profession	Adaptation of Four Dimension Scale of Perceived Benefit (Forsythe, 2006).	The four dimensions include convenience, product selection, ease/comfort and enjoyment. This instrument focuses on the consumer’s perception of gain.	This instrument will be used to measure the increased amount of perceived benefits among target population through the two post test surveys. There will be a short-answer question asking “What do you seek to gain out of choosing a pre-med profession?” The results will increase from the first to second post test, as the intervention continues.	The reliability of the perceived benefits assessment can be found by separating the engaged minority students into two groups. The groups will be the interested, but not yet committed to pre-med 6 months after the intervention group, and a committed to pre-med after 6 months group. A MANOVA test will reveal the perceived benefits/ risks based on their identification with the goal. P value was found to be (p<0.001) (Forsythe, 2006).

Part V. Process Evaluation and Data Collection Forms

15. 1. Recruitment and enrollment
       1. Recruitment will be identified through the recording of total number of undergraduate, minority students from the target population talked to at the orientation information fair. Enrollment will be identified as the number of email addresses acquired by the recruitment team at the information fair. This number will be compared to the total recruitment number mentioned earlier. The first form below (green, Pick Pre-Med Inquiry Info) will track those that originally choose to take the second step of the intervention, to sign up for continuing emails. The email process will be evaluated using the MailChimp website as it tracks the amount of opened and engaged emails. The second form displays this. The blue form (third, “Overall Tally”) is for someone of the recruitment team to track the total number of engagements (conversations or browsing of information) of all students of target population at information fairs. The fourth form labeled, “Recruitment Team Guide Form” will be in the hands of each recruitment team member as they speak with every student and as they create every email, ensuring the fidelity of the recruitment process. 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1. Attrition
   1. Attrition will be tracked through the collection of those that choose to remain in versus opt-out/ unsubscribe from an email sequence foretold to the students. This sequence labeled Undergraduate Pre-Med Track Follow Up/ Attrition Measure will look as follows. The data of how many emails are opened and engaged will be tracked by MailChimp and will look like the second diagram.

3. Fidelity of the program 

Fidelity of the program’s implementation will be measured through Intervention Team Reference Forms that will be in the hands of each team member and referenced when talking to each student at the information fair, referenced before each email, and every academic advising sessions. The form will look as follows:

Fidelity measures for the Summer Medical and Dental Education Program will look as follows:

References

Alfred, L., Beerman, P. R., Tahir, Z., LaHousse, S. F., Russell, P., & Sadler, G. R. (2011). Increasing underrepresented scientists in cancer research: the UCSD CURE program. Journal of Cancer Education, 26(2), 223-227.

Community Health Needs Assessment 2010. (2010, July). Retrieved from https://www.stmarysathens.org/pdfs/about-us/Community_Health_Needs_Assessment.pdf  

Current Athens, Georgia Population, Demographics and stats in 2016, 2015. (n.d.). US Census Bureau. Retrieved from https://suburbanstats.org/population/georgia/how-many-people-live-in-athens

Forsythe, S., Liu, C., Shannon, D., & Gardner, L. C. (2006). Development of a scale to measure the perceived benefits and risks of online shopping. Journal Of Interactive Marketing, 2055-75. doi:10.1002/dir.20061

Geiger JH. (2001). Racial stereotyping and medicine: the need for culturalcompetence. Can Med Association Journal.164:1699–700.

Guerrero, A. D., Holmes, F. J., Inkelas, M., Perez, V. H., Verdugo, B., & Kuo, A. A. (2015). Evaluation of the Pathways for Students into Health Professions: The training of under-represented minority students to pursue maternal and child health professions. Maternal and child health journal, 19(2), 265-270.

Huhman, M., Bauman, A., & Bowles, H. R. (2008). Article: Initial Outcomes of the VERB™ Campaign. Tweens’ Awareness and Understanding of Campaign Messages. American Journal Of Preventive Medicine, 34(Supplement), S241-S248. doi:10.1016/j.amepre.2008.03.006

Jabson, J. M., Mitchell, J. W., & Doty, S. B. (2016). Associations between non-discrimination and training policies and physicians’ attitudes and knowledge about sexual and gender minority patients: a comparison of physicians from two hospitals. BMC public health, 16(1), 1.

Majumdar, B., Browne, G., Roberts, J., & Carpio, B. (2004). Effects of Cultural Sensitivity Training on Health Care Provider Attitudes and Patient Outcomes. Journal Of Nursing Scholarship, 36(2), 161-166.

Medical College Of Georgia. (n.d.). Federal Education Budget Project. Retrieved from http://febp.newamerica.net/higher-ed/GA/157900#Demographics

Medical Minority Applicant Registry (Med-MAR). (n.d.). Retrieved June 05, 2016, from https://students-residents.aamc.org/choosing-medical-career/article/medical-minority-applicant-registry-med-mar/

Salto, L. M., Riggs, M. L., De Leon, D. D., Casiano, C. A., & De Leon, M. (2014). Underrepresented minority high school and college students report STEM-pipeline sustaining gains after participating in the Loma Linda University summer health disparities research program. PloS one, 9(9), e108497.

Sambamoorthi U, Moynihan PJ, McSpiritt E, Crystal S. (2001). Use of protease inhibitors and non-nucleoside reverse transcriptase inhibitors among Medicaid beneficiaries with AIDS. American Journal of Public Health, 91:1474–81.

University of Georgia – CollegeData College Profile. (n.d.). Retrieved from http://www.collegedata.com/cs/data/college/college_pg01_tmpl.jhtml?schoolId=975

Woloschuk, W., Harasym, P. H., & Temple, W. (2004). undergraduate medical education Attitude change during medical school: a cohort study. Medical Education, 38(5), 522-534.