{"id":118,"date":"2016-05-17T16:41:43","date_gmt":"2016-05-17T16:41:43","guid":{"rendered":"http:\/\/ctlsites.uga.edu\/hprb7010may2016\/?page_id=118"},"modified":"2016-06-07T20:20:49","modified_gmt":"2016-06-08T00:20:49","slug":"health-equity-out-of-school-time-academic-programs","status":"publish","type":"page","link":"https:\/\/ctlsites.uga.edu\/hprb7010may2016\/health-equity-out-of-school-time-academic-programs\/","title":{"rendered":"Health Equity: Out-of-School Time Academic Programs"},"content":{"rendered":"

Part I. Community Guide Update and Rationale for Intervention<\/u><\/strong><\/p>\n

Research<\/strong><\/p>\n\n\n\n\n\n\n\n
Author & Year<\/strong><\/td>\nIntervention Setting, Description, and Comparison Group(s)<\/strong><\/td>\nStudy Population Description and Sample Size<\/strong><\/td>\nEffect Measure (Variables)<\/strong><\/td>\nResults including Test Statistics and Significance<\/strong><\/td>\nFollow-up Time<\/strong><\/td>\n<\/tr>\n
Cutucache,\u00a0Luhr, Nelson, Grandgenett, &\u00a0Tapprich,\u00a02016<\/td>\nThe NE STEM (The Nebraska Science, Technology, Engineering, and Mathematics 4U) program was led by faculty members and run by college students; the program involved facilitating problem based learning activities that reinforced and built upon concepts learned during the school day in the STEM disciplines for economically disadvantaged students in grades K-8.\u00a0 The program with the possibility of benefiting both the college mentors and the K-8 students took place twice each week during the academic year in the after school hours at schools in the Omaha, Nebraska area.\u00a0 Students were engaged in hands on activities to increase their critical thinking skills while building their proficiency in the STEM fields.\u00a0 STEM 4U K-8 student participant knowledge growth was assessed via pre- and post-program assessments; the control group did not participate in any STEM 4U programming.<\/td>\nThe students in grades K-8 who participated in the STEM 4U program were all eligible for free and reduced lunch; they were the underperforming, minority, and underprivileged population at each participating school.\u00a0 The schools where the STEM 4U program was implemented were those with the lowest standardized test scores in both science and mathematics.\u00a0 The initial population size for participating students was n= 297.<\/td>\nThe variables being measured were two-fold; first measured were the pre and post-test assessment scores of those students who participated in the program.\u00a0 The second variable measured was the long term retention rate of the STEM based content by the program participants and the control group population not participating in the program.<\/td>\nStudents who participated in the program at least once per week saw an increase in STEM knowledge (1.7-2.5) in comparison to their non-participating counterparts.\u00a0 Participants also illustrated they had longer retention rates of the content than their non-participating peers by scoring better on long term retention STEM assessments, 45% for participants compared to 30% for non-participants (p=0.0026).<\/span><\/td>\nAt the completion of the program, students were given a post-assessment to measure the growth of their knowledge in the STEM disciplines.\u00a0 The program has continued since the first year of implementation.<\/span><\/td>\n<\/tr>\n
Paluta, Lower, Anderson-Butcher, Gibson, & Iachini,\u00a02016<\/td>\nThe researchers studied the quality indicators that were evident (or not evident) to stakeholders associated with the 21st<\/sup> Century Community Learning Centers (CCLCs), a type of out of school time program, and then they looked at the perceived quality of the programs in relation to favorable student outcomes.\u00a0 The CCLC programs consist of character education, academic enhancement, and literacy support programs.\u00a0 For this particular study, the CCLCs from which data were collected were all in their first or third year of their funding cycles and located in a midwestern state and served elementary, middle, and high school aged youth from local schools, YMCA\u2019s, or other educational service centers.\u00a0 Programs took place after school, on weekends, and during the summers.\u00a0 Stakeholders in 405 various CCLC programs throughout the state were asked to complete surveys either online or with paper\/pencil.\u00a0 Researchers were to establish the outcomes the program stakeholders thought were most affected by their out of school programs.<\/td>\n3928 surveys were taken by stakeholders (teachers, school administrators, community partners) with 332 deemed to be incomplete, n=3388.\u00a0 Demographics of the stakeholders varied across the board in terms of gender, ethnicity, age, and level of education.\u00a0 Survey respondents also encompassed various ranges of involvement in the CCLCs.<\/td>\nThe survey distributed to stakeholders consisted of response items following the Likert scale ranging from 1-5, with 1=strongly disagree and 5=strongly agree.\u00a0 The items included questions pertaining to such topics as youth development, academic learning, family engagement, diversity, and leadership.<\/td>\nAll outcome means were above the midpoint and deemed to be reliable.\u00a0 The quality indicator deemed most favorable was overall youth development (mean 4.49) and least favorable indicator was the professional development of the staff (mean 3.79).\u00a0 Academic outcomes (mean 4.09) being not as favorable as overall youth development, left room for improvement.\u00a0 Overall, perceptions across all domains measured were positive indicating to the researchers that the programs were achieving their purpose of increasing learning opportunities and support for youths.<\/td>\nAfter initial survey distribution and collection, no follow up has been conducted.<\/td>\n<\/tr>\n
Almus & Dogan, 2016<\/span><\/td>\nThe intervention, an intensive five- week long summer school program, took place in Kansas City, Missouri in the summer of 2013 at two urban public charter schools<\/span> with a free and reduced lunch population of 94%.\u00a0 The academic subject areas being targeted included science, mathematics, and reading. \u00a0During the summer school program for each course, the curriculum was created in order to reinforce those topics students did not \u201cget\u201d during the school year. \u00a0Intertwined with the program were also opportunities for students to participate in extracurricular activities such as PE classes and incentives for program attendance, engagement, and participation; incentives included extra PE time, free breakfast and lunch, free bus transportation, free t-shirts, and credits for the next school year.\u00a0 In addition to qualitative surveys conducted by the researchers, student achievement was measured based on a change in STAR test results prior to completion of the program compared to results on the same assessment after completion of the program.<\/span><\/td>\nStudents who participated in the program attended one of two high poverty urban charter schools and were in grades K-8.\u00a0 Of the 534 students who participated in the program, 80 were new to the school and voluntarily attended for orientation purposes, 24 were mandated to participate due to low academic performance during the school year, and the remaining participants did so voluntarily because of low standardized test scores or were otherwise targeted by administration for participation.\u00a0\u00a0<\/span><\/td>\nThe variables being measured were the standardized assessment STAR\u2019s results for the group of students before and after participation in the program.\u00a0 Statistical analyses were run on the data to determine significance of the changes in standardized test scores.\u00a0 Qualitative surveys were also distributed to students for them to assess their feelings about the program and how they feel they progressed during the summer and also to determine if students in different grade levels felt differently towards the program.\u00a0 Teachers and parents also completed Likert style surveys regarding the program.<\/span><\/td>\nFor a handful of reasons, only 386 of the 534 students completed the<\/span> posttest and therefore results are based on that number of participants.\u00a0 As a result of participation in the program, it was shown that the change in pre vs. post test scores is statistically significant for reading achievement as measured by the STAR test (t=2.371, p=0.018).\u00a0 As a result of the surveys, most students believed that their standardized test scores and achievement in other subjects in the future will be better because of their participation in the summer program.\u00a0 There was no difference found amongst students in various grade levels.<\/span><\/td>\nAt the end of the summer program, participating<\/span> students were given a post test (the same standardized STAR assessment given prior to the program) to measure their academic growth during the summer program.\u00a0 The summer school program has continued.<\/span><\/td>\n<\/tr>\n
Allen, 2015<\/td>\nAn after school tutoring program was modified in hopes of increasing the academic levels of students participating in the subjects of reading, English language arts, and mathematics.\u00a0 The program was implemented in an elementary school in a southern school district and operated once a week for ninety minutes for twenty-seven weeks; the school was described as having 500 students with 99% being African American and 98% being of low socioeconomic status. The effectiveness of the modified program was measured by the comparison of student grades before and after participation in the program; the comparison group consisted of non-program participants.\u00a0 For the modified after school tutoring program, lessons were created around the weak areas as illustrated by the previous year\u2019s standardized test scores.<\/td>\nThe students targeted for participation were in grades 3 and 4 and had scored lowest on the previous year\u2019s standardized test used to measure academic achievement in reading, English language arts, and mathematics.\u00a0 For the test group, n=100, and for the control group n=57.<\/td>\nThe effectiveness of the program was based on the change in standardized test scores for those students participating in the program compared to the change in test scores of those not participating in the program.\u00a0 Statistical analyses were performed on the various groups of data.<\/td>\nIn comparing the change in test scores, program participants were found to show more growth than the control group in all three subject areas being targeted.<\/span><\/p>\n

Reading<\/span> grades: (t=-3.345; p<0.001)<\/span><\/p>\n

English language arts grades: (t=-3.43; p<0.001)<\/span><\/p>\n

Mathematics grades: (t=-7.32; p<0.001)<\/span><\/td>\n

 <\/p>\n

In depth follow up interviews have been conducted with several teachers who implemented the program.\u00a0 The focus of each interview was on the qualitative aspects of an after school tutoring program.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Updated Recommendations and Justification<\/strong><\/p>\n

After completing reviews of a meta-analysis published in 2006 and twenty-five additional studies, the Task Force concluded that out of school time academic focused programs lead to achievement gains by at risk and low performing youths in grades K-12.\u00a0 The Task Force recommends programs that focus on reading, math, and general academic content out-of- school time academic programs because of their effectiveness in decreasing the achievement gaps for at risk students.\u00a0 With the implementation of these programs and subsequent rise in academic achievement, health equity is likely to be improved because academic achievement is linked to an individual\u2019s long term health.\u00a0 The programs target groups likely to fall into cycles of poverty: racial and ethnic minorities and those with low income.\u00a0 The findings and recommendations of the Task Force were published in 2015 (Guide to Community Preventive Services).<\/p>\n

As outlined in the table above, with the review and analysis of three studies conducted since the findings of the Task Force were published, it is also my belief that there is evidence to support that academic focused out-of-school programs be recommended for implementation in schools and community centers serving the at risk student populations.\u00a0 The Task Force based their recommendations on improvements made on standardized test scores by those students participating in out-of-school time academic programs; reading scores increased by 0.31 standard deviations, math scores by 0.12 standard deviations; in general content programs, reading improved by 0.09 standard deviations and math by 0.06 standard deviations (Guide to Community Preventive Services).<\/p>\n

The study conducted by Cutucache, Luhr, Nelson, Grandgenett, & Tapprich (2016) shows similar gains in academic achievement with the implementation of the NE STEM 4U program. The out-of-school time academic program geared towards problem solving and critical thinking in the STEM disciplines for underperforming and at risk youths in low income areas of Nebraska illustrated the academic achievement gains made by the students participating in the program versus those who did not participate in the program.\u00a0 The gains were measured by comparing pre and post program assessment scores and were shown to be significant for the test group of students who participated in the STEM 4U programming; the control group showed fewer gains and less long term retention of the content knowledge.<\/p>\n

In another research study conducted by Allen (2015), academic gains were also made by at risk elementary aged students participating in an out-of-school academic focused program.\u00a0 Third and fourth grade students who performed poorly on the previous year\u2019s high stakes standardized tests, participated in a twenty-seven week long after school tutoring program in an attempt to improve their achievement on the standardized test.\u00a0 The subject areas being targeted by the tutoring program were reading, English language arts, and mathematics.\u00a0 In comparison to students not participating in the after school tutoring program, the participating students saw more substantial gains in achievement in the targeted academic areas; reading scores for participants increased by 6.1% for participants and only 2.6% for non-participants; English language arts scores by 4.2% and 1.4%, respectively; mathematics scores saw the most dramatic change for participants with an increase of 8.0% and only 1.1% for non-participants.<\/p>\n

Another study by Paluta, Lower, Anderson-Butcher, Gibson, & Iachini (2016) focused on the perceptions of the quality of out-of-school academic programs.\u00a0 The Task Force did note that the differences in the quality and type of programs available can affect the outcomes.\u00a0 The study found that overall youth development is a positively perceived outcome of out-of-school programs and thus that the out-of-school programs are doing their job in providing learning opportunities and support for the at risk youth populations where implemented.<\/p>\n

The studies show sufficient evidence to recommend the implementation of out-of-school academic programs to improve at risk student achievement.\u00a0 The long term health of an individual is directly linked to his or her academic achievement.\u00a0 It has often been found that minority and low income groups do not attain academic achievement levels as high as those of the higher income and majority groups.\u00a0 As a consequence of this, the at risk students tend to become adults with lower income and thus poorer health.\u00a0 If the programs promoting academic growth and achievement of the minority and low income groups can be implemented effectively in areas of need, then a step towards future health equity is being taken (Guide to Community Preventive Services, 2015).<\/p>\n

Part II. Theoretical Framework\/Model<\/u><\/strong><\/p>\n

The Theory & its Constructs<\/strong><\/p>\n

If implemented effectively, the intervention strategy of out-of-school time programs focused on academics for at risk youth in grades K-12 improves not only academic achievement but in turn long term individual health as outlined by the Community Guide.\u00a0 Students falling into at risk socioeconomic and minority groups must be encouraged to participate in such programs if and when they are offered to them.<\/p>\n

The Theory of Planned Behavior (TPB) states that intention to perform a behavior is the most important factor in an individual deciding to or not to engage in a particular behavior. \u00a0The TPB examines an individual\u2019s relationship between behavior and beliefs, behavior and attitudes, and behavior and intentions regarding the behavior.\u00a0 An individual\u2019s perception of what others close to him\/her think about the behavior and the individual\u2019s attitude towards carrying out the behavior shape a person\u2019s intention regarding the behavior.\u00a0 The constructs of the Theory of Planned Behavior are as follows: behavioral intention, attitude, subjective norm, and perceived behavioral control.\u00a0 In order to improve participation in out-of-school academic programs, school officials should evaluate their eligible population via the TPB as the constructs influence behavior (National Cancer Institute, 2005).<\/p>\n

The perception that an individual has of his or her chance of performing a certain behavior is defined as behavioral intention.\u00a0 This can be measured via a question or discussion with an individual regarding whether he or she will be likely or not likely to engage in a behavior.\u00a0 When planning out-of-school programs for at risk youths, it would be important for program planners to talk to potential program participants about their intent to participate in such a program as behavioral intent in this case could lead to participation or non-participation in the school program;\u00a0school and program officials can only do so much as to encourage participation.\u00a0 In this case for the intervention of out-of-school time programs, the intent to participate is being defined as the student\u2019s and\/or parent of that student\u2019s likelihood to have the student participate in the program.<\/span><\/p>\n

Attitude impacts intent and is defined as an individual\u2019s evaluation of the behavior, in this case his or her attitude towards attending an out-of-school program.\u00a0 The individual must assess his or her belief in regards to whether the behavior (attending out-of-school program) would have a positive effect, negative effect, or no effect on him\/her.\u00a0 Throughout discussions regarding the program, school officials need to ensure that the program is marketed appropriately towards the intended audience (at risk students) so that attitudes towards the program are positive and attendance is high.<\/p>\n

The beliefs that a potential program participant has about how others in his\/her life feel regarding participation in the behavior is defined as subjective norm; subjective norm also influences an individual\u2019s intent to behavior.\u00a0 So, do the people in the individual\u2019s sphere of influence support participation in the behavior?\u00a0 A student who might consider participating in an out-of-school program would be motivated by the thoughts of those who have influence over them such as their parents, teachers, and peers.\u00a0 It becomes important that these other stakeholders (parents, teachers, peers) in a student\u2019s life know the potential impact and benefit of participation in out-of-school programs so that they can positively influence the student considering participation.<\/p>\n

The final construct, perceived behavioral control, pertains to whether an individual has control in performing a particular behavior.\u00a0 In the case of attending out-of-school programs this may pertain to a student feeling that he or she has a choice in attending or not attending the program.\u00a0 Ultimately the decision to intend to or not intend to participate in the program (behavior) lies with the student, and the student needs to know that while also being guided to make the decision that would benefit him\/her the most.<\/p>\n

The Model (revised)<\/span><\/strong><\/p>\n

 <\/p>\n

\"model<\/a><\/p>\n

Part III. Logic Model, Causal and Intervention Hypotheses, and Intervention Strategies<\/strong> <\/u><\/p>\n

Target Population & Program Setting<\/strong><\/p>\n

The out-of-school time academic focused program intervention will target the underperforming at risk students in minority and low socioeconomic status groups at the targeted schools and community centers.\u00a0 The schools targeted will be those in areas with standardized test averages lower than that of a state\u2019s overall average and thus underperforming for at least two consecutive years; these may be rural or urban schools in a district.\u00a0 The community centers targeted will be those that serve students at the targeted schools located in the surrounding areas of the identified schools.<\/span>\u00a0 As previously mentioned, academic achievement is linked to long term health.\u00a0 If the academic achievement of the underperforming at risk groups of students can be improved, then the chance of long term health is increased.\u00a0 The intervention will take place during the after school hours at the participating schools and community centers up to twice per week.\u00a0 The number of teachers who will conduct the program will be dependent upon the number of student participants at each site.<\/p>\n

Intervention Methods & Strategies<\/strong><\/p>\n\n\n\n\n\n\n\n
Intervention Method<\/strong><\/td>\nAlignment with Theory<\/strong><\/td>\nIntervention Strategy<\/strong><\/td>\n<\/tr>\n
Increase awareness of out-of-school programs and their short and long term benefits (persuasive communication)<\/td>\nAligned with the Theory of Planned behavior at the perceived behavioral control construct, using persuasive communication can lead prospective program participants (students) to feel they are able to make the decision to participate (or not) in the program<\/td>\n*distribute brochures\/flyers to prospective students<\/p>\n

*teacher led class discussions during the school day referencing the purpose and benefits of the programs<\/p>\n

*targeted small group meetings with school counselors\/administrators, giving students more opportunities to realize the benefits and to ask questions in a smaller setting<\/p>\n

*parent nights in which students and parents meet in the evening hours with program leaders to learn more about the benefits of the program<\/td>\n<\/tr>\n

Offering of incentives to program participants who reach certain benchmarks while participating in the program<\/td>\nBefore committing to program participation, students will know that there is an incentive plan in place for the duration of their participation, thus positively influencing the attitude of the prospective students, increasing the likelihood of their participation (intent).<\/td>\n*signed student contracts before program participation begins outlining the incentive plan; incentives for reaching benchmarks may include the opportunity for extracurricular activity while in the program (social time, gym time, computer time, phone time etc.)<\/p>\n

*if students close achievement gap via new test scores or other program measure, participation in the program may cease if student chooses<\/td>\n<\/tr>\n

Grant access to prospective student participants and their parents to students and parents of students who have participated and benefited from various out-of-school programs in the past (modeling)<\/td>\nThe subjective norm construct may be influenced with this particular intervention.\u00a0 If prospective participants and those close to them see that participation in the program had a positive influence on past participants, then prospective students will be more likely to commit to model themselves after successful participants.\u00a0 Participants will likely be motivated by the reactions of their parents and\/or peers who also have participated in this intervention.<\/td>\n*set up one on one mentoring sessions with a past participant and a prospective participant<\/p>\n

*set up sessions between student\/parent and past participant<\/p>\n

*once program begins, allow participating student to confide in and continue to communicate with past participant<\/p>\n

*past participants write notes of encouragement to participating students on regular basis<\/td>\n<\/tr>\n

Allow students to choose academic area in which they would like to further develop their skills<\/td>\nGiving prospective students choice in their program focus will lead to perceived control and thus more positive intent to participate<\/td>\n\u00a0*when targeted for the program, allow students to choose the subject area for their studies if they write a convincing reason\/story as to why the chosen subject should be their focus<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Logic Model<\/strong><\/p>\n\n\n\n\n
Inputs\/Resources <\/strong><\/td>\nActivities <\/strong><\/td>\nOutputs<\/strong><\/td>\nShort-term Outcomes<\/strong><\/td>\nIntermediate Outcomes<\/strong><\/td>\nLong-term Outcomes<\/strong><\/td>\n<\/tr>\n
*funding to train and pay educators; 1 educator per 15 students<\/p>\n

*funding to keep the school buildings\/rooms opened after hours; 1 room per 15 students (heat\/AC, lights, janitorial staff)<\/p>\n

*program curricula; 1 per subject area offered via the program<\/td>\n

*distribute brochures\/flyers to prospective students<\/p>\n

*teacher led class discussions during the school day referencing the purpose and benefits of the programs<\/p>\n

*targeted small group meetings with school counselors\/administrators, giving students more opportunities to realize the benefits and to ask questions in a smaller setting<\/p>\n

*parent nights in which students and parents meet in the evening hours with program leaders to learn more about the benefits of the program<\/p>\n

*signed student contracts before program participation begins outlining the incentive plan; incentives for reaching benchmarks may include the opportunity for extracurricular activity while in the program (social time, gym time, computer time, phone time etc.)<\/p>\n

*if student closes achievement gap via new test scores or other program measure, participation in the program may cease if student chooses<\/p>\n

*set up one on one mentoring sessions with a past participant and a prospective participant<\/p>\n

*set up sessions between student\/parent and past participant<\/p>\n

*once program begins, allow participating student to confide in and continue to communicate with past participant<\/p>\n

*past participants write notes of encouragement to participating students on regular basis<\/p>\n

*when targeted for the program, allow students to choose the subject area for their studies if they write a convincing reason\/story as to why the chosen subject should be their focus
\n<\/em><\/td>\n

*ALL students in the school receive flyer outlining the purpose and design of the program<\/p>\n

*school chooses one day to have all core subject educators to present the information to all of their classes at all grade levels<\/p>\n

*teachers recommend (based on test scores) students to participate in the small group meetings with counselors to further discuss details of the program<\/p>\n

*two parent meetings on different nights during the same week to introduce program to parents (parents attend one)<\/p>\n

*design and distribute program contract with incentives outlined to all students who have attended small group or parent meeting or who have otherwise shown interest<\/p>\n

*initial meeting (make it fun) between past and prospective participants, keep ratio low, no more than two participants per one past participant<\/p>\n

*<\/em>on a monthly basis, past participant will write notes of encouragement to current participants
\n<\/em><\/td>\n

*student and parent awareness of the program will increase, therefore the realization of the benefits of the program will increase<\/span> (attitude & subjective norm)<\/p>\n

*students will decide to participate in and commit to the program (intent)<\/p>\n

*students will actually attend the weekly program sessions (perceived control)<\/td>\n

*student standardized test scores will increase<\/p>\n

*student classroom grades will increase<\/p>\n

*<\/em>more students will begin to realize the value of and enjoy their time in school
\n<\/em><\/td>\n

*the achievement gap between groups of students will begin to close<\/p>\n

*the long-term health of the minority groups of students participating will be better than it would have been without participation in the program
\n<\/em><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Intervention Hypotheses<\/strong><\/p>\n

*By distributing brochures\/flyers to prospective students and parents, the awareness of the benefits of the program will increase.<\/p>\n

*If teachers lead class discussions during the school day referencing the purpose and benefits of the programs, student awareness of the benefits of the program will increase along with number of students deciding to participate in the program.<\/p>\n

*In targeted small group meetings with school counselors\/administrators, the intent of the student to participate in the program will increase, thus increasing actual participation in the program.<\/p>\n

*At parent nights in which students and parents meet in the evening hours with program leaders, the realization of the benefits of the program will increase amongst both parents and students potentially increasing participation.<\/p>\n

*If signed student contracts are secured before program participation begins outlining the incentive plan, then student participation in the weekly sessions will increase.<\/p>\n

*If a student closes achievement gap via new test scores or other program measure and student has the choice to end participation in the program, then initial commitment along with student attendance in the program will increase.<\/p>\n

*By allowing communication, interaction, and relationship building between prospective\/current student participants with past participants, increases in program participation and attendance will be realized.<\/p>\n

*When students are able to choose the subject area for their studies during the program, an increase in student attendance will be observed.<\/p>\n

Causal Hypotheses<\/strong><\/p>\n

*An increase in student and parent awareness of the benefits of the program will lead to increases in the enjoyment of school by program participants.<\/span><\/p>\n

*An increase in participation and commitment to the program by students will lead to increases in standardized test scores and classroom grades thus closing the achievement gap between minority groups of students and their higher achieving peers.<\/p>\n

*By increasing student attendance at the weekly program sessions and the closing of the achievement gap, an increase in long-term health of the minority groups of students participating in the program will be realized.<\/p>\n

 <\/p>\n

SMART Outcome Objectives<\/strong><\/p>\n

Goal 1:<\/strong> Increase awareness of the benefits of the out-of-school time program.\u00a0 Objective 1: <\/strong>Within two weeks of<\/span>\u00a0brochure distribution, classroom sessions, and parent meetings, awareness of the program and its benefits will increase at least 30%.\u00a0 Objective 2:<\/strong> Awareness of the program will increase by at least 15% in each subsequent year of implementation.<\/p>\n

Goal 2:<\/strong> Student commitment to the program will increase.\u00a0 Objective 1: <\/strong>After awareness of the program benefits has increased, initial student commitment to the program will increase at least 30% within one month<\/span>.\u00a0 Objective 2:<\/strong> Before the program begins because of increased awareness of the program, the commitment rate to the program by prospective students will increase 20% each year.<\/p>\n

Goal 3:<\/strong> Attendance at the weekly program sessions will increase.\u00a0 Objective 1:<\/strong> At least 85% of the students committing to the program will actually show up for at least 90% of the program sessions.\u00a0 Objective 2:<\/strong> Actual attendance of students who committed to participation in the program will increase 20% with each implementation of the program.<\/p>\n

Goal 4:<\/strong> Standardized test scores will increase.\u00a0 Objective 1:<\/strong> Pre and post-test scores of program participants will increase at least 15%.\u00a0 Objective 2:<\/strong> Within two months, p<\/span><\/span>articipation in standardized assessments will increase 15% as a result of participation in the program (ex: fewer students will be absent on test days).<\/p>\n

Goal 5:<\/strong> During the regular school day\/year, classroom grades of student program participants will increase.\u00a0 Objective 1:<\/strong> By the end of the grading period (or semester), t<\/span><\/span>he classroom grade in the content area in which each student participated in during the program will increase by at least 10% (a letter grade).\u00a0 Objective 2:<\/strong> Because engagement and grades of the student will increase, within six weeks,<\/span> teacher reported discipline problems will decrease by at least 20%.<\/p>\n

Goal 6:<\/strong> Student realization of the value of school and the enjoyment of school will increase.\u00a0 Objective 1:<\/strong> In six weeks time,<\/span> positive student perceptions of school will increase by at least 20% because of participation in the program.<\/p>\n

Goal 7:<\/strong> The achievement gap between minority groups of students and their higher performing peers will begin to close.\u00a0 Objective 1:<\/strong> As measured by classroom grades and standardized test scores, the achievement gap will begin to close by 1-2% each school year when the out-of-school program is being implemented.<\/p>\n

Goal 8:<\/strong> The long-term health of the groups of students participating in the out-of-school time program will be better than it would have been without participation in the program.\u00a0 Objective 1:<\/strong> In comparing minority groups of students each year out of the program compared to minority students who did not participate in the out-of-school time program, better health will be self-reported by the students who participated in the program at each year increment.\u00a0 Objective 2:<\/strong> Over the long-term with subsequent implementations of the program and increased minority student participation, gains in long-term health of the minority groups will be realized.<\/p>\n

Part IV. Evaluation Design and Measures<\/u><\/strong><\/p>\n

Stakeholders<\/strong><\/p>\n\n\n\n\n\n\n\n
Stakeholder<\/strong><\/td>\nRole in Intervention<\/strong><\/td>\nEvaluation Questions from Stakeholder<\/strong><\/td>\nEffect on Stakeholder of a Successful Program<\/strong><\/td>\nEffect on Stakeholder of an Unsuccessful Program<\/strong><\/td>\n<\/tr>\n
\u00a0Students<\/td>\nParticipants in the program<\/td>\nIf I participate in the program, how will I benefit?\u00a0 What incentives are there for me to participate in the program?<\/td>\nClassroom grades will improve; standardized test scores will improve; enjoyment of school will increase; better long-term health outcomes<\/td>\nGrades and test scores will remain stagnant or even decrease; attendance may decrease, drop out may occur; poor long-term health outcomes likely<\/td>\n<\/tr>\n
\u00a0Parents<\/td>\nSupport students throughout duration of program<\/td>\nHow do I know this is the right thing for my student to be doing?\u00a0 How will it help him\/her and me?\u00a0 How will I know the program is working?<\/td>\nChild will become a better, more engaged student, grades and test scores will increase; behavior may improve, long-term health outcomes of child will be better<\/td>\nGrades and test scores of child in program will not improve and may even decrease; no improvement of long-term health outcomes<\/td>\n<\/tr>\n
Educators\/Peer Mentors<\/td>\nCreate and deliver curricula to program participants; monitor academic achievement and growth; accurate record keeping; provide motivation and encouragement to participants<\/td>\nIs it an expectation that I contribute to this program or is there compensation associated with the extra effort?\u00a0 How will I know what I am doing is working and that the program is successful? How much time am I expected to devote to the program on a weekly basis?<\/td>\nEducator\/mentor will want to continue working for the program from year to year; will encourage others to participate; positive relationships established<\/td>\nEducator\/mentor will not continue with the program from year to year; will dissuade others from participating by sharing their experiences; meaningful relationships won\u2019t be established and carried forward; wasted time to develop curricula for program<\/td>\n<\/tr>\n
School\u00a0Administrators & Program Directors<\/td>\nOversee development and implementation of entire program<\/td>\nWhat are the expectations of the program?\u00a0 How much will it cost to implement the program; where will the funds come from?\u00a0 How will we know if the program is meeting its goals?<\/td>\nWill continue working with the program from year to year; will want to work to make program improvements each year<\/td>\nWill not want to continue working with the program from year to year; waste of time and resources in implementation; decreased support for future implementation of program<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Evaluation Design<\/strong><\/p>\n

To evaluate the effectiveness of the out-of-school time academic program intervention strategy, a randomized controlled trial design will be used.\u00a0 After similar schools are recruited and then elect to participate in the study, they will be randomly assigned to either receive the intervention or to continue moving forward with whatever teacher\/individual classroom tutoring programs already in place (no intervention program).\u00a0 Because the study will take place over the full school year, surveys and pre-assessments will be completed prior to the program beginning by participating students in both the intervention and control groups; assessments will then be given in bi-monthly intervals throughout the school year to program participants to measure academic growth.<\/p>\n

R O1\u00a0\u00a0 \u00a0 X\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 O2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 O3\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 O4\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 O5\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 (intervention)<\/p>\n

R O1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 O2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 O3\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 O4\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 O5\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 (no intervention)<\/p>\n

Standardized assessments will also be given to all students via the state mandated testing schedule; data from these exams may also be employed to gauge the effectiveness of this intervention.\u00a0 Surveys will be given to participating students prior to the beginning of the intervention\/program and at the conclusion of the program\/school year; surveys will be used to see where students stand on various aspects of the intervention\/program.<\/p>\n

Threats to Validity<\/strong><\/p>\n

By using a randomized evaluation design, most threats to internal validity are eliminated and internal validity should be maximized.\u00a0 With that being said, some threats may still exist.\u00a0 Bias in the instrumentation being used, specifically the pre and post program surveys, will exist if the instrument was not tested first with focus groups during the design of the intervention prior to its implementation.\u00a0 Even with the randomization of the study, attrition bias may exist; students may drop out of their school\u2019s program, whether a part of the intervention group or not, students may drop out of school, students may move schools, etc.\u00a0 Because the study will not be running for longer than one school year, some attrition bias may be eliminated.\u00a0 Maturation may also be a threat to this study; it might have been found that academic growth would have occurred with the targeted groups of students even without the intervention.\u00a0 By having a control group in this study, maturation bias will be eliminated if the intervention group shows more academic growth than the control group.<\/p>\n

Variables to be Measured and How<\/strong><\/p>\n\n\n\n\n\n
Short-term or Intermediate Outcome Variable<\/strong><\/td>\nScale, Questionnaire, Assessment Method<\/strong><\/td>\nBrief Description of Instrument<\/strong><\/td>\nExample item (for surveys, scales, or questionnaires)<\/strong><\/td>\nReliability and\/or Validity Description<\/strong><\/td>\n<\/tr>\n
Awareness of the program (if awareness of the program and its benefits are high\/positive, then attitudes of prospective students towards the program will increase along with their evaluation of subjective norm)<\/span><\/td>\nYoung and Rubicam\u2019s Brand Asset Valuator; the instrument will be revised to reflect relevance, esteem, knowledge, and differentiation of the out-of-school time program (Zaichkowsky, Parlee, & Hill,\u00a02008)<\/td>\nThe measure is designed to measure four dimensions of brand awareness: relevance, esteem, knowledge, and differentiation. It is administered in the form of a Likert style questionnaire, 1=strongly disagree, 5=strongly agree.<\/td>\nQuestions would be designed to compare the four parameters being measured between the out-of-school academic programs and other after school and\/or summer activities taking place in a school setting (not necessarily academic).<\/p>\n

*The out-of-school time academic program is the best option to help me (my student)<\/span> meet my (his\/her)<\/span> academic potential.<\/p>\n

*The out-of-school academic program will meet my (my student’s)<\/span> needs.<\/p>\n

*I have other options to help me meet my academic goals. (My student has other options to help him\/her meet his\/her academic goals.)<\/span><\/td>\n

Reliability (Cronbach\u2019s alpha):*relevance 0.68<\/p>\n

*esteem 0.72<\/p>\n

*knowledge 0.30<\/p>\n

*differentiation 0.44<\/td>\n<\/tr>\n

Commitment to the program (intent)<\/td>\nTCM Employee Commitment Survey- revised to reflect student commitment; printed\/administered in both English and Spanish to better reach target participants(Meyer & Allen, 2004)<\/td>\nThe survey addresses three types of commitment: normative, affective, and continuance.\u00a0 Survey items are answered using a 7 point Likert Scale with 1 equal to strongly disagree and 7 equal to strongly agree.<\/td>\nNormative Commitment: I was taught to believe in the value of remaining loyal to one\u2019s organization.Affective Commitment: I enjoy discussing my organization with people outside of it.<\/p>\n

Continuance Commitment: One of the major reasons I continue to work for this organization is that leaving would require considerable personal sacrifice- another organization may not match the overall benefits I have here.<\/p>\n

*will be revised as follows organization: out-of-school time program (intervention)work: attend, participate in<\/td>\n

Reliability (Cronbach\u2019s alpha) 0.858 in a pilot study & 0.912 in the main studyThe face and content validity of the scale was confirmed by subject area experts.<\/p>\n

(Sarwar & Ashrafi, 2014)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Part V. Process Evaluation and Data Collection Forms<\/u><\/strong><\/p>\n

Recruitment and Enrollment<\/strong><\/p>\n

To begin the recruitment process, school districts will be contacted first for approval for program planners to contact schools within the district in regards to the proposed intervention.\u00a0 Once a district approves of the intervention, then the program planners will work with the district officials to identify schools willing to participate.\u00a0 Program planners will draft and send letters to school principals\/counselors and then follow up via phone call within two weeks of mailing the letter.\u00a0 The following form will be used to track schools contacted and their response:<\/p>\n

\"Out-of-School<\/a><\/p>\n

Once schools have elected to participate, participants will then be identified.\u00a0 With information from the individual schools, the program planners will write and distribute letters to prospective students and parents.\u00a0 Letters with a parent consent form to be returned will first be distributed at school to prospective students outlining the benefits, parameters, and details regarding the program.\u00a0 If consent forms are not received within one week of distribution, program planners will follow-up with parents via phone call.\u00a0 The following form will be used to track students contacted for participation in the program:<\/p>\n

\"Out-of-School<\/a><\/p>\n

Attrition<\/strong><\/p>\n

Detailed contact information for each student and parent will be collected via the consent form during the recruitment and enrollment period.\u00a0 To ensure that each student\/parent pair has received all relevant information before consenting to participate in the program, the following form will be used to track receipt of information, attendance at meetings, etc.<\/p>\n

\"prospective<\/a><\/p>\n

Once a commitment has been made to the program and the program is implemented, the following form will be used to track student participation in scheduled program observations.<\/p>\n

\"participant<\/a><\/p>\n

Fidelity<\/strong><\/p>\n

To capture the fidelity of the program, the following form will be distributed at each participating school site; the administrator overseeing the program at each location will be asked to complete the form.\u00a0 Forms from all locations will then be compared to get an idea of what worked, what did not, and what might need to be changed for future implementations.<\/p>\n

\"program<\/a><\/p>\n

Classroom teachers conducting initial information sessions during class time, \u00a0the counselors conducting small group meetings, the individuals running the evening parent meetings, and past participants (mentors) will each be asked to complete minutes\/a survey regarding their part in the process.\u00a0 This data will then be compiled at the school level on the following form:<\/span><\/p>\n

\"fidelity<\/a><\/p>\n

References<\/strong><\/p>\n

Allen, B. (2015). After school tutoring increases academic performance.\u00a0 Journal of Teaching, Learning, and Scholarship<\/em>, 2(3), article 1.<\/p>\n

Almus, K., & Dogan, B. (2016). A study of summer school enrichment program in high-poverty urban public charter school.\u00a0 Reading Improvement<\/em>, 53(1), 1-16.<\/p>\n

Cutucache, C., Luhr, J., Nelson, K., Grandgenett, N., & Tapprich, W. (2016).\u00a0 NE STEM 4U: an out-of-school time academic program to improve achievement of socioeconomically disadvantaged youth in STEM areas.\u00a0 International Journal of STEM Education<\/em>, 3(1), 1-7.<\/p>\n

Guide to Community Preventive Services.\u00a0 Promoting health equity through education programs and policies: out-of-school time academic programs.\u00a0 \u00a0http:\/\/www.thecommunityguide.org\/healthequity\/education\/outofschooltime.html. Last updated: 05\/18\/2016.<\/p>\n

Meyer, J., & Allen, N. (2004).\u00a0 TCM Employee Commitment Survey: Academic Users Guide 2004.\u00a0 Ontario, Canada: University of Western Ontario.<\/p>\n

National Cancer Institute. (2005).\u00a0 Theory at a glance: a guide for health promotion and practice<\/em> (NIH Pub. No. 05-3896).\u00a0 Washington, DC: National Institutes of Health.<\/p>\n

Paluta, L., Lower, L., Anderson-Butcher, D., Gibson, A., & Iachini, A.\u00a0 (2016).\u00a0 Examining the quality of 21st<\/sup> century community learning center after-school programs: current practices and their relationship to outcomes.\u00a0 Children and Schools<\/em>, 38(1), 49-56.<\/p>\n

Sarwar, M., & Ashrafi, G. (2014).\u00a0 Students\u2019 commitment, engagement and locus of control as predictor of academic achievement at higher education level.\u00a0 Current Issues in Education<\/em>, 17(3).\u00a0 Retrieved from http:\/\/cie.asu.edu\/ojs\/index.php\/cieatasu\/article\/view\/1314.<\/p>\n

Zaichkowsky, J., Parlee, M., & Hill, J. (2010).\u00a0 Managing industrial brand equity: developing tangible benefits for intangible assets.\u00a0 Industrial Marketing Management<\/em>, 39(5), 776-783.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Part I. Community Guide Update and Rationale for Intervention Research Author & Year Intervention Setting, Description, and Comparison Group(s) Study Population Description and Sample Size Effect Measure (Variables) Results including Test Statistics and Significance Follow-up Time Cutucache,\u00a0Luhr, Nelson, Grandgenett, &\u00a0Tapprich,\u00a02016 The NE STEM (The Nebraska Science, Technology, Engineering, and Mathematics 4U) program was led by … Continue reading Health Equity: Out-of-School Time Academic Programs<\/span><\/a><\/p>\n","protected":false},"author":834,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"open","template":"","meta":{"_acf_changed":false,"footnotes":""},"class_list":["post-118","page","type-page","status-publish","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/ctlsites.uga.edu\/hprb7010may2016\/wp-json\/wp\/v2\/pages\/118","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ctlsites.uga.edu\/hprb7010may2016\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/ctlsites.uga.edu\/hprb7010may2016\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/ctlsites.uga.edu\/hprb7010may2016\/wp-json\/wp\/v2\/users\/834"}],"replies":[{"embeddable":true,"href":"https:\/\/ctlsites.uga.edu\/hprb7010may2016\/wp-json\/wp\/v2\/comments?post=118"}],"version-history":[{"count":0,"href":"https:\/\/ctlsites.uga.edu\/hprb7010may2016\/wp-json\/wp\/v2\/pages\/118\/revisions"}],"wp:attachment":[{"href":"https:\/\/ctlsites.uga.edu\/hprb7010may2016\/wp-json\/wp\/v2\/media?parent=118"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}