8b. The results are in! Part 2

Foundation Study Framework students are currently in the midst of completing their end of year assessments, and we wish them the best of luck with these!

Their modules finished just before Easter, and before they went off on exam leave, students were asked to complete a second COGS survey (identical to the one they completed in term one) to assess whether their level of self-efficacy had changed throughout the programme (see blog 2 and 3 for details on the COGS survey).

Whilst we were unable to capture results from the full student cohort, both COGS surveys were completed in full by 59 students, allowing us to compare a mean self-efficacy score at the start and end of their first year on the Foundation Study Framework. Whilst this may seem like a short time period, self-efficacy can be context-specific and therefore change fairly quickly (Ouweneel et al., 2013).

We were mainly interested to know if their mean self-efficacy score hand changed over time, and whether there was a difference between those that had attended the GRIT programme (see blog 5 for information on GRIT).

Statistically there was no difference in mean self-efficacy scores over time (term one and term two), and regardless of whether they attended the GRIT programme or not. The results of a Two-way mixed ANOVA showed that there was no difference in mean self-efficacy scores over time (F(1, 34)= .16, p= .69, ƞp2= <.005), and there was no significant interaction between time and attending the GRIT programme (F (1, 34) = .29, p= .58, ƞp2= <.009).

Total mean scores in term one were 3.18 (moderately true), and in term two were 3.2 (moderately true), thus indicating that students that engaged with both surveys generally had fairly good levels of self-efficacy to begin with, therefore it could be that any further development has little effect on those scoring 3 or 4 on the scale.

As there was also no difference found between those who attended the GRIT programme, it could be suggested that the GRIT programme is not of any benefit for self-efficacy in isolation. However participants reported a number of positive feelings regarding the GRIT programme that suggest other benefits aside from self-efficacy can be gained. Benefits surrounding self-esteem, relationships with peers, and acknowledging responsibility for their own future (see blog 6 and 7 for full details) formed part of the feedback from those that engaged with the GRIT programme.

It may also suggest that students that fully engaged with the research were already experiencing more positive levels of self-efficacy, and that those students that opted out are actually the ones who would benefit more from the GRIT programme. Tahmassian and Moghadam (2011) highlighted the link between self-efficacy and motivation, social engagement, and depression. Whilst there are very likely a host of other legitimate reasons for students not engaging with the research and the GRIT programme, it would be interesting to learn if those that didn’t engage had lower self-efficacy scores. In light of this, the mean scores of students who only completed the term one survey were compared with the scores of students who only completed the term two survey (therefore only partially engaging with the research). As term two survey completers were made up of students who were late enrollers, and generally had experienced difficulty engaging with their course to begin with, it was hypothesised that a lower self-efficacy score may be more prevalent in these students. Whilst an independent t-test showed no difference (t=1.61, df= 53, p=0.06) this was marginal and perhaps with a larger sample size a difference in mean self-efficacy scores may be discovered.

In terms of the methodology used, the four point LIKERT scale design within the COGS survey may be limited in detecting more subtle changes in self-efficacy. Research by Johns (2010) concluded that results from LIKERT scales (and those with similar rating scales) becomes ‘significantly less accurate when the number of scale points drops below five or above seven’. Also other studies have reported higher reliability in five point LIKERT scales (Bouranta et al. 2009) and seven point LIKERT scales (Lewis, 1993; Finstad, 2010).

Another methodological difficulty is the reliance on participants’ self-reporting of their self-efficacy. Whilst this allows for straightforward and wide scale data collection across the student cohort, it assumes that participants will a) report honestly and not succumb to Social Desirability Bias and / or Acquiescence Bias, and / or, b) be self-reflective enough to fully understand their levels of self-efficacy. Evidence suggests that the ability to self-evaluate (along with other higher level cognitive functions such as problem solving and not giving into peer pressure) develops in the pre-frontal cortex of the brain during our mid-twenties (Arain et al. 2013). The FSF cohort had 89% of its students aged under 24, therefore it could be suggested that self-scoring of self-efficacy is not going to yield accurate results in students under the age of 25. Also, things that hinder this development within the brain include negative social influences, social isolation, substance abuse, and high levels of stress. These kind of issues may have affected a number of students on the FSF, as the course is designed specifically for those who may not have had an easy time in life.  The good news is that the brain is still flexible after this critical period, therefore individuals experiencing low levels of self-efficacy and other negative thought patterns can still do things to improve their way of thinking.

Going forward, it would now be interesting to see how self-efficacy might affect performance, therefore the plan is to compare final year grades with self-efficacy scores.

 

 

8a. The results are in! Part 1

Foundation Study Framework students are currently in the midst of completing their end of year assessments, and we wish them the best of luck with these!

Following on from a mostly positive review of the GRIT programme from those that attended (see blogs 6 and 7 for details on this), we were interested to know how well the GRIT programme’s own survey compared with the UoN’s own COGS survey that student’s also completed in term one and term two (see blogs 2 and 3 for more details on this). Both these surveys have been based on previously published self-efficacy scales and have previously both shown good internal reliability.

The GRIT survey consisted of 10 statements whereby students self-scored on a four-point LIKERT Scale, and a further 17 key skills items whereby students self-scored on how good they felt they were (see Appendix 1 below for full details).

All scales showed good internal reliability using the current data when tested using Cronbach’s alpha (COGS Survey= 20 items; α= .846; GRIT LIKERT Scale= 10 items; α=.829, and GRIT Rating Scale= 17 items; α= .883). The GRIT LIKERT Scale contained a mixture of positive and negative statements. Whilst this can reduce the chances of acquiescence bias (a tendency to agree with statements, especially when in doubt) from the respondent (Dodd-McCue and Tartaglia, 2010), it can cause problems for statistical analysis, particularly when measuring internal reliability (Salazar, 2015). Negative statements data was therefore reverse coded for analysis.

Interestingly, whilst there were a few weak to moderate correlations between individual questions, overall there was no significant correlation between the mean scores of the GRIT LIKERT scale and the COGS LIKERT scale (r (59)= .045, p= .728).

As self-reporting scales are particularly vulnerable to acquiescence bias (Dodd-McCue and Tartaglia, 2010), it could be recommended that the COGS survey incorporates a mixture of positive and negative statements rather than just positive to avoid potential this potential bias.  

However there was a moderate positive correlation between the GRIT Rating Scale and the COGS LIKERT Scale (r(59)= .414, p= .001). This could be because the statements within both surveys are more similar in their design. However there was a also a moderate positive correlation between the GRIT Rating Scale and the GRIT LIKERT Scale (r(59)= .480, p= .001) which contradicts the idea that surveys benefit from being of similar design. A final argument could be that the COGS survey and the GRIT surveys are based on two different previously published self-efficacy surveys, which could explain some of the discrepancies when comparing the data.

On a final note, no correlations were found between any of the three self-efficacy scores and the highest level of qualification students held. There was also no association with highest level of qualification held and whether  students engaged with the GRIT programme. However the majority (n= 42 (69%)) of students held level 3 qualifications, with only 4 (7%) students holding Entry Level qualifications, 4 (7%) students holding level 4 qualifications, and 11 (17%) holding level 2 qualifications. It could be argued that there is not enough data in the latter categories to identify any patterns.  

Appendix 1: GRIT Self-efficacy baseline survey

Four Point LIKERT Scale (Strongly agree, Agree, Disagree, Strongly Disagree) Rating scale (1-10, with 1 being Poor and 10 being Excellent)
On the whole I am satisfied with myself Self-esteem
I often think I am no good at all Self-confidence
I feel that I have a number of good qualities Speaking in front of other people
I am able to do things as well as most other people Setting and reviewing goals for the future
I feel I do not have much to be proud of Developing positive relationships with new people
I feel useless sometimes Ability to work with others
I feel that I’m a person of worth, at least on an equal plane with others Supporting others
I wish I could have more respect for myself Accepting others
I am inclined to feel that I am a failure Developing relationships with adults
I take a positive attitude toward myself Being on time
 

Outside of the LIKERT scale students were also asked ‘What’s the highest qualification you have got?‘ and given a choice ranging from Entry Level to Level 4, and ‘other- please specify’

Handling mistakes
Taking responsibility for your actions
Commitment
Keeping promises
Ability to solve problems
Seeing possibility
Dealing with challenging situations

 

 

 

 

7. These students really have got some GRIT!

A reflection on how the GRIT programme has helped students studying on the Foundation Study Framework at the University of Northampton

On Thursday 22nd February 2018, students that had taken part in the training in November 2017 hosted by the GRIT organisation (for details about this organisation see Blog 5: GRIT! and visit https://grit.org.uk/) attended a follow up day to talk about how the training may have helped them, and to celebrate their successes.

The GRIT staff were thrilled to see familiar faces return for the follow up day- it was clear that they felt privileged and proud to see a transformation in the students, and as the session discussions began, there was evidently some students who perhaps would not still be engaged in their degree programme if it wasn’t for the GRIT training.

Students’ reflections on the GRIT programme echoed similar feedback from the initial group interviews (see Blog 6: an update on the research so far…).

“The first day was odd…”

“I kept my guard up at first, but the activities slowly allowed me to let this down.”

“I wasn’t impressed after day one, but I came back out of curiosity. The second day was very intense, and it revealed a bit about me- day two broke me! (On) day three we learnt about how we might interpret things differently. I have learnt more about myself.”

“I didn’t have a clue what the first day was about, but I was curious about the rest. It’s nice that people cared about your problems as well as their own.”

It is clear that the GRIT programme perhaps doesn’t make sense at first, but if you stick with it and trust their approach, there are many gains to be had. Students reflected on how it had helped them, and what they had taken forward into both their academic and personal life.

Many reported that they had benefitted from an activity called ‘Already always listening’ that challenged the way they react when someone says something that upsets them, or if they perceive someone to have something in a certain way that in reality is very different.

“(It) allowed me to reflect on why I think and react this way. Has there been a moment in my life that has caused that?”

“(It) helped me acknowledge the conversations I had in my head, and how these are an assumption, and not always the truth.”

“I wouldn’t have stood in front of the room and spoken to the group at the start, but today I am doing it! I can now see past assumptions e.g. thinking someone has it all when maybe they don’t- it’s just your assumption.”

Some reported just how much the programme had helped them stick with their studies by developing their self-discipline skills and building meaningful relationships with their peers. We assume that it is easy to make friends, but as you get older this gets harder. It shouldn’t be taken for granted when we build relationships in adult life, and this is so important for university being a positive experience.

“I have made friends from the three days (GRIT programme), and improved my ability to sit and stay focused- its helped me with my lectures.”

“(It) made me realise if you want to achieve something, it’s down to you. (I) need to stop living in the past and take responsibility for my future. You can change your future. I am booking holidays and making friends- making a future for myself.”

“I’m not a very patient person- the first day was confusing, but it taught me to be patient with the process. (Its) not just about making friends, but being open to the connection of making friends.”

 

Creating a future for yourself!

Students were next challenged to think about where they want to be in the next 5 years, but to talk about it in the present tense, like it was real and happening now. They also had to think about all the possibilities, and what things could get in the way of this goal. “Dreams are free, so why be cheap on your dreams?!”

Just talking about their goals ignites the process of planning them- the power of speaking is very influential in making things a reality. Students were encouraged to reflect on their past experiences, but to only ‘glance’ at these, as looking back too much won’t allow you to move forward.

Students then shared their dreams with the whole group, with big smiles on their faces as they dared to think about what kind of positive future they could have. Whilst there were a variety of different dreams, they all had similar aspirations interwoven. Being successful and having families were reoccurring themes, and this allowed students to realise that we may come from different situations, but really the things that are important to us are the same.

“The future is always in my head, but sharing it with someone else is exciting!”

 

Making a stand!

“I am the greatest!” (Muhammed Ali, 1964)

The final activity of the day was to encourage students to ‘make a stand’ for who they are and what they believe in. The GRIT team emphasised the importance of this- “If you don’t stand for something, you will fall for anything, or people that don’t have your best interests at heart will make a stand for you.” A stand does not have to contain any evidence, or have other people agree with or believe it. But it is important that you believe it, and it should incorporate how you are going to make a positive future for yourself.  Even if you get lost along the way, you can always come back to your stand.

Students had an hour or so to come up with their stand by completing the following sentence:

“I declare the possibility that who I am is the stand I take. The stand I take is…”

Each student then made their stand in front the group. Some students embraced it, whilst others clearly found it tough, perhaps because they were exposing their vulnerabilities. But it’s important to make a stand and really believe in it, so everyone had a huge round of applause for their courage.

The stands were varied, but many had common themes, as shown in the word cloud below. The larger the words, the more they occurred amongst the stands. Being a strong, kind, and caring person was important to a lot of the students, and this can only be a positive thing.

 

The day ended with students receiving certificates to celebrate their success on completing the programme. Students clearly felt they had gained a lot from the programme, and the skills they have developed will not only help see them through their degree programme, but will enhance their employability on completion. The financial cost of incorporating the GRIT programme into the academic year is also very efficient- only two students need to be retained to make it cost-effective. With the Foundation Study Framework aimed at students with non-traditional entry qualifications (see Blog 1: Introducing the Foundation Study Framework… for more details), drop-out rates are at risk of being higher compared with students with more traditional backgrounds. Retaining students on this programme will not only helps individual people succeed in life, but may help widen participation in degree programmes on a larger scale.

6. An update on the research progress so far…

Engaging students could come down to how much they believe in themselves

The concept of self-efficacy, or how much you genuinely believe in your own abilities to succeed, could be the making or breaking of you when embarking on any kind of endeavour.

Scientists have long studied the idea that any accomplishment is driven by your self-belief. This includes how well you respond to challenges and obstacles, and the choices you make along the way. 

So how does self-efficacy impact on undergraduate students? Completing a degree is no mean feat, often full of hindrances, setbacks, trials and tribulations. Its these challenges that contribute to making it a degree, and upon graduation sets you apart from those that haven’t been on this turbulent journey.

Exciting new research at the University of Northampton, led by The Foundation Study Framework (FSF) team is looking to understand how we can better engage students who are enrolled on a four-year FSF programme.

The FSF is an exciting novel route into degree-level study that aims to engage students without standard entry qualifications. The first year of their chosen degree is studied over two years, incorporating both discipline-related modules and foundation framework modules, with an aim to support students in developing higher level academic and transferable skills that will ensure they succeed into their chosen subject area (https://www.northampton.ac.uk/study/foundation-framework/). This contributes to the government’s  agenda to double the numbers of young people going to university by 2020.

Starting a degree course from a non-traditional background presents new challenges for staff and students alike. Skills that would usually be gained during A-level study ready for Uni may not have been developed in students undertaking the FSF, and this could be where self-efficacy, determination and ‘grit’ play an even bigger role in student success. This gives us a fantastic opportunity to carry out research into how important self-efficacy is for students on the FSF!

The Research Project

Over 250 students enrolled on 20 different degree courses as part of the FSF in the academic year 2017-18. To measure and monitor self-efficacy over the academic year, students will take part in The GRIT Breakthrough Programme and complete two self-efficacy scale surveys, one produced by GRIT, and one produced by the University of Northampton’s Changemaker Employability Self-assessment tool, which combines questions from the General Self-Efficacy Scale (Schwarzer, 1995) with the University of Northampton’s Changemaker Outcomes for Graduate Success (COGS). 

 In November 2017 students took part in The GRIT Breakthrough Programme–  a three-day  course led by the GRIT organisation (www.grit.org.uk) that believes that ‘when you change your thinking, you can change the world‘.  GRIT aims to reach young people through transformational training, leading the idea that changing the way they think can radically alter their outcomes in life. Much of this training inspires students to believe in themselves and that they are capable of success.

Attending a three-day GRIT programme  is no easy task though! Participants are challenged to look at the reasons why they might have low levels of self-belief. Workshops are geared towards participants facing some uncomfortable thoughts and feelings in order to get to the root cause.

In January 2018 a small focus group was held with students to reflect on The GRIT Breakthrough programme, and how it had affected them. Students emphasised how on the first day, they had no clue as to what was going on, and that many of the activities didn’t make sense! However they stuck with it, and after attending the second and third day, things started to make sense and students experienced almost epiphany-type states in relation to their self-belief.

“…when I came back after the 2nd or 3rd day I was like… you know the feeling you get after you have meditated? I felt very…like I know myself a bit more.”

“There was a lot to it, some of it did not make any sense at all to begin with, and then it became logical after day 3. I enjoyed it tremendously!”

Older members of the student group did however say that whilst it was good, they perhaps had less to gain from it. Age could well be a contributing factor to levels of self-efficacy, in that the older you are, the more you have confidence in your abilities, as life experience has taught you so.

“…there wasn’t a huge gap that they fulfilled, some small part maybe… I found it was thrilling all in all but I wouldn’t say I found anything new. I’m 52 years old…”

Data from this focus group will soon undergo thematic analysis- results from this to be published soon, and a follow up GRIT day is scheduled for February 2018 to further inform this research project.