Chapter 3

Exploratory Study

Post-Graduate Thinking and the Identity Compass

Introduction

The aim of this pilot study was to initially explore the feasibility of running a larger study and to test the methodological approach, before investigating almost 200 post-graduate student profiles. In the Literature Review it was ascertained that people in general are not aware of how they think in the moment, nor of how their thinking is deconstructed. This exploratory study aimed to test this awareness and thinking construction with a small contingent of post-graduate students at Coventry University using the Identity Compass profile tool. The intention was to deconstruct their thinking using fifty individual Meta-Programmes (MP’s) as per the literature review in the research by Daniels (2010) and Brown (2003). In order to identify common patterns of MP use using the Identity Compass tool, there were several sub-objectives, each of which will be considered in turn later:

  1. To determine if there are Meta-Programmes common to all post-graduate students.
  2. To determine if a specific combination of MP’s creates an academic thinking style.
  3. To determine if there are driver Meta-Programmes as suggested in the literature.

The objectives and the aims helped to formulate the main hypotheses:

  • Certain Meta-Programmes have more of an effect on the profile than others and thus might be classed as “driver programmes”.
  • Different combinations of Meta-Programmes will be discovered.

The Research Aim

In higher education, one must attain higher-order thinking as it is integral to the post-graduate process (Halpern, 1998). However, the reliability and validity of the measure of the higher-order thinking was not well-established (Williams, 1999). Despite the many labels on higher-order thinking, there was very little operational validity of the complexity of the academic tasks, and the types of responses provided by students when questioned about their thinking (Crone-Todd, 2007). Thus, to be a successful post-graduate student, one must develop the critical and logical thinking skills required, an awareness of one’s own learning process and studying strategies was essential (Gassner, 2009). Higher Education aims to produce students who ‘have learned how to learn and are capable of continuously adapting themselves…’ (Kelly, et al., 1999). Formal logical thinking based on sequential procedural decision-making was sufficient for under-graduate level courses, but such thinking should be insufficient for a more dialectical approach to a post-graduate degree. For example: Paglis, (2013) argues for the use of a competency-based pedagogy for undergraduate students which Spence and McDonald (2015) argue was not sufficient for post-graduates in order to develop future leaders in an increasingly demanding and complex world (Petrie, 2011).

Given the more complex nature of Higher Education then, a simplistic behavioural approach to a student’s interaction with the post-graduate journey was insufficient (Haselgrove, 1994). Because universities are now measured on efficacy and deliverables, how post-graduate students think in relation to the teaching offered was becoming increasingly important. For example: the development offered by traditional university courses is, by definition, lateral or horizontal development, which Heifetz, et al., (2009) define as the answer to technical challenges that have known solutions, implementable by existing knowhow. On the other hand, vertical development was defined as ‘possessing greater mental capacity to encounter completely different ways of understanding the world’ (Kegan & Lahey, 2009) and Rooke and Torbert (2005) define this vertical growth as: “becoming more aware of how an individual interprets their surroundings and reacts when their power or safety is challenged”, which was achievable by increasing in their own complexity (Petrie, 2011, p.12). It has also been suggested that students might achieve academic success were they to engage in those learning activities mentioned here (Duncan and Barrett, 2007; Vermunt and Vermetten, 2004).

The research aim was thus to demonstrate in the current study that it was possible to deconstruct a post-graduate student’s thinking in order to determine if Meta-Programmes are influential in a student’s construction of self in an academic context.

Method

Research Philosophy

The difference between an exploratory study and an experimental study is determined by the structure and outcome of the study. Cooper and Schindler (2000), suggest that exploratory studies tend to be loosely structured and aim to discover new ideas or research tasks. The aim of exploratory research was to develop a hypothesis or research question for further research. An experimental study would continue from where the exploratory study ends. From the literature review, there was a need for both an exploratory study followed by an experimental study.

Constructivism is when individuals create their own meaning in the way they map the world in which they operate, which is contingent upon human interactions and transmitted in a social context such as academia (Crotty, 1998). Positivism is a useful approach when engaging an online questionnaire, such as the Identity Compass tool as it “allowed the data to emerge” (Bryant, 2003; Charmaz, 2013). Measuring the objective reality that occurred and developing numeric measures of observations was paramount for positivists.

The current study was a unique blend of both Constructivism and Positivism in that the Identity Compass profile tool allowed for an interpretation of the awareness of the participants’ construction of their thinking, and although the profile output was quantitative, this unique construction is more typically linked to qualitative research (Mertens, 2009). Also, one cannot be entirely positive about one’s claims when studying the behaviour of humans (Phillips & Burbules, 2000). Meaning from the data was generated and any potential for confirmation or researcher bias was countered by interpreting only the quantitative data according to the profile training. In order to be time and cost-efficient, a survey methodology was selected to generate the necessary quantitative data (Walonick, 2010).

Research Design

Online questionnaires provide a relatively inexpensive and efficient way of obtaining large amounts of data (such as opinions and attitudes) from a large pool of participants as the researcher would not need to be present when the questionnaires were fulfilled. This is useful for large populations when interviews would be impractical (Frey and Oishi, 1995).

As per the Methodology (chapter 2), there were a number of profile tools available that reportedly measured Meta-Programmes, including the Motivation Profile Questionnaire (MPQ) (Arthur and Engel, 2000) as used by Miller and Deere (2000) Brown (2002; 2003) and Brown and Graff (2004). The latter were undertaken with undergraduate students to determine meta-programme differences between students and lecturers. Brown (2005) went to improve the MPQ with his MPI (Inventory) questionnaire. A third tool was the Inventory for Work Attitude and Motivation (iWAM) (Merlevede, 2001) which is an online assessment tool that measures motivational and attitudinal patterns. It was developed in the United States and consists of 40 questions, each of which has five potential responses that determine a meta-programme-based response. The decision to use the Identity Compass profile tool was made because the tool interpreted Meta-Programmes in the way that was conducive to defining polar relationships between the pairs, such as Internal/External, and importantly, had evidence in efficacy from previous research (Daniels, 2010). The researcher also had some experience with the Identity Compass profile tool as it was the preferred method of profiling post-graduate students at Coventry University London.

Participants

 

The study utilised an opportunity sample of 32 Post-Graduate students. Twenty- one were attending an MBA course at Coventry University London campus, and eleven were at Coventry university’s main campus, as PhD candidates. Seven were in stage 1, and three were in stage 2 of their PhD, from a group consisting of 3 males and eight females. There were twenty-one MBA students from Coventry University London: 7 male and 14 female. Participants’ age ranged from 21 to 32 years. Their countries of origin were, but not limited to: England, India, Pakistan, Korea, China, Japan and Nigeria.      

The justification for selecting the participants was directly related to the research question on post-graduate student thinking and how it was deconstructed using the Identity Compass profile tool. The MA students at the London campus were asked to participate as they formed part of the then-current High Flyers programme at the time. The PhD students from Coventry’s main campus were asked to participate as it was intended that a differentiator between the construction of students at the two levels of post-graduate study would take place. However, this was not the case.

Measures

The Identity Compass Profile Tool

The Identity Compass (IC) was developed in Germany by Arne Maus. Maus, (2011, p.167) states the validity of the Identity Compass profiling system was measured by Prof. Scheffer of Hamburg University against the existing personality tests such as NEO-FFI (Big Five), MBTI, Operational Motivation Test and the CFT. In this study, the Identity Compass profile questionnaire used has twenty-two years in industry, with over 25,000 profiles corroborating its efficacy (discovered in conversation with the profile owner in November 2017).

Studies show that the Identity Compass correlates with implicit personality structures (Scheffer, 2003, as cited in Maus, 2019). It has been repeatedly validated against the main personality profile systems, such as NEO-FFI and MBTI, as well as CFT (General intelligence) where it shows substantial correlations. The correlations at .40 are not so high as to nominate the Identity Compass as a personality test. (Maus, 2011. p167).

The Identity Compass deconstructs the thinking of the participant into groups of Meta-Programmes as per their collective intention (see Appendix 2). For example, ‘Seeing’, ‘Hearing’ and ‘Feeling’ are all considered senses, so the Identity Compass puts them together in a group called “Sensory Channel”. Another grouping was called “Comparison” and contains the Meta-Programmes of ‘Sameness’ and ‘Difference’. Shulman (2002, p.37) states that: “One of the central ways we make sense of experience is by making differences”. As a post-graduate student, an important ability was to be able to notice ‘difference’ in one’s research material (Brown, 2002), by asking oneself: how have other researchers written what has been said, and how was it similar or different to what has been written before? This ‘comparison’ helps to differentiate the direction of intention (of the student) when researching as this highlights the direction of attention from a research perspective. With this in mind, it would be anticipated we see a student’s IC output having ‘Difference’ higher in their percentage score than ‘Sameness’ due to the need for an objective look at the content of previous researchers’ studies and what they said about them. This predictability was also true of other Meta-Programmes.

Reliability of the Identity Compass

Although reliability and validity claims for learning style instruments are not well substantiated (Coffield, et al., 2004; Hawk & Shah, 2007), the scores for the validity of the Identity Compass are presented here. Cronbach’s alpha should be higher than 0.7 and ideally between 0.90 and 0.95 (Kline, 1999). Even if a measure is highly reliable, this does not show that the measure is assessing the theoretical constructs stated. The internal consistency of the Identity Compass, using Cronbach’s alpha coefficients (Cronbach, 1971) for the 11 items in the first dimension of the factor analysis was 0.894. Internal reliability of the individual Meta-Programmes scored above .90 thus demonstrating excellent internal reliability of the Identity Compass (Kline, 1999).

Procedures

Ethical approval was granted from the Coventry University ethics committee. Participants were treated in accordance with BPS ethical guidelines. Written consent was gained from all participants (see Appendix 4). PhD students were contacted by email via the Faculty Heads at Coventry University. These emails contained details of the research, including all ethical considerations and reasons for the study. Posts were left on social media for volunteers to undertake the profile as part of the research. These were specifically targeted at the Coventry University PhD forum on Facebook. It was made apparent that although the profile would be taken by the student, the resultant data was for research purposes only and no feedback was offered.

All MBA students were contacted separately from PhD students, via direct email, as each had undertaken an Identity Compass profile as part of their High Flyers programme with CU London. The Identity Compass profile forms the basis of the High Flyers programme from a self-awareness perspective for each student. The programme encourages students to go above and beyond their normal academic achievements. Participants were presented with the same question set and asked to use academia as the context when filling in the questionnaire. The questions were contextual and mainly open-ended with a number of closed questions or situational scenarios requiring perspective answers.

Responses were made on a 6-point Likert scale ranging from 1 = Not True for Me, to 6 = Very True for Me. Example items include:

  1. On projects I mostly tend to:

a. push things forward.

b. first analyse and check alternatives.

  • In meetings, it is more important to place emphasis on:

a. who is talking.

b. the purpose of the meeting.

c. how the purpose can be achieved.

d. the process of communication.

e. how one talks.

f. where the meeting takes place

  • In a meeting it is more important to pay attention to:

a. my perspective.

b. the perspective of the others.

c. the view of someone outside of the situation.

  • I know best that I have done a good job on this project:

a. by my own judgment.

b. by getting feedback from others.

For example: question 4 was designed to elicit a participant’s locus of evaluation which would determine their ‘Internal’ / ‘External’ score. Question 3 focused on the participant’s capacity to do second position in that answer ‘b’ was about the other person’s position. Should this be significantly higher than the participant’s score for ‘Own’ then they were potentially stuck in second position and subsume their own ideas for the ideas of others (Maus, 2011, p50). There were a number of questions that focused on these, from both directions of intention, and thus a score was devised based on the participant’s Likert scale response.

Results

Initially, fifty Meta-Programmes were included in the exploratory factor analysis for determining the factor structure, using IBM SPSS Statistics 25 software. Most of the variables used in this study were normally distributed, according to the Shapiro-Wilk normality test (36 out of 50 Meta-Programmes, Table A1 in Appendix 5), and all histogram distributions were close to normal. The sample size of n = 32 was too small (Tabachnick, & Fidell, 2007), and hence inadequate for factor analysis. Accordingly, principal component analysis was used as a method of factor analysis. According to the correlation matrix, there were many intercorrelations amongst the items higher than r = .30.

The principal component analysis method revealed 13 dimensions (or components) with eigenvalues greater than λ = 1.00, explaining 85.89% of the variance. According to the scree plot, keeping the first four factors seemed the most appropriate solution, according to the Cattell’s criteria (1966).

The four factors explained 56.97% of the variance. Direct oblimin rotation was used in order to get more accurate results, as there was a moderate correlation between the first and the second component (Table 3.6). Many items had communalities higher than .40, except for Places, Activity, Things, Affiliation, and Difference. These items, due to low communality, did not correlate with the other items. However, none of the items had factor loadings lower than .32, so no item was excluded from the final factor structure (Tabachnick, & Fidell, 2007).

Table 3.1: Intercorrelations amongst the obtained components

Component1234
11.00   
2.231.00  
3.36.251.00 
4.17-.06.081.00

N = 32.

Table 3.7 presents the component structure created based on Pattern Matrix (see Table A2 in Appendix 5), along with the internal consistency coefficients for each of the newly-made subscales. Each subscale had a high reliability, except for the 5-item Dimension 4. Some items had factor loadings of similar sizes on more than one component, for example, Options, Towards, Information, Places, Caring for Self and Trustful.

Table 3.2: Four dimensions of Meta-Programmes, and reliability of their subscales

Cognitive dimensions
Dimension 1, α = .95Dimension 2, α = .89Dimension 3, α = .86Dimension 4, α = .56
InfluencePre-ActiveQuality ControlCaring for Others
SeeingIndividualistHearingRelationship
LookingPastExternalSameness
InternalScepticLong-termCaring for Self
AbstractShort-termDetailsTrustful
VisionConcreteReading 
GlobalOwnRealization 
Team PlayerTaskConsensus 
ObserverActivityThings 
Re-ActiveTowardsProcedures 
PeopleInformationPlaces 
Partner   
Doing   
Listening   
Achievement   
Future   
Difference   
Affiliation   
Feeling   
Present   
Away From   
Polar   
Options   

Based on the results of the principal component analysis, there exists a subset of dimensions (Objective 3) as a summary of the fifty investigated Meta-Programmes amongst post-graduate students. These dimensions are: Dimension 1, Dimension 3, Dimension 2, and Dimension 4. Each one of which was created by calculating the mean value of five and 23 different Meta-Programmes after obtaining the results of the PCA.

Discussion

The current study starts from the premise that Meta-Programmes exist as high-level abstracted patterns of data sorting, thinking and responding (Maus, 2011). These MP’s can be reliably identified and measured by various existing methods as mentioned in the Methodology chapter (Daniels, 2010). The main aim was to test the methodological process on a small number of participants to determine how a post-graduate student’s thinking at Coventry University could be deconstructed by Meta-Programmes using the Identity Compass profile tool. The main objectives were to:

  1. To determine if there are Meta-Programmes common to all post-graduate students.
  2. To determine if a specific combination of MP’s creates an academic thinking style.
  3. To determine if there are driver Meta-Programmes as suggested in the literature.

The main hypotheses were:

  • Certain Meta-Programmes have more of an effect on the profile than others and thus might be classed as “driver programmes”.
  • Different combinations of Meta-Programmes will be discovered.

From the findings it can be summarised thus: (1) there are MP’s common to all post-graduate students; (2) a specific combination of MP’s does occur in the data; (3) a dominant sub-set cannot be determined due to participant numbers.

Objective 1 asked: if there are Meta-Programmes common to all post-graduate students. Each Meta-Programme demonstrates an unconscious intention towards a particular way of thinking in context. The results have given a potential hierarchy of Meta-Programmes in Dimension 1. However, it was not entirely clear at this stage what this meant. In this discussion, there have been judgements made on the meaning of the hierarchy and the individual Meta-Programmes. For example, the most-used preference by Median score was ‘Towards’ (see Table 3.8). The question that required further and longitudinal research was: was ‘Towards’ the most important Meta-Programme because every student demonstrated they use it, or was it the least significant for the same reason?

Table 3.3: Meta-Programmes ranked by Median score

Towards90
Information85
Global85
Activity85
Task85
Achievement80
Vision85
Re-Active78
Present80
Realisation80
Concrete78
Caring for Self75
Consensus78
Seeing80
Future80
Options75
Details75
Short term75
Quality Control80
Long term75

By virtue of this question, it was argued that the outliers needed further investigation, and the study required more participants for robustness. Because of the inability to have dialogue with the participants, it cannot be stated accurately what the correlations meant, therefore a logical assumption was to repeat the current study with a greater number of participants. Further to this, a qualitative study that discovered the lived experience of the participants would shed light on their proclivities and the construction of their thinking.

It was asked from a social-constructionist perspective, what the contributory factors to this hierarchy of Meta-Programmes were, and would it have been beneficial to research the background of the students to discover what led to the use of these MP’s specifically? If the hierarchy were socially derived, contributing factors such as the students all coming from the same country/region, or their age range was not diverse enough (thus emphasising a generational bias), or they were all simply post-graduate students, and embedded in their academic context, which created specific shared intentions as per social construction theory (Burr, 1995). If these are the case, future study could mitigate these points by including profiles by under-graduates and also supervisors/lecturers to discover how they perceive their environment and how they feel they cope with the post-graduate process, akin to the findings of Brown (2004).

From the cognitive view, Dweck, (2012) would ask what they might believe about their own learning ability, as this will directly affect their performance. Also, what are the relationships between Meta-Programmes such as ‘Information’ and learning preferences? This could better be answered in a qualitative face-to-face interview to determine a student’s previous awareness and post-questionnaire awareness in a future study.

Objective 2 asked if an academic thinking style derived from a specific combination of Meta-Programmes could be inferred. With n=32, this was not possible. However, the results of the direct oblimin rotation suggested that with many items having communalities higher than .40, there was a moderate correlation between the first and second components (Tabachnick & Fidell, 2007).

The principle seen in Table 3.8 was an important aspect of objective 2 as it allowed the reader to understand that there were component parts to a post-graduate student’s thinking style. This principle was replicated in future larger studies to gain a deeper understanding of how a student’s thinking was deconstructed in order to offer a greater understanding of their Intention in context.

Objective 3 asked if a subset of dominant Meta-Programmes can be found in the data (see Table 3.9). Dimension 1 had a reliability score (internal consistency coefficient) of .95 (a = .95) and offered a subset of influential Meta-Programmes.

Table 3.4: Meta-Programmes in Dimension 1

Dimension 1
Influence
Seeing
Looking
Internal
Abstract
Vision
Global
Team Player
Observer
Re-Active
People
Partner
Doing
Listening
Achievement
Future
Difference
Affiliation
Feeling
Present
Away From
Polar
Options

 

However, this finding will better be tested on a greater number of participants in a future study to verify or refute the findings here.

Chapter Summary

This exploratory study provided small-scale empirical support for considering the different ways of knowing one’s thinking construction as a lens through which to understand the contextual experiences of 32 post-graduate students at Coventry University.

It was also demonstrated that this was an appropriate approach to discover if student thinking could be deconstructed using Meta-Programmes, and if said deconstruction produced viable patterns in their combination. From the metacognition literature, it suggests that the accuracy of metacognitive judgments on an individual’s capacity to perform self-evaluation on their performance varies considerably, (Rounis, Maniscalco, Rothwell, Passingham, & Lau, 2010), thus the use of Meta-Programmes offers an alternate way to deconstruct how we know what we know about our thinking.

The study has extended the research of Brown (2004) that stated there are important Meta-Programmes in the context of post-graduate study, but there was insufficient information to definitively determine if those MP’s were directing the thinking of the student in their post-graduate course, or if they were an outcome of the student being on their post-graduate course.

The data did demonstrate that the Meta-Programmes clustered differently depending on the thinking of the post-graduate student. The findings showed a clear separation between direct and indirect variables (MP’s), or those variables that are tied to behaviours rather than thinking alone (De Bolle et al., 2015).

As the Identity Compass is a profile system, it was inevitable that students change over time, so it should be understood that each profile was a snapshot in time of thinking for each participant. This also lends support for a longitudinal study on changing levels of self-awareness. As there was no control group for comparative data, it was argued that the contextual meaning behind the Meta-Programmes for the individual 32 post-graduate students was no different to the meaning created by any participant of the Identity Compass profile. Only further research would help to determine a contextual bias (Academia) for the MP’s and their clustering.

The findings of this exploratory study suggested there was a specific need to create a new tool for determining how Meta-Programmes were filtered as dependent or independent variables. With further research, the new tool would evolve out of the Meta-Programme differences and establish levels of self-awareness, giving participants a better explanation for their habituated thinking and behaving in context (Kahneman, 2011).

For those post-graduate students who wanted to discover ‘how’ they think or increase awareness of validation for ‘why’ they think the way they did, it was evident at this stage that the Identity Compass used too many questions for too many Meta-Programmes.

Finally, further to this, for those students who wished to make changes in their thinking construction, and thus their world view, there were potential interventions for each Meta-Programme that would progress their thinking towards a more balanced approach, however, this was out of scope for the current study.

Further Study

The testing of the Methodological approach in the pilot study uncovered a number of interesting findings when looking at the deconstruction of post-graduate student’s thinking. From here, it was established that the researcher could move on to the second study with a greater number of participants using this same method. Study 2 would thus map the aims and objectives onto a larger post-graduate student dataset in order to discover to what extent the objectives and results were replicable. The objectives were also updated. For example: does a greater number of participants produce a different dimension in the factor analysis? Does this validate objective 2? Would it convince the researcher that different combinations of Meta-Programmes are different thinking styles?

However, as the current study was based on 32 profiles, there was a question of extrapolation to the wider population of post-graduate students. A second study that takes the results of the current study and attempts to replicate them on a larger number of post-graduate students would further investigate the hypotheses and go some way to validating or refuting the findings of the current study. For this, study 2 was devised, using the larger post-graduate population from the large data set within the Identity Compass database.

As discussed, the IC output differentiates between intention and attention in context, and the post-graduate student’s awareness of this intention/attention is currently not measured by any profile tool. With this in mind, it was evident that a student’s level of self-awareness of their intention in the moment was limited, which also warranted further study.