Reliability and Validation | Assessments and Methodology
- How reliable is the test?
- Does the test consistently measure what it is supposed to measure?
- Imagine that you test the same person twice, subsequently, using the same tool. Will you – as expected – have two almost identical results?
…these are just some of the questions we get asked when we meet clients and audiences all over the world. The following information addresses these questions and explains what Motivation Factor does to maximize the quality, reliability and user-value of our tools and methodology.
First of all..
Motivation Factor’s assessments are not based on psychometric profiling (such as one would find with Jung’s theory and others). There is nothing wrong with profiling tools – most of them are great – but we measure the level of motivation by focusing on the “Why” instead of the “What” and the “How”, and we identify the key motivational drivers, such as Needs and Talents.
Please notice that – in particular – Needs can change over time, depending on the individual’s situation and life circumstances.
Motivation Factor is:
- Based on 100+ highly validated questions
- Individualized with mathematically calculated, algorithm based scoring that is added up and combined with the validation data
- Measured and focused on motivation – and motivation only
- A live snap-shot of what the individual’s or team’s motivation looks like, right now!
- Inspired by several motivation theories taken from Maslow, Hertzberg, The Flow Theory, Positive Psychology, Appreciative inquiry, etc.
Motivation Factor is not:
- A psychometrical tool
- A typological profiling tool
- Based on psykometrisk theories (Jung, etc.)
- About predictive behavior
- Just another employee satisfaction survey
Validation and quality control
Believe it or not, international standards for how to validate assessment tools, like the ones presented by Motivation Factor, do exist. Only a few companies are able to undertake and achieve the complicated job of validating questionnaires and assessments according to these international standards.
All our questionnaires and assessments are validated by Boston Research Group.
Correlation: How well do the 5 elements: Energy, Needs, Talents and Purpose connect and correlate to each other?
- All five concepts are related to each other yet
- Some relationships are stronger than others:
- Strategic Connection and Purpose have the closest relationship to one another
- Needs and Strategic Connection as well as Needs and Energy are also closely related
- The “weakest” (weakest because it is still a positive and significant correlation) is between Talents and Energy. In fact, Talents has the lowest correlation with each of the other concept
Boston Research Group | March 2018
Regression: If we consider a person to be 100% motivated (Total Motivation), how important is each of the five elements: Energy, Needs, Talents, Purpose and Strategic Connection?
“The results from the Regression Analysis clearly demonstrate several important findings;
- Each of the five concepts being tested – Strategic Connection, Energy, Needs, Talents, Purpose – play a role in driving Total Motivation.
- The degree to which each concept impacts Total Motivation varies from a high of 35% for Purpose (i.e., Purpose accounts for 35% of the variance in Total Motivation) to a low of 10% for Talents.
Notes: Beta value is a measure of how strongly each predictor variable influences the dependent variable (Total Motivation). The sum of the beta weights, in this analysis, total more than 1 due to the degree of multicolinearity (or overlap) among the independent variables. This overlap essentially means that, to a modest degree, the independent variables explain some of the same variance in the dependent variable”.
In order to present the results in a more concise manner, the beta values are reproportioned to equal 1 (and thus represent the total variance in Total Motivation).
Thus, the strongest two contributors to Total Motivation are, by far, Purpose and Energy. More modest contributions are derived from Needs and Talents.Further, it is interesting to note that the Strategic Connection 1) plays a significant role in Total Motivation; however 2) that role is less critical to Total Motivation than either Purpose or Energy.
Boston Research Group | March 2018
Cronbach’s alpha is the most common measure of internal consistency (“reliability”). It is most commonly used when you have multiple Likert questions in a survey/questionnaire that form a scale and you wish to determine if the scale is reliable.
The following displays scores and interpretations (the interpretation is a general rule of thumb):
Cronbach’s alpha ratings for Motivation Factor’s Measurement:
We asked Boston Research Group to factor analyze the question related to motivation capabilities and intrinsic motivation. The questions related to Motivation Capabilities variables formed a clear factor with other Motivation Capabilities variables and the same applied to Intrinsic Motivation. This confirms the basic concept that Motivation Capabilities and Intrinsic Motivation address unique areas within the respondents’ way of thinking.
The results from the Factor Analysis are presented below:
Motivation Factor meets and exceeds international standards regarding reliability and validation. All validation results are available in reports from Boston Research Group. Click here to email-request the full validation report.
Motivation Factor’s methodology, exercises and workshops are described and documented in the Book: The Motivated Brain, by Motivation Factor Founder Helle Bundgaard and MIT Brain Scientist Jefferson Roy.
In Chapter 10 of The Motivated Brain, Jefferson Roy connects Motivation Factor’s methodology to modern brain science. Click here to email-request Chapter 10: Motivation Lives in the Brain.
We are continuously developing and improving Motivation Factor’s products, program, methodology, and services. As new information is discovered from research, completed assessments and successful consulting workshops, we adapt and advance the Motivation Factor offering
To ensure that all these activities meet our requirements and ambitions, and to keep a high level of quality, we have appointed an Expert Panel. The panel, consisting of experienced Motivation Factor Practitioners, psychologists and a brain researcher, are reviewing all changes being made to any area of the Motivation Factor portfolio. This will guarantee a high success rate in a quality launch.