Measurement Processes
Measurement is the process by which the fundamental quantities and qualities
of members of a system, together with their relationships are derived and
documented. The clearest example of this process is measurement of
physical properties such as counts - how many, mass, weights, distance. In
these examples the processes are well know. This is the most accurate example of measurement. Sometimes one uses the same numbers (1,2,3,etc.) to given order. From the statistical perspective these two processes require different methods for proper analysis.
Issues of how measurement processes focus on the nature of phenomenon are key
criteria for understanding measurement processes and appropriately interpreting
results. Based on the nature of the focus of measurement, four
classifications are presented.
 |
Observations and counts
 |
As the description identifies, this process of measurement is based on
counting. This implies focus on observable phenomena. These may
be objects, behaviors, events, etc. In many ways this is a simple form
of criterion-reference measurement described below. |
|
Example:
 |
How many...? |
|
|
Constraints and caveats:
|
Not all measurement focuses on observable things.
|
|
|
|
Opinion and belief
|
When ask, a respondent can provide use information about a wide variety
of their actions and beliefs. These responses may, or may not, be
accurate and/or truthful, but in many situations a research accepts them as
"truth." This acceptance is very appropriate in many settings.
In addition to basic statements of belief we can also ask about strength of
value and belief. |
|
Example: Frequently this type of measurement user a Likert scale
in which a respondent indicates a strength of response.
|
Voting is an important action for every citizen.
___Strongly Agree ___Agree ___Disagree
___Strongly Disagree |
|
|
Constraints and caveats:
|
The most obvious challenge is getting people to report "truthfully."
While there are many questions that individuals will answer truthfully,
there are others that are much hard to collect.
|
|
|
|
Ipsative measurements
|
The term "ipsative" (sometimes seen as ipsitive [sic.] ) is based on "ipso
facto" which means "by the very fact itself, the very fact." In
this usage, ipsative measurement means measurement against itself.
While different usages focus on different traits of this type of
measurement, two critical characteristics emerge.
|
First, these measurements are self-reported evaluations. This
then gets into the issues of how self-aware an individual is (see Johari
window), and how
situational demands bias a self-report. |
|
Second, these measurements utilize a forced choice ranking of two or
more indicators. Even when some strength of responses is added to
these rankings, the forced choices used to build scales indicate
relative, not quantitative results. This then effects the
statistically tools and interpretation of these findings. (see
Barton, 1996) One of the key issues is that the sum of the
scores build on such forced choice items tend to sum to the same total or
a very narrow range of possible scores. This then means that the
scales tend to be significantly correlated with each other. Such
that any one scale can be determined from knowing the sum of the other
scales. |
This form of measurement frequently focuses on hidden, or latent traits
that are not "known" to the individual such as those in personality
instruments. In this form of measurement the respondent chooses,
ranks, or associates themselves with some indicators, and disassociates
themselves from other indicators. These associations are then
aggregated into multiple scales.
|
|
Example: (Each of the five items is an indicator of a trait.
The items are not a single dimension of theoretical ideas, but a list, with
each item indicating a different trait. Notice how the respondent is
forced to order and select or reject items.)
Indicate which of the following items are most to least
like you
Use 1 = Most like you down to 5 = Least like you.
___ Young at heart, ___
Introspective, ___ Pessimistic,
___ Persuasive,
___ Obedient
|
|
Constraints and caveats:
|
Scale scores are relative to the individual, not to comparison among
individuals. One does not know how a score of x for responder A
relates to that same score for a different responder. |
|
Scores are relative to each other, not measures of a quantity. One
knows that the individual reports more association with the indicators of
one trait rather than another, but self-reporting twice as many indicators
does not mean that the level of a trait is twice as high. |
|
Scores can be highly influenced by situational expectations.
Individuals may see themselves one way in a work setting and a different way
in a social setting. Thus, depending on the instrument scores may vary
depending on in what context an instrument is utilized. |
|
|
|
Norm-referenced measurements
|
This measurement process focuses on relative performances of individuals
compared with that of the others in a set of data. It is the form of
measurement most frequently used in school. A test is given, students
produce a range of results from top performers to lowest. If the test
was very hard a teacher might curve it, or not. In any case any one
student's performance can be compared with the collective performances of
all the other students. With norm-references measurement, when a
standard is set for acceptable performance it is specified in terms
specifying minimum acceptable levels given the history of other
performances. In some more sophisticated systems relative performance
may be translated from raw scores into "standardized" scores in which the
characteristics of the standardized scores are known. Two examples of
these standardized scores are Z-scores, or T-scores. |
|
Example:
|
Scores from ACT, College Boards, may teachers' tests |
|
|
Constraints and caveats:
|
Scores may not be be consistent across instruments, one test may be a
"killer" and another "easy."
|
|
|
|
Criterion-reference measurements
|
In many ways this is just a count form of measurement. In this
process some criterion is specified, and then some person, or event is
compared with that criterion, and an evaluation made that the criterion is
meet or not. The measurement is a yes/no result. |
|
Example:
|
Disability is the inability of an individual to complete
three or more of the following actions.
|
Dress themselves |
|
Bath themselves |
|
Feed themselves |
|
and the list would continue etc. |
|
|
|
Constraints and caveats:
|
Just as with some of the other measurement processes criterion-reference
assessments are not always, easily defined in a yes/no judgment. In
the case of many learning and behavior settings there may be many levels of
competence. |
|
|
Johari window: An exploration of what
we know and what others know about ourselves.
Barton, Helen. 1996. "Strengths and Limitations of Ipsative Measurement," Journal of Occupational and Organizational Psychology, 69, 49-56.
