THE RESEARCH PROCESS
1993 © David S. Walonick, Ph.D.
All rights reserved.
We understand the world by asking questions and
searching for answers. Our construction of reality
depends on the nature of our inquiry.
Until the sixteenth century, human inquiry was
primarily based on introspection. The way to know things
was to turn inward and use logic to seek the truth. This
paradigm had endured for a millennium and was a
well-established conceptual framework for understanding
the world. The seeker of knowledge was an integral part
of the inquiry process.
A profound change occurred during the sixteenth and
seventeenth centuries. Copernicus, Kepler, Galileo,
Descartes, Bacon, Newton, and Locke presented new ways of
examining nature. Our method of understanding the world
came to rely on measurement and quantification.
Mathematics replaced introspection as the key to supreme
truths. The Scientific Revolution was born.
Objectivity became a critical component of the new
scientific method. The investigator was an observer,
rather than a participant in the inquiry process. A
mechanistic view of the universe evolved. We believed
that we could understand the whole by performing an
examination of the individual parts. Experimentation and
deduction became the tools of the scholar. For two
hundred years, the new paradigm slowly evolved to become
part of the reality framework of society. The Age of
Enlightenment had arrived.
Scientific research methodology was very successful at
explaining natural phenomena. It provided a systematic
way of knowing. Western philosophers embraced this new
structure of inquiry. Eastern philosophy continued to
stress the importance of the one seeking knowledge. By
the beginning of the twentieth century, a complete schism
had occurred. Western and Eastern philosophies were
mutually exclusive and incompatible.
Then something remarkable happened. Einstein's
proposed that the observer was not separate from the
phenomena being studied. Indeed, his theory of relativity
actually stressed the role of the observer. Quantum
mechanics carried this a step further and stated that the
act of observation could change the thing being observed.
The researcher was not simply an observer, but in fact,
was an integral part of the process. In physics, Western
and Eastern philosophies have met. This idea has not been
incorporated into the standard social science research
model, and today's social science community see
themselves as objective observers of the phenomena being
studied. However, "it is an established principle of
measurement that instruments react with the things they
measure." (Spector, 1981, p. 25) The concept of instrument
reactivity states that an instrument itself can
disturb the thing being measured.
Problem Recognition & Definition
All research begins with a question. Intellectual
curiosity is often the foundation for scholarly inquiry.
Some questions are not testable. The classic
philosophical example is to ask, "How many angels
can dance on the head of a pin?" While the question
might elicit profound and thoughtful revelations, it
clearly cannot be tested with an empirical experiment.
Prior to Descartes, this is precisely the kind of
question that would engage the minds of learned men.
Their answers came from within. The modern scientific
method precludes asking questions that cannot be
empirically tested. If the angels cannot be observed or
detected, the question is considered inappropriate for
A paradigm is maintained as much by the process of
formulating questions as it is by the answers to those
questions. By excluding certain types of questions, we
limit the scope of our thinking. It is interesting to
note, however, that modern physicists have began to ask
the same kinds of questions posed by the Eastern
philosophers. "Does a tree falling in the forest
make a sound if nobody is there to hear it?" This
seemingly trivial question is at the heart of the
observer/observed dichotomy. In fact, quantum mechanics
predicts that this kind of question cannot be answered
with complete certainty. It is the beginning of a new
Defining the goals and objectives of a research
project is one of the most important steps in the
research process. Clearly stated goals keep a research
project focused. The process of goal definition usually
begins by writing down the broad and general goals of the
study. As the process continues, the goals become more
clearly defined and the research issues are narrowed.
Exploratory research (e.g., literature reviews,
talking to people, and focus groups) goes hand-in-hand
with the goal clarification process. The literature
review is especially important because it obviates the
need to reinvent the wheel for every new research
question. More importantly, it gives researchers the
opportunity to build on each others work.
The research question itself can be stated as a
hypothesis. A hypothesis is simply the investigator's
belief about a problem. Typically, a researcher
formulates an opinion during the literature review
process. The process of reviewing other scholar's work
often clarifies the theoretical issues associated with
the research question. It also can help to elucidate the
significance of the issues to the research community.
The hypothesis is converted into a null hypothesis in
order to make it testable. "The only way to test a
hypothesis is to eliminate alternatives of the
hypothesis." (Anderson, 1966, p.9) Statistical
techniques will enable us to reject a null hypothesis,
but they do not provide us with a way to accept a
hypothesis. Therefore, all hypothesis testing is
Creating the Research Design
Defining a research problem provides a format for
further investigation. A well-defined problem points to a
method of investigation. There is no one best method of
research for all situations. Rather, there are a wide
variety of techniques for the researcher to choose from.
Often, the selection of a technique involves a series of
trade-offs. For example, there is often a trade-off
between cost and the quality of information obtained.
Time constraints sometimes force a trade-off with the
overall research design. Budget and time constraints must
always be considered as part of the design process
Many authors have categorized research design as
either descriptive or causal. Descriptive
studies are meant to answer the questions of who, what,
where, when and how. Causal studies are undertaken to
determine how one variable affects another. McDaniel and
Gates (1991) state that the two characteristics that
define causality are temporal sequence and concomitant
The word causal may be a misnomer. The mere
existence of a temporal relationship between two
variables does not prove or even imply that A causes B.
It is never possible to prove causality. At best,
we can theorize about causality based on the relationship
between two or more variables, however, this is prone to
misinterpretation. Personal bias can lead to totally
erroneous statements. For example, Blacks often score
lower on I.Q. scores than their White counterparts. It
would be irresponsible to conclude that ethnicity causes
high or low I.Q. scores. In social science research,
making false assumptions about causality can delude the
researcher into ignoring other (more important)
There are three basic methods of research: 1) survey,
2) observation, and 3) experiment (McDaniel and Gates,
1991). Each method has its advantages and disadvantages.
The survey is the most common method of
gathering information in the social sciences. It can be a
face-to-face interview, telephone, or mail survey. A
personal interview is one of the best methods obtaining
personal, detailed, or in-depth information. It usually
involves a lengthy questionnaire that the interviewer
fills out while asking questions. It allows for extensive
probing by the interviewer and gives respondents the
ability to elaborate their answers. Telephone interviews
are similar to face-to-face interviews. They are more
efficient in terms of time and cost, however, they are
limited in the amount of in-depth probing that can be
accomplished, and the amount of time that can be
allocated to the interview. A mail survey is generally
the most cost effective interview method. The researcher
can obtain opinions, but trying to meaningfully probe
opinions is very difficult.
Observation research monitors respondents'
actions without directly interacting with them. It has
been used for many years by A.C. Nielsen to monitor
television viewing habits. Psychologists often use
one-way mirrors to study behavior. Social scientists
often study societal and group behaviors by simply
observing them. The fastest growing form of observation
research has been made possible by the bar code scanners
at cash registers, where purchasing habits of consumers
can now be automatically monitored and summarized.
In an experiment, the investigator changes one
or more variables over the course of the research. When
all other variables are held constant (except the one
being manipulated), changes in the dependent variable can
be explained by the change in the independent variable.
It is usually very difficult to control all the variables
in the environment. Therefore, experiments are generally
restricted to laboratory models where the investigator
has more control over all the variables.
It is incumbent on the researcher to clearly define
the target population. There are no strict rules to
follow, and the researcher must rely on logic and
judgment. The population is defined in keeping with the
objectives of the study.
Sometimes, the entire population will be sufficiently
small, and the researcher can include the entire
population in the study. This type of research is called
a census study because data is gathered on every
member of the population.
Usually, the population is too large for the
researcher to attempt to survey all of its members. A
small, but carefully chosen sample can be used to
represent the population. The sample reflects the
characteristics of the population from which it is drawn.
Sampling methods are classified as either probability
or nonprobability. In probability samples, each
member of the population has a known probability of being
selected. Probability methods include random sampling,
systematic sampling, and stratified sampling. In
nonprobability sampling, members are selected from the
population in some nonrandom manner. These include
convenience sampling, judgment sampling, quota sampling,
and snowball sampling. The other common form of
nonprobability sampling occurs by accident when the
researcher inadvertently introduces nonrandomness into
the sample selection process. The advantage of
probability sampling is that sampling error can be
calculated. Sampling error is the degree to which a
sample might differ from the population. When inferring
to the population, results are reported plus or minus the
sampling error. In nonprobability sampling, the degree to
which the sample differs from the population remains
unknown. (McDaniel and Gates, 1991)
Random sampling is the purest form of
probability sampling. Each member of the population has
an equal chance of being selected. When there are very
large populations, it is often difficult or impossible to
identify every member of the population, so the pool of
available subjects becomes biased. Random sampling is
frequently used to select a specified number of records
from a computer file.
Systematic sampling is often used instead of
random sampling. It is also called an Nth name
selection technique. After the required sample size
has been calculated, every Nth record is selected from a
list of population members. As long as the list does not
contain any hidden order, this sampling method is as good
as the random sampling method. Its only advantage over
the random sampling technique is simplicity.
Stratified sampling is commonly used
probability method that is superior to random sampling
because it reduces sampling error. A stratum is a subset
of the population that share at least one common
characteristic. The researcher first identifies the
relevant stratums and their actual representation in the
population. Random sampling is then used to select
subjects for each stratum until the number of subjects in
that stratum is proportional to its frequency in the
Convenience sampling is used in exploratory
research where the researcher is interested in getting an
inexpensive approximation of the truth. As the name
implies, the sample is selected because they are
convenient. This nonprobability method is often used
during preliminary research efforts to get a gross
estimate of the results, without incurring the cost or
time required to select a random sample.
Judgment sampling is a common nonprobability
method. The researcher selects the sample based on
judgment. This is usually and extension of convenience
sampling. For example, a researcher may decide to draw
the entire sample from one "representative"
city, even though the population includes all cities.
When using this method, the researcher must be confident
that the chosen sample is truly representative of the
Quota sampling is the nonprobability equivalent
of stratified sampling. Like stratified sampling, the
researcher first identifies the stratums and their
proportions as they are represented in the population.
Then convenience or judgment sampling is used to select
the required number of subjects from each stratum. This
differs from stratified sampling, where the stratums are
filled by random sampling.
Snowball sampling is a special nonprobability
method used when the desired sample characteristic is
rare. It may be extremely difficult or cost prohibitive
to locate respondents in these situations. Snowball
sampling relies on referrals from initial subjects to
generate additional subjects. While this technique can
dramatically lower search costs, it comes at the expense
of introducing bias because the technique itself reduces
the likelihood that the sample will represent a good
cross section from the population.
There are very few hard and fast rules to define the
task of data collection. Each research project uses a
data collection technique appropriate to the particular
research methodology. The two primary goals for both
quantitative and qualitative studies are to maximize
response and maximize accuracy.
When using an outside data collection service,
researchers often validate the data collection
process by contacting a percentage of the respondents to
verify that they were actually interviewed. Data editing
and cleaning involves the process of checking for
inadvertent errors in the data. This usually entails
using a computer to check for out-of-bounds data.
Quantitative studies employ deductive logic,
where the researcher starts with a hypothesis, and then
collects data to confirm or refute the hypothesis. Qualitative
studies use inductive logic, where the researcher first
designs a study and then develops a hypothesis or theory
to explain the results of the analysis.
Quantitative analysis is generally fast and
inexpensive. A wide assortment of statistical techniques
are available to the researcher. Computer software is
readily available to provide both basic and advanced
multivariate analysis. The researcher simply follows the
preplanned analysis process, without making subjective
decisions about the data. For this reason, quantitative
studies are usually easier to execute than qualitative
Qualitative studies nearly always involve in-person
interviews, and are therefore very labor intensive and
costly. They rely heavily on a researcher's ability to
exclude personal biases. The interpretation of
qualitative data is often highly subjective, and
different researchers can reach different conclusions
from the same data. However, the goal of qualitative
research is to develop a hypothesis--not to test one.
Qualitative studies have merit in that they provide
broad, general theories that can be examined in future
Modern computer software has made the analysis of
quantitative data a very easy task. It is no longer
incumbent on the researcher to know the formulas needed
to calculate the desired statistics. However, this does
not obviate the need for the researcher to understand the
theoretical and conceptual foundations of the statistical
techniques. Each statistical technique has its own
assumptions and limitations. Considering the ease in
which computers can calculate complex statistical
problems, the danger is that the researcher might be
unaware of the assumptions and limitations in the use and
interpretation of a statistic.
Reporting the Results
The most important consideration in preparing any
research report is the nature of the audience. The
purpose is to communicate information, and therefore, the
report should be prepared specifically for the readers of
the report. Sometimes the format for the report will be
defined for the researcher (e.g., a dissertation), while
other times, the researcher will have complete latitude
regarding the structure of the report. At a minimum, the
report should contain an abstract, problem statement,
methods section, results section, discussion of the
results, and a list of references (Anderson, 1966).
Validity and Reliability
Validity refers to the accuracy or truthfulness
of a measurement. Are we measuring what we think we are?
"Validity itself is a simple concept, but the
determination of the validity of a measure is
elusive" (Spector, 1981, p. 14).
Face validity is based solely on the judgment
of the researcher. Each question is scrutinized and
modified until the researcher is satisfied that it is an
accurate measure of the desired construct. The
determination of face validity is based on the subjective
opinion of the researcher.
Content validity is similar to face validity in
that it relies on the judgment of the researcher.
However, where face validity only evaluates the
individual items on an instrument, content validity goes
further in that it attempts to determine if an instrument
provides adequate coverage of a topic. Expert opinions,
literature searches, and pretest open-ended questions
help to establish content validity.
Criterion-related validity can be either
predictive or concurrent. When a dependent/independent
relationship has been established between two or more
variables, criterion-related validity can be assessed. A
mathematical model is developed to be able to predict the
dependent variable from the independent variable(s). Predictive
validity refers to the ability of an independent
variable (or group of variables) to predict a future
value of the dependent variable. Concurrent validity
is concerned with the relationship between two or more
variables at the same point in time.
Construct validity refers to the theoretical
foundations underlying a particular scale or measurement.
It looks at the underlying theories or constructs that
explain a phenomena. This is also quite subjective and
depends heavily on the understanding, opinions, and
biases of the researcher.
Reliability is synonymous with repeatability. A
measurement that yields consistent results over time is
said to be reliable. When a measurement is prone to
random error, it lacks reliability. The reliability of an
instrument places an upper limit on its validity
(Spector, 1981). A measurement that lacks reliability
will necessarily be invalid. There are three basic
methods to test reliability : test-retest, equivalent
form, and internal consistency.
A test-retest measure of reliability can be
obtained by administering the same instrument to the same
group of people at two different points in time. The
degree to which both administrations are in agreement is
a measure of the reliability of the instrument. This
technique for assessing reliability suffers two possible
drawbacks. First, a person may have changed between the
first and second measurement. Second, the initial
administration of an instrument might in itself induce a
person to answer differently on the second
The second method of determining reliability is called
the equivalent-form technique. The researcher
creates two different instruments designed to measure
identical constructs. The degree of correlation between
the instruments is a measure of equivalent-form
reliability. The difficulty in using this method is that
it may be very difficult (and/or prohibitively expensive)
to create a totally equivalent instrument.
The most popular methods of estimating reliability use
measures of internal consistency. When an
instrument includes a series of questions designed to
examine the same construct, the questions can be
arbitrarily split into two groups. The correlation
between the two subsets of questions is called the split-half
reliability. The problem is that this measure of
reliability changes depending on how the questions are
split. A better statistic, known as Chronbach's alpha
(1951), is based on the mean (absolute value) interitem
correlation for all possible variable pairs. It provides
a conservative estimate of reliability, and generally
represents "the lower bound to the reliability of an
unweighted scale of items" (Carmines and Zeller, p.
45). For dichotomous nominal data, the KR-20
(Kuder-Richardson, 1937) is used instead of Chronbach's
alpha (McDaniel and Gates, 1991).
Variability and Error
Most research is an attempt to understand and explain variability.
When a measurement lacks variability, no statistical
tests can be (or need be) performed. Variability refers
to the dispersion of scores.
Ideally, when a researcher finds differences between
respondents, they are due to true difference on the
variable being measured. However, the combination of
systematic and random errors can dilute the accuracy of a
measurement. Systematic error is introduced
through a constant bias in a measurement. It can usually
be traced to a fault in the sampling procedure or in the
design of a questionnaire. Random error does not
occur in any consistent pattern, and it is not
controllable by the researcher.
Scientific research involves the formulation and
testing of one or more hypotheses. A hypothesis cannot be
proved directly, so a null hypothesis is established to
give the researcher an indirect method of testing a
theory. Sampling is necessary when the population is too
large, or when the researcher is unable to investigate
all members of the target group. Random and systematic
sampling are the best methods because they guarantee that
each member of the population will have an known non-zero
chance of being selected. The mathematical reliability
(repeatability) of a measurement, or group of
measurements, can be calculated, however, validity can
only be implied by the data, and it is not directly
verifiable. Social science research is generally an
attempt to explain or understand the variability in a
group of people.
Anderson, B. (1966) The Psychology Experiment: An
Introduction to the Scientific Method. Belmont, CA:
McDaniel, C. and R. Gates (1991) Contemporary
Marketing Research. St. Paul, MN: West.
Carmines, E., and R. Zeller, (1979) Reliability and
Validity Assessment. Beverly Hills: Sage.
Spector, P. (1981) Research Design. Beverly
Walonick, D. (1993) StatPac Gold IV: Marketing
Research and Survey Edition. Minneapolis, MN: