Preparation of this manuscript was supported by a grant awarded to Susan M. Sheridan and colleagues (IES #R305C090022) by the Institute of Education Sciences. The opinions expressed herein are those of the authors and should not be considered refl ective of the funding agency.
Correspondence concerning this manuscript should be addressed to Natalie Koziol, University of Nebraska-Lincoln,
CYFS, 303 Mabel Lee Hall, Lincoln, NE 68588-0345 (nkoziol@ huskers.unl.edu).
The Journal of Research in Rural Education is published by the Center on Rural Education and Communities, College of Education, The Pennsylvania State University, University Park, PA 16802. ISSN 1551-0670 array of disciplines, and each of these defi nitions has its own strengths and weaknesses. Although the challenges of defi ning rural are well-documented (e.g., Coladarci, 2007;
Cromartie & Buc holtz, 2008; Hart, Larson, & Lishner, 2005; Howley, Theobald, & Howley, 2005; Isserman, 2005),
discussions have primarily occurred at a theoretical level or do not delve into the issues that arise once a defi nition has been chosen. Concrete examples and guidelines are needed to ensure that researchers fully understand the extent to which the rural defi nition impacts the study’s sampling design, analysis plan, and generalizability.
The purpose of this article is twofold. First, we aim to remind (or potentially inform) rural researchers how to
(a) identify an operational defi nition of rural given their theoretical perspective and the context and goals of their
study, (b) appropriately analyze their data given the chosen operational defi nition, and (c) accurately communicate
their fi ndings given the chosen operational defi nition. Upon examining several quantitative articles recently published in JRRE, we were generally encouraged by JRRE authors’ “Rural” is a theoretical construct, so identifying
a theoretical perspective of rural is a critical fi rst step in conducting research on rural education. However, an equally critical (and closely related) step is identifying an operational defi nition of rural, which is necessary for conducting quantitative rural research. Numerous theoretical and operational defi nitions have been proposed across a wide Defi ning rural is a critical task for rural education researchers, as it has implications for all phases of a study. However, it is also a diffi cult task due to the many ways in which rural can be theoretically, conceptually, and empirically operationalized.
This article provides researchers with specifi c guidance on important theoretical and operational considerations relevant to conducting quantitative rural education research: identifying a rural defi nition, selecting appropriate analytic methods, and thoroughly communicating rural details to situate the fi ndings within the broader rural literature base. In addition, this article uses the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K) and three rural defi nitions to illustrate how parameter estimates and substantive interpretations are impacted by the statistical model, rural defi nition, and exclusion/inclusion of covariates. We believe that informed consideration and implementation of the article’s guidelines will enhance and clarify the quantitative literature on rural education.
Citation: Koziol, N. A., Arthur, A. M., Hawley, L. R., Bovaird, J. A., Bash, K. L., McCormick, C., & Welch, G. W. (2015). Identifying, analyzing, and communicating rural: A quantitative perspective.
Journal of Research in Rural Education, 30(4), 1-14.
Natalie A. Koziol
Ann M. Arthur
Leslie R. Hawley
James A. Bovaird
Kirstie L. Bash
Carina McCormick
Greg W. Welch
University of Nebraska-Lincoln
National Center for Research on Rural Education (R2Ed)
The Nebraska Center for Research on Children, Youth, Families & Schools (CYFS)
Journal of Research in Rural Education, 2015, 30(4)
Identifying, Analyzing, and Communicating Rural:
A Quantitative Perspective
2 KOZIOL, ARTHUR, HAWLEY, BOVAIRD, BASH, MCCORMICK & WELCH
demographic characteristics such as population density and size, and spatial delimiters. Demographic and population density-based defi nitions of rural largely date back to Emile Durkheim’s 1893 work, The Division of Labor in Society. Durkheim differentiated societies by the nature of their solidarity, which he theorized to be a direct function of population density (Durkheim, 1964). According to Durkheim, low population density societies (i.e., rural
societies) lend themselves to a mechanical form of solidarity characterized by collectivist orientations, homogeneous
backgrounds and belief systems, and agrarian lifestyles.
In contrast, high population density societies (i.e., urban societies) increasingly demonstrate an organic form of
solidarity characterized by an interdependency among others that stems from the division of labor. Along with population characteristics such as size and density, rural places are often spatially defi ned. Spatial conceptualizations of rural focus on the where (i.e., space, distance, and relationship to the city) of places (Lobao, Hooks, & Tickamyer, 2007) and highlight issues of spatial inequality or spatial disparities in the allocation of resources.
Most operational defi nitions of rural used in quantitative research are grounded in the demographic and spatially
based theories, as these theories provide a straightforward means for classifying geographies. However, just because a theory readily enables classifi cation (which in turn enables quantifi cation) does not mean it is superior. Other placebased theories that emphasize political-economic and sociocultural distinctions have also garnered attention. With respect to these alternative theories, Brown and Schafft (2011) note that although “the focus is not on the space
(or place) itself … locales often create a powerful context for collective identity and social interaction” (p. 39) such
that demographic and spatially based classifi cations can sometimes serve as proxies for getting at these alternative
(and generally harder to operationalize) ways of thinking about rural.
As the name suggests, political-economic theories of rural focus on distinctions among sectors that are primarily
politically and economically driven (Cloke, 2006). Such distinctions include the tendency for rural economies to
be more specialized (Deavers, 1992) and more dependent on the government sector (Deavers & Brown, 1985).
Tilly (1974) attributes this dependence to state-making, urbanization, industrialization, and commercialization.
Much of the literature on class relations (e.g., Stinchcombe, 1961), including peasant studies (e.g., Wolf, 1969), can
also be considered here. While conceptually distinct, many of these issues relate, at least indirectly, to Durkheim’s
(1964) theorizing on rural, suggesting that demographic characteristics could be used in some instances to
approximate political economic classifi cations.
Socio-cultural theories frame rural in terms of formal careful attention to defi ning and discussing rural at both the
theoretical and operational levels. Our goal is to explicate and thus foster and sustain this good practice by providing
a comprehensive guide for education researchers seeking to identify, analyze, and communicate rural phenomena.
Second, we use data from the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K; developed by the U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics [NCES], n.d.-a) to illustrate the impact of the rural defi nition on statistical results and substantive inferences.
It is important to acknowledge that our discussion is primarily intended for researchers conducting quantitative
research. Some of our recommendations may not apply to, or may even counter what is recommended for, qualitative research. We believe that both forms of research, in addition to mixed-methods research, are important for understanding rural issues. Thus, we strongly encourage other researchers to advise the fi eld on rural defi nition issues in the context of qualitative and mixed-methods research.
Identifying, Analyzing, and Communicating Rural Identifying an Appropriate Defi nition
Choosing a rural defi nition infl uences the entire scope
of a study. At the initial planning stages of a study, the
defi nition affects the selection of a sampling design and
statistical analysis plan. At the concluding stages of a study,
the defi nition affects the generalizability of the research
fi ndings. In this section we provide guidance on identifying
an appropriate operational defi nition of rural given a
particular theoretical perspective, while taking into account
practical considerations.
Theoretical perspectives. First and foremost,
operationalizing rural requires formulating a theoretical
perspective of rural. Numerous theories of rural have
been postulated. In following the organization scheme of
Brown and Schafft (2011), such theories can be broadly
classifi ed into one of two groups: place-based theories (e.g.,
demographic, population, spatial, political economic, and
socio-cultural theories) and social constructivist theories.1
We briefl y touch on each of these theories below. A full
discussion is beyond the scope of this article, so we strongly
encourage researchers to consult the original sources and
seek out additional references for a deeper and more
comprehensive understanding of rural theory.
One means for conceptualizing rural is in terms of
“population and settlement structure and landscape”
(Brown & Schafft, 2011, p. 5), where emphasis is placed on
1We are particularly grateful for the feedback of an
anonymous reviewer who outlined and summarized the
theories discussed in this section.
IDENTIFYING, ANALYZING, AND COMMUNICATING RURAL 3
has created an Urban and Rural Classifi cation. Finally,
NCES has developed the Urban-Centric Locale Codes
(NCES, n.d.-b), a revised version of the previously used
Metro-Centric Locale Codes (or simply, the Locale Codes;
NCES, n.d.-c). Detailed descriptions and comparisons of
the defi nitions’ strengths and weaknesses are provided by
Arnold, Biscoe, Farmer, Robertson, and Shapley (2007);
Coburn et al. (2007); Hart (2012); and Hart et al. (2005).
Given the prominence and usage of these ways of
classifying geographies, we strongly encourage researchers
to consider the defi nitions seriously. At the same time, we
recognize that the defi nitions have limitations and may not
be appropriate for all study contexts. As an alternative, a
number of researchers have created their own rural defi nition
by modifying or combining one or more of the existing
defi nitions. For instance, Isserman (2005) introduced a
Rural-Urban Density Typology that combines elements of
the U.S. Census Bureau and OMB defi nitions—including
county population density, percentage of county population
in urban/rural areas, and presence/absence of urban areas
of 10,000+ or 50,000+—to acknowledge the presence
of “mixture” counties that contain both rural and urban
areas. As another example, Waldorf (2006) proposed the
Index of Relative Rurality, a continuous measure of rural
comprised of population size and density, extent of urban
area, and remoteness. A primary advantage of the index is
that it avoids the need to impose arbitrary thresholds such as
10,000+ and 50,000+.
Primary considerations. In creating a new defi nition
or choosing among well-established defi nitions, researchers
must ultimately bear in mind their theoretical perspective of
rural. Different defi nitions of rural use different operational
indicators; the most relevant indicators are those that match
the researcher’s theoretical perspective. For example, the
U.S. Census Bureau (2013a) uses a defi nition of urban
and rural that is based primarily on population size and
density. This defi nition clearly links to demographic-based
theories of rural. A moderately strong relationship has been
identifi ed between population density and human capital
investment (Garces-Voisenat, 2011), suggesting that in
education research population density might be important
for explaining job growth for teachers or other school
offi cials. The OMB and RUCA classifi cation systems
additionally take commuting patterns into account (OMB,
2010; WWAMI RHRC, n.d.-b) and thus relate to spatially
based theories of rural. Commuting patterns might be
particularly relevant to education researchers who want
to understand the impact of access to big-city resources,
such as art museums or institutions of higher education, on
child outcomes. Similarly, the Urban-Centric Locale Codes
stem from spatial conceptualizations of rural, as they rely
on geographic information systems (GIS) technology to
classify locations (NCES, n.d.-d). Researchers might use
and informal social and cultural networks (e.g., Schafft,
2000) and interactive community fi elds (e.g., Kaufman,
1959; Wilkinson, 1972). Schafft and Brown (2000) note that
“embedded intra-community relations, including individual
and group-level social ties, cultural practices, and political
behavior, reinforce the affi liative networks within a given
locality” (p. 204). Strong ties among members within a
community promote social cohesion, while ties (albeit
weaker) with wider networks (e.g., ties between rural
communities and neighboring cities) ensure suffi ciency of
resources and prevent isolation (Schafft, 2000). Although
these networks are socially defi ned, spatial infl uences are
clearly evident. With respect to interactional communities,
Kaufman (1959) explains that “at the cultural level,
integration is effected through the widely shared values
and objectives pertaining to the community fi eld and, at the
ecological level, through a ‘functional relation’ of services”
(p. 12). Discussions of social norms (e.g., reciprocity
and civic engagement) as they relate to social capital
(e.g., Putnam, 1993) also contribute to social-cultural
conceptualizations of rural.
The issues discussed in the remainder of this article
relate most directly to place-based theories of rural,
particularly the demographic and spatially based theories
of rural. However, we would be remiss not to mention
social constructivist approaches to conceptualizing rural.
In particular, Halfacree (1993, 2006) emphasizes the
importance of considering non-tangible indicators of space
such as cognitive structures and social representations.
Place identity construction, specifi cally the notion of urban
and rural identities (Bell, 1992; Creed & Ching, 1997), falls
under this approach. Bell (1992) observes that, although
“the difference between country life and city life may only
ever be true in the mind” (p. 66), perceived differences have
real social consequences.
Operational defi nitions. To conduct quantitative
research, one’s theoretical perspective of rural must be
translated into operational terms. A reasonable fi rst step
in searching for an appropriate operational defi nition is
to examine the defi nitions that are currently in use. Most
well-established and commonly used defi nitions have been
developed by one of four federal agencies/centers. The U.S.
Department of Agriculture, Economic Research Service
(ERS) has developed the Rural-Urban Continuum Codes
(RUCCs, also referred to as the Beale Codes; ERS, 2013a),
Urban Infl uence Codes (UICs; ERS, 2013b), and Rural-
Urban Commuting Areas (RUCAs; ERS, 2013c), developed
jointly with the Washington, Wyoming, Alaska, Montana,
and Idaho Rural Health Research Center (WWAMI RHRC,
n.d.-a). The U.S. Offi ce of Management and Budget (OMB)
has defi ned Core Based Statistical Areas (CBSAs), which
include Metropolitan and Micropolitan Statistical Areas
(OMB, 2010). In addition, the U.S. Census Bureau (2013a)
4 KOZIOL, ARTHUR, HAWLEY, BOVAIRD, BASH, MCCORMICK & WELCH
level are likely preferred over rural defi nitions applied at
the school level.
The choice of geographic unit strongly infl uences
conclusions about rural phenomena, a consequence that has
been defi ned in the literature as the modifi able areal unit
problem (MAUP). Waller and Gotway (2004) describe
MAUP as the “geographic manifestation of the ecological
fallacy in which conclusions based on data aggregated to
a particular set of districts may change if one aggregates
the same underlying data to a different set of districts” (p.
104). MAUP involves both a scale/aggregation problem
and a grouping/zoning problem. The scale/aggregation
problem relates to the fact that statistical results will vary
as a function of the level of aggregation applied. As Arnold
et al. (2007) note, greater aggregation generally results in a
greater loss of information—e.g., many counties classifi ed as
metropolitan contain signifi cant rural spaces and vice versa
(Isserman, 2005). The grouping/zoning problem relates to
the fact that statistical results will vary depending on how
groups are formed, given a particular level of aggregation.
For example, census tract boundaries often change across
census years, resulting in different groupings across census
years (Hart et al., 2005). While there is no defi nitive solution
to MAUP, researchers should be aware of the implications
of choosing a particular geographic unit, and shape their
inferences about rural phenomena accordingly.
Supplemental considerations. Sometimes logistical
concerns make a theoretically ideal rural operationalization
practically infeasible. One such concern is fi nancial
constraints. Consider a defi nition that is applied to counties.
The power to detect a rural effect is primarily infl uenced
by the number of counties sampled. Sampling counties is
likely to be much more expensive than sampling schools,
which tend to exist in greater numbers and in narrower
geographical regions. Researchers may not have the time
and resources to sample at the county level. Likewise,
choosing one of the broader defi nitions of rural (e.g., the
U.S. Census Bureau’s defi nition) will result in more places
qualifying as in-need based on policy or program eligibility
requirements, but programs may be limited in their funding
(Coburn et al., 2007).
Another concern is the availability of data. Coburn
et al. (2007) offer a health policy example in which they
note that provider claims are based on ZIP codes. Without
additional information, rural defi nitions applied to counties
cannot be used to address questions about provider claims,
because counties and ZIP codes are not directly comparable.
ZIP codes were created by the U.S. Postal Service as mail
delivery areas and are not bound to counties or even states
(“Can ZIP codes cross,” 2011) and can change monthly
(“How many changes are made,” 2011). Although ZIP
codes may be loosely aggregated with GIS techniques
into approximate counties (DuScheid, 2011), this practice
this defi nition to study the impact of geographic remoteness
on schools’ reliance on distance technology.
Consistent with the guidelines of Hart et al. (2005), the
indicators described above are quantifi able and relatively
objective. However, as noted by Coladarci (2007),
some researchers question the meaningfulness of such
“traditional constructs of demography” and argue for “more
important notions of ‘local commitments’ and ‘meaningmaking’”
(p. 2). In particular, none of the aforementioned
defi nitions directly captures the tenets of the politicaleconomic,
socio-cultural, and social constructivist theories.
If researchers deem demographic and spatial characteristics
to be insuffi cient proxies for operationalizing rural under
these alternative theories, then other indicators should be
explored. In doing so, however, researchers must be willing
and able to provide convincing evidence that the potentially
subjective indicators do in fact measure what they are
purported to measure.
Related to choosing a rural indicator(s) is determining the
geographic unit to which the indicator(s) should be applied.
This consideration is important because the geographic
unit represents the experimental unit—the smallest unit to
which the “treatment” is independently applied (Milliken &
Johnson, 2009)—for the rural/urban predictor. As discussed
in the “Analyzing Rural Data” section, inferences about
rural exist at the level of the experimental unit (e.g., the
county), which is not always the same as the lowest-level
sampling unit (i.e., the lowest-level unit for which data are
being collected, for example, the student).
Common geographic units include schools, school
districts, ZIP code areas, census tracts, and counties.
Obviously, researchers should choose a geographic unit that
matches their target sampling unit. For example, the schoollevel
defi nitions provided by the Metro- and Urban-Centric
Locale Codes might be particularly useful for comparing an
intervention’s effi cacy in rural versus urban schools. On the
other hand, county-level defi nitions provided by the RUCCs
and UICs would be more appropriate for evaluating rural
and urban county health and wellness initiatives. Countylevel
defi nitions may also be advantageous in the context
of multi-year longitudinal studies, as counties tend to be
more stable over time compared to other geographical
units (Hart et al., 2005). When the ideal geographic unit
is not immediately apparent, researchers may benefi t from
choosing the geographic unit that has the most variation on
the outcome variable. For instance, suppose “professional
development opportunities” is the outcome variable. If most
of the variability in professional development opportunities
exists between school districts as opposed to within school
districts, then school-district-level predictors will generally
account for more variability in professional development
opportunities than, say, school-level predictors. Hence, in
this context, rural defi nitions applied at the school-district
5
The sampling design involves two levels where level 1 units
(the lowest-level sampling units; = 200 children) are nested
within level 2 units (the experimental units for the rural/
urban predictor; = 20 schools). At fi rst glance, it might seem
that an independent samples t-test comparing the average
BMI of the 100 children in rural schools to the average
BMI of the 100 children in urban schools is suffi cient for
testing the “rural” effect on BMI. However, the assumption
of independent error terms is violated. Children who go to
the same school will tend to be more similar than children
who go to different schools, so in actuality, there are 10
independent observations per group instead of 100. Only
considering data at the lowest level of sampling is referred to
as disaggregation (Snijders & Bosker, 2012). In the simplest
case, when predictor variables are assigned at a higher level
of sampling, disaggregation increases the chance of a Type
I error (e.g., fi nding urban and rural differences when there
are no differences). In contrast, when nesting is present (and
ignored) but predictor variables are measured at the lowest
level of sampling, disaggregation increases the chance of a
Type II error (e.g., failing to fi nd urban and rural differences
when there are in fact differences).
An alternative is to aggregate the lower-level data to
the higher level. This approach would involve computing
the average BMI for each school and then performing
an independent samples t-test to compare the average
BMI of the 10 rural schools to the average BMI of the 10
urban schools. Like disaggregation, aggregation has its
disadvantages. Aggregation throws away within-cluster
(i.e., within-school) variability, which prevents researchers
from fully understanding all of the variability in the outcome.
For instance, research has shown that the prevalence of
obesity among children is greater for low-income families
(Ogden, Lamb, Carroll, & Flegal, 2010), suggesting that
socioeconomic status (SES) is an important predictor to
include in any analysis of BMI. In the case of an aggregated
analysis of BMI, the only means for including SES as a
predictor would be to aggregate family-level SES to the
school level and then regress average BMI on average SES
and rural status. Importantly, family-level SES and schoollevel
SES are distinct predictors with potentially distinct
effects on BMI. The effect of school-level SES on BMI may
be weaker, stronger, or even in the opposite direction2 of
the effect of family-level SES. Drawing inferences about
lower-level relationships based on higher-level relationships
is referred to as an ecological fallacy (Snijders & Bosker,
2012). In an attempt to avoid such a fallacy a researcher
might perform two separate regressions, one at the child
level and one at the school level. This approach is not ideal.
First, all else being equal, the estimators of the regression
2For instance, we might fi nd this case if a county
initiative aggressively targeted low SES schools for healthy
lunch programs.
introduces additional error as it does not always provide an
exact match. In the context of a secondary data analysis,
researchers are limited to the data at hand. If the dataset
does not provide NCES school IDs or census tract or county
codes (either the American National Standards Institute
[ANSI] codes or the older Federal Information Processing
Series [FIPS] codes; U.S. Census Bureau, 2013b), then the
corresponding defi nition codes cannot be merged with the
dataset.
Finally, a dataset may include all the necessary
identifi cation and coding information, but whether it is
appropriate to make comparisons based on higher-level
geographic units depends on the sampling design of the
original study. For instance, although the restricted-license
ECLS-K dataset provides information on higher-level units,
the study’s website states that “the ECLS-K sample was not
designed to support state-level (or city- or county-level)
estimates, as the sample is not necessarily representative of
children in particular states (or cities or counties)” (NCES,
n.d.-e).
To conclude this section, we stress the hierarchy of
theoretical and practical considerations. Ultimately, theory
should be the driving force in selecting an operational
defi nition of rural. Practical considerations, while important,
should serve more as an evaluation of the feasibility of the
chosen rural defi nition, and potentially as an indication of
the need for additional or alternative study resources.
Analyzing Rural Data
Once a rural defi nition is chosen, the next step is to
determine an appropriate analytic strategy. The analysis
plan follows directly from the sampling design, which in
large part depends on the geographic unit to which the rural
defi nition is applied. In education and rural research, simple
random samples are generally impractical and ineffi cient.
Instead, researchers often use complex sampling procedures
that involve clustering (also referred to as nesting). Clustering
is a sampling method in which the units being sampled
contain multiple related observation units (Kish, 1965).
For instance, rather than directly sampling students, it may
be more convenient to randomly sample schools and then
observe a selection of students within each school. Whereas
clustering is often more time and cost effi cient than simple
random sampling (Kish, 1965), a disadvantage of clustering
is that the assumption of independent observations, implicit
to most basic statistical models and tests, is violated.
Consider the following scenario. Suppose a researcher
is interested in comparing the body mass index (BMI) of
rural and urban children, where the terms rural and urban are
defi ned at the school level. The researcher randomly selects
10 rural schools and 10 urban schools, and within each
school, measures the BMI of 10 randomly selected children.
IDENTIFYING, ANALYZING, AND COMMUNICATING RURAL
6 KOZIOL, ARTHUR, HAWLEY, BOVAIRD, BASH, MCCORMICK & WELCH
In Equation 3, yij represents the BMI of the ith child who
attends the jth school, γ00 is the grand mean, uoj is the
difference between the jth school’s mean and the grand
mean, and eij is the difference between the ith child’s BMI
and the jth school’s mean. The only difference between a
traditional regression model and Equation 3 is the presence
of the additional school-level error term (uoj). Figure 1
provides a small illustration of each of the terms in Equation
3 based on hypothetical data for two schools (j=1,2), each
with two children (i=1,2). We see that the mean BMI for
School 1 (y·1=25) is lower than the grand mean (γ00=26),
so u01 is negative. In contrast, Child 1 from School 1 has
a higher BMI (y11=25.54) than the mean BMI for his or her
school, so e11 is positive. If we did not include the schoollevel
error terms, the residual would instead represent the
distance between the child’s BMI and the grand mean,
which would result in correlated error terms due to the
fact that children from the same school tend to have more
similar BMIs than children from different schools (in our
hypothetical example).
Earlier we noted that MLM serves to partition variance
in the outcome into its between- and within-level sources.
The proportion of variability that exists at the higher level
can be estimated via an intraclass correlation coeffi cient
(ICC)
(4)
where
is the between-level (e.g., between-school)
variance in the outcome, and
is the within-level (e.g.,
within-school or between-child) variance in the outcome.
The larger the ICC, the greater the proportion of variability
coeffi cients will be less statistically effi cient (resulting in
greater standard errors) when the analyses are conducted
separately. Second, separate analyses preclude the
possibility of cross-level interactions (e.g., the interaction
between child-level SES and school-level rural status). This
limitation is serious. Rural education research is complex,
and examining interactions is one way to acknowledge the
complexity (Howley et al., 2005).
Aggregation and disaggregation leave much to be
desired. An alternative approach to handling clustering
is multilevel modeling (MLM),3 also referred to as linear
mixed modeling, hierarchical linear modeling (not to be
confused with hierarchical multiple regression), and random
coeffi cients modeling (Raudenbush & Bryk, 2002). MLM
avoids the limitations of aggregation and disaggregation by
partitioning variance in the outcome into its multiple sources
and then modeling these sources simultaneously. It is useful
(albeit an oversimplifi cation) to conceptualize MLMs
simply as regression models with more than one error term.
As an example, the unconditional (i.e., no predictor) model
for the two-level BMI scenario can be expressed in equation
form as
Level 1: yij = βoj + eij (1)
Level 2: βoj = yoo + uoj (2)
which can be simplifi ed by replacing the β0j placeholder in
Equation 1 with Equation 2:
Combined: yij = γoo + uoj + eij · (3)
3Another option for handling clustered data is to
calculate empirical standard errors that adjust for the
complex sampling design (e.g., Wolter, 2007).
Figure 1. An illustration of the regression terms presented in Equation 3.
0 2
24.46
25.00
25.54
26.00
26.17
27.00
27.83
e22
e12
e11
e21
y12
y11
y21
00
u02
u01
y22
y1
y2
=
+
IDENTIFYING, ANALYZING, AND COMMUNICATING RURAL 7
effect is only based on 18 degrees of freedom resulting in
a larger standard error. In examining the estimated variance
components for Model 3, it is not a coincidence that the
within-level variance is the exact same as the withinlevel
variance estimated for Model 1, but the betweenlevel
variance is reduced (again, this result may not hold
perfectly in more complex data situations). Rural status is
the same for all children within a particular school so it
cannot explain within-school variance.
A key consideration when analyzing rural data is
whether covariates should be included in the model. As
Coladarci (2007) emphasizes, “without adequate controls
in place, the obtained [rural/urban] differences may be
either unwittingly exaggerated or understated (although
exaggeration is more likely)” (p. 4). Earlier in this section
we mentioned the relationship between SES and BMI. Table
1 indicates that, upon controlling for child- and schoollevel
SES (Model 4), we would conclude that school rural
status is not a signifi cant predictor of BMI (p = .827). What
appeared to be a rural phenomenon based on Models 2 and
3 was actually explained by SES.
Our discussion and examples of MLM focused on only
one possible source of dependency among observations,
dependency due to nesting or clustering of lower-level
units within higher-level units. Dependency can occur for
a multitude of other reasons. Particularly relevant to rural
education research is the possibility of geographic, or
spatial, dependency (see Kučerová & Kučera, 2012, for a
that exists at the higher level. By calculating the proportion
of variability in the outcome that exists at each level we can
determine which types of predictors will be most helpful
in explaining the outcome. If very little variability in BMI
exists at the school level then a rural defi nition applied at
the school level will probably not tell us much about BMI.
We conclude this discussion of MLM with a
demonstration to illustrate the consequences of ignoring
clustering and omitting important covariates. Our review of
the quantitative articles published in JRRE between 2009
and 2013 revealed only one study (Stockard, 2011) that used
MLM.4 This fi nding by no means indicates that the studies
that did not use MLM should have used MLM (MLM is only
applicable under certain conditions), but it suggests that a
demonstration might be worthwhile. The interested reader
should see Durham and Smith (2006), Reeves and Bylund
(2005), and Roscigno and Crowley (2001) for additional
examples of MLM in rural education research. Readers who
are new to MLM and interested in learning more about the
topic should see one of the many comprehensive textbooks
on MLM methods (e.g., Raudenbush & Bryk, 2002; Singer
& Willett, 2003; Snijders & Bosker, 2012).
Drawing on our hypothetical example, we simulated
BMI data for 200 children from a total of 20 schools (10
“rural” and 10 “urban”). To use MLM, the data must be in
stacked (also referred to as person-period or long) format—
that is, each row should correspond to the lowest-level unit
(i.e., the child, in our example). The datafi le and model
syntax (both SAS and SPSS syntax) are available from the
fi rst author upon request.
As a fi rst step we calculated the ICC by analyzing the
unconditional two-level model (Model 1) corresponding to
Equation 3. Table 1 provides partial results. Approximately
(2.77/[2.01 + 2.77]) * 100% = 58% of the variability in
BMI is at the between-school level. This bodes well for our
school-level rural predictor.
As an example of disaggregation, we evaluated the
effect of rural status on BMI using a single-level model that
ignored the existence of school-level variance (Model 2).
Based on 198 degrees of freedom, we would determine that
rural status has a signifi cant effect on BMI (p < .001). Of
course, this conclusion is untrustworthy because the model’s
assumption of independent error terms is violated. Next we
evaluated Model 3, a multilevel model that accounted for
the nesting of children within schools. Table 1 shows that
the estimated beta coeffi cient representing the effect of
school rural status is the same for Models 2 and 3 (note
that this result may not hold perfectly in more complex
data situations). However, for Model 3, the test of the rural
4Irvin, Farmer, Leung, Thompson, and Hutchins (2010)
discuss MLM but do not actually employ the method.
Table 1
Single-Level and Multilevel Model Estimates for Simulated
BMI Data
SE dfa p
Model 1 2.01 2.77 — — — —
Model 2 3.20 — 2.42 0.25 198 < .001
Model 3 2.01 1.31 2.42 0.55 18 < .001
Model 4 0.90 0.90 0.24 1.06 16.98 .827
Note. aDenominator degrees of freedom were estimated using a
Satterthwaite approximation.
= within-cluster variance
(residual variance).
= between-cluster variance. = effect
of rural on BMI. Model 1 = unconditional two-level model.
Model 2 = single-level model with rural predictor. Model 3 =
two-level model with rural predictor. Model 4 = Model 3 +
child-level and school-level SES predictors.
8 KOZIOL, ARTHUR, HAWLEY, BOVAIRD, BASH, MCCORMICK & WELCH
a particular fi nding holds across different operational
defi nitions is noteworthy (e.g., by exploring alternative
ways of combining the Beale Codes and observing no
signifi cant impact on the results, Jordan, Kostandini, and
Mykerezi [2012] were able to have more confi dence in the
robustness of their results).
As with any method section, researchers should provide
readers with enough detail to replicate the study. Limiting
the description to generic terms such as “rural” and “urban”
does not facilitate replication. At a minimum, it is necessary
to specify the indicators used in defi ning rural and the
unit to which the defi nition was applied. Along with this
description, researchers should provide a strong rationale
for why they chose the defi nition. A nice example of such a
rationale is given by Jordan et al. (2012):
Beale Codes were used here because they were
designed specifi cally to examine the continuum
between urban and rural areas. They were
developed for the analysis of trends in non-metro
areas that are related to the population density
and metropolitan infl uence. Beale Codes allow a
more detailed analysis of the survey data than the
more common urban-suburban-rural classifi cation
systems (p. 4).
Not only does this explanation inform the reader, it forces
the researchers to carefully consider whether the defi nition
is indeed appropriate given their theoretical perspective and
the context of their study.
Appropriate communication concludes with a
discussion section that is framed by the researcher’s
theoretical perspective of rural but couched in terms of the
operational defi nition that was applied. Again, operational
defi nitions are merely proxies, so researchers should avoid
making unjustifi ed generalizations. Hannum, Irvin, Banks,
and Farmer (2009) provide a great example of limiting their
conclusions to the population from which they sampled.
They emphasized that “the results are generalizable to rural
schools meeting the defi nitions in the REAP program. The
results may or may not be the same in urban, suburban, or rural
schools other than those identifi ed in the REAP defi nition”
(p. 13). As recognized by Hannum and colleagues, it is
important to limit generalizations to the geographic unit to
which the defi nition was applied. For example, if the rural
defi nition was applied at the county level then the effect
of rural should be discussed at the county level. A higherlevel
predictor cannot explain lower-level variation in the
outcome. That is, a county-level rural predictor cannot
explain why individuals within counties perform differently
on the outcome of interest; rather, it can only explain why
counties perform differently on the outcome of interest.
Likewise, it is more revealing and precise to relate fi ndings
discussion of “the geographical aspects of education,” p.
3). Goodchild (1992) defi nes spatial dependence as “the
propensity for nearby locations to infl uence each other
and to possess similar attributes” (p. 33). For instance, it
certainly seems plausible that counties in the Northeastern
region of the United States are more similar to one another
than to counties in the Southeastern region of the United
States. A number of methods have been developed for
analyzing spatially dependent data (see, for example, the
two-level spatial model described by Verbitsky-Savitz &
Raudenbush, 2009). In fact, an entire branch of statistics
is devoted to the study and analysis of spatial dependency.
While an in-depth discussion of spatial methods exceeds
the scope of this article, interested readers should seek
out additional information (e.g., Cressie, 1993; Gelfand,
Diggle, Fuentes, & Guttorp, 2010).
The purpose of this section was to remind readers about
some of the critical issues that arise when analyzing rural
education data. Although educational outcomes are most
often measured at the level of individual children, teachers,
principals, etc., rural defi nitions are most often applied at
higher levels such as census tracts or counties. Dependency
among observations must be taken into account when
considering the effect of rural on the outcome. In addition,
alternative explanations of rural fi ndings should be evaluated
through the inclusion of covariates.
Communicating Rural Findings
A study is not complete until results have been
disseminated. Providing transparent and detailed accounts
should be a goal of all researchers, but it is perhaps even
more critical for rural education researchers given that
many theoretical perspectives and operational defi nitions of
rural exist. Using the nondescript label of rural is to commit
a nominal fallacy; rural alone does not actually explain what
is being measured.
Appropriate communication of rural fi ndings starts
with the literature review. In describing previous rural
studies, it is important to note how researchers defi ned (or
did not defi ne) rural. For example, in reviewing the fi ndings
of Farmer et al. (2006), Irvin, Farmer, Leung, Thompson,
and Hutchins (2010) condition their discussion based on
the operational defi nition that was used: “African American
youth attending schools identifi ed as Rural Low Income by
the Rural Education Achievement Program are four times
less likely to meet Adequate Yearly Progress” (p. 2). The
goal is to appropriately situate the present study within
the broader rural research context rather than limit the
discussion to studies that use a similar defi nition. In fact, the
presence of different operational defi nitions adds richness
to the fi eld. Operational defi nitions are merely proxies for
theoretical defi nitions of rural, so determining whether
IDENTIFYING, ANALYZING, AND COMMUNICATING RURAL 9
Metro-Centric Locale Codes additionally distinguish among
types of incorporated places (e.g., central city vs. town) and
census-designated places. The OMB and RUCA defi nitions
additionally take commuting patterns into account. The
RUCA defi nition goes further by distinguishing areas based
on their primary and secondary fl ows.6 In their full form,
the Metro-Centric Locale Codes and RUCAs provide a
much fi ner-grained measure of rurality than the OMB
classifi cation. However, for practical reasons, researchers
often combine the codes into considerably fewer categories.
Consequently, differences among defi nitions become less
pronounced. For this illustration we used a two-category
classifi cation scheme (urban vs. rural) that was obtained
from the Rural and Low-Income School Program for the
Metro-Centric Locale Codes (codes 1-5 were designated
as urban; all other codes were designated as rural; U.S.
Department of Education, Offi ce of Communications and
Outreach, 2012), the WWAMI RHRC website for the
RUCAs (codes 1.0, 1.1, 2.0,2.1, 3.0, 4.1, 5.1, 7.1, 8.1,
and 10.1 were designated as urban; n.d.-c), and based on
common practice by many federal programs for the OMB
classifi cation (metropolitan counties were designated as
urban; Coburn et al., 2007). Note that the two-category
classifi cation scheme for the RUCAs is an approximation
of the two-category classifi cation scheme for the OMB
defi nition, but at the census tract level (WWAMI RHRC,
n.d.-c).
Analyses were performed in Mplus Version 6.1
(Muthén & Muthén, 1998-2010). Syntax is available from
the fi rst author upon request. We fi rst analyzed a set of
single-level models that ignored the dependency among
observations. We then estimated a set of multilevel models
that accounted for the nesting of children within geographic
units (where the geographic unit varied by defi nition).
Finally, we estimated the same multilevel models but
additionally controlled for child- and location-level SES.
Analyses were limited to cases with complete data on all the
variables of interest, which resulted in an effective sample
size of 12,270 children, 2,440 schools, 2,240 census tracts,
and 290 counties.7
Results
Approximately 14% of sample schools were classifi ed
as rural based on the Metro-Centric Locale Codes, 11%
of sample census tracts were classifi ed as rural based on
the RUCAs, and 19% of sample counties were classifi ed
as rural based on the OMB defi nition. Although there was
6See the respective sources for a more complete
description of each defi nition’s codes.
7Sample size numbers have been rounded to the nearest
10 per Institute of Education Sciences’ restricted-use data
reporting guidelines.
to the observable indicators used to operationally defi ne
rural (e.g., “remoteness alone did not compromise access
to technology” [Howley, Wood, & Hough, 2011, p. 6])
rather than use the elusive terms “urban” and “rural.” This
practice is critical for public policy and the development of
educational programs that rely on tangible indicators.
As we demonstrate in the next section, different
operational defi nitions of rural can lead to very different
results. This consequence does not mean that certain
defi nitions are inherently wrong, but it does mean that
choosing a defi nition requires careful thought, and the
interpretation and discussion of results should be intimately
tied to the chosen defi nition.
An Illustration
In this section we provide an empirical example using
the ECLS-K to demonstrate the impact of the rural defi nition
on parameter estimates and substantive interpretations. The
ECLS-K is a longitudinal study that followed a nationally
representative sample of children from kindergarten
entry (1998-1999 school year) through eighth grade. The
restricted-use license provides access to the census tract
and county FIPS codes for each participating child’s school,
which allowed us to merge existing rural defi nition codes
with the dataset.
It is important to stress that this example is provided
solely for demonstration purposes. The results from our
analyses should not be interpreted in any real manner. We
did not apply sampling weights or adjust for additional
layers of design complexity (e.g., stratifi cation, additional
levels of clustering) as is normally required when analyzing
data from the ECLS-K. Our goal is simply to highlight
differences that may result when choosing among different
modeling approaches and different rural defi nitions.
Method
Using data from the spring third grade wave, we
compared students’ science scores across urban and rural
locations as defi ned by the school-level Metro-Centric
Locale Codes5 (drawn from the 1999-2000 Private School
Universe Survey [PSS] and 2000-2001 Common Core of
Data [CCD]; NCES, 2003), census-tract-level RUCAs
(Version 2.0, based on data from the 2000 U.S. Census;
WWAMI RHRC, n.d.-b), and 2003 county-level OMB
designations (based on the 2000 OMB Standards; OMB,
2000). As a whole, these defi nitions map on most directly
to the demographic- and spatially-based theories of rural.
All three defi nitions are derived directly or indirectly from
population size and density, and relation to a CBSA. The
5The ECLS-K dataset does not include the newer
Urban-Centric Locale Codes.
10 KOZIOL, ARTHUR, HAWLEY, BOVAIRD, BASH, MCCORMICK & WELCH
and rural census tracts based on the RUCAs (B = -0.03,
SE = 0.02, p = .150). The results under “Multilevel, SES
Omitted” correspond to the second set of analyses. Urban
counties signifi cantly outperformed rural counties based
on the OMB classifi cation (B = -0.13, SE = 0.06, p =
.031). In contrast, rural schools signifi cantly outperformed
urban schools based on the Metro-Centric Locale Codes
(B = 0.10, SE = 0.03, p = .003). There was no signifi cant
difference between urban and rural census tracts based on
the RUCAs (B = 0.05, SE = 0.04, p = .185). The results
under “Multilevel, SES as Covariate” correspond to the fi nal
set of analyses. Upon controlling for child- and locationlevel
SES, all three defi nitions indicated that rural locations
signifi cantly outperformed urban locations (B = 0.20, SE
= 0.02, p < .001 for the Metro-Centric Locale model; B =
0.19, SE = 0.03, p < .001 for the RUCA model; and B =
0.12, SE = 0.04, p = .004 for the OMB model).
Discussion
As evidenced by our demonstration, the statistical
model (single-level vs. multilevel model), rural defi nition
(the indicators used to defi ne rural, the geographic unit to
considerable overlap among defi nitions, cross-classifi cation
did occur. Four percent of rural schools were located in
urban census tracts, and 3% of urban schools were located
in rural census tracts. In addition, 4% of rural schools were
located in urban counties, and 5% of urban schools were
located in rural census tracts. Finally, 2% of rural census
tracts were located in urban counties, and 4% of urban
census tracts were located in rural counties. The estimated
ICCs indicated that 33% of the variance in science scores
existed at the school level or higher, 32% existed at the
census tract level or higher, and 15% existed at the county
level or higher.
Table 2 provides the model-estimated means and
mean differences in third grade science scores across rural
defi nitions. The results corresponding to the fi rst set of
analyses are listed under “Single-Level, SES Omitted.”
There was a signifi cant mean difference in science scores
between urban and rural counties based on the OMB
classifi cation (B = -0.07, SE = 0.02, p < .001) such that
urban counties tended to outperform rural counties. There
was no signifi cant difference in science scores between
urban and rural schools based on the Metro-Centric Locale
Codes (B = 0.02, SE = 0.02, p = .308) or between urban
Table 2
Estimated Means and Mean Differences in 3rd Grade Science Scores for “Rural” and “Non-Rural” Locations
Single-Level,
SES Omitted
Multilevel,
SES Omitted
Multilevel,
SES as Covariate
B SE p B SE p B SE p
Metro-Centric Locale Codes
Mean Urban Science Score -0.33 0.01 < .001 -0.40 0.02 < .001 -0.38 0.01 < .001
Mean Rural Science Score -0.31 0.02 < .001 -0.30 0.03 < .001 -0.18 0.02 < .001
Mean Difference 0.02 0.02 .308 0.10 0.03 .003 0.20 0.02 < .001
RUCAs
Mean Urban Science Score -0.32 0.01 < .001 -0.40 0.02 < .001 -0.37 0.01 < .001
Mean Rural Science Score -0.35 0.02 < .001 -0.34 0.03 < .001 -0.19 0.02 < .001
Mean Difference -0.03 0.02 .150 0.05 0.04 .185 0.19 0.03 < .001
OMB Classification
Mean Urban Science Score -0.31 0.01 < .001 -0.26 0.03 < .001 -0.33 0.02 < .001
Mean Rural Science Score -0.38 0.02 < .001 -0.38 0.05 < .001 -0.21 0.04 < .001
Mean Difference -0.07 0.02 < .001 -0.13 0.06 .031 0.12 0.04 .004
Note. B = unstandardized estimate of mean or mean difference. The third set of analyses controlled for child-level and locationlevel
(school-level, census tract-level, or county-level) SES, where SES was grand-mean centered.
IDENTIFYING, ANALYZING, AND COMMUNICATING RURAL 11
Due to the many theoretical perspectives and
operational defi nitions of rural, rural researchers must
shoulder extra responsibility to ensure that their work is
maximally informative and easily replicable. To this end,
we noted throughout our discussion that researchers should
familiarize themselves with the most common defi nitions
available today, and choose (or develop) a defi nition
that, above all else, refl ects their theoretical perspective.
Supplemental considerations, such as the feasibility of a
defi nition given the structure of the data and the intended
analyses, are also important. Since rural research typically
involves clustered data (e.g., children within classrooms,
schools within counties), we described multilevel modeling
as a potentially useful means for analyzing rural data.
We also encouraged the use of covariates to account for
additional variance in the model and rule out alternative
explanations to rural phenomena. Finally, we reminded
researchers to fully communicate the rural nature of their
study, from their theoretical perspective and choice of an
operational defi nition to situating their research fi ndings
within the context of other rural defi nitions. This approach
provides the necessary detail to spur replication and
encourage inclusion in meta-analytic work.
Researchers also need to be cognizant of how modeling
decisions and the choice of an operational defi nition
of rural affect research outcomes and policy decisions.
Using the ECLS-K dataset, we demonstrated the extent to
which parameter estimates and substantive interpretations
can differ across statistical models, rural defi nitions, and
exclusion/inclusion of covariates. Inappropriate selection,
analysis, and/or communication of the rural defi nition may
result in misinformed conclusions about rural phenomena,
which in turn may result in misinformed policy and program
eligibility decisions.
It is our hope that this article has provided researchers,
both new and seasoned, with fi rm guidance in several
critical aspects of rural research. We believe that more
consistent and informed consideration of the presented
guidelines will enhance and clarify the quantitative
literature on rural education. We strongly encourage
qualitative and mixed-methods researchers to continue our
conversation on rural in order to provide a more complete
set of guidelines. In addition, we encourage conversations
such as those of Howley et al. (2005) that extend beyond a
particular method. A blending, or at least a comparison, of
perspectives is desired. Finally, we certainly acknowledge
that rural education research is complex and each study is
unique. There is no simple solution to the issues presented
in this article. Informed and deliberate decision-making
should always trump strict adherence to guidelines.
which the defi nition was applied, and the way in which
the codes were combined),8 and exclusion/inclusion
of covariates greatly impacts parameter estimates and
substantive conclusions. If this study was real and we
had used a single-level or multilevel model based on the
OMB defi nition without consideration of SES, we would
have concluded that urban locations tend to have higher
science test scores than rural locations. If we had instead
used a single-level model without consideration of SES
based on the Metro-Centric Locale Codes, or a single-level
or multilevel model without consideration of SES based on
the RUCAs, we would have concluded that urban and rural
locations tend to perform equally well. Finally, if we had
used a multilevel model without consideration of SES based
on the Metro-Centric Locale Codes, or if we had taken into
account SES and used a multilevel model based on any one
of the rural defi nitions, we would have concluded that rural
locations tend to have higher science test scores than urban
locations.
Rather than try to provide a substantive interpretation
of these results, we use this exercise instead to comment on
the appropriateness of the various defi nitions and analytic
decisions given the context of our demonstration. With
respect to operationally defi ning rural, the Metro-Centric
Locale and RUCA defi nitions are preferred over the OMB
defi nition, as the OMB defi nition was not designed to make
inferences about rural areas (OMB, 2013). In addition, the
OMB defi nition is applied at the county level, and counties
tend to be much more heterogeneous than schools and
census tracts. Choosing between the Metro-Centric Locale
and RUCA defi nitions is less clear and would require careful
theoretical consideration by subject-matter researchers.
Of course, a different defi nition altogether may be more
appropriate given an alternative theoretical perspective. With
respect to the various modeling approaches, the multilevel
models are more appropriate than the single-level models
because they account for similarities among children who
are nested within the same geographic location. Likewise,
including relevant covariates is preferred over examining
the rural effect in isolation, as covariates help to disentangle
seemingly rural differences from differences that are
actually explained by other variables such as SES.
Conclusion
8Our comparison of the three rural defi nitions is
confounded in the sense that we do not know whether the
observed differences are due to differences in the indicators
used to defi ne rural, the geographic unit to which the
defi nition was applied, or the way in which the codes were
combined. Rather than using existing defi nitions, researchers
could create new defi nitions that are more comparable by
fully crossing the levels of these three factors.
12 KOZIOL, ARTHUR, HAWLEY, BOVAIRD, BASH, MCCORMICK & WELCH
Durham, R. E., & Smith, P. J. (2006). Nonmetropolitan
status and kindergarteners’ early literacy skills: Is there
a rural disadvantage? Rural Sociology, 71, 625-661.
doi:10.1526/003601106781262052
Durkheim, E. (1964). The division of labor in society (G.
Simpson, Trans.). New York, NY: Free Press.
DuScheid, S. (2011). GIS: Geotags replacing ZIP codes,
census tracts for analyses. Government Technology.
Retrieved from http://www.govtech.com/geospatial/
GIS-Geotags-Replacing-ZIP-Codes-Census-Tracts.
html
Farmer, T. W., Leung, M.-C., Banks, J., Schaefer, V.,
Andrews, B., & Murray, R. A. (2006). Adequate yearly
progress in small rural and rural low-income schools.
Rural Educator, 27, 1-7. Retrieved from http://www.
ruraleducator.net/archive/27-3/27-3_Farmer.pdf
Garces-Voisenat, J.-P. (2011). Population density, human
capital and productivity (Working Paper No. 102).
Winston-Salem, NC: Wake Forest University.
Gelfand, A. E., Diggle, P. J., Fuentes, M., & Guttorp, P.
(Eds.). (2010). Handbook of spatial statistics. Boca
Raton, FL: CRC Press.
Goodchild, M. F. (1992). Geographical information science.
International Journal of Geographical Information
Systems, 6, 31-45. doi:10.1080/02693799208901893
Halfacree, K. (1993). Locality and social representation:
Space, discourse and alternative defi nitions of the rural.
Journal of Rural Studies, 9, 23-37. doi:10.1016/0743-
0167(93)90003-3
Halfacree, K. (2006). Rural space: Constructing a threefold
architecture. In P. Cloke, T. Marsen, & P. Mooney
(Eds.), Handbook of rural studies (pp. 44-62). Thousand
Oaks, CA: Sage.
Hannum, W. H., Irvin, M. J., Banks, J. B., & Farmer, T.
W. (2009). Distance education use in rural schools.
Journal of Research in Rural Education, 24(3).
Retrieved from http://jrre.vmhost.psu.edu/wp-content/
uploads/2014/02/24-3.pdf
Hart, L. G. (2012). Frontier/remote, island, and rural
literature review (Version 2.99). Retrieved from
University of North Dakota, School of Medicine &
Health Sciences, Center for Rural Health website:
http://ruralhealth.und.edu/pdf/frontierreview.pdf
Hart, L. G., Larson, E. H., & Lishner, D. M. (2005). Rural
defi nitions for health policy and research. American
Journal of Public Health, 95, 1149-1155. doi:10.2105/
AJPH.2004.042432
Howley, C. B., Theobald, P., & Howley, A. (2005). What rural
education research is of most worth? A reply to Arnold,
Newman, Gaddy, and Dean. Journal of Research in
Rural Education, 20(18). Retrieved from http://jrre.
vmhost.psu.edu/wp-content/uploads/2014/02/20-18.
pdf
References
Arnold, M. L., Biscoe, B., Farmer, T. W., Robertson, D.
L., & Shapley, K. L. (2007). How the government
defi nes rural has implications for education policies
and practices (REL 2007—No. 010). Retrieved from
U.S. Department of Education, Institute of Education
Sciences website: http://ies.ed.gov/ncee/edlabs/
regions/southwest/pdf/REL_2007010.pdf
Bell, M. M. (1992). The fruit of difference: The rural-urban
continuum as a system of identity. Rural Sociology, 57,
65-82. doi:10.1111/j.1549-0831.1992.tb00457.x
Brown, D. L., & Schafft, K. A. (2011). Rural people
and communities in the 21st century: Resilience and
transformation. Malden, MA: Polity Press.
Can ZIP codes cross state, county, political jurisdictions
(cities, congressional district), and metro areas? (2011).
Retrieved from ZIP Code FAQ website: http://www.
carrierroutes.com/ZIPCodes.html
Cloke, P. (2006). Conceptualizing rurality. In P. Cloke,
T. Marsen, & P. Mooney (Eds.), Handbook of rural
studies (pp. 18-28). Thousand Oaks, CA: Sage.
Coburn, A. F., MacKinney, A. C., McBride, T. D., Mueller,
K. J., Slifkin, R. T., & Wakefi eld, M. K. (2007).
Choosing rural defi nitions: Implications for health
policy (Issue Brief No. 2). Retrieved from Rural Policy
Research Institute Health Panel website: http://www.
rupri.org/Forms/RuralDefi nitionsBrief.pdf
Coladarci, T. (2007). Improving the yield of rural education
research: An editor’s swan song. Journal of Research
in Rural Education, 22(3). Retrieved from http://jrre.
vmhost.psu.edu/wp-content/uploads/2014/02/22-3.pdf
Creed, G. W., & Ching, B. (1997). Recognizing rusticity:
Identity and the power of place. In B. Ching & G.
W. Creed (Eds.), Knowing your place: Rural identity
and cultural hierarchy (pp. 1-38). New York, NY:
Routledge.
Cromartie, J., & Bucholtz, S. (2008). Defi ning the “rural”
in rural America. Amber Waves, 6, 28-34. Retrieved
from http://www.ers.usda.gov/amber-waves/2008-
june/defi ning-the-%E2%80%9Crural%E2%80%9Din-
rural-america.aspx#.VHSSv8m7jIU
Cressie, N. A. C. (1993). Statistics for spatial data. New
York, NY: Wiley.
Deavers, K. (1992). What is rural? Policy Studies Journal,
20, 184-189. doi:10.1111/j.1541-0072.1992.tb00146.x
Deavers, K. L., & Brown, D. L. (1985). Natural resource
dependence, rural development, and rural poverty
(Rural Development Research Report No. 48).
Retrieved from U.S. Department of Agriculture,
National Agricultural Library website: http://naldc.nal.
usda.gov/naldc/download.xhtml?id=CAT86865629&c
ontent=PDF
IDENTIFYING, ANALYZING, AND COMMUNICATING RURAL 13
Raudenbush, S., & Bryk, A. (2002). Hierarchical linear
models: Applications and data analysis methods.
Thousand Oaks, CA: Sage.
Reeves, E. B., & Bylund, R. A. (2005). Are rural schools
inferior to urban schools? A multilevel analysis of school
accountability trends in Kentucky. Rural Sociology, 70,
360-386. doi:10.1526/0036011054831215
Roscigno, V. J., & Crowley, M. L. (2001). Rurality,
institutional disadvantage, and achievement/
attainment. Rural Sociology, 66, 268-292.
doi:10.1111/j.1549-0831.2001.tb00067.x
Schafft, K. A. (2000). A network approach to understanding
post-socialist rural inequality in the 1990s. Eastern
European Countryside, 6, 25-40.
Schafft, K. A., & Brown, D. L. (2000). Social capital
and grassroots development: The case of Roma selfgovernance
in Hungary. Social Problems, 47, 201-219.
doi:10.1525/sp.2000.47.2.03x0288b
Singer, J., & Willett, J. (2003). Applied longitudinal data
analysis: Modeling change and event occurrence. New
York, NY: Oxford University Press.
Snijders, T., & Bosker, R. (2012). Multilevel analysis: An
introduction to basic and advanced multilevel modeling
(2nd ed.). Thousand Oaks, CA: Sage.
Stinchcombe, A. L. (1961). Agricultural enterprise and rural
class relations. American Journal of Sociology, 67,
165-176. doi:10.1086/223082
Stockard, J. (2011). Increasing reading skills in rural areas:
An analysis of three school districts. Journal of Research
in Rural Education, 26(8). Retrieved from http://jrre.
vmhost.psu.edu/wp-content/uploads/2014/02/26-8.pdf
Tilly, C. (1974). Rural collective action in modern Europe
(CRSO Working Paper No. 96). Ann Arbor: The
University of Michigan, Center for Research on Social
Organization.
U.S. Census Bureau. (2013a). 2010 Census urban and
rural classifi cation and urban area criteria. Retrieved
from http://www.census.gov/geo/reference/ua/urbanrural-
2010.html
U.S. Census Bureau. (2013b). American National Standards
Institute (ANSI) codes. Retrieved from http://www.
census.gov/geo/reference/ansi.html
U.S. Department of Agriculture, Economic Research
Service. (2013a). Rural-urban continuum codes:
Documentation. Retrieved from http://www.ers.usda.
gov/data-products/rural-urban-continuum-codes/
documentation.aspx
U.S. Department of Agriculture, Economic Research Service.
(2013b). Urban infl uence codes: Documentation.
Retrieved from http://www.ers.usda.gov/data-products/
urban-infl uence-codes/documentation.aspx
U.S. Department of Agriculture, Economic Research
Service. (2013c). Rural-urban commuting area codes.
Howley, A., Wood, L., & Hough, B. (2011). Rural
elementary school teachers’ technology integration.
Journal of Research in Rural Education, 26(9).
Retrieved from http://jrre.vmhost.psu.edu/wp-content/
uploads/2014/02/26-9.pdf
How many changes are made to ZIP codes monthly? (2011).
Retrieved from ZIP Code FAQ website: http://www.
carrierroutes.com/ZIPCodes.html
Irvin, M. J., Farmer, T. W., Leung, M.-C., Thompson, J. H., &
Hutchins, B. C. (2010). School, community, and church
activities: Relationship to academic achievement of
low-income African American early adolescents in
the rural deep south. Journal of Research in Rural
Education, 25(4). Retrieved from http://jrre.vmhost.
psu.edu/wp-content/uploads/2014/02/25-4.pdf
Isserman, A. M. (2005). In the national interest: Defi ning
rural and urban correctly in research and public policy.
International Regional Science Review, 28, 465-499.
doi:10.1177/0160017605279000
Jordan, J. L., Kostandini, G., & Mykerezi, E. (2012).
Rural and urban high school dropout rates: Are they
different? Journal of Research in Rural Education,
27(12). Retrieved from http://jrre.vmhost.psu.edu/wpcontent/
uploads/2014/02/27-12.pdf
Kaufman, H. F. (1959). Toward an interactional
conception of community. Social Forces, 38, 8-17.
doi:10.2307/2574010
Kish, L. (1965). Survey sampling. New York, NY: Wiley.
Kučerová, S., & Kučera, Z. (2012). Changes in the spatial
distribution of elementary schools and their impact on
rural communities in Czechia in the second half of the
20th century. Journal of Research in Rural Education,
27(11). Retrieved from http://jrre.vmhost.psu.edu/wpcontent/
uploads/2014/02/27-11.pdf
Lobao, L. M., Hooks, G., & Tickamyer, A. R. (2007).
Introduction: Advancing the sociology of spatial
inequality. In L. M. Lobao, G. Hooks, & A. R.
Tickamyer (Eds.), The sociology of spatial inequality
(pp. 1-28). Albany: State University of New York Press.
Milliken, G. A., & Johnson, D. E. (2009). Analysis of messy
data: Designed experiments (Vol. 1, 2nd ed.). Boca
Raton, FL: CRC Press.
Muthén, L. K., & Muthén, B. O. (1998-2010). Mplus user’s
guide (Version 6.1). Los Angeles, CA: Author.
Ogden, C. L., Lamb, M. M., Carroll, M. D., & Flegal,
K. M. (2010). Obesity and socioeconomic status in
children: United States, 2005-2008 (NCHS Data Brief
No. 51). Retrieved from Centers for Disease Control
and Prevention website: http://www.cdc.gov/nchs/data/
databriefs/db51.pdf
Putnam, R. D. (1993). Making democracy work: Civic
traditions in modern Italy. Princeton, NJ: Princeton
University Press.
14 KOZIOL, ARTHUR, HAWLEY, BOVAIRD, BASH, MCCORMICK & WELCH
Verbitsky-Savitz, N., & Raudenbush, S. W. (2009).
Exploiting spatial dependence to improve measurement
of neighborhood social processes. Sociological
Methodology, 39, 151-183. doi:10.1111/j.1467-
9531.2009.01221.x
Waldorf, B. S. (2006, July). A continuous multi-dimensional
measure of rurality: Moving beyond threshold
measures. Paper presented at the annual meeting of the
American Agricultural Economics Association, Long
Island, CA.
Waller, L. A., & Gotway, C. A. (2004). Applied spatial
statistics for public health data. Hoboken, NJ: Wiley.
Washington, Wyoming, Alaska, Montana, and Idaho Rural
Health Research Center. (n.d.-a). RUCA data: General
methods. Retrieved from http://depts.washington.edu/
uwruca/ruca-methodology.php
Washington, Wyoming, Alaska, Montana, and Idaho Rural
Health Research Center. (n.d.-b). RUCA data: Code
defi nitions: Version 2.0. Retrieved from http://depts.
washington.edu/uwruca/ruca-codes.php
Washington, Wyoming, Alaska, Montana, and Idaho Rural
Health Research Center. (n.d.-c). RUCA data: Using
RUCA data. Retrieved from http://depts.washington.
edu/uwruca/ruca-uses.php
Wilkinson, K. P. (1972). A fi eld-theory perspective for
community development research. Rural Sociology,
37, 43-52.
Wolf, E. R. (1969). Peasant wars of the twentieth century.
New York, NY: Harper & Row.
Wolter, K. M. (2007). Introduction to variance estimation
(2nd ed.). New York, NY: Springer-Verlag.
Retrieved from http://www.ers.usda.gov/data-products/
rural-urban-commuting-area-codes/documentation.
aspx
U.S. Department of Education, Offi ce of Communications
and Outreach. (2012). Guide to U.S. Department of
Education programs: Fiscal year 2012. Retrieved from
http://www.ed.gov/programs/gtep/gtep.pdf
U.S. Department of Education, Institute of Education
Sciences, National Center for Education Statistics.
(n.d.-a). Kindergarten class of 1998-99 (ECLS-K).
Retrieved from http://nces.ed.gov/ecls/Kindergarten.
asp
U.S. Department of Education, Institute of Education
Sciences, National Center for Education Statistics.
(n.d.-b). School locale defi nitions. Retrieved from
http://nces.ed.gov/surveys/ruraled/defi nitions.asp
U.S. Department of Education, Institute of Education
Sciences, National Center for Education Statistics.
(n.d.-c). Prior urban/rural classifi cation systems.
Retrieved from http://nces.ed.gov/surveys/ruraled/
priorclassifi cation.asp
U.S. Department of Education, Institute of Education
Sciences, National Center for Education Statistics.
(n.d.-d). Identifi cation of rural locales. Retrieved from
http://nces.ed.gov/ccd/rural_locales.asp
U.S. Department of Education, Institute of Education
Sciences, National Center for Education Statistics.
(n.d.-e). Frequently asked questions: Data fi le
information. Retrieved from http://nces.ed.gov/ecls/
kinderfaq.asp?faq=5
U.S. Department of Education, Institute of Education
Sciences, National Center for Education Statistics.
(2003). User’s manual for the ECLS-K third grade
restricted-used data fi le and electronic code book
(NCES 2003-003). Washington, DC: Author.
U.S. Offi ce of Management and Budget. (2000). Standards
for defi ning metropolitan and micropolitan statistical
areas; Notice. Federal Register, 65(249). Retrieved
from http://www.whitehouse.gov/sites/default/fi les/
omb/fedreg/metroareas122700.pdf
U.S. Offi ce of Management and Budget. (2010).
2010 Standards for delineating metropolitan and
micropolitan statistical areas; Notice. Federal Register,
75(123). Retrieved from http://www.whitehouse.gov/
sites/default/fi les/omb/assets/fedreg_2010/06282010_
metro_standards-Complete.pdf
U.S. Offi ce of Management and Budget. (2013). Revised
delineations of metropolitan statistical areas,
micropolitan statistical areas, and combined statistical
areas, and guidance on uses of the delineations of
these areas (OMB Bulletin No. 13-01). Retrieved from
http://www.whitehouse.gov/sites/default/fi les/omb/
bulletins/2013/b-13-01.pdf