Current Controversies in Nurse Staffing Research Barbara A. Mark PhD, RN, FAAN Sarah Frances Russell Distinguished Professor School of Nursing The University of North Carolina at Chapel Hill
Theoretical Issues • Research Design Issues • Data Issues • Interpretation Issues and Strength of Evidence
The Need for Better Theory • To explain why empirical results occur • To predict conditions under which observed relationships are most and least likely to hold true • To develop theoretically-based, targeted interventions aimed at improving quality of care
Better Theory Requires • Precise conceptual and operational definitions of major constructs • Clear explication of proposed causal relationships • Identification of critical mediators and moderators of the proposed relationships
Conceptual Definitions of Nurse Staffing • What does “nurse staffing” mean? • Is it the NUMBER of RNs who are providing care? • Is it the number of HOURS of care delivered to patients by RNs?
Is it the PROPORTION of care delivered to patients that is delivered by RNs? • Is it WHAT RNs do? • Is it the COMPETENCE of the RNs who are providing care? • Is it the QUALITY of the actual care delivered by RNs?
Common Operational Definitions of “Nurse Staffing” • RN hours of care • RN hours as a percent of all hours • Percent of total staff who are RNs • FTEs per 1000 patient days • FTEs per 1000 case mix adjusted patient days
These Arise from Different Conceptual Approaches • Capacity: actual or potential ability to perform • Availability: ready for use • Actuality: actual existence; reality
Nurse hours reflect average staffing across a 24 hour period, not fluctuations in census, scheduling patterns in different shifts and period of the year • Interpretation of an “FTE” • Overlooks the issue of staffing “adequacy” … Do we need more under-performing nurses (i.e., “warm bodies”)?
Example: Unit with All RN Staff • Staffing is operationally defined as “proportion of total staff who are RNs” • Assumption is that patients will receive care from RNs. However… • Poor management, inadequate support services, poor medication/delivery and supply systems, etc…. • Results in patients actually receiving few hours of care …. albeit all from RNs
The primary question • What is the theoretical rationale for expecting that variations in nurse staffing will cause variations in quality of care? • WHY more RNs, better outcomes? • Fagin’s surveillance function • Silber’s failure to rescue • Cure, care, coordination (Mauksch, 1966!)
“Nurse Sensitive” Outcomes • Issues raised about staffing conceptualization and operationalization also apply to selection of nurse sensitive measures • What does it really mean when we say a selected variable is “nurse sensitive?” • How do we demonstrate that an outcome is “nurse sensitive”?
Are there outcomes of hospital care that are not nurse sensitive? • Might there be a continuum of nurses’ contributions to patient outcomes? • Does nurse staffing relate differently to adverse outcomes (mortality, complications, etc.) than more positive outcomes (self-care ability, illness-related self-efficacy, disease management knowledge and skill)?
Opening The Black Box: Mediation Y X M Addresses the WHY and HOW of a hypothesized relationship
Possible Mediators of the Staffing-Quality Relationship • Professional practice environment • Communication quality • Nurse-physician collaboration
2) 30% variance Outcome Staffing 3) 9% variance 1) 32% variance 3) 19% variance Communication Quality
Opening the Black Box: Moderation X1 X2 Z X1X2 The interaction terms represents a joint relationship between the two independent variables and accounts for additional variance in the outcome beyond that explained by either variable alone. Addresses the issue of “under what conditions” does a relationship exist?
Possible Moderators • Managed care penetration • Safety climate • Urban/rural location
Moderating Effect of HMO Penetration Marginal effect of one-unit increase in RN FTEs/1000 inpatient days on mortality ratio (Mark, Harless & McCue, 2005)
Research Design Issues • Unit of analysis • Cross-sectional vs. longitudinal designs • Risk adjustment
Unit of Analysis • “Levels” in organizations • Hospital (i.e., organizational) • Work/Nursing unit • Individual • Implies a conceptual hierarchy in which one level is “nested” in higher levels • Conceptual approach should drive selection of unit of analysis (or multi-level approach) • Little agreement regarding which level is the most appropriate
Challenges • Hospital level analyses assume homogeneity of effects across work/nursing units • Adjusting standard errors: HLM-type models to deal with clustered data • Evaluating appropriateness of aggregating data from one level to another
Sampling issues are important, particularly if data are collected from individuals and aggregated to the hospital level • Concerns about data reliability, validity and representativeness when very small numbers of individuals provide data
Multi-Level Analyses • Few studies examine multi-level models • Mark, Salyer & Wan (2003) • McGillis-Hall et al., (2003) • Specification of correct multi-level model: • extremely important • conceptually demanding • statistically challenging (sometimes!)
Advantages Efficient, relatively easy to do Disadvantages Difficult to establish causality Takes a “one-time” slice of organizational action that may not be representative of critical processes that can best be observed over time Advantages Examines change over time Allows stronger causal inference than cross-sectional designs Disadvantages Tricky issue of lag structure and temporal ordering Extremely costly and time-consuming Research Designs Cross SectionalLongitudinal
Unaddressed Questions • How long does it take for changes in nurse staffing to have an impact on quality? Are they immediate or delayed? • What is the functional form of that trajectory? • Are some outcome measures more likely to be affected by short-term vs. long-term staffing changes? • Are there certain thresholds of nurse staffing that can be identified?
Risk Adjustment • Risk: “the possibility of suffering harm or loss” • Risk of what? death, complications, functional status, resources used (costs, LOS) • Severity from a medical perspective vs. severity from a nursing perspective (i.e., intensity of nursing care or acuity)
Common Risk Adjustment Strategies • Medicare Case Mix Index • developed as a measure of resource use, not for risk • used for Medicare patients • all patient CMI • Nursing Intensity Weights (developed in New York; updated in 2004) • DRG based • measure nursing resource utilization
APR-DRGs • AHRQ’s Clinical Classifications Software and comorbidity software • Medstat (or other proprietary) disease staging algorithms • Disease specific risk models (i.e. CABG, AMI) • Study specific risk models
Cautions with any risk model • Generally contain substantial measurement error • Depend on coded data from standard administrative and claims databases • Subject to inaccurate and incomplete coding
Systematic coding differences across institutions • Unmeasured aspects of risk known to the provider but not captured in the risk model • Complication or comorbidity?
Data Issues • Research question and unit of analysis drive data issues (although sometimes it is true that data availability drives the research!) • Different databases and data elements are differentially available depending upon unit of analysis and on whether primary or secondary data are used • In general, hospital level data are obtained through secondary data; unit level data are obtained through primary data collection
Ideal Data Characteristics • Reliable: should, as accurately as possible, reflect actual staffing and be free of systematic measurement error • Valid: should allow for direct measurement of staffing at the level theoretically appropriate for the study • Should be possible to disaggregate to the appropriate unit of analysis • Unfortunately, there is no database that meets all these requirements
Hospital Level Staffing Data • American Hospital Association • Total facility (i.e., hospital and long-term care unit) • Does not differentiate RNs working in in-patient and out-patient facilities • Does not differentiate RNs in direct care from those in support/supervisory roles • Does not report UAPs • Does not differentiate paid vs. productive time
CMS Provider of Services File • RNs and LPNs only • State databases • not available in all states • little consistency in definitions across states • allocation of nurses to relevant subunits may be extremely difficult
Unit Level Nurse Staffing Data • Frequently relies on primary data collection, which is extremely costly and time intensive • CalNOC, VANOD, MILNOD, NDNQI • California Office of State Health Planning and Development (OSHPD) has arguably the best data on nurse staffing • Service level, not unit level, however • Has agency hours • Reports productive hours
Staffing Data from Primary Data Collection • Still would like the data to meet the ideal characteristics addressed earlier • Now also have an additional human element (the data collector) • In multi-site studies, definitional problems reign supreme • For example, how is an FTE measured?
Consistency across sites is also problematic • Requires an enormous amount of follow up and obsessive attention to detail to “get it right.”
Strength of Evidence Issues • Measurement error • Omitted variable bias • The functional form of the relationship • Endogeneity • Selection bias
Measurement Error • Nurse staffing estimates based on data that do not measure staffing directly • Rely on assumptions in terms of allocating staffing estimates to the appropriate acute care unit • Administrative hours method • Adjusted patient days method • Revenue proportion method
Omitted variable bias • Improperly specified model • Most studies do not include other care providers (i.e., MDs), who clearly interact with RNs in provision of patient care • Strong conceptual basis for study, valid and reliable data, valid and reliable risk adjustment model all reduce likelihood of omitted variable bias
Functional Form • Many studies assume linearity in relationship between nurse staffing and outcomes • Curvilinear relationships • Examining marginal effects is important in terms of policy implications
Endogeneity • Correlation between an independent variable and the error term of the model • Example: a hospital experiences an unexpected increase in deaths or complications
Hospital responds by increasing the number of nurses • Difficult to estimate the true effect of nurse staffing on outcomes, since, in this case, the change in outcomes resulted in a change in nurse staffing
Possible Remedy • Instrumental variable approach • Theoretically difficult, since selection of the instrument must • Actually cause variation in the independent variable, but • Affect the outcome variable ONLY through its influence on the independent variable
Selection bias • Occurs when selection/participation is contingent upon the value of the dependent variable • Generally not dealt with, in either cross-sectional or longitudinal studies of nurse staffing and quality of care • Two-stage Heckman models; conceptually challenging • Propensity scores: Aiken, Smith & Lake (1994) matched 39 magnet hospitals along 12 control variables with 195 non-magnet hospitals