Do I have enough data after 24 months of time?

Steve Simon

2005-04-05

Categories: Blog post Tags: Clinical importance Early stopping

Someone asked me about a correlation coefficient that he computed on a data set that represented 24 months of data collection. A particular correlation of interest (a correlation between staff turnover and resident falls) was not significantly different from zero, but this person wanted to know how much more data to collect before safely concluding that no relation has been or likely will be established.

First compute a confidence interval for the correlation coefficient. If that interval is so narrow that you can rule out the possibility of a clinically important shift, then your sample size is large enough. How large a correlation is clinically significant? That's very hard to say. The correlation is a unitless quantity, and usually you need some measure in physical units (meters, kilograms, etc.) before you can talk about clinical importance.

You might want to look instead at the regression coefficient which does have units of measure in it. I assume that turnover is your independent variable and falls is your dependent variable. Think, then, about how much of an increase in falls per unit change in turnover is important from a clinical perspective. If that value is (I'm just making up a number) 0.5, then your sample size is adequate as long as the confidence interval for the slope is entirely inside plus/minus 0.5.

Please realize that an outsider like me can't tell you what's clinically important, because that requires clinical judgment, something I lack. A good general overview about clinical importance is on my web pages at

--> Stats: Confidence intervals

If this is an ongoing project, perhaps you might also find some value to using a control chart. A control chart allows for continuous monitoring of important processes. Who knows, maybe something that is not apparent now will become apparent because of some of the recent changes in health care? I have a brief outline of control charts at

--> Stats: Guidelines for quality control models

Another issue is that it is dangerous to look at 12 months worth of data, then 13, then 14, etc. because you are testing multiple times on a single hypothesis. It's sort of like being dealt three poker hands and choosing which one you like best. It would be better to select a sample size (time interval) prior to data collection and then test only once. If you do test multiple times, you need to adjust your alpha level. See

--> Stats: Interim analysis and

--> Stats: Early stopping in an animal study (July 1, 2004)

You can find an earlier version of this page on my original website. You can find an earlier version of this page on my original website. Sampling the entire population You can find an earlier version of this page on my original website. More thoughts on equipoise You can find an earlier version of this page on my original website. Two articles debating equipoise You can find an earlier version of this page on my original website. Ethical principles for Complementary and Alternative Medicine You can find an earlier version of this page on my original website. The ethics of randomization You can find an earlier version of this page on my original website. Overview of evidence-based-medicine You can find an earlier version of this page on my original website. Fair Use of copyrighted material You can find an earlier version of this page on my original website. The costs of a false positive test You can find an earlier version of this page on my original website. Finding more information about a gene You can find an earlier version of this page on my original website. More on discovering gene information You can find an earlier version of this page on my original website. Web page for Fisher’s Exact test You can find an earlier version of this page on my original website. Forest plots You can find an earlier version of this page on my original website. Fractions are funny You can find an earlier version of this page on my original website. Analysis of Gene Expression Data Short Course You can find an earlier version of this page on my original website. Geometric distribution You can find an earlier version of this page on my original website. Counterpoint on Google Scholar You can find an earlier version of this page on my original website. Group Sequential Monitoring of Clinical Trials in R You can find an earlier version of this page on my original website. Growth curves You can find an earlier version of this page on my original website. The HapMap project You can find an earlier version of this page on my original website. Hard learned lessons You can find an earlier version of this page on my original website. A surprising application of the harmonic mean You can find an earlier version of this page on my original website. Hedge’s G You can find an earlier version of this page on my original website. Incidence density ratio You can find an earlier version of this page on my original website. Inferential and descriptive statistics You can find an earlier version of this page on my original website. More on information theory models You can find an earlier version of this page on my original website. Information theory and microarrays You can find an earlier version of this page on my original website. Information content of a continuous distribution You can find an earlier version of this page on my original website. How good is your intuition? You can find an earlier version of this page on my original website. Language resources You can find an earlier version of this page on my original website. Two nice R libraries You can find an earlier version of this page on my original website. Optimization using the MM algorithm You can find an earlier version of this page on my original website. Using Mathematica and Matlab for Statistics You can find an earlier version of this page on my original website. Measuring agreement You can find an earlier version of this page on my original website. MedStats discussion group You can find an earlier version of this page on my original website. Media interview tips You can find an earlier version of this page on my original website. Merging in R You can find an earlier version of this page on my original website. Some articles on meta-analysis You can find an earlier version of this page on my original website. Responding to a critique of meta-analysis You can find an earlier version of this page on my original website. Meta-analysis talk You can find an earlier version of this page on my original website. Review articles on microarrays You can find an earlier version of this page on my original website. More articles on microarrays You can find an earlier version of this page on my original website. More on normalization You can find an earlier version of this page on my original website. Publicly available microarray data You can find an earlier version of this page on my original website. Statistical Analysis of Microarrays by Insightful You can find an earlier version of this page on my original website. Microarray data analysis, again You can find an earlier version of this page on my original website. RMA normalization of microarrays You can find an earlier version of this page on my original website. Moderator variables You can find an earlier version of this page on my original website. Searching for information about the molasses with milk enema You can find an earlier version of this page on my original website. Expected value and moments You can find an earlier version of this page on my original website. Monetary incentives You can find an earlier version of this page on my original website. Moving R objects You can find an earlier version of this page on my original website. Step-down procedures for multiple comparisons You can find an earlier version of this page on my original website. Naming conventions for genes, proteins, etc. You can find an earlier version of this page on my original website. A totally negative microarray experiment You can find an earlier version of this page on my original website. Non-destructive data editing You can find an earlier version of this page on my original website. Non-random samples You can find an earlier version of this page on my original website. A nonspecific diagnostic test You can find an earlier version of this page on my original website. Computing normal probabilities You can find an earlier version of this page on my original website. Presenting Numbers, Tables, and Charts You can find an earlier version of this page on my original website. Object oriented features of R You can find an earlier version of this page on my original website. Odds ratios less than one You can find an earlier version of this page on my original website. Open-ended questions on a survey You can find an earlier version of this page on my original website. Another open site closes You can find an earlier version of this page on my original website. Determining the optimal threshold for a diagnostic test You can find an earlier version of this page on my original website. Summing ordinal data You can find an earlier version of this page on my original website. The paired availability design You can find an earlier version of this page on my original website. Patients’ reactions to finding out they were in the placebo You can find an earlier version of this page on my original website. Permutation tests for microarrays You can find an earlier version of this page on my original website. Post hoc power is never justified You can find an earlier version of this page on my original website. PowerPoint Counterpoint You can find an earlier version of this page on my original website. Developing good practice guidelines You can find an earlier version of this page on my original website. Profile analysis and MANOVA You can find an earlier version of this page on my original website. PubMed tags You can find an earlier version of this page on my original website. Public access to publications from NIH-funded research You can find an earlier version of this page on my original website. Quality of published research You can find an earlier version of this page on my original website. Publicon software You can find an earlier version of this page on my original website. Quality control exercises You can find an earlier version of this page on my original website. Quality control exercises, Part 2 You can find an earlier version of this page on my original website. Quotes for February You can find an earlier version of this page on my original website. Quotes for the month of January You can find an earlier version of this page on my original website. Quotes for the month of March You can find an earlier version of this page on my original website. Application of the ROC curve to microarray data You can find an earlier version of this page on my original website. Coding race/ethnicity You can find an earlier version of this page on my original website. An inefficient approach to randomization You can find an earlier version of this page on my original website. More on the weaknesses of randomized trials You can find an earlier version of this page on my original website. Effective communication about randomized clinical trials You can find an earlier version of this page on my original website. More on regular expressions You can find an earlier version of this page on my original website. Stats: Report cards You can find an earlier version of this page on my original website. The fate of retracted articles You can find an earlier version of this page on my original website. More on the retroactive prayer study You can find an earlier version of this page on my original website. Re-weighting the data You can find an earlier version of this page on my original website. What’s New in SPSS version 14.0 You can find an earlier version of this page on my original website. Seminar notes, S-PLUS Clinical Safety Miner You can find an earlier version of this page on my original website. Relationship between sample size and p-values You can find an earlier version of this page on my original website. Sample size calculation for a nonparametric test You can find an earlier version of this page on my original website. Sample size for a binomial confidence interval You can find an earlier version of this page on my original website. Sample size for a binary endpoint You can find an earlier version of this page on my original website. Science mentoring You can find an earlier version of this page on my original website. Allegations of scientific misconduct You can find an earlier version of this page on my original website. IRBs and scientific validity You can find an earlier version of this page on my original website. Searching the Internet You can find an earlier version of this page on my original website. Searching the literature You can find an earlier version of this page on my original website. More on searching the literature You can find an earlier version of this page on my original website. Another search for evidence You can find an earlier version of this page on my original website. A third search for the evidence You can find an earlier version of this page on my original website. Selective reporting of research findings You can find an earlier version of this page on my original website. Self experimentation You can find an earlier version of this page on my original website. An error slips through the peer review process You can find an earlier version of this page on my original website. Seventeen years between research and practice You can find an earlier version of this page on my original website. Side effects of Cox-2 inhibitors You can find an earlier version of this page on my original website. When one group only has a single observation You can find an earlier version of this page on my original website. What does a 60% drop mean? You can find an earlier version of this page on my original website. Slow progress on my weblog You can find an earlier version of this page on my original website. A small p-value does not mean a large difference You can find an earlier version of this page on my original website. Small relative risks You can find an earlier version of this page on my original website. Preserving spacing in html code You can find an earlier version of this page on my original website. Spectrum Bias You can find an earlier version of this page on my original website. The S+ CorrelatedData Library You can find an earlier version of this page on my original website. S-plus version 7 You can find an earlier version of this page on my original website. Standard deviation versus standard error You can find an earlier version of this page on my original website. Free statistics software You can find an earlier version of this page on my original website. Stepwise regression to screen for covariates You can find an earlier version of this page on my original website. When can I stop my CQI study? You can find an earlier version of this page on my original website. Stratified Cox regression models You can find an earlier version of this page on my original website. String manipulations in R You can find an earlier version of this page on my original website. Summary Receiver Operating Characteristic Curve You can find an earlier version of this page on my original website. Surrogate outcomes You can find an earlier version of this page on my original website. Taguchi methods You can find an earlier version of this page on my original website. Ten research studies that anyone teaching EBM should be familiar You can find an earlier version of this page on my original website. Another top ten study in EBM You can find an earlier version of this page on my original website. More on the top ten studies in EBM You can find an earlier version of this page on my original website. I abhor Lilliefor and other tests of normality You can find an earlier version of this page on my original website. Tolerance limits You can find an earlier version of this page on my original website. Registration of clinical trials You can find an earlier version of this page on my original website. A simple trick in R You can find an earlier version of this page on my original website. When the F test is significant, but Tukey is not You can find an earlier version of this page on my original website. Importing value labels from Access into SPSS You can find an earlier version of this page on my original website. Vote for me You can find an earlier version of this page on my original website. Interesting web links and quotes for the month of April You can find an earlier version of this page on my original website. Interesting web sites, publications, and quotes for the month of You can find an earlier version of this page on my original website. Interesting web sites, publications, and quotes for the month of You can find an earlier version of this page on my original website. Recommended web links for the month of February You can find an earlier version of this page on my original website. Interesting web links for the month of January You can find an earlier version of this page on my original website. Interesting web links and quotes for the month of May You can find an earlier version of this page on my original website. Interesting quotes, web pages, and publications for the month of You can find an earlier version of this page on my original website. Interesting web links for the month of March You can find an earlier version of this page on my original website. Interesting web links and quotes for the month of May You can find an earlier version of this page on my original website. Interesting web sites, publications, and quotes for the month You can find an earlier version of this page on my original website. Interesting web sites, publications, and quotes for the month You can find an earlier version of this page on my original website. Withholding information