Questionnaire statistical analysis guide

1. Preparing the data in Excel

• Each row is one respondent and each column is one question or variable; the first row holds only the column names.
• Enter Likert responses as numbers (for example strongly disagree = 1 to strongly agree = 5).
• Reverse-score negatively worded items before analysis (1↔5, 2↔4).
• Put grouping variables such as gender or education in their own columns with consistent labels.
• Leave missing answers blank; do not enter zero, because zero counts as a real value.
• Consistent names for each construct's items (for example Q1–Q5 for “satisfaction”) make building scales easy.

2. Descriptive statistics

Before any test, describe the data: frequencies and percentages for grouping variables (such as gender), and the mean, standard deviation, minimum and maximum for numeric variables and construct scores. These tables are usually the first part of the results chapter.

3. Reliability with Cronbach's alpha

Cronbach's alpha shows how consistent the items of a construct are with each other. It is calculated separately for each construct:

If alpha is low, look at the “alpha if item deleted” column; an item whose removal raises alpha noticeably probably has a problem (or is a reversed item that was not reverse-scored).

4. Testing for normality

Choosing between parametric tests (such as t-tests and ANOVA) and non-parametric tests (such as Mann–Whitney and Kruskal–Wallis) depends on whether the data are normal. The common tests are Kolmogorov–Smirnov and Shapiro–Wilk:

• If p > 0.05, normality is not rejected and a parametric test is suitable.
• If p ≤ 0.05, the data are not normal and a non-parametric test is the safer choice.
• In large samples (above about 100), these tests react to tiny departures; also check whether skewness and kurtosis lie between −2 and +2.

5. Choosing the right test

6. Interpreting p-values and effect sizes

• p < 0.05: the null hypothesis is rejected; the difference or relationship is statistically significant.
• p ≥ 0.05: there is not enough evidence to reject the null hypothesis; this does not “prove” there is no difference.
• Significant is not the same as important; also report the effect size (such as Cohen's d, eta squared or the correlation coefficient).
• Correlation does not prove causation; talk about an “effect” only when the research design allows it.

7. Reporting results in APA style

An example of reporting an independent t-test:

• Give the statistic, degrees of freedom, the p-value and the effect size.
• Report exact p-values (p = .014) unless they are very small (p < .001).
• Number and title every table and do not repeat its figures in the text; state only the main point.

Common mistakes

• Not reverse-scoring negative items before calculating alpha and construct scores.
• Running t-tests or ANOVA on clearly non-normal data with a small sample.
• Running several t-tests in a row instead of ANOVA to compare more than two groups.
• Reporting only p without the statistic and effect size.
• Drawing causal conclusions from correlation.

Need the sample size first? Use the free sample size calculator.