4.0. When to design your research

Design your research after you have identified the weaknesses in previous research, and before you conduct it.

After you have conducted the literature review you know the weaknesses of previous research. When you go over the limitations discussed in previous research, you know which problems you want to avoid in your own research. To some extent these weaknesses may be unavoidable, or difficult for you to repair. If you are allowed to work with existing data, use the best possible source of data you can lay your hands on. Ask your advisor for suggestions on existing data sets that do not have commonly identified weaknesses. Don’t take the advice for granted though: ask critical questions about the quality of the datasets that your supervisor suggests. We’ll go over some of the most common problems in research design in this chapter.

Think hard about the best possible design of your research before you actually do the research. Preregister the choices you make in the collection and analysis of data. Preregistration forces you to think critically about these choices. Raising the bar for the quality of the research prevents disappointing conclusions. You would like to avoid are that you were not able to answer your question for methodological reasons. Examples of such disappointing conclusions are: the sample analyzed (students) is not representative of the target population (humanity); the dataset analyzed did not contain measures of the main concepts in the theory; the measure of the dependent variable refers to a period before the period that the measure of the alleged cause refers to; or the measures do not capture the concepts that you intended to measure. We will go into issues of sampling and measurement in the next sections. But first let’s discuss the kinds of research questions that various research designs allow you to answer.