Asking research questions

Asking research questions

The purpose of this building block

Introduction

You always need information for creating and realizing suitable IT solutions, for example (based on the goals of 5 research spaces of the DOT framework):

  • to better understand the situation, so that you can come up with a suitable solution
  • to be able to use existing expertise, and not waste your time reinventing the wheel
  • gain insight into various solution options, and be able to make a suitable choice,
  • to determine whether your solution meets what is needed and indeed solves the problem
  • and whether your solution adds something compared to what already existed

You can state these kinds of questions in your Plan of Action, so that you can make clear what you are going to do, how this all contributes to your final solution, and whether the whole will be feasible. But sometimes it is difficult to think in advance what kind of information you actually need. You need to know what you do not know – that is very hard! Or maybe you have an idea of what to do, but you have trouble expressing the corresponding question.

This building block gives you some tools for thinking up and formulating research questions. In order to gain inspiration about what we could ask, we look at the goals you could have with these kinds of questions. Next, we will look at how you can turn questions that you are dealing with in practice (e.g. about how you should tackle something, or which decision to take) into research questions that give you the information you need. Then there is a short checklist that you can use to see if your questions are well-formulated. There is an assignment attached to apply the skills you learn here in your own project.

Tip: For this building block it is useful if you already have a rough idea of your goal. Otherwise it becomes very difficult to think about what information you need to achieve that goal.

Prerequisites
One of the components in this building block uses the DOT framework. For this, it can be useful to briefly go through the corresponding DOT framework building block. However, the teaching material in this building block is set up in such a way that it can be used independently.

P.S. Depending on what you are asked to deliver, you will or will not explicitly mention these kinds of questions. However, whether or not it is asked of you, it is always useful to be aware that you these are the questions you want to answer. That way you do not waste time on side tracks that actually do not contribute so much to your final goal.

Learning goals

After this building block:

  • You can turn practical questions into research questions (information questions)
  • You can think of the questions from what you want to achieve with the research with the DOT framework
  • You can formulate research questions well

Relation to the DOT-framework

The DOT framework. cc-by-sa HAN (link)

Within the DOT framework, you try to solve a central question. Every request or assignment can in principle be reformulated into a question. This is called the main question.

In order to develop a solution for this central assignment, you need information such as "What are the wishes and needs of the end users?" (Field), "What kind of ICT solutions have already been tried for this problem?" (Library), or "Is this prototype indeed faster than the old application?" (Lab). These are sub-questions.

From goals to research questions

The DOT framework can help you determine what kind of research can be useful for your assignment. First read the chapter on the considerations of research of the DOT framework building block, which deals with various goals that you could pursue with your research.

In short, you can use research to ensure:

  • Fit or expertise
    A better connection to what is needed in the context where the solution will be used (e.g. with regard to users or other stakeholders and existing systems or processes to which your solution must adjust), or better use of already available expertise in the field of problem or your solution (by looking at what is already known about the problem in question, what has already been tried by others, what best practices there are, etc.).
     
  • Overview or certainty
    More overview and insight into the problem and possible solutions (e.g. by talking to users or experts) or certainty that your solution indeed improves and meets the requirements (e.g. by testing your solution, having it reviewed by an expert, or compare with existing solutions).
     
  • Data or inspiration
    Justification of decisions (e.g. by comparing different options and checking which meet your requirements) or innovation within your solution (for example by brainstorming, perhaps together with potential users).

What is important within your assignment? What questions could you ask to ensure that that aspect receives sufficient attention?

 

Praktijkvragen per onderzoeksstrategie
Common general questions for the DOT framework research strategies.
(The framework cc-by-sa HAN (link))


Tip: if you would like to know more about this framework or want to see more examples, take a look at the building block about the DOT framework.

 

Sources
Turnhout, K. van, Craenmehr, S., Holwerda, R., Menijn, M., Zwart, J. P., & Bakker, R. (2013). Tradeoffs in design research: development oriented triangulation. In Proceedings of the 27th International BCS Human Computer Interaction Conference (p. 56). British Computer Society.

From practical questions to research questions

Practical questions are questions that you, as a professional(-to-be), deal with in practice. There are different types of questions. A question can, for example, concern:

  • Standards: Do we have to have all the documentation and code reviewed by another team member by default?
  • Decisions: Are we going for a centralized system where everything is on one server, or are we going to spread the tasks over multiple separate systems?
  • Actions: How do I ensure that I understand what the client really wants?
  • Knowledge: Can the user perform the task faster than before with this new application?
  • Reasons: Why do people choose unsafe passwords?

Sometimes a question is not important enough to conduct an entire research for. Sometimes research is not necessary, and do you know the answer from your own knowledge and experience or is it just a matter of questioning. Sometimes you can go further without examining it.

But sometimes:

  • the answer is not readily available on the shelf (research is possible and necessary),
  • it is important that there is an answer that you want to know that it is correct (a thorough answer is desirable),
  • and you really do need that answer to continue (it is relevant).

Then you can look for the research question in your practical question. Knowledge and why questions are often already researchable, but you can also turn other practical questions into research questions by asking yourself the following question:

What kind of information do I need to be able to answer this practical question?

Example

Suppose you want to develop a chat app such as WhatsApp, but one that is safe to use in traffic. A practical question could then be: What demands should my solution meet?
What kind of concrete information questions can you think of?

(Tip: first try to come up with some examples yourself!)

  • What kinds of wishes and requirements do potential users think of, based on their own experiences with existing chat apps?
  • Which chat apps do these people already use? What expectations does this result in, concerning what the app could do or how certain things should work in the app?
  • What exactly is dangerous about chatting while cycling or driving and what kind of requirements does that lead to?

There are, of course, many more questions to consider that could help you come up with a secure chat app to be used in traffic. You will therefore probably have to limit your research to those questions that are most important to your solution. You can also quickly come to a good enough answer for some questions, for example by Googling or briefly talking to an expert.

 

Sources
Harinck, F. (2010). Basisprincipes praktijkonderzoek. Sixth edition. Garant. Consulted on 14-09-2018 from Google Books.

Formulating research questions

Formulating a research question well can be tricky. Here are a set of checks that you can use, both for your main question and for your sub-questions:

Is the question relevant (does it fit in with the goal)?
If the question is not necessary for realizing your solution, why would you spend time on it? This also means that your main question indicates the central question you ask to come to your solution. The main question is then divided into various aspects to form the sub-questions. So if a sub-question does not fall within the scope of your main question (and therefore your goal), the question is irrelevant (or you may have to reformulate your main question...).

Is the function clear (in terms of the type of professional product)?
Is it clear how the answer contributes? Does it, for example, does provide information for a better understanding of the problem or does it help with determining the design?

Is the question neutral? (not an opinion or already including a direction for the solution)
A big pitfall is that secretly a decision is hidden in the question, while you are actually still in a phase in which it is not yet clear that this decision is the right one for your situation.

Is the question feasible (time, resources)?
If your question is going to cost a lot of time or money to answer, then it is important to consider whether this is possible at all and if it might be possible: whether this is the best way to spend your resources. 

Is it an open question? (not yes / no)
The goal of a research question is to bring out certain information. The choice for yes or no (which contains very little information by itself) is usually the result of a practical question that you wanted to answer with the help of this information.

Is the question clear? (one explanation, concise)
It is important that your question can only be understood one way, making it clear to yourself what you want to know, but also for your client, supervisor or teacher.

Tip: If you already have experience with the SMART formulation of goals and requirements, you will recognize a number of aspects from above (specific, acceptable, realistic).

Examples

A. Suppose your goal is to speed up the handling of orders at Bol.com. Which question better matches the goal (relevance)? Why?

  1. How can the number of complaints about the handling of orders at Bol.com be reduced?
  2. Which measures could speed up the handling of orders at Bol.com?
Even though there may be a relationship between the number of complaints and the speed of the handling, if the goal is to speed up the handling itself, the second question is more relevant to your goal.


B. Imagine you have devised an ICT solution to speed up the handling of orders at Bol.com. Now you want to evaluate that solution. Which question would more suitable as a research question? Why?

  1. Is the new situation better than the old situation?
  2. How much does the processing time in the new situation differ from the old situation?

Firstly, the first question is a closed question. The answer is yes or no. The second question will give you more information: how large is the difference? Then you can still try to draw the conclusion for your practical question whether the new situation with your solution is better.
The second point s that the first question is very unspecific. Which aspect is better? What do you want to look at? This is much more concrete in the second question: you look at the processing time.

C. Say your grandmother is feeling lonely in the old people's home, and you want to help her using your experience with robots. What is a more appropriate question to start with?

  1. How can a robot chat with Grandma about the weather?
  2. How can a robot help grandmother feel less lonely?
Both questions are still fairly abstract. They could be starting points as a central question, but they are not really concrete information questions that are ready to be investigated.
Apart from that, question 1 is more specific than question 2. You might think: but specific is good! Unfortunately, there is a hidden assumption here: if a robot would chat with grandma about the weather, then she would feel less lonely. This question therefore already includes a solution direction, while you have not yet determined that chatting about the weather will help against loneliness. This question is not neutral enough for the phase you are currently in. If your research into question 2 shows that talking about the weather could be very helpful, then you have a good reason to make the step to question 1.

 

Sources
Losse, M. (2011). Onderzoeksvaardigheden voor docenten. Boom Lemma.

Assignments

Project assignment

One of the first deliverables in a project is your Action Plan. Your plan must clarify what you are going to do, what you are going to deliver, and whether this is all feasible within the stipulated time a planning. The explicit formulation of research questions helps you to clarify what kind of information you need (and what you need to do) to realize the proposed solution.

The assignment for your project is: Brainstorm and formulate at least 1 research question per team member for your project.

  1. Together as a team:
    1. Brainstorm a list of questions from the different goals you can have with your research (fit or expertise, overview or security, data or inspiration). Just write down your questions as they come up (probably in the form of a practical question).
  2. Each team member now chooses the most important question from the list that has not yet been chosen by another team member, and goes through the following two steps:
    1. Reformulate your practical question in the form of an information question (research question).
    2. For your information question, apply each of the checks: relevance, function, neutrality, feasibility, openness, and clarity. If necessary, adjust your question to make it more well-formed.
  3. Discuss each of the reformulated research questions as a team. If everyone agrees, you include the question in your action plan.

Practical tips

  1. By formulating your research questions as concretely as possible, you clearly define your work and you have a clearer idea of what you have to do.

  2. Research questions may change! Maybe you will find out during your project that your direction for a solution will not work at all, or that there is another possibility with a greater chance of success. Do not get stuck in your original plan because this is what you happened to have thought up at the beginning of your project. Keep focused on the final goal. Adjusting your plans based on insights gained in the mean time is a must.

  3. At the end of your project, look back at your research questions and reflect on what was or was not useful and what you overlooked. That way you will improve at thinking up and formulating research questions.

  4. Do not forget to answer your questions and apply the answer to your final solution! If your question was important enough to ask and put time into researching it, then the answer must also be available somewhere, and it should have an effect on your final solution (relevance!). The answer could, for example, be included in a research report or as part of a design or test report.

Bibliography

Harinck, F. (2010). Basisprincipes praktijkonderzoek. Zesde druk. Garant. Geraadpleegd op 14-09-2018 van Google Books.

Losse, M. (2011). Onderzoeksvaardigheden voor docenten. Boom Lemma.

Turnhout, K. van, Craenmehr, S., Holwerda, R., Menijn, M., Zwart, J. P., & Bakker, R. (2013). Tradeoffs in design research: development oriented triangulation. In Proceedings of the 27th International BCS Human Computer Interaction Conference (p. 56). British Computer Society.

For the teacher

This section will not be translated into English (for now).

Het doel van deze bouwsteen
Onderzoeksvragen formuleren helpt met het concreet maken van wat voor informatie nodig is om een bepaalde oplossing te ontwerpen of te realiseren. Deze bouwsteen geeft handvatten voor het bedenken en formuleren van onderzoeksvragen.

Plaats in het curriculum en module
Onderzoeksvragen worden aan het begin van een project geformuleerd en gedurende en aan het eind van het project beantwoord, zodat aan het eind er een goede eindoplossing staat. Het is aan te bevelen deze bouwsteen in een van de eerste weken van een project aan te bieden. Aangezien dit een nuttige vaardigheid is bij alle projecten en verder geen speciale voorkennis vereist kan het vroeg in de opleiding worden aangeboden, bijv. leerjaar 1 kwartiel 1 of 2.

Er is in principe geen vereiste voorkennis. Het is wel nuttig deze bouwsteen te combineren met kennis van het DOT framework. Dit helpt namelijk zowel bij het bedenken van vragen als bij het bedenken van een aanpak om deze vragen te beantwoorden. Een voorbeeld zou dus kunnen zijn: Week 1: Probleemdefinitie en Onderzoeksvragen stellen; Week 2: Onderzoeksaanpak bepalen met het DOT framework.

Het is goed om studenten regelmatig tijdens de opleiding te laten reflecteren op de zelf-geformuleerde informatievragen, bijvoorbeeld aan de hand van de lijst van checks. Onderzoeksvragen bedenken en formuleren is en blijft een moeilijke vaardigheid.

Voorbeeld lesplan

2u Laat de studenten het thuis de bouwsteen doornemen Individueel, thuis
10min Geef in de les een korte samenvatting (nut van deze vaardigheid, praktijkvraag vs. informatievraag, doelen onderzoek, checks) en beantwoord eventuele vragen na aanleiding van de voorbereiding. Klassikaal
15min

Een gezamenlijke brainstormsessie voor praktijkvragen binnen het team, die helpen met het realiseren van hun eindproduct.

De docent kan hier wat input geven om het proces op gang te brengen wanneer een team vast zit.

 In projectteam
15min

Werk een gekozen praktijkvraag uit tot een informatievraag die voldoet aan de basischecks (relevantie, functie, neutraliteit, haalbaarheid, open en duidelijkheid).

De docent kan hier vragen stellen die problemen in de huidige formulering zichtbaar maken voor de student en de checks verder toelichten bij onduidelijkheid.

 Elk teamlid individueel
15min Bespreek de geformuleerde onderzoeksvragen binnen het team en voeg na consensus toe aan het PvA. In projectteam
20min Laat een paar studenten een voorbeeld geven van hun praktijkvraag en uiteindelijke onderzoeksvraag, en beantwoord veelvoorkomende vragen. Klassikaal

 

Dit voorbeeld duurt iets minder dan een blok van 2 lesuren van 45 minuten. De overige tijd kan worden besteedt aan het noteren van het doel en de deelvragen in een concept Plan van Aanpak en het verder uitwerken van andere belangrijke onderzoeksvragen.

Tips voor projectbegeleiding
De belangrijkste punten om studenten op te begeleiden is:

  1. Dat de onderzoeksvragen duidelijk en specifiek zijn geformuleerd, en
  2. dat de antwoorden op de onderzoeksvragen duidelijk bijdragen aan het eindproduct.

Stel dus vooral vragen die studenten hierover aan het nadenken zetten, door bijv. met opzet een onderzoeksvraag anders te interpreteren dan waarschijnlijk is bedoeld maar wat wel past bij de formulering, of door expliciet te vragen naar wat er op basis van het antwoord gedaan gaat worden.

Beoordelen
De projectopdracht zou als volgt beoordeeld kunnen worden per teamlid:

  Onvoldoende Voldoende Goed
Per vraag De vraag is geen informatievraag, maar heeft bijvoorbeeld nog de vorm van een praktijkvraag. De vraag is een informatievraag, relevant voor de opdracht en eenduidig verwoord. De informatievraag is relevant voor de opdracht, heeft een duidelijke functie, is neutraal, haalbaar, en open en eenduidig geformuleerd.

 

  • Het arrangement Asking research questions is gemaakt met Wikiwijs van Kennisnet. Wikiwijs is hét onderwijsplatform waar je leermiddelen zoekt, maakt en deelt.

    Auteur
    hbo-ict open-oio
    Laatst gewijzigd
    2018-09-17 09:30:30
    Licentie

    Dit lesmateriaal is gepubliceerd onder de Creative Commons Naamsvermelding 4.0 Internationale licentie. Dit houdt in dat je onder de voorwaarde van naamsvermelding vrij bent om:

    • het werk te delen - te kopiëren, te verspreiden en door te geven via elk medium of bestandsformaat
    • het werk te bewerken - te remixen, te veranderen en afgeleide werken te maken
    • voor alle doeleinden, inclusief commerciële doeleinden.

    Meer informatie over de CC Naamsvermelding 4.0 Internationale licentie.

    Aanvullende informatie over dit lesmateriaal

    Van dit lesmateriaal is de volgende aanvullende informatie beschikbaar:

    Toelichting
    Je hebt altijd informatie nodig voor het bedenken en realiseren van goede ICT oplossingen, maar soms is het lastig te bedenken wat voor informatie je dan concreet nodig hebt. Deze bouwsteen geeft je wat handvatten voor het bedenken en formuleren van onderzoeksvragen. Dit is een HBO-ICT bouwsteen voor Onderzoek in Onderwijs.
    Eindgebruiker
    leerling/student
    Moeilijkheidsgraad
    gemiddeld
    Trefwoorden
    bouwsteen, building block, hbo ict oio, onderzoek, onderzoekend vermogen, onderzoeksvragen, research, research questions

    Gebruikte Wikiwijs Arrangementen

    hbo-ict open-oio. (2018).

    Onderzoeksvragen stellen

    https://maken.wikiwijs.nl/130300/Onderzoeksvragen_stellen

  • Downloaden

    Het volledige arrangement is in de onderstaande formaten te downloaden.

    Metadata

    LTI

    Leeromgevingen die gebruik maken van LTI kunnen Wikiwijs arrangementen en toetsen afspelen en resultaten terugkoppelen. Hiervoor moet de leeromgeving wel bij Wikiwijs aangemeld zijn. Wil je gebruik maken van de LTI koppeling? Meld je aan via info@wikiwijs.nl met het verzoek om een LTI koppeling aan te gaan.

    Maak je al gebruik van LTI? Gebruik dan de onderstaande Launch URL’s.

    Arrangement

    IMSCC package

    Wil je de Launch URL’s niet los kopiëren, maar in één keer downloaden? Download dan de IMSCC package.

    Voor developers

    Wikiwijs lesmateriaal kan worden gebruikt in een externe leeromgeving. Er kunnen koppelingen worden gemaakt en het lesmateriaal kan op verschillende manieren worden geëxporteerd. Meer informatie hierover kun je vinden op onze Developers Wiki.

  • Gebruik
    Weergave
  • Wikiwijs is een dienst van