1. Leeds University Business School
  2. Departments
  3. Work and Employment Relations
  4. Research projects
  5. Questionnaire design

Research project

Questionnaire design

Description

Dr. Mark Robinson of WBRC is engaged in an ongoing programme of research examining social science research methods. Effective research methods are essential to provide the evidence to underpin effective decision making about organizational performance and wellbeing.

Keywords: measurement, multi-item scales, psychometric scales, questionnaires, surveys, psychometric questionnaires, questionnaire design, survey design, questionnaire items, survey items, rating scales, response points, verbal anchors, Likert scales, Likert rating scales, scale development, psychometric scale development, factor analysis, questionnaire identification codes, numerically coding responses, calculating scale scores, research methods, quantitative research methods, organizational psychology, work psychology, occupational psychology, organizational behaviour

Publications and outputs

One of Mark’s recently published journal papers examined and evaluated existing best practice in questionnaire design and administration, focusing particularly on the use of multi-item psychometric scales. The full reference of this paper is as follows, together with a weblink from which the published PDF file of the paper can be freely downloaded (open-access):

  • Robinson, M. A. (2017). Using multi-item psychometric scales for research and practice in human resource management. Human Resource Management. http://dx.doi.org/10.1002/hrm.21852 

The paper offers evidence-based practical guidance for academics, researchers, students, and practitioners who use questionnaires to collect data. Some of these key guidance points are now summarized below:

  1. In the main section of your questionnaire, each question or statement should be accompanied by a set of 5-7 equidistant response points with balanced verbal response anchors (e.g., “strongly agree”).
  2. Use a minimum of 3-4 questions/statements to measure each variable and check they measure the variable in a reliable and valid way.
  3. A reliable and valid set of 3+ items measuring the same variable is referred to as a psychometric scale, or multi-item scale, or simply a scale.
  4. When developing scales, the following process should be followed: (1) generate preliminary items, (2) evaluate preliminary items, (3) administer preliminary items, (4) implement participant feedback, (5) analyze preliminary item data, (6) administer revised items, (7) analyze revised item data, (8) criterion validate psychometric scales.
  5. Throughout the scale development process, the following types of reliability and validity should be assessed: internal reliability, test-retest reliability (where relevant), content validity, construct validity, and criterion validity.
  6. Detailed step-by-step practical guidance on all of these issues is provided in the journal paper, together with accompanying diagrams and tables.

Important note: Among other uses, this journal paper is intended as a teaching resource for training courses and educational courses in questionnaire design. It has been published open-access with a CC BY 4.0 Licence, so that it may be freely accessed, freely distributed, and its content freely used provided that attribution is made to the original source:

  • Robinson, M. A. (2017). Using multi-item psychometric scales for research and practice in human resource management. Human Resource Management. http://dx.doi.org/10.1002/hrm.21852

Finally, if you find this guidance about questionnaire design useful, you may also find the following recently published book chapter by Dr. Mark Robinson useful as it explains the research contexts in which questionnaires are used:

Specifically, this chapter provides general guidance about social science quantitative research methods, including measuring variables (operational definitions, reliability, validity) and research design (scientific principles, experimental designs, correlational designs)