Template:Pilot Testing

From Design Science Research Methods
Jump to navigation Jump to search

Pilot Testing

Description

  • Pilot testing is an often overlooked but extremely important part of the research process!
  • helps detect potential problems in research design and/or instrumentation (e.g., whether the questions asked is intelligible to the targeted sample)
  • ensures that the measurement instruments used in the study are reliable and valid measures of the constructs of interest
  • pilot sample = small subset of the target population

Examples

Further Readings

[1] Bhattacherjee, A. (2012). "Social Science Research: Principles, Methods, and Practices", Textbooks Collection. 3. Available at: https://scholarcommons.usf.edu/oa_textbooks/3

Sampling Strategy

Description

choose target population and a strategy of how to choose samples related to the unit of analysis in a research problem avoid a biased sample


Sampling approaches:

Probability sampling:

  • technique in which every unit in the population has a chance (non-zero probability) of being selected
  • chance to be selected can be accurately determined
  • sampling procedure involves random selection at some point

methods: random sampling, systematic sampling, stratified sampling, cluster sampling, matched pairs sampling, multi-stage sampling

Non-probability sampling:

  • some units of the population have either zero chance of selection or the probability of selection cannot be determined accurately
  • selection criteria are non-random (e.g. quota, convenience)
  • estimation of sampling errors is not allowed
  • information from a sample cannot be generalized back to the population

methods: convenience sampling, snowball sampling, quota sampling, expert sampling

Examples

Probability sampling:

Cluster sampling:

  • divide population into clusters
  • randomly sample a few clusters and measure all units within that cluster

Matched pairs sampling:

  • two subgroups of population
  • compare two individual units from subgroups with other
  • ideal way to understand bipolar differences

Multi-stage sampling:

  • combine the previously described sampling technique
  • e.g. combine cluster and random sample

Random sampling:

  • all subsets are given equal probability of being selected
  • unbiased estimates of population parameters

Stratified sampling:

  • divided into homogeneous and non-overlapping subgroups
  • simple random sample within each subgroup

Systematic sampling:

  • sampling frame is ordered according to some criteria
  • elements are selected at regular intervals

Non-probability sampling:

Convenience sampling:

  • take a sample from a population that is close to hand
  • e.g. outside of shopping mall
  • may not be representative, therefore limited generalization

Expert sampling:

  • choose respondents in a non-random manner based on their expertise on the phenomenon
  • findings are still not generalizable to a population
  • e.g. in-depth study on an institutional factor such as Sarbanes-Oxley Act

Quota sampling:

  • segment population into mutually exclusive subgroups
  • take a non-random set of observations to meet pre-defined quota
  • pre-defined quota can either proportional (as the overall population) or non-proportional (less restrictive)
  • both are not representative of the population

Snowball sampling:

  • start criteria-based
  • ask respondents for further potential participants

Further Readings

[1] Bhattacherjee, A. (2012). "Social Science Research: Principles, Methods, and Practices", Textbooks Collection. 3. Available at: https://scholarcommons.usf.edu/oa_textbooks/3