Design And Analysis Of Sample Surveys Pdf
File Name: design and analysis of sample surveys .zip
- How to analyze survey data: best practices for actionable insights from survey analysis
- How to Design and Analyze a Survey
- The Design and Analysis of Longitudinal Surveys: Controversies and Issues of Cost and Continuity
This course provides practical methods and tools to analyze complex survey data with a hands-on introduction to the use of specialized statistical software procedures. Relevant design features of the NCS-R, NHANES and HRS include survey weights that take into account differences in probability of selection into the sample and differences in response rates, as well as stratification and clustering in the multistage sampling procedures used in identifying the sampled households and individuals. After introducing essential concepts related to complex sample designs, the course will turn to the construction of survey weights, estimation of sampling variance, descriptive analysis, regression analysis, and finally special topics in the analysis of survey data. Participants can expect to work on homework exercises, computer lab exercises, and a final analysis project.
How to analyze survey data: best practices for actionable insights from survey analysis
Survey research is sometimes regarded as an easy research approach. However, as with any other research approach and method, it is easy to conduct a survey of poor quality rather than one of high quality and real value. This paper provides a checklist of good practice in the conduct and reporting of survey research. Its purpose is to assist the novice researcher to produce survey work to a high standard, meaning a standard at which the results will be regarded as credible.
The paper first provides an overview of the approach and then guides the reader step-by-step through the processes of data collection, data analysis, and reporting.
It is not intended to provide a manual of how to conduct a survey, but rather to identify common pitfalls and oversights to be avoided by researchers if their work is to be valid and credible. Survey research is common in studies of health and health services, although its roots lie in the social surveys conducted in Victorian Britain by social reformers to collect information on poverty and working class life e.
Charles Booth [ 1 ] and Joseph Rowntree [ 2 ] , and indeed survey research remains most used in applied social research. The researcher therefore uses information from a sample of individuals to make some inference about the wider population.
Data are collected in a standardized form. This is usually, but not necessarily, done by means of a questionnaire or interview. There is no attempt to control conditions or manipulate variables; surveys do not allocate participants into groups or vary the treatment they receive. Surveys are well suited to descriptive studies, but can also be used to explore aspects of a situation, or to seek explanation and provide data for testing hypotheses.
As with any research approach, a choice of methods is available and the one most appropriate to the individual project should be used.
This paper will discuss the most popular methods employed in survey research, with an emphasis upon difficulties commonly encountered when using these methods. The aim is to examine a situation by describing important factors associated with that situation, such as demographic, socio-economic, and health characteristics, events, behaviours, attitudes, experiences, and knowledge. Descriptive studies are used to estimate specific parameters in a population e. Analytical studies go beyond simple description; their intention is to illuminate a specific problem through focused data analysis, typically by looking at the effect of one set of variables upon another set.
These are longitudinal studies, in which data are collected at more than one point in time with the aim of illuminating the direction of observed associations. Data may be collected from the same sample on each occasion cohort or panel studies or from a different sample at each point in time trend studies.
This form of research collects data to ascertain the effects of a planned change. The research produces data based on real-world observations empirical data. The breadth of coverage of many people or events means that it is more likely than some other approaches to obtain data based on a representative sample, and can therefore be generalizable to a population. Surveys can produce a large amount of data in a short time for a fairly low cost.
Researchers can therefore set a finite time-span for a project, which can assist in planning and delivering end results. The significance of the data can become neglected if the researcher focuses too much on the range of coverage to the exclusion of an adequate account of the implications of those data for relevant issues, problems, or theories.
The data that are produced are likely to lack details or depth on the topic being investigated. Securing a high response rate to a survey can be hard to control, particularly when it is carried out by post, but is also difficult when the survey is carried out face-to-face or over the telephone. Good research has the characteristic that its purpose is to address a single clear and explicit research question; conversely, the end product of a study that aims to answer a number of diverse questions is often weak.
This is a trap novice researchers in particular fall into. Therefore, in developing a research question, the following aspects should be considered [ 4 ]:. Widen the base of your experience, explore related areas, and talk to other researchers and practitioners in the field you are surveying. Consider using techniques for enhancing creativity, for example brainstorming ideas. Avoid the pitfalls of: allowing a decision regarding methods to decide the questions to be asked; posing research questions that cannot be answered; asking questions that have already been answered satisfactorily.
The survey approach can employ a range of methods to answer the research question. Common survey methods include postal questionnaires, face-to-face interviews, and telephone interviews.
This method involves sending questionnaires to a large sample of people covering a wide geographical area. As response rates are low, a large sample is required when using postal questionnaires, for two main reasons: first, to ensure that the demographic profile of survey respondents reflects that of the survey population; and secondly, to provide a sufficiently large data set for analysis.
The researcher then asks the respondent a series of questions and notes their responses. The response rate is often higher than that of postal questionnaires as the researcher has the opportunity to sell the research to a potential respondent. Face-to-face interviewing is a more costly and time-consuming method than the postal survey, however the researcher can select the sample of respondents in order to balance the demographic profile of the sample. Telephone surveys, like face-to-face interviews, allow a two-way interaction between researcher and respondent.
Telephone surveys are quicker and cheaper than face-to-face interviewing. Whilst resulting in a higher response rate than postal surveys, telephone surveys often attract a higher level of refusals than face-to-face interviews as people feel less inhibited about refusing to take part when approached over the telephone.
Whether using a postal questionnaire or interview method, the questions asked have to be carefully planned and piloted. The design, wording, form, and order of questions can affect the type of responses obtained, and careful design is needed to minimize bias in results. When designing a questionnaire or question route for interviewing, the following issues should be considered: 1 planning the content of a research tool; 2 questionnaire layout; 3 interview questions; 4 piloting; and 5 covering letter.
The topics of interest should be carefully planned and relate clearly to the research question. It is often useful to involve experts in the field, colleagues, and members of the target population in question design in order to ensure the validity of the coverage of questions included in the tool content validity. Researchers should conduct a literature search to identify existing, psychometrically tested questionnaires.
A well designed research tool is simple, appropriate for the intended use, acceptable to respondents, and should include a clear and interpretable scoring system.
A research tool must also demonstrate the psychometric properties of reliability consistency from one measurement to the next , validity accurate measurement of the concept , and, if a longitudinal study, responsiveness to change [ 5 ]. The development of research tools, such as attitude scales, is a lengthy and costly process.
It is important that researchers recognize that the development of the research tool is equal in importance—and deserves equal attention—to data collection. If a research instrument has not undergone a robust process of development and testing, the credibility of the research findings themselves may legitimately be called into question and may even be completely disregarded.
Researchers who are unable or unwilling to undertake this process are strongly advised to consider adopting an existing, robust research tool. Questionnaires used in survey research should be clear and well presented. The use of capital upper case letters only should be avoided, as this format is hard to read. Questions should be numbered and clearly grouped by subject.
Clear instructions should be given and headings included to make the questionnaire easier to follow. Questions may be open where the respondent composes the reply or closed where pre-coded response options are available, e. Closed questions with pre-coded response options are most suitable for topics where the possible responses are known. Closed questions are quick to administer and can be easily coded and analysed.
Open questions should be used where possible replies are unknown or too numerous to pre-code. Open questions are more demanding for respondents but if well answered can provide useful insight into a topic.
Open questions, however, can be time consuming to administer and difficult to analyse. Whether using open or closed questions, researchers should plan clearly how answers will be analysed.
Open questions are used more frequently in unstructured interviews, whereas closed questions typically appear in structured interview schedules. A structured interview is like a questionnaire that is administered face to face with the respondent. When designing the questions for a structured interview, the researcher should consider the points highlighted above regarding questionnaires.
The interviewer should have a standardized list of questions, each respondent being asked the same questions in the same order. If closed questions are used the interviewer should also have a range of pre-coded responses available. If carrying out a semi-structured interview, the researcher should have a clear, well thought out set of questions; however, the questions may take an open form and the researcher may vary the order in which topics are considered.
A research tool should be tested on a pilot sample of members of the target population. This process will allow the researcher to identify whether respondents understand the questions and instructions, and whether the meaning of questions is the same for all respondents.
Where closed questions are used, piloting will highlight whether sufficient response categories are available, and whether any questions are systematically missed by respondents. When conducting a pilot, the same procedure as as that to be used in the main survey should be followed; this will highlight potential problems such as poor response. All participants should be given a covering letter including information such as the organization behind the study, including the contact name and address of the researcher, details of how and why the respondent was selected, the aims of the study, any potential benefits or harm resulting from the study, and what will happen to the information provided.
The covering letter should both encourage the respondent to participate in the study and also meet the requirements of informed consent see below.
The concept of sample is intrinsic to survey research. Usually, it is impractical and uneconomical to collect data from every single person in a given population; a sample of the population has to be selected [ 7 ].
This is illustrated in the following hypothetical example. A hospital wants to conduct a satisfaction survey of the patients discharged in the previous month; however, as it is too costly to survey each patient, a sample has to be selected. In this example, the researcher will have a list of the population members to be surveyed sampling frame. It is important to ensure that this list is both up-to date and has been obtained from a reliable source. The method by which the sample is selected from a sampling frame is integral to the external validity of a survey: the sample has to be representative of the larger population to obtain a composite profile of that population [ 8 ].
There are methodological factors to consider when deciding who will be in a sample: How will the sample be selected?
What is the optimal sample size to minimize sampling error? How can response rates be maximized? The survey methods discussed below influence how a sample is selected and the size of the sample.
There are two categories of sampling: random and non-random sampling, with a number of sampling selection techniques contained within the two categories.
The principal techniques are described here [ 9 ]. Generally, random sampling is employed when quantitative methods are used to collect data e. Random sampling allows the results to be generalized to the larger population and statistical analysis performed if appropriate. The most stringent technique is simple random sampling. Using this technique, each individual within the chosen population is selected by chance and is equally as likely to be picked as anyone else.
Referring back to the hypothetical example, each patient is given a serial identifier and then an appropriate number of the population members are randomly selected. Alternative random sampling techniques are briefly described.
In systematic sampling, individuals to be included in the sample are chosen at equal intervals from the population; using the earlier example, every fifth patient discharged from hospital would be included in the survey.
How to Design and Analyze a Survey
Not a MyNAP member yet? Register for a free account to start saving and receiving special member only perks. This chapter begins with an overview of sample surveys, which are not widely used in the aviation community. Other federal agencies have been using sample surveys successfully for a long time. Section 3.
PDF | Contents Preface 1 1 Introduction 2 2 Planning Experiments and Overview of Minimal Sample Sizes 5 Regression Analysis.
The Design and Analysis of Longitudinal Surveys: Controversies and Issues of Cost and Continuity
Flawed data can guide even the greatest leaders to the wrong conclusions. When success hangs in the balance, you need to be absolutely sure that you're gathering the right data with the right methods. So we asked our data scientist, Christopher Peters , to craft this guide about how to collect and analyze data. It's like a college-level course in survey design: you'll learn how to write questions, distribute them, and synthesize the responses.
Collected all of your survey data? Confused about what to do next and how to achieve the optimal survey analysis? Use this post as a guide to lead the way to execute best practice survey analysis in Customer surveys can have a huge impact on your organization. Whether that impact is positive or negative depends on how good your survey is no pressure.
Enter your mobile number or email address below and we'll send you a link to download the free Kindle App. Then you can start reading Kindle books on your smartphone, tablet, or computer - no Kindle device required. A unique feature of the book is that a large number of exercises with real sets of data from various fields is included either as illustrative examples to demonstrate the method of analysis or unsolved problems to be attempted by the reader so as to make concepts and procedures more clear.
Publications Summary. Hardcopies of publications are intended primarily for those in developing countries where internet connections are limited or unavailable. Abstract: This analytical study has been described as the handbook "champion" of sampling errors and design effects in survey samples. The first part of the report describes the general features of the sample designs of 48 nationally-representative household surveys conducted under the DHS program. A large number of identically defined survey estimates is considered in the analysis: 37 in total, covering fertility, family planning, fertility intentions, child health and child mortality.