Survey Guidelines

Need to survey Beaverton School District students or staff? All research generated from inside and outside the District needs to be approved. BSD administration asks that you take the time to consider the following guidelines, and then submit applications to do research (including surveys) to Dee Carlson, Research Committee Coordinator, BSD Department of Teaching and Learning. A committee will meet to review the application.

Meanwhile, consider these guidelines:

1. Avoid surveying if you can.
2. Work with your administrators.
3. Sampling considerations.
4. Managing the survey project.
5. Writing the questions and building the survey.
6. Steps for conducting a survey at BSD.

1. Avoid surveying if you can. This applies to surveying with pencil and paper, and web- or e-mail-based surveying. Electronic surveys are also time-consuming and can be annoying. There is a lot of information already available that might allow you to find out what you need to know, rather than doing a new survey. Consult a BSD research professional about your research interests before you make big decisions about collecting data. Call Information and Technology, Teaching and Learning, or Community Involvement to inquire about the wonderful sources of existing BSD data, and what might apply to your research questions. Too much surveying is not good because people may neglect great data that is already available, and because BSD staff resent answering too many surveys. Consider conducting focus groups instead of a survey if your research goals are a little vague, or the topic is quite sensitive. Do not conduct research at the very end of the year or at the beginning of the year. Avoid hectic times such as testing days.
   
   
2. Work with your administrators. BSD administrators need to approve all new research. Because respondents are more likely to complete a survey if administrators ask them to do so, recruit the right administrator for your purposes, and address the survey cover memo or e-mail from that person. This will maximize your survey’s RESPONSE RATE. Unless you get a high response rate (at least 60-70%) without known bias in who chose to respond, you may not be able to trust your findings, and readers may reject your conclusions, no matter how valid you may believe them to be.
   
   Working closely with administrators should also help to ensure that the survey is well-timed, and does not interfere with other school or District activities. Although a survey may seem harmless, it might inadvertently raise issues already receiving District attention. Administrators should be able to help.
   
   
3. Sampling considerations. Data from a survey of the population (not a sample) can be assumed to represent the population as long as the response rate is at least 60-70%, and no sub-groups are badly under-represented. If you need to know about a large population (say, all high school students), consider a sampling approach. This means that you can survey only some of the people you are interested in, chosen to represent the entire group statistically. Consult someone who can tell you about how to draw a random or purposive sample, if this is the route you decide to take. This person should be able to tell you how to identify the sample you need, and how to interpret the data statistically.
   
   
4. Managing the survey project. Leave enough time both for planning at the start, and for analysis and reporting at the end. Developing the questions should involve other people to get the best results. Choose questions based on what topics you wish to focus on in your report on survey findings (i.e., only include what you intend to discuss; keep the survey as short as possible).
   
   Designate a project manager, and develop a strategy for careful follow up on every survey to maximize RESPONSE RATE. Pay close attention to and streamline as much as possible the logistics of distributing and collecting the survey. Always identify the denominator (the total people who will receive the survey and might answer it) for the RESPONSE RATE.
   
   After data collection, someone will need to enter the data on a database, or format and analyze the database if data were collected electronically. The survey analyst might create tables or graphs to help the project manager to interpret the results in the light of project goals and other context both statistical and qualitative. The analysis and report writing phase should last long enough for the writer to include all the relevant issues, check data results thoroughly, and refine the message based on reviews of early drafts.
   
   
5. Writing the questions and building the survey There are many ways to go wrong here. Get other people to help you with wording. Ask potential respondents, administrators, and your peers for suggestions about content and wording, to help you identify hidden issues and avoid pitfalls. Secretaries who work with respondents are also a good source for what language to use and not use. Field-test your questions if possible. Some rules of thumb:
   
   
   • Frame each question in FEW WORDS and about ONLY ONE ISSUE at a time. If you have complicated issues, you need to break them down into separate questions. Otherwise you will get responses that can’t be interpreted. Respondents will be frustrated because they have to think too long about the question and still may not know what it means. A respondent should be able to answer each question quickly and should feel a little sense of accomplishment at the end of the survey.
     
     
   • Only ask good questions to which you honestly need to know the answers, and to which reasonable people could disagree. Avoid emotionally-laden terminology. Do not ask respondents to incriminate themselves. If you know that everyone always thinks X, don’t ask X. If you have to take action in Y direction, don’t ask respondents whether you should do Y. Only ask questions where you are open to different answers, and respondents are not incriminating themselves to respond honestly.
     
     
   • DO NOT USE JARGON OF ANY KIND. It is safer to stay away from or minimize terms of art and other specialized vocabulary. Even though it may surprise you, not everyone will know what you mean. More importantly, jargon is freighted with connotations (implied no-no’s and sacred cows) that bias responses. Simple wording allows more respondents to answer more freely. However, if you survey “experts” you may need to use special terminology in order to make survey questions understandable to them. Keep this to an absolute minimum.
     
     
   • Order of questions should be from broad and general, to particular and specific, and make sense. Ask for RELEVANT characteristics in the beginning (avoid income). Then ask their general opinions about things, and finally ask about specific opinions or facts, one topic area at a time. Very sensitive material should be handled discreetly, often at the end.
     
     
   • Consider what message you send by the order and wording of questions. Group similar questions together, maybe with a few sub-headings. Adopt an ordering of questions that will not bias the answers that respondents tend to give. Avoid ordering or wording questions to give an unpleasant message or feeling of any type. Be positive. Use the active voice. You want to leave the respondent in a good mood.
     
     
   • Handle sensitive material at the end. Ideally, the early questions should be easy to answer, encouraging the respondent to continue. The rapport you build with the respondent will help you get their responses on the more difficult or sensitive questions, near the end. If the respondent quits here, at least they will have answered the earlier questions.
     
     
   • Entitle the survey, and welcome the respondent by telling who or what organization is conducting the survey and how the data will be used, in a SHORT 1 or 2-sentence preamble at the top. At the end, thank the respondent for completing the survey.
     
     
   • Number the questions. A simple thing often neglected, this will help you, analysts, and readers handle and discuss the survey efficiently.
     
     
   • Response options should be chosen carefully. A respondent gets used to the response options he or she is going to have, and this allows them to proceed more quickly, which respondents prefer. Best is to choose one or two “response sets” and use one within each section or throughout. A typical response set is “Strongly agree, Agree, Unsure, Disagree, Strongly disagree, NA.” Another at BSD is to grade something “A, B, C, D, or F.” Present the options with the positive end of the continuum first, or on the left. Frame the question as a statement if this fits the response set best, such as the Agree/Disagree response set. Make sure the options offered include all possible answers. Be sure that any rating scale labels are meaningful. It is best to include a “don’t know” or “not applicable” option unless you are certain that all respondents will be able to respond.
     
     
   • Open-ended questions. When you want more detail and you don’t know what might come up, you can ask for an open-ended response to a simple question to elicit broad views. Only do one or a FEW of these. Often the most valuable part of results, this material is colorful and allows readers to hear respondents voice their own ideas about a topic.
     
     
6. Steps for conducting a survey at BSD.
   
   
   • Speak to your administrator to establish the goals of the project and determine the feasibility of conducting research (see above)
     
     
   • Visit this website for more details about research design, especially the Steps in a Survey Project: http://www.surveysystem.com/sdesign.htm
     
     
   • Design the survey using guidelines (above)
     
     
   • Submit a research application to the Research Coordinator, and discuss it with the Coordinator, and/or the research committee members
     
     
   • Revise plans based on committee input, and re-submit
     
     
   • Proceed based on guidelines (above)