Introduction
Surveys can be useful tools for gathering opinions and perceptions from a large number of people. They provide information on which to base or test decisions; however, effective surveys are time-consuming and expensive.
Although surveys are difficult and time-consuming, there are times when a survey is the most appropriate tool to use to gather data. For library planners who, after thoughtful consideration, decide that a survey is the best approach for their Looking-Around effort, this chapter provides a detailed guide to carrying out effective surveys. It begins with a series of four questions to be answered before the decision to conduct a survey can be made, describes four types of surveys typically used by libraries, and provides a detailed, step-by-step guide to planning and implementing a survey.
Levels of Effort
Any survey requires an extensive level of effort. The decision to undertake a survey should be made only after all other possible options are considered.
Questions to ask before the survey
Four series of questions should be answered before the decision to conduct a survey is made:
Types of surveys
Four types of surveys that libraries use are surveys of staff, citizens, students, and users. Here is a description of those options, along with the advantages and disadvantages of each.
staff attitudes toward library service
priorities for service
what services should be added
what services should be eliminated if funding is decreased
evaluation of library collections and services
job achievement and satisfaction.
The advantages of this type of survey include:
staff are most informed about library operations
the survey is easy to administer
staff often have opinions about change.
The disadvantages include:
anonymity must be preserved in survey design
staff may focus on internal management problems rather than service
staff may be protective of their own service area.
where respondents go for information
participation in community groups
business and school use
reading patterns
access to library service
convenience of library service
knowledge of library service
use of the library
priorities for library service
personal information.
and can determine the following information needs:
material needed in major life coping areas
importance or depth of need
preference for priority service roles
where respondents go for information to satisfy the need
satisfaction of respondents with information received from each source
use of library
education level
age and race
disability and handicap.
The advantages of this type of survey include:
obtains the opinion and perceptions of ALL citizens rather than just library users, can verify demographics from secondary sources
can obtain information needs rather than perceptions about the library
makes citizens feel a part of the planning process, increasing their sense of ownership for the library, whether or not they use it.
The disadvantages of such a survey include:
respondents may not know enough about the library to provide informed input
difficult to obtain accurate data about information needs
base for random sampling may include a bias (e.g., telephone numbers exclude
those without phones and those with unlisted telephone numbers; registered voters excludes people who do not vote)
if done by telephone, requires extensive planning, volunteers or paid workers making the call, training for the callers, and making calls in the early evening to assure an accurate cross-section of the community
people may not be able to articulate their information needs
people may not accurately remember their library use patterns
mail surveys typically do not receive a sufficient response.
personal characteristics
use of public library
access to public library
"wants" from public library
source of information for school assignments
source of recreational information and reading
use of audio-visual materials.
The advantages of this type of survey include:
if done in the school, the students are a captive audience extremely high return rate
promotes positive relations between library and school and library and parents
young people tend to be candid in their remarks.
Disadvantages of such a survey include:
schools may not cooperate
random samples outside of the school are very difficult to identify.
access and ease of use of library
purpose of using library
success in getting materials and information
use of staff
satisfaction with library
what services might be eliminated if funding decreases
personal information needs.
The potential advantages include:
can be done with a minimum of help
users presumably know about the library and its services, thus can give informed opinions
survey can include questions about users' perceptions of the library and actual use of the library
makes patrons feel a part of their library's planning process.
The disadvantages include:
may disrupt library service
must be done over at least a week's period to get a random sample of use by various user groups at typical times (e.g., students after school, working people after work and on weekends)
ease of administration encourages proliferation of unnecessary questions
does not represent the community at large.
Steps in survey research: planning and implementation
Once the decision has been made that a specific type of survey is appropriate, the planners should follow the steps below during the planning and implementation stage:
Step 1: Identify issues
The most common mistake in survey research is beginning to design an instrument before identifying the issues to be addressed. This step is essential to the success of a survey. It is not enough to decide to conduct a survey as part of a community analysis project. Library staff, board members, and other interested parties--as well as any consultant or other third party who may be enlisted--may have to collaborate in defining this task. That may be done in various ways. Two popular strategies are having a committee write a charge for the survey and holding focus groups to identify issues on which data is needed.
Step 2: Check for available data
Before deciding to conduct a community analysis survey, check for available data. Frequently, readily accessible figures from the library's patron, catalog, and circulation files can be used instead of asking questions of users about their personal characteristics and library use. Usually, such available data can be combined with newly collected data without great difficulty. Asking for a user's library card number on a survey may be sufficient to make this link.
Step 3: Review survey research methods
Before beginning to design a community analysis survey, review the basics of survey research methods. A good, practical, plain-English text that describes the principles of survey research is Earl Babbie's The Practice of Social Research, 6th edition (Babbie 1992 OK). For more specific guidance on the survey as a library management tool, consult Robert Swisher and Charles R. McClure's Research for Decision-Making (Swisher and McClure 1984).
Step 4: Determine whom to survey
After identifying issues to be addressed, determining whom to survey is the next important question. The issues will dictate whom one should survey. Frequently surveyed groups include the general public, library users and non-users, library staff, and students.
After deciding whom to survey, a related decision is how to select those who will be surveyed. Obviously, some of the above-mentioned groups are more easily sampled than others. One reason library users are frequently surveyed is that drawing a sample from the library's patron files is relatively simple. Likewise, library staff and students are easily defined populations. Sampling the general public is the most difficult and presents the greatest difficulties in terms of costly survey administration and low response rate.
Deciding how to go about sampling any of these groups is very important. Options include random, stratified, quota, and convenience sampling. These options are described in most research texts. Notably, one must choose between random and stratified sampling, if it is important to be able to estimate the margins for error in survey results. Margins for error cannot be calculated for quota or convenience samples.
Step 5: Determine how to survey them
The next step is to determine how the sample chosen will be surveyed. When most people hear the word "survey," they think of a questionnaire. Indeed, this is the most efficient method of survey research as it enables researchers to reach the greatest number of people with the greatest number of questions at the least cost. Interviews, however, are a particularly useful alternative when dealing with complex issues.
If the questionnaire method is chosen, there is a further choice between mailing the form to potential respondents, distributing it by hand at the library or other selected locations, and publishing it in the local newspaper. Knowing the issues and who is being surveyed will equip one to make this choice.
Step 6: Design the survey
In designing a survey, there are several sets of general issues to address.
The general format of a survey is important. Its length should be the minimum required to address the issues identified. While brevity is important, a survey should not look too crowded. Techniques for exploiting space on a page include: using columns and the back of the page; using different sized fonts; and defining sections and emphasizing key items using lines, boxes, shading, and placement.
A variety of item formats are possible. The most basic distinction is between open-ended questions and those with structured responses. The former are easier to write, but the latter are easier to tabulate and analyze. Structured response types include: yes/no or true/false items, multiple choice, check off or ranking lists, Likert (strongly agree to strongly disagree) and semantic differential (excellent to poor) scales, and matrix items. Detailed information about and examples of these different item types may be found in the two references mentioned in Step 3 above.
When using structured responses, it is important to ensure that they meet three standards. They should be standardized, comprehensive, and mutually exclusive. Again, these standards are addressed in more detail in the research texts.
In wording survey items, there are several rules of thumb to follow. Items should be worded in neutral or, at least, balanced terms. Items should be clear and brief. Compound questions should not be used since respondents may wish to answer each half differently. Survey items should not be redundant, unless there is some reason to check the reliability of respondents.
In ordering survey items, logical or time sequence should be followed. Controversial items are probably best placed in the middle of a survey. This will keep them from being the first thing potential respondents read or the last thing they remember about the survey.
Beyond these general rules of survey research, there are several excellent national examples on which to draw. Between 1989 and 1993, there were at least five national polls on public library issues.
In addition to these national examples, many examples of local community analysis surveys are usually available from library development staff at state library agencies.
Step 7: Foster a high response rate
In designing and administering a community analysis survey, there are several practices one can follow to foster a high response rate:
Step 8: Conduct a pre-test
Once the survey instrument is designed, do not distribute or send it immediately. First, conduct a pre-test. Select a small, diverse group of potential respondents and ask them to complete the instrument, identifying any items that they find difficult or impossible to answer, verifying that structured responses meet the three above-mentioned criteria, and evaluating the design in general. This takes time, but is well worth it. Once a survey has been administered, changing the items or responses is no longer an option.
Step 9: Administer the survey
Once the survey is designed, pre-tested, and revised, administer it. Be sure the survey instrument identifies prominently the due date and how or to whom it should be returned. Give telephone and fax numbers and e-mail addresses for anyone a potential respondent may call if he has questions about the survey. And be prepared for calls from these potential respondents.
Step 10: Receive returns
Once the survey is administered, be prepared to receive and store returns. If follow-up contacts are planned, be prepared to "check in" responses. This will permit identification of non-respondents for follow-up. Again, if the survey is anonymous, follow-up contacts will not be a possibility.
Steps in survey research: analyzing survey data
Step 1: Monitor response rate
From the date the survey is administered to the due date, monitor the response rate regularly. This will be especially important if there are plans for follow-up.
Step 2: Make follow-up contact
If the survey is not anonymous and interviewers are willing to make some extra effort to ensure as high a response rate as possible, make follow-up contacts. Such contacts may include sending a reminder postcard, sending out a second copy of the survey, or making follow-up contacts by telephone. If respondents are not anonymous, additional follow-up contacts may be desirable in order to clarify ambiguous responses to more important questions.
Step 3: Close out returns
It might seem obvious that survey returns should stop after the announced due date. They will not. Indeed, when setting the due date for a survey, it is wise to set that date at least one to two weeks before one actually needs to closeout the returns. Regrettably, there will be a few additional ones even after that close-out date. But, returns have to be closed out sometime.
Step 4: Review and edit survey returns
It is also important to review and edit survey returns before expecting anyone to enter them into a computer. Respondents do not always mark their intended responses unambiguously, and they frequently rewrite questions before answering them and add important marginal notes qualifying their responses. To ensure the most accurate data possible, review each return, clarifying ambiguous responses and correcting those that are inaccurate in light of respondent rewrites and marginal notes.
Step 5: Enter data
If a survey is very long and has very many respondents, it will be well worthwhile to have the data entered into a computer file. There is a wide variety of spreadsheet, database, and statistical software available for this purpose. It will be important to locate an experienced data entry operator to ensure that this job is done as quickly and accurately as possible. If experienced data entry staff are not available, some organizations have the same data entered twice into separate files to be reconciled.
Step 6: Analyze response
One of the first substantive sets of questions in analyzing survey results revolves around who responded and how representative they are of the larger population under study. Even if a random sample is used, it will be important to demonstrate its similarity to the population it represents.
Step 7: Select analytical techniques
In analyzing survey data, several techniques are available:
Step 8: Process data
A variety of spreadsheet, database, and statistical software packages are available for data entry and analysis. Spreadsheets such as Lotus 123, Excel, and Quattro Pro; database packages such as Dbase, Foxpro, and Paradox; and statistical packages such as SPSS and SAS will enable computer analysis and reporting. Most libraries and library-related organizations have ready access to experienced users of such software. Researchers in library schools or other departments of nearby colleges or universities are good resources for statistical analysis if that skill is not available in the library.
Step 9: Generate and display findings
Once data provided by respondents has been entered and processed, findings can be generated and displayed in ways that facilitate interpretation. To communicate overall results of a survey, a frequently chosen display method is to add summary data about responses to the survey form itself. For more detailed findings, ranked lists, tables, and cross-tabulations will be helpful. Charts and diagrams illustrating findings may also be helpful for analytical as well as presentation purposes.
Step 10: Draw conclusions
Once survey findings are generated and displayed, drawing conclusions about them can be shared with other participants in the community analysis project. Drawing conclusions about survey findings involves interpreting what they mean for the issues at hand and making practical recommendations about how to address those issues.
Steps in survey research: applying survey findings
Step 1: Write narrative and select graphics
The first step in applying survey findings is to produce a report about them. A survey report should include a concise, well-organized narrative that describes the issues, the sample, the survey instrument, how it was analyzed, the findings of the survey, and the conclusions.
Consider who will be the audience for this document: library staff, board members, local or state government officials, the general public. Knowing the intended audience, the author of this narrative should tell that group what it needs to know. Generally, the higher the rank of the audience, the briefer the report should be. If a lengthy document is unavoidable, provide a one to three-page executive summary.
In composing and designing the appearance of such a narrative, make it as readable and quotable a document as possible. Liberal use of white space, lists, and bullets help to make any document an "easy read."
Survey reports usually include selected tables and graphics to summarize and illustrate the findings. Appending a copy of the survey form to the report is a good idea.
Step 2: Produce and disseminate report
The next step in applying survey findings is to produce and disseminate the report. Copies should be distributed to all appropriate decision-makers and should be made available somehow to interested respondents. Sending a copy of the complete report to each respondent is inadvisable both because of the expense involved and because they are unlikely to read it. Issuing a press release about the survey to local newspapers, radio and television stations, and electronic mail listservs and bulletin boards is appropriate.
Step 3: Design and make presentation
In a more perfect world, producing and disseminating a report about a community analysis survey would suffice. In reality, though, merely publishing a report is insufficient. If the survey is to have any significant impact on library decision-making, someone must be prepared to design and make presentations about it. If consultants are involved, they may help to design such a presentation, to be made by appropriate library leaders -- director, staff, board. Efforts should be made to schedule such presentations to all groups of interested parties involved in the community analysis survey -- staff, board, friends, citizens, local government officials. Such presentations will give stakeholders and interested parties opportunities to focus on major findings and conclusions and to ask questions about the issues.
Step 4: Integrate results into decision-making
Once a report document is released and presentations about the survey have been made, library and community leaders are equipped to use survey results in their decision-making. When (and only when) this is achieved, the survey has fulfilled its purpose. Beyond applying the results to immediate decision-making, library leaders should review such survey projects critically and maintain records of lessons of experience that might benefit anyone who might repeat the survey in the future.
Step 5: Contribute to the profession
While it is important for community analysis surveys to inform local decision-making bodies about immediate or foreseeable issues, it is equally important for them to make a contribution to the profession of librarianship. Consultants and staff involved in such surveys should consider submitting a copy of the report to ERIC or another clearinghouse, writing articles for state and national professional publications, or contributing to sessions at state and national conferences. At the very least, the survey form and report should be made available to others who may benefit from examples and mistakes.