How the 1999 CBECS Was Conducted

1999 Commercial Buildings
Energy Consumption Survey:
How the Survey Was Conducted

The Commercial Buildings Energy Consumption Survey (CBECS) is conducted quadrennially by the Energy Information Administration (EIA) to provide basic statistical information about energy consumption and expenditures in U.S. commercial buildings and information about energy-related characteristics of these buildings. The survey is based upon a sample of commercial buildings selected according to the sample design requirements described below. A “building,” as opposed to an “establishment,” is the basic unit of analysis for the CBECS because the building is the energy-consuming unit. The 1999 CBECS was the seventh survey that had been conducted since 1979.

The CBECS is conducted in two data-collection stages: a Building Characteristics Survey and an Energy Suppliers Survey. (For the 1999 CBECS, the Energy Suppliers Survey was initiated only if the respondents to the Building Characteristics Survey could not provide the energy consumption and expenditures information.) The Building Characteristics Survey collects information about selected commercial buildings through voluntary interviews with the buildings’ owners, managers, or tenants. In the 1999 survey, these data were collected using Computer-Assisted Telephone Interviewing (CATI) techniques. (In previous CBECS cycles, the information was collected during personal interviews.)

During the Building Characteristics Survey, respondents are asked questions about the building size, how the building is used, types of energy-using equipment and conservation measures that are present in the building, the types of energy sources used, and for the 1999 survey, the amount and cost of energy used in the building. Building respondents could provide the consumption and expenditures information for approximately 60 percent of the sampled buildings. For the remaining 40 percent of buildings, the energy supplier names, addresses and account numbers were obtained.

Upon completion of the Building Characteristics Survey, the Energy Suppliers Survey is initiated for those cases that did not provide consumption and expenditures information. This Suppliers Survey obtains data about the building’s actual consumption of and expenditures for energy from records maintained by energy suppliers. These billing data are collected in a mail survey conducted under EIA’s mandatory data collection authority. A survey research firm, under contract to EIA, conducts both the interviews for the Building Characteristics Survey and the mail survey for the Energy Suppliers Survey.

This document describes how the 1999 Commercial Buildings Energy Consumption Survey (CBECS) was conducted. It includes information about what constitutes a commercial building for the CBECS, how the questionnaire was designed, the type of interviewer training that occurred, and how the data were collected (including procedures to minimize nonresponse), and edited. Other information about survey procedures and methodology can be found in "Technical Information on CBECS" and "Background Information on CBECS".

Highlights of Changes in the 1999 CBECS

Longitudinal revisit of the 1995 CBECS sample.

New construction sample (buildings constructed between 1995 and 1999) limited to buildings over 10,000 square feet.

Data collected by computer assisted telephone interview rather than personal interview.

Energy consumption and expenditures information collected during the Building Survey and only collected from the energy suppliers when it was unavailable at the building level.

Worksheets mailed to the respondents before the telephone interview to assist them with difficult questions.

More detailed information collected about the principal building activities.

More information collected about the presence of office and medical equipment in the building.

A building is eligible for the CBECS if it meets three criteria: building definition, building use, and building size. Determining this eligibility is a two-step process. In 1999, the first step occurred during the development of the sample and the second step occurred during the interview with the building respondent.

Criterion 1—Building Definition: The definition of a building was the same one used in the past: a structure totally enclosed by walls that extend from the foundation to the roof that is intended for human access. Therefore, structures such as water, radio, and television towers were excluded from the survey. Also excluded were: partially open structures, such as lumber yards; enclosed structures that people usually do not enter or are not buildings, such as pumping stations, cooling towers, oil tanks, statues, or monuments; dilapidated or incomplete buildings missing a roof or a wall; and, beginning with the 1995 CBECS, stand-alone parking garages. There is one exception to the building definition criterion—structures built on pillars so that the first fully enclosed level is elevated are included. These types of buildings are included because such buildings fall short of meeting the definition due only to the technical shortcoming of being raised from the foundation. They are totally enclosed, are used for common commercial purposes, and use energy in much the same way as buildings that sit directly on a foundation.

Criterion 2—Building Use: In order to be included in the CBECS, a building had to be used primarily for some commercial purpose; that is, more than 50 percent of the building’s floorspace must have been devoted to activities that were neither residential, industrial, nor agricultural. The primary use of the sampled building governed whether the building was included in the CBECS. Beginning with the 1995 CBECS, there was one exception to this criterion: commercial buildings on manufacturing sites were considered out of scope. (In previous CBECS, if a commercial building, such as an office building, was located on a manufacturing site, it would have been considered in scope.)

Examples of nonresidential buildings that were not included in the CBECS samples are:

farm buildings, such as barns, (unless space is used for retail sales to the general public);

industrial or manufacturing buildings that involve the processing or procurement of goods, merchandise, or food (again, unless space is used for retail sales to the general public);

buildings on most military bases; buildings where access is restricted for national security reasons;

single-family detached dwellings that are primarily residential, even if the occupants use part of the dwelling for business purposes; and

mobile homes that are not placed on a permanent foundation (even if the mobile home is used for nonresidential purposes).

Criterion 3—Building Size: In general, a commercial building had to measure more than 1,000 square feet (about twice the size of a two-car garage) to be considered in scope for the 1999 CBECS. The one exception to the general size criterion was buildings constructed between 1995 and 1999. These new buildings had to be 10,000 square feet or more. (New small buildings will be imputed for the 1999 CBECS.) This building size criterion was met in two successive size cutoffs, which were enacted during the sample development and interviewing stages. During the development stage, buildings judged to be less than 500 square feet were not enumerated. Then during the interviewing stage, interviewers asked screening questions designed to terminate the interview when the square footage was reported to be 1,000 square feet or less for buildings constructed prior to 1995 and less than 10,000 square feet for buildings constructed between 1995 and 1999.

Data collection encompasses several phases, including: (1) designing the questionnaire, (2) pretesting the questionnaire, (3) training supervisors and interviewers, (4) conducting interviews, (5) minimizing nonresponse, and (6) processing the data.

Although a set of core questions remained the same or very similar to those used in previous surveys, the 1999 Building Questionnaire was redesigned by EIA to improve data quality and to allow the data to be collected by use of Computer-Assisted Telephone Interviewing (CATI) techniques. The CATI instrument was programmed by EIA using Blaise software.

Two pretests were conducted prior to fielding the full-scale survey to assess the major changes that were planned for the 1999 CBECS. The first pretest focused primarily on the procedures that were new to the 1999 CBECS, such as identifying the correct respondent using telephone interviewing methods; delivery and contents of the package of advance materials for respondents; ability to conduct the interview for the correct building without viewing the structure; and the ability to collect the consumption and expenditures data at the building level. The second pretest provided an opportunity to test procedures that changed as a result of the first pretest findings and to evaluate the CATI questionnaire.

The CBECS building questionnaire is a complex instrument designed to collect data during a telephone interview with a respondent at the building site. Well-trained interviewers are imperative to the collection of technical information. Training for the 1999 CBECS included two in-person training sessions: one session for the telephone staff supervisors and team leaders and a separate seven-day session for the interviewers. Because this was the first time collecting CBECS data using CATI, all interviewers were trained in the general use of the computer and in interviewing and administering the CATI questionnaire. Training sessions included lectures, slide presentations, and small group sessions where the interviewers practiced administering the questionnaire using computers. In addition, a variety of ad-hoc training sessions were held throughout the data collection period that covered specific topics such as refusal conversion, handling of special building types and understanding technical questions about the heating, cooling and lighting systems within a building. EIA personnel participated in all training sessions, provided an overview of the CBECS, and presented key 1999 CBECS energy concepts.

CBECS interviews began January 2000, and ended September 2000. The data were collected by the survey contractor’s telephone research center staff. This staff consisted of approximately 100 telephone interviewers. Each building in the sample began with a prescreener and then proceeded with the extended interview.

Prescreener: Initial contacts with the building representatives were made through a prescreening telephone call to the respondent of the previous 1995 survey or, in the case of a new building, to a contact provided by F.W. Dodge. The purpose of this prescreener was four-fold. First, the sampled building was validated to determine if the structure was the same one that was interviewed in 1995, or in the case of a new structure, that it was the sampled building and that it met the eligibility requirements of the survey. Second, a respondent who was knowledgeable about the building was identified. Third, an address was obtained for the purpose of mailing a packet of introductory materials for the interview, and fourth, an appointment was scheduled to conduct the interview in approximately 10 days from the prescreening call.

Mailed Materials: An introductory packet of materials was mailed to the respondents in order to further explain the purpose of CBECS and to assist them in obtaining answers to some of the more technical questions about the buildings and in collecting the consumption and expenditures data needed for the actual interview. The packet included: (1) an introductory letter, signed by a representative of EIA, explaining that the building had been selected for the survey, introducing the survey contractor, assuring the building manager that the data would remain confidential, and discussing the uses and needs for the CBECS data in setting national energy policies; (2) three worksheets with core questions that the respondent may need advance knowledge of; (3) a reference guide to assist with CBECS definitions; (4) several examples of the type of CBECS data that are published; (5) a sheet of Frequently Asked Questions; and (6) a letter from the survey contractor. To protect the respondents’ confidentiality, the packet of material was addressed and mailed by the survey contractor.

Extended Interview: Each interview began with a series of screening questions designed to verify the building’s address and eligibility for the survey. Respondents were asked about the building as a whole rather than individual establishments located within the building. The CBECS questions covered: physical characteristics such as building activity, size and year constructed; building use patterns such as operating hours, number of workers, ownership and occupancy questions; types of energy-using equipment such as heating, cooling, refrigeration, lighting and office equipment; conservation features and energy management practices; types of energy used in the building and whether that energy is used for heating, cooling, water heating, cooking, manufacturing or electricity generation; and the amount of and expenditures for energy used in the building in 1999.

The 1999 CBECS was the first CBECS that collected energy consumption and expenditures information from the building respondents instead of the energy suppliers. This information was obtained for approximately 60 percent of the buildings. For the other 40 percent of buildings, the energy suppliers were contacted during a mail survey.

The completed building interview lasted an average of 40 minutes. The average time to obtain each completed interview, including interviewer preparation, prescreening, interviewing and callbacks, was approximately 6 hours.

Interviewer Supervision: The main procedure used to ensure that the interviews were conducted as intended was silent monitoring of the telephone interview. This involved listening to the telephone conversation and watching the interviewer’s entries using remote equipment. Interviewers were informed that silent monitoring of their work would occur at any time during an interview, but they were not informed when it would occur. Either the telephone center supervisors or EIA staff observed approximately 4,440 silent monitoring sessions. If an interviewer was found to be having difficulty, they were provided extra training or one-on-one assistance. If that did not correct the problems, they were removed from the survey.

Several approaches were employed in an effort to minimize nonresponse. These included: (1) mailing an advance packet of materials to building owners or managers; (2) establishing a toll-free “hot-line” number to address respondents’ concerns or questions; (3) mailing personalized letters to documented refusals; (4) using extensive telephone tracing in order to locate telephone numbers for problem buildings; (5) varying the day and hour when respondents were called in order to reach those who were initially unavailable; and (6) employing specially trained telephone interviewers in nonresponse conversion techniques. These approaches dealt with the two categories of nonresponse for CBECS: refusals and cases where the respondent was unavailable during the field data collection period or it was difficult to locate a telephone number for the building.

The initial processing of the CBECS data occurred at the survey contractor’s home office and included editing the questionnaires, calculating the survey weights for each building, and masking the data for confidentiality before it was transmitted to EIA. Final data preparation occurred at EIA and consisted of checking the data for internal consistency, checking the data against data from previous surveys, conducting imputation procedures for missing data, and preparing cross-tabulations for release to the public. Additionally, for those buildings that could not provide the energy consumption and expenditures data, authorization forms were requested that permitted the survey contractor to contact the energy supplier for that information. Upon receipt of the authorization form, the information was entered into the CATI questionnaire.

Data Editing: While data editing for the 1999 CBECS Building Characteristics Survey occurred at several points during data collection and processing, the primary editing occurred during the CATI interview. CATI controlled for skip pattern errors and the entry of ineligible codes. It also permitted some editing of the data as the interview proceeded. Arithmetic checks were conducted for some items, consistency checks between items prompted interviewers to confirm unlikely responses, and internal regression programs allowed for on-line editing of the energy consumption and expenditures. Most edits were “soft” meaning that they could be suppressed after the interviewer verified the information with the respondent, but for a few crucial questions or “impossible” data combinations, the interview could not continue until the edit was resolved. Additional editing occurred upon completion of the telephone interview that resolved the edit checks, updated data based on clarifying comments that were provided by the interviewers and incorporated responses to open-ended questions after EIA’s review of the questions. In some hard-to-resolve cases, EIA personnel provided technical guidance on how to reconcile questionnaire responses. The final data editing occurred at EIA during the review of data.

Item Nonresponse: Prior to publication of the 1999 CBECS, adjustments were made for item nonresponse. Item nonresponse is a specific piece of information that is missing in an otherwise completed interview. In the case of building interviews, the usual cause for item nonresponse was that the building representative lacked the knowledge to answer a questionnaire item.

Item nonresponses were treated by a technique known as “hot-deck imputation.” In hot-decking, when a certain response is missing for a given building, another similar building (a donor building) is randomly chosen to furnish its reported value for that missing item. That value is then assigned to the building (a receiver building) with the item nonresponse. This procedure was used to reduce the bias caused by different nonresponse rates for a particular item among different types of buildings.

Donor buildings had to be similar to the nonrespondent in characteristics correlated with the missing item. The characteristics that were used to define similar depended on the nature of the item to be imputed. The most frequently used characteristics were principal building activity, floorspace category, year constructed category and Census region. Other characteristics (such as type of heating fuel and type of heating and cooling equipment) were used for specific items. To hot-deck values for a particular item, all buildings were first grouped according to the values of the matching characteristics specified for that item. Within each group defined by the matching variables, donor buildings were assigned randomly to receiver buildings.

As was done in previous surveys, the 1999 CBECS used a vector hot-deck procedure. With this procedure, the building that donated a particular item to a receiver also donated certain related items if any of these were missing. Thus, a vector of values, rather than a single value, is copied from the donor to the receiver. This procedure helps to keep the hot-decked values internally consistent, avoiding the generation of implausible combinations of building characteristics.

EIA does not receive or take possession of the names or addresses of individual respondents or any other individually identifiable energy data that could be specifically linked with an individual sample building or building respondent. All names and addresses are maintained by the survey contractor for survey verification purposes only. Geographic identifiers and NOAA Weather Division identifiers are not included on micro-data files delivered to EIA. Geographic location information is provided to EIA at the Census division level. In addition, building characteristics that could potentially identify a particular responding building, such as number of floors, building square footage, and number of workers in the building, are masked to protect the respondent’s identity.