EIA STANDARDS MANUAL

 

Foreword

 

Interagency Standards

1.         2002-1, Office of Management and Budget Requirements

 

Systems Standards for Information Collection, Processing, and Dissemination

2.         2002-2, Information Technology (IT) Systems

3.         2002-3, Information System Documentation

 

Information Collection and Processing Standards

4.         2002-4, Survey Collection/Processing Planning, Design, and Testing

5.         2006-5, Frames Development and Maintenance

6.         2002-6, Respondent Contact Records (RCRs)

7.         2002-7, Response Rates and Imputation

8.         2002-8, Measuring Response Rates

9.         2002-9, Edit Procedures

10.     2006-10, Survey Data Evaluation

11.     2002-11, Data Quality Measures

 

Information Dissemination Standards

12.     2002-12, Policy for Releasing Information

13.     2006-13, Revisions

14.     2002-14, Dissemination of Information Based on Reported and Derived Data (Estimates)

15.     2002-15, Rounding

16.     2002-16, Codes, Abbreviations, Acronyms, and Definitions

17.     2002-17, Information Utility

18.     2002-18, Information Integrity

19.     2002-19, Accuracy Measures of Data and Estimates

20.     2002-20, Quality Assurance Reviews

21.     2007-21, Data Protection and Accessibility

22.     2008-22, Nondisclosure of Company Identifiable Data in Aggregate Cells

23.     2002-23, Reproducibility

24.     2002-24, Documentation for Public-Use Electronic Products

25.     2009-25, Statistical Graphs

 

Model Standards

26.     2002-26, Model Documentation

27.     2002-27, Model Archival

28.     2002-28, Proprietary Models

 

Public Comments on Agency Compliance with Quality Guidelines Standard

29.     2002-29, Public Comments on Compliance with Information Quality Guidelines

 

Business Process Documentation, Continuity of Operations, and Records Management Standards

30.     2002-30, Business Process Documentation

31.     2002-31, EIA Continuity of Operations

32.     2002-32, EIA Records Management

 

Additional Materials

1.      Standard 2002-4 Supplementary Materials, Survey Form Design Checklist

2.      Standard 2002-10 Supplementary Materials, Developing A Survey Data Evaluation Plan and Suggested Approaches for Evaluating Different Types of Survey Data Nonsampling Error

3.      Standard 2002-11 Supplementary Materials, Additional Suggested Data Quality Measures, and Periodic Quality Reviews

4.      Standard 2002-15 Supplementary Materials, Guidelines on the Standard for Rounding

5.   Standard 2002-16 Supplementary Materials, Abbreviations and Codes in Data Tables

6.      Standard 2002-20 Supplementary Materials, Quality Assurance Review Guidelines

7.      Standard 2008-22 Supplementary Materials, Guidelines for Implementation of a Disclosure Limitation Rule

8.      Standard 2009-25 Supplementary Materials, Guidelines for Graphs

9.      Standard 2002-26 Checklist A: Explanatory Model Documentation Components

10.  Standard 2002-26 Checklist B: Supplementary Model Implementation Documentation Components

11.  Standard 2002-26 Guidelines for Mathematical Specifications in Model Documentation

 

 

NOTE:  Some Related Information used in conjunction with EIA’s Standards is available only on EIA’s Intranet site accessible to persons working for EIA.

FOREWORD

 

Standards are used by the Energy Information Administration (EIA) in support of EIA’s Information Quality Guidelines to help ensure and maximize the quality, objectivity, utility, and integrity of information disseminated by EIA.  Standards document the professional basis upon which EIA expects to be judged by our stakeholders and the level of quality and effort expected in all our activities, including those of our contractors.  Standards provide a means for and assurance of consistency among and within activities conducted by EIA.  Finally, standards provide users of EIA products information on the methods and principles employed in the collection, analysis, and dissemination of information.

The standards in this Manual must be followed by all EIA staff and contractors and are effective as of the approval date indicated in each standard.  This Manual supersedes the previous Standards Manual issued by EIA.

This Manual contains 32 EIA standards designed for general application to EIA models, surveys, data systems, and information products.  EIA conducts over 50 surveys, utilizes over 100 data systems, operates over 25 models for analysis and forecasting, and disseminates information in numerous electronic and printed products.  The application of standards over such a wide diversity of activities, of course, requires judgment.

In those rare instances where the strict application of a standard is impractical or infeasible, an affected EIA program office should consult with the Statistics and Methods Group (SMG) to consider alternative methods of achieving the objective of the standard or to request an exemption from the standard.  

The SMG Director will decide whether an exemption should be granted based on a review of the circumstances as well as consultation with other members of EIA Senior Staff, as appropriate.  If an exemption is not granted by the SMG Director, the affected program office has the option of requesting an exemption granted by the EIA Deputy Administrator.

 

Date Issued:     September 29, 2002

                       

Guy F. Caruso

Administrator

Energy Information Administration

 

 

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-1

 

Title:  Office of Management and Budget Requirements

 

Superseded Version:  88-02-03

 

Purpose:  To identify Office of Management and Budget (OMB) requirements that are mandatory for Federal information programs and activities.

 

Applicability:  All EIA information collection and dissemination activities.

 

Required Actions:  The following OMB requirements must be adhered to:

·        Metropolitan Statistical Areas for use by all Federal statistical agencies to establish nationally consistent definitions for collecting, tabulating, and publishing statistics for a set of geographic areas

·        North American Industry Classification System (NAICS) for use by all Federal statistical agencies to classify business establishments (NAICS replaced the Standard Industrial Classification [SIC])

·        Data on Race and Ethnicity for use by all Federal agencies to establish a minimum standard for maintaining, collecting, and presenting data on race and ethnicity for all Federal reporting purposes

·        Standard Occupational Classification (SOC) User Guide for use by Federal statistical agencies to classify workers into occupational categories for the purpose of collecting, calculating, or disseminating data

·        Federal Statistical Confidentiality Order applies to statistical agencies or units as defined in the Order (within EIA the Order applies only to the Energy Consumption Division which the Order refers to as the Energy End Use and Integrated Statistics Division); the Order clarifies, and make consistent, government policy protecting the privacy and confidentiality interests of individuals or organizations which furnish data for Federal statistical programs

·        Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies (February 22, 2002, Volume 67 of the Federal Register [FR], pages 8452-8460); requirement for Federal agency information quality guidelines
·        Controlling Paperwork Burdens on the Public (August 29, 1995, 60 FR pages 44978-44996); OMB approval requirements for an information collection (e.g., survey) involving 10 or more persons in a 12-month period
·                           OMB Circular A-130, Transmittal Memorandum #4, Management of Federal Information Resources (November 28, 2000); requirement for all Federal agencies to establish a policy for the management of Federal information resources
·                           OMB Circular A-16, Coordination of Geographic Information and Related Spatial Data Activities (August 19, 2002); provides direction for Federal agencies that produce, maintain or use spatial data (i.e., information about places or geography traditionally shown on maps) either directly or indirectly in the fulfillment of their missions.
 
Related Information:  None
 
Approval Date:  September 26, 2002

 

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-2

 

Title:  Information Technology (IT) Systems

 

Superseded Version:  Standards 88-02-01, 88-03-02, 88-03-03, and 88-03-04

 

Purpose:  To ensure that the EIA IT systems are developed, maintained, and secured using best IT practices and that they adhere to established policies, standards, and procedures.

 

Applicability:  All IT systems at EIA.

 

Background:  EIA has standards for developing and documenting computer systems.  These standards ensure that systematic procedures are used when new IT systems are developed and existing systems are revised; that these systems meet business and functional requirements; and that the systems can be operated, modified, and backed up.  With the advent of the Internet, the EIA IT standards have been expanded to address data access and security issues arising in the Web environment.

 

Required Actions:  All EIA and contractor staff involved in developing and maintaining IT systems are required to follow applicable Federal Information Processing (FIPS), Department of Energy, and EIA-specific IT system standards, policies, guidelines, and procedures listed below.

 

Related Information: 

1.      Federal Information Processing Standards (FIPS)

2.      Department of Energy IT Standards

3.      2002-30, Business Process Documentation

4.      2002-31, EIA Continuity of Operations

5.      2002-32, EIA Records Management

6.      EIA Information Systems Security Policy

7.      EIA Password Generation, Protection, and Use Procedures

8.      EIA Operations Security Program Manual

9.      EIA Software Systems Life Cycle Standard

10.  EIA Web-Based Data Collection Systems Standard

11.  EIA Security Architectural Standard For Web-Based Data Collection Systems

12.  EIA Desktop PC Standard

 
Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-3

 

Title:  Information System Documentation

 

Superseded Version:  Standards 88-03-02 and 88-03-03

 

Purpose:  To ensure that EIA information systems are documented adequately to allow personnel unfamiliar with them to become knowledgeable about them and to operate them, if necessary.

 

Applicability:  All EIA information systems.

 

Required Actions:    

1.      If an EIA information system is a modeling system, it must follow the requirements of EIA Model Documentation Standard 2002-26.

 

2.      All other EIA information systems (e.g., survey systems, secondary data systems, and other systems supporting other key EIA business practices) must have documentation of all operations (both automated and manual) necessary to operate, maintain, and update the systems. 

  • All new information systems should have up-to-date Users and Developer’s Reference Manuals (see section 7.8 of DOE Systems Engineering Methodology).   
  • All information systems created prior to the approval date of this standard should have up-to-date Operations and Programmer’s Maintenance Manuals as required by the predecessor EIA Standards Manual.
  • When documenting an information system that is not a modeling system, the following topics should be covered, if applicable:  1) an overview of integrated manual and automated operations, workflow, interfaces, and personnel requirements; 2) frames development and updating; 3) sampling design and methodology; and 4) a detailed description of each step in collection/processing/dissemination, including distribution of survey materials, establishment of access to on-line reporting options, data collection methodologies, respondent contact handling, non-response procedures, data entry procedures, data editing procedures, error correction/adjustment procedures, estimation methodology, quality comparison procedures, withholding procedures, revision procedures, dissemination, and back-up and archiving procedures.  This information may be incorporated into the existing documentation or written as a separate document.

 

Related Information: 

1.         2002-30, Business Process Documentation

2.         2002-31, EIA Continuity of Operations

3.         2002-32, EIA Records Management

 
Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-4

 

Title:  Survey Collection/Processing Planning, Design, and Testing

 

Superseded Version:  Standard 88-04-01

 

Purpose:  To ensure that appropriate procedures are followed in the design, development, and testing of EIA survey programs and to obtain EIA and Office of Management and Budget (OMB) approvals.

 

Applicability:  All EIA survey  programs, including existing programs undergoing major revisions.  Major revisions include the following:  change from census to sample survey or vice versa; change in sample size of over 25% from the size of the previous survey collection; change in the type of reporting entity from which information is collected; major sample redesign; or substantive changes in the information collected.

 

Required Actions:

1.      EIA Review:  Before beginning the EIA approval process, a survey sponsor should consider the data needs, collection methods, processing system, dissemination plans, and estimated resource requirements.   A proposal for a new survey or a major revision of an existing survey should then be developed for review and approval by the Sponsoring Office Director and the Statistics and Methods Group (SMG) Director prior to submission of the proposal to OMB.  The proposal is typically documented in a supporting statement that includes:

  • The energy subject-matter background and rationale for a new/revised effort, including a discussion of uses of the data/data products.
  • An explanation of the reasons why the new/revised effort is needed at this time (e.g., Congressional request or directive, information requests from other public or private data users, EIA management decision, etc.).
  • A brief discussion of survey methodology, including the universe being covered, the type of  reporting entity being considered for data collection and analysis, sampling approach and sample size if the collection is to be a sample survey, frequency of collection, mode(s) of collection, topics of information to be collected, etc.
  • Information on how the data will be disseminated, including intended release times to the public and data dissemination media.
  • Initial estimates of the per-year resources (staff and contract dollars) and annual respondent burden of the proposed survey.

 

2.   OMB Review:  Any new survey collecting information from 10 or more persons in a 12-month period or any major revision to an existing survey generally requires OMB approval.  In addition, a renewal of OMB approval is required at least once every three years, even if no changes will be made to an existing survey.  The sponsoring office should coordinate with SMG to ensure all OMB requirements are satisfied.  The actions required to obtain OMB approval include:

  • Consulting with stakeholders (e.g., survey respondents and data users) who may be affected by the proposed action.  At a minimum, OMB requires publication of a Federal Register notice requesting public comments.  The Federal Register notice will be included on EIA’s Web site.  Consultations with stakeholders may also be expanded to include any other means for collecting comments, such as individual meetings, focus groups, presentations at conferences/workshops, cognitive testing, and pretests/pilot tests.
  • Preparing a clearance package for submission to OMB.  A clearance package contains a supporting statement (written according to OMB’s requirements), draft respondent letter, survey form, and instructions.
  • Reviewing the clearance package within EIA prior to submission to OMB.
  • Submitting the clearance package to OMB.  Simultaneously, issuing a second Federal Register notice requesting that public comments be sent to OMB.

 

Related Information: 

1.      Controlling Paperwork Burdens on the Public (8/29/95, 60 FR 44978-44996); OMB approval requirements for an information collection (e.g., survey) involving 10 or more persons in a 12-month period

2.      Standard 2002-4 Supplementary Materials, Survey Form Design Checklist

 

Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2006-5

 

Title:  Frames Development and Maintenance

 

Superseded Version:  Standard 2002-05

 

Purpose:  To require plans to ensure that necessary steps are taken to develop and maintain frames, and that a frame’s coverage is evaluated and documented.

 

Applicability:  All EIA surveys.

 

Required Actions for Creating New Frames:

 

A plan for constructing a frame file should be developed and implemented when a new frame needs to be created. The plan should include the following descriptions:

 

  • The frame’s major applications and frequency of use.

 

  • The extent that the new frame is expected to reflect the population(s) of interest.

 

  • Clear, standardized, and comprehensive definitions of terms and codes.

 

  • Data items necessary to support unique and efficient identification of elements in the frame.

 

  • Data items necessary for sampling purposes such as:

(a)    Information referencing a geographic area;

(b)   Measures of size and/or other sampling attributes;

(c)    Information relating to company structure that shows how frame elements are organized such as identification of sites, facilities, plants, outlets, subsidiaries, joint ventures and other corporate structures; and

(d)   Important industrial, technical, legal, or historical relationships between or within frame elements.

 

  • Identification, description, and evaluation of any sources, including any or all parts of an existing frame that may be used to construct a new frame.

 

  • What measures will be used to evaluate the quality of the new frame (under/over coverage, accuracy of sampling and identification information, timeliness, etc). 

 

  • Describe any important checks, counts, totals, or statistics that will be calculated from the data fields for the frame elements and the performance measures that will be produced to measure and monitor the frame with respect to quality (see Standard 2002-11 and supplement to 2002-11).  Use the measures to evaluate the frame to provide comparisons and/or some trend analysis between the old and updated frame.

 

  • The frequency and conditions for updating the new frame, the methods and sources that will be used, and a projected schedule and resource budget for this task (see Required Actions for Maintaining and Updating Existing Frames).

 

  • Any manual or automated matching/merging/un-duplication methods used. 

 

 

Required Actions for Maintaining and Updating Existing Frames:

 

Plans for maintaining and updating existing frames must be documented, implemented, and revised as needed.  Maintenance includes:

 

  • Identifying and adding new frame elements (births).
  • Identifying and coding frame elements that are no longer in business (deaths).
  • Checking and accurately coding whether frame elements are in or out of scope.
  • Checking and accurately coding whether frame elements are active or inactive, or other operational status classifications.
  • Revising the ownership, name, address, contact, etc. of any data field for a frame element.
  • Incorporating suggestions to improve frame quality.
  • Incorporating cross-reference information between surveys or survey systems.
  • Resolving conflicts between frame elements and other sources of information.
  • Recording the date of changes to the content of an element and documenting the person who made the change.

 

All frames should be maintained and updated based on a predetermined schedule. Updates may also occur anytime as described in the pre-determined frame update plan as information is discovered prior or after the pre-determined scheduled update.  The update process should be examined to consider the sources of the update information, the timing, and how the update is performed to avoid any potential source of bias to the survey.  Plans to update under a pre-determined schedule should document:

 

  • The frequency or the necessary conditions for updates.
  • The projected schedule and resource budget for an update.
  • Comprehensive searches that examined the most current information sources and how information from these sources will be incorporated into the frame
  • How sources of information exogenous to the frame will be evaluated and used, the timeliness and data limitations of any exogenous information sources, and their associated Internet links if available.
  • How updates will be incorporated into all appropriate files, mailing lists, and 

other survey systems that use the frame.

  • The measures that will be used to evaluate the updated frame for quality. These include under/over coverage, accuracy of sampling and identification information, and performance measures that will be produced to measure and monitor quality. 
  • Descriptions of any important checks, counts, totals, or statistics that will be calculated from the data fields for the frame elements. Use the measures to evaluate the frame to provide comparisons and/or some trend analysis between the old and updated frame.

 

Required Actions for Frames Retention:

 

To ensure that the most recent information is retained, the record for each element of the frame must have appropriate transactional variables such as the most current known status of the frame element and the last date the record was modified.  Before each major update of a frame, the current version must be archived electronically or be preserved in a manner that allows its recreation with minimal time and effort.  Historical versions of frames should be retained in accordance with each survey office’s record retention schedule and include:

 

  • all respondents whether active or inactive when the historical version was archived for retention ;
  • all information about an individual frame element, including all transactional and descriptive data needed for sampling or survey control.

 

A respondent's record should not be deleted from a frame file unless it was added to the frame because of either a factual mistake or administrative error.   If a survey office determines that an inactive respondent should no longer be included as part of the current frame file, then the survey office should first consult with the users of the frame to insure that this action results in no adverse consequences to users of the frame or to future frame updates, or any conflicting data requirements from other survey files.  Respondents no longer considered part of the frame should be appropriately coded in the status code field on the frame file so that the record does not appear in future archives of the frame.

 

Related Information:  None

 

Approval Date:  March 22, 2006

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-6

 

Title:  Respondent Contact Records (RCRs)

 

Superseded Version:  Standard 88-04-03

 

Purpose:  To ensure that all contacts with survey respondents are recorded.

 

Applicability:  All EIA surveys.

 

Required Actions:

1.      A Respondent Contact Record (RCR) form must be completed for each communication (e.g., Fax, telephone, mail, e-mail, or any other form of contact) with a respondent.  An RCR must be prepared regardless of who initiated the communication or the reason.

 

2.      The Respondent Contact Record should provide the following information:

  • Survey form number
  • EIA Respondent Identification Number
  • Responding entity’s reporting name
  • Responding entity’s contact person  information

a)      Name

b)      Telephone number

c)      Mailing address

d)      Fax number (if applicable)

e)      Internet e-mail address (if applicable)

  • Date of contact
  • Reporting period (if applicable)
  • Survey data collection staff member’s name
  • Reason for the contact (questions/issues addressed)
  • Information received from respondent
  • Actions taken, if any.

 

Related Information: 

1.      EIA Operations Security Program Plan for information on retention of survey records

 

Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-7

 

Title:  Response Rates and Imputation

 

Superseded Version:  Standard 88-04-02

 

Purpose:  To maximize survey response and minimize the combined variance and bias.

 

Applicability:  All EIA surveys.

 

Background:  Data disseminated by EIA are to be based on surveys with acceptable response rates.  The minimum desired level for the unweighted response rate (see Standard 2002-8) is 80 percent of the eligible respondents.  The minimum desired level for a response coverage rate (see Standard 2002-8) is 80 percent for “key” national-level totals as identified by the survey manager.  Imputation methods (including weight adjustment) are to be used to account for the data not submitted.

 

Required Actions:

1.      Systematic effort must be taken to maximize response rates, subject to resource constraints.  For a survey not meeting the minimum desired response level, the manager must consider steps to help ensure high response rates in the future:

  • Communicate ahead of time.  Notify respondents that they have been selected to report on a survey prior to the first collection period.  Provide advance notice so that selected respondents are prepared to report.
  • Use a cover letter signed by a high-ranking executive within the EIA survey office explaining the purpose of the survey, the value to the participant, and how the survey will be used. 
  • Increase follow-up communications after the questionnaire is distributed by use of  mail (e.g., letter or postcard), phone, e-mail, and fax to stimulate participation.
  • Develop and use a series of letters for late respondents and nonrespondents.
  • Develop lists of high-level executives in respondent companies to contact if other efforts are not successful.

 

2.      Summarize in the survey documentation any imputation methods (including weight adjustment) used to account for survey nonresponse.  Monitor response rates over time.  If response rates deteriorate, analyze the reasons for nonresponse to assess the effects on the survey and to improve future survey operations.  Also, analyze the imputation results to determine the changes in the variances due to imputation.

 

3.      If a response rate is below 80%, consider variance estimates and bias before disseminating the information

 

Related Information: 

1.      Standard 2002-8, Measuring Response Rates

2.      Standard 2002-11, Data Quality Measures

Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-8

 

Title:  Measuring Response Rates

 

Superseded Version:  None

 

Purpose:  To ensure that uniform procedures are followed to measure response rates.

 

Applicability:  All EIA surveys.

 

Required Actions: 

The following general formulae should be used to measure response rates:

 

·        Unweighted unit response rate = R / N

 

·        Response coverage rate (sometimes called weighted response rate) =

      Switi  /Switi

             R       N

      where   

 

R = the number of eligible units completing the survey and responding,

 

N = the number of eligible units in the survey,

 

Additionally for the response coverage rate formula:

 

·        The numerator represents a weighted total contributed by eligible responding units.

 

·        The denominator represents the corresponding weighted total for all eligible units in the survey.

 

·        wi = the (sampling) weight for the ith unit.

 

·        ti =  a measure of size or a “key” variable for the ith unit.

 

·        For sample surveys N = n,  the number of sampling units.

 

·        For census surveys wi =1 for all i.

 

In the above formulae,

  • The denominator includes all original survey units that were identified as being eligible, including units with pending responses with no data received, new eligible units added to the survey, and an estimate of the number of eligible units among the units of unknown eligibility.  The denominator does not include units deemed out-of-business, out-of-scope, or duplicates.
  • The numerator includes all survey units that have submitted sufficient information (based on criteria established by the survey staff) to be considered complete responses for the survey period.

 

The unweighted response rate is used to indicate the proportion of eligible reporting units that responded to the survey, while the response coverage rate is generally used to indicate the (sampling-weighted) proportion of an estimated national total that is contributed by respondents.  The response coverage rate provides a measure of the survey’s coverage of key variables.

 

Related Information:

1.      Standard 2002-7, Response Rates and Imputation

2.      Standard 2002-11, Data Quality Measures

 

Approval Date:  September 26, 2002

 

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-9

 

Title:  Edit Procedures

 

Superseded Version:  Standard 88-04-05

 

Purpose:  To ensure that survey data are subjected to appropriate edits, edit procedures are applied consistently, and edits are evaluated in a systematic and quantitative manner.

 

Applicability:  All EIA surveys.

 

Required Actions:

1.      The survey documentation must describe procedures for handling records that fail edit checks.  The documentation must explain the purposes of the edit checks, the decision procedures for resolving edit messages, and how edit message records are maintained.  Edit checks should address the following indicators of data quality (as applicable):

a.       Consistency 

b.      Range 

c.       Completeness

d.      Arithmetic/calculation  accuracy

e.       Comparability with other sources

f.        Validity of codes.

 

2.      For key edits as identified by the survey staff, maintain measures for the number of:

a.       Edit messages generated

b.      Edit messages resulting in revisions of the originally submitted data.

 

3.      Document the procedures for evaluating the performance of the edits and the results of any evaluations. 

 

Related Information:

1.      Standard 2002-2,  Information Technology (IT) Systems

2.      Standard 2002-7, Response Rates and Imputation

3.      Standard 2002-11, Data Quality Measures

 

Approval Date:  September 26, 2002

 

ENERGY INFORMATION ADMINISTRATION STANDARD 2006-10

 

Title:  Survey Data Evaluation

 

Superseded Version: Standard 2002-10

 

Purpose: To periodically evaluate survey data quality and limitations.

 

Applicability: All EIA surveys.

 

Required Actions: 

EIA surveys are subject to periodic assessments of data quality. An assessment, conducted either by the survey office or by the Statistics and Methods Group, should focus on one or more of the following topics:

 

1.      Identify and discuss potential sources of nonsampling error - i.e., coverage, measurement error, nonresponse, data collection and processing, survey methodology, imputation, estimation, and revision. Describe how those sources may affect the survey results and assess those effects, if any, over time.  (See Standard 2002-10 Supplementary Materials, Developing A Survey Data Evaluation Plan and Suggested Approaches for Evaluating Different Types of Survey Data Nonsampling Error.)

 

2.      Compare the survey data with similar data available from other sources. Assess differences, and identify and discuss specific reasons - e.g., coverage, survey methodology, definitions, time periods, etc.- for any differences. Also, identify areas of research for resolving any unexplained differences that persist over time.  Document the data sources, including Internet links, and any adjustments to the other data series for comparison with the survey data.

 

3.      Identify any changes to the survey over time. Discuss how those changes affect the estimates. Assess the effects on the data from those changes, including evaluating the effectiveness of those survey changes on the accuracy and quality of the data.

 

Related Information:

1.      Standard 2002-10 Supplementary Materials, Developing A Survey Data Evaluation Plan and Suggested Approaches for Evaluating Different Types of Survey Data Nonsampling Error

2.      Federal Committee on Statistical Methodology, Subcommittee on Measuring and Reporting the Quality of Survey Data.  Statistical Policy Working Paper 31, Measuring and Reporting Sources of Error in Surveys, Washington, D.C., June 2001.

3.      Examples of EIA Survey Data Evaluation Materials:

·        Energy Information Administration.  Accuracy of Petroleum Supply Data 2001.  “Petroleum Supply Monthly” Feature Article, DOE/EIA-0109 (2002/09),  Washington, D.C. September 2002.

·        Energy Information Administration.  Comparisons of Independent Petroleum Supply Statistics.  “Petroleum Supply Monthly” Feature Article, DOE/EIA-0109 (1999/12), Washington, D.C., December 1999.

·        Energy Information Administration.  Energy Consumption Series:  Residential Energy Consumption Survey Quality Profile.  DOE/EIA-0555(96)/1, Washington, D.C., March 1996.

 

Approval Date:  March 22, 2006

 

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-11

 

Title:  Data Quality Measures

 

Superseded Version:  Standard 88-04-06

 

Purpose:  To collect information for use in evaluating and improving the quality of EIA survey data. 

 

Applicability:  All EIA surveys.

 

Required Actions: 

Each survey should collect the measures below to support EIA-wise measures or should have a plan with milestones for collecting those measures not currently available. 

1.      Frames - Frame size, volatility (e.g., births, deaths, mergers), and estimated coverage.

 

2.      Response Rates - Unweighted unit response rate and response coverage rate (as defined in Standard 2002-8).

 

3.      Revisions of Disseminated Data - Percent difference between first disseminated data and the final disseminated data for key data series.

 

4.      Timeliness - Time from the close of the survey reference period until the dissemination of key data series.

 

5.      Data Edits - For key edits (as identified by the survey staff), the number of survey responses with that edit flag and the percentage of flagged responses that are revised.

 

6.      Sample Surveys - Sampling variances (or confidence intervals or relative standard errors).

 

7.      Imputation - Item imputation rates for key data items.

 

Related Information: 

1.      Standard 2002-11 Supplementary Materials, Additional Suggested Data Quality Measures and Periodic Quality Reviews

2.      Standard 2006-5, Frames Development and Maintenance

4.      Standard 2002-8, Measuring Response Rates

5.      Standard 2002-9, Edit Procedures

6.      Standard 2006-13, Revisions

7.      Standard 2002-19, Accuracy Measures of Data and Estimates

 

Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-12

 

Title:  Policy for Releasing Information

 

Superseded Version:  None.

 

Purpose:  To ensure that information intended for public release is released according to a dissemination plan that provides fair access to customers.

 

Applicability:  All EIA information products.

 

Required Actions:  

1.      To provide fair access to customers, major information products should follow publicly available release schedules.

·        Establish the schedule and mode for the release of information products.

·        Provide security procedures to prevent unauthorized premature release of the information.

 

2.      Sensitive information must be protected against any unauthorized pre-release and may be released only according to an established release schedule.  Information is considered sensitive by EIA if it substantively affects financial or energy markets.  

·        Prior to the official release time, sensitive information should not be read or discussed in public, or otherwise exposed on airplanes, restaurants, public transportation, or in other public places and/or in any media. 

·        The office responsible for producing the sensitive information must follow strict security procedures to protect the information from any unauthorized release prior to the official release schedule.  The security procedures for protecting sensitive information against an unauthorized release include:

a.       Establish and adhere to office lock-up procedures to physically secure the information and to restrict access to it.  Depending upon the circumstances, these security procedures may include placing locks on desk drawers, filing cabinets, and office room doors.

b.      Maintain a list of authorized individuals who have access to the information and designate a primary responsible party for safeguarding the information prior to its official release.

 

3.      Information products that are non-sensitive may be released prior to their release schedule if available earlier

 

Related Information: 

1.      DOE/EIA Operations Security Program Manual, including Section 3, “Fair Practice Data Disclosure,” Policy Reference No. 3, “News and Data Releases,” and Appendix F, “Embargoed Data Products and Release Guidelines”

 

Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2006-13

 

Title:  Revisions

 

Superseded Version:  Standard 2002-13

 

Purpose:  To provide EIA customers with information about revisions in disseminated data.

 

Applicability:  All EIA information products.

 

Required Actions:  

1.      Establish a policy for anticipated revisions. Show the date the policy became effective, and make it available electronically to users. Program offices should document any change to their revision policy and the effective date for each change. 

 

2.      The first dissemination of a data value in an information product should be identified as "preliminary" if revisions are anticipated in a subsequent dissemination.  Scheduled revisions to these values should be identified as "revised" (or “final,” where appropriate) when a revised value is disseminated.

 

3.      Preliminary and revised data must be identified through the following:

      A.  data value labeling such as:

            (i.) data marked "P" for preliminary, and

            (ii.) data marked "R" for revised); or,

 

      B.  text in the

            (i.) product title,

            (ii.) table titles,

            (iii.) headers or footnotes; or,

 

      C.  other text accompanying the release of the information product.  

 

         Historical databases accessed by the public should identify preliminary data or any other data that are subject to revision either by appropriately marking the data or by text in the product title, header or footnotes of the output table that shows the data.  After the revised data are released with the appropriate notice to users, no additional data labeling is required for final or revised data that are archived in an historical database. 

 

4.      When unscheduled revisions of data that have not been released as “final” are required due to previously unrecognized errors or respondent resubmissions, clearly identify the revisions and communicate the reasons for revising the data in a notice to data users (see item 3 above).

 

5.      Data previously released as “final” should not be revised without consulting with and notifying all other program offices within EIA.  The program office seeking to revise data previously released as “final” must contact the Revisions Coordinator in EIA’s Statistics and Methods Group (SMG) (Renee Miller as of March 2006) or, in his/her absence, the Director of SMG regarding the proposed change, reason(s) for the change, and the procedures for notifying representatives of the other offices.  A program office may proceed with its proposed change if no office objects to the revision within three weeks following notification.  If there is any disagreement between the program office and another EIA office concerning a proposed revision, an inter-office team will meet to discuss the proposed change and resolve the differences.

 

6.      Do not disseminate information if “errata sheets” are anticipated.

 

Related Information:  

1.      Standard 2002-7, Response Rates and Imputation

 

Approval Date:  March 22, 2006

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-14

 

Title:  Dissemination of Information Based on Reported and Derived Data (Estimates)

 

Superseded Version:  Standard 88-05-03

 

Purpose:  To ensure that EIA customers are aware of how information is produced; i.e., whether it is prepared solely from data reported on EIA surveys, is based on data from non-EIA sources, or is derived using models, and the methodology used to produce the information. 

 

Applicability:  All EIA information products.

 

Required Actions:

1.      For information based solely on reported data from surveys:

·        If the data reflect imputation or other adjustment methods, the methods must be described and be accessible from the data product.

·        For sample surveys, a brief discussion of sampling error and its potential effects on the data must be accessible from the data product.

·        Any data item that is a balancing item (i.e., it is not itself reported but is computed as the remainder after other components are taken into account) must be identified.

·        The sources and known limitations of any information from sources outside of EIA must be described.

 

2.      For products with information derived using models:  

·        Forecasts and derived estimates must be clearly identified (e.g., in the title, header, footnote, or text of the information product).  If only selected cells in a table are estimates, the cells may be marked “E”.  A description of the forecasting model or derivation procedure must be accessible from the product along with any available evaluation of its accuracy.  Any total or subtotal comprised partially or wholly of estimates must be labeled as well.

·        Any number originally disseminated as a projected value must be identified in subsequent releases. 

 

Related Information:

1.      Standard 2002-7, Response Rates and Imputation

2.      Standard 2002-19, Accuracy Measures of Data and Estimates

3.      Standard 2002-26, Model Documentation

4.      Standard 2002-27, Model Archival

5.      Standard 2002-28, Proprietary Models

 

Approval Date:  September 26, 2002

                                              


ENERGY INFORMATION ADMINISTRATION STANDARD 2002-15

Title:  Rounding

Superseded Version:  88-05-07        

Purpose:  To ensure consistency in rounding.

Applicability:  To all data released by EIA.

Required Actions:

1.      To round a number to n digits (decimal places), add one unit to the nth digit if the (n+1) digit is 5 or larger and keep the nth digit unchanged if the (n+1) digit is less than 5.

Note:  This is a simplified description of what should be done when rounding.  See Guidelines on the Standard for Rounding for further details on how to round, including rounding negative numbers.

 

2.      All calculations should be carried out prior to rounding.  In particular, tabulations to produce summary data and computations performed for purposes of estimating standard errors should be done on data as collected; i.e., no rounding should take place prior to the completion of these kinds of tabulations.

 

3.      Sums of the column (row) values in a table should be derived by using the unrounded column (row) values, with appropriate rounding of the column (row) total being done after its derivation.  To handle the problem of the rounded column (row) values not adding to the rounded column (row) total, a footnote should be used such as, "Totals may not equal the sum of their components because of independent rounding."

 

Related Information:

1.      Standard 2002-15 Supplementary Materials, Guidelines on the Standard for Rounding

 

Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-16

 Title:  Codes, Abbreviations, Acronyms, and Definitions


Superseded Version:  88-02-02


Purpose:  To ensure consistency in the use of codes, abbreviations, acronyms, and definitions throughout EIA.


Applicability:  All EIA information products.


Required Actions:


1.      Codes, Abbreviations, and Acronyms

 

2.      Definitions

  • EIA has a Glossary of established definitions for commonly used energy terms.  When using a term in the Glossary, use the established definition.
  • If you have questions, would like to create a definition for a new energy term, or would like to propose a revision to an established definition, contact the Glossary Coordinator, Grace Sutherland in EIA’s Statistics and Methods Group for procedures on coordinating with affected EIA staff and customers.

Related Information:  None

Approval Date:  September 26, 2002

 

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-17

Title:  Information Utility

Superseded Version:  None

Purpose:  To maximize utility (i.e., usefulness to intended users) of energyinformation products disseminated by EIA. 

Applicability:  All EIA information products.

Required Actions:

1.      Each program office must ensure the continuing usefulness of the information products it disseminates by:

  • Monitoring information needs of its customers and responding to customer feedback.
  • Reviewing on a regular basis the frequency and mode of access of information products.
  • Assessing customer input.
  • Developing new information sources and release methods or revising existing information collection methods, models, and information products, when appropriate.

 

2.      EIA must conduct periodic customer surveys to evaluate how effectively information meets users’ needs.

 

3.      Users of EIA information must have access to explanatory materials on possible sources of error, data definitions, collection methodology, estimation and imputation, and any data adjustments to assist in the understanding and interpretation of information disseminated by EIA.

 

Related Information: 

1.      Standard 2002-4, Survey Collection/Processing Planning, Design, and Testing

2.      Standard 2002-20 Supplemental Materials, Quality Assurance Review Guidelines

 

Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-18

Title:  Information Integrity

Superseded Version:  None.                                                                                                                                

Purpose:  To ensure that EIA data, analyses, and forecasts are:  1) secure from unauthorized access or revision or destruction, and 2) not compromised through corruption or falsification.

Applicability:  All data submitted to EIA by survey respondents or obtained by EIA from secondary sources and all data, analyses, and forecasts releasable to the public by EIA.

Required Actions:

1.      EIA will ensure that:

·        Individually identifiable EIA survey data are protected.

·        EIA data systems and electronic products are protected from unwarranted intervention.

·        Only EIA authorized personnel can access EIA information resources.

·        EIA data files, network segments, servers, and desktop PCs are electronically secure from malicious software and intrusion using best available information resource security practices, which are periodically monitored and updated.

 

2.      Options for respondents to submit forms must be consistent with the Draft EIA Policy on Security of Sensitive Unclassified Survey Data Transmitted to EIA.

·        EIA will provide secure electronic methods for its survey respondents to use when they transmit sensitive unclassified survey data to EIA.

·        EIA will inform its survey respondents about the incoming data transmission methods that EIA can accommodate and which are secure.

·        EIA Office Directors may request waivers to this policy.  Waiver requests will be reviewed by the Information Technology Council and decided by the Administrator.

 

3.      EIA will ensure controlled access to data sets so that only specific, named individuals working on a particular data set can either read, write, or both read and write that data set.  Data set access rights are to be periodically reviewed by the project manager responsible for that data set in order to guard against unauthorized release or alteration.

 

4.      EIA will ensure that all changes to EIA data are documented and that they are both warranted and follow EIA established data revision procedures.

 

Related Information:                                                                                                      

1.      DOE/EIA Operations Security Program Plan

2.      EIA Cyber Security Program Plan

3.      EIA Information Systems Security Policy

4.      EIA Password Generation, Protection, and Use Procedures

5.      EIA Security Architectural Standard for Web-Based Data Collection Systems

6.      Standard 2002-13, Revisions

7.      Standard 2007-21, Data Protection and Accessibility

8.      Standard 2002-30, Business Process Documentation

9.      Standard 2002-31, EIA Continuity of Operations

 

Approval Date:  September 26, 2002

 

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-19

 

Title:  Accuracy Measures of Data and Estimates

Superseded Version:  None.                                                                                                                                

Purpose:  To provide users with information concerning the accuracy or reliability of survey data and estimates.

 

Applicability:  All EIA information products.

 

Required Actions: 

1.      Revision and sampling error information, when applicable, must be available for an EIA product. 

 

·        Revision error (applicable to all disseminated data and estimates when they are subsequently revised) is the difference (often expressed as a percentage) between an initial release of data/estimates and its corresponding final disseminated data/estimates for key data series.  The office disseminating data/estimates should provide revision error information designed to help users better understand the variability between initial key data/estimates and final key data/estimates.  Some ways to present revision error information include the average revision error, the maximum revision error, or the distribution of revision errors during a specified time period.  Also, when presenting a measure of revision error, an indication should be made with respect to what data/estimates the error applies (e.g., the difference between the first estimate and the final data, the difference between preliminary and final data). 

 

If revision error for a key data series shows an initial release is an unreliable indicator of the final data/estimate, consider whether publishing the data/estimate with a measure of revision error or withholding the initial data/estimate is the best way to serve EIA’s customers.

          

·        Sampling error (applicable only to data/estimates derived from a sample surveys).  Include in products the sampling errors or tools to calculate these errors together with a description of sample selection and estimation procedures.

 

If the sampling error for a key data series shows the data may be an unreliable measure, consider whether publishing the data with sampling error information or withholding the data is the best way to serve EIA’s customers.

 

2.      Nonsampling errors affect all data/estimates and may occur for a number of reasons  including incomplete coverage of the units of interest, nonresponse, respondent difficulties in understanding/reporting, mistakes in recording/coding data, and other errors of collection, response, coverage, and estimation.  While nonsampling error is generally difficult to measure, a discussion of nonsampling error and its potential impact must be available for an EIA information product. 

 

3.      If the revision error, sampling error, or nonsampling error discussion is too complicated or extensive to include in an information product, a reference or link to a Web product with the required information should be provided.

 

Related Information:

 

1.      Standard 2002-7, Response Rates and Imputation

2.      Standard 2002-11, Data Quality Measures

3.      Standard 2002-13, Revisions

4.      Standard 2002-14, Dissemination of Information Based on Reported and Derived Data (Estimates)

 

Approval Date:  September 26, 2002

ENERGY INFORMATION ADMINISTRATION STANDARD 2002-20

 

Title:  Quality Assurance Reviews

Superseded Version:  None.                                                                                                                                

Purpose:  To ensure pre-dissemination quality assurance reviews of all EIA information products.

 

Applicability:  All EIA information products.

 

Required Actions:

1.      Before public dissemination, an EIA information product must undergo a quality assurance review within the originating office.  (For guidelines on the pre-dissemination review, see the Standard 2002-20 Supplemental Materials.)

 

2.      The originating office may submit an information product for review by other EIA offices before dissemination.

 

3.      As needed, a program office may use EIA’s Independent Expert Review (IER) program to provide technical advice from highly qualified, nationally and internationally recognized subject matter experts.  For additional information about the program, contact EIA’s IER Coordinator, Preston McDowney, of EIA’s Statistics and Methods Group.

 

Related Information:

1.      Standard 2002-20 Supplemental M