[Report on the Assessment of Electronic Government Information Products]
[Appendices]
[Appendix G - Electronic Government Information Products Assessment Agency Meetings Held and Discussion Questions]
[From the U.S. Government Printing Office, www.gpo.gov]

Appendix G

Electronic Government Information Products Assessment Agency Meetings
Held and Discussion Questions


   Electronic Government Information Products Assessment Agency Meetings Held

Agency                                                       Number of Attendees


Department of Health and Human Services                                       10
Meeting Date:  September 14, 1998


Supreme Court of the United States                                            11
Meeting Date: September 15, 1998


Environmental Protection Agency                                               12
Meeting Date: September 17, 1998


Department of Education                                                        5
Meeting Date: September 22, 1998


Department of Commerce                                                         8
Meeting Date: September 23, 1998


National Archives and Records Administration                                   7
Meeting Date: September 24, 1998






              Electronic Government Information Products Assessment
                         Agency Meeting Discussion Questions


1. Does your agency have preferred medium and format standards for Government 
electronic information products at the permanent public accessibility stage?  If 
so, what are the top three?  What factors does your agency consider in 
determining preferred standards (e.g., user needs, agency's dissemination 
requirements or policies, cost, security etc.)?  What about specific standards 
for CD-ROMs as they relate to user documentation, installation, search 
software, etc.?


2. Can you give us any examples of particularly innovative and creative product 
formats, mediums and/or online approaches?  We have in mind formats, mediums, or 
online approaches that may well point to the wave of the future, not only for a 
particular product but also for other kinds of products, yet is neither an 
agency-mandated standard nor even a common agency practice.


3. Is there any difference between your agency's preferences for mediums and 
formats as opposed to the preferences of intermediary distributors?  If so, what 
are those differences, and why are the two preferences different?


4. Has your agency involved external user groups in assessing the value and 
effectiveness of the dissemination of Government electronic information 
products?  If so, are there formats and mediums that seem particularly 
appropriate for public dissemination to users who may be economically, 
technically, or physically disadvantaged?


5. Does your agency follow any internally or externally prescribed guidelines 
for the presentation and organization of products in online formats?  If so, 
what are they (e.g., WWW Federal Consortium, FIPS Guidelines, agency or 
departmental publication specifications or guidelines)?


6. Has your agency undertaken any kind of cost benefit analyses for producing or 
creating products in preferred or emerging formats, mediums, or online 
approaches for distribution to the FDLP?  If so, which ones appear to be the 
most cost-effective? 


7. What factors does your agency consider in deciding to create or retain 
products in more than one medium?  Is this a common agency practice?


8. Are there trends with respect to migrating specific families of products from 
pre-electronic mediums to electronic mediums or formats?  For example, are 
loose-leaf publications, training manuals, annual reports, conference 
proceedings, newsletters, rules and regulations, scientific journals, etc., 
targeted for migration to a particular medium?  If so, which mediums and formats 
are used for specific families of products?


9. Has your agency identified any medium and format standards that seem 
particularly appropriate for use throughout a product's entire information life 
cycle, not just at one stage (i.e., creation, storage and retrieval, 
communication and dissemination, archiving and disposition) for electronic 
Government information products?  If so, which ones?
 
 
10. How do you determine whether a product should be made permanently publicly 
accessible when you create or produce it?  If so, what criteria do you use to 
determine which products will be permanently publicly accessible?  Can you give 
us any examples of how you ensure permanent public accessibility for a given 
product?
 

11. Does your agency routinely provide locator tools (e.g., GILS or specific 
agency locators) to enhance access to information sources and services available 
to external users and customers?  If so, is this an official policy, common 
agency practice, or both?


12. Are there trends for facilitating public access to your agency products by 
including them in broad electronic Federal Government information services such 
as GPO Access, LOC Thomas, and NTIS FedWorld?  Are you using any particular 
guidelines to facilitate that decision, and if so, what are they?


                 Summary of Responses to Agency Meeting Questions


1. Does your agency have preferred medium and format standards for Government 
electronic information products at the permanent public accessibility stage?  If 
so, what are the top three?  What factors does your agency consider in 
determining preferred standards.  

Agencies reported using the following electronic mediums most often: CD-ROM, 
Internet, and Bulletin Board System (BBS).  The most frequently used formats 
include:  

* HTML, PDF, ASCII 
* TIFF, GIF, JPEG
* Lotus/Domino

In determining medium and format standards, agencies consider the amount of 
information or files used, the timeliness of the information (e.g., more recent 
products or publications often placed on the web), and user needs for easy and 
quick access to information.  For example, some agencies are looking to Windows 
as an interface for CD-ROM products since the public is used to seeing and using 
Windows.


2. Do you have any examples of particularly innovative and creative product 
formats, mediums, and/or online approaches?  We have in mind formats, mediums or 
online approaches that may well point to the wave of the future for not only a 
particular product but also for other kinds of products, yet is neither an 
agency-mandated standard nor even a common agency practice.

Almost all the agencies interviewed are exploring a wide range of innovative and 
creative format, medium, and web applications.  Below is a sampling of some of 
the interesting online approaches and formats used by the agencies interviewed. 

* Data warehousing: Taking information not previously publicly accessible and 
integrating  it into an online format.  The format used is an Oracle database 
using SQL to query.

* Interactive Geographic Information System (GIS).  Provide mapping capability 
through GIS combined with regulatory information to create dynamic maps.  

* Online catalog of all products on the agency's website, using Oracle with a 
ColdFusion interface with their search engine (Verity).

* Creating user guides for CD-ROMs as pop-up HELP or short Read-Me files so 
users will be more inclined to use HELP.

* Live "real-time" web casting of selected speeches.  Format: RealPlayer 
software, available free from the web. 

* Radio news broadcast news service.  Provides daily radio sound-bits for news 
reports. Format: RealAudio RealPlayer software and WAV file format for 
downloading.

* Real-time forecasting of air pollution levels for 22 states from one site. 
Format: animated GIFs created by nonproprietary software designed by computer 
center in North Carolina.  

* Multimedia CD-ROM (i.e., art, music, animation, film, and video).

* Searchable electronic inventory of all proposals funded over the last 30 years 
so the agency can analyze its own information and make it available to others.  
Format: legacy database put into WAV database using HTML on the fly. 

* Documents stored in TIFF format for image and textual data.  As customers 
request documents, the agency converts them to PDF so customer can download.


3. Is there any difference between your agency's preferences for mediums and 
formats as opposed to the preferences of intermediary distributors? 


Generally, agencies indicated that intermediary distributors do not find agency 
formats to be restrictive.  However, the distributors often modify formats 
(e.g., from HTML to ASCII, or reformat data using compression technology).


4. Has your agency involved external user groups in assessing the value and 
effectiveness of the dissemination of electronic Government information 
products?  If so, are there formats and mediums that seem particularly 
appropriate for public dissemination to users who may be economically, 
technically, or physically disadvantaged?

Yes, all agencies reported that they involve users in assessing some aspects of 
their products, per OMB Circular A-130.  Examples include the following:

Focus groups are used to: 

* Determine the capability of new products (e.g., Can you use the same 
technology for the newer version of a product? Do users lose anything (e.g., 
Macros) when they update a product?).

* Determine ways to create more user-friendly CD-ROMs that resulted in the 
agency establishing three principles for producing CD-ROMs: make them simple to 
use, intuitive, and self-tutorial.

* Determine how information is presented on the web (Alpha and Beta testing).

* Solicit feedback on usability and accessibility; focus groups conducted with 
tribal Governments, teachers, librarians, children, etc.

* Learn about expectations, behavior, and problems in accessing products on the 
web experienced by the elderly.

* Interview and videotape users to assist agency redesign of website. 

* Solicit ideas from educators at professional conferences about topics that 
they would like to see presented in online products. 

For technologically or economically disadvantaged customers, one agency sub-unit 
faxes free copies of information printed from the CD-ROM or the Internet.  
Another agency sub-unit reported they try to reach economically disadvantaged 
customers by training rural community leaders in isolated areas on ways to 
access health-related information on the web .


5. Does your agency follow any internally or externally prescribed guidelines 
for the presentation and organization of products in online formats?  If so, 
what are they (e.g., WWW Federal Consortium, FIPS Guidelines, agency or 
departmental publication specifications or guidelines)?

Most agencies have developed guidelines or "best practices" for presentation of 
products in online formats.  However, several agency representatives indicated 
that the real challenge is in convincing agency staff to comply with the 
guidelines.  The following agencies provided information on their guidelines:

1. NARA: NARA Guidelines for Digitizing Archival Materials for Electronic Access 
(not to be considered a standard for digital imaging).   

(http://www.nara.gov/nara/vision/eap/eapspec.html)

2.  Federal Web Consortium's guidelines are based on Dept. of Education's 
guidelines (http://www.ed.gov/internal/wwwstds.html).  One sub-unit, NCES, also 
has developed guidelines.

3. EPA used WWW Federal Consortium guidelines 
(http://www.dtic.mil/staff/cthomps/guidelines/) to develop their own guidelines 
for presentation. 

4. Census uses a process and structure for submitting items for the web, but it 
is not yet formalized.

5. No departmental guidelines exist for the fifth agency, but most sub-units 
have some kind of guidelines for presentation and organization, although they 
might vary among sub-units.



6. Has your agency undertaken any kind of cost-benefit analyses for producing or 
creating products in preferred or emerging formats, mediums, or online 
approaches for distribution to the FDLP?  If so, which ones appear to be the 
most cost-effective? 


Generally, agencies have not conducted a formal cost-benefit analysis.  Agency 
representatives made the following observations, however:  

* One agency sub-unit tracks the number of customers who purchase a product and 
compares this amount against the cost of producing it.  They discontinued a 
product on CD-ROM because so few people could afford to buy it.

* The web reduces administrative costs for printing and mailing hard copies of 
publications.

* One agency reported a dramatic decrease (by 25,000) in the number of 
publications 
requested under the Freedom of Information Act due to the web.

* One agency tracks the number of people that a product can potentially reach 
(e.g., they sell 2,000 paper copies through GPO, but have 9,000 hits on the 
website).

* One agency reported that they order fewer publications to fill customer 
requests as a result of the web.


7. What factors does your agency consider in deciding to create or retain 
products in more than one medium?  Is this a common agency practice?


Agency representatives reported that they consider a variety of factors in 
creating and retaining products in more than one medium, although they did not 
characterize these factors as a common agency practice.  Several agencies 
reported that these issues are considered on a case-by-case basis or by the 
individual program unit.  The key factors considered are:

1. Budget (e.g., some products in CD-ROM are too expensive to make available to 
a small audience).

2. Cost (e.g., cost to print and mail product as opposed to make it available on 
the web).

3. Needs of technologically disadvantaged users (e.g., one agency maintains its 
Fax on Demand service, even though it is not cost-effective).

4. Accessibility (e.g., one sub-unit stores products in TIFF image format so 
they can produce them in whatever medium of output customers want).

5. Size of audience (e.g., agencies survey users and use web software to track 
use).

6. Number of queries or type of customer requests.  Customers will often request 
a publication or product in more than one medium.

7. Regulatory requirements.  Some products must be produced in paper regardless 
of whatever other medium it is produced in.

8. Type of product/publication  (e.g., searchable databases are only suitable 
for electronic mediums).

8. Are there trends with respect to migrating specific families of products 
(e.g., loose-leaf publications, training manuals, annual reports, conference 
proceedings) from pre-electronic mediums to electronic mediums or formats?  If 
so, which mediums and formats are used for specific families of products?


The general trend reported by agency representatives is to migrate more products 
to the web, especially recent ones.  Some examples include:

* Conference proceedings and presentations online in PowerPoint or PDF.

* Newsletters in HTML. 

* Training manuals and annual reports in HTML and PDF.

* Information for records managers are posted to Gopher, but will move to the 
agency's website in 1999.


9. Has your agency identified any medium and format standards that seem 
particularly appropriate for use throughout a product's entire information life 
cycle, not just at one stage (i.e., creation, storage and retrieval, 
communication and dissemination, archiving and disposition) for electronic 
Government information products?  If so, which ones?
 
 
Most agencies have either not addressed this issue of information life cycle or 
are struggling with it. 

* NARA has established medium and format standards for transferring permanent 
records to the National Archives in 36CFR 1228.188. 

* One sub-unit reporting putting documents in ASCII, but using Oracle for 
database management.

* One sub-unit is beginning to think about standardization for some documents.  
They draft documents in Lotus Notes (GroupWare) and publish final document in 
another database that goes onto the web.  They use Rich Text Format (RTF) to 
accommodate images and text.

* Several sub-units indicated that the technology is changing so rapidly they 
cannot establish standards.

 
10.  How do you determine whether a product should be made permanently publicly 
accessible when you create or produce it?  If so, what criteria do you use to 
determine which products will be permanently publicly accessible?  Can you give 
us any examples of how you ensure permanent public accessibility for a given 
product?
 
No agencies could provide responses to this question or indicated that this 
issue has not been resolved.  Some observations:

* One sub-unit is committed to making paper and CD-ROM-based products available 
for permanent access, but they are less clear about their commitment to products 
on the Internet.

* Some agency representatives did not understand the differences between 
permanent public access and permanent records. 

(The experts interviewed for this study provide some insight into the reasons 
that agencies are not addressing this issue.)


11. Does your agency routinely provide locator tools (e.g., GILS or specific 
agency locators) to enhance access to information sources and services available 
to external users and customers?  If so, is this an official policy, common 
agency practice, or both?

Most agencies indicated that the web format supercedes the original GILS 
concept.  However, most agencies have their own locators:

* NTIS has a catalog and maintains some GILS records.

* EPA's website has a GILS record and they put all Internet products on one 
server so there is one access point for all their products.

* Development and maintenance of GILS records is official agency policy for 
NARA.


12. Are there trends for facilitating public access to your agency products by 
including them in broad electronic Federal Government information services such 
as GPO Access, LOC Thomas, and NTIS FedWorld?  Are you using any particular 
guidelines to facilitate that decision, and, if so, what are they?


About half of the agencies use GPO Access or NTIS FedWorld.  The other half 
relies more heavily on individual agency websites with good links.