During the past few years considerable attention has been given to a collection of technologies usually found under the rubric "virtual environments" or "virtual reality." This rapidly maturing field, founded in the 1960s, offers a novel approach to the solution of many problems in the aerospace domain. Virtual environment systems generally provide immersive visual displays of computer graphics rendered at a rate that provides a "real time" environment for the user. These systems can also incorporate a three-dimensional acoustic environment and allow for user interaction through head tracking and the recognition of hand gestures. A few systems now offer one or more haptic (the term "haptic" describes sensory inputs received through the skin) display devices to deliver tactile, force, or thermal feedback to the user.

The NASA-Johnson Space Center (JSC) Graphics Research and Analysis Facility (GRAF) and the University of Houston Virtual Environment Technology Laboratory, a joint resource of JSC and the University of Houston, have been established to support research and development directed at aerospace applications of virtual environment technology. A common interest of these two laboratories is the incorporation of graphically-generated human figures into virtual environments for training, human factors analysis, and design of human-occupied facilities.

Background
A number of issues in the use of virtual environments for education, training, scientific/engineering data visualization, delivery of medical services, manufacturing, engineering design, and architecture arise due to the need to include one or more humans in the virtual environment. NASA and the Department of Defense have long been interested in the modeling of the human figure. This interest has focused on work conducted by Dr. Norman I. Badler of the University of Pennsylvania. Dr. Badler has created "Jack™", a computational model of the human body that can be rendered, with sufficient computing capability, in near real time. With Jack™ now available to NASA, the next level of research to be addressed is the control of Jack™ models by "real" humans and by intelligent agents.

Objectives
The joint activity between NASA-JSC's Graphics Research and Analysis Facility and the University of Houston's Virtual Environment Technology Laboratory is manifested postdoctoral fellow who will work cooperatively with NASA civil servants and NASA contractor personnel in the JSC GRAF and in the University of Houston Virtual Environment Technology Laboratory to further refine Jack™ and provide Jack™ with:

• autonomous capability,
• event responses,
• human communication skills,
• cognitive behavior, and
• human-based problem-solving strategies, making Jack™ a truly "synthetic actor" that can be employed in a wide range of human modeling tasks. The results of this research will be directed at such NASA needs as:
• flight deck and workstation design,
• planning IVA/EVA tasks, and
• training for IVA tasks,

• camera location and light sensitivity,
• field of view,
• lighting/illumination, and
• fidelity needed to plan for or train in cooperative task performance.

Researchers will engage in joint activity between NASA-JSC's Graphics Research and Analysis Facility and the University of Houston's Virtual Environment Technology Laboratory.

Approach
The University of Pennsylvania Jack™ "product," together with refinements of Jack™ developed by the JSC GRAF and the University of Houston Virtual Environment Technology Laboratory, form the starting point of this activity. The first phase of the work consists of parallel development of (1) the capability of driving a Jack model with sensors attached to a human and (2) creating intelligent agents to autonomously provide motion and behavioral data to a Jack model. The first element will be achieved using COTS technology and software developed by the postdoctoral researcher, working in cooperation with NASA and University of Houston scientists and engineers. The second element will build on technologies developed by both GRAF and the Virtual Environment Technology Laboratory. A combination of production rules, neural networks, and genetic algorithms will be explored as a means of producing realistic motions and behaviors in Jack models in response to a narrow range of stimuli.

In the second phase of the project the postdoctoral researcher will work with scientists and engineers in the GRAF and in the University of Houston Virtual Environment Laboratory to address issues of human performance in virtual environments. Topics to be studied will include lighting models (both number and placement of sources, degree of diffusion, and intensity), needed fields of view, and needed degrees of sensory fidelity to achieve given levels of performance.

The third phase of the project will consist of employing the resulting synthetic actors in a wide range of experiments relevant to NASA's needs. For example: design of multi-crew workstations for humans in a range of sizes, production of virtual environments for team training when one or more team members is absent, and development of crew procedures for a multi-person EVA task.

Facilities and Personnel
The postdoctoral researcher will have access to two major facilities:

NASA-JSC Graphics Research and Analysis Facility (GRAF)

Using computer models of humans and their environments, this facility performs computational human factors and human engineering for design, analysis and operations involved with humans working in space.

University of Houston Virtual Environment Technology Laboratory

Established by a Space Act Agreement between NASA-Johnson Space Center and the University of Houston, this laboratory (1) develops and refines graphics software for the rendering of complex virtual environments, (2) investigates the efficacy and develops applications of virtual environments for training and education, and (3) supports the use of virtual environments for scientific/engineering data visualization.

Anticipated Project Results
As at the conclusion of this project it is anticipated that:

• Human models will have the ability to perform autonomous behavior with some degree of reasoning and the ability to react to events (both planned and unplanned) within the computer-modeled environment. The computer model will be able to report success and failure the same way a person might report them. Communication will be by speech, gesture and words.
• The computer-modeled environment will have all of the critical attributes of the real world, such as light and sound.
• At least two "real" NASA problems will be addressed such as workstation design/layout, EVA procedures development, and IVA training.

Current Results
Although no postdoctoral fellow is in place as of September 1996, two offers have been made to qualified candidates. It is anticipated that the position will be filled in the near future. Meanwhile, joint activity between the GRAF and the Virtual Environment Technology Laboratory has commenced. In cooperation with Hughes Training, Inc. and Hughes Research Laboratories, the GRAF and the Virtual Environment Technology Laboratory have begun the development of an International Space Station IVA training application. The application (see figures) depicts the interior of a space station science module and contains a high-fidelity model of the Human Research Facility. Human figure representations allow two astronauts to carry out training for cooperative task performance even though they physically reside in different locations.