     
» Demos» Published Papers » Tech FAQ's        |
| PUBLISHED PAPERS |
|
Remote Visualization and Management Tools for Underwater Operations appeared in Sea Technology Magazine (April 2002 issue) PDF version of this paper
I. Introduction
Any scenario that deprives or diminishes our senses will always produce challenges because our ability to make good decisions lessens with poor perception. The ocean is without a doubt a perfect example of such a place. Depths of only a few hundred feet already pose serious challenges for any kind of operation. The drastic loss of visibility associated with depth, combined with the enormous pressures and low temperatures makes it a place where only tele-operated robots can function. These robots provide limited feedback to the people that operate them, making underwater construction a very expensive and time-consuming process. There are several factors responsible for the lack of useful feedback, many of which are bounded by the laws of physics. Communication technologies that thrive in air simply fail to work in water (e.g., radio waves). Position technologies such as GPS or laser tracking cannot be used underwater. Light can only travel a limited distance in water. As a result, the sensors currently used provide limited accuracy and frequency. The cameras available today can only provide an image of the immediate vicinity even under good visibility conditions. To complicate things even further, the data collected by all these sensors and cameras is often scattered across many systems, making its perception and analysis very difficult. All these factors lead to working scenarios where the people involved must make decisions with very little information and scattered data. Only when poor perception conditions can be improved to provide better awareness through the use of visualization and data consolidation and management, the resulting increase in awareness will lead to better decisions, hence, better job performance. This article is structured in three sections. Section II introduces a comprehensive solution that can be used to improve the perception and understanding of underwater scenes where near real-time data is available. The proposed solution can be used in all the stages of a job and provides a common framework that increases awareness and promotes better decision-making. Section III provides a summary of the application of the proposed solution in the recovery of the Ehime Maru. This particular job showcases all the concepts presented in this article. Section IV concludes the article and provides references to real-world applications and companies that have benefit with the use of these tools. II. Proposed solution: XYZ World This section contains an overview of the structure of the solution and the rationale behind its design. XYZ World's architecture is simple and concise. The architecture consists of three families of network-enabled applications and services: data distribution, data acquisition, and data visualization. The core of the data distribution suite consists of a real-time database server and a publish-and-subscribe service library. The real-time database server is responsible for maintaining an accurate representation or world model of all the elements that compose the underwater scene. The publish-and-subscribe library allows all other applications to synchronously and concurrently receive update notifications and query information about the world model. The data acquisition suite consists of applications customized to gather data from specific sources and publish the information to the real-time database server. This suite of applications also includes database access stubs and general-purpose simulators. Together, the data acquisition applications are responsible for updating the world model so that it accurately represents the underwater scene. The data visualization suite consists of applications that subscribe to the real-time database server, receive updates every time the state of the world model changes, and present the most current state of the scene to the user using 2D or 3D perspectives. In this manner, different viewers at different locations in the network can display the state of the underwater scene in a synchronous fashion. The following three sections describe the three components of the system architecture in more detail and Figure 1 shows a diagram illustrating the components that perform the three main functions of the system.
1) Data Distribution The main objective of the real-time database server is to maintain and distribute an accurate representation of the underwater scene. The server represents the scene using an efficient data structure termed the world model, which consists of a list of entities with properties designed to represent their real-world counterparts in an underwater scenario. This model is expandable and flexible enough to adapt to the unpredictable nature of sub sea tasks. A scene in the XYZ World is made of five types of entities: Surfaces: Due to the amount of points that surveying instruments can produce it was necessary to use a surface model capable of representing surfaces with hundreds of millions of polygons yet fast enough to render them at acceptable frame rates. A multi-resolution approach accomplishes both of these requirements by adjusting the amount of detail according to the location of the observer in the virtual scene [2]. The multi-resolution surface model used in the XYZ World can be updated in near-real time making it useful for surveying applications and navigation as well as underwater construction. Objects: Static and dynamic objects are represented using CAD geometry or basic shapes (e.g., cubes, cylinders, spheres, cones, etc.). Complex objects with high polygon counts can be handled through the use of interactive level of detail (LOD) management. Dynamic objects are updated through the use of bindings that link objects in the virtual environment with their counterparts in the world model. These objects can have multiple cameras, multiple lights and multiple indicators. In addition, parent/children relations between objects can also be modeled. Cameras: This entity does not have a real-world counterpart, but it is used to represent the concept of a camera in the virtual environment. They can be attached to moving objects and can be configured to track entities as well. Indicators: These entities are used to represent the value of a field or property according to some predefined behavior and/or appearance. These entities can also represent a conceptual property that exists in the real world; for example, the distance between two objects or a projection distance between an object and a surface. Lights: These entities may not have a real-world counterpart in most scenarios, but they are used to represent the concept of a light source in the virtual environment. 2) Data Acquisition The main objective of the data acquisition applications is to update the state of the world model by acquiring and publishing the data originating from disparate data sources. Specifically, each application represents the interface between a given data source and the world model since its main function is to acquire the data corresponding to the source and update the relevant components in the world model by publishing such data. There are three different groups of data acquisition applications: Sensor gathering: These applications interface directly with the sensors that provide the data. Common examples range from simple embedded microcontrollers with Analog-to-Digital (A/D) converters to sophisticated survey computers communicating through serial cables. Data processing: These applications commonly generate and publish new information by subscribing to the data gathered and published by other applications. Common examples are data filters and general-purpose simulators. Database stubs: These applications serve as gateways to high-end databases and they are responsible for publishing information that is relevant in the world model. 3) Data visualization These tools are a collection of specialized component-based modules, called 3D Gear, designed to shorten the development cycle of complex virtual environment applications [1]. XYZ's 3D Gear provides three different levels of abstraction. The first level of abstraction (first gear) provides direct access to the software components that model the key elements of a virtual scene. At this level, the 3D Gear is a collection of interfaces and components that define the elements of a virtual environment, such as surfaces, static and dynamic objects, cameras, lights, and indicators. The second level of abstraction (second gear) consists of managing components that create, destroy, manipulate, and track all the components in the world model. The third level of abstraction (third gear) includes specialized versions of Microsoft Foundation Classes (MFC) for the Document and View classes that can give a custom application instant access and visualization of a world model. The next section briefly discusses the use of XYZ World during the Ehime Maru recovery mission that ended successfully on November 26, 2001 when the Ehime Maru was brought to its final resting place. III. Ehime Maru Recovery This section illustrates the use of XYZ World during the Ehime Maru recovery (see also [4],[5]). The underwater job involved many different tasks including underwater navigation, construction, and surveying, and therefore showcases the use of XYZ World as a decision-making support tool. The first subsection describes the project and its objectives. The second subsection provides a detailed description of how the XYZ World tools were used in each phase to improve the decision process. A. Project Description The Ehime Maru salvage showcases all the concepts presented in this article. The mission objective was to recover the vessel that rested on the seabed at approximately 2000 feet. The project consisted of four stages. The first stage was a reconnaissance survey of the area to establish the exact location of the vessel, to collect the bathymetry of the area, and prepare the vessel for lifting. Stage two involved the installation of a sensor array, the removal of debris from the area, and the installation of a lifting harness around the vessel. The third stage encompassed the lifting and towing of the vessel to a shallow water area where it was accessible to Navy divers for the search and recovery of 8 victims. The fourth stage consisted of the relocation of the Ehime Maru to its final resting place. The XYZ World was put to use during the planning, execution, and evaluation phases of the first and second stages of the recovery mission. During the second stage, the system consisted of 5 viewing stations located around the recovery vessel (Rockwater 2). The location of the computers were as follows: one at the bridge to support the captain of the Rockwater 2, one at the logistics room to support the project manager and other observers, one at each of the three ROV vans to support the operators of the ROVs (Manta, XL16, and Quest). The next subsections provide a brief description of how the XYZ World tools where used in each phase to improve the decision process. Figure 10 shows snapshots of the XYZ World and real photos taken during the recovery job B. Planning The key concept that allows XYZ World tools to be used during the planning phase is the fact that the server and the viewer applications are independent of the source of the information. Simulated positions, orientations, and other variables can be published, visualized, and logged in the same fashion as the real data. This allows users to plan the job using the same environment they will later use during the execution of the project. The only difference between planning and execution is that during execution, the sources of the data are real sensors instead of simulators. It is important to note that the usefulness of this analysis depends on the accuracy and fidelity of the simulators that are used. In this particular job, the bathymetry data collected during the first stage of this project played an important role in the planning phase. Once the survey was completed, a more in depth analysis was conducted to determined optimal paths for ROV and vessel navigation, task durations, risk of collisions, etc. An additional application of a simulated scene was to present ideas to all the project members and clients. An interactive three-dimensional scene provided an understanding that otherwise could require time to explain. C. Execution During the execution of this job, position sensors for the ROVs involved collected their position and orientation and published it in real-time to the server. The five viewing stations subscribed to the server and obtained current data for all the objects in the world model. Virtual cameras were placed in key locations to provide feedback to the people operating the ROVs, the vessel and other pieces of equipment such as cranes and drillers. Distance indicators designed to measure clearances in critical locations were invaluable and alerted viewers of potential collisions before they happened. During the towing phase, GPS position was published and used in the viewer stations to place the Rockwater 2 within the scene and show the crew what lies beneath the surface of the water. This allowed them to maneuver the Rockwter 2 and raise or lower the Ehime Maru. Figure 2 shows snapshots of the XYZ World and real photos taken during the recovery job.
D. Evaluation Since all the data published to the server was logged, a total reconstruction of the entire operation could always be accomplished. Visualization of the logged data was performed through the same viewer used during the planning and execution phases. The data was logged by the server with time stamps that can be used during playback to reconstruct the events with temporal accuracy. Since navigation within the viewer is independent of the events taking place, the logged events can be viewed and studied from new locations and perspectives. Furthermore, televiewing can also take place, allowing viewers in remote sites to discuss and analyze the operation without having to travel to a meeting location. IV. Conclusion Besides the successful application of the technology described in this article during the Ehime Maru recovery project, other clients have further validated the technology in other fields. Applications in the areas of marine navigation, underwater construction and underwater surveying have been developed using XYZ Technology. Kongsberg Simrad's SPS2000 digital chart navigator uses a 3D viewer that displays a 3D surface of a digital chart area along with a virtual representation of the boat and surrounding objects (e.g., lighthouses, buoys, underwater structures, etc.). The view provided by the 3D display improves the perception of the boat's location in relation to the seabed and allows the crew to make better decisions while sailing (see Figure 3). QPS's 3D Viewer uses a multi-resolution grid to visualize multi-beam data as it is collected, allowing the surveyors to visualize the area of interest and make sure that the target zone is covered. This prevents having to go back to the site due to missing spots. Several survey companies use XYZ's data collectors to publish data from sensors to an XYZ Data Server. By subscribing to it from multiple locations using XYZ 3D Viewers, engineers can visualize the entire operation as it takes place from local or remote locations. This system has allowed survey companies to visualize not only the execution phase of projects but has assisted the company during the planning phases of key projects. In addition, the data logged during the execution phases of these projects has been useful to present results to the survey companies' clients and keep a record of the tasks completed. Existing clients have validated the usefulness of XYZ World tools in a variety of scenarios. The quality and performance of the jobs they have accomplished while using the XYZ World tools improved dramatically. They have attributed the reason for their success to the new level of awareness provided by these tools, which have led them to make time and cost-saving decisions in their projects.
References [1] D. Box, Esential COM. Addison Wesley, 1998. [2] Lindstrom, P., Koller, D., Ribarsky, W., Hodges, L., Faust, N., and Turner, G. "Real-time, continuous level of detail rendering of height fields," In Computer Graphics (SIGGRAPH '96 Proceedings) (1996), 109-118. [3] M. Kirtland, Designing Component-Based Applications. Microsoft Press, 1999. [4] Canyon Offshore. "The Recovery of the Ehime Maru". Ocean News & Technology. Nov./Dec. 2001. 34-36. Technology Systems Corporation, 2001. [5] Santamaria, J.C. & Opdenbosch, A.. "Monitoring Underwater Operations with Virtual Environments," In Proceedings of Offshore Technology Conference (OTC 2002), (in press). Authors
PDF version of this paper |
