Visualizing the ocean is not an easy task – moreso when it comes to 3D. It requires accurate geodata, spatial tools like CAD and GIS and knowledge about the processes involved. The work is directly applicable to marine monitoring, pipeline construction and other engineering related projects. V1 Magazine editor Jeff Thurston spoke with Christian Relling of IVS 3D about that company’s work in world’s oceans and to learn more about the efficiency gains through the use of their technologies.

V1 Magazine: How did IVS 3D begin as a company? Can you describe a little bit about the history of the company? 

CR: IVS was founded in 1995 to provide interactive 3D visualization and analysis software and services for the ocean mapping industry. The Fledermaus software suite is an interactive 3D visualization and analysis software that originated from research at the Ocean Mapping Group (OMG) at the University of New Brunswick. OMG led the world in the research and development of the application of multibeam sonars to ocean mapping for nautical charting, habitat mapping, and geologic survey.

CR: That tagline embodies three concepts.  First, it IS a “pretty picture”, but this picture is more than that—it is an intuitive presentation of complex data and communicates so much more information to the viewer then say a bunch of numbers and a 2D plot.  Second, there was the misconception that 3D visualization in general was ONLY a pretty picture at the end of the processing workflow.

CR: Fledermaus is a suite of software products that are bundled to meet the needs of specific disciplines and industries. FM Viz4D is the standard visualization package, and meets the needs of most geoscience and geospatial professionals.  FM Habitat combines the 4D visualization with multibeam sonar backscatter processing and bathymetric analysis for seabed characterization, and ideal package for interdisciplinary habitat mapping teams.  
FM Hydro is the premier tool for geospatial processing, quality control, and verification of bathymetric survey data, used by surveyors mapping for nautical charting, geohazards, and construction.  FM Offshore is a set of interactive 4D geo-spatial processing and analysis tools for planning, monitoring, and review of seabed engineering and construction projects.  Finally, FM Pro brings you the entire suite of software products—the ultimate bundle.  

In general, the software suite is used for working with and visualizing extremely dense multibeam sonar data of the seafloor.  In addition to the visualization, which in itself provides an avenue for better analysis of the data, we have tools to process and edit, mosaic and interpret multibeam backscatter data of the seafloor, plan pipeline routes, and to monitoring the location of vessels like ships and AUVs (Autonomous Underwater Vehicles) in real time.  

The greatest value we add is in the improved efficiency of the process and the accuracy of the results; this is achieved through the use of interactive visualization, and the ability to utilize all of the underlying data leading to greater insight to support decision making.  We improve the workflow in the traditional sense, but what we want to get across is that the visualization itself is the added value.  

Put on top of that the ability to interface and export to a variety of other programs on your desktop, and you will see that the software not only helps you to get out your normal required “products” faster, but also opens up the ability to deliver new and evolving products above and beyond what you customer has asked for but will come to expect.  This can include high quality perspective images, fly-through movies, Google Earth files, and visualizations that your customer can view and interact with using our free viewer (iView4D).

CR: Visualization is the key role of our company, and we feel we really led the trend in the world of ocean mapping.  We definitely see the increased use of 3 and 4D visualization across all domains and applications, as folks not only start to grasp how it is an essential element of their tools, but start to expect it. There are many trends we can see—the importance of time in visualization, the need to deal with ever-increasing amounts of data, the desire to visualize large areas but easily be able to mine down to the attributes that made up that surface.  

We also feel (or hope) that people are starting to change they way they work; not only finish with a nice 3D visualization, but actually spend most of their time in that same 3D environment.

V1 Magazine: I notice that you have something called ‘ShiftScape.’ What is that exactly? 

CR: ShiftScape consists of a number of complex algorithms that automatically adjust the amount of detail in the scene when exploring data. The effects of ShiftScape can be seen when zooming into a data set. As the viewpoint gets closer to a surface, more and more detail is displayed. The detail is adjusted based on the performance of your graphics hardware: for faster machines, more detail will be shown and for slower machines, less detail is shown.

 

CR: Assuming we are talking about environmental sustainability:  Mapping provides the very basis, and background dataset, for the management of resources.  It is very difficult to come up with any kind of resource management plan with out a high quality base map.  So, in the ocean realm, we often see ourselves becoming part of sustainability projects.  For many years we have seen our clients in government and academia mapping our nations shorelines and shallow water areas to help them study shoreline change and to help delineate habitat areas for marine protected status.  

Recently we have seen an increase in mapping for the sites of offshore wind turbines.  Another big effort over the last couple of years has been mapping for the United Nations Convention on the Law of the Seas (UNCLOS).  As nations create proposals to extend their continental shelves based on Article 15 of the UNCLOS, they are vying for the rights to exploit larger areas of the seafloor.  The rest of the seafloor, outside these national limits, will have to be managed for the good of all nations.  Anytime resource exploitation comes into play, we are or should also be thinking about sustainability.

Outside of basic seafloor surface mapping, we are currently working on tools to visualize what is in the water column.  Researchers everywhere are using multibeam sonars to map fish populations, methane vents, and plumes using the mid-water returns of the sonar.  This is all valuable data that is essentially lost to most people if it isn’t extracted, analysed and visualized.

V1 Magazine: Data quality is an important of part of visualisation but often misunderstood. What does data quality involve in your mind as it pertains to visualisation?  

CR: First, I’d like to turn that around and say that data visualization is an important part of ensuring good quality data.  Every year we get better sensors, giving us much higher resolution data that is more accurate. Better sensors are providing huge numbers of measurements, giving the redundancy necessary to produce a better seafloor model for analysis in 3D visualization.  

However, there is still a challenge in assessing which data are really good, and this is where visualization plays an essential part of the workflow.  For example, at the survey level, you may get 1000 measurements for one spot on the seafloor, but if other measurements are wrong (your location, your tidal offsets, etc.) you still don’t have good data.  If you are utilizing visualization throughout your workflow, you will catch these kinds of systematic errors earlier and increase the likelihood that the data you collect is of good quality.  

The next level involves cleaning and checking that data to produce the best model.  If you have a thousand measurements for one spot on the seafloor, how do you decide which is correct or the best model that represents the depth?  Algorithms help us quantitatively, but at some points the trained hydrographer has to review the model and make a qualitative decision—at this point, visualization is absolutely vital.  

CR: At this stage we don’t do visualization of real-time sensor processes, such as during multibeam sonar acquisition.  That kind of visualization is important but tends to be done by the sonar companies as part of the sensors’ control and logging package.  At the real-time level, we have a tool called the Vessel Manager that allows you to monitor the location of a vessel, remote operated vehicle (ROV) or autonomous underwater vehicle (AUV).  

Recently we were out with clients doing testing of their AUV.  We brought in imagery of the lake we were working in and used the Vessel Manager to keep track of where the AUV was, as well as to find features of interest.  As the AUV collected data, we brought in that data “pseudo” real-time to produce a developing model of the seafloor.  

This is something we do see our clients doing quite often—start using the application just after they acquire the data, to start visualizing the data in 3 and 4D as quickly as possible.  So many more issues can be easily resolved soon after data collection while those involved in the operation are still available versus trying to work out problems later that become much more serious as you move temporally and spatially out of the survey area.  Some of our future work will take us further along this path of pseudo real-time visualization and processing.  We are currently working on tools to process and visualize data being sensed in the water column.  

So as ships or AUVs are collecting sonar data of the seafloor, they often see a lot of “noise” in the water column that had previously been of no interest to those mapping the seafloor.  This noise is quite often something that itself if of extreme interest to other disciplines—variations in oceanographic properties, schools of fish or gas plumes.  It is essential to get these data processed, analysed and visualized as quickly as possible, as they are moving targets, and further observation may not be possible if delayed.

V1 Magazine: Are people porting the visualisations they create with you software to mobile devices? Can you share a few examples? 

CR: We have been at conferences or visiting with clients, and they might pull out their iPhone to show us a Fledermaus fly-through they’ve made or a movie posted on YouTube showing a visualization created with our software.

V1 Magazine: What are the challenges you see ahead in the area of visualisation at the present time, aside from the economic situation and focusing on the technical and user perspectives.

CR: People now have a lot of options for 3D visualization in many applications, and the challenge for them is to know what is the right software tool for their particular domain and workflow. 3D visualization in particular means many things to different people, and we often see confusion and mis-application of 3D visualization tools in various application areas. Even though they may all be 3D visualization tools, it is quite a different software tool that is used in quality control and processing of multibeam, as compared to a ship simulator or data distribution or discovery.  

Google Earth is a well known example of the  latter; its use has led to a huge increase in general public geospatial awareness, and wider acceptance of and expectation for 3 and 4D visualization. Free and widely used products like this have helped us to move thinking to the next level—from visualization as “nice to have” to visualization as expected and necessary in any geo-spatial workflow, even though with Google Earth it is primarily for distribution and collaboration at the end of the workflow.   

This has also presented some challenges.  People see something in the free public tools, and immediately want to get their data into that visualization environment.  On the other end, their clients are also asking to see the data in that environment, so they are pressured to get data cleaned, published, and accessible as quickly as possible.  

This is where we can help.  As we mentioned earlier visualization is about providing people with insight about scientific data, and IVS 3D has been at the forefront of this with respect to ocean mapping data. We understand the requirement to take the sensor data, process it rapidly and provide diverse products that in themselves provide yet another set of users with an easily understood view of the variety of data. I can think of two examples of this from our client base.  

First, from the oil and gas industry, there is the example of a complex offshore engineering project with a major oil company; before any construction can be done, a very diverse group (surveyors, geophysicist, geologists, pipeline and civil engineers, and ship masters) must be made to understand the impact of the seabed geomorphology and hazards on the project.  The survey most go from a collection of measurements to a very rich demonstration very quickly.  

The second example would be a group like the United States Geological Survey.  They are out collecting survey data, and their “clients” are the American people.  One of their products tends to be animations of  underwater geologic features as outreach for middle school children.  We support a variety of data on your desktop, so it is not a problem importing data for analysis, and to process quickly and prepare it for publication, and export.  One direct export is to create KML and KMZ files of your data for Google Earth.

 

Another challenge has been the economic one.  Budgets are cut and researchers may find it harder to get ship time.  The challenge here is to get more out of the data by using the data over and over again for multiple purposes.  Here again, we are responding to the challenges.  We provide tools to quickly let you visualize your data during survey, near real-time, so errors can be caught and dealt with more quickly.  We are creating tools for processing and visualizing mid-water data, which is data that is already there but often not stored and currently just ignored.  Finally, we provide really great tools to help you show off that data, and get as many people to use it as possible (and subsequently they may book your services later).

 

V1 Magazine: How interoperable is your product? I assume it interfaces most standard GIS products and CAD related software. But do architects and other professional fields also transfer data into Fledermaus, for example? 

CR: We are a vital part of the ocean mapping workflow, so interoperability is essential.  One of the real powers of visualization is the ability to bring in a large variety of data and “see it all” together.  So within the ocean mapping world, our software is very much interoperable and we work hard to support the various formats coming out of multibeam sonar, lidar sensors, and now laser scanners, plus all the associated and supporting data types from charts, grid formats, imagery, GIS and CAD software.  Surveyors have embraced Fledermaus for quite awhile, and it has been interesting watching the other disciplines move into working in a 3D environment.

As an example, the undersea pipeline engineer has mostly worked in a 2D (hard copy or CAD) environment and relied on the surveyor to provide the seabed terrain information mostly as contours and profiles. In difficult areas, the planning activity can extend over weeks with data going back and forth between all involved in the planning. Now they are beginning to use tools such as the FM Routeplanner, to interactively adjust their route, interactively check slope or seabed type, and check potential areas the pipe may span—all while in a meeting with the rest of their team—and then directly export that route to engineering analysis software, AutoCAD or GIS.  

We continue to strive to improve our clients workflow. Last year IVS 3D became an ESRI Business Partner and we are working with them to develop a direct interface to ArcGIS to better support those that work in the marine environment. Once released this will provide interoperability across applications, and not just across the data.

 

V1 Magazine: If you could advise people about things regarding visualisation today, what would they be?   

CR: The main purpose of any scientific visualization (2D, 3D or 4D) is not about computing and data, but about presenting all that data in the most intuitive presentation possible to provide the users with new insight. We should aim for the most intuitive presentation, in a way that we perceive the real world, so that users can rapidly gain new insights and more information can be extracted from the underlying data.

It must be fully quantitative and allow them to integrate data from all possible sensors and sources in the single environment to ensure they can efficiently make decisions in their processing and analysis. 3D visualization should be an essential element throughout any geo-spatial process and not only an add-on product at the end of the process.

This has been the goal of IVS 3D from the very beginning and it requires us to optimize the rapid advances of computing technology (processors, graphics, storage, etc.) to provide our clients with the best possible solution to their problems of huge volume of data coming from the latest ocean mapping sensors.  The amount of data and computing is part of visualization, but it should be something that the visualization providers deal with by keeping pace with the emerging technology.

V1 Magazine: Where do you see your company in 5 years time? 

CR: The goal will always be to provide our users with an intuitive environment that allows them to gain better insight from their data. This depends on the data they have and the technology they can use to deliver their products, and IVS 3D continues to invest in R&D to ensure we meet this goal. Advances in computer hardware allows us to support this goal with improved processors, graphics, network and storage, and we see an ongoing evolution from high end desktop applications, to enterprise solutions and through to mobile and web enabled services.

Visualization has moved from 2D to 3D, and 3D stereo, to full 3D volume visualization with temporal changes.  Our goal has always been to be on the forefront of the use of scientific visualization in the ocean realm; we will continue on this path.