Earth Eye is a relatively new company that has been focused on LiDAR data capture, and the fusion of LiDAR with other sensor information from both an aerial and mobile mapping platform. Their aircraft has three different sensors on it: a LiDAR, a color electro-optical (EO) imager, and a hyperspectral imager. Their vans have 360 imagers, LiDAR, high-resolution pavement systems, rut bars — a number of different sensor technologies on one robust solution platform. Special correspondent Matteo Luccio spoke with Mark Romano, vice president and CTO of Earth Eye, about this sensor fusion approach, about trends in data capture, and about the future of earth observation.
Earth Eye is a relatively new company that has been focused on LiDAR data capture, and the fusion of LiDAR with other sensor information from both an aerial and mobile mapping platform. Their aircraft has three different sensors on it: a LiDAR, a color electro-optical (EO) imager, and a hyperspectral imager. Their vans have 360 imagers, LiDAR, high-resolution pavement systems, rut bars — a number of different sensor technologies on one robust solution platform. Special correspondent Matteo Luccio spoke with Mark Romano, vice president and CTO of Earth Eye, about this sensor fusion approach, about trends in data capture, and about the future of earth observation.
S&S: How did your experience developing electro-optical hardware influence your approach at Earth Eye?
Romano: I learned my tradecraft working in the aerospace industry for about 11 years. In 1994, I entered the commercial sector applying the principles and know-how from what I learned there to try to “build a better mouse trap”. Since then, I have been developing hardware and software specific to the geopositioning and georeferencing systems we use today. I moved from the classic photogrammetric and surveying world to the direct georeferencing system environment.
S&S: In your private life, do you build, repair, and improve devices? Do you have a workshop at home?
Romano: Yes, I am a tinkerer. I like to always do something different, better. I love to repair things. I am always trying to think of the next greatest widget, whether it is for personal use or hoping to make that one great invention someday. I certainly have a large garage and workshop at home and am always repairing, improving, or building something.
S&S: What is Earth Eye’s unique selling proposition (USP)? How does it differ from, for example, those of Aerometric, Airborne 1, GeoDigital, Merrick, or WSI?
Romano: Earth Eye is basically a combination of what I learned along the way from advancements in technology, continued learning from some of the best people in the industry, and not being afraid of change. Every time you do something from scratch, you can apply the lessons you learned and do it better. As far as uniqueness, I believe we have one of the few and the best combined solutions for both LiDAR and imaging systems and that seems to be something that is currently lacking.
Most people are very sensor-specific today — they are not really dealing with multiple sensors, multiple platforms. For example, our aircraft has three different sensors on it: a LiDAR, a color electro-optical (EO) imager, and a hyperspectral imager. Our vans have 360 imagers, LiDAR, high-resolution pavement systems, rut bars — a number of different sensor technologies on one robust solution platform. Generally, most of the other companies today fly or drive one or two sensors, at most, on a particular platform. We are unique in being able to integrate all of those solutions into one combined solution.
In addition, we have developed our own software application to process and integrate (fuse) multiple sensor data in a common processing environment. This adds tremendous value by allowing us to process data more accurately and exploit additional content that are often precluded with single sensor environments.
S&S: What is a rigidly coupled or single-piece system?
Romano: Both our airborne and mobile systems are referred to as “rigidly coupled”. Basically, multiple sensors are mechanically coupled to a single position and orientation system (POS) — using a GPS receiver for position and an inertial measurement unit (IMU) for orientation. With this approach we work with one direct georeferencing system for all of them. So, there is one calibration, one solution for all the sensor platforms. It makes for a very robust data set that provides for the highest accuracy and precision (registration) between all sensor data.
S&S: What is your company’s relationship to Leica? Are you committed to using Leica sensors or are you hardware agnostic? Whose sensor do you use for your mobile solution?
Romano: Our relationship to Leica is long-tested. I really like their airborne platforms; they are the most flexible over a wide range of altitudes. They are probably the best choice for a broad range of mapping services. There are many good systems available today, some of which may be better suited for specific applications, but as a small company with one airborne LiDAR I believe Leica is the right choice for us. As far as mobile solutions, we are using the Riegl LiDAR System, which has high precision and accuracy, a well-established work flow, and very good calibration routines. Again there are many solutions available today but I try to do my homework and choose the best technology and value at any given point in time based on research and experience.
For mobile imaging we design and build those ourselves and have a complete hardware and workflow solution to support it —including camera calibration, data logging (collection), post processing, and close range photogrammetry for extraction. All the companies competing today continually leapfrog each other with improvements to be the best. This heavy competition fosters rapid advancement in all the technologies ultimately benefiting us as end users.
We’re a pretty new company, so we don’t have seven or eight sensors like some of the bigger companies. However, we process all kinds of data. We process data from Riegl, Trimble, Optech, and Leica, and government-specific data. So, I’m pretty familiar with all of them. There is a lot of collaboration between companies these days and we tend to share collection and processing resources on a multitude of projects and applications.
S&S: What are some examples of custom application development?
Romano: We are pretty much game to develop anything that can solve a problem for our clients. I’d say that is our greatest strength. We basically have turn-key processing for any type of LiDAR data — that includes calibration, filtering, feature extraction, hydro modeling, transmission line modeling, and the list goes on. We have developed our own in-house solution to process and produce deliverables in multiple industries.
On the mobile side, we also incorporate mobile LiDAR and imaging in a comprehensive platform, so we can extract content from either source with high precision and accuracy. We find that there is no one tool, especially in the mobile arena. For example, you can determine the most precise and accurate positions from LiDAR, but you don’t know anything about a particular asset. While you can see a sign in mobile LiDAR, you can’t tell what kind of sign it is or its condition. There are many attributes that require building a database for either transportation or asset inventory that require both LiDAR and imaging products. We comprehensively put all that information together in one software environment. I think that is very unique in this industry.
There is another technology called a pavement camera which is a high resolution line-scan camera that comes off the back of the vehicle. They are utilized to determine the rating or quality of the pavement. It’s called “crack or distress mapping” — we get a lot of jokes about that. It is a pavement imager, so it only collects a lane width of panchromatic imagery that allows us to detail the different types of distresses or cracks that are on the road surfaces. Conventionally, this has been just a statistical process, so a certain number of cracks or types of cracks per unit of road. We’ve taken that product beyond the statistical analysis to make a geospatial product. So, not only do we know statistically the quality of a road or its distress rating, we also know quantitatively where they are and the extent of the damage. Basically, instead of delivering tabular data and statistics we now physically map the cracks as points, polylines, and polygons producing a geodatabase that our customers can use to make informed decisions about budgets and remediation planning. Here in the United States, municipal, state, and federal agencies have to inventory their roads to get funding and also prioritize the areas in which they need to work. This new process provides actionable intelligence and is a paradigm shift from the traditional processes!
S&S: At what speed can you drive the roads to collect these data?
Romano: It depends on the project, but for LiDAR, video, and crack mapping we are typically driving at highway speed, 50-60 miles an hour.
S&S: Your Web site’s Mobile LiDAR Solutions page says: “We employ a flexible, modular approach that allows us to configure our vehicle specifically for your project.” Please explain.
Romano: We have 360 degree imaging, mobile LiDAR, high-resolution pavement imagery (in a 1 millimeter resolution), and a rut bar on the vehicle’s front bumper. Different transportation districts use different rating criteria. The rut bar is for rut measurement and ride quality, the pavement camera is for road distress assessment, the video is for asset mapping/management, and the mobile LiDAR is typically used for engineering — so the remediation, the physical milling and paving of the road. The image data is used to attribute and type the condition of the asset. So, you have four different pieces of hardware there, each with a specific function, which you can run on a single vehicle.
S&S: Wouldn’t you run with all five of them all the time?
Romano: We don’t collect with them all the time. We collect what we are being asked to collect, as a service — but we are well equipped to tackle any specific transportation needs.
S&S: How and for which applications has LiDAR replaced photogrammetric mapping?
Romano: There are a few, very large-scale mapping projects for which photogrammetry is still the preferred method, but they are few and far between. LiDAR has essentially replaced photogrammetric mapping, probably 95 percent of it. Photogrammetric mapping still finds niches in small regional service sectors. Many are people who fly the same mine or landfill and are a mom and pop basement shop, but almost all of the companies today are either very little or have eliminated photogrammetric mapping entirely.
We continue to push the envelope on precision and accuracy all the time, because LiDAR gets better and better every year. The few applications left for which airborne LiDAR isn’t going to work very well are extremely large scale photogrammetry projects, at 1:20 or 1:40. People who are still doing a lot of compilation, a lot of feature-based mapping, still have some use for photogrammetry. Increasingly, however, as the LiDAR resolution goes up and it becomes more and more like a high-resolution image, we are doing less and less of that every year. The pulse rate continues to increase, the densities or resolutions continue to increase, and it just continues to squeeze photogrammetry further into large scale mapping. When I started here, we were probably doing 4,000 pulses per second and maybe mapping at 1:500 or 1:1,000 scale and we are now mapping at 1:50 scale and there are people encroaching on even tighter accuracies. It is just the evolution of the technology.
S&S: How did the transition to LiDAR correlate with the retirement age of photogrammetrists? Did many of them have to retire early or did they transition as well?
Romano: Some certainly retired, if they were older, but many of them transitioned. What you learn from photogrammetric mapping — that skill set — is still applicable to any real mapping, cartography, or surveying project. It is not a stale skills set. It may be stale in the method, but all of the basic principles still apply.
S&S: What might replace LiDAR in a few years?
Romano: The new flash or Geiger-mode systems are the big buzz now. They are fairly niche market, right now, very small footprint so they are not too efficient at collection yet. They are mainly used in military applications at the moment and are relatively expensive, but as with all technology it’s only a matter of time. They are still LiDAR, but, ultimately, that type of system, integrated with imagery, I think, will become a technology used in the years to come. Think of a flash system as a three-dimensional terrain camera and you can also get color or four-band or multi-band imagery at the same time. Eventually, when the sensor resolutions get there, we will not be making orthophotos anymore, because they will be a result of the collection process and I think that will be a paradigm shift at some point from the linear (scanning) systems that we are using today.
S&S: What are some of the applications possible today that were not possible five or ten years ago? What new applications might new imaging technology make possible in five or ten years?
Romano: The biggest drivers are increased accuracy, precision, and resolution which continue to drive the map scales lower. We do less and less classic mapping and surveying than we ever did before and are going to continue to shrink in terms of the way that we do those processes. I think that data fusion, which I was alluding to with flash LiDAR, is also an area that everyone is working on today, to try to better reference multi-sensor data. Data becomes much more feature-rich when you can enable it with other types of data — specifically, in this case, imagery. You can do a lot more from a single multimodal data set that is highly calibrated (fused) than you could from just one technology or the other.
S&S: In five or ten years, what will be the biggest application for airborne and mobile LiDAR? Will it be for surveying, engineering, and construction projects or for mass market applications, such as 3D city models for car navigation systems?
Romano: Surveying, engineering, and construction are always going to be there; it is just a necessity of life. 3D city models, car navigation, personal navigation… this is new to the United States but, in Japan, you can walk around a department store with your iPhone and know what store you are standing in front of. The inside and the outside domains are going to converge. Google has definitely shown us the way with regards to 3D city models. Those are going to get better; they are going to get more detailed. Right now, they are fairly monolithic models. As all these resolutions increase, both from top-down imaging and from oblique imaging, from higher resolution LiDAR systems and more precise integration of systems, our world is going to become virtual in three dimensions.
I don’t think that we are going to be looking at topographic maps ten years from now. We are all going to be looking at exactly where we are standing, driving, or navigating. It is all there for the taking. Microsoft is out there flying imagery. Pictometry is out there, covering the world with oblique imagery, and the ability to just go beyond monoliths with texture is going to become reality in five or ten years. We thought Google Earth was cool, but it is about to get injected with steroids. Google was a leader in the beginning, but who knows who is going to come out on top of all this. I think our world will be virtually 3D in five to ten years.
S&S: How will unmanned aerial systems (UAS) change what you do and who your clients are? Do you see them as a game changer?
Romano: I do. There are a few issues. One is when, because this kind of change comes very slowly. Ultimately, it will come. I think the first legal systems will carry too small a payload for anything beyond some small imagers. Ultimately, I can foresee a fleet of repeatable aircraft out there that are going to be visiting the Earth as often as a satellite constellation does today. I hope that’s going to happen in my lifetime, but I think it’s still going take some time.
Five to ten years from now, if this really takes off and there are sensors just revisiting every inch of the Earth or of the United States, however they end up being deployed over time, it is going to become very much like the satellite model we have today, where you are licensing that data and you will be adding value to that data. So, we may not be in this proverbial aerial collection business maybe ten years from now. I think that it is going to take some time. There are still many obstacles out there. [It requires UAS] that can handle the sizable payload that we need to do the type of work that we do but, like everything, It is only a matter of time.
