Photogrammetry continues to evolve and today the study of photogrammetry remains important. Armin Gruen has previously served as Head of the Chair of Photogrammetry at the Institute of Geodesy and Photogrammetry, Federal Institute of Technology (ETH) in Zuerich, Switzerland. V1 Magazine editor Jeff Thurston interviewed him recently to learn about his life long passion of photogrammetry, the beginnings of what would become the CyberCity company and his work as one of the original members of the Board for Digital Earth.
V1 Magazine: How did you get started in photogrammetry? What interested you?
AG: Originally I studied geodetic sciences. Photogrammetry was much broader in scope than surveying and geodesy and that interested me. It was strange at the beginning, because I was really aiming for geodesy , and photogrammetry was not expected as subject. .
Then I realised one could do many things with photogrammetry and that triggered my interest. Upon graduation, I needed to decide what direction would be the best to follow. I chose photogrammetry because I’d already realised that I could still perform geodesy work and become involved in surveying. Since photogrammetry included a large analytical part and image analysis, that interested me significantly because that was a growing field at that time with many new innovations happening.
V1 Magazine: How has photogrammetry changed over time?
AG: There have been many changes in the field of photogrammetry over time and they continue to be. In 1968 when I first graduated most of the photogrammetry involved analog methods. There were many different processing systems at that time, but only a small number of sensors and approaches. Today we see a large number of different sensors being used and a large industry oriented toward processsing imagery. We also see more and more multi-sensor approaches. For example, the combination of images, GPS/INS, LiDAR and other sensors is quite common today.
V1 Magazine: Can you explain the scale issues?
AG: The changes in scale today cross many applications. Most of our earlier work was at the landscape level and involved satellite and aerial sensors. Today we find many applications that include terrestrial approaches right down to the street and individual building level. Industrial quality control is firmly established. Significant applications with increasing potential are found in Cultural Heritage and archaeology.
V1 Magazine: What are the challenges for photogrammetry today?
AG: There is a wide variety of issues that are challenges to photogrammetry. It begins with the new sensors and includes cameras, laser scanners, radar sensors and so on. From my perspective it is important that we understand individual sensors and the mathematical models behind them. This is called sensor modelling and leads to an understanding of how they can best be exploited.
We need to know how to calibrate them not only geometrically, but also radiometrically. This applies to every new sensor that comes to the market. It is at this point where our profession really starts, but we also need to debate and discuss how these sensors can be applied.
For example, in the cultural heritage area where I now work, we might be interested in coupling digital object models to imagery derived through digital cameras. Such work includes network design and network analysis. It also includes geo-referencing and computing orientation. Finally, the most difficult part would include the reconstructing of the geometric and appearance parts of the model itself, especially if it has to be done in an automated mode
V1 Magazine: There must be a point where manual versus semi-automated approaches require a decision?
AG: Image interpretation remains the key for most image analysis tasks today. That is because when we talk about added-value within the imagery market, we are really talking about deriving information from the pixels. Fully automated production is a big topic today. But the human eye remains most capable of deciphering the uniqueness of visual information. An important question being asked today is “how can we fully automate the information extraction from images ?” In many cases manual or semi-automated techniques are the only valid procedures of choice, if high quality results are required. At the end it really depends on the quality of results to what extent manual or semi-automated or fully automated methods should be used.
V1 Magazine: What are the considerations then when considering full automation?
AG: The largest issue surrounds the automated interpretation of raw sensor output. We have not made much progress on this front in the past 30 years. Occasionally I listen to people who talk about the human brain. They discuss issues related to the brain and how it interprets visual cues – how the eye connects to the brain. The eyes transfer two images to the brain at which point spatial and content understanding begins. The brain can achieve a high level of correlation image patch by image patch and interprets relative to inter-relatedness and neighbourhoods.
The latest research today suggests that experience and emotions are part of this ability and understanding. These are of course things that we have no clue how to model with a computer. This is a frontier area in photogrammetry and other fields concerned with vision, and it would be highly interesting to anyone considering photogrammetry as a career in research.
V1 Magazine: How do photogrammetry and geographic information systems (GIS) come together?
AG: Photogrammetry serves as data acquisition technique for GIS, but at the same time delivers algorithms for data processing and value adding. The simplest relationship between the two involves orthophotos. This relationship is purely a geometric one and does not include any level of image understanding. The problems begin when we start talking about adding value to imagery. That leads us immediately toward interpretation and the earlier issues we mentioned. GIS as a technology works to add value when spatial analysis is taking place. When pixels provide information and that information is locatable then we can begin to think about value added products and services.
V1 Magazine: What is you involvement in CyberCity company?
AG: We initiated CyberCity AGback in 1999 as a spin-off company of ETH Zurich,based on the development of CyberCity-Modeler, a method and software that can turn images into 3D city models in a semi-automated way. I never was involved in the operational aspects of the company, but only served on the board. Today a daughter company is operating in the USA.
Today we see people talking about 3D city models that involves taking LiDAR data and draping imagery upon it. But this is not what we need. Instead, we want to be able to turn the pointcloud into structured models that can be understood by CAD systems and that GIS analysis can be enabled.
V1 Magazine: Could you explain the value added connection to GIS further?
AG: It is the topology that spatially enables CAD systems and GIS. This is a big step for many organisations today – still. That is because it is a value added step and therefore requires human intervention to ensure it is done properly. Few CAD models for digital cities today are capable of digital spatial analysis. In that respect they are quite limited. The highest level or value is reached when these models can be fully used in GIS and CAD and then coupled to the entire processing chain.
V1 Magazine: Why do you think the potential is not being realised?
AG: It is very difficult to generate complete and topologically correct 3D models fully automatically, be it from images or laserscans.. This was also the case in the late 1990’s. When we started CyberCity Modeller, we actually began by measuring key points manually. Later we would develop functions that were automated to do this, but we still had to involve humans into the process to ensure quality. The idea is to move from unstructured data to highly structured data.
V1 Magazine: What is the value of 3D?
AG: Reality is not 2-D or 2.5D but 3D. Consequently 3D is a closer approximation to reality. Our desire is to model in 3D because it allows us to understand the relationship of objects and processes better and that leads to better decision making. There are a wide variety of users for 3D models. They include telecommunication, environmental monitoring, energy, health, movie industry, TV, computer gaming and other fields of application.
V1 Magazine: Yous spoke about Digital Earth earlier. Can you tell me a bit about your work there?
AG: I am one of the original members of the Digital Earth Board and continue that work. From a society point of view we meet on a regular basis. The group is attempting to influence policy making, to support the identification of relevant R&D topics and has also started a journal (The International Journal of Digital Earth).
At the same time there are many people outside the Society who are nevertheless contributing significantly to Digital Earth. But there are still not enough connections between policy makers and senior scientists on the one side and those performing valuable scientific and practicalwork at the basis. I feel there is still a gap between these groups as far as the Society is concerned. We need to connect the groups and there are two ways to go about that. The first involves more publishing and the second includes holding more symposiums to discuss the wider Digital Earth goals. Both are related to the expansion of the platform for communication. The current top-down approach has to be coupled with bottom-up procedures.
V1 Magazine: How do you see spatial data infrastructure (SDI) connecting to Digital Earth?
AG: SDI is a part of Digital Earth. They both involve digital data towards achieving similar objectives. Some people call this Virtual Earth. By it’s very definition it is about modelling our whole environment, various processes within and changes over time.
Digital Earth is more than a static understanding though. It is a dynamic approach that considers and includes 4D (the time and change factors). There are also many research questions that surround Digital Earth and SDI that still need addressing.
V1 Magazine: What does photogrammetry bring to Digital Earth table?
AG: Photogrammetry brings the basic static model of the Earth. It is delivered through imagery, laser scanning and other sensors. It also brings the capacity and capability to monitor change over time.
V1 Magazine: Is there a role for Volunteer Geographic Information in photogrammetry?
AG: Quite frankly I don’t think so. I am aware ofthe discussions in the mapping communities, but the problem is that photogrammetry requires specialised skills and knowledge. The work cannot be accomplished by amateurs. There are no black boxes in photogrammetry. To make things happen, one has to understand the basics of the technology and of its applications. An example of poor results is the low quality of 3D building and city models generated by sketch-up techniques today.
V1 Magazine: What would you say to someone contemplating a career in photogrammetry today?
AG: I would tell them that photogrammetry is very exciting today. It is associated with constant development of new technologies and new applications, but it also includes the development and research for multi-sensor approaches that couple new technologies to traditional instrumentation.
The questions surrounding the understanding of images, their content and how to transfer them automatically into object models, are key opportunity areas for those interested. We have barely begun work in this area and it is very likely to grow in the future.
————————————————————-
Armin Gruen is a professor at the Institute of Geodesy and Photogrammetry Swiss Federal Institute of Technology ETH – Hoenggerberg CH – 8093, Zurich, Switzerland – E-mail: [email protected]
V1 Magazine: How did you get started in photogrammetry? What interested you?
AG: Originally I studied geodetic sciences. Photogrammetry was much broader in scope than surveying and geodesy and that interested me. It was strange at the beginning, because I was really aiming for geodesy , and photogrammetry was not expected as subject. .
Then I realised one could do many things with photogrammetry and that triggered my interest. Upon graduation, I needed to decide what direction would be the best to follow. I chose photogrammetry because I’d already realised that I could still perform geodesy work and become involved in surveying. Since photogrammetry included a large analytical part and image analysis, that interested me significantly because that was a growing field at that time with many new innovations happening.
V1 Magazine: How has photogrammetry changed over time?
AG: There have been many changes in the field of photogrammetry over time and they continue to be. In 1968 when I first graduated most of the photogrammetry involved analog methods. There were many different processing systems at that time, but only a small number of sensors and approaches. Today we see a large number of different sensors being used and a large industry oriented toward processsing imagery. We also see more and more multi-sensor approaches. For example, the combination of images, GPS/INS, LiDAR and other sensors is quite common today.
V1 Magazine: Can you explain the scale issues?
AG: The changes in scale today cross many applications. Most of our earlier work was at the landscape level and involved satellite and aerial sensors. Today we find many applications that include terrestrial approaches right down to the street and individual building level. Industrial quality control is firmly established. Significant applications with increasing potential are found in Cultural Heritage and archaeology.
V1 Magazine: What are the challenges for photogrammetry today?
AG: There is a wide variety of issues that are challenges to photogrammetry. It begins with the new sensors and includes cameras, laser scanners, radar sensors and so on. From my perspective it is important that we understand individual sensors and the mathematical models behind them. This is called sensor modelling and leads to an understanding of how they can best be exploited.
We need to know how to calibrate them not only geometrically, but also radiometrically. This applies to every new sensor that comes to the market. It is at this point where our profession really starts, but we also need to debate and discuss how these sensors can be applied.
For example, in the cultural heritage area where I now work, we might be interested in coupling digital object models to imagery derived through digital cameras. Such work includes network design and network analysis. It also includes geo-referencing and computing orientation. Finally, the most difficult part would include the reconstructing of the geometric and appearance parts of the model itself, especially if it has to be done in an automated mode
V1 Magazine: There must be a point where manual versus semi-automated approaches require a decision?
AG: Image interpretation remains the key for most image analysis tasks today. That is because when we talk about added-value within the imagery market, we are really talking about deriving information from the pixels. Fully automated production is a big topic today. But the human eye remains most capable of deciphering the uniqueness of visual information. An important question being asked today is “how can we fully automate the information extraction from images ?” In many cases manual or semi-automated techniques are the only valid procedures of choice, if high quality results are required. At the end it really depends on the quality of results to what extent manual or semi-automated or fully automated methods should be used.
V1 Magazine: What are the considerations then when considering full automation?
AG: The largest issue surrounds the automated interpretation of raw sensor output. We have not made much progress on this front in the past 30 years. Occasionally I listen to people who talk about the human brain. They discuss issues related to the brain and how it interprets visual cues – how the eye connects to the brain. The eyes transfer two images to the brain at which point spatial and content understanding begins. The brain can achieve a high level of correlation image patch by image patch and interprets relative to inter-relatedness and neighbourhoods.
The latest research today suggests that experience and emotions are part of this ability and understanding. These are of course things that we have no clue how to model with a computer. This is a frontier area in photogrammetry and other fields concerned with vision, and it would be highly interesting to anyone considering photogrammetry as a career in research.
V1 Magazine: How do photogrammetry and geographic information systems (GIS) come together?
AG: Photogrammetry serves as data acquisition technique for GIS, but at the same time delivers algorithms for data processing and value adding. The simplest relationship between the two involves orthophotos. This relationship is purely a geometric one and does not include any level of image understanding. The problems begin when we start talking about adding value to imagery. That leads us immediately toward interpretation and the earlier issues we mentioned. GIS as a technology works to add value when spatial analysis is taking place. When pixels provide information and that information is locatable then we can begin to think about value added products and services.
V1 Magazine: What is you involvement in CyberCity company?
AG: We initiated CyberCity AGback in 1999 as a spin-off company of ETH Zurich,based on the development of CyberCity-Modeler, a method and software that can turn images into 3D city models in a semi-automated way. I never was involved in the operational aspects of the company, but only served on the board. Today a daughter company is operating in the USA.
Today we see people talking about 3D city models that involves taking LiDAR data and draping imagery upon it. But this is not what we need. Instead, we want to be able to turn the pointcloud into structured models that can be understood by CAD systems and that GIS analysis can be enabled.
V1 Magazine: Could you explain the value added connection to GIS further?
AG: It is the topology that spatially enables CAD systems and GIS. This is a big step for many organisations today – still. That is because it is a value added step and therefore requires human intervention to ensure it is done properly. Few CAD models for digital cities today are capable of digital spatial analysis. In that respect they are quite limited. The highest level or value is reached when these models can be fully used in GIS and CAD and then coupled to the entire processing chain.
V1 Magazine: Why do you think the potential is not being realised?
AG: It is very difficult to generate complete and topologically correct 3D models fully automatically, be it from images or laserscans.. This was also the case in the late 1990’s. When we started CyberCity Modeller, we actually began by measuring key points manually. Later we would develop functions that were automated to do this, but we still had to involve humans into the process to ensure quality. The idea is to move from unstructured data to highly structured data.
V1 Magazine: What is the value of 3D?
AG: Reality is not 2-D or 2.5D but 3D. Consequently 3D is a closer approximation to reality. Our desire is to model in 3D because it allows us to understand the relationship of objects and processes better and that leads to better decision making. There are a wide variety of users for 3D models. They include telecommunication, environmental monitoring, energy, health, movie industry, TV, computer gaming and other fields of application.
V1 Magazine: Yous spoke about Digital Earth earlier. Can you tell me a bit about your work there?
AG: I am one of the original members of the Digital Earth Board and continue that work. From a society point of view we meet on a regular basis. The group is attempting to influence policy making, to support the identification of relevant R&D topics and has also started a journal (The International Journal of Digital Earth).
At the same time there are many people outside the Society who are nevertheless contributing significantly to Digital Earth. But there are still not enough connections between policy makers and senior scientists on the one side and those performing valuable scientific and practicalwork at the basis. I feel there is still a gap between these groups as far as the Society is concerned. We need to connect the groups and there are two ways to go about that. The first involves more publishing and the second includes holding more symposiums to discuss the wider Digital Earth goals. Both are related to the expansion of the platform for communication. The current top-down approach has to be coupled with bottom-up procedures.
V1 Magazine: How do you see spatial data infrastructure (SDI) connecting to Digital Earth?
AG: SDI is a part of Digital Earth. They both involve digital data towards achieving similar objectives. Some people call this Virtual Earth. By it’s very definition it is about modelling our whole environment, various processes within and changes over time.
Digital Earth is more than a static understanding though. It is a dynamic approach that considers and includes 4D (the time and change factors). There are also many research questions that surround Digital Earth and SDI that still need addressing.
V1 Magazine: What does photogrammetry bring to Digital Earth table?
AG: Photogrammetry brings the basic static model of the Earth. It is delivered through imagery, laser scanning and other sensors. It also brings the capacity and capability to monitor change over time.
V1 Magazine: Is there a role for Volunteer Geographic Information in photogrammetry?
AG: Quite frankly I don’t think so. I am aware ofthe discussions in the mapping communities, but the problem is that photogrammetry requires specialised skills and knowledge. The work cannot be accomplished by amateurs. There are no black boxes in photogrammetry. To make things happen, one has to understand the basics of the technology and of its applications. An example of poor results is the low quality of 3D building and city models generated by sketch-up techniques today.
V1 Magazine: What would you say to someone contemplating a career in photogrammetry today?
AG: I would tell them that photogrammetry is very exciting today. It is associated with constant development of new technologies and new applications, but it also includes the development and research for multi-sensor approaches that couple new technologies to traditional instrumentation.
The questions surrounding the understanding of images, their content and how to transfer them automatically into object models, are key opportunity areas for those interested. We have barely begun work in this area and it is very likely to grow in the future.
