Environmental Data Exchange Network for Inland Water

Understanding and protecting our environment is a key component of environmental development, yet access to a wide range of high-quality information is currently based on very limited data due to lack of the exchange of data between source and recipient. This three part book that first discusses the importance of data exchange and describes why it is essential for gathering data in the environmental sciences. Part Two takes the results of the Environmental Data Exchange Network for Inland Water project (EDEN-IW), and addresses its objectives for ensuring that the needs of citizens and enterprises of the environmental sciences community are met. Finally, Part Three takes a look at the wide variety of data policies and addresses how environment administrators in Europe can enhance their efficiency, openness and accountability.

• Discusses the importance of data exchange, as well as database integration and distribution of data with software agents
• Provides the results, objectives, and focus of the EDEN-IW project for sharing knowledge
• Addresses current data exchange policies and its future impact within the environmental fields

• Language: English
• Publisher: Elsevier Science
• Release date: Aug 11, 2011
• ISBN: 9780080467252

Book preview

UK

Part I

Environmental Data Exchange

Chapter 1

Environmental Data Exchange for Inland Water

P. Haastrup; J. Würtz

1.1 INTRODUCTION

In a world where computing power, storage capacity and network size all tend to grow exponentially, exchange of data is growing at a similar rate. From a purely information-technology point of view, any data packet sent over the network is exchange of data, and is thus linked to the overall network traffic.

Searching the Web for relevant information has become commonplace; indeed, it is at times difficult to imagine how cumbersome the search for information was before the World Wide Web.

It is amazing to see the ease with which it is now possible to search for and find, for instance, telephone numbers, addresses, flight information and scientific publications (just to mention a few). However, for other types of data, the search is more difficult and has yet to realise the full potential of the full data exchange.

On the philosophical level, one could speculate that the exchange of document information through the World Wide Web has been so successful due to the fact that the HyperText Markup language (HTML) was invented at the same time as the first browsers became operational and that the protocol and the language were so simple and easy to use.

For data held in databases, this was not the case, with different standards and proprietary software already in place on stand-alone computers, making data exchange an inherently difficult problem.

These general problems on data exchange are also relevant for data exchange in the environmental field, and in the field of Inland Water.

For the environmental topic of surface water, one additional physical characteristic is fundamental: water runs downhill (!) Environmental data for one water body upstream of another is therefore always relevant and allows the inference of some scientific and technical information about the water body downstream.

With this in mind, the Environmental Data Exchange Network for Inland Water (EDEN-IW) project (described in detail in Part II) was started, aiming to facilitate and increase data exchange.

The perspectives and the visions of the needs are that a citizen or a user, including policy makers, needs to be able to get rapid answers to environmental questions on inland waters, without any knowledge about what data sources might be available or how to connect to them. The questions may range from global to continental, from national to local, following political boundaries or catchment areas as required.

In principle, a large number of existing Inland Water databases in the world could be linked together, providing citizens with access to a worldwide inland water database, together with tools to assess the information and regulate the inland water environment in a more efficient and effective way than in the current situation. The present status is illustrated in Figure 1.1.

Fig. 1.1 Status of environmental databases.

A key driving force for the project was to meet the needs and expectations of high-quality affordable services related to environmental data, initially on the Inland Water field, but with perspectives and visions that include more general information related to the environment.

As illustrated in Figure 1.1, the current situation is a mix of all possible combinations, with some databases disconnected from the network, through situation with remote accessibility on the corporate or local area network, all the way to databases fully connected to the Internet. However, in the situation, where the databases are connected to the Internet, they typically have different interfaces and use different protocols.

The aim was to change this situation, to enable individuals and organisations to innovate and be more effective and efficient in their work related to environmental management, thereby providing the basis for sustainable growth while also improving the quality of life.

The goals of the project – with an integrative approach as a fundamental point of departure – are well adapted to the requirements for policy issues in areas where the information society technologies, by nature, require close co-ordination with various research programmes and where the deployment plays a critical role.

The challenges were and are significant. Previous experience, relevant technologies and technological advancements had to be examined, leading to research and development of specific information technology solutions.

The background is examined in the next sections, while previous experiences in environmental data exchange are examined in Chapter 2 with an example of a French water system, and the shared information of the EEA (Chapter 3).

1.2 GENERAL BACKGROUND

A significant number of public institutions and organisations gather, analyse, and disseminate environmental data from many sources. Traditionally, each organisation has developed its own databases with different database softwares and on different platforms: Each database addresses different aspects of the data. Both from a practical and from a theoretical point of view, it is important to discover inconsistencies in the data and minimise redundancies. Because the databases differ by their structure, query language, location, and by the way they represent data, it has been virtually impossible, even for a skilled database professional, to retrieve common information from these databases simultaneously. This isolation of environmental information limits its usefulness and increases the risk of inconsistency.

Many organisations gather environmental multilingual data from many sources and make them available to public and private users – though lack of consistent data descriptions has made it very difficult to share data and use them as a knowledge source in decision-making. The difficulty of organising and managing what we know about the data has led to the development of distributed systems that address isolated functions. However, the physical separation of these systems hinders or prevents access by secondary users and frustrates attempts to draw data together to form a more comprehensive understanding of environmental conditions and actions.

Previous research (US-EPA, 2005) has demonstrated the potential of the elements using distributed, heterogeneous information management by providing a conceptual demonstration of how the use of metadata registries, such as the Environmental Data Registry (EDR) of the US-EPA, can be combined with thesaurus systems, such as the Terminology Reference System (TRS) (also of the US-EPA), and integrated using agent technology.

The EDR is a single source of metadata information about the definition, source and location of environmental data. These metadata are information about data such as field name, field type, field length, value domains and system source. The TRS is a Web-accessible thesaurus of environmentally related terms, concepts, data dictionaries, and regulatory definitions. The TRS incorporates elements of a multilingual thesaurus, which will be used in EDEN-IW. These two reference systems provide a foundation for the construction of ontological reasoning systems.

Previous work, for instance by Lawrence Berkeley National Laboratory, has demonstrated the usefulness of the Extended Markup Language (XML) in integrating information from various sources.

Extended Markup Language is an enabling technology that separates content from formatting to facilitate the retrieval, publication and exchange of environmental data and metadata from databases, and has been extended to OWL and RDF. Although the use of XML does not by itself address the meaning of the data in an application, the recent combined initiative by World Wide Web Consortium (W3C) and DARPA to produce a more agent-oriented markup language, DAML (DARPA Agent Markup Language), is a step in this direction.

The previous work thus demonstrated the powerful technology, addressed by combining the use of:

• Agents – co-operating specialised objects distributed across a network, intranet, or Internet that reason about concepts, respond to queries, achieve goals, and detect complex events within the system.

• Semantic brokering – dynamic incorporation of new agents into the system and identification of agents that can provide desired services or data, based on semantic knowledge about their capabilities.

• Ontologies – specification of domain-specific vocabularies to represent real-world concepts and their relationships.

• Multi-level task and query planning – separation of the task plan requested by the user from the determination of how to process a specific query.

These features, combined as they are in distributed semantic agents, allow for a flexible, plug-and-play approach to integrated query processing and data analysis that would require many developer-years of effort to duplicate.

Some of the ideas developed in the hybrid agent multi-database system have been addressed in academic database and multi-database research projects in the past: for instance, the use of agents to accomplish these goals in a dynamic environment, integrated standard Web meta-language and metadata models, and open agent technology and relational databases in a larger, more distributed environmental application. Forward-looking developments could include incorporating an important multilingual thesaurus into the application, formalisation of value-mapping techniques using ISO 11179 metadata registries, enhancement of distributed hybrid semantic agents, and the development of reasoning engines that can take advantage of the newly developed XML data transport layer of Distributed Hybrid Semantic Agents in order to explain the occurrence of retrieval problems such as missing resources and absence or mismatch of data, as well as to identify the agent or component responsible for a slow response.

Fortunately, it is possible to address a part of these problems with state-of-the-art multi-agent technology, by employing multiple co-operative intelligent agents to conduct concept-based searches of heterogeneous, distributed information in a dynamically changing environment of databases and the World Wide Web. This type of technology can help organisations retrieve relevant information that currently is not easily accessed, filtered and retrieved by Web browsers. It may also help to harmonise the knowledge held in the databases, while maintaining the autonomy of data custodians. This multi-agent technology can be thought as dynamic virtual data warehousing, where physically separate databases are logically integrated into an efficient tool for decision support by a shared knowledge base. An overview of the technology is given in the next section, and described in detail in Chapter 4. The related security issues are outlined in Chapter 6.

1.3 TECHNICAL BACKGROUND

The objective of the EDEN-IW project is to make existing distributed environmental data available to citizens and other users through an intelligent interface acting as a one-stop shop. The users, who may also be public authorities (for instance, environmental regulatory agencies) and the public, will be able to address their needs for Inland Water data through one common interface, independent of physical or logical location of the databases, the database languages used, or of the specific nomenclature used in a specific database, and without knowing which database or databases contain the relevant information, by data mining the most complete set of information sources.

1.3.1 Agent Technology

The technology explored was the use of independent software agents. A pilot implementation is described in Haastrup and Würtz (2001). Each of the agents has a specific task in the complex process leading from the question raised by the user, through the formulation of specific (but database-independent) queries, through to the specific queries sent to databases which the agents evaluate as potentially having an answer or part of the answer for the actual question. Such requests for information require a common language— i.e. a list of accepted and well-defined words–which is the basis for an ontology, relevant for inland waters.

While data exchange for inland water, by definition, is centred on a regional level, within a common watershed, significant benefits can be reaped, by exploring the issues in a broader context.

Besides data collated directly by the users, potentially useful information is increasingly becoming available on the Internet on a global scale by a variety of information providers. However, the increase in quality and quantity of environmental data and information on the Internet highlights the difficulties of accomplishing effective and efficient retrieval. Terminology and language differences are two major barriers to information access and dissemination. Information services to the public, industry and environmental program staff can be greatly enhanced by deploying emerging Internet technologies that draw upon new semantic management techniques and tools based on types of multi-agent systems that communicate using an agent communication language based on speech acts and explicit ontologies.

Agent systems differ from competing database integration technology because they make use of intelligent agents to provide a flexible and dynamic information environment in which the arrival or departure of an agent only affects the availability of a particular service or set of data; such dynamic events do not affect the correct performance of the system as a whole.

The applied agent system, unlike most other agent systems, focuses on the use of a standard Agent Communication Language (ACL), which makes a seamless interlink and control of existing database technology possible at a much richer semantic level. The layout of the implemented agent system is illustrated in Figure 1.2.

Fig. 1.2 Structure of the agent system in EDEN-IW.

1.3.2 Inland Water Databases

The aforementioned issues are relevant to a number of environmental areas, and certainly to the area of water management, where the rivers connect the inland waters among the various member countries, and where the costal and marine environment is shared.

On the issue of data, data gathering and data mining, the technical challenge is to be able to provide the necessary overview of the various data and information, while leaving the data in the original databases. This should happen in real time, and on-line, so that laborious transformation is avoided as much as possible. Since the databases are independent and geographically distributed, on different hardware and software platforms, issues of heterogeneity must be