FLIPS:
Flexible Integration Platforms for
Networked Information Systems

Randi Karlsen and Weihai Yu

Motivation

Network based applications are appearing and growing rapidly. Strong interests are shown in networked information systems, either within enterprises or beyond organisation boundaries. The need to access information stored in independently developed autonomous and heterogeneous information systems has become more and more important. Also, management of legacy systems and reuse of data generated in different database systems have gained importance. The need for integrated access to multiple, heterogeneous, and autonomous information systems over the network is the motivation behind the FLIPS project.

The FLIPS project aims at providing integrated OLTP (on-line transaction processing) services over multiple autonomous and heterogeneous information systems. The integration will allow for association of domain-specific knowledge and take into consideration factors like timeliness, data availability, level of consistence, cost-effectiveness, and so on. The integration may include both schematic and materialized information of the source information systems, as well as its own domain-specific integration logic.

Related Work

Enormous research activities have aimed at providing integrated services among multiple autonomous and heterogeneous data sources. The most common approaches are:

• Gateway. A simple integration through peer-to-peer gateway-augmented connections between all pairs of data sources.
• Mediation. A mediator is a software component that provides a logical database system on top of existing data sources. The mediator does not store any data. It requests the relevant data from its sources and uses that data to formulate a response to a user's query. Both the multidatabase and the federated database approaches can be seen as using a form of mediation.
• Warehousing. Copies of data from several sources are stored in a single database, called a data warehouse. Data stored in the warehouse may be processed before storage in order to fit the applications needs. They are periodically reconstructed from the current data in the sources. The warehouse is generally a read-only database supporting business decision making.
• Caching. Distributed caching may help bring data closer to the clients, mediator or data warehouse, therefore enhancing performance and availability. Basically caching can only prevent repetition of costly operations. Cached data can even be better utilized by exploring semantic information like invariants of query results. The lifetime of cached data is totally up to the dynamics of data access. Caching in general does not provide an integrated system.
• Materialization. View materialization corresponds to the full evaluation of a view/query, and a storing of the result as a relation. Materialization has gained increased attention in connection with supporting both single-source and integrated views. Materialization is an important issue in warehousing, and has also, but to a much less extent, been an issue for data integration mediators in general.

Characteristics of the state-of-the-art in relation with FLIPS goals are summarized below. The FLIPS project seeks to combine the nice features of the available integration approaches, namely mediator and data warehousing, and other relevant techniques like caching, materialized views and data replication.

Approach

The FLIPS project adopts a three-tier approach. An integrated information system in the middle tier integrates data and services from source information systems in the bottom tier. The integration includes both schematic and materialized information of the source information systems, as well as its own integration logic.

The goals of the FLIPS approach are

• Support for full integration of operational services, with value-added integration logic.
• Flexible materialization strategies to meet different requirements like performance, availability or consistency.

The middle tier provides flexible partial materialization of data from the source information systems.

Materialization is partial. Some data are replicated in the middle tier, while other data are stored only in the bottom tier and are integrated schematically in the middle tier. Materialization is preferable in situations where response-time is critical, where the network connection is unreliable, or where it is more cost-effective to incrementally maintain the information at the middle tier rather then re-compute and transmit them each time they are needed. It also enables information to be pre-integrated before it is actually in use. Schematic integration is preferable when updates at the sources are frequent and maintaining materialized information is too costly. Degree of autonomy of source information systems may also restrict maintaining consistency between materialized and original information.

Materialization is adaptive. The collection of materialized data will vary depending on the current use of the system. A particular set of data may be materialized in the middle tier for a period in which they are of high current interest, while they are excluded from the middle tier during periods of low current interest. Factors that may influence materialization include performance, current data access patterns, consistence levels, autonomy of source information systems, connectivity between source information systems and the integrated information system, and so on.

The middle tier also provides value-added integration logic that requires domain-specific knowledge unavailable at individual source information systems. This includes, among others, relating data at different source information systems, supplying domain-specific information, applying particular consistency strategies, and so on.

Example applications

A health-care system consists of many autonomous information systems, for instance, of different health-care centers, hospitals and different divisions of the same hospital. During the treatment of one patient, information from different sources need to be gathered. Integration logic such as synchronized schedule of different divisions is needed. Some of the critical information at the source, like the information about the patient, should be pre-integrated and materialized at the integrated service to increase timeliness, performance and tolerance to network disconnections.

An integrated geographical database may consist of a number of special-purpose geographic databases, such as the ones for routes and natural resources, as well as other databases like hotel systems. The integration logic may include synchronization of maps from the different geographical databases, very likely with conversions to appropriate formats, proportions or resolutions, and association of information about hotels with the maps. The information should be materialized since the process for synchronization, conversion and association of information is costly and does not need frequent updates.