|
http://www.dbta.com/frontpage_archives/7-06.html
FRONT PAGE STORY: The Information Layer Matters
July 2006
By Joe McKendrick
eBay is a data giant. The
online auction provider currently stores more than two petabytes
of data - over 200 times the size of the Library of Congress. In addition, the
company rolls out more than 100,000 lines of new code each week.
To manage multiple data sources and
applications, eBay has built what it calls a software-based “integration
tier,” James Barrese, vice president of systems
development at eBay, told DBTA. “This contains both a data access
layer and a services framework,” he explained. “The
integration tier acts as an abstraction layer for software engineers to work
with many disparate back-end data sources through a consistent set of
abstractions.”
eBay is not alone
with the challenge of more effectively managing large reservoirs of data
streaming into its organization through multiple channels. Cliff Longman, CTO
for Kalido, told DBTA that the increasing data burden
is the major driver behind interest in enterprise data management. “The
data deluge that organizations are buried under is creating a heavy burden on
the consumers of information,” he explained. “Information is often
inconsistent and difficult to find, making the consumer, whether they be a
partner or a customer or an employee, a victim of this data challenge. In
addition, most systems require the user to know where the data is in order to
find it, a daunting task for the business user.”
There was a time, about a decade ago, when
“middleware” was considered passé. But in recent times, eBay
has not been alone in finding it more profitable and manageable to abstract
enterprise information from underlying systems. There is, however, little
agreement as to what types of technologies, approaches or architecture should
constitute what has been called the “information layer,”
“integration tier,” “enterprise data management,” or,
in some cases, an “enterprise data fabric.” Many industry watchers
say service-oriented architecture (SOA) fills the bill, while others look to
data extract, transfer and load (ETL) approaches as the purveyor of the
information layer.
It’s also a term that has been
heavily misused. In many cases, the information layer
“is something that a lot of consultants have used to their advantage to
mystify people about what’s out there,” observed Anthony Politano, partner of Business-Edge Solutions and author of
Chief Performance Officer. “I’ve seen it referred to as everything
from pure middleware, to just something sitting in between as a communication
vehicle, to an integrated data layer, including operational data stores and
data warehouses, to TIBCO.”
In an interview with DBTA, Politano defined the information layer “as a middle
layer that removes or abstracts data - takes it away from its system of record
and isolates it from there.” It doesn’t matter what the enabling
architecture is, Politano noted, whether it’s
“done through data movement, ETL, data warehousing, or data integration,
or a combination of all of the above.”
Compelling Business Case
The business case for integrating key data sources into an information layer is
compelling. "For most companies, isolated business process reengineering
is no longer enough," said Ginni Rometty, senior vice president of IBM Enterprise Transformation
Services. Business demands can be huge, Rometty said.
She observed that in the
“Many companies are recognizing the
need for consistent, shared information services that provide a comprehensive
view of information while abstracting the complexity of source systems,”
said Michael Curry, Information Management SOA lead for IBM. “These
services are designed and maintained by information specialists, ensuring
accuracy and long-term viability. Master data management is a good starting
point for this approach since these are the areas where consistency, accuracy,
completeness, and reuse are at a
premium.”
Curry cites an IBM survey of global CEOs,
which found that “60 percent believe that their organizations need to do
a better job of leveraging the information that they have in their environment.
However, most companies don’t even understand what information they have, much less know how to use it. It’s just
everywhere; it’s coming in through many different channels. They’re
starting to put into place information architectures that allow them to deal
with the complexity of their information layer in a consistent way.”
A recent survey of more than 800
enterprises conducted by Unisphere Research, the
research unit of DBTA, for the Independent Oracle Users Group (IOUG), finds
that middleware environments have now been almost universally adopted across
enterprise database sites, with 89 percent of respondents reportedly working
with one of the major brands of middleware or application servers. The question
is how deeply these middleware environments are being used to support the
information layer.
The tools and platforms are there, but
deployments tend to be spotty,
Managing this information layer for
consistency is another challenge, Ken Rugg, vice
president of Progress Software, pointed out in an interview with DBTA.
“The information layer, at a very basic level, is a way of providing
consistent, up-to-date information across all of the functions of the business.
It will vary. The level of consistency, the level of up-to-dateness,
will vary as you get further from the center of where the ownership of the data
lives. In many cases, that ownership is distributed, so you want to have a
consistent view throughout the enterprise.”
New Possibilities
To many industry watchers, SOA – in conjunction
with strategies such as master data management – is opening the door to
the first information layer truly detached from underlying systems. SOA may
represent the key enabler for an information layer strategy going forward.
For example, at eBay, “SOA is part of
the base infrastructure,” said Barrese.
“We leverage both component-oriented and service-oriented architecture
technologies. eBay has built a service architecture
and uses it to enable integration across disparate technology stacks. Additionally,
we see that by accelerating the integration of acquisitions such as PayPal, service-oriented approaches allow us to realize
reduced cost and faster time to market. Services are part of eBay's
architecture strategy.”
But companies’ approaches to developing information layers vary widely.
“I am seeing two approaches – from the bottom up and from the top
down,” Majid Abai,
professor of Business Intelligence and Enterprise Data Architecture at UCLA and
co-author of Data Strategy, told DBTA. “Some companies are utilizing Web
services, and they are accessing one or two source systems utilizing these
services, with plans to move the services to an enterprise level.”
According to Abai,
“Other organizations are adopting a five-year type of plan, to combine
the source systems and build all the common interfaces, capture data from the
source system in a near real-time standpoint, put in a data layer, and then
change the consumer systems to access the data layer - as opposed to going
directly to the source system.”
A governance structure for managing the
information flow also helps organizations get their arms around this challenge.
“Given the emphasis on compliance and corporate transparency,
organizations require a comprehensive understanding of what has
occurred,” Kalido’s Longman explained.
“Striving for one version of the truth has led companies to centralize an
integrated copy of disparate data.”
Joe McKendrick
is a contributing editor to Database Trends and Applications.
Back to
Ari Kaplan's Home Page