Virtual Data Marts
Federate and deliver real-time information without data replication
No Replication Required
Operational data stores, data marts, data warehouse are proven solutions for complex data integration problems. And while much has been written about the differences in each, all are based on replication and therefore share a common set of strengths and limitations.
Virtual data marts solve many of the same data integration issues without the consequences of replication.
When Virtual is the Smart Move
When choosing between replication-based and virtual data integration approaches, architects have to consider a number of key factors with respect to the end solution requirements and the source data structures.
- Schema. How does data relate? And how should it? Normalized and/or denormalized models? How much to replicate? How much to virtualize?
- Agility. How well understood are the requirements? Will they change? How “set in stone” do you want the schema to be? Does your integration toolset support both rapid and iterative development?
- Volatility. How often does the source data change? How up-to-the-minute does the user require the data to be? Nightly refreshes via ETL? Changed data capture? High performance, federated query on demand?
- Performance. What is the SLA for the consuming application? And the SLA for the source systems? How complex are the queries? How much volume? Pure replication, pure virtual, or a mix along with some caching?
- Transformation. How much transformation required? Is the goal dimensioning the data for heavy duty analysis? Or to combine unlike types (XML, relational) into an easy to understand and use tabular form?
- Quality. How much cleansing? And where will you do it? Fix the source data? Cleanse as you replicate? Virtualize an agreed best version of the truth?
- Security. What are the field and row level access and authentication rules? Are there constraints on replication due to compliance rules or ownership boundaries? Is encryption needed?
- Reuse. Will the source data be used by other consuming applications? How will you implement the reuse? Go atomic with reusable views and data services? Or create the all-inclusive warehouse?
- Cost. How much can you spend for data integration on this project today? Can you afford the extra costs required for replication?
The Composite Advantage
Composite’s “virtual data mart” is the better choice than operational data stores, data marts, and data warehouse for projects where:
- Time to solution and frequent change place a premium on agility.
- The consuming solution requires real-time insight from fast changing sources.
- Data volumes, transformation, and cleansing workloads are supportable at runtime.
- Replication is constrained by data ownership or compliance rules.
- Development and support costs must be reduced.
How Composite Enables the Virtual Data Mart
Composite is the recognized leader in the virtualized approach to data integration that groups like DAMA and TDWI often call Enterprise Information Integration (EII), data federation, or data virtualization. Unlike replication-based approaches, Composite lets you:
- Virtualize data silos. All your data appears in one logical location. Up to the minute. Readily available, on demand.
- Abstract away complexity. Data the way your business solutions want to consume it. Easy to understand. Reusable.
- Federate heterogeneous data. Securely access and combine diverse operational and historical data. Provide single views and other composites. Query optimization for high performance.
Composite Virtual Data Marts In Action
Composite’s virtual data mart solution is proven at large customers like Wall Street investment banks, large pharmaceuticals, and the US Federal government. Selected use cases include:
- Mortgage Loan Virtual Data Mart. Virtual data mart integrates loan origination, risk analysis, approval and funding systems data to support real-time management and monitoring of over ten thousand loans in process.
- Well Maintenance & Repair Data Mart. Virtual data mart combines repair rig status, staffing availability, best practice procedures, maintenance records, flow rates, and more from disparate systems to enable real-time dispatching of repair resources.
- Single View of Cell Phone Customer Activities Mart. Virtual data mart combines customer activities data from Siebel CRM and historical data warehouse so customer service representatives’ screens provide a complete, up-to-the-minute picture.
The Bottom Line
Virtual data marts federate and deliver real-time information without the extra costs and longer development cycles associated with replication-based approaches such as operational data stores, data marts, and data warehouses. Only Composite provides the critical data virtualization, abstraction, and federation capabilities you need to build and run your virtual data marts.
