Data Warehouse Prototyping
Problem – Data Warehouse Development Takes Too Long
Everyone understands that building a new data warehouse from scratch is a large undertaking that requires significant design, development, and deployment efforts. One of the biggest issues is the level of effort required to affect a schema change, a frequent activity early in a warehouse’s lifecycle. This change process requires modification of both the ETL scripts and physical data in the warehouse and thus becomes a bottleneck that slows new warehouse deployments. This problem does not go away later in the lifecycle; it just lessens as the pace of change slows.
Solution – Use Data Virtualization to Rapidly Prototype and Quickly Meet New Requirements
You can use Composite data virtualization to rapidly prototype and quickly meet new requirements in an early stage of a new data warehouse initiative or later as you add new data sources, federate data in different ways, and or meet new reporting needs.
In this integration pattern, the Composite Information Server serves as the prototype development environment for a new data warehouse shown below. At this prototype stage, you build a virtual data warehouse rather than a physical one, saving the time to build the physical warehouse This virtual warehouse includes a full schema that is easy to rapidly iterate as was as a complete functional testing environment.
Once the actual warehouse is deployed, the views and data services built during the prototype stage still have value for prototyping and testing subsequent warehouse schema changes that arise as business needs or underlying data sources change.
Virtual Data Warehouse Serves as Prototype to Enable Rapid Development
Prototype Virtual Data Warehouse Replaced by Actual Data Warehouse
Selected Examples
- Prototyping New Data Warehouses – To reduce time to solution for new data warehouse projects and changes to existing ones, this government agency uses Composite data virtualization. Time spent getting the data right has proven to be four times faster than a directly building the ETL and warehouse, even when including the subsequent translation of Composite views into ETL scripts and physical warehouse schemas.
