.png)
Data Platforming:
Streamlining
from
Data Slush
to
Data Flow
One of the largest Pharmacy Retail store in the US specializing in filling prescriptions, retailing health and wellness products, information/consultation with a revenue of over $100B, was challenged in their goal of establishing a centralized Cloud Mart with the robust, scalable and Quick To Completion approach that of moving On Prem server data to cloud while ensuring Audit and Governance, Standardization and seamless transition.
​
Challenges
The customer was shifting the data from several on-premise and external service on a single unified platform. However, the technology landscape of applications was vast, and supported by multiple vendors and agencies. There was no data standard, and a high need of clearing our the data ambiguity through proper data governance across the data platform, while ensuring teams were working efficiently.
Solutions Implemented
To address these challenges, there were strategically employed frameworks to reduce time to production readiness, standardize processes, and unify data formats.
Data governance and traceability was achieved leveraging Collibra. This enabled a consistent process of data cleansing and created accountability across teams.
Data Mart: Standardization of Ingestion Process, Applying Governance Audit, DQ, Curation
The architecture spanned from processing landing to curated and semantic layers, supporting diverse data types and sources.
Key capabilities included
- Standardization rules engines
- DQ rules engines
- Data consolidation
- Technical metadata storage
- Integration with Collibra facilitated metadata synchronization
Frameworks across the End-to-End Flow
Custom DQ Implementation
Custom DQ was introduced in the Ingestion framework to facilitate specific rules and logic to be implemented at a Business Requirement level apart from a robust Generic Rules and Correction Logic that are parameterized and configurable.
Approach to Data Mart centralization
The process of Intake consistent of an independent framework that works as a gateway to the Azure Ecosystem, where Data Intake is configured for schedule, Data Quality and Quarantine, Metadata Updates for Governance/Audit Trail and location of Landing.
The Curation Framework further is configured for schedule, location of consumption from Landing [dynamic Batch-ID based identification], data transformations, data consolidation, Logical Transformation via defined WISIWIG Functions, or UDFs [User Defined Functions] and placed into Symantec Layer of choice as per approved Enterprise list of Databases.
Here, the use of framework streamlines the End-to-End process, including the scheduling and ensures not only standardization, but also Audit Trail of Metadata for Governance through the end-to-end cycle.
Success Factors
Covid Dashboard
An Interactive Dashboard that showcases the timeline of impact Omicron has had, and the progress being made across all outlets in terms of delivering vaccines, spliced across multiple factors like, age, gender, ethnicity.
A huge challenge in terms of designing the End-to-End Solution, bringing in data from On Prem, performing all the required data modelling and data enhancement via the framework and making the same available within the Sematic Layer was achieved in a span of 3 Week, with incremental enhancements as per business usage requirements being made over the next 4 weeks.
Clinical Trials
A unique business requirement where apart from consuming organization data of Pharmacy, Doctor, Patient, Prescription, the need to be able to guide patient to nearest pharmacy, was achieved by connecting to US Postal Services. Our teams recommendation of using Ethnicity Census of a region also enabled the Clinical Trials to further focus on where trials could be conducted and the nearby stores to be considered as part of the prescription trials.
Financial Cloud Migration
This migration project showcased the success to not only quickly perform migrations and data massage and modeling as required, but also the ability to do so with utmost precedence to Confidentiality and Authorized Access of data. The volume of transfers not only were in Big Data Volumes of TB, but the schema of data varies from small scale 10s of columns to 20K columns as well.
A look back
Our success extended beyond delivering an enterprise-wide solution. Our team expertize helped our customer in developing UDFs and valuable add-ons. External data ingestion was brought in by our teams for Clinical trial enrichment, and was highly appreciated.
​
Our ability to provide optimized solutions for critical development and business needs, translated to a well appreciated core team for our customers long term programs and goals.