One of the biggest challenges you’re likely to face when selecting a new Laboratory Information Management System (LIMS) is the management and handling of your existing data. While this can be a daunting task for any laboratory, the benefits of adopting a more up-to-date and technology-rich LIMS like LabVantage are significant — from enhanced productivity and efficiency to automation, integration, and lower operating costs.
This article is the first of a two-part series covering the right tools and processes to use when performing a data migration with LabVantage. Feel free to contact us directly for more information, or if you have specific questions about migrating your organization’s current systems.
Challenges of data migration
Many uncertainties surround the data migration process, regardless of the system you’re upgrading from.
- What approach is best?
- What tools should you use?
- How should the migrated data be validated in the new system?
- How can you ensure access after the migration?
In addition to these concerns, some industries are required to maintain their data records for decades (e.g., pharma, diagnostics, food & beverage, or public health). This often holds organizations back from embracing a more up-to-date and technology-rich platform. Even in situations where a new LIMS is already up and running, data migration may also be needed when legacy systems are to be decommissioned.
Finally, the migration and verification of data needs to happen as quickly and accurately as possible. It is therefore critical to use the right tools for the job, and for the migration team to have a thorough knowledge of the LIMS, databases and laboratory processes.
The building blocks for data migration
Certain high-level activities, detailed in the diagram below, make up the “building blocks” of data migration. Each can be iterative, and the planning activity is revisited during each phase.
Migration of production data requires detailed planning and is typically staged on complete copies of your data in least two pre-production environments. This ensures the accuracy and completeness of migration prior to its execution in your production environment.
The 5 phases of data migration
The data migration process can be divided into five key phases, which require careful planning and review by subject matter experts (SMEs) and stakeholders. The five phases are:
- Data Extraction — At the start of the process, all data that needs to be moved to the migrated environment gets identified and an approach to extraction is defined.
- Data Transformation — Mapping and evaluating the data that can be used in the new format.
- Data Cleansing — Removing data quality anomalies such as duplicates, data types, sizes, format, etc.
- Data Validation — Testing and verifying that the data is usable for its intended purpose.
- Data Load — Importing the data into the destination system and making it usable to operators.
Migration stages
Data migration requires a staged approach, typically distributed into two broad categories: staged environments and staged data.
- Staged Environment — LabVantage recommends performing migration with a copy of the production database on at least two different non-production in-house environments prior to executing on the production environment. This allows the creation, execution and testing of scripts before performing the actual migration, minimizing the risks of errors and downtime.
- Staged Data — This relates to migrating specific, complex objects, as well as verifying the completeness and accuracy of the migrated data and its compatibility in the new migrated environment.
The CASCADE Approach
Data migration services limit the challenges of data migration by relieving you of the following critical processes:
- Planning and configuration of the LIMS.
- Mapping of legacy data to new LIMS.
- Identifying legacy data required for read-only versus used in LIMS processes.
- Identifying duplicate data to avoid redundancy.
- Addressing data loss concerns that may result from the new structure.
- Handling data integrity
Whether you’re performing a complex migration from a legacy LIMS into LabVantage or migrating the database platform from Oracle to MSSQL (or vice-versa), LabVantage Managed Services can assist you with the process. You can also get help when starting a new LabVantage LIMS implementation in order to start fresh or eliminate unsustainable customizations of legacy systems.
In addition, LabVantage Managed Services minimizes your risk and downtime during data migration with a process we refer to as the ‘CASCADE’ approach. Here’s a quick breakdown of each element:
C — Complete
To ensure the integrity of your data in the migrated repository, LabVantage Managed Services performs a full review of your existing structure, including object mapping and data cleansing. This establishes a detailed understanding of your system’s patterns and processes to help ensure that your data continues to perform as needed in the migrated platform.
A — Accurate
The migration team leverages LabVantage APIs, plus its years of data migration experience, to ensure the accuracy of your data in its migrated platform.
S — Secure & Safe
System crashes, power failures, and other challenges can cause migration to fail. LabVantage protects your data against these possibilities by preparing a fallback plan that enables complete restoration of the entire system to its pre-migrated state in the event any issues occur.
C — Consistent
Data migration can be time consuming if not planned and executed well. This is especially true in situations where validated data must be migrated into multiple “lower” environments prior to execution. LabVantage Managed Services speeds up the migration process by ensuring consistency in performance and results with modern ETL tools, LabVantage APIs, and our team’s years of extensive expertise.
A — Analyzed
Because the LabVantage data model is highly normalized, the migration team will assist the validation process by providing an extract of the migrated data from the destination environment or queries that that will fetch the migrated data. This validation and verification process ensures the integrity of your post-migration data.
D — Dynamic Data Handling
When transferring from one system or application to another, some of your data may need to be translated, split, or merged in the new environment. LabVantage Managed Services helps to identify those areas by providing procedures, functions, and methods to convert and translate such data.
E — Execution & Exception Handling
System-related exceptions can cause costly downtime. These exceptions come about in several ways, but are usually related to integrity constraints or occur as a result of the normalized structure used when adopting to the new workflows and data model. LabVantage Managed Services minimizes downtime by using a combination of ETL and its own APIs to handle exceptions as quickly and efficiently as possible.
Migration of master and static data – as well as the importing of transaction, dynamic and system configuration data – can be simplified using utilities built directly into LabVantage products and taking advantage of the expertise of Managed Services. ETL tools can be used to migrate large-volume data that does not require significant transformation. Finally, LabVantage APIs can be leveraged to transform and handle complex data that gets stored in a highly-normalized form.
Regardless of the approach, verification and reporting are critical to ensuring data integrity. No matter what type of lab you operate, this is often the most time-consuming and critical task in any data migration. To avoid unpredictable transformation results and ensure incremental quality improvement, it is necessary for your validation planning to ensure:
- Acceptable quality limits and reject criteria for migrated data.
- Use of the ASTM Z1.4 Standard for Sampling the migrated data.
- Physical review of data from the originating system and LIMS.
- Systematic approach towards gaining knowledge about the data.
For more details on which tools LabVantage uses to make data migration as seamless as possible as well as data migration best practices, watch for the second article in this series.
Read part II: Managed Services: How LabVantage Handles Data Migration (Part II of II)
