In this blog we will discuss how ENZO DataZen helps setup a secured Business to Business (B2B) data exchange topology, including one-time data transfers and ongoing data replication, and how this solution offers a loosely coupled, eventually consistent, and system agnostic architecture. We will also review some of the key principles of B2B data sharing and how information is secured and kept up to date.
DataZen is a replication technology that can detect changes made to any source system through a proprietary
Change Data Capture (CDC) engine. Because the CDC engine built into DataZen is universal by design, it can
detect changes (adds, updates, and deletes) regardless of the source system (relational, no-sql, or any API exposed through ENZO Server
or an ODBC driver).
Let's assume Corporation A wants to send data to Corporation B regularly, such as the next day's expected distribution route. The data is deemed sensitive, and Corporation A wants to control how often it sends updates. Corporation A also wants to make sure that it can stop sending data if it so chooses at any time.
In this setup, Corporation A uses DataZen to read from its internal database(s); this operation
can be scheduled at any frequency (weekly, daily, hourly or even every few minutes). Corporation A
keeps control of the frequency. DataZen creates an "Initialization File" that contains all the
records that Corporation A wants to send to Corporation B, and continues to create Change Logs over
time as records change. These logs can be sent encrypted using PGP to a secured FTP site for example, or any other
supported cloud drive (such as Dropbox, Egnyte, Azure Blobs, AWS S3 or GoogleDrive).
Corporation B also runs DataZen in its data center, and listens for incoming change logs on the same drive. Corporation B only needs read access to the change log; once changes are available, they are applied to the desired target system(s). Note that it is possible to apply the changes to multiple systems. For example, Corporation B could choose to push changes to a MySQL database, an Azure SQL Database, and a SharePoint Online list.
Corporation B can apply the change log to any target system, including HTTP endpoints by leveraging the Enzo Server platform or an ODBC driver.
ENZO DataZen provides three main capabilities that are the key principles to B2B data replication:
loosely coupled, eventually consistent, and system agnostic.
Neither the source (Corporation A) nor the target (Corporation B) DataZen applications are aware of each other's presence, and as a result function entirely independently. They only share a common cloud drive in which Change Logs are securely exchanged.
Because Change Logs represent data changes from the source system over time, the target systems can apply these changes at their own pace, guaranteeing that all the systems involved in the replication topology will eventually return the last updated values.
DataZen communicates to source and target systems through ODBC drivers and/or Enzo Server. In addition, Change Logs contain both Schema and Data from the source system, stored in a universal format. This allows the source and target systems to be entirely different platforms.
From a roles and responsibilities standpoint, the following shows what each party can perform,
using the previous example involving Corporation A and Corporation B:
|Responsibility||Corporation A (source)||Corporation B (target)|
|Read from Source Systems and filter data to replicate||Yes||-|
|Change filter to add/remove records being replicated||Yes||-|
|Set/Modify the replication schedule||Yes (reader schedule)||Yes (writer schedule)|
|Customize which records are sent, and which columns||Yes||-|
|Configure a Shared Cloud Drive or FTP Site||Yes (usually)||Possibly (not needed)|
|Encrypt Change Logs using PGP||Yes (optional)||-|
|Apply Change Logs to target systems||-||Yes|
|Replay Change Logs any number of times||-||Yes|
|Start/Stop Replication Jobs||Yes (at the source)||Yes (at the target)|
DataZen can read data from any relational database through the use of ODBC drivers, including SQL Server,
Oracle, MySQL and Teradata. Because any ODBC driver can be used, any other database
platform or system can also be the source and target system.
In addition, DataZen works with Enzo Server, which eliminates the need for ODBC drivers. Enzo Server allows DataZen to read and write data to any supported platform, including HTTP endpoints and flat files. In addition, Enzo Server allows DataZen to share its change logs through cloud drives, such as Dropbox, GoogleDrive, AWS S3 and Azure Blobs to name a few.
Loosely coupled, system agnostic, and eventually consistent replication.
By leveraging ODBC drivers and Enzo Server, DataZen can replicate data from any source system to any target system. For example, the above diagram shows a possible configuration in which Corporation A shares its MySQL data through an FTP site, and Corporation B pushes changes to two platforms: SharePoint Online and Oracle. In this example, ODBC is used by Corporation A to read data from MySQL, and Enzo Server is used by Corporation B to write data to a SharePoint Online list.
This blog discusses how two corporations can easily share data securely through a shared cloud drive or FTP site.
Because DataZen's architecture offers loose coupling, eventual consistency, and system agnostic replication,
corporations can share data regardless of the actual
database technology used on either end. In addition, using ODBC drivers and Enzo Server allows both corporations
to push data from/into any system, including flat files, HTTP endpoints, SaaS platforms, and legacy systems.
To try DataZen at no charge, download the latest edition of DataZen on our website.
Integrate data from virtually any source system to any platform.
Support for High Watermarks and fully automated Change Data Capture on any source system.
Extract, enrich, and ingest data without writing a single line of code.
To learn more about configuration options, and to learn about the capabilities of DataZen, download the User Guide now.
© 2023 - Enzo Unified