PLATFORMS SOLUTIONS BLOGS CONTACT

Pipeline Adapter



Implement Change Data Capture and Initial Data Sync from SharePoint, SQL Server, SalesForce and Twitter.


This adapter is currently in Preview



Configuration Settings



Name Description Default
Boolean _isDefault Make this the default configuration. Defaults to true if configName is already the default.
String enzoConnectionString The central connection string to Enzo (loopback)
String storeConnectionString The central connection strong for the store engine

Handlers



ClearErrors Deletes the error log for a listener. details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
listenerName R The name of the listener

SQL Call

exec Pipeline.ClearErrors 'sharepoint-usstates', 'sql-listener'


Http Call

/bsc/pipeline/clearerrors

Output Columns

This call does not return data.

CreateListener Creates a new listener for the pipeline. details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
name R The name of the listener
system R The destination system (AzureBusTopic,Twilio,SQL...)
config The name of the destination adapter configuration
remoteObject The name of the remote object, table or endpoint
mode The processing mode of the listener (each,batch) each
initialize When 1 send all initialization data False
deleteCDC When 1 send all delete operations False
upsertCDC When 1 send all upsert operations False
columns List of columns for forward (leave blank for all)
filter Destination SQL-Like filter
options JSON object providing options specific to the destination adapter

SQL Call

exec Pipeline.CreateListener 'sharepoint-usstates', 'sql-listener-usstates'


Http Call

/bsc/pipeline/createlistener

Output Columns

This call does not return data.

CreatePipeline Creates a new pipeline. details 

Arguments

Name Description Default
name R The name of the pipeline
adapter R The name of the source adapter
config R The configuration name of the source adapter
sourceTable R The source table to read from
interval The polling interval to the source system as a cron expression
mode The pipeline mode (CDC,Sync,CDCSync)
filter Adapter-specific filter if required (ex: Twitter)

SQL Call

exec Pipeline.CreatePipeline 'sharepoint-usstates', 'sharepoint', 'sp-config', 'USStates', 'az-config', '* * /5 * * ?', 'cdcsync'


Http Call

/bsc/pipeline/createpipeline

Output Columns

This call does not return data.

DeleteListener Deletes a listener in a pipeline. details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
listenerName R The name of the listener

SQL Call

exec Pipeline.DeleteListener 'sharepoint-usstates', 'sql-listener-usstates'


Http Call

/bsc/pipeline/deletelistener

Output Columns

This call does not return data.

DeletePipeline Deletes a pipeline and associated listeners. details 

Arguments

Name Description Default
name R The name of the listener

SQL Call

exec Pipeline.DeletePipeline 'sharepoint-usstates'


Http Call

/bsc/pipeline/deletepipeline

Output Columns

This call does not return data.

DisableListener Prevents a listener from forwarding events to the destination system. details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
listenerName R The name of the listener

SQL Call

exec Pipeline.DisableListener 'sharepoint-usstates', 'sql-listener'


Http Call

/bsc/pipeline/disablelistener

Output Columns

This call does not return data.

DisablePipeline Prevents a pipeline from gathering change events from the source system. details 

Arguments

Name Description Default
name R The name of the pipeline

SQL Call

exec Pipeline.DisablePipeline 'sharepoint-usstates'


Http Call

/bsc/pipeline/disablepipeline

Output Columns

This call does not return data.

EnableListener Allows a listener to forward events to the destination system. details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
listenerName R The name of the listener

SQL Call

exec Pipeline.EnableListener 'sharepoint-usstates', 'sql-listener'


Http Call

/bsc/pipeline/enablelistener

Output Columns

This call does not return data.

EnablePipeline Enables a pipeline from gathering change events from the source system. details 

Arguments

Name Description Default
name R The name of the pipeline

SQL Call

exec Pipeline.EnablePipeline 'sharepoint-usstates'


Http Call

/bsc/pipeline/enablepipeline

Output Columns

This call does not return data.

GetCreateDDL Returns the DDL command to execute in order to create the destination table details 

Arguments

Name Description Default
dbName The database name to connect to (leave blank for Azure SQL Databases)
schemaName R The schema name to use dbo
tableName R The table name to create
connectionStr R The Central Connection String name
fields R The list of fields selected from the source pipeline

SQL Call

exec Pipeline.GetCreateDDL 'datalake', 'twitterschema', 'twitter-capture1', 'localsqldb001', 'string id,int age' 


Http Call

/bsc/pipeline/getcreateddl

Output Columns

String sql
GetExecutionLog Returns the execution log for a given execution id details 

Arguments

Name Description Default
executionId R The execution id
pipelineName R The name of the pipeline

SQL Call

exec Pipeline.GetExecutionLog '87632ad6-9d4c-4ff1-bf61-91159f3857be'


Http Call

/bsc/pipeline/getexecutionlog

Output Columns

String executionId
DateTime eventDate
String message
String level
String pipelineName
String listenerName
String pipelineId
String listenerId
GetExecutions Returns the list of recent executions for a given pipeline or listener details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
listenerName The name of the listener

SQL Call

exec Pipeline.GetExecutions 'pipelineName'
exec Pipeline.GetExecutions 'pipelineName', 'listenerName'


Http Call

/bsc/pipeline/getexecutions

Output Columns

String ExecutionId
DateTime ExecutionDate
String ErrorMessage
String StackTrace
Int32 RecordCount
Int32 TotalMin
Int32 Year
Int32 Month
Int32 Day
Int32 Hour
Int32 Minute
String PipelineName
String ListenerName
String PipelineId
String ListenerId
GetJobStats Returns saved jobs with current statistics. details 

Arguments

Name Description Default
pipelineName The name of the pipeline to filter on

SQL Call

exec Pipeline.GetJobStats


Http Call

/bsc/pipeline/getjobstats

Output Columns

String pipelineName
String listenerName
String status
String lastSyncId
Int64 lastProcessedCount
Int64 lastErrorCount
Double lastTotalKB
Int64 totalProcessedCount
Int64 totalErrorCount
String lastRunTime
String resultMessage
String stackTrace
Boolean replaying
String pipelineIdentifier
String listenerIdentifier
String disabled
ListListeners Gets available listeners. details 

Arguments

Name Description Default
pipelineName The name of the pipeline
listenerName The name of the listener (starts with)

SQL Call

exec Pipeline.GetListeners 'sharepoint-usstates' 


Http Call

/bsc/pipeline/listlisteners

Output Columns

String pipelineNameThe name of the pipeline
String nameThe name of the listener
String identifierThe internal unique id for this pipeline
String ownerThe owner of the pipeline
Boolean enabledtrue when the pipeline is enabled
DateTime createdOnThe date the pipeline was created
String systemThe destination system (AzureBusTopic,Twilio,SQL...)
String configThe name of the destination adapter configuration
String remoteObjectThe name of the remote object, table or endpoint
String modeThe processing mode of the listener (each,batch)
Boolean initializeWhen 1 send all initialization data
Boolean deleteCDCWhen 1 send all delete operations
Boolean upsertCDCWhen 1 send all upsert operations
String columnsList of columns for forward (leave blank for all)
String filterDestination SQL-Like filter
String optionsJSON object providing options specific to the destination adapter
DateTime lastPingedOnLast date/time the listener sent a Ping while running
Boolean isInitTrue when the listener is currently processing items from the INIT log
Boolean isReplayingTrue when the listener is replaying from the CDC log as requested
Boolean hasRetriesTrue when the listener has items to retry at the next execution cycle
Boolean hasErrorsTrue when the listener has errors that will not be retried automatically
Boolean clearErrorsTrue when errors will be deleted at the next execution cycle, as requested
Boolean replayErrorsTrue when errors will be replayed at the next execution cycle, as requested
ListPipelines Gets list of pipelines. details 

Arguments

Name Description Default
name The name of the pipeline (starts with)

SQL Call

exec Pipeline.ListPipelines


Http Call

/bsc/pipeline/listpipelines

Output Columns

String nameThe name of the pipeline
String identifierThe internal unique id for this pipeline
String ownerThe owner of the pipeline
Boolean enabledtrue when the pipeline is enabled
DateTime createdOnThe date the pipeline was created
String adapterThe name of the source adapter
String configThe configuration name of the source adapter
String sourceTableThe source table to read from
String intervalThe polling interval to the source system as a cron expression
String modeThe pipeline mode (CDC,Sync,CDCSync)
String filterAdapter-specific filter if required (ex: Twitter)
String columnListList of columns available from the source system
String IDsThe Primary Key of the source system (if any)
Reinitialize Reinitializes a listener by replaying the INIT log and then the complete CDC log. details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
listenerName R The name of the listener

SQL Call

exec Pipeline.reinitialize 'pipeline001', 'listener001'


Http Call

/bsc/pipeline/reinitialize

Output Columns

This call does not return data.

Replay Replays a listener from a specific start time. details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
listenerName R The name of the listener
startTime R Replay date (in UTC). Send an empty string to replay the complete CDC log

SQL Call

exec Pipeline.replay 'pipeline001', 'listener001', '2019-03-02t03:41:12.4760949z'


Http Call

/bsc/pipeline/replay

Output Columns

This call does not return data.

ReplayErrors Replays the error log for a listener. details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
listenerName R The name of the listener

SQL Call

exec Pipeline.ReplayErrors 'sharepoint-usstates', 'sql-listener'


Http Call

/bsc/pipeline/replayerrors

Output Columns

This call does not return data.

RunListener Runs a listener as soon as possible if it is enabled. details 

Arguments

Name Description Default
pipelineName R The name of the pipeline
name R The name of the listener

SQL Call

exec Pipeline.RunListener 'sharepoint-usstates'


Http Call

/bsc/pipeline/runlistener

Output Columns

This call does not return data.

RunPipeline Runs a pipeline as soon as possible if it is enabled. details 

Arguments

Name Description Default
name R The name of the pipeline

SQL Call

exec Pipeline.RunPipeline 'sharepoint-usstates'


Http Call

/bsc/pipeline/runpipeline

Output Columns

This call does not return data.

UpdatePipelineSchedule Updates a pipeline schedule. details 

Arguments

Name Description Default
name R The name of the pipeline
interval R The polling interval to the source system as a cron expression

SQL Call

exec Pipeline.UpdatePipelineSchedule 'sharepoint-usstates',  '* * /5 * * ?'


Http Call

/bsc/pipeline/updatepipelineschedule

Output Columns

This call does not return data.








601 21st St Suite 300
Vero Beach, FL 32960
United States

(561) 921-8669
info@enzounified.com
terms of service
privacy policy

PLATFORM

ENZO SERVER
ENZO DATAZEN

SOLUTIONS

SOLUTIONS OVERVIEW
INTEGRATION
SaaS
CLOUD ANALYTICS

RESOURCES

DOWNLOAD
BLOGS & VIDEOS
IN THE NEWS
ENZO ADAPTERS
ONLINE DOCUMENTATION
TCO CALCULATOR

COMPANY

LEADERSHIP TEAM
PARTNERS


© 2023 - Enzo Unified