SOLUTIONS PARTNERS BLOGS CONTACT

 Introduction
 Concepts
   Sync Agents
   Jobs Readers & Writers
   Synthetic CDC
 Security
 Connections 
     Overview
     Databases:
      SQL Server
      MySQL
      Oracle
      Teradata
      ODBC
     Drives/FTP:
      Local Path/UNC
      AWS S3 Bucket
      Azure Container/ADLS
      Google Drive
      FTP
     HTTP/S:
      HTTP/S Connection
      Rate Limiting
      OAuth Tokens
     Messaging:
      AWS SQS
      Azure Event Hub
      Azure Service Bus
      Google PubSub
      Kafka
      MSMQ
      RabbitMQ
     Big Data
      Google BigQuery
      Azure CosmosDB
 Jobs
     High Watermark Values
     Dynamic Jobs
     Job Readers 
      Database Reader
      Drive Reader
      HTTP/S Reader
      Messaging Consumer
      Big Data

     Job Writers 
      Create From Reader
      Database Target
      Drive Target
      HTTP/S Target
      Messaging Producer
      Big Data Target

     Triggers
     Data Pipelines
 DataZen Functions
 Resync & Replay
 Sync Agent API

DataZen Functions

DataZen Functions are available throughout DataZen Manager in most fields that display a function icon or show a function replacement window. DataZen Functions fall under the following categories, some of which may not be available depending on the context.

  • Enzo Functions: a list of built-in functions that can be used inline
  • Dataset Schema: a list of fields available from the data source or after being modified by a data pipeline to choose from
  • Job Parameters: a list of job-specific parameters that can be used inline

Uses Enzo Functions can be used in multiple places throughout DataZen, namely:

  • Data Pipelines Dynamic Columns
  • URL of a Sync Job using an HTTP Connect as the source
  • Custom SQL Scripts in a Database Target operation
  • Name of a database table a database Target
  • URI Endpoint, XML or JSON payloads in an HTTP Target
  • Initialization and Finalization database scripts when using a DB Target

Example: DataZen Functions in HTTP Payload

In this example, the HTTP Payload provided to a target system contains a DataZen Function and a field from the source data:

  - a random guid: #rndguid()
  - a field name replacement: {{ids}}

To insert a field or function, place the cursor at the desired insertion point, then double-click on the field or function to insert.

Certain fields display a DataZen Function icon to show a secondary screen with a similar layout.

Click on a DataZen function to see a description and example on how to use it.

You can nest functions by adding them together in a call. For example, the following operation gets the current time in UTC, returns an ISO8601 compliant date format, and URL encodes the output so it can be passed in a URL function:
#urlencode(#toiso8601(#utcnow()))

List of Functions

Function Description Comments
#decrement(a)
#decrement(a, b)
Decrements a value by one, or by the amount specified. #decrement({{field1}}, 1)
#eval(code) Treats the inner parameter as a C# script and executes its content. Expects the output of the script to be a string or an object that supports the .ToString() method. Use this function to execute a more complex C# operation. See the Microsoft documentation for the CSharpScript.EvaluateAsync operation.
#format(a, b, …) Uses the C# string.Format syntax to build a string with any number of parameters The C# syntax uses {0} for the first parameter, {1} for the second, and so forth.

#format('User {0} lives in {1}', {{userId}}, {{state}})
#formatdate(date, format)
#utcformatdate(date, format)
Formats a string value into a date with a specific format (following the C# DateTime.ToString(format) notation) #formatDate(11/01/2000T08:00:00.0000, 0:MM/dd/yy H:mm:ss)
#increment(a)
#increment(a, b)
Increments a value by one, or by the amount specified. #increment({{field1}}, 25)
#http_get(…)
#http_post(…)
#http_put(…)
#http_delete(…)
Performs an HTTP/S call to an internet resource and returns a string as output. If the call to the HTTP resource is an image, the content is returned as an Hexadecimal representation of the bytes of the image. See the HTTP Functions section below for further information

#http_get({{field_url}})
#isnullorempty(a,b) Returns value ‘b’ if value ‘a’ is a NULL value or an empty string You can use this function to replace a NULL value with a constant, or the content of another field

#isnullorempty({{field1}}, 0)
#left(a, n) Returns the left-most n characters of a string #left(‘this is a test’, 5)
#now() Returns the current datetime in the local timezone of the server
#pick(a,b,c…) Randomly selects a value from the list provided as parameters Any number of parameters can be provided
#replace(a, b, c) Replaces all occurrences of b with c, in string a #replace(‘this is a cat’, ‘cat’, ‘dog’)
#right(a, n) Returns the right-most n characters of a string #right(‘this is a test’, 5)
#rnddouble(a,b) Generates a random double value between two numbers Both the lower and upper bound values are required
#rndguid() Returns a new random GUID
#rndint(a,b) Generates a random integer value (Int32) between two numbers Both the lower and upper bound values are required
#tohex64(a) Transforms a value into an Hex64 representation #tohex64(‘this is a test’)
#toiso8601(x) Converts a datetime field into a ISO 8601 compliant format This function converts a datetime field into a string in the ISO 8601 format assuming the date is in the current timezone
#torfc1123(x) Converts a datetime field into a RFC 1123 compliant format
#utcnow() Returns the current datetime in UTC format
#utctoiso8601(x) Converts a UTC datetime field into a ISO 8601 compliant format This function converts a datetime field into a string in the ISO 8601 format assuming the date is in the UTC timezone
#utctorfc1123(c) Converts a UTC datetime field into a RFC 1123 compliant format
#urlencode(c) URL encodes the value provided so it is safe to pass as a URL parameter.

HTTP Functions

Calling HTTP Endpoints within a Data Pipeline is a feature that allows you to use the data provided by the job as input records. In other words, for each input record, an HTTP method will be called. Enzo HTTP Functions can be invoked in multiple ways to make both authenticated and unauthenticated calls to HTTP REST endpoints.

The parameters used by the HTTP functions depend on whether or not the call is authenticated and the type of call being made. The following signatures are allowed:

Function Param 1 Param 2 Param 3 Param 4 Description
http_get
http_delete
uri Call to HTTP endpoint without authentication
headers uri Call to HTTP endpoint with custom headers without authentication
@conn(key) uri Call to HTTP endpoint using a pre-defined HTTP connection string
@conn(key) headers uri Call to HTTP endpoint with custom headers using a pre-defined HTTP connection string
http_postt
http_put
body uri Call to HTTP endpoint without authentication with the specified body
body headers uri Call to HTTP endpoint with custom headers without authentication with the specified body
@conn(key) body uri Call to HTTP endpoint using a pre-defined HTTP connection string with the specified body
@conn(key) body headers uri Call to HTTP endpoint with custom headers using a pre-defined HTTP connection string with the specified body

When specifying a pre-defined connection, use the following format: @conn(key). The key should be the name of the Connection String to use that represents the HTTP Endpoint, its authentication mechanism, and Rate Limiting setting. HTTP Enzo Function calls can use any of the supported authentication mechanisms including API Keys and OAuth 2.0 bearer tokens.

When specifying custom headers, the headers parameter should be comma-separated, and the header itself should be in the normal HTTP header notation: header:value.
When specifying multiple headers, the parameter should be single-quoted: 'header1:val1,header2:val2'.

Example 1: Unauthenticated HTTP Get

The following example performs an HTTP GET operation and returns a string from a service endpoint running on the localhost without authentication:

http_get(https://localhost/func1)

Example 2: Unauthenticated HTTP Post with Headers

Sends an HTTP POST operation to a localhost endpoint unauthenticated, passing a body and a custom Content-Type header. Sends the userKey and loginId column values for each row processed by the job.

http_post(‘{ “id”: {{userKey}}, “userId”: “{{loginId}}”}’,‘Content-Type:text/json’, https://localhost/func1)

Example 3: Authenticated HTTP Get

Sends an HTTP GET operation to the “TWITTER” HTTP connection string, using the /users relative URI, and passing the userid value found in the source dataset for each record processed by the job. This call returns a JSON document of the tweets for the given userid; if this function is used in a Dynamic Column (in a Data Pipeline), the new column value will contain the JSON document returned by this HTTP request.

http_get(@conn(TWITTER), /users/{{userid}}/tweets)






601 21st St Suite 300
Vero Beach, FL 32960
United States

(561) 921-8669
info@enzounified.com
terms of service
privacy policy

PRODUCTS

ENZO SERVER
ENZO DATAZEN

SOLUTIONS

SOLUTIONS OVERVIEW
LOW-CODE INTEGRATION
SHAREPOINT INTEGRATION
RABBITMQ INTEGRATION
HYBRID SHARDING
READ/WRITE PARQUET FILES
SQL SERVER HYBRID QUERIES

RESOURCES

ENZO AZURE VM
BLOGS & VIDEOS
IN THE NEWS
ENZO ADAPTERS
ONLINE DOCUMENTATION
TCO CALCULATOR

COMPANY

LEADERSHIP TEAM
PARTNERS


© 2023 - Enzo Unified