Job Writer: Big Data
To send data to a Big Data or NO-SQL platform, select a Big Data target system. Big Data targets have additional field-mapping options as explained below.
At this time, Google BigQuery and Azure CosmosDB are supported as Big Data connections.
Example: CosmosDB
In this example, the settings use a CosmosDB target system. The Database, Container and PartitionKey Path values are set as needed. This target system allows you to build a custom JSON payload; however, this example shows you how to build the payload using the source field names automatically. These fields can use DataZen Functions and you can choose to create the Database or Container automatically as needed.
When Selected Columns is left blank, all the columns from the source system are forwarded to the database.
You set the Partition Key (id) as a random Guid, from an existing source 'id' field, or by using the value of another field.
Example: Google BigQuery
In this example, the settings use a Google BigQuery target system. The DataSet and Table values are set as needed. This target system requires you to map source columns to target columns. You can choose to create the DataSet or Table automatically as needed.
When Selected Columns is left blank, all the columns from the source system are forwarded to the database.
Data-type conversion is automatic.