Twitter
How to configure DataZen to access Twitter.
Pre-Requisite
The recommended approach is to register a new application project with Twitter and request
specific access to the Twitter API for this registered application. At a high level, the following steps may
be required to register a new Twitter application project.
- Register a Twitter developer account
-
Create a new Twitter project
In order to access certain aspects of the Twitter API you may need to apply for an Elevated account -
Once your project has been created, create an application
Make a note of your API Keys and API Secret - they will be needed later
WARNING: These values are generated once and cannot be accessed later; make sure to save them somewhere
Connection String
Once you have configured an application, you can create a Central Connection String in DataZen. Twitter Bearer Tokens do not expire;
however, they may be revoked manually. If revoked, you will need to go through these steps again to gerenate a new token.
- In DataZen, choose Configuration -> Central Connection Strings
- Choose New -> HTTP/S Endpoint
- Enter a friendly name for the connection, and specify this HTTP Endpoint URI: https://api.twitter.com/2/
- Choose Bearer for the authentication
-
Note: you may be able to directly generate a Bearer Token from the Application screen on the Twitter Dashboard; if so, copy and paste the
Bearer Token directly here and proceed to the next step. To create a Bearer Token with DataZen, follow these steps:
- Click on the Obtain OAuth Token... link
- On the OAuth Setting dropdown, select Twitter OAuth 2.0 Settings option
- Enter your application Client ID in the appropriate field
- Enter your application Client Secret in the appropriate field
- Click on Get Bearer Token; the Token and Refresh Token will be automatically filled out
- Click OK
- Specify a Call Rate Limit if desired; this is important to avoid throttling by Twitter.
- Click OK to create/update the Connection String
Reader
You can create a Job Reader that reads data from Twitter. The endpoints you can read from require you to have sufficient permissions as
defined by the Scopes associated with your OAuth 2.0 token. Follow the following steps to read data from Twitter:
- In DataZen, select an Agent, and choose New -> Data Sync...
- Select the connection previously created for your Source Connection
- Choose a Quick Setup option to assist with configuration (Twitter V2 API User Timeline with filter and paging)
-
Modify the URL as needed (the URL may container a {userid} parameter that should be replaced with the ID of a user to monitor)
Refer to the Twitter API documentation for more information -
The Paging Strategy should be set to Bookmark/Token to allow DataZen to fetch all available records
- Token Param Name: pagination_token
- Value Path: meta.next_token
-
Optionally set the High Watermark if you want DataZen to start reading from the last retrieved tweet every time:
- High Watermark: id
- URL Param Name: since_id
- Make sure the Apply XML/JSON Transformation is checked
- Document Root Path: data
- To try the call, modify the Max Rows value if desired and click the preview link
