Configuring Your Redshift Connector to DataHub
Now that you have created a DataHub user in Redshift in the prior step, it's time to set up a connection via the DataHub UI.
Configure Secrets
- Within DataHub, navigate to the Ingestion tab in the top, right corner of your screen
If you do not see the Ingestion tab, please contact your DataHub admin to grant you the correct permissions
- Navigate to the Secrets tab and click Create new secret
- Create a Redshift User's Password secret
This will securely store your Redshift User's password within DataHub
- Click Create new secret again
- Enter a name like
REDSHIFT_PASSWORD
- we will use this later to refer to the secret - Enter your
datahub
redshift user's password - Optionally add a description
- Click Create
Configure Recipe
- Navigate to the Sources tab and click Create new source
- Select Redshift
- Fill out the Redshift Recipe
Populate the Password field by selecting Redshift Password secrets you created in steps 3 and 4.
Schedule Execution
Now it's time to schedule a recurring ingestion pipeline to regularly extract metadata from your Redshift instance.
- Decide how regularly you want this ingestion to run-- day, month, year, hour, minute, etc. Select from the dropdown
- Ensure you've configured your correct timezone
- Click Next when you are done
Finish Up
- Name your ingestion source, then click Save and Run
You will now find your new ingestion source running
Validate Ingestion Runs
- View the latest status of ingestion runs on the Ingestion page
- Click the plus sign to expand the full list of historical runs and outcomes; click Details to see the outcomes of a specific run
- From the Ingestion Run Details page, pick View All to see which entities were ingested
- Pick an entity from the list to manually validate if it contains the detail you expected
Congratulations! You've successfully set up Redshift as an ingestion source for DataHub!
Need more help? Join the conversation in Slack!