Snowflake

Last updated on Oct 10, 2023

Snowflake offers a cloud-based data storage and analytics service, generally termed as data warehouse-as-a-service. Companies can use it to store and analyze data using cloud-based hardware and software.

In Snowflake, you can create both data warehouses and databases to store your data. Each data warehouse can further have one or more databases, although this is not mandatory. Snowflake provides you one data warehouse automatically when you create an account.

The Snowflake data warehouse may be hosted on any of the following Cloud providers:

  • Amazon Web Services (AWS)

  • Google Cloud Platform (GCP)

  • Microsoft Azure (Azure)

For Hevo to access your data, you must assign the required permissions. Snowflake uses Roles to assign permissions to users. You need ACCOUNTADMIN, SECURITYADMIN or SYSADMIN privileges to create the required roles for Hevo. Read more about Roles in Snowflake.

The data from your Pipeline is staged in Hevo’s S3 bucket before being finally loaded to your Snowflake warehouse.

To connect your Snowflake instance to Hevo, you can either use a private link which directly connects to your Cloud provider through Virtual Private Cloud (VPC) or connect via a public network using your Snowflake account URL.

A private link enables communication and network traffic to remain exclusively within the cloud provider’s private network while maintaining direct and secure access across VPCs. It allows you to transfer data to Snowflake without going through the public internet or using proxies to connect Snowflake to your network. Note that even with a private link, the public endpoint is still accessible and Hevo uses that to connect to your database cluster.

Note: The private link is supported only for the Hevo platform regions.

Please reach out to Hevo Support to retrieve the private link for your cloud provider.

The following image illustrates the key steps that you need to complete to configure Snowflake as a Destination in Hevo:

Configure Snowflake as a Destination


Handling Source Data with Different Data Types

For teams created in or after Hevo Release 1.58, Hevo automatically modifies the data type of a Snowflake table column to accommodate Source data with a different data type. Data type promotion is performed on tables that are less than 50GB in size. Read Handling Different Data Types in Source Data.

Note: Your Hevo release version is mentioned at the bottom of the Navigation Bar.


Limitations

  • Hevo replicates a maximum of 4096 columns to each Snowflake table, of which six are Hevo-reserved metadata columns used during data replication. Therefore, your Pipeline can replicate up to 4090 (4096-6) columns for each table. Read Limits on the Number of Destination Columns.

See Also


Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Oct-03-2023 NA Updated sections, Creating a Snowflake Account and Create and Configure your Snowflake Warehouse as per the latest Snowflake UI.
Aug-11-2023 NA Fixed broken links.
Apr-25-2023 2.12 Updated section, Configure Snowflake as a Destination to add information that you must specify all fields to create a Pipeline.
Dec-19-2022 2.04 Updated section, Configure Snowflake as a Destination to reflect the latest Hevo UI.
Dec-19-2022 2.04 Updated the page overview to add information about Hevo supporting private links for Snowflake.
Nov-24-2022 NA Added a step in section, Create and Configure your Snowflake Warehouse.
Oct-10-2022 NA Added the section (Optional) Creating a Snowflake Account.
Jun-16-2022 NA Modified section, Prerequisites to update the permissions required by Hevo to access data on your schema.
Jun-09-2022 NA Updated the page to provide a script containing all the user commands for creating a Snowflake warehouse.
Mar-31-2022 NA Updated the screenshots to reflect the latest Snowflake UI.
Feb-07-2022 1.81 Updated the page to add the step, Create and Configure Your Snowflake Warehouse, and other permission related content.
Mar-09-2021 1.58 Added section, Handling Source Data with Different Data Types.
Feb-22-2021 NA - Updated the page overview to state that the Pipeline stages the ingested data in Hevo’s S3 bucket, from where it is finally loaded to the Destination.
- Formatting-related edits.

Tell us what went wrong