ON THIS PAGE
Shopify uses the concept of an app to allow access to store data for a merchant. The app is configured with the requisite permissions to fetch the different types of data from the store using Shopify’s REST APIs. The Pipeline in Hevo then integrates with the Shopify app to access the data and loads it into the Destination.
The transfer of data from your Shopify store to the Destination location, therefore, comprises the following one-time setups:
Creating an app in Shopify.
Assigning permissions to the app to read and transform the data using Shopify’s Rest API.
Creating a Pipeline in Hevo for transferring data from Shopify to the Destination database or data warehouse.
A Pipeline only transfers data to the specified Destination. You need to use appropriate tools for transforming the data for further analysis or for publishing it through your e-commerce portal. See Models for further information.
Note: For Pipelines created with this Source, Hevo provides you a fully-managed BigQuery data warehouse Destination if you do not already have one set up. You are only charged the cost that Hevo incurs for your project in Google BigQuery. The invoice is generated at the end of each month and payment is recovered as per the payment instrument you have set up. You can now create your Pipeline and directly start analyzing your Source data. Read Hevo Managed Google BigQuery .
OAuth authentication is not supported in private apps.
A valid Shopify account.
Data types in Source and Destination are the same.
Creating an App in Shopify
As a first step, you must create an app, using which, you can make API calls to your Shopify store and fetch data. To do this:
Log in to your Shopify Store Admin account here.
In the left menu bar, click Apps, and then, Manage private apps.
Click Create new private app.
Note: Private apps function exclusively for your Shopify store unlike public apps, which are built to work with many stores. Therefore, with private apps, you can access your store’s data directly using Shopify’s APIs.
In app details:
Provide a name for the private app.
In Emergency developer email, enter the email address that Shopify should use for contacting you regarding your app.
Scroll to the Admin API section and complete the steps listed in the Configuring API Permissions in Shopify task below to enable the app to fetch the data from Shopify.
Configuring API Permissions in Shopify
To enable the app to make API calls, you must configure it with the required permissions to the different resources and the data within them and generate the API credentials. To do this:
Under Admin API, click Show inactive Admin API permissions to view the list of Shopify API resources and the permissions you can configure for them.
Specify Read Access permission for the following resources:
Click Save. Click Create app in the confirmation dialog box.
Copy the API Credentials that are displayed and store them at a secure location.
Go back to Apps > Manage private apps and click on the app you just created.
Copy the following details. You will enter them in Hevo when connecting to the Shopify data Source.
- API Key
- Password (located below the API key)
Creating a Pipeline in Hevo for Shopify
Once you have set up the app and generated the API credentials, create a Pipeline to complete the framework for replicating the data from Shopify into the Destination. To do this:
Navigate to the Hevo user interface.
Click Pipelines in the left navigation bar, and then, + Create Pipeline in the workspace on the right.
Select Source Type as Shopify.
Define the following settings:
Pipeline Name: A unique name for your Shopify Pipeline.
Shop Name: Your shop name. You can find this in the address bar when logged into your shop. For example, in the URL https://mynewshop.myshopify.com/admin, the shop name is “mynewshop”.
Admin API Password: Your app password. See Step 6 of the procedure above.
No additional settings are needed, as Hevo connects to Shopify using the app details that you provide.
Note: You can transfer data of all data types.
Data Replication Window and Schedule
As part of creating the Pipeline, Hevo configures the schedule and frequency at which the Pipeline must synchronize the Destination with the Source.
Historical data: In the first run of the Pipeline, the Destination database is empty, and legacy data from the Source needs to be replicated into it. By default, Hevo replicates object data for the past 60 days. Contact Hevo Support team if you need to change this duration.
Incremental data: Once the legacy data is available in the Destination, subsequent runs of the Pipeline sync the objects between the Source and Destination only on an incremental basis. The default frequency of uploading the objects is 10 minutes. You can modify this value, with the minimum frequency being 5 minutes and maximum being 24 hours. To do this:
Click the More icon on the top right corner of the pane, and then, Change Schedule.
Select from the available frequencies or specify a custom duration and click Schedule. The new frequency is displayed below the Source name in the Pipeline. For example, in the image above, the sh_25 Shopify Source is synchronized every three hours.
Hevo uses the following schema to upload the records in the Destination database:
Refer to Appendix 1 - Destination Tables for the list of tables created at the Destination.
If you have selected the AutoMapping option, Hevo creates the tables in the Destination by default. Else, you must create the appropriate tables and map them accordingly. See sample image here:
Diagnostics and Error handling
Appendix 1: Destination Tables
The following is the list of tables (objects) and their primary keys that are created at the Destination when you run the Pipeline.
Note: The table names are written in small case, except for the Snowflake data warehouse tables which are written in uppercase.