Stripe

Stripe is a payment gateway that facilitates payments (in person or online) by transferring money between a merchant account and a payment processor. It can be incorporated into websites and mobile applications and provides technical, fraud prevention and banking infrastructure to facilitate online payments.

For creating Pipelines using this Source, Hevo provides you a fully managed BigQuery data warehouse as a possible Destination. This option remains available till the time you set up your first BigQuery Destination irrespective of any other Destinations that you may have. With the managed warehouse, you are only charged the cost that Hevo incurs for your project in Google BigQuery. The invoice is generated at the end of each month and payment is recovered as per the payment instrument you have set up. You can now create your Pipeline and directly start analyzing your Source data. Read Hevo Managed Google BigQuery.


Prerequisites

  • An active Stripe account with either Administrator or Developer role. Read Teams and user roles.

  • Either a Restricted API Key or a Secret Key, which allows Hevo to read your Stripe data. Read Creating the API Key.


Configuring Stripe as a Source

Perform the following steps to configure Stripe as a Source in Hevo:

  1. Click PIPELINES in the Asset Palette.

  2. Click + CREATE in the Pipeline List View page.

  3. In the Select Source Type page, select Stripe.

  4. In the Configure Your Stripe Source page, specify the following:

    Stripe settings

  5. Click TEST & CONTINUE.

  6. Proceed to configuring the data ingestion and setting up the Destination.


Obtaining the API Key

You may obtain a Restricted API Key or a Standard Secret API Key and share it with Hevo. The restricted API key allows you to select specific objects to give Hevo access to, whereas the standard secret key allows Hevo to access all objects without any restriction. If you have an Administrator or Developer role, your API keys are always available in the Stripe Dashboard. Read Stripe’s documentation on API keys.

Do one of the following:

  • Create a restricted API Key:

    1. Log in to the Developer Dashboard.

    2. Disable the Viewing test data option if it is enabled, to view live data.

    3. In the API keys page, under the Restricted keys section, click Create restricted key.

      Create restricted key

    4. Specify a unique name for this API Key. For example, Stripe connector for Hevo.

    5. Under the PERMISSIONS column, select the Read permission corresponding to the following resource types:

      1. All core resources

      2. All Billing resources

      3. All Checkout resources

      4. All Orders resources

      5. All Connect resources

      6. All Issuing resources

      7. All webhook resources

      8. All Reporting resources

    6. Scroll to the bottom of the page, and click Create key. You will be redirected to the API keys page.

      Restricted Key

    7. Under the Restricted Keys section, click Reveal live key corresponding to the name of the API key that you just created, and copy it. This key is required while setting up a Pipeline in Hevo.

      Reveal Key

  • View the standard secret key:

    1. Log in to the Developer Dashboard.

    2. Disable the Viewing test data option if it is enabled, to view live data.

    3. In the API keys page, under the Standard keys section, click Reveal live key, and copy it. This key is required while setting up a Pipeline in Hevo.

      Standard key


Object Selection

After you have specified the Stripe Source connection settings, as mentioned in the Configuring Stripe as a Source section above, select the objects to be ingested in the Select the Objects you want to replicate page, and click CONTINUE.

All the objects for ingestion are selected by default. You can deselect the objects you do not want to ingest. Hevo displays a warning if your Stripe restricted API key does not have the required permissions for the objects. All such objects can be found at the bottom of the objects list. Hevo displays the specific permission required for ingesting data when you hover over the warning icon.

Stripe object selection

To grant permissions in Stripe to ingest these objects, do the following:

Note: You need to have the Admin or Developer role in Stripe to view the API keys in your account.

  • Provide a different API key

    To do this:

    1. Click EDIT API KEY.

    2. In the Configure your Stripe Source page, provide a different API key from your Stripe account that has the required permissions to access the objects.

      New API key

    3. Click TEST & CONTINUE.

  • Edit the existing API key in Stripe: You can edit your restricted API key in Stripe to provide the required permissions for Hevo to access the selected objects. After editing the API key, in the Select the Objects you want to replicate page, click REFRESH to see the updated list of objects that Hevo has access to. Read Stripe FAQs for steps to edit you existing API key.

In case you continue without providing the required permissions, the objects are not ingested and Hevo displays the PERMISSION DENIED message for these objects in the Pipeline Overview page.

Permission denied alert


Source Considerations

  • Stripe informs the user of a change in an object in their account through an event. For example, when a charge succeeds, it creates a charge.succeeded event. Stripe maintains a record of all the events that it created for the past 30 days in an events object. Hevo refers to this events object while replicating the data to the Destination to make sure all the deletes and updates are duly accounted for.

    Select Events Object

    Make sure to select Stripe’s events object while creating your Pipeline, else, Hevo will not be able to capture any updates. To enable Hevo to capture deletes for your Pipeline, you need to contact Hevo Support.

  • Pausing a Pipeline for more than 30 days can lead to a loss of data, as the events object tracks updates and deletes of the past 30 days only.


Data Replication

Default Pipeline Frequency Minimum Pipeline Frequency Maximum Pipeline Frequency Custom Frequency Range (Hrs)
30 Mins 15 Mins 24 Hrs 1-24

Note: The custom frequency must be set in hours, as an integer value. For example, 1, 2, 3 but not 1.5 or 1.75.

  • Historical Data: Hevo ingests your historical data in reverse order, starting from the latest to the earliest Event for all Pipelines. This enables you to have quicker access to your most recent data.

    • For Pipelines created before Release 1.80: Hevo ingests historical data on the basis of the offset selected at the time of creating the Pipeline. By default, it fetches the entire past data associated with your Stripe account.
    • For Pipelines created after Release 1.80: You can select the Historical Sync Duration as per your requirement while creating a Pipeline. Default value: 3 Months.

    Note: The time taken to fetch the historical data depends on the number of days selected at the time of creating the Pipeline. Hevo defaults this period to 30 days. If required, you may customize it to any number or even select the starting date of your Stripe account.

  • Incremental Data: Once the historical load is completed, each run of the Pipeline fetches the next available new and modified data. Hevo also fetches the Events object from Stripe, which tracks the updates and deletes made in the last 30 days, in order to sync the related Events with your Destination.

Updated and Deleted Objects

Hevo replicates the updated and deleted records using the Stripe events object. For the deleted records, Hevo sets the is_row_deleted column to True and the deleted_timestamp column to the timestamp when the record was deleted in Stripe, in the ingested data and the Destination.

Hevo identifies the updated and the deleted records from the Stripe events object and replicates that information for the following objects:

Object Name Event Type Tracked
ACCOUNT updated
BANK_ACCOUNT deleted
CARD deleted
CHARGE updated
CUSTOMER - updated
- deleted
DISPUTE updated
INVOICE - updated
- deleted
INVOICE_ITEM deleted
ORDER updated
PAYMENT_INTENT updated
PAYOUT updated
PERSON deleted
PRICE deleted
PRODUCT - updated
- deleted
REFUND updated
SETUP_INTENT updated
SUBSCRIPTION updated

Note: As Hevo also replicates the deleted data now, you might see more Events in your Destination than in the Source.


Schema and Primary Keys

Hevo uses the following schema to upload the records in the Destination:

Data Model

The following is the list of tables (objects) that are created at the Destination when you run the Pipeline:

Table Description
Account Contains all the properties of a Stripe account, such as its current email address and whether the account is enabled to make live charges.
Application Fee Contains the transaction fee amount collected on top of a charge made for a user.
Application Fee Refund Contains the reference to the application fee collected from the user and the portion of it to be refunded in case of a transfer reversal.
Authorization Contains the information that must be approved for any purchase to be completed successfully from an issued card.
Balance transaction Contains the details of funds moving through an account. It is created for every in-flow and out-flow of funds to and from your Stripe account. Read Balance transaction types.
Bank Account Contains information of the bank accounts connected to the payment methods of the customers object.
Capability Contains the details of the capabilities associated with a connected account. The level of information required for an account is determined by its capabilities.
Card Contains details such as cardholder, and the month and year of expiry of the physical or virtual cards that are issued to the cardholders.
Cardholder Contains information such as the name, email, and phone number of the individual or business entity who issues a card.
Charge Contains the details of a charge made on a credit or a debit card. You can retrieve and refund individual charges as well as list all charges. They are identified by a unique, random ID.
Credit Note Contains the adjusted or refunded amount to be added to an already finalized invoice instead of recreating the invoice.
Credit Note line item Contains the individual adjustments or refunds that you are adding to a credit note associated with an invoice.
Customer Contains the details of a customer, such as the name, email, default payment method and delinquency status.
Customer Balance Transaction Contains the amount by which a customer’s account balance needs to be incremented or decremented.
Customer Tax IDs Contains details of the tax ID(s) of a customer.
Coupon Contains the information about a percent-off or amount-off discount you might want to apply to a customer. It can be applied to invoices or orders.
Discount Contains the information such as the beginning and ending of the discount and the coupon object. It represents the actual application of a coupon to a particular customer.
Disputes Contains details of customer disputes about charges they do not recognise, suspect may be fraud.
Dispute Balance Transaction Contains the mapping between each dispute and transaction entities.
It is a custom table generated by Hevo to maintain data sanity.
Early Fraud Warning Contains details about early fraud warnings sent by the card issuer to Stripe for a transaction.
Events Contains the details about the events generated by your Stripe account.
Fee Detail Contains the detailed breakdown of fees paid for each transaction present in the balance transaction table.
File Link Contains a URL that you can use to retrieve the contents of a file without any authentication.
Invoice Contains the statements of amounts owed by a customer. Invoices are either generated one-off or periodically from a subscription.
Invoice Item Contains the details about the charges or credits made to the customer’s credit cards.
Invoice Line Item Contains the details of an invoice, such as amount, currency, period and proration. It is stored inside the invoice object.
Issue Disputes Contains the details of the disputes raised by the customers to recover money for transactions completed using the Issuing API.
Mandates Contains the details of the permissions given by the customer to debit their payment method.
Orders Contains details of the customer’s purchases, such as products and quantities, order status, payment information, and billing and shipping information.
Payment Intent Contains the guide for the process of collecting a payment from your customer during a particular session.
Read How intents work to know more about the payment flow followed at Stripe.
Payment Link Contains the URLs of pages that customers use to make payments.
Payment Method Contains the details of the payment method that a customer has used on one or more payment intent(s) along with details. Supported Payment Methods are Card (card/wallet), FPX, and ideal.
Payment Method Details Contains the detailed data for each payment method that is created on your Stripe Account, will be individually stored in your destination to maintain data sanity.
Examples:
- payment_method_card
- payment_method_fpx
- payment_method_ideal
Payout Contains the details of the received funds from Stripe. It is created when you initiate a payout to either a bank account or debit card of a connected Stripe account.
Person Contains the details of the person linked to a Stripe account.
Plan (pricing) Contains the details such as the base price, currency and billing cycle for recurring purchases of products. It helps you track the pricing.
Price Contains the details such as the unit cost, currency, and billing cycle for recurring and one-time purchases of the different products you offer to your customers. For example, a standard or premium version of your service.
Product Contains a list of all offers for the product. Each product can have multiple tiers of pricing plans.
Promotion Code Contains a customer-redeemable code that can be used to create single or multiple codes for a coupon.
Quote Contains the details of the quotation document provided to the customer for certain products or services. A quote is a document that contains a fixed price of a product.
Refund Contains the details of a charge that is to be refunded due to a dispute, return, or failure to deliver a service.
Review Contains human-added information to supplement automated fraud detection cases.
Sessions Contains the details of the customer’s session when they pay for one-time purchases or subscriptions.
Setup Attempts Contains the details of the attempts at setting up a payment method by the customers.
Setup Intent Contains the guide for the process of saving a customer’s payment credentials for future payments.
Shipping Rates Contains the details of the shipping rates applied to products being delivered to the customers.
Source Contains the details of a customer’s payment methods that can be used with the Stripe API.
Subscription Contains the details of a recurring charge for a customer. Each subscription has multiple line items.
Subscription Item Contains line item data for each subscription. Each line item is associated with a plan.
Subscription Schedule Contains the attributes required to manage the lifecycle of customers’ subscriptions by predefining existing changes.
Tax Code Contains the details of the tax codes used to classify the goods and services for tax purposes.
Tax Rate Contains the details of the tax rates applied to the invoices, subscriptions, and checkout sessions to collect tax.
Tokens Contains the details of the tokens used by Stripe to collect sensitive customer information such as bank accounts and credit cards.
Top-Up Contains the amount by which customers wish to top up their Stripe account balance.
Transaction Contains details of funds entering or leaving a Stripe account. A transaction object is created whenever an issued card is used for making a purchase or a refund.
Transfer Contains the details to move funds between Stripe accounts such as amount, destination and so on. It is used when you need to directly move a fund from your customer to the seller who’s offering the service through your platform.
Transfer Reversal Contains the amount that is to be deducted from the seller’s account (who originally received it via a successful transfer) and added back to your platform.
Usage Record Contains a report of a customer’s usage and other metrics that are sent to Stripe.
Value Contains the details of the values in a value list, for example, an email ID. Value lists group values together so that they can be referenced in rules. For example, a value list containing email addresses can be referenced for the rule that blocks any payment with the email address on this list.
Value List Item Contains the details of the items that can be added to a Stripe Radar value list that is used for fraud prevention.
Verification Report Contains the details of the data of your users while collecting and verifying their identities.
Verification Sessions Contains the details of the processes you need to follow for collecting and verifying the identities of your users, and the verified data after the verification is complete.

Limitations

  • For new and existing Pipelines, Hevo replicates only the data that has been deleted after Release 1.86.



Revision History

Refer to the following table for the list of key updates made to this page:

Date Release Description of Change
Sep-21-2022 1.98 Updated sections, Schema and Primary Keys, and Data Model to add information about the new Stripe objects supported by Hevo.
Aug-24-2022 1.96 Added section, Object Selection.
Jul-27-2022 NA - Renamed section, Creating the API Key to Obtaining the API Key.
- Updated section, Obtaining the API Key to include information about standard and restricted API keys.
Apr-11-2022 1.86 - Updated the Data Replication and Source Considerations sections to add information about how Hevo handles deletes in Stripe objects.
Mar-07-2022 1.83 Added information about reverse historical load in the Data Replication section.
Jan-24-2022 1.80 Added information about configurable historical sync duration in the Data Replication section.
Jan-03-2022 1.79 Added objects to section, Data Model.
Oct-25-2021 NA Added the Pipeline frequency information in the Data Replication section.
Aug-23-2021 1.70 Added the section, Source Considerations.
Last updated on 21 Sep 2022

Tell us what went wrong