Creating your First Connector

Before you create your first custom connector, follow the given instructions to ensure your development environment is properly set up. This includes meeting the technical requirements, installing the necessary tools, and accessing the required repository.

Setting up your Development Environment

Software Requirements

Before you start development, ensure you have the following software installed:

• Integrated Development Environment (IDE): Choose an IDE you’re comfortable with, such as IntelliJ IDEA or Eclipse.

• Java Development Kit (JDK): You will need JDK 17. If it is not already installed, you can use the following commands:

  curl -s "https://get.sdkman.io" | bash 
  source "$HOME/.sdkman/bin/sdkman-init.sh" 
  sdk version 
  sdk install java 17.0.9-tem 
  sdk env install
  

• Gradle

• Git

Connector Repository Access

After you have installed the necessary software, you need to access the repository where your custom connector code will reside.

• Request a Connector Repository: The test_connector repository houses your custom connector code. If you don’t have access to the repository, ensure you request one.

• Clone the Repository: Next, clone the repository to your local machine using the command line or GitHub’s web interface. Read Cloning a Repository. Ensure you have permission to clone and push code changes to the repository. If not, contact Hevo Support.

  git clone https://github.com/hevoio/test_connector.git
  

After cloning the repository, you need to set up your IDE to work with the project.

1. Open the cloned repository in your preferred IDE.

2. Ensure the IDE is using the correct JDK version (JDK 17) for the project. You can verify this by running java -version and javac -version in your terminal or command prompt. If the IDE is not using JDK 17, configure the correct JDK version in your IDE settings and reload the project to apply the changes.

Note: The repository includes a parent build.gradle.kts file that manages utilities for code cleaning and handles all the dependencies. It also provides the required interfaces and classes to help you implement a custom connector.

git clone https://github.com/hevoio/test_connector.git

Implement your custom connector

Now that your development environment is set up, you can begin implementing your custom connector logic.

The test_connector repository includes a base structure within the src/main/java directory. You can rename this package to reflect your specific connector or create a new package. Ensure the new package name starts with io.hevo.connector. For example, you could use io.hevo.connector.NewConnector.

The test_connector repository also includes a sample connector class, test_connector.java, located in src > main > java/io/hevo/connector > test_connector. This file serves as a starting point for your custom connector development. Copy this file into your io.hevo.connector.NewConnector package to implement the specific logic required for your connector to interact with the target Source and Hevo.

The TestConnector.java file serves as the entry point for your custom connector. This file is responsible for handling the connection, fetching metadata and data from the Source, and closing the connection once processing is complete.

Setting up the Connector Class

Before you start, you need to create a connector class that encapsulates the entire process of initializing, retrieving data, and closing the connection. This is necessary for managing your connection logic and adding testing methods.

The following code snippet shows how to set up the TestConnector class. Every connector must implement the GenericConnector interface, which defines the core operations for interacting with your Source.

public class TestConnector implements GenericConnector { 
    private static final Logger log = LoggerFactory.getLogger(TestConnector.class);

Core Functionalities of the Connector

After, setting up the connector class, within TestConnector.java, you need to implement the following core functionalities:

• UI Configuration: Define the UI group where the user will input their connection details, such as Host, Port, User, and Password.

• Connection Setup: Initialize the connection to your Source.

• Fetch Objects: Fetch a list of tables or objects from the Source.

• Fetch Schema: Fetch the schema for the selected objects, defining field types and constraints.

• Fetch Data: Fetch the data from the Source and transform it into Hevo’s internal format.

• Close Connection: Terminate the connection after the data processing is complete.