Connect to External Systems¶
Confluent Cloud offers pre-built, fully managed, Apache Kafka® Connectors that make it easy to instantly connect to popular data sources and sinks. With a simple UI-based configuration and elastic scaling with no infrastructure to manage, Confluent Cloud Connectors make moving data in and out of Kafka an effortless task, giving you more time to focus on app development.
- Source Connector
- A source connector, such as the Microsoft SQL Server Source Connector, ingests entire databases and streams table updates to Kafka topics. It can also collect metrics from all of your application servers and store these in Kafka topics, making the data available for stream processing with low latency.
- Sink Connector
- A sink connector delivers data from Kafka topics into secondary indexes, such as Google BigQuery or batch systems like Amazon S3, for offline analysis.
Note
See Limitations for Connect Confluent Cloud limitations.
Supported connectors¶
The following Confluent Cloud connectors are supported by Confluent:
- Amazon CloudWatch Metrics Sink Connector
- Amazon Kinesis Source Connector
- Amazon Redshift Sink Connector
- Amazon SQS Source Connector
- Amazon S3 Sink Connector
- AWS Lambda Sink Connector
- Azure Blob Storage Sink Connector
- Azure Cognitive Search Sink Connector
- Azure Cosmos DB Sink Connector
- Azure Data Lake Storage Gen2 Sink Connector
- Azure Event Hubs Source Connector
- Azure Functions Sink Connector
- Azure Service Bus Source Connector
- Azure Synapse Analytics Sink Connector
- Datadog Metrics Sink Connector
- Datagen Source Connector (development and testing)
- Elasticsearch Service Sink Connector
- Google BigQuery Sink Connector
- Google Cloud Functions Sink Connector
- Google Cloud Spanner Sink Connector
- Google Cloud Storage Sink Connector
- Google Pub/Sub Source Connector
- HTTP Sink Connector
- Microsoft SQL Server CDC Source Connector (Debezium)
- Microsoft SQL Server Sink Connector (JDBC)
- Microsoft SQL Server Source Connector (JDBC)
- MongoDB Atlas Sink Connector
- MongoDB Atlas Source Connector
- MySQL CDC Source Connector (Debezium)
- MySQL Sink Connector (JDBC)
- MySQL Source Connector (JDBC)
- Oracle Database Source Connector
- PostgreSQL CDC Source Connector (Debezium)
- PostgreSQL Sink Connector (JDBC)
- PostgreSQL Source Connector (JDBC)
- Salesforce Platform Event Sink Connector
- Salesforce CDC Source Connector
- Salesforce Platform Event Source Connector
- Salesforce PushTopic Source Connector
- Salesforce SObject Sink Connector
- ServiceNow Sink Connector
- ServiceNow Source Connector
- Snowflake Sink Connector
- Solace Sink Connector
Preview connectors¶
Caution
Preview connectors are not currently supported and are not recommended for production use.
The following Confluent Cloud connectors are available for preview:
Cloud platforms support¶
The following table shows the cloud platforms supported by each Confluent Cloud connector.
| Cloud Connector | AWS | Azure | GCP |
|---|---|---|---|
| Amazon CloudWatch Metrics Sink Connector | Yes | No | No |
| Amazon Kinesis Source Connector | Yes | Yes | Yes |
| Amazon Redshift Sink Connector | Yes | No | No |
| Amazon SQS Source Connector | Yes | Yes | Yes |
| Amazon S3 Sink Connector | Yes | No | No |
| AWS Lambda Sink Connector | Yes | No | No |
| Azure Blob Storage Sink Connector | No | Yes | No |
| Azure Cognitive Search Sink Connector | No | Yes | No |
| Azure Cosmos DB Sink Connector | No | Yes | No |
| Azure Data Lake Storage Gen2 Sink Connector | No | Yes | No |
| Azure Event Hubs Source Connector | Yes | Yes | Yes |
| Azure Functions Sink Connector | No | Yes | No |
| Azure Service Bus Source Connector | Yes | Yes | Yes |
| Azure Synapse Analytics Sink Connector | No | Yes | No |
| Datadog Metrics Sink Connector | Yes | Yes | Yes |
| Datagen Source Connector | Yes | Yes | Yes |
| Elasticsearch Service Sink Connector | Yes | Yes | Yes |
| Google BigQuery Sink Connector | No | No | Yes |
| Google Cloud Dataproc Sink Connector | No | No | Yes |
| Google Cloud Functions Sink Connector | No | No | Yes |
| Google Cloud Spanner Sink Connector | No | No | Yes |
| Google Cloud Storage Sink Connector | No | No | Yes |
| Google Pub/Sub Source Connector | Yes | Yes | Yes |
| HTTP Sink Connector | Yes | Yes | Yes |
| Microsoft SQL Server Sink Connector | Yes | Yes | Yes |
| Microsoft SQL Server Source CDC Connector (Debezium) | Yes | Yes | Yes |
| Microsoft SQL Server Source Connector | Yes | Yes | Yes |
| MongoDB Atlas Sink Connector | Yes | Yes | Yes |
| MongoDB Atlas Source Connector | Yes | Yes | Yes |
| MySQL Source CDC Connector (Debezium) | Yes | Yes | Yes |
| MySQL Sink Connector | Yes | Yes | Yes |
| MySQL Source Connector | Yes | Yes | Yes |
| Oracle Database Source Connector | Yes | Yes | Yes |
| PostgreSQL CDC Source Connector (Debezium) | Yes | Yes | Yes |
| PostgreSQL Sink Connector | Yes | Yes | Yes |
| PostgreSQL Source Connector | Yes | Yes | Yes |
| Salesforce CDC Source Connector | Yes | Yes | Yes |
| Salesforce Platform Event Sink Connector | Yes | Yes | Yes |
| Salesforce Platform Event Source Connector | Yes | Yes | Yes |
| Salesforce PushTopic Source Connector | Yes | Yes | Yes |
| Salesforce SObject Sink Connector | Yes | Yes | Yes |
| ServiceNow Sink Connector | Yes | Yes | Yes |
| ServiceNow Source Connector | Yes | Yes | Yes |
| Snowflake Sink Connector | Yes | Yes | Yes |
| Solace Sink Connector | Yes | Yes | Yes |
Cloud Console connector controls¶
You can use the GUI buttons to start, stop, pause, and delete a connector. Select and display one of your listed connectors to view the controls.
API for fully-managed connectors¶
For information and examples to use with the Confluent Cloud API for fully-managed connectors, see the Confluent Cloud API for Connect documentation.
Internet access to resources¶
Consider the following cluster network types when determining the public Internet access configuration for resources that fully-managed connectors must access. For full Confluent Cloud networking details, see Networking in Confluent Cloud.
Public and Private IP addresses¶
Refer to the following sections to determine your cloud networking configuration for connectors that do not use static egress IP addresses.
Public IP address accessible over the Internet¶
The following table provides connectivity information for a managed connector with a public IP address accessible over the Internet.
| Cluster Network Type | Connectivity Supported? | Data Plane Network Access | Control Plane Network Access |
|---|---|---|---|
| Public Endpoint | Yes | Public IP/CIDR range where the Confluent Cloud cluster is located. | Public IP/CIDR range for us-west-2 |
| VPC Peering and Transit Gateway | Yes | Public IP/CIDR range where the Confluent Cloud cluster is located. | Public IP/CIDR range for us-west-2 |
| Private Link | Yes | Public IP/CIDR range where the Confluent Cloud cluster is located. | Public IP/CIDR range for us-west-2 |
Tip
See the following cloud provider documentation for additional information:
Private IP address hosted in a customer VPC¶
The following table provides connectivity information for a managed connector with a private IP address hosted in a customer VPC.
| Cluster Network Type | Connectivity Supported? | Data Plane Network Access | Control Plane Network Access |
|---|---|---|---|
| Public Endpoint | No | N/A | N/A |
| VPC Peering and Transit Gateway | Yes | Source IP address used is from the /16 CIDR configured by the customer for the Confluent Cloud Cluster. | None |
| Private Link | No | N/A | N/A |
Static Egress IP Address Support¶
To determine your cloud networking configuration, refer to the following: for connectors running on AWS that use static egress IP addresses.
Important
To use static egress IP addresses, you must be running a multitenant or Dedicated cluster on AWS and on the public Internet (without VPC peering or AWS PrivateLink).
Public IP address accessible over the Internet¶
The following table provides connectivity information for a managed connector with a public IP address accessible over the Internet.
| Cluster Network Type | Connectivity Supported? | Data Plane Network Access | Control Plane Network Access |
|---|---|---|---|
| Public Endpoint (AWS) | Yes | Fixed set of static egress IP addresses (see static egress IP addresses). | None |
| Public Endpoint (Azure/GCP) | Yes | Public IP/CIDR range where the Confluent Cloud cluster is located. | Public IP/CIDR range for us-west-2 |
| VPC Peering and Transit Gateway | Yes | Public IP/CIDR range where the Confluent Cloud cluster is located. | Public IP/CIDR range for us-west-2 |
| Private Link | Yes | Public IP/CIDR range where the Confluent Cloud cluster is located. | Public IP/CIDR range for us-west-2 |
Tip
See the following cloud provider documentation for additional information:
Private IP address hosted in a customer VPC¶
The following table provides connectivity information for a managed connector with a private IP address hosted in a customer VPC.
| Cluster Network Type | Connectivity Supported? | Data Plane Network Access | Control Plane Network Access |
|---|---|---|---|
| Public Endpoint (AWS) | No | N/A | N/A |
| Public Endpoint (Azure/GCP) | No | N/A | N/A |
| VPC Peering and Transit Gateway | Yes | Source IP address used is from the /16 CIDR configured by the customer for the Confluent Cloud Cluster. | None |
| Private Link | No | N/A | N/A |
Connector data previews¶
For information about connector data previews, see Connector Data Previews.
View connector events¶
For information about viewing Confluent Cloud connector events, see View Connector Events.
Single message transforms¶
For information about using single message transforms (SMTs), see Single Message Transforms for Managed Connectors.
API for fully-managed connectors¶
For information and examples to use with the Confluent Cloud API for fully-managed connectors, see the Confluent Cloud API for Connect documentation.
Service accounts¶
For information about setting up service accounts, see Service Accounts.
Dead letter queue¶
For information about accessing and using the Confluent Cloud Dead Letter Queue, see Dead Letter Queue.
Connector limitations¶
To view a list of connector limitations, see Limitations.
About preview features¶
For a description of the Confluent Cloud Connect preview program, see Confluent Cloud Connect Preview.