Confluent Kafka Single Node Image 8.0.2
Amazon-Linux 2023
Features (Default) at a glance
Available as single node only on AWS Marketplace (Confluent License Restrictions)
- Confluent Kafka Node
- KRaft Controller, Kafka Broker, Kafka Connect, Schema-Registry, Kafka-Rest, KSQL & ControlcenterNextgen systemd processes
- KSQL Requires two nodes
- Connector-plugins Installed - providing integration via connectors
- Mandatory mTLS and Kafka Access Control Lists for connecting to Kafka
- Certificate & Kafka-ACL scripts for easy application onboarding or for mTLS
- Certificates provided for KRaft Controller+Kafka-Broker, Kafka-Connect, Schema-Registry, Kafka-Restand ControlcenterNextgen
- ACLs created for KRaft Controller+Kafka-Broker, Kafka-Connect, Schema-Registry, Kafka-Rest, KSQL
- ACL script provided
- KRaft Controller, Kafka Broker, Kafka Connect, Schema-Registry, Kafka-Rest, KSQL & ControlcenterNextgen systemd processes
- Easily integrate with organization's Observability stack
- JMX and Jolokia Enabled Metrics Collection for Kafka-Components
- OS Metrics via Node-Exporter
- Controlcenter Nextgen aldos provided Prometheus & Alertmanager
- Secure Client Access - mandatory mTLS+Kafka ACLs
- Java-Clients, curl and Non-Java Clients - integrate via Kafka-streams
- Use the installed connector-plugins or easily add new plugins
- No development costs + Register for deployment assistance
- No maintenance overhead
- Expert Support available - Apache-Kafka, Confluent
Confluent Ports
The following are the Confluent Kafka Component Ports. Use this as a guide when creating the security groups (for applications access), NACLs etc.
|
Component |
Port# |
Confluent-Service |
Purpose |
|
Kafka Broker |
9092 |
confluent-kafka-server.service |
Kafka Client Port (mTLS) |
|
KRaft Controller |
9093 |
confluent-kcontroller.service |
Controller Port (mTLS) |
|
Kafka Connect |
8083 |
confluent-kafka-connect.service |
Connect Port (mTLS) |
|
Schema Registry |
8081 |
confluent-schema-registry.service |
Connect Port (mTLS) |
|
Kafka Rest |
8082 |
confluent-kafka-rest.service |
Connect Port (mTLS) |
|
KSQL |
8088 |
confluent-ksqldb.service |
Connect Port (mTLS) |
Observability Ports
The following are the Observability Ports. Use this as a guide when creating the security groups (for observability access), NACLs etc.
|
Component |
Port# |
Service |
Provider |
|
Node Exporter |
9100 |
node_exporter.service |
CYLLiX |
|
Prometheus |
9090 |
prometheus.service |
Confluent-ControlCenterNextGen |
|
Alertmanager |
9098 |
alertmanager.service |
Confluent-ControlCenterNextGen |
Metrics Ports
Use this to integrate with the Observability framework in your organization. Optionally we provide an Observability image for Kafka
|
Component |
JMX Port# |
Jolokia Port# |
Purpose |
|
KRaft Controller |
30000 |
7770 |
KRaft Controller |
|
KRaft Broker |
30010 |
7771 |
Broker |
|
Kafka Connect |
30020 |
7772 |
Connector |
|
Schema-Registry |
30030 |
7773 |
Schema-Registry |
|
Kafka Rest |
30040 |
7774 |
Kafka-Rest |
|
KSQL |
30050 |
7775 |
KSQL |
|
ControlCenter |
30060 |
N/A |
ControlCenterNextGen |
Prerequisites
- Networking will need to be defined
- Vnet
- Private Subnets
- Kafka Security Group
- Secure SSH Connectvitiy
- DNS - will need to be defined post deploy
Deployment
- From the AWS Console, select the ami and click Launch instance from AMI
- AWS Console -> EC2 -> Images -> AMIs
- Provide a Name for the instance
- You can use the hostname.domain format or e.g., cfkr04.cyllix.ami
- For Instance type, select t4g.2xlarge (Minimum t4g.xlarge)
- Provide a Key pair
- You can select from the list or create one (don't forget to download this if creating)
- Create a security group with all the following (Recommend creating separate groups)
- Confluent ports (Confluent ports section)
- Observability ports (Observability ports section)
- Metric ports (Metrics ports section)
- Configure storage as required
- Expand Advanced details (Mandatory) - In User data (Replace puthostname.domain with your preferred hostname.domain)
|
#!/bin/bash |
|
- Click on Launch instance (You will find this on the bottom right)
Support
For CYLLiX Support and Inquiries, please contact us at https://www.cyllix.io/customer-portal
For Confluent Support and Inquiries please see https://www.confluent.io