Atlan crawls and maps the following assets and properties from Confluent Kafka.
Once you've crawled Confluent Kafka, you can use connector-specific filters for quick asset discovery. The following filters are currently supported for these assets:
- Topics β Message count filter
Topics
Atlan maps topics from Confluent Kafka to its KafkaTopic
asset type.
Source property | Atlan property |
---|---|
Topic | name |
PartitionCount | kafkaTopicPartitionsCount |
ReplicationFactor | kafkaTopicReplicationFactor |
segment.byte | kafkaTopicSegmentBytes |
compression.type | kafkaTopicCompressionType |
cleanup.policy | kafkaTopicCleanupPolicy |
isInternal | kafkaTopicIsInternal |
sizeInBytes | kafkaTopicSizeInBytes |
recordCount | kafkaTopicRecordCount |
Consumer groups
Atlan maps consumer groups from Confluent Kafka to its KafkaConsumerGroup
asset type.
Source property | Atlan property |
---|---|
GROUP | name |
memberCount | kafkaConsumerGroupMemberCount |
ReplicationFactor | kafkaTopicReplicationFactor |
topic_names | kafkaTopicNames |
TOPIC | kafkaConsumerGroupTopicConsumptionProperties.topicName |
PARTITION | kafkaConsumerGroupTopicConsumptionProperties.topicPartition |
LAG | kafkaConsumerGroupTopicConsumptionProperties.topicLag |
CURRENT-OFFSET | kafkaConsumerGroupTopicConsumptionProperties.topicCurrentOffset |