What does Atlan crawl from Confluent Kafka?

Have more questions? Submit a request

Atlan crawls and maps the following assets and properties from Confluent Kafka.

Once you've crawled Confluent Kafka, you can use connector-specific filters for quick asset discovery. The following filters are currently supported for these assets:

  • Topics β€” Message count filter

Topics

Atlan maps topics from Confluent Kafka to its KafkaTopic asset type.

Source property Atlan property
Topic name
PartitionCount kafkaTopicPartitionsCount
ReplicationFactor kafkaTopicReplicationFactor
segment.byte kafkaTopicSegmentBytes
compression.type kafkaTopicCompressionType
cleanup.policy kafkaTopicCleanupPolicy
isInternal kafkaTopicIsInternal
sizeInBytes kafkaTopicSizeInBytes
recordCount kafkaTopicRecordCount

Consumer groups

Atlan maps consumer groups from Confluent Kafka to its KafkaConsumerGroup asset type.

Source property Atlan property
GROUP name
memberCount kafkaConsumerGroupMemberCount
ReplicationFactor kafkaTopicReplicationFactor
topic_names kafkaTopicNames
TOPIC kafkaConsumerGroupTopicConsumptionProperties.topicName
PARTITION kafkaConsumerGroupTopicConsumptionProperties.topicPartition
LAG kafkaConsumerGroupTopicConsumptionProperties.topicLag
CURRENT-OFFSET kafkaConsumerGroupTopicConsumptionProperties.topicCurrentOffset

Related articles

Was this article helpful?
0 out of 0 found this helpful