Atlan crawls and maps the following assets and properties from Microsoft Azure Event Hubs.
Once you've crawled Microsoft Azure Event Hubs, you can use connector-specific filters for quick asset discovery. The following filters are currently supported for these assets:
- Event hubs — Message count, size (MB), partition count, and cleanup policy filters
- Consumer groups — Member count and topic name filters
Event hubs
Atlan maps event hubs from Microsoft Azure Event Hubs to its AzureEventHub asset type.
| Source property | Atlan property |
|---|---|
| Topic | name |
| PartitionCount | kafkaTopicPartitionsCount |
| ReplicationFactor | kafkaTopicReplicationFactor |
| segment.byte | kafkaTopicSegmentBytes |
| compression.type | kafkaTopicCompressionType |
| cleanup.policy | kafkaLogTopicCleanupPolicy |
| isInternal | kafkaTopicIsInternal |
| sizeInBytes | kafkaTopicSizeInBytes |
| recordCount | kafkaTopicRecordCount |
| status | azureEventHubStatus |
| retention.ms | kafkaTopicRetentionTimeInMs |
Consumer groups
Atlan maps consumer groups from Microsoft Azure Event Hubs to either of the following asset types:
-
KafkaConsumerGroup— managed via Kafka clients. -
AzureEventHubConsumerGroup— managed via Azure portal, SDK, or Azure Resource Manager templates.
💪 Did you know? Consumer groups are most likely to show up only in streaming scenarios. This is because if a topic is not being consumed actively, Microsoft Azure Event Hubs will purge the consumer group. So, if a consumer group is inactive while the workflow runs in Atlan, it will not be cataloged as an asset.
| Source property | Atlan property |
|---|---|
| GROUP | name |
| memberCount | kafkaConsumerGroupMemberCount |
| ReplicationFactor | kafkaTopicReplicationFactor |
| topic_names | kafkaTopicNames |
| TOPIC | kafkaConsumerGroupTopicConsumptionProperties.topicName |
| PARTITION | kafkaConsumerGroupTopicConsumptionProperties.topicPartition |
| LAG | kafkaConsumerGroupTopicConsumptionProperties.topicLag |
| CURRENT-OFFSET | kafkaConsumerGroupTopicConsumptionProperties.topicCurrentOffset |