Atlan crawls and maps the following assets and properties from AWS Glue.
Databases
Atlan maps databases from AWS Glue to its Database asset type.
| Source property | Atlan property |
|---|---|
| TABLE_CATALOG | name |
| SCHEMA_COUNT | schemaCount |
Schemas
Atlan maps schemas from AWS Glue to its Schema asset type.
| Source property | Atlan property |
|---|---|
| TABLE_SCHEMA | name |
| TABLE_COUNT | tableCount |
| VIEW_COUNT | viewsCount |
| TABLE_CATALOG | databaseName |
| COMMENTS | description |
| CreateTime | sourceCreatedAt |
Tables
Atlan maps tables from AWS Glue to its Table asset type.
| Source property | Atlan property |
|---|---|
| TABLE_NAME | name |
| COMMENTS | description |
| COLUMN_COUNT | columnCount |
| ROW_COUNT | rowCount |
| BYTES | sizeBytes |
| Parameters (recordCount) | rowCount |
| Parameters (sizeKey) | sizeBytes |
| TABLE_TYPE | subType |
| StorageDescriptor (Location) | externalLocation |
| Parameters (typeOfData, classification) | externalLocationFormat |
| PartitionKeys | isPartitioned |
| PartitionData | partitionCount, partitionList |
| CreatedBy | sourceCreatedBy |
| CreateTime, CreationTime | sourceCreatedAt |
| UpdateTime, LastAccessTime | sourceUpdatedAt |
Views
Atlan maps views from AWS Glue to its View asset type.
| Source property | Atlan property |
|---|---|
| TABLE_NAME | name |
| COMMENTS | description |
| COLUMN_COUNT | columnCount |
| VIEW_DEFINITION | definition |
| Parameters (recordCount) | rowCount |
| PartitionKeys | isPartitioned |
| PartitionData | partitionCount, partitionList |
| CreatedBy | sourceCreatedBy |
| CreateTime, CreationTime | sourceCreatedAt |
| UpdateTime, LastAccessTime | sourceUpdatedAt |
Columns
Atlan maps columns from AWS Glue to its Column asset type.
Atlan also supports nested columns up to level 15 for AWS Glue to help you enrich your semi-structured data types:
- Atlan retrieves raw
STRUCTandARRAYtype objects for nested columns up to 15 levels. - View nested columns in the column preview and overview sidebar for your table assets.
- Column-level lineage is supported. Search, enrich metadata, and view lineage for nested columns.
- Tag propagation is currently only supported from parent to nested columns.
- Atlan currently does not parse
MAPtype objects for columns and nested columns.
| Source property | Atlan property |
|---|---|
| COLUMN_NAME | name |
| COMMENTS | description |
| ORDINAL_POSITION, COLUMN_ID | order |
| TYPE_NAME, DATA_TYPE | dataType |
| IS_PARTITION | isPartition |
| PARTITION_ORDER | partitionOrder |