System Metadata

While CDAP allows users to tag entities with metadata properties and tags, it also tags entities with system properties and tags by default. These default properties and tags can be retrieved using the CDAP Metadata HTTP RESTful API by setting the scope query parameter to system. These default annotations can be used to discover CDAP entities using the Metadata Search API.

This table lists the system metadata annotations of CDAP entities:

Entity System Properties System Tags
  • Plugins (name and version)
  • Artifact name
  • Programs (name and type) contained in the application
  • Plugins
  • Schedules (name and description)
  • Application name
  • Artifact name
  • n/a
  • Program name and type
  • Program mode (batch or realtime)
  • Workflow node names (for workflows only)
  • Schema (field names and types)
  • Dataset type (FileSet, Table, KeyValueTable, etc.)
  • TTL (Time To Live)
  • Dataset name
  • batch for Datasets accessible through Map Reduce or Spark programs
  • explore for Datasets that can be queried through Explore interface
  • Schema (field names and types)
  • TTL
  • Stream name
Stream Views
  • Schema (field names and types)
  • View name