Set Up System Cube
What is System Cubeâ
For better supporting self-monitoring, a set of system Cubes are created under the system project, called "KYLIN_SYSTEM". Currently, there are five Cubes. Three are for query metrics, "METRICS_QUERY_EXECUTION", "METRICS_QUERY_SPARK_JOB", "METRICS_QUERY_SPARK_STAGE". And the other two are for job metrics, "METRICS_JOB", "METRICS_JOB_EXCEPTION".
How to Set Up System Cubeâ
Please check document: How to use System Cube in Kylin4.0
Details of System Cubeâ
Common Dimensionâ
For all of these Cube, admins can query at four time granularities. From higher level to lower, it's as follows:
| KYEAR_BEGIN_DATE | year |
| KMONTH_BEGIN_DATE | month |
| KWEEK_BEGIN_DATE | week |
| KDAY_DATE | date |
METRICS_QUERYâ
This Cube is for collecting query metrics at the Spark Execution level.
METRICS_QUERY_SPARK_JOBâ
This Cube is for collecting Spark Job level query metrics.
METRICS_QUERY_SPARK_STAGEâ
This Cube is for collecting Spark Stage level query metrics.
METRICS_JOBâ
In Kylin, there are mainly three types of job:
- "BUILD", for building Cube segments from HIVE.
- "MERGE", for merging Cube segments in storage engine.
- "OPTIMIZE", for dynamically adjusting the precalculated cuboid tree base on the base cuboid in storage engine.
This Cube is for collecting job metrics. The details are as follows:
| Dimension | |
|---|---|
| PROJECT | |
| CUBE_NAME | |
| JOB_TYPE | |
| CUBING_TYPE | in kylin, there are two cubing algorithms, Layered & Fast(InMemory) |
| Measure | |
|---|---|
| COUNT | |
| MIN, MAX, SUM of DURATION | the duration from a job start to finish |
| MIN, MAX, SUM of TABLE_SIZE | the size of data source in bytes |
| MIN, MAX, SUM of CUBE_SIZE | the size of created Cube segment in bytes |
| MIN, MAX, SUM of PER_BYTES_TIME_COST | = DURATION / TABLE_SIZE |
| MIN, MAX, SUM of WAIT_RESOURCE_TIME | a job may includes several MR(map reduce) jobs. Those MR jobs may wait because of lack of Hadoop resources. |
METRICS_JOB_EXCEPTIONâ
This Cube is for collecting job exception metrics. The details are as follows:
| Dimension | |
|---|---|
| PROJECT | |
| CUBE_NAME | |
| JOB_TYPE | |
| CUBING_TYPE | |
| EXCEPTION | when running a job, exceptions may happen. It's for classifying different exception types |
| Measure |
|---|
| COUNT |