Skip to main content
Version: 5.0.0

System Log

After being successfully started, Kylin will create a directory named logs/ by default, all logs generated during Kylin runtime will be stored in this directory.

Log files

Log files generated by Kylin is as followings.

kylin.log

This file is Kylin's main log file, whose default logging level is DEBUG.

kylin.out

The standard output of Kylin process will be redirected to this file, including the output of Tomcat and Hive.

kylin.gc

This file is the GC(Garbage Collection) log of Kylin Java process. And it appends pid as filename suffix to avoid being overwritten.

access.log

This file stores the Tomcat access log. It records all HTTP request response information.For example, User-Agent, access URL, etc.

jstack.timed.log

This file records Java stack traces of Java threads of Kylin, which is used to record some threads running status. To avoid the storage overused, only 20 log files can be kept. The new file will replace the oldest one, when exceeding the maximum number.

Note:Because the execution of jstack depends on the .java_pid file written by jvm in the /tmp directory, if the file is deleted (for example, a scheduled clean up script), it will cause the jstack to not run properly, so that the jstack.timed .log will not be generated

check-env.out

The standard output of executing check-env.sh script will be redirected to this file.

check-env.error

The error message of executing check-env.sh script will be redirected to this file.

shell.stderr

The result of running command lines will be stored in this file.

shell.stdout

The standard output of running command lines will be redirected to this file.

kylin.security.log

The log of system start, stop, upgrade, login and logout will be redirected to this file.

Notes: When using LDAP service to implement user authentication, two logs will be recorded for each login failure. Because when using the LDAP service, if the login fails, another method will be used for authentication.

kylin.schedule.log

This file records logs related to task scheduling, whose default logging level is DEBUG.

kylin.query.log

This file records query related logs, whose default logging level is DEBUG.

kylin.build.log

This file records build-related logs, whose default logging level is DEBUG.

kylin.metadata.log

This file records metadata and transaction operations related logs, whose default logging level is DEBUG.

dump.hprof

When Out of Memory (OOM) occurs in Kylin, it will dump the entire heap, which is convenient for checking the cause.

Note: When you have a large memory setting and Out of Memory OOM occurs, the file dump.hprof will occupy a large storage space, which may cause your disk space to be insufficient and the node to be abnormal. You can manually clean up the historical file.

Logging Analysis

Take query as an example, submitting a query on Web UI, and we'll see the following information in kylin.query.log

==========================[QUERY]===============================
Query Id: 8586e718-67b4-c840-61b4-a8898415a154
SQL: select lo_revenue as from p_lineorder;
User: ADMIN
Success: true
Duration: 1.243
Project: ssb100_10
Realization Names: [P_LINEORDER_1]
Index Layout Ids: [30001]
Snapshot Names: []
Is Partial Match Model: [false]
Scan rows: [35000]
Total Scan rows: 35000
Scan bytes: [246530]
Total Scan Bytes: 246530
Result Row Count: 280
Shuffle partitions: 1
Hit Exception Cache: false
Storage Cache Used: false
Storage Cache Type: null
Is Query Push-Down: false
Is Prepare: false
Is Timeout: false
Time Line Schema: massage,end calcite parse sql,end_convert_to_relnode,end_calcite_optimize,end_plan,collect_olap_context_info,end select realization,end_rewrite,to_spark_plan,seg_pruning,fetch_file_status,shard_pruning,executed_plan,collect_result
Time Line: 6,1,4,11,0,0,1,1,14,6,0,0,1,1198
Message: null
Is forced to Push-Down: false
User Agent: null
Scan Segment Count: 1
Scan File Count: 1
Is Refused: false
==========================[QUERY]===============================

The main fields in the above clip are described as follows:

  • Query Id: Query id
  • SQL: Query statement
  • User: The user name to execute the query
  • Success: Status flag of query result, true execution succeeded, false execution failed
  • Duration: Query time (unit: seconds)
  • Project: The name of the project used in the query
  • Realization Names: The name of the model hit by the query
  • Index Layout Ids: ID of the layout hit by the query
  • Snapshot Names: Query the hit snapshot
  • Is Partial Match Model: Partial match model, such as a left B, you can check table a alone
  • Scan rows: Query the number of data rows scanned
  • Total Scan rows: Total rows of data scanned by query
  • Scan bytes: Query the number of data bytes scanned
  • Total Scan Bytes : Query the total number of bytes of scanned data
  • Result Row Count: The number of data rows returned by the query
  • Shuffle partitions: A spark query parameter that affects how many partitions / tasks are generated after a shuffle. It is calculated by kylin.query.engine.sparkl-sql-shuffle-parittions or dynamic calculation. The calculation formula is min (the estimated value of data size and the total number of cores of spark cluster)
  • Hit Exception Cache: Whether to hit the cache of failed queries
  • Storage Cache Used: Whether to hit the cache successfully queried
  • Storage Cache Type: The cache type of the hit query
  • Is Query Push-Down: Is it a push down query
  • Is Prepare: Whether it is a probe query (this item will be true for the query sent by BI)
  • Is Timeout: Whether to timeout
  • Time Line Schema: Steps in query module
  • Time Line: Time spent in each step of the query module (MS)
  • Message:Query the prompt information on the page. The query is successful. This item is null
  • Is forced to Push-Down: Whether to force down
  • User Agent: The environment information used to submit the query
  • Scan Segment Count: Number of scanned segments
  • Scan File Count: Number of scanned files
  • Is Refused: Whether to refuse to query

Logging Configuration

Kylin uses log4j2 to configure logs. Users can edit the kylin-server-log4j.xml file in the $KYLIN_HOME/server/conf/ directory to modify the log level, path, etc. After modification, you need to restart Kylin for the configuration to take effect.

The configuration of all logs starting with kylin and ending with log is in kylin-server-log4j.xml, the configuration code is as follows.

        <Routing name="routing">
<Routes pattern="$${ctx:logCategory}">
<Route>
<RollingFile name="rolling-${ctx:logCategory}"
fileName="${env:KYLIN_HOME}/logs/kylin.${ctx:logCategory}.log"
filePattern="${env:KYLIN_HOME}/logs/kylin.${ctx:logCategory}.log.%i">
<Policies>
<SizeBasedTriggeringPolicy size="268435456"/>
</Policies>
<DefaultRolloverStrategy max="10"/>
<PatternLayout pattern="%d{ISO8601} %-5p %X{request.project}[%t] %c{2} : %mask{%m}%n"/>
</RollingFile>
</Route>

<Route ref="server" key="$${ctx:logCategory}"/>
</Routes>
</Routing>

In the default configuration, log rolling is triggered when the log file reaches 256MB, keeping the last 10 log files.

If you need to configure one of the log files (such as kylin.query.log) separately, you need to add a new Route under the Routes configuration in the above configuration code, and configure the key as the corresponding log file name (query, schedule). It should be noted that the new route needs to be configured before the existing route, otherwise it will not take effect.

The following is an example, modify the rolling strategy of kylin.query.log to trigger at 0:00 every day, back up the last 5 logs.

<Route key="query">
<RollingFile name="rolling-${ctx:logCategory}" fileName="${env:KYLIN_HOME}/logs/kylin.${ctx:logCategory}.log" filePattern="${env:KYLIN_HOME}/logs/kylin.${ctx:logCategory}.log.%i">
<Policies>
<CronTriggeringPolicy schedule="0 0 0 * * ?"/>
</Policies>
<DefaultRolloverStrategy max="5" />
<PatternLayout pattern="%d{ISO8601} %-5p %X{request.project}[%t] %c{2} : %mask{%m}%n" />
</RollingFile>
</Route>

If you need to configure kylin.log, you can modify the RollingRandomAccessFile configuration, for example, change the number of reserved files to 5

<RollingRandomAccessFile name="server" fileName="${env:KYLIN_HOME}/logs/kylin.log" append="true"
filePattern="${env:KYLIN_HOME}/logs/kylin.log.%i" immediateFlush="false" >
<Policies>
<SizeBasedTriggeringPolicy size="268435456"/>
</Policies>
<DefaultRolloverStrategy max="5"/>
<PatternLayout pattern="%d{ISO8601} %-5p %X{request.project}[%t] %c{2} : %mask{%m}%n"/>
</RollingRandomAccessFile>

Error Code In Log

The format of log error code is KE-AABBBCCC, AA refers to the error reporting module, BBB refers to the more detailed business error reporting and CCC refers to the error number.

AADescription
00common
10server
20query
30build
40system
50tool
BBBDescription
000general
001project
002model
003user
004user group
005password
006column
007table
008database
009measure
010dimension
011cc
012index
013job
014sql expression
015license
016email
017file
018kerberos
019catalog
020recommendation
021server
022segment
023diag
024auth
025shell
026metadata
027frequency query
028json