Hive Logging (Hive 2.3 and Later)

For Hive 2.3 and later starting with EEP 6.3.0, this topic describes the folder structure of the Hive logs and includes details about the log-file contents and how log files are installed in multinode installations.

Hive Log Folder Structure

Table 1 shows the Hive log folder structure:

Table 1. Hive Logs
Folder or File				Description
`${HIVE_HOME}/log`				Root folder for all Hive logs
	`hive-${ADMIN_USER}-hiveserver2-${HOSTNAME}.out hive-${ADMIN_USER}-metastore-${HOSTNAME}.out init_derby_db_${TIMESTAMP}.log`			Each service has a separate file for logging
`${HIVE_HOME}/log/${ADMIN_USER}`				Root folder for admin cluster logs
		`${ADMIN_USER}-hiveserver2-${HOSTNAME}.log ${ADMIN_USER}-metastore-${HOSTNAME}.log ${ADMIN_USER}-cli-${HOSTNAME}.log`		Cluster admin log files
`${HIVE_HOME}/log/${ADMIN_USER}/webhcat/`				Root folder for webHcat logs
			`webhcat.log webhcat-console.log webhcat-console-error.log`	WebHcat log files
`${HIVE_HOME}/log/${OTHER_USER}`				Root folder for a user other than the admin user
		`${OTHER_USER}-cli-${HOSTNAME}.log`		CLI log for a user other than the admin user

In Table 1:

This element	Represents
`${HIVE_HOME}`	The Hive home folder, which is usually `/opt/mapr/hive/hive`.
`${ADMIN_USER}`	The admin user of a cluster that runs HiveServer2 and HiveMetastore daemons. Usually, this is the `mapr` user.
`${HOSTNAME}`	The name of the host where the daemon runs.
`${TIMESTAMP}`	The date and time of log creation.
`${OTHER_USER}`	A user other than the admin user.
`${OTHER_USER}-cli-${HOSTNAME}.log`	The log file that is created when `${OTHER_USER}` launches the Hive CLI.

Content of Log Files

Table 2 shows the content of the log files:

Table 2. Log File Contents
File Name	Description
`hive-${ADMIN_USER}-hiveserver2-${HOSTNAME}.out`	Contains information about when the HiveServer2 daemon was started and the PID of the file.
`hive-${ADMIN_USER}-metastore-${HOSTNAME}.out`	Contains information about when the HiveMetastore daemon was started and the PID of the file.
`${ADMIN_USER}-hiveserver2-${HOSTNAME}.log`	Contains information from the HiveServer2 daemons. This file also contains the job progress.
`${ADMIN_USER}-metastore-${HOSTNAME}.log`	Contains information from the HiveMetastore daemons.
`${ADMIN_USER}-cli-${HOSTNAME}.log`	Created when a user runs the Hive CLI over the `${ADMIN_USER}`. This file contains the job progress.
`${OTHER_USER}-cli-${HOSTNAME}.log`	Created when a user runs the Hive CLI over the ${OTHER_USER}. This file also contains the job progress.
`init_derby_db_${TIMESTAMP}.log`	Created if and only if Hive was configured for Apache Derby through cluster installation.

Log Files in a Multinode Hive Installation

Table 3 shows a Hive multinode installation (that is, Hive packages installed on different nodes):

Table 3. Cluster Configuration
	Hive Metastore	HiveServer2	HiveWebHCat
node1	✓
node2		✓
node3			✓

See Table 4 for the log configurations:

Table 4. Log Configuration on Hive Multi-Node Installation
File or Folder Name	node1	node2	node3
`hive-${ADMIN_USER}-metastore- ${HOSTNAME}.out`	✓
`${ADMIN_USER}-metastore- ${HOSTNAME}.log`	✓
`hive-${ADMIN_USER}-hiveserver2- ${HOSTNAME}.out`		✓
`${ADMIN_USER}-hiveserver2- ${HOSTNAME}.log`		✓
`/webhcat/`			✓
`webhcat.log`			✓
`webhcat-console.log`			✓
`webhcat-console-error.log`			✓
`/${ADMIN_USER}/`	✓	✓	✓