Dec 2, 2020

Understanding Pulse Elasticsearch data

Starting with v6, the open-source version of Elasticsearch is embedded into Pulse. Elasticsearch is a search engine that helps Pulse store all the information about your TM1 application second by second.

This information can then be visualized with the Pulse Explorer. Pulse Explorer is powered by the open-source version of Kibana. Kibana helps you visualize and build your own dashboard on top of the Pulse Elasticsearch data.

Accessing the Pulse Explorer

To access Pulse Explorer, first log-in to Pulse with an admin user and then go to Reports > Pulse Explorer:

Kibana will open in a new tab:

The open-source version of Kibana comes with the following features:

List of recently viewed items

Discover: To explore Pulse Elasticsearch data

Visualize: To create visualization

Dashboard: Group your visualizations together

Dev tools: Access Kibana console for advanced tasks

Administration: Administration tasks such as Import/Export dashboars

The Discover mode

Before building dashboards on top of the Pulse Elasticsearch data, you will need to be familiar with the data structure. The Discover mode is here to help you understand the data.

After clicking on the Discover icon, you should be able to see this, just below filder you can change the index:

Indices

Instead of a database, Elasticsearch is using indices. To learn more about Elasticsearch indices, you can read the following article:

MySQL structure vs Elasticsearch:

  • MySQL => Databases => Tables => Columns/Rows
  • Elasticsearch => Indices => Types => Documents with Properties

Pulse Elasticsearch data is structured into 10 indices:

NameDescription
pulse-alert-historyInformation about Pulse Alerts
pulse-chore-process-historyChores and processes history execution time
pulse-server-action-historyInformation about Pulse actions such as stop/start services and disconnect users
pulse-server-stateAll Server information such as server stats including all active threads
pulse-tm1-client-historyInformation about the }Clients dimension
pulse-tm1-object-historyInformation about all TM1 objects
pulse-tm1-server-logServer logs information
pulse-tm1-session-historyAll Sessions history
pulse-tm1-topInformation about all threads
pulse-tm1-user-historyUser usage tracking (Excel, TM1 Web, Canvas and Pulse)

Indices split by periods

To easier manage the volume data, the indices are split by periods (daily, monthly, yearly), indices that will contain lots of data are split daily (a new index is created every day) whereas indices that store less information are split monthly or yearly (a new index is created every month or year):

  • New indices are created daily for:

    • pulse-tm1-top (disabled by default)
  • New indices are created monthly for:

    • pulse-server-state
    • pulse-tm1-server-log
  • New indices are created yearly for:

    • pulse-alert-history
    • pulse-chore-process-history
    • pulse-server-action-history
    • pulse-tm1-object-history
    • pulse-tm1-session-history
    • pulse-tm1-user-history

On the Pulse status page, you can see all indices per # of documents and size in the disk. In the example below, we can see there is one pulse-server-state index for 2020-12 (pulse-server-state-2020-12) whereas there are one pulse-tm1-top index per day (pulse-tm1-top-2020-12-11, pulse-tm1-top-2020-12-10…):

The DocumentType property

Some indices such as the pulse-server-state group different types of information such as information about server, instances and TM1top info. Pulse is using the DocumentType properties to group these different type of information.

Once the index selected, adding a DocumentType filter will reduce the fields to the relevant information:

Please find below the list of DocumentType per indices:

Index NameDocument TypeDescription
pulse-alert-historyAlertLoginfo about Pulse alerts
pulse-chore-process-historychoreandprocesshistoryChore and Process history info
pulse-server-action-historyServiceControlRequestinfo about who started or stopped instances from Pulse
pulse-server-action-historyDisconnectRequestinfo about who cancel threads from Pulse
pulse-server-stateDriveDrive infromation (DriveFreeSpace, DiveSize…)
pulse-server-stateTM1TopItemActive threads info (DisplayName, Function…)
pulse-server-stateTM1TopStateActive threads status (ThreadsRun, ThreadsWait…)
pulse-server-stateServerStateServer info (CPU, Memory…)
pulse-server-stateServiceStateInstance info (StartUpTime, Status…)
pulse-tm1-client-historyClientTM1 users info (IsAdmin, IsActive…)
pulse-tm1-object-historyRelationRelation info between objects
pulse-tm1-object-historyCubeStatsStats by cube (FedCells, PopulatedNumericCells…)
pulse-tm1-object-historyCubeDimensionDimensions per cube (DimensionName, IndexNumber…)
pulse-tm1-object-historyProcessProcess info (PrologLines, DataLines…)
pulse-tm1-object-historyCubeCube info (Rules info, Views…)
pulse-tm1-object-historyDimensionDimension info (Hierarchies, Subsets…)
pulse-tm1-object-historyDimensionAttributeAttributes info (Type, CreateDate…)
pulse-tm1-object-historyApplicationApplication info (Name, Path…)
pulse-tm1-object-historyChoreChores info (IsActive, Processes…)
pulse-tm1-server-logtm1serverlogMessage log info (MessageLevel, MessageText…)
pulse-tm1-session-historySessionSessions info (Session time, IsAdmin…)
pulse-tm1-topTM1TopItemTM1 Top info (ObjectName, Function…) 
pulse-tm1-user-historyPulseLogWhat Pulse users are doing
pulse-tm1-user-historyWebLogWhat TM1Web users are doing
pulse-tm1-user-historyWorkbookWorkbook info (OpenTime, Path, Type…)
pulse-tm1-user-historyCanvasLogWhat Canvas user are doing

Related articles:

Related content

Loading related content