Memory profiling a nano node

This is a primer for developers interested in profiling the memory consumption of the nano node. By completing these steps you'll be able to generate a memory consumption trace file and load this trace in to the heaptrack_gui tool for inspection.

Requirements:

  • A Linux distribution
  • The heaptrack and heaptrack-gui tools. These are available on Debian based distributions with: "apt install heaptrack heaptrack-gui". Additional information on the heaptrack tool can be obtained from their GitHub repository: GitHub - KDE/heaptrack: A heap memory profiler for Linux
  • The nano_node executable with debug symbols.

These steps are not able to inspect a node running within a container such as Docker. To ensure debug symbols are retained set the cmake variable CMAKE_BUILD_TYPE=RelWithDebInfo. Without debug symbols the traces will be illegible.

Generating a trace:
Instruct heaptrack to profile nano_node: "heaptrack ./nano_node --daemon"
Heaptrack will indicate where it is generating the trace file: 'heaptrack output will be written to "/tmp/heaptrack.APP.PID.gz"'
Run the node during test conditions.

Interpreting a trace:
Load the trace file in to the heaptrack_gui analyzer: "heaptrack_gui heaptrack.nano_node.2958.gz" Analyzing the trace files can take some time, especially if they're big.

Analyzing the trace file can be daunting at first but there are 3 major categories of things to look for.

Number of "allocations": This indicates the number of calls to malloc by a particular section of code. If this is in a critical section of code this can impact performance and needs to be rewritten.

Amount of memory allocated: Large allocations are make it easier to run in to out of memory conditions. If large sections of memory need to be allocated, they should be allocated infrequently.

Leaks: Memory leaks are calculated on process shutdown. If the process shuts down gracefully, the leak tracking will be generally accurate on reporting leaks. If the process crashes, possibly from out of memory conditions, the leak counts will be indicative of where memory was allocated when the process crashed.

This is a sample of what the heaptrack_gui tool can output.

6 Likes