Blocks Server Diagnostics

Sometimes, it's valuable to be able to "look inside" a running Blocks server, to see various metrics on what's going on in terms of memory consumption, etc. This is mainly for internal PIXILAB use, but could occasionally be useful also for partners.

Enabling Logging of Metrics

The first thing you can do is to enable logging of diagnostic information. This is done by adding the following to the server configuration file:

metrics:
  frequency: 1 day
  reporters:
    - type: log
      logger: metrics

The "frequency" option determines how often you want metrics written to the log file. You may want to set this to a shorter value (such as "5 minutes") if you're diagnosing an issue with a shorter time span. Remember to restart Blocks to make it pick up the config file change.

Accessing Metrics on Demand

The above is useful for after-the-fact diagnostics, but will only log information at the configured interval. In some cases, you may want to obtain metrics at will, rather than through the log file. This can be accomplished by also enabling a separate diagnostics endpoint on your Blocks server. This endpoint can then be used to pull all metrics at any moment. The data is provided in JSON format, allowing it to be used from a browser as well as programmatically. To enable this method, add the adminContextPath adminConnectors and under the server heading in your configuration file:

server:
  adminContextPath: /diag
  adminConnectors:
    - type: http
      port: 8081
      acceptorThreads: 1
      selectorThreads: 2
      idleTimeout: 5 seconds

:!: IMPORTANT: If you already had a top-level server entry in your configuration file, take only the items under this heading above and add them to your server entry – do not add a second top-level server entry. Also note that you need to have both these settings and the aforementioned top-level metrics settings to obtain full details.

Once this has been added (and the server restarted) you'll find all metrics under the following URL:

http://<hostname>:8081/diag/metrics?pretty=true

Replace <hostname> with the proper name or IP address of your Blocks server. Note that the diagnostics endpoint for security reasons run on a separate server port (above set to 8081). Hence, if you run a firewall on your Blocks server, you must provide for access to this port. This can be done either by opening a port in the firewall, using the "TCP tunneling" feature of Anydesk or through an SSH tunnel.

Data Provided

Both the methods described above provide the same data (albeit in somewhat different forms). This data describes the internal state of the Blocks server. So it will tell you what's going on, but generally won't tell you why. But it should be helpful in diagnosing issues related to low memory, deadlocks and similar issues.