Analyzing Graphistry visual session debug logs#
Life happens. This document describes how to inspect the backend logs for loading a visualization and how that may narrow down failures to specific services. For example, if a firewall is blocking file access, the data loader may fail.
It covers the core visualization service. It does not cover the graph upload service nor the investigation template environment.
Prerequisites#
Graphistry starts (seeing
docker pssection of your install guide) with no restart loopsGraphistry documentation loads: going to
mygraphistry.comshows a page similar tohttps://hub.graphistry.com/.Logged into system terminal for a Graphistry server
Setup#
Ensure log location
Modify /etc/docker/daemon.json to send logs to the intended location:
Local: JSON recommended
Remote: see Graphistry AWS/Azure marketplace images for sample Splunk docker driver
Restart docker and Graphistry if modified.
Enable debug logs
In folder ~/data/config, modify custom.env to turn on debug logs:
LOG_LEVEL=DEBUG
GRAPHISTRY_LOG_LEVEL=DEBUG
For more fidelity, use TRACE. Make sure to set back to INFO or higher after: This fixes performance and prevents leaking secrets.
To debug a specific service, you can selectively modify its log level through an environment variable in data/config/custom.env and restart: ./graphistry up -d --force-recreate.
Restart Graphistry (
docker restart <containerid>)Ensure all workers reported in and are ready:
GPU:
docker ps=> healthy forstreamgl-gpu,forge-etl-pythonCPU:
docker ps=> healthy forforge-etl,streamgl-viz
Watch logs:
docker logs -f -t --tail=100: watch all running services, including starting with the past 100 lines eachdocker logs -f -t streamgl-viz streamgl-gpu forge-etl-python: focus on a few viz services
Clear screen before starting the test session.
Start test session:
Navigate browser to https://www.yourgraphistry.com/graph/graph.html?dataset=Facebook
Client logs#
The browser developer panel, especially around console logs and history of network requests, may provide hints at the first failing request and corresponding error
Nginx logs#
Nginx in debug mode should show the graph-page redirect and a viz socket connection. The current route is /graph/socket.io/; worker-number URLs belong to older deployments. A proxy that blocks WebSocket upgrade causes the client to use HTTP polling, with repeated GET and POST requests carrying the same session value.
GET /graph/graph.html?dataset=FacebookRedirect to
GET /graph/graph.html?dataset=Facebook&session=<SESSION_ID>GET /graph/socket.io/?EIO=4&transport=websocket&session=<SESSION_ID>(upgrade), orGET/POSTwithtransport=polling&session=<SESSION_ID>(fallback)Visualization data requests under
/streamgl/as the graph loads