How to score engagement quality, not just quantity

How to score engagement quality, not just quantity

Message count is the vanity metric of communities — easy to inflate, weakly tied to health. A server can be loud and dying. Here's a procedure to score the quality underneath the volume.

Step 1: compute conversation depth
For a sample of threads/replies, measure replies-per-conversation and unique-participants-per-conversation. Monologues and '+1' walls score low even at high volume.

Step 2: compute reciprocity
Build a who-replies-to-whom map. Healthy channels show many-to-many edges; unhealthy ones show a hub-and-spoke (one person answers everyone, or nobody answers anyone).

Step 3: track median reply latency
The time between a question and its first answer is a leading indicator. Rising latency precedes visible decline, because people stop bothering before they stop showing up.

What the network shape reveals
Servers with dense reciprocal graphs retain better than equally-active hub-and-spoke ones — distributed answering means the community, not one hero, holds it together.

The caveat
Network metrics need a logging bot and enough volume to be stable; small channels are too noisy to graph meaningfully. Don't over-read a sparse week.

Open question: can you intervene to make a hub-and-spoke graph more reciprocal, or is that structure baked in by who the founding members are?