Logs, alerts, and evidence

Logs are records; alerts are bets; evidence has provenance

A log is a record that something emitted: “account jordan.h@riverstone.example authenticated from 203.0.113.40 at 02:14.” An alert is a rule or model that decided a human should look: “impossible travel” or “new device plus export.” Evidence is a record you are willing to cite in a decision, with enough provenance that someone else can tell what it is. NIST SP 800-61 Revision 3 treats incident handling as an evidence-preserving process, not as a chat thread of hunches.

Maya’s first skill is to copy the record before decorating it. The log does not say Jordan the human was awake. It does not say the address is a cafe. It does not say data left TrackPort. Those are hypotheses. CSF 2.0’s Detect and Respond functions only work if the organization can tell an observation from a story.

Worked note: the 02:14 login

Observed: identity-provider success log for jordan.h at 02:14 UTC from 203.0.113.40, user-agent FleetLink/4.2, MFA prompt approved. Observed: TrackPort application log of GET /exports/addresses for trailer 44 at 02:16, HTTP 200, bytes 1.2 MB. Missing: whether 203.0.113.40 is Jordan’s known phone path; whether trailer 44 was assigned; whether the export API should have authorized this identity; whether the identity-provider clock is synchronized using Network Time Protocol (NTP). Hypothesis, labeled as such: account takeover using an approved MFA prompt.

Provenance matters. If the export log is sampled at 10 percent, Maya cannot treat silence as proof that no other export happened. If a security information and event management (SIEM) platform parsed the user-agent into a friendly “mobile app” field and discarded the raw string, later investigators cannot check for a forged agent. Keep raw copies for the case, note truncation, and never overwrite.

• Observation: quoted record plus source system plus time plus collection method.
• Hypothesis: a possible explanation, written so it can be killed by the next fact.
• Missing: the next useful record, not a demand for omniscience.
• Provenance: original, copy, parse, and sampling all change what the record can prove.

Precision, recall, and why noisy alerts still need a human

Precision is true positive alerts divided by all positive alerts: TP / (TP + FP). Recall is detected true events divided by all true events: TP / (TP + FN). Define the event class, counting unit, and labeled evaluation set first. In a synthetic set containing 20 known incidents, a detector raises 10 alerts: six match distinct incidents and four are false positives. Precision is 6/10, or 60 percent; recall is 6/20, or 30 percent. In production, undiscovered incidents make the recall denominator uncertain. Report that limit instead of inventing a score.

Changing a threshold can trade precision against recall; measure the effect with labeled cases rather than assuming a universal rule. One alert does not itself have precision. The export increases concern, but its authorization and trailer assignment are still unknown here. Do not let the detector name become the incident conclusion. Write “TrackPort address export by account jordan.h; authorization and human attribution under investigation” until additional evidence supports a stronger title.