The problem: Four times a year, risk and compliance emails the security operations team (the SOC): “Please send the list of alerts you handled this quarter, by Friday.” The two most senior analysts stop what they are doing and copy numbers into a spreadsheet by hand. The SOC sees it as a distraction from “real” work. Compliance sees a team that will not help. The figures arrive late, and they do not always match.
Words used here: risk and compliance is the team that writes the rules and checks they are followed. The SOC is the team that watches alerts and deals with attacks. Evidence (proof) is something that shows a check really happened.
Think of a restaurant on a Friday night
The health inspector’s aide asks the chef to count everything in the fridge by Friday and write it on a form. It is the Friday night rush. The chef does not say no because he dislikes forms. He says no because the request lands in the one hour he cannot spare.
The aide needs the count and the chef needs to cook. Both are right. The trouble is the timing and the method: counting by hand, on request, in the middle of work.
What it looks like at work
The rule says the SOC’s alert handling must be reviewed every quarter, with proof. A compliance analyst sends the request for the Q3 list on Monday, due Friday. That week the SOC has a live incident.
The senior analysts export alerts from three tools and join them in a spreadsheet at night. Two of the numbers differ from last quarter because the export was filtered differently. Compliance sends the figures to the auditor with a note: “These may not match.” The auditor asks for the review itself, and there is no record that it happened.
- 11. Request sentQ3 list, due on Friday
- 22. Live incidentThe SOC has no time that week
- 33. Done by handThree exports joined in a spreadsheet
- 44. Figures differFilters changed since last quarter
- 55. No recordThe auditor asks for the review itself
Both teams have a fair point. Compliance has a fixed audit date and cannot wait. The SOC cannot drop a live incident to count tickets. Neither is being difficult. They have two different clocks, and nobody has put the proof on a shared one.
You will notice it when:
- The request arrives by email with a short deadline, every quarter.
- The people asked are the most senior, and the most busy.
- Last quarter’s figures cannot be reproduced the same way.
- The SOC says “we already did this” and compliance says “we need it again.”
- Evidence is built the week before the audit, never during the year.
What exactly needs fixing
The fix is not to ask earlier. It is to stop collecting by hand. If one saved report in the tool produces the proof, nobody has to be asked. This card shows the whole problem on one page:
Quarterly alert proof is collected by hand by the two busiest analysts, and the figures do not match.
Compliance’s quarterly request, and three SOC tools that are exported by hand.
A quarter with no usable proof, and a finding for it.
- Senior analyst days lost
- Figures differ each time
- Strained teams
- Gaps in the proof
- 11. Agree the proofWrite the list of figures in plain words with the auditor
- 22. Build one reportOne saved report that gives the same figures every time
- 33. Fix a dateOne date each quarter, in both diaries, read-only access
Who fixes it: SOC lead and compliance lead
By when: report built before the next quarter ends
A blank copy of this card is in the Tools section.
Why it happens
- The proof lives inside the tool, and only the SOC can log in. Compliance has to ask.
- Each quarter is treated as a new request. Nobody builds the report once.
- The two teams plan on different calendars. Compliance plans around audit dates. The SOC plans around incidents.
- The work is done by the busiest people. They are the only ones who understand the data.
- Compliance does not know what the tool can export. So it asks for a custom list.
What happens if you ignore it
The same scramble happens four times a year, and gets worse as the SOC grows.
- Senior analysts lose days each quarter that they cannot give to real alerts.
- Figures differ from one quarter to the next, so the auditor trusts them less.
- The two teams stop talking to each other except to ask for things.
- One missed quarter leaves a gap in the proof that cannot be filled later.
At a glance
| Question | Answer |
|---|---|
| How likely is it to hurt us? | Very likely. It happens wherever proof is asked for, not produced. |
| Worst case? | A quarter with no proof at all, and a finding for it. |
| What does it cost? | Several senior-analyst days a quarter, and a strained working relationship. |
| What if we wait? | The same scramble comes back, with more alerts to count. |
| What does the fix cost? | A few days once to build the report, then almost nothing. |
How to fix it, step by step
Build the report once, agree a calendar, then let compliance read it.
- Agree what proof is needed. Sit with the auditor’s checklist and the SOC lead. Write the list of figures in plain words.
- Build one saved report in the tool. It should produce those figures, the same way, every time.
- Fix the definitions. What counts as an alert handled, and from which date to which date.
- Agree a calendar. One fixed date each quarter. Put it in both teams’ diaries.
- Let compliance read the report itself. Give read-only access, or have the tool email it on the date.
- Ask only when something looks odd. A question, not a request for a new list.
- Keep each quarter’s report. Save it in one folder so the year’s proof is already there.
The picture shows the order: build once, agree a date, then let compliance read it. After that, no one has to ask.
Who to bring in: The SOC lead (builds the report), the compliance lead (owns the calendar), the person who administers the SOC’s tool (gives read access), and the internal auditor (agrees what counts as proof).
Tools and templates
Quarterly proof calendar
| Proof | Saved report | Date each quarter | Who builds | Who reads |
|---|---|---|---|---|
| Alerts handled | Alert summary report | 5th working day | SOC lead | Compliance analyst |
| Time to respond | Response time report | 5th working day | SOC lead | Compliance analyst |
| Review sign-off | Signed review sheet | 10th working day | SOC lead | Compliance lead |
Blank finding card
| Field | Write here |
|---|---|
| What is broken | One line, in plain words |
| Where it is | The system, team, or process |
| What could go wrong | In one sentence |
| What it costs us | Money, time, a failed audit, lost trust |
| The fix | The exact action |
| Who fixes it | One name or one team |
| By when | A date, not “soon” |
Email to start the talk
Subject: Let us stop the quarterly evidence scramble
Hi both. Every quarter we ask the SOC for a list by hand, and it costs the senior analysts days. I would like to try something different: the SOC builds one saved report that gives us the figures, and compliance reads it on a fixed date. Can we meet for 30 minutes this week? I am not looking for who is at fault. I am looking for a way that nobody has to be asked.
Field tip
If you find yourself asking for the same list a second time, it is time to ask for a report instead.
Words used in this chapter
- SOC: the security operations team that watches alerts and deals with attacks.
- Risk and compliance: the team that writes the rules and checks they are followed.
- Evidence (proof): something that shows a check really happened.
- Saved report: a report set up once in a tool that can be run again the same way.
- Quarter: a three-month period of the business year.
- Finding: a problem the auditor writes down.