Most IT operations reviews arrive as a long document, full of observations everybody already agreed with, ranked by how easy they are to fix. They make people feel examined and they change nothing.
A useful one answers three questions, and an ordinary one answers the first only.
The three questions
How does this actually run today? Not how the process document says. What people actually do, including the workarounds, because the workarounds are the findings. Somebody keeping a spreadsheet beside the system is telling you the system does not do the job.
What is it costing? In money, with the arithmetic shown. "Handover delays are a concern" is an observation. "Handovers between the two teams cost about £40,000 a year in waiting time, worked out as follows" is a finding somebody can act on or argue with.
What is worth doing about it, and what is not? Ranked by what each is worth, not by how easy it is. The easy ones get done anyway. A review earns its fee by naming the expensive problem nobody wanted to look at.
What an IT operations review typically assesses
Which areas matter depends on the decision the review is for, and few reviews need all of them. The usual candidates:
- Service desk and support. How requests arrive, how they are triaged, handed over and closed, and where they wait or bounce between teams.
- Incidents, problems and changes. How failures are handled, whether the recurring ones ever get fixed, and how changes are approved and released.
- Ownership and handovers. Who owns each service, how work passes between teams and suppliers, and how escalation works out of hours.
- Suppliers and contracts. What you pay for, what you actually receive, and how anybody would know the difference.
- Tools and data. Whether the systems fit the work, where spreadsheets have grown up beside them, and whether the ticket data can be trusted enough to measure anything.
- Operational costs. Staff time, licences, supplier charges, and the cost of rework and delay, which is usually the largest and the least visible.
- Risk and resilience. Knowledge held by one person, access nobody reviews, and what happens when something fails at a bad moment.
What a finding needs
- Evidence attached. Where the number came from, so a sceptical person can check it.
- A cost, or an honest statement that it cannot be costed. Some things genuinely cannot be, and saying so is better than inventing a figure.
- What it would take to change, roughly. A finding with no sense of effort cannot be weighed against anything.
- Who would have to act. A finding nobody owns is a paragraph.
A worked example (hypothetical)
This example is invented to show the shape of a finding. It is not taken from any client, and every number in it is an assumption stated below.
The finding. About one ticket in five passed from the service desk to the infrastructure team is sent back because it lacks the information needed to start work.
The assumptions.
- 1,200 tickets a month are passed from the service desk to the infrastructure team.
- 20% of them are returned for missing information, which is 240 a month.
- Each return takes 15 minutes of staff time in total: reading it, chasing the person who raised it, and passing it on again.
- Staff time costs £35 an hour, including employment costs.
The arithmetic.
- 240 returns × 15 minutes = 3,600 minutes, or 60 hours a month.
- 60 hours × £35 = £2,100 a month.
- £2,100 × 12 = £25,200 a year in handling time.
That figure leaves out the delay to the person waiting for the fix. It is real, but it cannot be priced honestly from these numbers, so the finding says so rather than adding a guess.
How it would be validated before anybody relied on it.
- Count the actual returns from the ticket history over at least 90 days, rather than trusting one month that may not be typical.
- Time a sample of returns, for example thirty, to test the 15 minute figure.
- Read a sample of returned tickets to confirm that missing information is really the reason, and not a routing or priority problem.
- Confirm the hourly cost with finance rather than using a rule of thumb.
What it would take, and who would act. Agreeing the minimum information the infrastructure team needs, and making the handover form ask for it. The service desk manager and the infrastructure lead would own it together. Whether it is worth doing depends on the validated figure, not on this one.
Two audiences, one document
A board needs to read it and an engineer needs to be able to check it. Most reviews pick one and lose the other: either a summary nobody can verify, or forty pages of detail no director will open.
The way through is to make every claim short, and make the evidence for it available underneath. The board reads the claims. The engineer reads the evidence. Neither is asked to trust the other.
Say when the answer is to change nothing
The most valuable sentence in a review is often "this is fine, leave it". It is also the rarest, because a review that recommends nothing looks like a review that was not needed.
If a reviewer finds something worth doing in every area they look at, that is worth noticing about the reviewer.
What you should receive
The deliverables from a review typically include:
- A short summary for decision makers. The findings, what they cost and what is recommended, readable in one sitting.
- The findings in full. Each with its evidence, its cost or a plain statement that it cannot be costed, a rough sense of effort, and who would have to act.
- The workings. The arithmetic and the data behind every figure, in a form somebody else can check and rerun.
- Recommendations, ranked by value. Including the areas that should be left alone.
- A session to go through it. With the people who commissioned the review, so questions are answered before the document is circulated.
These are typical rather than fixed. The scope of each engagement, which areas it covers, who is interviewed, how long it takes and exactly what is delivered, is agreed with the customer before the work starts.
How to commission one so it is useful
Say what decision it is for. A review commissioned to inform a specific decision is sharper than one commissioned to have a look. "We are deciding whether to renew this contract" produces a different and better document.
Give access to the people doing the work, not only their managers. The gap between the two is usually where the findings are.
Ask for the arithmetic, not just the conclusions. If a number cannot be explained, it cannot be defended when somebody senior disagrees, and it will be quietly dropped.
Agree in advance what happens to it. A review with no scheduled decision after it is a document that gets read once. The best time to agree the follow-up meeting is before the work starts.
The uncomfortable part
A good review will say something the person who commissioned it did not want to hear. That is most of what you are paying for. If you know in advance which findings would be unwelcome, say so at the start. It is better than discovering it in the room.
Thinking about an IT operations review? We agree the scope with you before any work starts: the decision it is for, the areas it covers and what you receive at the end. See what our Operation review service covers, or talk to us about what you need.