Debug Detective: curl was lying to me
The pipeline was green. The deploy had finished, the health check was happy, and somebody was telling me the page did not work. One of those things was wrong and I spent a few hours assuming it was the person.

The scene
A deploy went out. The pipeline ran its smoke test against the new version, got what it wanted, and went green. Twenty minutes later somebody said the page was blank.
Blank. Not slow, not erroring, not half rendered. Nothing on it.
My first move was to doubt the report, and I want to be honest that this is what I did rather than dress it up as triage. The pipeline had checked. The check had passed. So the most likely explanation, to me, was a stale tab or a cached bundle on somebody else's machine.
It was not.
Everything I checked was fine
The deploy had completed. Right image tag, pods ready, rollout finished clean. The health endpoint answered exactly as it should.
So I went to the logs, looking for the 5xx that had to be in there somewhere.
There were none. Not one. No stack trace, no error, no warning, nothing raised and swallowed. Hours of reading a log that had nothing to say, because the server was not failing.
That is the part worth sitting with. Everything I checked came back green, and everything I checked really was green. The green was not a lie. The green was the problem.
The witness
This is what my smoke test did:
It asked the server for a status code. The server gave it one. Two hundred, honestly and correctly, because the server really did have a document to hand over and really did hand it over.
The document was an empty shell that fills itself in the browser. The script that fills it was not there.
curl never lied to me. It answered the question I asked with complete accuracy, and the question I asked was not the question I meant. I wanted to know whether the page worked. I asked whether the server replied.
What a real browser said
The moment you run something that executes the page rather than merely fetching it, the case closes. The container is empty. It was always going to be empty, and it would have been empty on every single green build.
This is a reproduction, run locally on a throwaway server, not a capture from the system where it happened. The mechanism is identical and you can run it yourself in about a minute, which is rather the point.
The fix, and what it cost
The smoke test now runs a real browser and looks for a string that only appears once the application has mounted. It is slower. It adds most of a minute to every pipeline.
I will take the minute. The old test cost me a few hours and, worse, it cost me trust in the report from the person who was right.
Detective's note
The witness was honest. I had asked a question whose true answer told me nothing, and then I read the answer as reassurance.
- A status code is a claim about the transaction, not about the page. HTTP 200 means the server had something to send.
- If your check cannot distinguish a working page from a broken one, it is not checking the thing you care about.
- When everything you inspect is green and somebody says it is broken, consider that you may be inspecting the wrong things rather than that they are wrong.
- Believe the human with the blank screen. They are looking at the artefact. You are looking at a proxy for it.
- A slower test that can fail beats a fast one that cannot.
Is your pipeline testing the thing you think it is?
Green pipelines that never catch anything are a common and expensive habit. If you want a second look at yours, get in touch.
Get in touch
flochai
