Skip to main content

Your logs are not a black box

· 4 min read
Ilgiz Khusnullin
Founder, Attestsys

After a production incident, someone always asks the timeline question: who changed what, and when? Ask experienced engineers how they answer it and you'll hear the same thing, almost word for word: logs. Application logs, change logs, auth logs, git history. And they're right — logs are exactly the right tool for that question.

Until the person asking it stands outside your trust boundary.

The flight-recorder analogy, taken seriously​

A popular analogy in these discussions: logs are the black boxes of software. After a crash you have wreckage, witnesses, radar tracks — but the flight recorder tells you what actually happened in the cockpit.

The analogy is better than it looks, because it points at the one property that makes a black box trustworthy — and it isn't the recording. Investigators trust a flight recorder because it sits outside the operator's control. The pilots can't open it. The airline can't edit it mid-flight. The data was captured outside the trust boundary of the people being investigated.

Application logs don't have that property. They're created, stored, and administered under the same control as the systems being investigated. For diagnosis that's completely fine — you're not adversarial with yourself. But it creates a credibility ceiling the moment the audience changes.

Where the ceiling actually bites (and where it doesn't)​

Let's be honest about the boundaries, because overclaiming helps nobody:

Internal accountability? Logs win. A central log server with separated admin access — rsyslog, Loki, your SIEM — where engineers can't touch the records of their own systems is a genuinely good architecture. Most internal audits accept exactly that: personalized logins, separation of privileges, a well-backed-up log server. If your audience is your own organization, you likely don't need anything stronger.

The ceiling bites in two situations:

  1. The audience is outside your trust boundary. A customer disputing what your team did and when. A counterparty in a contract disagreement. A security review by someone who has no reason to take your word. Their first question is the one your log server can't answer from the inside: "how do I know these weren't edited after the fact?" The honest answer with conventional logs is "our policy prevents it" — which is a promise, not a proof.

  2. The system itself is compromised. This one comes from practitioners, not from us: after a breach, the hacked system's own logs can't be trusted — and the remote copies, the ones the attacker couldn't reach, decide the investigation. Notice what made those copies trustworthy: placement, not proof. The investigators still had to take the log server's integrity on faith; it just happened to be faith better placed.

From placement to proof​

The step beyond "the attacker couldn't reach it" is "anyone can check it" — making each record carry its own evidence of integrity instead of inheriting trust from wherever it happens to be stored:

  • Sign every entry at the moment it's captured (we use ECDSA P-256, with a per-workspace key held in a KMS).
  • Chain the entries: each record embeds the SHA-256 hash of the previous record's signed payload, so editing any past entry visibly breaks everything after it.
  • Timestamp independently: each entry gets an RFC 3161 token from an external Timestamp Authority — not from the operator's clock, and not from ours.
  • Make verification portable: the export bundle ships with a self-contained verify.html that re-checks every signature, link, and timestamp in the recipient's browser, offline, with zero calls to us.

That last point is the part that changes the conversation. The person you hand the record to doesn't have to trust your log server, your policy — or us. They run the math themselves.

We build this for Jira workflows (Tamper-Evident Audit Log for Jira, free tier on the Atlassian Marketplace), but the principle is general, decades old, and — from what we've seen in practice — strangely underused: RFC 3161 timestamping plus an append-only signed chain is the cheap way to give ordinary records black-box properties.

Try the uncomfortable test​

If you want to know whether your current setup has this property, try the question on yourself: if a customer demanded proof tomorrow that your incident timeline wasn't edited after the fact, what exactly would you show them — and would it convince someone who doesn't trust you?

If the answer is a process document, you have accountability. If the answer is something they can verify themselves, you have evidence. You can see the difference in your own browser: drop our sample bundle into the live verifier — no install, no account, no network requests.