Advanced · 45 minutes · Practical project

Project: an evidence-led incident report

Combine service, network and DNS evidence into a defensible diagnosis.

By the end of this lesson

  • Separate observations from assumptions
  • Identify a supported cause from cross-layer evidence
  • Plan verification and rollback before proposing a change

Your environment: Fully offline evidence exercise. No service installation, network probing or administrator access required. Setup help

Write a report another engineer can use

A useful incident report tells the next person what happened, who is affected, when it began and what you actually observed. It distinguishes facts from hypotheses. Include the affected endpoint and relevant timestamps, but do not paste credentials, private keys or unnecessary personal data.

For this project, a team’s internal training web service works on its VM but not from a client. You receive a fixed evidence bundle. Treat it as an offline case study, not an invitation to contact the documentation addresses. Your job is to connect the clues and specify the smallest justified next action.

A proposed fix needs a verification plan

Good troubleshooting does not end with a plausible explanation. State how you would confirm the intended service exposure, what configuration you would inspect and which authorised change would address the demonstrated limitation. Do not promise that one change resolves every unknown: another access control might still matter.

Plan the retest from the affected client as well as locally, and describe how to return to the previous configuration if the change fails. Include what you have not yet tested. This project records your own completion of the practical report; its short knowledge check cannot remotely verify the state of a real server.

See it in context

Worked example

Structure a concise incident handover.

Symptom:
Scope and time:
Observed evidence:
Supported hypothesis:
Unverified assumptions:
Proposed authorised change:
Verification:
Rollback:

Use this structure in a local text file. Evidence should be specific enough for another person to reproduce the investigation.

Your turn

Practical project

Diagnose the fictional service outage and write a change-and-verification plan.

Run commands in your own lab. This website displays instructions; the practice terminal below helps you rehearse the syntax.

1. Prepare your workspace

mkdir -p "$HOME/training4linux/incident-project"
cd "$HOME/training4linux/incident-project"
printf "%s\n" "09:00 UTC: deployment finished" "09:03 UTC: client reports TCP 8080 connection refused" "DNS A: app.example -> 192.0.2.10; NOERROR" "Server interface: 192.0.2.10" "Service: active (running)" "Listener: 127.0.0.1:8080" "Local request: HTTP 200" "Remote firewall path: not yet verified" > evidence.txt

2. Complete the task

  1. Create report.txt with each heading from the worked example.
  2. Explain which evidence supports a loopback-bind hypothesis, and why active status and DNS success do not prove remote service availability.
  3. Propose checking the intended exposure and bind setting with the service owner before changing it.
  4. Specify local and client-side retests, a remaining firewall check and a rollback to the recorded previous configuration.

3. Check your result

  • Your report identifies the loopback-only listener as a supported limitation.
  • It keeps the unverified firewall path separate from established facts.
  • It includes an authorised change proposal, two-sided verification and rollback.
  • It avoids disabling all security controls or promising an unverified recovery.
Need a hint?

Combine local HTTP success with the exact listening address. Treat the firewall as an open question.

Reveal a worked solution

Compare this with your approach after trying the task. More than one valid solution may exist.

cat evidence.txt
cat > report.txt <<'REPORT'
Symptom: remote client cannot connect to app.example:8080 after deployment.
Scope and time: one reported client, beginning 09:03 UTC.
Evidence: DNS matches the server; service active; local HTTP 200; listener is 127.0.0.1:8080.
Hypothesis: loopback-only binding prevents direct remote access.
Unknowns: intended service exposure and firewall path.
Proposal: confirm intended exposure, preserve configuration, then seek approval for a scoped bind change if appropriate.
Verification: inspect the listener; test locally and from the affected client; check scoped firewall policy if remote access still fails.
Rollback: restore the recorded previous configuration using the approved service procedure, then verify local behaviour.
REPORT
cat report.txt

After the lab: This is a documentation-only project. Keep report.txt as your troubleshooting portfolio piece.

Common mistakes to avoid

  • A correct hypothesis is not the same as a verified recovery.

Check your understanding

Knowledge check

Answer all 3 questions correctly to pass. Retry as often as you need; your best score is kept.

1. What is the strongest demonstrated limitation?
2. Which statement should remain an open question?
3. What belongs in a responsible change proposal?

Your lesson progress

Not started · Best score: 0% · Attempts: 0

Practical completion is self-reported. Progress is linked to a private cookie in this browser and stored on the site. Clearing that cookie loses access to it. Manage or erase progress.

Keep the reference close

Further reading