# Discovery Interview — Switch Power Issue Agent

**Skill:** `/nucleus-discovery-interview` · **SME (Subject-Matter Expert):** Adam, Senior Network Engineer
**Working name:** `switch_power_issue` · **Duration:** 32 minutes · **Mode:** Narration-first
**Deliverable:** 6-sheet use case card workbook (`switch_power_issue.xlsx`)

---

**Adam**
I need a workflow for troubleshooting device power issues on our Cisco access switches.

---

**Claude — Alert payload**
Before we start: do you have a real alert payload we can use as context? It saves us 10–15 questions later because I can read device type, alert family, severity, and the exact field names straight from it.

- **Yes — I'll paste the JSON next (Recommended if you have one)**
- Yes — I'll attach a file
- No, but I can describe the alert in 1–2 sentences
- No — proceed with no context

**Adam** ✓ *Yes — I'll paste the JSON next*

```json
{
  "source": "SolarWinds",
  "severity": "3",
  "cmdb_ci": "GB-LON-ACC-SW-014",
  "assignment_group": "Network Operations",
  "short_description": "Power supply failure detected on GB-LON-ACC-SW-014 (PS-2 status: Faulty)",
  "additional_information": {
    "sys_message": "%IOSXE_PEM-3-PEMFAIL: The PEM in slot P2 has failed or been removed",
    "psu_slot": "P2"
  }
}
```

**Claude**
Thanks — I can see this is a **hardware power** alert from **SolarWinds** on a **Cisco Catalyst access switch** (`GB-LON-ACC-SW-014`), assigned to **Network Operations**, referencing PSU (Power Supply Unit) slot **P2** via the `%IOSXE_PEM-3-PEMFAIL` syslog signature. I'll use these as defaults throughout — you can override anything I get wrong.

---

## Phase A — Product brief

**Claude — Workstream**
Which workstream does this agent belong to?

- **Hardware Health (Recommended)** — the alert's event family is hardware power; the existing PSU / fan / thermal agent already lives in this workstream
- Power Assurance — new standalone workstream
- Device Reliability — new standalone workstream

**Adam** ✓ *Hardware Health*

---

**Claude — Problem statement**
What's broken or missing today that this agent fixes? Here's my draft from what you've told me so far:

- **"Power-supply alerts on Cisco access switches are frequent and noisy, and every one is triaged by hand today. Engineers spend 15–20 minutes per alert on the CLI confirming whether a PSU has genuinely failed, whether redundancy is lost, or whether the event was a transient blip." (Recommended)**
- Shorter: "Manual triage of switch power alerts is slow and inconsistent."
- I'll write my own

**Adam** ✓ *Recommended — but add that transient events often sit open for hours before anyone closes them. That's the bit that annoys everyone.*

**Claude**
Added. Problem statement captured with the transient-backlog point.

---

**Claude — Scope**
How should we scope the MVP (Minimum Viable Product)?

- **Catalyst 9200 / 9300 access switches, standalone and stacked (Recommended)** — matches your payload's device estate and the patterns in the existing hardware-health agent
- Access switches plus modular / chassis platforms
- Single model family only (9300)

**Adam** ✓ *Recommended. Keep chassis kit out of the MVP — different beast entirely.*

---

**Claude — Success KPIs** *(pick all that apply)*
- **Reduce Mean Time to Resolve (MTTR) (Recommended)**
- Time Savings
- Cost Savings

**Adam** ✓ *MTTR + Time Savings*

---

**Claude — Process owner**
- **Network Operations Lead (Recommended)** — derived from the payload's assignment group
- Network Manager
- Service Owner

**Adam** ✓ *Network Operations Lead*

---

**Claude — Operators** *(pick all that apply)*
Who runs this day-to-day?

- **Network Operations team (Recommended)**
- Network Manager
- IT Leader
- Customer Success

**Adam** ✓ *Network Operations team + Network Manager*

---

**Claude — API documentation**
Do you have API documentation for the platform the agent will query during investigation? If yes, I'll use it to suggest accurate endpoints and parsing logic.

- Yes — I have a URL **(Recommended)**
- Yes — I'll paste the docs as text
- Yes — I'll attach a file
- **No — proceed without**

**Adam** ✓ *No — proceed without. These are all CLI checks over SSH, no controller involved.*

**Claude**
No problem — the bundled reference agents already cover the SSH / CLI patterns, so I'll ground my proposals in those.

---

## Phase B — Troubleshooting flow

**Claude — Interview mode**
Before we capture the troubleshooting flow — would you like me to walk you through it one check at a time, or describe the whole thing in your own words first?

- Walk through one check at a time **(Recommended)** — step-by-step with dropdown options
- **Narration-first — I'll describe the flow in my own words** — usually 10–15 prompts total instead of 70+

**Adam** ✓ *Narration-first. I've done this at 3am enough times, I can tell it as a story.*

---

**Claude — Troubleshooting narration**
Go ahead — cover (a) what you'd check first, (b) what each check tells you, and (c) where each path ends: another check, an auto-close, or an escalation.

**Adam**
First thing, always: can I actually get on the box? If SSH fails on a *power* alert, treat it as possible total power loss and hand straight over to the device-down flow — don't try to be clever. If I'm on, I check whether the alert is still real: `show environment power`. If every supply shows OK and the logs are clean for the last half hour, it was a blip — close it and write down what you saw.

If a PSU shows Faulty or Not Present, sanity-check whether that slot should even have a supply in it. Some of our smaller sites run single-PSU by design and the redundancy alert is just noise — those go back to the monitoring team for tuning, not to an engineer. If the slot *should* be populated: Faulty means a genuine hardware fault, so that's TAC and an RMA. Not Present means someone's unseated it — site visit.

If both supplies look OK but one is drawing zero watts, the switch is fine — the feed or the UPS is the problem, and that goes to facilities, not us. On PoE switches I also look at the inline power budget, because "power issue" tickets are sometimes just ports being denied PoE. And I always scan the logs for repeated PEM fail/restore messages — a flapping supply means dirty input power, not a dead PSU. Anything that doesn't fit those buckets stays open for an engineer, with everything I gathered attached to the ticket.

---

**Claude — Confirm the draft**
From your narration, I extracted:

— **10 decisions** (7 scenario-specific + 3 common pre-checks)
— **8 investigation steps**
— **8 outcomes** (1 auto-closeable, 7 leave-open)

**Decisions:** change-in-progress check → parent / MI (Major Incident) check → is device up? → is the alert still active? → any PSU Faulty / Not Present? → should that slot be populated? → failed vs removed? → healthy PSU with zero input? → PoE budget exhausted? → is the PSU flapping?

**Investigation steps:** SSH reachability → `show clock` → `show version` → `show environment power` → `show environment all` → `show power inline` → `show logging` → `show switch`

**Outcomes:** PSU hardware fault (TAC / RMA) · PSU removed (site visit) · single-PSU by design (monitoring tuning) · input feed lost (facilities / UPS) · transient — auto-close · PoE budget exhausted · flapping supply · unclassified — engineer review

Does this match what you'd do, or would you like to adjust?

- **Looks right — proceed to gap-fill (Recommended)**
- Adjust a decision / step / outcome first

**Adam** ✓ *Looks right — proceed to gap-fill*

---

## Gap-fill *(only fields the narration didn't cover)*

**Claude — PSU status source**
For the "is the alert still active" and PSU status checks, which command is the primary diagnostic?

- **`show environment power` (Recommended)** — the existing hardware-health agent parses per-slot OK / Not Present / Faulty from exactly this output
- `show environment all` only
- `show power` (older platforms)

**Adam** ✓ *Recommended*

---

**Claude — Input-feed detection**
To tell a healthy PSU on a dead feed apart from a failed PSU, I'm proposing `show environment all` and flagging any supply that reports OK with **0 W input**. Confirm?

- **Yes — flag OK-with-zero-input as feed loss (Recommended)**
- Different signal

**Adam** ✓ *Yes — that's exactly the tell.*

---

**Claude — PoE budget threshold**
For "PoE budget exhausted", what's the trigger?

- **Remaining inline power below 30 W (Recommended)** — fixed floor
- Any port in power-deny state
- Other

**Adam** ✓ *Other → make it relative: flag when remaining is under about 10% of the total budget, not a fixed number — a 9300 stack budget dwarfs a lone 9200. Keep the power-deny port check too.*

**Claude**
Captured: remaining < 10% of budget **or** any port in power-deny.

---

**Claude — Syslog signatures**
For the log scan, I'm proposing these search patterns from the reference agents plus your payload:
`%IOSXE_PEM-3-PEMFAIL` · PEM inserted / removed · `%PLATFORM_ENV-1-PWR` · `%ILPOWER-*` (PoE grants / denials), with timestamp correlation against the alert. Confirm?

- **Yes — use these patterns (Recommended)**
- Add / change patterns

**Adam** ✓ *Yes — and for flap detection call it three or more fail/restore events on the same slot inside 24 hours.*

---

**Claude — Transient window**
How long must the device be clean before we call the event transient and auto-close?

- **30 minutes (Recommended)** — the canonical cooldown used by the existing device-down, interface and hardware-health agents
- 15 minutes
- 60 minutes

**Adam** ✓ *30 minutes — consistency with the others is worth more than shaving minutes.*

---

**Claude — Timeouts and retries**
Standard timing from the reference agents: reachability **10 s / 1 min retry for 10 min**; each command **10 s / 30 s retry for 3 min**. Keep?

- **Yes — keep the standard timings (Recommended)**
- Adjust

**Adam** ✓ *Keep them.*

---

**Claude — Auto-close authority** *(pick all that can auto-close)*
- **Transient — alert cleared (Recommended)**
- PSU hardware fault
- Input feed lost
- PoE budget exhausted

**Adam** ✓ *Only the transient one. Everything else needs a human — it's someone's hardware or someone's building power.*

---

## Metrics

**Claude — Time saved?**  ✓ **Yes (Recommended)** — *Adam: "15–20 minutes of CLI evidence-gathering per alert, and we see about 30 of these a month."*

**Claude — MTTR affected?**  ✓ **Yes (Recommended)** — *Adam: "Transients close in minutes instead of hours, and real faults land with the evidence pack ready, so the RMA starts immediately."*

**Claude — Cost affected?**  ✓ *Somewhat* — *Adam: "Fewer wasted site visits for reseat checks, and TAC cases open with full diagnostics, so less back-and-forth."*

---

## Release plan

**Claude — Internal stakeholders** *(pick all that apply)*
✓ *Network Operations team channel · Network Manager · Service Owner*

**Claude — External stakeholders**  ✓ *None for MVP*

**Claude — Pre-sales briefing?**  ✓ *Not sure — PM decides*

**Claude — Customer CAB (Change Advisory Board) required?**
- yes · no · **depends on customer environment (Recommended)** · not applicable

**Adam** ✓ *Depends on environment — worth noting the agent is read-only, no config changes, so most customers won't need CAB.*

---

## Playback — the captured flow

**Claude**
Here's the flow read back, end to end. We start with the common pre-checks: any in-progress **change** or existing **parent / MI** sends the alert straight out with no automation. Then **Step 1** confirms SSH reachability — if the box is unreachable on a power alert, we treat it as possible total power loss and hand off to the device-down flow. If we're on, **Steps 2–8** gather evidence: `show clock`, `show version` (a reload during the event is brownout evidence), `show environment power`, `show environment all`, `show power inline`, `show logging`, `show switch`.

**DEC-4** asks whether the alert is still active — all PSUs OK and clean logs for 30 minutes means **OUT-005: transient, auto-close with evidence**. **DEC-5** looks for a Faulty / Not Present slot; **DEC-5a** checks whether that slot should even be populated — if it's a single-PSU-by-design site, **OUT-003: monitoring noise, suppression recommendation**. If it should be populated, **DEC-5b** splits Faulty → **OUT-001: hardware fault, TAC / RMA** from Not Present → **OUT-002: unseated, site visit**. **DEC-6** catches the healthy-PSU-zero-input case → **OUT-004: feed lost, facilities / UPS**. **DEC-7** checks the PoE budget (remaining < 10% or ports in power-deny) → **OUT-006**. **DEC-8** counts PEM fail/restore events — three or more on one slot in 24 hours → **OUT-007: flapping supply, dirty input**. Anything left → **OUT-008: engineer review with the full evidence pack attached**.

Does that match what you'd do in real life? Anything missing, wrong, or out of order?

**Adam**
That's exactly how I'd run it. Honestly cleaner than how I'd have written it down myself. Ship it.

---

**Claude — Wrap-up**
Captured: **`switch_power_issue`** — **10 decisions**, **8 investigation steps**, **8 outcomes** (1 auto-close, 7 leave-open), plus product brief, metrics and release plan. Workbook saved as `switch_power_issue.xlsx` with this transcript alongside it. Send both files to the engineering team — they'll assign the agent ID, place it in the scenario folder, generate the Azure DevOps work items and build the n8n workflow. You'll hear back when there's something to smoke-test.

---

*Demonstration transcript — Nucleus `/nucleus-discovery-interview` skill.*