Troubleshooting with your Favorite AI

What's your troubleshooting process?

Up until 10 or 15 years ago it was probably "Google it" and, if it was something you did frequently, you got good at a writing search statements. Maybe you also had a few sharp co-workers or interns you could Slack or IM, or, if you were lucky, that guru hiding in a back corner who needed to be bribed with good coffee or a nice craft beer.

Before that, you probably dug through the stack of books on your shelf, asked questions around the water cooler, called people (remember doing that?) and bribed the guru with Jolt Cola.

Google (or Yahoo) was nicer because the chance of being ridiculed was MUCH lower than then co-worker/guru option, unless you actually had to post something. If you were searching then results could be variable at best. If you were talking to actual people or posting online then you had a limited number of questions you could ask before the eye-rolls and sighs started. If you were REALLY lucky then you had a mentor who actually helped walk you through your problem and answered your questions politely.

Fun, right?

Welcome to the new world of AI-based troubleshooting! You now have a totally non-judgmental, awake-24-hours-a-day, widely read assistant ready to help you without ever rolling an eye or heaving a sigh, unless you ask!

I've been using ChatGPT (and other LLMs) heavily for the past six months or so for all sorts of things. I have friends who use the heck out of them, too, and we all trade tips and tricks. One of the recent ones I learned was "ask the AI to ask you questions". Hmm - that sounds kinda like what an expert might do if you brought them a sticky issue and said "I don't know where to start with this."

I was recently trying to figure out a security issue with AWS and found I was going in circles with the documentation and UI. After about 15 minutes of this (short attention span) I decided to give the new "Ask the AI to ask you questions" trick to see what happens. Nothing to lose, right? No eye-rolls, sighs, or office gossip about how you're an idiot (not this year anyway - maybe in 2025 or 2026).

My first prompt was this (leaving out 30 lines of policy code):

I'm working in AWS with Multi-account permissions. We're using Control Tower and an organizational structure and we manage users and permissions from our Management account. 

I'm trying to set up permissions on an S3 bucket in a different account - sandbox-jmarlow. To do this I created a Group in the Management Account, added my user to the group, and then added a Permission Set that should give read/write permissions to the S3 bucket in sandbox-jmarlow. I did this with an Inline Policy in a Permission Set.

here's my policy:
{
	"Version": "2012-10-17",
	"Statement": [
		<LOTS OF STUFF REMOVED>
	]
}

When I log in to the sandbox-jmarlow account and go to S3 buckets I get an error saying I don't have List Bucket permissions.

The response was a list of 7 questions that I needed to answer - typical "did you check this, did you check that, show me the code for this other setting" sort of thing somebody would ask.

I responded with various answers and bits of code and was asked a few more questions, which I could mostly answer.

Typical troubleshooting stuff.

Then I was asked for a specific setting on a specific Role. If you've ever worked with Roles in AWS you know you can end up with pages of them, and they can have all sorts of weird names if they're being auto-generated (looking at you, Control Tower). I wasn't asked for information about a specific role - just to find a role that matched some requirements and provide a bit of code from a setting.

Great! I have 50 plus roles and no way to know which one the AI wants info from. Sounds like copy-paste hell to me.

Except! Except that I remembered that ChatGPT 4o could handle images as input!

Screenshot that list of roles, paste it in to the chat pane, and add "I don't know which role you want that information from" as the prompt.

The result?

The name of the specific role the AI wanted info from. I opened the role, got the info, passed it to the AI, and continued troubleshooting. Found the answer in another couple minutes.

What's the big deal? Well, AI can be a lot of help with troubleshooting, but sometimes you have something on screen that really just needs to be looked at. And now, at least with ChatGPT, that's just a screenshot away!