Information Gathering For EMs In The LLM Era
Updated:
By far the most important thing to say about information gathering as an engineering leader is that it's a key activity of being an effective manager, and that you should be gathering information both actively & intentionally (on your calendar) and passively (looking at every task as an opportunity to gather info).
That said, it's worth considering what information gathering means and how to do it well. Let's break it down into categories. And then a few ideas on how to do them better. informed by the new presence of ChatGPT, Claude, Gemini et al:
How the business and its industry work: First-time managers often feel whiplashed by business decisions they don't understand and seem to make no sense. I'm supposed to explain that to my team? This is normal. Also, a sign that the understanding of the business that you thought you had as an IC was/is incomplete.
Idea: If you work for a public company, this is easy: there's enough data available for an LLM to get context on the whys of company strategy. Startups can be harder, but the press release of the last fundraise are good place to start. Another good prompt can be "Say you were running a startup that was solving X problem in the Y industry. How would you reason about a decision around Z" where Z is the decision that's got you confused.
How the technical architecture, systems and software development lifecycle work: Your Observability Environment is useful on so many levels. Likely moreso for than a working Dev Env for an Eng Manager, though both are valuable.
Idea: Spend time in Cursor or your coding tool of choice in Ask mode. Block a half hour each week to just ask it questions about the software you own's inner architecture. How does the API framework bootstrap into the app. How and where does DataDog init. Where are the env variables located. Are there any unusual patterns at play. What parts of the codebase have seen the most changes or refactors. What parts of the codebase contain pure logic that's not covered by unit tests. Etc.
What causes errors, slowness, scalability failures in the software: This is about using the tools you have available, and at higher levels about influencing tool choice and/or purchasing new tools. DataDog, New Relic, Grafana, BigQuery, Snowflake, Read-only relational replicas, CloudWatch logs, all of it can tell you what's really happening if you set yourself up for success.
Ideas: Try out in-tool AI, but the broader LLMs are really useful for both general and specific questions in popular tooling. Choose popular tooling so your team can leverage LLMs to problem solve. A good prompt: Write a DataDog query for using RUM to see users who are hitting errors on the /domain/my-feature route.
Company org chart and how it works: This starts with knowing the org chart. Continues with identifying which nodes are key for your team's success, and knowing how each leader and their tree work in practice. And finally it's about spending the time, which could be an entire chapter of its own but usually comes down (especially in big orgs) to "letting the people you collaborate with know you're a human too and not just another source of aggravation in their Discord/Slack/Teams/Email.
Ideas: Put any title you're not familiar with into a prompt, asking not just for the expectations but the mindset and motivations. Remove the names and just put the titles in, and prompt for which of your coworkers the LLM thinks you should get to know better.
People & who they are: A subset of the above, but reserved for your direct reports, direct partners and peers and upline. The people you work with on a daily basis are usually worth getting to know, and it's hard to get to know someone without deliberate effort.
Ideas: Calendar recurring 1:1s, don't wait for them to happen organically. For recurring 1:1s, prompt your past agendas and then ask for ideas for the next session's topics, working to rotate between strategic and tactical.
Rumors & gossip: Just say no. I know, it's hard. But it's probably not worth your time or energy.