Heilmeier’s Catechism

Heilmeier Catechism

When George Heilmeier was the director of ARPA in the mid 1970s, he had a standard set of questions he expected every proposal for a new research program to answer. These have been called the Heilmeier Catechism. It’s a good exercise to answer these questions for an individual research project, too, both for yourself and as a way to convey to others what you hope to accomplish.

A nice set of questions to ask yourself when doing research. They are part of Heilmeier Catechism, who was the director of research at ARPA.

They’re also questions you can ask when you’re working on something that could be turned into a research paper (which is a lot of things in the computer world unsurprisingly).

Or when you want to execute a marketing or sales project that relies on a new technology.

Update 2018:

heilmeier catechism research project steps

Example of using Heilmeier’s Catechism

Below we will explore each question that is part of the Heilmeier Catechism in the context of a software development team TeamSaas working on a SaaS product adopting GraphQL for the API.

The questions to ask: Heilmeier Catechism in an example context

1. What is the problem? Why is it hard?

In our example, TeamSaas needs to cater to multiple clients and consumers of their API and maintain a maximum payload size of some amount. This is difficult because different clients/consumers have different needs (some may need a lot of data, others may need less data). TeamSaas is aiming to minimize the payload size and give clients/consumers of their API only the data they need.

2. How is it solved today?

Today, TeamSaas would use a REST API.

3. What is the new technical idea; why can we succeed now?

GraphQL is the new technical idea, it allows a client/consumer of an API to select and filter data before it is passed back. The reason it can succeed now is that GraphQL has an integration with the database and framework and language that TeamSaas is using; there are also client libraries that can consume a GraphQL API and they are (relatively) stable. The main reason though is that there is enough adoption and resources aimed at making GraphQL implementable and stable.

4. What is the impact if successful?

In our example, the impact for TeamSaas is that they will be able to send the exact data that each client/consumer of their API needs; additional impacts include a cleaner codebase on their server-side and on the client-side since the data is handled through a GraphQL layer and the API response doesn’t have to be transformed in an awkward/hacky way.

5. How will the program be organized?

TeamSaas could organize their project with a client developer and backend developer prototyping and developing one or two API calls using GraphQL and then further expanding from there.

6. How will intermediate results be generated?

In our example, TeamSaas can use their prototype to generate API responses and operate the new GraphQL APi in parallel with the old REST API.

7. How will you measure progress?

Progress for TeamSaas’s switch from a REST API to a GraphQL-based API can be measured by the number of clients/consumers using the new GraphQL API and by the number of complaints there are. If there are more clients switching and there are few complaints/feedback, then things are going smoothly. Additionally, the payload sizes for each request can be measured and averaged and compared in the 90th to 99th percentile range to the old REST API to ensure payload sizes overall are smaller. The total payload sizes delivered over a week or a month could be measured, aiming for a decreased payload size total in comparison to the REST API.

8. What will it cost?

This is an exercise for the reader; the cost of the project depends on the tasks and time estimates and the team.

Categories: Leadership

Notes on “The New Manager Death Spiral”, a presentation from Michael Lopp/Rands In Repose

Here are some notes from the event that I attended where the VP of Engineering at Slack, and more famously, the writer behind the Rands In Repose blog (and author of three books), Michael Lopp presented on becoming a manager and the mistakes that will be made by new managers.

Some of the talk was based on the blog post written by Mr Lopp.

The Notes on the New Manager Death Spiral

Began with some questions about who’s a manager, designer, engineer, introvert or extrovert. Talk described as anti-advice. Walks through the manager journey, beginning with hey congrats you’re now a manager. Lopp asked how many people have formal managerial training. Personally started with “taking care” of a few team members, started with setting up one on ones.

When starting out as a new manager, you will assume that you are still an individual contributor. You will forget that you’re responsible for the team, forget to delegate tasks or you will be hesitant to give away some power. The first failure mode is that the quality of work drops, which leads to being behind schedule.

The overarching goal behind management and why it’s important

Your job as a manager is to get things done at scale.

Delegation and Delegating Tasks

Fake delegation is giving not enough context and not enough power to shape things. Successful delegation builds trust in the team. Saying, “Go figure it out….or else” is when the team starts talking to each other about the manager not being trusting of the team.

The manager’s job is to delegate aggressively.

Opinions become facts, and team is demoralized (within the new manager death spiral).

Management can be seen as a career restart. With our industry’s habit of not formally training people…

Lessons learned…let others shape your thinking…augment your obvious and non obvious weakness with a diverse team… delegate more than is comfortable.

Notes on the Q&A

How do you recognize management potential in software developers? Empathy tied into emotional intelligence, can the engineer read the room and being situationally aware, it’s a good leading indicator.

How to build trust with management above? Managing up is the same as managing team, clear communication and trust. During rapid growth, bad politics happens a lot within companies.

What’s the biggest management mistake you’ve made in the past year? Would assume data literacy of team but didn’t frame the narrative and shared too much data which led to drama, thought was doing right thing through transparency.

How do I get team to give critical feedback? Started with one on ones, always have an agenda. Dial it up and give feedback that is positive and negative. Slight increase every month and then ask whether they have feedback for you. At some point they will trust you enough to give feedback. Just listen, don’t try to understand, stop and say it back “what I heard was this”. Then they correct, and it builds trust. Anecdote it took a year to get this feedback and to build that trust. “Feedback is a gift.”

Recommendations for newly promoted to manager? What if you are promoted above your friends? First couple of months with friends, be explicit about friend hat and manager hat, same person just changing the context. Set boundaries. You can’t fire your friends but as manager you can fire them. You have to be deliberate about this.

Measure manager, 4 things to look for. Vision, can you describe a compelling future. Strategy, can you design the road signs and deconstruct the vision into road signs to guide the team. Tactics/execution…Judgement and decision making, you’re coming to crossroads all the time so why choose one path or another and how well do you explain it. Subjective and kinda objective.

Asks team leads about plans they have for growth of team.

Serial lack of training, where to get it? More important is getting company culture to train internally and usually first question he asks is how do we train managers. If small set up mentorship circles or use external resources.

How do you interview for judgment? My job is soup tasting (metaphor) what I’m looking for is how you explain path A or path B selection and need you to explain to me why did you choose that path. If you can explain it well and your strategy around that choice is explained well and you walk through it, is how he susses out judgment, was it a flip of a coin or decision with real consideration.

Do we need managers? Holacracy isn’t working out at Zappos or Medium. He thinks that managers serve a force multiplication function. Managers are there as information conduits and to lead growth of team. Self managing teams at scale need some form of management even if it isn’t called that.

From coaching and trust standpoint, how to build trust on remote team? It’s hard to do this, has bias to look face to face to know what’s really going on. Tries to travel a lot to meet the team. Depends on culture of team, how they answer questions and deference to managers. Moment when it blows up is if remote employees start to feel like second class citizens. Avoids remote, being able to see the whole team gives ability to read the room better.

Should managers be direct contributors? There are four roles to grow into, be CEO CTO VP of engineering or chief architect. Aspiration to be CEO moves to see other parts of business. VP of engineering is people and process. CTO is operations and still coding. Chief architect is still writing code. You need to be able to draw a clear path to each of those roles.

When you learn how well you’re doing is when there’s an emergency, it’s okay to fail and reflect and learn. Fail faster and make managers feel safe. Say what happened, and say it’s ok if you screw up, and work on the problem instead of worrying about the failure.

How do you efficiently delegate? At a small startup under 20 people you have the team and don’t need managers, but when you get past 20 to 50 or more, you need to start bringing in managers with good judgment to connect things together. They give the scaling function to the organization.

When you delegate, do you focus on delegating to people’s strengths? Sometimes things need to go fast and needs velocity, delegate to strength, tends towards growth mindset and will delegate based on learning/growth.

How do you improve reading the room as a skill? Advise extroverts to listen, introverts already listen more.

How do you manage people who don’t take feedback well? They need to have a safe environment of feedback, the manager needs to show that this is a valuable transaction and valuable communication.

Technical details…A minute a slide? Something like that. Slide text was at bottom and hard to see.

Categories: Leadership

Scrum Methodology Diagram

For project managers who are training in SCRUM agile, here is a scrum methodology diagram for reference:

Diagram of SCRUM Agile Methodology with sprints, product owners, daily scrums
SCRUM Agile Methodology Diagram. [Click on the diagram to view the original size image.]

Scrum Agile Methodology

  1. User stories (and tasks, features, etc.) are written and added to the product backlog. These can be written by the team, the project manager, project owner, or any other stakeholder.
  2. The Product Owner (or project manager) organizes the product backlog.
  3. During a sprint planning meeting, user stories are selected for design and development and implementation and prioritized into the sprint backlog by the product owner.
  4. The team works on the items in the sprint backlog with the Scrum Master helping to unblock team members.
  5. Every day the team meets for a daily scrum meeting where they discuss what they are working on and if they are blocked or stuck on anything.
  6. At the end of a sprint, there is a retrospective to see what went well and what needs to change.
  7. Then the cycle begins again with new user stories added, existing user stories selected for development and so on.

What is the Product Backlog?

The product backlog is the list of potential user stories that can be worked on. The list can contain tasks bugs and features as well.

It can be organized by priority, by estimated dollar value, by urgency, by number of customers who would be happy, or by some other criteria.

What is a User Story?

In contrast to traditional project management where work packages are worked on, in agile you have a user story. Each user story includes the user persona or actor. It also includes what they would like to be able to do in the system.

Examples of user stories

  • As an author, I would like to write a book (for a word processor)
  • As the system administrator, I would like to view all performance metrics of our active systems
  • As an educator, I want to create a course outline
  • As an teacher, when creating a course outline, I want to include PDF presentation files and images
Categories: Leadership

4 Things To Know About Teamwork

Here are the four things to know about teamwork:

  1. When the situation is tense, call a timeout, it’s the only thing that will work
  2. Information needs to be as close as possible to the team members that can make use of it
  3. Be aware of the message you’re sending as a team leader, senior developer, CTO because it affects how the team works together
  4. Some team members want to learn new things and develop their skills, you absolutely must help them do that whenever possible
Continue reading “4 Things To Know About Teamwork”
Categories: Leadership

How to apply cause and effect diagrams in IT and Software Development

Cause and effect diagrams, also known as Ishikawa diagrams, are one of 7 basic tools of quality. You won’t see them used very often in software development or IT projects though they should be. So today we’re going through what cause and effect diagrams are, why they’re useful, an example diagram, and case studies.

You will learn:

  1. What a cause and effect diagram is
  2. The usefulness of cause and effect diagrams
  3. Using industry-wide common causes as a starting point
  4. Common causes in the IT & software development industries
  5. Example cause and effect diagrams
  6. Case studies, example cause explorations based on real-world experience
  7. Applying cause and effect diagrams, applications of the diagrams to explore root causes
Continue reading “How to apply cause and effect diagrams in IT and Software Development”
Categories: Software Development

Lessons from “Producing Open Source Software”

The free book, Producing OSS / Producing Open Source Software, is one I’ve referred to now and again when thinking about how free/open source projects should be run and lately I’ve been thinking about how a big company like Red Hat works when they’re based around free/open source projects.

Here are some choice quotes that I think can serve as useful lessons for open organizations.

Continue reading “Lessons from “Producing Open Source Software””

Categories: Leadership

SEMAT Kernel Example – walking through a small task

I have been reading the SEMAT Kernel book, The Essence of Software Engineering: Applying the SEMAT Kernel, and it provides a new way of managing software development projects. It is supposed to make it easy to check the health of a project by classifying the major components, known as Alpha states in SEMAT, and the progress level that they’re at.

Continue reading “SEMAT Kernel Example – walking through a small task”

Categories: Leadership

Effective Technical Leadership

Some notes from the Effective Technical Leadership talk given by David Byttow.

Attributes of an effective technical lead

  • Knowledge: “A strong tech lead’s knowledge is broad and deep…A tech lead should be a master of several technologies.”
  • Speed: “be ultra-responsive and capable of making instant decisions, always kicking the ball forward”
  • Awareness: “You should be able to keep the current state of the entire project in your head at all times.”


Some key actions

  • Help create and stack rank project priorities
  • Define best practices for issue tracking
  • Coach other engineers
  • Review code in detail and provide useful feedback
  • Shield engineers from management when needed
  • Explain why decisions are made
  • Fight for the right design decisions
  • Load-balance work among the team