Introduction

This is a talk I’ve had several times and outlines my way of categorizing an organization’s level of clarity around their status, goals, and progress. I use a sequence of light metaphors: no light, flashlight, headlight, and searchlight. My goal as a consultant is to bring your organization to the next level of clarity.

No Light

The first stage is no light. This is the worst position an organization can be in and, fortunately, is pretty uncommon. Imagine you are trying to get somewhere in the woods on a cloudy night but you have light. In this stage of clarity, you have no awareness of your surroundings and when you trip and fall, you don’t really know why, you just pick yourself up and continue to feel your way around.

When you’re in this situation, your highest priority should be gaining visibility into what’s going on. Make diagrams, build monitoring and alerting, track projects and progress. Getting visibility into your current situation and understanding what systems you have currently deployed, how they are functioning, and what errors they are producing gets you to the next level of clarity, flashlight.

Flashlight

Once you have a flashlight in the dark you are much better off than before, you can see the tree before you run into it, and (if you are doing a good job of scanning) you’ll see the roots before they trip you. Many tech organizations I’ve worked with fall into this category. For this you need:

  • a relatively clear picture of your current deployments and systems
  • monitoring and alerting around production functionality
  • up-to-date documentation on development and support systems

If you’re missing any of these pieces, then I wouldn’t really consider you at complete flashlight level of clarity yet; that is, there are definitely gaps where things could jump out at you and you’d have to feel around to fix the problem.

When you’re in this situation, you have good visibility into what’s happening right now and can respond to issues as they come up. But at this point you are only responding to events as they occur. You don’t have visibility into longer term issues and challenges. To get to the next level of clarity, you’ll need to understand your trajectory and make plans. Once you have action plans and are tracking progress you will have entered the next level of clarity, headlight.

Headlight mode

Once you have short term plans and are tracking progress, you are at the headlight stage. I would say that the vast majority of my clients are fall somewhere around here, and during certain phases of your growth, this is exactly where you want to be. You are making progress, becoming aware of challenges and obstacles before they actually hit you, and are able to plan ahead for capacity (both meat-space and digital-space). For this you need (in addition to all of the above):

  • a clear picture of current development and production capacity and projections for the immediate future,
  • proactive monitoring and alerting around production functionality which warns you of situations before they arise (alerts which trigger as a system is approaching failure, not after it fails),
  • minimal manual steps during your development and produciton lifecycle, enabling you to change direction (or go back) with few manual steps.

Although the majority of my clients fall somewhere around here, most of them didn’t meet the full complement of bullets above. I do think that you should be striving to meet all of the bullets above, doing so well enable you to make progress (and make projections around that progress) and not be surprised (too often) by issues which suddenly appear. For key areas of your business, once you do have all the above, you are may need to reach for the next level of clarity.

Searchlight

To be honest, I have never worked in or with an organization which was entirely at this level of clarity. Some teams, in particular the data group and the data center group, within Facebook certainly were here. Mostly it’s an aspirational goal of being not only aware of your short term challenges and issues before they affect your deliveries, it also includes having a clear picture of your realistic 1-2 years objectives. Think of it as being on a road with a map and a spotter with a searchlight above and ahead of you. To be here you need:

  • a picture of your long term objectives and goals and be tracking your progress towards them,
  • a good idea of the knowns and unknowns on your path towards those goals,
  • active plans and tracking towards clarifying that path.

Most organizations probably don’t need to be at this level of clarity; because the business and technical environment can change so rapidly, attempting to be here all the time across all functionality is a waste of time and resources. However, for those pieces of your business upon which you absolutely depend or which stand at a key fulcrum which can lever your business from profitable to unprofitable (I would say that the two groups I called out for Facebook definitely fall into this category), you want to be at this level of clarity. To get here, you need to invest, not only in day-to-day and month-to-month planning and tracking, you need to invest in

  • potential big win projects which may not pay-off for several months or even years,
  • have a map of your objectives and your progress towards them.
  • be continuously updating and evaluating all of the above.

Conclusion

As I mentioned in the introduction, I’ve given this talk several times to prospective clients, underlining my goal of bringing the organization to the next level of enlightenment and I’ve had the good fortune to work with several organizations where we got to the next level. In particular I specialize in:

  • Payment systems,
  • complex computing challenges,
  • operations and systems architecture.

If your company/organization/engineers are working to achieve the next level of clarity, I can help with understanding and focusing your organization and achieving the next level of lucidity.

Send your thoughts, feedback, interest by emailing me at dlee@lucid-operations.com or