Data Mesh Accelerate Workshop
Any organization hoping to survive in a technological, globalized world has to effectively work with data. With 97% of organizations investing in big data and AI, the need to effectively manage this data is more critical than ever. Data Mesh helps companies to generate valuable insights and live up to the promise of becoming data driven organizations.
The Data Mesh paradigm is founded on four principles:
- domain-oriented decentralization of data ownership and architecture
- domain-oriented data served as a product
- self-serve data infrastructure as a platform to enable autonomous, domain-oriented data teams
- federated governance to enable ecosystems and interoperability
Data mesh is a decentralized socio-technical approach to remove the dichotomy of analytical data and business operation. Unfortunately, as with many socio-technical approaches, many organisations struggle to align on the goals and strategy that is really needed to make a data mesh transformation a success. The transformation to a data mesh is hard. It represents a paradigm shift in how teams are organized, how work is prioritised and how to apply the newest data related technology advances. This is a wide reaching change that impacts people across the whole organization.
We have been part of several Data Mesh transformations, across a wide variety of organizations. We have also exchanged and learned from Thoughtworkers around the world who have been involved in many more.
We have seen two common challenges facing Data Mesh transformations: technology focused and big design upfront.
In one approach, teams start building data products without aligning their work with higher level goals. This approach (technology focused) is based on a genuine thinking that Data Mesh is all about technologists and new ways of collecting, transforming and serving data. The main goal of Data Mesh is to unlock the value of data at scale, and while technology is a key part of this, alignment with business goals is critical for success.
To start from the technical side without being very aligned with the business strategic goals is like packing for a holiday without knowing where you are going. Don´t do it. Make sure there is an initial alignment before starting the journey.
Another common approach is to do a deep dive analysis before starting with Data Mesh implementation. This is Big Design Up Front, with long periods for analysis and design, often a few months. The result: lots of diagrams and plans for the path going forward.
Diagrams and plans are great, but the value comes from making them real and delivering value to the users and the business. A key concept of Data mesh is product thinking, where we incrementally evolve our products based on user needs and real usage. Our approach to the data mesh transformation as a whole also needs to follow this evolutionary approach, with short feedback loops.
The Data Mesh Accelerate Workshop
At Thoughtworks, our response has been a process called Data Mesh Accelerate. We have put together a good sequence of activities to set an initial direction, to build understanding and to create strategic alignment. It provides enough initial analysis and design to get started, and a good balance between technology and business alignment.
The Data Mesh Accelerate workshop provides a series of activities focusing on collaboration, the capture of the current state and the mapping of aspirations. The main goal is to build the initial alignment across stakeholders, to understand what kind of Data Mesh outcomes the group is aiming at, and explore the process for identifying, designing and building data products.
Leaving the workshop, stakeholders should have a shared understanding of Data Mesh concepts, what the journey ahead looks like and alignment on the next steps. The Data mesh workshop represents the first step on an organization’s data mesh transformation, and while there are many steps ahead, we have found that taking the time to make the first step successful, pays off massively for the journey ahead.
Where the accelerate workshop fits in
The Data Mesh Accelerate Workshop is one part of a data mesh transformation. To kick off a data mesh transformation you start by selecting an appropriate domain to work with, then you go from the vision to the use case (accelerate), then you identify the data products, platform and organizational changes needed to support that use case (discovery and inception) before actually building the data products and adding them to the data mesh.
The Data Mesh Accelerate workshop fits in when you have got the domain stakeholders (business and technical people) interested and available for a few hours to kick start a successful Data Mesh transformation. It provides the means to get everyone aligned on the initial path.
Facilitating any workshop, let alone a multi-day workshop like Data Mesh Accelerate, requires preparation to make it a successful experience. The sub-sections below explore what you need to do before the day of the workshop.
Materials, tools and logistics
At the most basic level you will need to arrange the material and tools (or their digital equivalents) and logistics required for the workshop:
- Gather Post-Its and pens
- Prepare collaboration boards (see below)
- Give access to the needed tools (video-conferencing, online boards etc.)
- Book a room and time in the participants calendars
Preparing the audience
Along with understanding what the workshop is about, each participant needs to know what is expected from them. It is your job to help clarify it as well as to make sure the right people are invited to the workshop.
You should communicate the goal of the workshop and where it fits with the context of the group going through it and the organization. You have to put it in context on what has happened before and what will happen after it. You can achieve this in several different ways:
- Send a short introductory email
- Share a explanatory video
- Schedule a conference call
- Share a link to this article and to Zhamak’s articles
- Schedule one-on-ones with participants
Besides preparing the audience, you will be gaining an insight into how technically skilled and familiar with data mesh the participants are. This will also help you adapt the content to their specific needs.
Every workshop needs an agenda. Clarifying the agenda and inviting people well in advance will help put your participants at ease and help them find times in their busy schedules.
In the next section is a typical agenda we have been using. We recommend you start from there and adjust it according to your specific context and needs.
Preparing specific activities
Each activity needs a working area with its instructions. These are described in the next section. But it is your job to prepare these areas for your workshop. In a face to face setting, you might prepare a few flipcharts, while if you are remote, you will arrange the template on your collaboration tool of choice. We describe the activities in terms of their outcomes and their step-by-step, leaving you to adapt to your specific context and setting.
Please find below a sample agenda and the explanation for each of the activities.
Here is a link to a MURAL template following the above agenda.
The sample agenda lays the workshop out as four afternoons. We typically go with such an agenda when running the workshop remotely. It avoids videoconferencing fatigue and makes it easier for participants to fit it in their busy agendas.
Whenever we run this workshop in person we do it in two consecutive days. We reserve a good conference room and fill the boards and walls with many post-is as we go through the activities.
A Sample Agenda for a data mesh accelerate workshop
The week starts with a kick-off, followed by a sequence of intense sessions and ends with a review. The agenda and expectations for the week are covered on the kick-off. The results obtained from the workshop are presented in the review.
Step by Step
- Ask the main sponsors of the organization to open the workshop with a speech about the importance of this engagement.
- Make a brief presentation about the intention of the overall workshop, its agenda, and each activity. This is also your chance to bring attention to the ground rules (see below).
- Ask everyone to quickly introduce themselves (tip: besides name and role, it is nice to ask people to share something else, such as favorite food or vacation spot).
The kick-off should end with a clear understanding of the goal for the workshop and the upcoming activities. It may be the first time some people get to work with each other, therefore the importance of the tip, making some space for people to connect to each other.
Setting clear ground rules, such as switching off phones or expecting people to be open to other perspectives will help participants understand what is expected of them and help the whole workshop run smoothly.
Data Mesh Four Principles and challenges ahead
The four principles of Data Mesh activity has two main goals: (1) build a shared understanding of the four principles of Data mesh — Domain Ownership, Data as a Product, Self-serve Data Platform and Federated Computational Governance –, and (2) foster an open conversation about the challenges to implement each of these in the current context of the organization and the group.
Step by Step
- Share a quick introduction to the four principles
- Ask the participants to share the threats to applicability for each principle: “What are the challenges in the way for us getting there?”
- Have a conversation about the challenges ahead
It is important to get everyone on a similar level of understanding about Data Mesh and its principles. Someone with a good understanding of data mesh should give a short presentation to the audience, covering at a minimum:
- The “Why” of Data Mesh, what problems it aims to solve.
- The Four Principles of Data Mesh, explaining the reasoning and details of each.
After sharing the information, give the audience time to clarify, share and ask questions about the principles.
Ask the participants to share their previous experiences, concerns and where they see the challenges ahead for each of the four principles. Talking about the challenges helps the group get an initial feeling about the areas to pay closer attention to, especially in the beginning of a transformation driven by data.
“If we were to start applying this principle tomorrow, what challenges would we face?”
As participants are sharing their perspectives, ask the rest of the group to compare their perspectives to those being presented by asking three questions; what was the same, what was different and what surprised them. Asking these questions is a great way to compare and contrast the different perspectives in the group, uncovering similarities and blind spots.
By exploring the challenges faced by the organisation and the pain points raised by the participants, you are able to understand the current context and frame subsequent discussions appropriately. Given that Data Mesh aims to address common failure modes experienced by data organisations, understanding the specific challenges faced by your organisation is a great place to start!
Data Mesh Nirvana
The goal of this activity is to understand what good looks like in the context of Data Mesh by creating a nirvana statement that clearly articulates an ideal future state. This activity helps the group look forward and provides an opportunity to introspect on some of our current pain points.
“Nirvana is that special place, that end state of perfection (many times not achievable, but important to aspire for). It is very important to align and understand what nirvana is, for this group of people and within this organizational context.”
Step by Step
- Break the participants into smaller groups.
- Have each group create a nirvana statement, then present it back to the larger group.
- Combine to create a shared nirvana statement.
Breaking up the participants into smaller groups helps facilitate lively discussion. People increase their participation when working in smaller groups. We find that groups of 4-6 participants made active discussion easier.
Give each group the same instructions: to write a short statement about the ideal future state for the Data Mesh transformation. Stickies with the prompt “Our Nirvana is…” can help kick off this discussion.
After the groups have created their nirvana statements, reconvene and have a volunteer from each group read out their statement and share the thinking behind it.
The final task is to combine the different statements into a single shared nirvana statement. We recommend a fishbowl approach to stimulate discussion and collaboration. By the end of this process, you should have a single nirvana statement that everyone in the group agrees represents a desirable goal.
It is very interesting to see how this simple question gets participants very engaged and then aligned: what is our nirvana? Once the group is in agreement about that, it’s much easier to clarify the steps towards it.
Before talking about options, actions, initiatives, or any execution task towards achieving something big, it is essential to align on that high level goal.
4 Key Metrics
The four key metrics activity has two main goals: (1) to foster a conversation about the current state of the four key metrics, and (2) to make it visible where the team aims to get to.
The book Accelerate and the associated research demonstrates the importance of the 4 Key Metrics for achieving great organizational and software delivery performance. The 4 Key Metrics are Lead time, Deploy frequency, Mean Time to Restore (MTTR) and Change fail percentage.
Tell me how you measure me and I will tell you how I behave
Step by Step
- Share details of the 4 Key Metrics and have a discussion about their importance.
- Ask the participants to share where they consider they are today on each of these 4 metrics
- Discuss future expectations around the 4 Key Metrics.
Start by introducing the 4 Key Metrics so that participants are familiar with their usage and the research behind them.
Within the workshop participants there should be representatives of different areas and different roles. This diverse group of people is needed to build a complete understanding of where they are for each of the metrics.
Start at the first metric, lead time, for example. Ask the participants to share where they consider they are today as per the Lead time metric:
- More than six months
- One to six months
- One week to one month
- One day to one week
- Less than one day
- Less than one hour
As the dots are gathered, ask the participants to share the stories behind the dots. The outliers in particular can offer valuable insights into the experience of the teams.
To close the activity, you should foster a conversation about how the participants think they could improve on these metrics over the next year. Even though the group is not yet deciding exactly how to get there, it is important to open the conversation and pay attention to the effort ahead of them. To kick off this conversation, you could say: “consider a year from now, where do you believe we could be at for each of these four metrics?”
Metrics bring an important conversation about desirable outcomes and how to measure progress. But even more, it raises everyone’s attention to the current state and to a desirable future state, including an initial conversation about how far away the desirable state is. This activity brings tech and business together, not only by having a common goal, but also by looking at progress via similar (measurable) lenses.
Objectives and Key Results
This activity’s goal is to align on the organisation’s main objectives and start a conversation about measurable progress towards success. There are many ways to have this conversation. In this article, we have chosen OKRs – Objectives and Key Results, but building a Lean Value Tree is another effective approach that links business vision to our daily work. Of course, if your organisation is already using a goal/objective framework, you might be able to save some time by pulling in your last set of objectives from that!
Objectives and Key Results (OKRs) are a goal-setting framework that helps a group of people define goals — or Objectives — and then track the outcome via quantitative metrics, the Key Results. OKRs has been around since the 1970s from Intel, and many popular organizations, such as Google, Oracle, Twitter, LinkedIn and Dropbox have adopted them.
Objectives (Where do we want to go?) and Key Results (How do we know if we are getting there?)
Step by Step
- Identify the high level objectives of the organization
- Brainstorm key results that can measure progress towards those objectives
- Iterate over the OKRs as a group
OKRs can be tricky to write well. Are the objectives really worthwhile outcomes or are they outputs? Do the Key Results really capture what is needed to deliver on the Objectives? Avoid common mistakes by giving the participants time to refine their OKRs.
Identifying Objectives and Key Results helps make the journey ahead more tangible. As we move ahead and start to define the (data) products we intend to build, keeping these OKRs in mind helps bridge the gap between outcome and output.
Explore the Use Cases
Having identified your Objectives in the previous step, you now turn your attention to how you can actually deliver on those goals. Use cases are a translation of the desired outcome for a customer or a user. In the context of analytical data, use cases are usually either:
e.g. Forecast service usage to optimise efficiency
e.g. Recommend a service to users based on their past purchases
This activity is the moment for the group to brainstorm and align on Data Mesh Use Cases. To help focus the discussion, we provide the following Use Case template:
We believe that <this Use Case> will help achieving <this GOAL>
Step by Step
- Divide the participants into groups and ask them to brainstorm use cases, using the template above.
- Have each group present their use cases to the wider group, followed by discussion and clustering
- Refine and select Use Cases
You should kick off this activity with a definition and discussion of Data Mesh Use Cases. This will make the brainstorming more productive. At this stage, you want to gather as many ideas as you can, because you will filter down later.
When the groups return, have a volunteer for each group present the use cases they identified and open up to a wider discussion. This is a great opportunity to combine (or cluster) similar use cases and discard non-analytical use cases.
Once you have a set of analytical use cases, you need to decide which ones to take forward to the next activities. If you have a few Use Cases, you might take all of them to the next activity. Although if there are too many, you should prioritize them. There are quite a few prioritization techniques, choose the one that fits your context.
“Build it and they will come” is a pitfall that many transformation efforts fall into, so by keeping the Use Cases connected to the Objectives previously identified, via the presented template, ensures that the work is aligned with the high level engagement outcomes.
Discovering Data Products
Data Mesh applies Product Thinking to remove friction, deliver value and truly delight our data users. Data Products are the architectural quantum of Data Mesh, the “smallest unit of architecture that can be independently deployed with high functional cohesion, and includes all the structural elements required for its function.” (definition from Evolutionary Architectures).
These Data Products represent the building blocks of Data Mesh and in this activity, we identify Data Products that can help us satisfy the Use Cases we identified above. The interaction of one or more Data Products fulfills a Use Case.
Having a small set of Use Cases in the previous activity, you now need to start comprehending, proposing and mapping the relationship between Use Cases and Data Products.
Step by Step
- Identify Data Products for the selected Use Cases
- Identify sources and consumers for each data product
- Map the connections between the data products
“If you were to hire a data product to help address this Use Case, what would its job be?”.
Jobs to be done (JTBD) is a framework created by Clayton Christensen for understanding customers and their motivations for adopting a new product or service. For defining a Data Product, we suggest you follow the same approach. The simple question above helps the participants start a discussion about how to put their data to good use.
Once you have identified the data product’s job, you can dig a little deeper by asking what data would be needed to actually carry out that job. This may include existing master data or operational systems, manual data entry or even other data products.
With the sources of the data product identified, you should ask who would actually be interested in consuming this data. This may include particular people, roles, other data systems or other data products.
Part of the vision behind Data Mesh is a mesh of reusable, interoperable data products. In the final part of this activity, you want to map out how the data products interact with each other. Wherever a data product consumes from another data product, draw that link on the board. At the end of this activity, you should start to see a data product interaction map
Identifying Data Products, their sources, their consumers as well as the connections between them are an essential part of a successful Data Mesh transformation. At this stage of the workshop, the group will have connected very important pieces of the puzzle for the first few Use Cases.
Wrap up and review
Wrapping up the workshop is important in order to provide a sense of closure to the experience, review the progress made, and initiate a conversation around next steps. The accelerate workshop will feel much more complete when you give a short summary at the end and provide some context for next steps.
You should wrap up the workshop by summarizing the takeaway points and the connections from the different activities of the workshop. Some topics you may want to cover in the review include:
Step by Step
- The goals, use cases and data products identified during the workshop
- Any interesting observations, learnings or recommendations
- Details of next steps
It may also be helpful to invite external stakeholders, who may have an interest in the outcomes of the workshop to this session.
After the review, there are a few activities that we recommend wrapping up with:
Running a short futurespective activity like Future LinkedIn Posts helps the participants visualise where this journey will take them.
If time permits, a short check-out activity like One Word Before Leaving can be a great way to share experiences and gather feedback from your participants before bringing the workshop to a close.
With your last activities done, you can bring the workshop to a close as you look forward to kicking off your data mesh transformation!
The Data Mesh Accelerate workshop is an intensive workshop, bringing together participants from across your organization to build a shared understanding of Data Mesh and alignment on the next steps.
Things are moving really fast in the technical and business data world. Workshops and activities such as the ones we presented here are the means to bring people together, align and plan for a successful transformation.
We and many of our colleagues have been using what we shared here to achieve great results. We hope these work for you as well and you use it, adapt it, add to it and keep on sharing with our great data community.