How To Create a Documentation Framework

Writing data documentation belongs to the famous category of “important but not urgent” tasks.

There's always something more urgent to do … Or more exciting. You might also not know where to start or how to proceed.

Last week, we dove into three different use cases for documentation in the second article of our of our “Data Documentation Demystified” series. This third article is about setting up a data documentation framework. You can subscribe to receive the rest of the series directly to your inbox.

Data documentation is painful. People don’t proactively rush to document the data. Building a strong data documentation framework incentivizes your stakeholders to collaborate on writing documentation. It is essential for any team or organization seeking to improve collaboration and productivity.

‍

In this article, we will explore how to create a comprehensive and effective framework for documenting your data, ensuring that documentation is easily accessible to your team and is well-organized for long-term use. We will cover the following topics:

1. Establishing a data documentation structure: Learn how to develop a structure for organizing and maintaining your data documentation, including best practices for categorizing and labeling information.
2. Delegating the documentation effort: Discover strategies for assigning documentation responsibilities across your team, ensuring that everyone is equipped to contribute to the documentation process.
3. Answering common questions: Explore how to use your documentation to quickly and accurately answer common questions that arise in your daily data operations.
4. Spring Cleaning: Finally, we'll discuss how to regularly review and update your data documentation to ensure it remains relevant and useful over time.

1. Establish a data documentation structure

To create a documentation framework, it's important to start with a clear and well-organized structure. This structure should not be too detailed, but at least should include the main categories that your team will be working on. For example, you may want to include categories such as "Onboarding documentation" "Technical documentation," or "Business knowledge documentation". You can read more about these three main use cases for documentation here.

Without a clear hierarchy, it's easy for the documentation to become disorganized and difficult to navigate, especially as more information is added over time. By establishing a clear structure from the beginning, team members can quickly and easily find the information they need, without wasting time searching for it.

Crafting a homepage is also an important step in creating clean and organized documentation. The homepage should be clear, easy to understand, and provide an overview of the documentation structure. It's also a good idea to pin the homepage in relevant places, such as the team's shared drive or communication platform, so that team members can easily access it.

‍

Once you have established a clear data documentation structure, you can focus on delegating. Which leads us to the next section…

2. Delegate

When it comes to documentation, delegating responsibility to knowledgeable individuals is vital to ensure information is complete. Documentation is especially important when it comes to key data assets such as tables, dashboards, and KPIs, which are used frequently by team members to make important decisions.

Before the cloud era, the data steward was in charge of metadata management, ensuring all data assets were properly documented. This ensured that those who consumed the data would understand it. The issue is, this can’t be a one-man job anymore. There is simply too much data for all documentation to be handled by one person. Plus, data is continuously evolving, and documentation should be too. Automating data documentation has solved part of the problem. The other part can only be solved by leveraging collaboration.

Collaboration requires organization. And as I previously mentioned, no one rushes to document the data, so you can’t leave documentation to people’s good will. Here are a few tips to organize a collaborative documentation effort:

• Make specific individuals responsible for ensuring key data assets are maintained, up to date, and accurate.
• Ensure that these individuals are knowledgeable about the data and its use, so they can provide accurate descriptions and schemas of the data. This will help team members to understand the data and use it effectively.
• Require these individuals track any known issues, related tickets, or works in progress related to the data. This will help ensure that any issues or problems are quickly identified and resolved, preventing them from causing issues down the line.
• Get owners to add relevant tags to optimize search results. By including appropriate tags, team members can quickly and easily find the information they need, improving efficiency and productivity.
• Establish a refresh schedule to ensure that the data is kept up to date. This will remind owners to update documentation when needed, and prevent team members from making decisions based on outdated or incorrect data.
• Get owners to use a visual warning on deprecated assets. By highlighting assets that are no longer in use, team members can avoid using outdated or incorrect data, and ensure that their decisions are based on the most up-to-date information available.

Once you manage to delegate the documentation effort in a way that suits your needs, the next step is to ensure you have a framework in place for answering common question. This is what comes next…

3. Answer common questions

Documentation is meant to be USED. Its primary objective is to enable data consumers to independently answer data-related queries. Therefore, it should include a structure to address routine questions.

Integrating a mechanism for addressing day-to-day inquiries into the documentation framework can alleviate the workload of your team and guarantee that everyone has access to precise and current information.

To start, it's important to educate your data consumers to ask their questions directly in your data catalog. This encourages team members to search for the information they need before asking for help. By doing so, they can quickly find the information they need without having to wait for a response.

To do this, you need to reduce the number of questions asked in Slack or other messaging channels. While Slack can be a useful tool for communication, its knowledge has a short lifetime. Messages can quickly become outdated, and it can be difficult to retrieve information from old messages. By encouraging team members to ask their questions in the data catalog, you can ensure that the information they receive is accurate and up-to-date.

‍

When the same question is asked several times, it's a sign that something needs to be improved. Perhaps the documentation is missing or unclear, or it's difficult to find. In some cases, team members may not even be aware that documentation exists. In other cases, they may be taking the easy way out and asking for help instead of searching for the information themselves.

To encourage team members to search for information, be patient when answering their questions. Provide them with a quick response and a link to the relevant documentation. By doing so, you can help them find the information they need quickly and easily. Over time, they will become more self-sufficient and start searching for information on their own.

While establishing a framework to handle daily inquiries is beneficial, it is crucial to ensure its longevity. Sustaining such a framework over time necessitates periodic "spring cleaning," which is the key to its long-term success. In the following section, we will explore this concept in greater detail.

4. Spring cleaning 🧹

Once a year, it's important to organize a big campaign to refresh your documentation and ensure that everything is accurate and up-to-date. For your data documentation spring cleaning, you should pay attention to four things:

• Reorganize your documentation structure
• Update your labels and tags
• Tag your deprecated assets
• Update your FAQ.

We’ll look at each of these aspects individually.

‍

The first step in this process is to reorganize your documentation structure if needed. As your team and data assets grow, you may find that your current documentation structure no longer works as well as it used to. Take the time to review your structure and make any necessary changes to ensure that it's organized in a way that makes sense for your team.

Next, it's important to update your labels and tags. Over time, your team's terminology may change, or new tags may be needed to help organize your documentation effectively. By updating your labels and tags, you can ensure that your team can easily find the information they need.

As part of your annual documentation refresh, it's important to tag any deprecated assets. By doing so, you can ensure that team members will know that these assets are no longer in use, and they can avoid using outdated or incorrect information.

Finally, review the list of frequent questions from the past year. Are there any questions that have come up repeatedly? If so, consider adding an entry to your FAQ to help other team members find the information they need quickly. Additionally, take the time to review any dashboards or tables that remain undocumented, and document them as needed. By doing so, you can ensure that your team has access to all the information they need to be successful.

‍

Subscribe to the Castor blog

‍

Conclusion

In this article we delved into how to create a comprehensive and effective framework for documenting data, and the four different parts of the process. We hope these tips help make documentation more easily accessible and organized for long-term use.

About us

We write about all the processes involved when leveraging data assets: from the modern data stack to data teams composition, to data governance. Our blog covers the technical and the less technical aspects of creating tangible value from data.

At Castor, we are building a data documentation tool for the Notion, Figma, Slack generation.

Or data-wise for the Fivetran, Looker, Snowflake, DBT aficionados. We designed our catalog software to be easy to use, delightful and friendly.

Want to check it out? Reach out to us and we will show you a demo.

‍

‍