Data Documentation: Vital Piece of the Data Puzzle

Writing data documentation belongs to the famous “important but not urgent” tasks.

There's always something more urgent to do … Or more exciting! You might also not know where to start or how to proceed.

This article launches a series on data documentation, with the goal of addressing 4 key questions about the subject:

‍

🎯 Why?: Why is it vital to pay attention to data documentation? It prevents repeated questions from data consumers, allowing data producers to allocate their time towards more valuable pursuits.

‍

📖 What?: What are the different forms of data documentation? Although data dictionary or business glossary may come to mind first, there are additional capabilities that are just as important.

‍

✍🏼 How: How can you approach data documentation effectively? Documentation is not magic, just common sense. In this part, we will set up a framework for curating documentation.

‍

🔍 Where: Where should data documentation live? Organization is one of the most critical aspects of documentation. True success in documentation is achieved when readers are able to independently access the information they need and are satisfied with what they find.

‍

We’ll explore the significance of the first question, "Why?", in this first article, as well a few rules for effective documentation. Want to know more? Subscribe to receive the rest of the series directly to your inbox.

‍

Let's dive in!

I - Data Documentation: The Secret to Organized and Meaningful Data

Technical documentation is an essential component of workplace productivity and collaboration. It serves several key purposes, including:

• Reducing the number of recurring questions: Technical documentation provides clear and concise information. By having this information easily accessible, employees can find answers to their questions without having to constantly ping their colleagues on slack, saving a lot of time. Since investing in data documentation, Printify has reduced the number of data-related slack messages to their data engineering team by 90%.

• Putting an end to tribal knowledge: Technical documentation centralizes information to help avoid tribal knowledge, a common challenge in larger teams where important information is only known by a small group of people. This ensures that all team members have access to the same information and can work collaboratively towards common goals.

• Simplifying handover: Technical documentation can make it easier for new team members to onboard and understand the systems, processes, and procedures in place. Therefore, it can significantly reduce the time it takes for new employees to become productive and contribute to the team.

• Sharing knowledge: Writing down information about systems and processes can be beneficial for all team members, especially those who encounter similar problems in the future. By documenting information, team members can share their knowledge and save their colleagues time and effort. As a rule of thumb, if you spent time figuring out something tricky: write it down. You might save the day of another teammate… if not yourself!

• Unburdening your mind: Knowledge can be a burden, especially if you are a veteran at your company. Do yourself a favor: write it and get rid of it. Technical documentation can help relieve the burden by providing a centralized and easily accessible repository of information. This habit can help team members focus on their core tasks and be more productive.

II - 3 Rules for Effective Documentation

Properly executed data documentation can open many doors. However, to achieve this outcome, you must follow certain guidelines. To turn your data team into a powerhouse, make sure you utilize only reliable and up-to-date data, document in a manner that promotes dashboard reuse, and combat metric inconsistencies.

‍

‍

Avoid using untrustworthy data

‍When starting on your data documentation project, avoid using data that is out of date, data that is known to be duplicated, work tables, and sandboxes. These types of data can be unreliable and mislead data users. Only document data that is reliable and up-to-date, to ensure everyone in your company is working with the right information.

‍

Don’t reinvent the wheel

‍Instead of creating a brand new dashboard from scratch, you can potentially take an existing one and modify it to suit your needs. Dashboard reuse can be a great way to save time, energy, and resources, as it leads to less maintenance work and a reduction in storage and processing costs.

For this reason, your documentation should promote dashboard reuse. Data documentation can support dashboard reuse by providing clear and comprehensive information about existing dashboards, including their purpose, data sources, calculations, and visualizations.

This information can be used to evaluate whether an existing dashboard can be adapted to meet new requirements, or if a new dashboard needs to be created from scratch. By having this information readily available, decision-makers can make informed choices about dashboard reuse and avoid duplicating effort.

‍

Fight inconsistencies

It’s common to get different figures for the same metric from one table to another. Without a unified source of truth for metrics, different departments may adopt their own method of calculation, leading to conflicting metric interpretations across the various departments within the same company.

To avoid this, naming conventions and folder organization rules should be strictly followed. This is essential in order to maintain consistent data across the board and ensure that any discrepancies can be easily identified and fixed. It is also important to keep track of a timeline of all changes so any sudden shifts in figures can be quickly and accurately traced back to their source and addressed. These measures will help to promote accuracy and avoid any confusion or errors in the future.

Final words

Data documentation is vital to ensure data users can extract value from the data. It brings clarity to information, eliminates repetitive questions, keeps information organized, makes transitions smooth, and reduces the burden of knowledge.

This article emphasized why data documentation is important, and the crucial elements of effective data documentation including the prevention of inaccurate data, promotion of dashboard reuse, and avoidance of inconsistencies.

By keeping these principles in mind, data documentation can be a powerhouse practice that elevates productivity, streamlines processes, and fosters teamwork in the workplace.

‍

Subscribe to the Castor blog

About us

We write about all the processes involved when leveraging data assets: from the modern data stack to data teams composition, to data governance. Our blog covers the technical and the less technical aspects of creating tangible value from data.

At Castor, we are building a data documentation tool for the Notion, Figma, Slack generation.

Or data-wise for the Fivetran, Looker, Snowflake, DBT aficionados. We designed our catalog software to be easy to use, delightful and friendly.

Want to check it out? Reach out to us and we will show you a demo.

‍