What Is dbt Compile?

What is “dbt compile”?

dbt (which stands for "data build tool") is a software tool that helps data analysts and engineers transform data in the warehouse more effectively. It allows users to write, document, test, and execute SQL-based data transformation workflows.

The dbt compile command is used to compile dbt projects. Here's what it specifically does:

1. Jinja Rendering: dbt uses the Jinja templating engine, which allows users to write more dynamic SQL code. The dbt compile command will process these Jinja templates and produce the pure SQL code that can be run against a data warehouse.
2. Dependency Resolution: If you have models that depend on other models, dbt compile will ensure that they are compiled in the correct order. For instance, if Model A relies on Model B, then Model B needs to be compiled before Model A. This ensures that the final transformation scripts are correctly ordered.
3. File Generation: After processing the templates and resolving dependencies, dbt compile will create a set of SQL files in the target/ directory of your dbt project. These files represent the final SQL code that will be run against your data warehouse.
4. No Data Changes: It's important to note that dbt compile does not make any changes to your data warehouse. It simply prepares the SQL scripts. If you want to actually run the transformations, you would use the dbt run command.

In essence, dbt compile is a way to preview and prepare your dbt project's transformations without executing them. It's a useful step for debugging and ensuring that your Jinja templates are producing the desired SQL output.

dbt compilation - step by step

1. Jinja Rendering

Jinja is a templating engine for Python, and dbt uses it to allow for more dynamic SQL code. With Jinja, you can use logic and variables in your SQL.

Example:

Suppose you have a variable for the date and you want to filter a table for records from that date:

When dbt compile is run, this would be rendered to:

2. Dependency Resolution

dbt models can depend on other models. This dependency is typically specified using the ref() function.

Example:

Imagine you have two models: staging_orders and final_orders. The final_orders model relies on the staging_orders model.

When you run dbt compile, dbt understands that final_orders depends on staging_orders and will compile them in the correct order.

3. File Generation

After compiling, dbt will generate SQL files in the target/ directory of your dbt project. These files represent the final SQL code.

For instance, if you check the target/compiled/<project_name>/models/ directory after running dbt compile, you might find:

These files will contain the rendered SQL, free of any Jinja code, and ready to be run against the data warehouse.

4. No Data Changes

This isn't something that can be shown directly with a code example, but it's crucial to understand. When you run dbt compile, it doesn't execute any SQL against your database or data warehouse. Instead, it just prepares the SQL scripts for execution. If you were to run the dbt run command afterwards, that's when the transformations would actually be applied to your data.

So, for example, even if you have a model that says:

Running dbt compile will not delete anything. It will simply generate the SQL file. Only a subsequent dbt run would execute the dangerous deletion. Always be cautious and understand the implications of your dbt commands!

‍