Before anything else, we need to setup the dbt cloud. Apart from what we did 🔗 here, we also need to follow instructions of 🔗 this video to terminate the starting of the dbt project.
<aside> 🚨
When configuring the BigQuery connection (when defining the project), make sure to specify the same location as your BigQuery data. dbt cannot build models in a location different from where the source data resides. Check the data location in the BigQuery console. Go to the dataset details and look for the "Data location" field. This will show you where your data is stored (e.g., US, EU, or other regions). Make sure to use this same location when setting up your dbt project connection.
</aside>
Links
Content of the course
The development of dbt models involves creating and organizing SQL transformations in a structured way. We'll start by understanding the basics of model creation and how to write effective SQL queries within the dbt framework. Then, we'll explore best practices for model organization and documentation.
We already loaded the raw data as the trips data in BigQuery (in my case).Now, we’re going to do the development, the testing, and the documentation, as it’s shown on the two first layers of the image below 👇. And then, at the end of the lesson, we’re going to deploy it into a production environment, in order to use BI tools.
To start, we’re going to use modular data modeling approach. This means breaking down our data transformations into smaller, manageable pieces that can be easily maintained and reused. Each model will focus on a specific business concept or transformation step, making our codebase more organized and easier to understand. This approach also allows us to build complex transformations incrementally, testing each component as we go.
Typically, we will follow these steps :