Skip to main content

Behavior changes

Most flags exist to configure runtime behaviors with multiple valid choices. The right choice may vary based on the environment, user preference, or the specific invocation.

Another category of flags provides existing projects with a migration window for runtime behaviors that are changing in newer releases of dbt. These flags help us achieve a balance between these goals, which can otherwise be in tension, by:

  • Providing a better, more sensible, and more consistent default behavior for new users/projects.
  • Providing a migration window for existing users/projects — nothing changes overnight without warning.
  • Providing maintainability of dbt software. Every fork in behavior requires additional testing & cognitive overhead that slows future development. These flags exist to facilitate migration from "current" to "better," not to stick around forever.

These flags go through three phases of development:

  1. Introduction (disabled by default): dbt adds logic to support both 'old' and 'new' behaviors. The 'new' behavior is gated behind a flag, disabled by default, preserving the old behavior.
  2. Maturity (enabled by default): The default value of the flag is switched, from false to true, enabling the new behavior by default. Users can preserve the 'old' behavior and opt out of the 'new' behavior by setting the flag to false in their projects. They may see deprecation warnings when they do so.
  3. Removal (generally enabled): After marking the flag for deprecation, we remove it along with the 'old' behavior it supported from the dbt codebases. We aim to support most flags indefinitely, but we're not committed to supporting them forever. If we choose to remove a flag, we'll offer significant advance notice.

What is a behavior change?​

The same dbt project code and the same dbt commands return one result before the behavior change, and they return a different result after the behavior change.

Examples of behavior changes:

  • dbt begins raising a validation error that it didn't previously.
  • dbt changes the signature of a built-in macro. Your project has a custom reimplementation of that macro. This could lead to errors, because your custom reimplementation will be passed arguments it cannot accept.
  • A dbt adapter renames or removes a method that was previously available on the {{ adapter }} object in the dbt-Jinja context.
  • dbt makes a breaking change to contracted metadata artifacts by deleting a required field, changing the name or type of an existing field, or removing the default value of an existing field (README).
  • dbt removes one of the fields from structured logs.

The following are not behavior changes:

  • Fixing a bug where the previous behavior was defective, undesirable, or undocumented.
  • dbt begins raising a warning that it didn't previously.
  • dbt updates the language of human-friendly messages in log events.
  • dbt makes a non-breaking change to contracted metadata artifacts by adding a new field with a default, or deleting a field with a default (README).

The vast majority of changes are not behavior changes. Because introducing these changes does not require any action on the part of users, they are included in continuous releases of dbt Cloud and patch releases of dbt Core.

By contrast, behavior change migrations happen slowly, over the course of months, facilitated by behavior change flags. The flags are loosely coupled to the specific dbt runtime version. By setting flags, users have control over opting in (and later opting out) of these changes.

Behavior change flags​

These flags must be set in the flags dictionary in dbt_project.yml. They configure behaviors closely tied to project code, which means they should be defined in version control and modified through pull or merge requests, with the same testing and peer review.

The following example displays the current flags and their current default values in the latest dbt Cloud and dbt Core versions. To opt out of a specific behavior change, set the values of the flag to False in dbt_project.yml. You will continue to see warnings for legacy behaviors you’ve opted out of, until you either:

  • Resolve the issue (by switching the flag to True)
  • Silence the warnings using the warn_error_options.silence flag

Here's an example of the available behavior change flags with their default values:

dbt_project.yml
flags:
  require_explicit_package_overrides_for_builtin_materializations: False
  require_model_names_without_spaces: False
  source_freshness_run_project_hooks: False
  restrict_direct_pg_catalog_access: False
  require_yaml_configuration_for_mf_time_spines: False
  require_batched_execution_for_custom_microbatch_strategy: False

This table outlines which month of the "Latest" release track in dbt Cloud and which version of dbt Core contains the behavior change's introduction (disabled by default) or maturity (enabled by default).

Flagdbt Cloud "Latest": Introdbt Cloud "Latest": Maturitydbt Core: Introdbt Core: Maturity
require_explicit_package_overrides_for_builtin_materializations2024.042024.061.6.14, 1.7.141.8.0
require_resource_names_without_spaces2024.05TBD*1.8.01.10.0
source_freshness_run_project_hooks2024.03TBD*1.8.01.10.0
[Redshift] restrict_direct_pg_catalog_access2024.09TBD*dbt-redshift v1.9.01.9.0
skip_nodes_if_on_run_start_fails2024.10TBD*1.9.0TBD*
state_modified_compare_more_unrendered_values2024.10TBD*1.9.0TBD*
require_yaml_configuration_for_mf_time_spines2024.10TBD*1.9.0TBD*
require_batched_execution_for_custom_microbatch_strategy2024.11TBD*1.9.0TBD*
cumulative_type_params2024.11TBD*1.9.0TBD*

When the dbt Cloud Maturity is "TBD," it means we have not yet determined the exact date when these flags' default values will change. Affected users will see deprecation warnings in the meantime, and they will receive emails providing advance warning ahead of the maturity date. In the meantime, if you are seeing a deprecation warning, you can either:

  • Migrate your project to support the new behavior, and then set the flag to True to stop seeing the warnings.
  • Set the flag to False. You will continue to see warnings, and you will retain the legacy behavior even after the maturity date (when the default value changes).

Failures in on-run-start hooks​

The flag is False by default.

Set the skip_nodes_if_on_run_start_fails flag to True to skip all selected resources from running if there is a failure on an on-run-start hook.

Source definitions for state:modified​

info

You need to build the state directory using dbt v1.9 or higher, or the dbt Cloud "Latest" release track, and you need to set state_modified_compare_more_unrendered_values to true within your dbt_project.yml.

If the state directory was built with an older dbt version or if the state_modified_compare_more_unrendered_values behavior change flag was either not set or set to false, you need to rebuild the state directory to avoid false positives during state comparison with state:modified.

The flag is False by default.

Set state_modified_compare_more_unrendered_values to True to reduce false positives during state:modified checks (especially when configs differ by target environment like prod vs. dev).

Setting the flag to True changes the state:modified comparison from using rendered values to unrendered values instead. It accomplishes this by persisting unrendered_config during model parsing and unrendered_database and unrendered_schema configs during source parsing.

Package override for built-in materialization​

Setting the require_explicit_package_overrides_for_builtin_materializations flag to True prevents this automatic override.

We have deprecated the behavior where installed packages could override built-in materializations without your explicit opt-in. When this flag is set to True, a materialization defined in a package that matches the name of a built-in materialization will no longer be included in the search and resolution order. Unlike macros, materializations don't use the search_order defined in the project dispatch config.

The built-in materializations are 'view', 'table', 'incremental', 'materialized_view' for models as well as 'test', 'unit', 'snapshot', 'seed', and 'clone'.

You can still explicitly override built-in materializations, in favor of a materialization defined in a package, by reimplementing the built-in materialization in your root project and wrapping the package implementation.

macros/materialization_view.sql
{% materialization view, snowflake %}
  {{ return(my_installed_package_name.materialization_view_snowflake()) }}
{% endmaterialization %}

In the future, we may extend the project-level dispatch configuration to support a list of authorized packages for overriding built-in materialization.

No spaces in resource names​

The require_resource_names_without_spaces flag enforces using resource names without spaces.

The names of dbt resources (models, sources, etc) should contain letters, numbers, and underscores. We highly discourage the use of other characters, especially spaces. To that end, we have deprecated support for spaces in resource names. When the require_resource_names_without_spaces flag is set to True, dbt will raise an exception (instead of a deprecation warning) if it detects a space in a resource name.

models/model name with spaces.sql
-- This model file should be renamed to model_name_with_underscores.sql

Project hooks with source freshness​

Set the source_freshness_run_project_hooks flag to True to include "project hooks" (on-run-start / on-run-end) in the dbt source freshness command execution.

If you have a specific project on-run-start / on-run-end hooks that should not run before/after source freshness command, you can add a conditional check to those hooks:

dbt_project.yml
on-run-start:
  - '{{ ... if flags.WHICH != 'freshness' }}'

MetricFlow time spine YAML​

The require_yaml_configuration_for_mf_time_spines flag is set to False by default.

In previous versions (dbt Core 1.8 and earlier), the MetricFlow time spine configuration was stored in a metricflow_time_spine.sql file.

When the flag is set to True, dbt will continue to support the SQL file configuration. When the flag is set to False, dbt will raise a deprecation warning if it detects a MetricFlow time spine configured in a SQL file.

The MetricFlow YAML file should have the time_spine: field. Refer to MetricFlow timespine for more details.

Custom microbatch strategy​

The require_batched_execution_for_custom_microbatch_strategy flag is set to False by default and is only relevant if you already have a custom microbatch macro in your project. If you don't have a custom microbatch macro, you don't need to set this flag as dbt will handle microbatching automatically for any model using the microbatch strategy.

Set the flag is set to True if you have a custom microbatch macro set up in your project. When the flag is set to True, dbt will execute the custom microbatch strategy in batches.

If you have a custom microbatch macro and the flag is left as False, dbt will issue a deprecation warning.

Previously, users needed to set the DBT_EXPERIMENTAL_MICROBATCH environment variable to True to prevent unintended interactions with existing custom incremental strategies. But this is no longer necessary, as setting DBT_EXPERMINENTAL_MICROBATCH will no longer have an effect on runtime functionality.

Cumulative metrics​

Cumulative-type metrics are nested under the cumulative_type_params field in the dbt Cloud "Latest" release track, dbt Core v1.9 and newer. Currently, dbt will warn users if they have cumulative metrics improperly nested. To enforce the new format (resulting in an error instead of a warning), set the require_nested_cumulative_type_params to True.

Use the following metric configured with the syntax before v1.9 as an example:


    type: cumulative
    type_params:
      measure: order_count
      window: 7 days

If you run dbt parse with that syntax on Core v1.9 or the dbt Cloud "Latest" release track, you will receive a warning like:


15:36:22  [WARNING]: Cumulative fields `type_params.window` and
`type_params.grain_to_date` has been moved and will soon be deprecated. Please
nest those values under `type_params.cumulative_type_params.window` and
`type_params.cumulative_type_params.grain_to_date`. See documentation on
behavior changes:
https://docs.getdbt.com/reference/global-configs/behavior-changes.

If you set require_nested_cumulative_type_params to True and re-run dbt parse you will now receive an error like:


21:39:18  Cumulative fields `type_params.window` and `type_params.grain_to_date` should be nested under `type_params.cumulative_type_params.window` and `type_params.cumulative_type_params.grain_to_date`. Invalid metrics: orders_last_7_days. See documentation on behavior changes: https://docs.getdbt.com/reference/global-configs/behavior-changes.

Once the metric is updated, it will work as expected:


    type: cumulative
    type_params:
      measure:
        name: order_count
      cumulative_type_params:
        window: 7 days

0
Edit this page