Core Concepts

Understanding the rescile Processing Model

To effectively use rescile, it’s essential to understand its core processing model. rescile works by applying a series of declarative rules in a specific order, iterating through the resources in your graph at each step. This document explains this fundamental “mental model” to help you write powerful and predictable configurations.

1. The Building Blocks: Resources and Relationships

At its heart, rescile builds a directed property graph. This graph consists of two simple but powerful components:

  • Resources (or Nodes): A resource represents an entity in your environment, like an application, a server, or a firewall_rule. Every resource has:

    • A type (e.g., "server"), which is like its label.
    • A name, (e.g.m "billing-api_server"), which acts as its unique primary key within that type.
    • An optional set of properties (key-value pairs) that describe it.

  • Relationships (or Edges): A relationship is a directed link from one resource to another. Every relationship has:

    • A type (e.g., "HOSTS"), which describes the nature of the connection.
    • A direction (from a source resource to a target resource).
    • An optional set of properties.

Your rescile configuration is a set of declarative rules that tell rescile how to create and connect these resources and relationships to form a complete model of your environment.

2. The Core Iteration Pattern

The most important concept in rescile is its iteration pattern. For every Model, Compliance, and Output file you create, rescile performs an implicit loop based on your configuration.

The Golden Rule: A configuration file with origin_resource = "X" instructs rescile to iterate through every resource of type “X” currently in the graph. For each of these resources, rescile will then evaluate every rule block ([[create_resource]], [[control.target]], etc.) defined within that file.

You don’t write the loops; you just declare the rules. rescile handles the iteration automatically.

Consider a simple model file data/models/server.toml:

# data/models/server.toml
origin_resource = "application"

# Rule Block 1
[[create_resource]]
resource_type = "server"
relation_type = "HOSTED_ON"
name = "{{ origin_resource.name }}_server"
# ...

# Rule Block 2
[[create_resource]]
match_on = [{ property = "environment", value = "prod" }]
resource_type = "monitoring_agent"
name = "agent_for_{{ origin_resource.name }}"
# ...

This file contains one [[create_resource]] block, which defines three key instructions:

  • resource_type = "server": For each application, create a new resource of type server.
  • name = "{{ ... }}_server": Set the primary key for the new server. The combination of resource_type and name uniquely identifies a resource in the graph.
  • relation_type = "HOSTED_ON": Create a directed relationship (application) -[HOSTED_ON]-> (server) connecting the origin resource to the one just created.

The processing logic for this file can be visualized in pseudo-code:

for each "application" in the graph:
  // Evaluate Rule Block 1
  create a "server" resource for this application

  // Evaluate Rule Block 2
  if this application has environment == "prod":
    create a "monitoring_agent" for this application

This declarative, iterative approach is the foundation for all of rescile’s engines.

If origin_resource is not defined in a model file, the iteration pattern is bypassed. The file is processed exactly once, and its rules are evaluated in a global context. This is useful for creating singleton resources or resources based on static data defined within the model file itself. In this mode, the origin_resource variable is not available in templates.

3. Data Sources for Templating

rescile provides a rich context of data that you can access within your templates ({{ ... }}). This allows for highly dynamic and data-driven configurations. The data is available from three primary sources:

a. The origin_resource

As explained in the core iteration pattern, the origin_resource variable is available and refers to the specific resource being processed in the loop. This is the most common data source.

origin_resource = "application"

[[create_resource]]
# Accesses a property from the current 'application' resource
name = "server_for_{{ origin_resource.name }}"

b. Inline TOML Data

You can define static data directly in the header of your model, compliance, or output files. Any top-level key that is not a reserved rescile directive (like origin_resource or [[create_resource]]) is automatically made available to all templates within that file.

This is ideal for defining constants, configuration maps, or environment-specific values that you want to reuse across multiple rules.

# data/models/server.toml
origin_resource = "application"

# This is a custom TOML table, not a rescile directive.
# It becomes available in templates as 'server_specs'.
server_specs = { standard_cpu = 4, premium_cpu = 8 }

[[create_resource]]
resource_type = "server"
name = "server_{{ origin_resource.name }}"
[create_resource.properties]
# Use the inline data in a template
cpu_cores = "{% if origin_resource.environment == 'prod' %}{{ server_specs.premium_cpu }}{% else %}{{ server_specs.standard_cpu }}{% endif %}"

c. External JSON Files

For larger or more complex datasets, you can load data from external JSON files using the special json! directive. This keeps your configuration files clean and allows you to manage data separately. The path to the JSON file is relative to the data directory.

# data/models/network.toml
origin_resource = "server"

# Load data from 'data/shared/ip_ranges.json'
# The data becomes available in templates as 'ip_ranges'.
ip_ranges = { "json!" = "shared/ip_ranges.json" }

[[create_resource]]
# ...
[create_resource.properties]
subnet = "{{ ip_ranges.subnets[origin_resource.region] }}"

By combining these data sources, you can create flexible and powerful rules that adapt to both the dynamic state of your graph and static configuration data.

4. The Order of Operations

rescile processes your configuration files in a strict, multi-phase sequence. Understanding this order is crucial for predicting how the graph will be built.

flowchart TD A[ 1. Asset Loading] --> B B[ 2. Model Application] --> C C[ 3. Compliance Application] --> D D[ 4. Output Generation] --> E[ Final Graph] subgraph A [Phase 1] direction LR A1[Assets] end subgraph B [Phase 2] direction LR B1[Models] end subgraph C [Phase 3] direction LR C1[Compliance] end subgraph D [Phase 4] direction LR D1[Outputs] end A1 --> B1 --> C1 --> D1

  1. Asset Loading (data/assets/*.csv): All CSV files are read first. This phase populates the graph with the initial set of resources and creates the first simple relationships based on foreign key conventions.

  2. Model Application (data/models/*.toml): Model files are processed to build out the architectural graph. rescile automatically detects dependencies between models (e.g., if model_B.toml reads resources created by model_A.toml) and executes them in the correct order. The core iteration pattern is applied for each model file.

  3. Compliance Application (data/compliance/*.toml): After the complete architectural graph is built, compliance files are processed. They also use the iteration pattern to find resources and relationships, mutating the graph to enforce governance.

  4. Output Generation (data/output/*.toml): Finally, output files are processed. They query the final, governed graph to generate structured data artifacts.

5. Idempotency and Merging

rescile’s processing is idempotent. This means that if you define a rule to create a resource that already exists (identified by its type and primary key [name]), rescile will not create a duplicate or throw an error.

Instead, it intelligently enriches the existing resource by adding or updating its properties.

The exact behavior for handling property conflicts depends on the processing phase in which the rule is applied.

Merging in the Model Phase

During Model Application, when multiple [[create_resource]] rules operate on the same resource (i.e., they have the same resource_type and rendered name), the default behavior is to overwrite properties.

  • If a rule defines a property that doesn’t exist on the resource, it is added.
  • If a rule defines a property that does exist, the new value replaces the old one.
  • To help debug unintended changes, rescile will issue a WARN log message whenever an overwrite occurs.

This overwrite behavior is fundamental to building up a resource’s state across multiple models. For example, server.toml might create a server with a “provisioning” status, and a later network.toml model can update that same server’s status to “active” while adding network-specific properties.

# In server.toml
[create_resource.properties]
status = "provisioning"

# In network.toml, operating on the same server resource
[create_resource.properties]
# This will replace any existing 'status' property.
status = "active"
ip_address = "10.0.0.1"

Merging in the Compliance Phase

During Compliance Application, when [[control]] rules add properties to resources or relationships, those properties are always aggregated (merged). This is designed to accumulate controls and requirements from multiple sources without overwrites.

The aggregation logic is as follows:

  • If the existing property is not an array, it is converted into an array containing just that value.
  • The new value (or values, if it is an array) is added to this array.
  • Duplicate values in the final array are removed.
  • If the final array contains only one item, it is stored as a single scalar value rather than an array.

For example, if two different compliance files add control metadata to the same database connection:

  • iso27001.toml adds a controls property with [{ "control_id": "A.5.14" }].
  • dora.toml adds a controls property with [{ "control_id": "dora-rmf-std-encryption" }].

The final relation will have a controls property containing an array with both control objects, providing a complete view of all governance applied to that connection.

6. Automatic Relationship Creation

In addition to the explicit rules you define, rescile uses a powerful convention-over-configuration mechanism to automatically create relationships. This “auto-relation” phase runs after each major processing phase (Assets, Models, and Compliance), creating a cohesive graph without verbose configuration.

The rule is simple: rescile scans all resources for properties where the property name matches the type (or label) of another known resource in the graph. When a match is found, it creates a relationship.

Example:

  1. A model file, server.toml, creates resources of type server. One such resource has the name billing-api_server.

  2. An asset file, host.csv, defines physical hosts. It contains a server column.

    # data/assets/host.csv
name,ip_address,server
host-01,10.1.1.50,billing-api_server

During processing, rescile sees that the host-01 resource has a property named server. Since server is a known resource type (created by the model), rescile automatically creates a relationship: (host:host-01) -[server]-> (server:billing-api_server).

This feature significantly reduces the amount of configuration needed to wire your graph together. Because it runs after each phase, resources created by models can be automatically linked to by later models or by compliance rules, and so on.

Preventing Automatic Linking

Sometimes, a property name may coincidentally match a resource type, but you don’t intend for a relationship to be created. You can prevent this automatic behavior by prefixing the property name with an underscore (_).

rescile will create the property on the resource without the leading underscore but will exclude it from the auto-linking process.

Example in a CSV Asset:

# data/assets/application.csv
name,_comment # This will not create a link to a 'comment' resource
billing-api,"This is just a note."

The billing-api resource will have a property comment with the value "This is just a note.", but no relationship will be formed.

Example in a Model:

# data/models/server.toml
origin_resource = "application"

[[create_resource]]
resource_type = "server"
name = "{{ origin_resource.name }}_server"
[create_resource.properties]
# This property will be named 'service', but will not be auto-linked
_service = "{{ origin_resource.service_name }}"

This simple convention gives you full control over when and where automatic relationships are created, blending the power of convention with the precision of explicit configuration.

7. The origin_resource Context in Templates

Whenever you use Tera templating (e.g., {{ ... }}), the data is always evaluated in the context of the specific resource being processed in the main iteration loop.

The special origin_resource variable gives you access to the properties of that resource. This allows you to create dynamic values using expressions, filters, and conditional logic.

# In a model with origin_resource = "application"
[[create_resource]]
resource_type = "server"
name = "{{ origin_resource.name }}_server"
[create_resource.properties]
# Access a property from the resource being processed and apply a filter
hostname = "{{ origin_resource.name | upper }}" # Renders as "BILLING-API"
# Use conditional logic
tier = "{% if origin_resource.environment == 'prod' %}premium{% else %}standard{% endif %}"

When rescile is processing the application named billing-api, the template {{ origin_resource.name }} will render as billing-api. When it processes the next application, frontend-app, the same template will render as frontend-app.

This ensures that the rules you define are applied correctly and uniquely for every resource they operate on.

8. Conditional Logic and Traversal with match_on

The match_on directive allows you to apply rules selectively, filtering which resources are processed. The most common use is a simple property check:

# Only applies if the resource has a property `environment` with the value `prod`.
match_on = [{ property = "environment", value = "prod" }]

A more powerful feature is the ability to match on properties of related resources using dot notation.

# Applies if the resource is linked to a `domain` resource whose `owner` is `ops-team`.
match_on = [{ property = "domain.owner", value = "ops-team" }]

When rescile evaluates this rule, it traverses from the current origin_resource along the domain relationship to the connected domain resource(s). It then checks the owner property on those resources. If the domain property on the origin resource contains an array of related domains, the condition is considered true if any one of them matches the criteria.

This traversal capability is essential for creating context-aware rules that depend on the wider state of your graph, not just the properties of a single resource.

9. A Concrete Walkthrough

Let’s trace the process with a more detailed example that uses relationship traversal.

Phase 1: Assets

  • data/assets/application.csv

    name,environment,domain
billing-api,prod,example.com
auth-service,dev,example.com

    Result: Two application resources are created.

  • data/assets/domain.csv

    name,owner
example.com,ops-team

    Result: One domain resource is created. rescile’s auto-relation feature sees that the domain column in application.csv matches the domain resource type and automatically creates relationships: (application) -> (domain).

Phase 2: Models

  • data/models/server.toml 
    origin_resource = "application"

[[create_resource]]
resource_type = "server"
relation_type = "HOSTS"
name = "{{ origin_resource.name }}_server"
[create_resource.properties]
environment = "{{ origin_resource.environment }}"

[[create_resource]]
# This rule now checks a property on the related domain.
match_on = [{ property = "domain.owner", value = "ops-team" }]
resource_type = "firewall_rule"
relation_type = "NEEDS"
name = "fw_rule_for_{{ origin_resource.name }}"

Processing Walkthrough:

  1. rescile begins processing server.toml.
  2. It fetches the first application: billing-api. This application is linked to the domain named example.com, which has owner: "ops-team".
  3. Iteration 1: billing-api
    • It evaluates the first [[create_resource]] block.
      • A server resource named billing-api_server is created with environment: "prod".
      • A HOSTS relationship is created: (application:billing-api) -> (server:billing-api_server).
    • It evaluates the second [[create_resource]] block.
      • It evaluates the match_on condition domain.owner == "ops-team".
      • rescile traverses from billing-api to the example.com domain and checks its owner property. The value is ops-team, so the condition is true.
      • A firewall_rule resource named fw_rule_for_billing-api is created.
      • A NEEDS relationship is created: (application:billing-api) -> (firewall_rule:fw_rule_for_billing-api).
  4. It fetches the second application: auth-service. This application is also linked to the example.com domain.
  5. Iteration 2: auth-service
    • It evaluates the first [[create_resource]] block.
      • A server resource named auth-service_server is created with environment: "dev".
      • A HOSTS relationship is created: (application:auth-service) -> (server:auth-service_server).
    • It evaluates the second [[create_resource]] block.
      • The match_on condition domain.owner == "ops-team" is checked again. The traversal is performed, and the condition is true.
      • A firewall_rule resource named fw_rule_for_auth-service is created.

By understanding this simple, powerful iteration model, you can build complex and accurate digital twins of your environment with confidence.