A Policyfile is a way to create immutable collections of cookbooks, cookbook dependencies, and attributes defined in a single document that is uploaded to the Chef Infra Server. The Policyfile is then associated with a group of nodes. When these nodes perform a Chef Infra Client run, they utilize recipes specified in the Policyfile.

Why Policyfiles? 

Policyfiles make it easier to test and promote code safely with a simpler interface. Using a Policyfile improves the user experience and resolves real-world problems that some workflows built around Chef Infra must deal with. The following sections discuss in more detail some of the good reasons to use Policyfile, including:

• Focus the workflow on the entire system
• Safer development workflows
• Less expensive computation
• Code visibility
• Role mutability
• Cookbook mutability
• Replaces Berkshelf and the environment cookbook pattern

Focused System Workflows 

The knife command line tool maps closely to the Chef Infra Server API and the objects defined by it: roles, environments, run-lists, cookbooks, data bags, nodes, and so on. Chef Infra Client assembles these pieces at run-time and configures a host to do useful work.

Policyfile focuses that workflow onto the entire system, rather than the individual components. For example, Policyfile describes whole systems, whereas each individual revision of the Policyfile.lock.json file uploaded to the Chef Infra Server describes a part of that system, inclusive of roles, environments, cookbooks, and the other Chef Infra Server objects necessary to configure that part of the system.

Safer Workflows 

Policyfile encourages safer workflows by making it easier to publish development versions of cookbooks to the Chef Infra Server without the risk of mutating the production versions and without requiring a complicated versioning scheme to work around cookbook mutability issues. Roles are mutable and those changes are applied only to the nodes specified by the policy. Policyfile does not require any changes to your normal workflows. Use the same repositories you are already using, the same cookbooks, and workflows. Policyfile will prevent an updated cookbook or role from being applied immediately to all machines.

Code Visibility 

When running Chef Infra without a Policyfile, the exact set of cookbooks that are applied to a node is defined by:

• The node’s run_list property
• Any roles that are present in the node’s run-list or recursively included by those roles
• The environment, which may restrict the set of valid cookbook versions for a node based on a variety of constraint operators
• Dependencies, as defined by each cookbook’s metadata
• Dependency resolution picks the “best” set of cookbooks that meet dependency and environment criteria

These conditions are re-evaluated every time Chef Infra Client runs, which can make it harder to know which cookbooks will be run by Chef Infra Client or what the effects of updating a role or uploading a new cookbook will be.

Policyfile simplifies this behavior by computing the cookbook set on the workstation, and then producing a readable document of that solution: a Policyfile.lock.json file. This pre-computed file is uploaded to the Chef Infra Server, and is then used in each Chef Infra Client run that is managed by that particular policy name and policy group.

Less Expensive Computation 

When running Chef Infra without Policyfile, the Chef Infra Server loads dependency data for all known versions of all known cookbooks, and then runs an expensive computation to determine the correct set.

Policyfile moves this computation to the workstation, where it is done less frequently.

The Chef Infra Server now does no expensive calculation. It just sends to the node whatever set of Policyfile controlled cookbook versions are needed, based on the Policyfile version currently in place for that node.

Role and Environment Mutability 

When running Chef Infra without Policyfile, roles and environments are global objects. Changes to roles and environments are applied immediately to any node that contains that role in its run-list or belong to a particular environment. This can make it hard to update roles and environments and in some use cases discourages using them at all.

Policyfile effectively replaces roles and environments. Policyfile files are versioned automatically and new versions are applied to systems only when promoted.

Cookbook Mutability 

When running Chef without Policyfile, existing versions of cookbooks are mutable. This is convenient for many use cases, especially if users upload in-development cookbook revisions to the Chef Infra Server. But this sometimes creates issues that are similar to role mutability by allowing those cookbook changes to be applied immediately to nodes that use that cookbook. Users account for this by rigorous testing processes to ensure that only fully integrated cookbooks are ever published. This process does enforce good development habits, but at the same time it should not be a required part of a workflow that ends with publishing an in-development cookbook to the Chef Infra Server for testing against real nodes Policyfile solves this issue by using a cookbook publishing API for the Chef Infra Server that does not provide cookbook mutability. Name collisions are prevented by storing cookbooks by name and an opaque identifier that is computed from the content of the cookbook itself.

For example, name/version collisions can occur when users temporarily fork an upstream cookbook. Even if the user contributes their change and the maintainer is responsive, there may be a period of time where the user needs their fork to make progress. This situation presents a versioning dilemma: if the user does not update their own version, they must overwrite the existing copy of that cookbook on the Chef Infra Server, wheres if they do update the version number it might conflict with the version number of the future release of that upstream cookbook.

Opaque IDs 

The opaque identifier that is computed from the content of a cookbook is the only place where an opaque identifier is necessary. When working with Policyfile, be sure to:

• Use the same names and version constraints as normal in the Policyfile.rb file
• Use the same references to cookbooks pulled from Chef Supermarket
• Use the same branch, tag, and revision patterns for cookbooks pulled from git
• Use the same paths for cookbooks pulled from disk

Extra metadata about the cookbook is stored and included in Chef Infra Server API responses and in the Policyfile.lock.json file, including the source of a cookbook (Chef Supermarket, git, local disk, etc.), as well as any upstream identifiers, such as git revisions. For cookbooks that are loaded from the local disk that are in a git repo, the upstream URL, current revision ID, and the state of the repo are stored also.

The opaque identifier is mostly behind the scenes and is only visible once published to the Chef Infra Server. Cookbooks that are uploaded to the Chef Infra Server may have extended version numbers such as 1.0.0-dev.

Environment Cookbooks 

Policyfile replaces the environment cookbook pattern that is often required by Berkshelf, along with a dependency solver and fetcher. That said, Policyfile does not replace all Berkshelf scenarios.

 

Knife Commands 

The following knife commands used to set the policy group and policy name on the Chef Infra Server. For example:

knife node policy set test-node 'test-policy-group-name' 'test-policy-name'

Policyfile.rb 

A Policyfile file allows you to specify in a single document the cookbook revisions and recipes that Chef Infra Client will apply. A Policyfile file is uploaded to the Chef Infra Server, where it is associated with a group of nodes. When these nodes are configured during a Chef Infra Client run, Chef Infra Client will make decisions based on your Policyfile settings and will build a run-list based on that information. A Policyfile file may be versioned, and then promoted through deployment stages to safely and reliably deploy new configuration.

Syntax 

A Policyfile.rb is a Ruby file in which run-list and cookbook locations are specified. The syntax is as follows:

name "name"
run_list "ITEM", "ITEM", ...
default_source :SOURCE_TYPE, *args
cookbook "NAME" [, "VERSION_CONSTRAINT"] [, SOURCE_OPTIONS]

Settings 

A Policyfile.rb file may contain the following settings:

name "name"

Required. The name of the policy. Use a name that reflects the purpose of the machines against which the policy will run.

run_list "ITEM", "ITEM", ...

Required. The run-list Chef Infra Client will use to apply the policy to one (or more) nodes.

default_source :SOURCE_TYPE, *args

The location in which any cookbooks not specified by cookbook are located. Possible values: chef_repo, chef_server, :supermarket, and :artifactory. Use more than one default_source to specify more than one location for cookbooks.

default_source :supermarket pulls cookbooks from the public Chef Supermarket.

default_source :supermarket, "https://mysupermarket.example"" pulls cookbooks from a named private Chef Supermarket.

default_source :chef_server, "https://chef-server.example/organizations/example"" pulls cookbooks from the Chef Infra Server.

default_source :chef_repo, "path/to/repo" pulls cookbooks from a monolithic cookbook repository. This may be a path to the top-level of a cookbook repository or to the /cookbooks directory within that repository.

default_source :artifactory, "https://artifactory.example/api/chef/my-supermarket"" pulls cookbooks from an Artifactory server. Requires either artifactory_api_key to be set in config.rb or ARTIFACTORY_API_KEY to be set in your environment.

Multiple cookbook sources may be specified. For example from the public Chef Supermarket and a monolithic repository:

default_source :supermarket
default_source :chef_repo, 'path/to/repo'

or from both a public and private Chef Supermarket:

default_source :supermarket
default_source :supermarket, 'https://supermarket.example';

Note

If a run-list or any dependencies require a cookbook that is present in more than one source, be explicit about which source is preferred. This will ensure that a cookbook is always pulled from an expected source. For example, an internally-developed cookbook named chef-client will conflict with the public chef-client cookbook that is maintained by Chef. To specify a named source for a cookbook:

default_source :supermarket
default_source :supermarket, 'https://supermarket.example'; do |s|
  s.preferred_for 'chef-client'
end

List multiple cookbooks on the same line:

default_source :supermarket
default_source :supermarket, 'https://supermarket.example'; do |s|
  s.preferred_for 'chef-client', 'nginx', 'mysql'
end

cookbook "NAME" [, "VERSION_CONSTRAINT"] [, SOURCE_OPTIONS]

Add cookbooks to the policy, specify a version constraint, or specify an alternate source location, such as Chef Supermarket. For example, add a cookbook:

cookbook 'apache2'

Specify a version constraint:

run_list 'jenkins::master'

# Restrict the jenkins cookbook to version 2.x, greater than 2.1
cookbook 'jenkins', '~> 2.1'

Specify an alternate source:

cookbook 'my_app', path: 'cookbooks/my_app'

or:

cookbook 'mysql', github: 'opscode-cookbooks/mysql', branch: 'master'

or:

cookbook 'chef-ingredient', git: 'https://github.com/chef-cookbooks/chef-ingredient.git';, tag: 'v0.12.0'

named_run_list "NAME", "ITEM1", "ITEM2", ...

Specify a named run-list to be used as an alternative to the override run-list. This setting should be used carefully and for specific use cases, like running a small set of recipes to quickly converge configuration for a single application on a host or for one-time setup tasks. For example:

named_run_list :update_app, 'my_app_cookbook::default'

include_policy "NAME", *args

Specify a policyfile lock to be merged with this policy. Chef Workstation supports pulling this lock from a local or remote file, from a Chef Infra Server, or from a git repository. When the policyfile lock is included, its run-list will appear before the current policyfile’s run-list. This setting requires that the solved cookbooks appear as-is from the included policyfile lock. If conflicting attributes or cookbooks are provided, an error will be presented. See RFC097 for the full specifications of this feature.

Pull the policyfile lock from ./NAME.lock.json:

include_policy 'NAME', path: '.'

Pull the policyfile lock from ./foo.lock.json.

include_policy 'NAME', path: './foo.lock.json'

Pull the policyfile lock foo.lock.json from the example/foo Git repository on the git.example.com Git server.

include_policy 'NAME', git: 'https://git.example.com/example/foo';, path: 'foo.lock.json'

Pull the policyfile lock from ./bar.lock.json with revision ID ‘revision1’.

include_policy 'NAME', policy_revision_id: 'revision1', path: './bar.lock.json'

Pull the policyfile lock from a remote server https://internal.example.com/foo.lock.json.

include_policy 'NAME', remote: 'https://internal.example.com/foo.lock.json';

Pull the policyfile lock from a remote server https://internal.example.com/bar.lock.json and with revision ID ‘revision1’.

include_policy 'NAME', policy_revision_id: 'revision1', remote: 'https://internal.example.com/foo.lock.json';

Pull the policy NAME with revision ID revision1 from the http://chef-server.example Chef Infra Server:

include_policy 'NAME', policy_revision_id: 'revision1', server: 'http://chef-server.example';

Pull the policy foo with revision ID revision1:

include_policy 'NAME', policy_name: 'foo', policy_revision_id: 'revision1', server: 'http://chef-server.example';

Pull and lock the current revision for policy foo in policy group prod:

include_policy 'NAME', policy_name: 'foo', policy_group: 'prod', server: 'http://chef-server.example';

Example 

For example:

name 'jenkins-master'
run_list 'java', 'jenkins::master', 'recipe[policyfile_demo]'
default_source :supermarket, 'https://mysupermarket.example';
cookbook 'policyfile_demo', path: 'cookbooks/policyfile_demo'
cookbook 'jenkins', '~> 8.2'
cookbook 'mysql', github: 'sous-chefs/mysql', branch: 'master'

client.rb Settings 

The following settings may be configured in the client.rb file for use with Policyfile:

named_run_list

The run-list associated with a policy file.

policy_group

The name of a policy group that exists on the Chef Infra Server. policy_name must also be specified.

policy_name

The name of a policy, as identified by the name setting in a Policyfile.rb file. policy_group must also be specified.

use_policyfile

Chef Infra Client automatically checks the configuration, node JSON, and the stored node on the Chef Infra Server to determine if Policyfile files are being used, and then automatically updates this flag. Default value: false.

knife bootstrap 

A node may be bootstrapped to use Policyfile files. Use the following options as part of the bootstrap command:

--policy-group POLICY_GROUP

The name of a policy group that exists on the Chef Infra Server.

--policy-name POLICY_NAME

The name of a policy, as identified by the name setting in a Policyfile.rb file.

For a customized bootstrap process, add policy_name and policy_group to the first-boot JSON file that is passed to Chef Infra Client.

knife search 

The policy_name and policy_group settings for a node are stored as searchable attributes and as such are available when using a fuzzy matching search pattern. For example: knife search dev will return nodes that are part of the dev policy group.

Test w/Kitchen 

Kitchen may be used to test Policyfile files. Add the following to kitchen.yml:

provisioner:
  name: chef_zero

A named run-list may be used for each suite:

suites:
  - name: client
    provisioner:
      named_run_list: test_client_recipe
  - name: server
    provisioner:
      named_run_list: test_server_recipe

or globally:

provisioner:
  name: chef_zero
  named_run_list: integration_test_run_list

or testing with policies for each suite, once the Policyfile files are available in your repo:

suites:
   - name: defaultmega
      provisioner:
         policyfile: policies/default.rb
   - name: defaultultra
      provisioner:
         policyfile: policies/defaulttwo.rb

Note

As chef_zero explicitly tests outside the context of a Chef Infra Server, the policy_groups concept is not applicable. The value of policy_group during a converge will be set to local.

To view the full documentation, please see the link in the Resolution section

See the Policyfile docs for full detail at https://docs.chef.io/policyfile/