deepsea-policy.cfg

NAME

deepsea policy.cfg − configuration file for assignments

DESCRIPTION

The policy.cfg is a configuration file containing four sections. These sections are only descriptive. These are

Cluster Assignment
Role Assignment
Profile Assignment
Common configuration

The contents of all sections is the same: a relative pathname to YAML files ending with sls or yml. Globbing is optional, but normally encouraged to simplify the configuration. The base pathname is the location of the policy.cfg which is /srv/pillar/ceph/proposals. None of the files under this directory are directly used by Salt. This area is similar to a scratchpad where DeepSea creates many YAML fragments to fill in the blanks and allow the administrator to pick which parts to include and shape their Ceph cluster.

The strategy behind this approach allows minions with single or multiple roles to be managed with the same effort. No direct modifying of YAML is necessary.

SECTIONS

Cluster Assignment
DeepSea currently only supports one Ceph cluster in a Salt cluster. The default name is ceph. In the future, this setting should become more useful.

Entries in the policy.cfg often look like:

cluster-ceph/cluster/*.sls

The *.sls will match all minions constrained by deepsea-minions (7) but each minion could be explicitly listed.

Role Assignment
In a Ceph cluster, a minion can perform one or more roles. The roles are master, admin, mon, storage, mgr, mds, igw, rgw, ganesha and openattic. All must be defined explicitly except storage. Storage is addressed in the Profile section.

The roles have the following meanings:

Note that the admin role has no function. The role is mainly a convenience for specifying which minions that the admininistrator wishes to have the admin keyring. All other roles use their own keyrings and do not require the admin keyring.

Also note the conflated meanings between the Salt master and the general concept of the Ceph admin node. Both are centralized. In DeepSea, these are one in the same but the hostname is often called admin.

Entries in the policy.cfg follow this convention:

role-NAME/cluster/MINION.sls

where NAME is one of the roles above and MINION is normally the fqdn of the host.

Profile Assignment
This section defines how drives are configured for Ceph. This is the most complex part of the configuration. By default, profile-default will contain an arrangement of devices in a ratio of 1:5 (e.g one SSD for five HDDs) or failing that, independent OSDs. The default format is bluestore. For generating other configurations, see the utility known as a Salt runner salt-run proposal.help.

In the configuration file itself, two pathnames are included. The default entries are:

profile-default/cluster/*.sls
profile-default/stack/default/ceph/minions/*.yml

The second path references each disk configuration for all the minions that had available hardware. Each file contains unique pathnames in a data structure suitable for creating an OSD.

Depending on the site and the administrator preferences, these lines can be removed and replaced with a custom configuration. Another option is multiple profiles. One group of storage nodes can be assigned a particular type of hardware profile such as profile-archive while another group with different characteristics is assigned profile-applications. Beware that this may require editing the CRUSH map to achieve the desired results.

Although partial configurations can be created across different profiles for the same minion, this may result in heartarche. The most important point to remember is to have only one set of devices for a minion in a profile. If the policy.cfg includes the same devices for one minion from multiple profiles, failures will ensue.

For multiple profiles, removing the globbing and specifying which minions use which profile is often simpler.

Common Configuration
This section includes both a global.yml and cluster.yml. With a single cluster these are somewhat redundant. The entries are

config/stack/default/global.yml
config/stack/default/ceph/cluster.yml

The global.yml contains references to the time configuration and the cluster.yml contains the network settings, available roles and the Ceph fsid. These two lines are included unmodified.

EXAMPLES

The simplest cluster contains five nodes for minimal fault tolerance.

## Cluster Assignment
cluster-ceph/cluster/admin.ceph.sls
cluster-ceph/cluster/node[1-4].ceph.sls
## Role Assignment
role-master/cluster/admin.ceph.sls
role-mon/cluster/node[1-3].ceph.sls
role-mgr/cluster/node[1-3].ceph.sls
## Profile Assignment
profile-default/cluster/node[1-4].ceph.sls
profile-default/stack/default/ceph/minions/node[1-4].ceph.yml
## Common
config/stack/default/global.yml
config/stack/default/ceph/cluster.yml

With no gateways, this may also be considered the least accessible. This same example can be altered.

## Cluster Assignment
cluster-ceph/cluster/*.sls
## Role Assignment
role-master/cluster/admin*.sls
role-mon/cluster/node[1-3]*.sls
role-mgr/cluster/node[1-3]*.sls
## Profile Assignment
profile-default/cluster/*.sls
profile-default/stack/default/ceph/minions/*.yml
## Common
config/stack/default/global.yml
config/stack/default/ceph/cluster.yml

For five servers, these will give the same result. The difference is that the second configuration is more generic on the addition of storage nodes. Adding servers to the Salt cluster named node4 or foo23 will not require a configuration change assuming that the default profile is suitable for the new hardware. Running stages will be sufficient. Whether this is desirable is left to the preference of the administrator.

Note the removal of the domain names for the role assignment. Most examples contain the hostname followed by ’*’ since some domains names can be lengthy.

FILTERS

Two optional modifiers can be appended to any line, but are rarely necessary. These are slice and re. The slice filter will operate on an index and can be useful in virtual environments where hostnames are not known. For example,

role-mon/cluster/*.sls slice=[0:3]

will assign the first three nodes the monitor role.

The re filter will apply a regular expression. For example,

profile-default/cluster/*.sls re=.*[^7]*.sls
profile-default/stack/default/ceph/minions/*.yml re=.*[^7]*.yml

will exclude a host named node7.

Globbing and multiple entires is encouraged and typically easier to maintain. These filters remain for those that need them.

AUTHOR

Eric Jackson <ejackson@suse.com>

SEE ALSO

/usr/share/doc/packages/deepsea/examples, deepsea-stages(7)