deepsea commands


DeepSea uses different Salt commands. All commands are organized under orchestrations, states, modules and runners.

One difficulty with Salt is distinguishing between these types of commands. Running the wrong base command with the desired operational arguments results in confusing error messages. The separation below hopes to prevent some confusion.

Salt orchestrations run on the Salt master and typically perform multiple steps sequentially on different sets of minions. Some of these steps will run on multiple minions in parallel.

While these orchestrations are powerful, the files end with sls and can be easily confused with normal state files. Additionally, the state.orch is a salt-runner which only returns results when all steps have completed. The commands salt-run and salt-run state.event pretty=True can provide useful progress of an orchestration.

The list below is all orchestration commands.

salt-run state.orch ceph.stage.0
salt-run state.orch ceph.stage.prep
salt-run state.orch ceph.stage.prep.master
salt-run state.orch ceph.stage.prep.minion

salt-run state.orch ceph.stage.1
salt-run state.orch ceph.stage.discovery

salt-run state.orch ceph.stage.2
salt-run state.orch ceph.stage.configure

salt-run state.orch ceph.stage.3
salt-run state.orch ceph.stage.deploy

salt-run state.orch ceph.stage.4
salt-run state.orch
salt-run state.orch ceph.stage.cephfs
salt-run state.orch ceph.stage.ganesha
salt-run state.orch ceph.stage.iscsi
salt-run state.orch ceph.stage.openattic
salt-run state.orch ceph.stage.radosgw

salt-run state.orch ceph.stage.5
salt-run state.orch ceph.stage.removal

salt-run state.orch ceph.benchmarks.baseline
salt-run state.orch ceph.benchmarks.cephfs

salt-run state.orch ceph.diagnose.iperf3-concurrent
salt-run state.orch ceph.diagnose.iperf3
salt-run state.orch

salt-run state.orch ceph.maintenance.upgrade
salt-run state.orch ceph.maintenance.upgrade.master
salt-run state.orch ceph.maintenance.upgrade.minion

salt-run state.orch ceph.migrate.nodes
salt-run state.orch ceph.migrate.osds
salt-run state.orch ceph.migrate.policy
salt-run state.orch ceph.migrate.subvolume

salt-run state.orch ceph.purge

salt-run state.orch ceph.redeploy.nodes
salt-run state.orch ceph.redeploy.osds

salt-run state.orch ceph.restart.ganesha
salt-run state.orch ceph.restart.igw
salt-run state.orch ceph.restart.mds
salt-run state.orch ceph.restart.mgr
salt-run state.orch ceph.restart.mon
salt-run state.orch ceph.restart.osd
salt-run state.orch ceph.restart.rgw

Salt States are the essential building blocks for running a series of steps on a single minion. The state.apply is a Salt module that processes an sls file.

The states below can be run indepdently and are useful when debugging. Nearly all states are part of a larger orchestration which may have prerequisites of other states. Running these commands requires understanding what you are doing.

For guidance, the Salt targets are listed as pseudo hostnames. Some are intended to only run on the Salt master. These targets are master_minion. Others are expected to only run on minions assigned a specific role. These targets are labeled as role*. Running states against unintended minions will often result in confusing errors.

salt ’igw*’ state.apply ceph.igw.import
salt ’master_minion’ state.apply ceph.igw.auth
salt ’igw*’ state.apply ceph.igw.config
salt ’master_minion’ state.apply ceph.igw.key
salt ’igw*’ state.apply ceph.igw.keyring
salt ’igw*’ state.apply ceph.igw.restart
salt ’igw*’ state.apply ceph.igw.sysconfig
salt ’igw*’ state.apply ceph.igw

salt ’master_minion state.apply ceph.osd.auth
salt ’master_minion state.apply ceph.osd.key
salt ’storage*’ state.apply ceph.osd.keyring
salt ’storage*’ state.apply ceph.osd.restart
salt ’storage*’ state.apply ceph.osd.scheduler
salt ’storage*’ state.apply ceph.osd

salt ’storage*’ state.apply ceph.packages.common
salt ’*’ state.apply ceph.packages

salt ’master_minion state.apply ceph.pool

salt ’master_minion’ state.apply ceph.rgw.auth
salt ’rgw*’ state.apply ceph.rgw.buckets
salt ’rgw*’ state.apply ceph.rgw.key
salt ’rgw*’ state.apply ceph.rgw.keyring
salt ’rgw*’ state.apply ceph.rgw.restart
salt ’rgw*’ state.apply ceph.rgw.users
salt ’rgw*’ state.apply ceph.rgw

salt ’*’ state.apply ceph.time.ntp
salt ’*’ state.apply ceph.time

salt ’*’ state.apply ceph.updates.kernel
salt ’master_minion’ state.apply ceph.updates.master
salt ’*’ state.apply ceph.updates.regular
salt ’*’ state.apply ceph.updates.restart
salt ’*’ state.apply ceph.updates.salt
salt ’*’ state.apply ceph.updates

salt ’master_minion’ state.apply ceph.configuration.check
salt ’*’ state.apply ceph.configuration

salt ’master_minion state.apply ceph.wait.1hour.until.OK
salt ’master_minion state.apply ceph.wait.2hours.until.OK
salt ’master_minion state.apply ceph.wait.4hours.until.OK
salt ’master_minion state.apply ceph.wait.until.OK
salt ’master_minion state.apply ceph.wait

salt ’master_minion state.apply ceph.admin.key
salt ’*’ state.apply ceph.admin

salt ’master_minion state.apply ceph.ganesha.auth
salt ’master_minion state.apply ceph.ganesha.config
salt ’ganesha*’ state.apply ceph.ganesha.configure
salt ’ganesha*’ state.apply ceph.ganesha.install
salt ’master_minion state.apply ceph.ganesha.key
salt ’ganesha*’ state.apply ceph.ganesha.keyring
salt ’ganesha*’ state.apply ceph.ganesha.restart
salt ’ganesha*’ state.apply ceph.ganesha.service
salt ’ganesha*’ state.apply ceph.ganesha

salt ’master_minion state.apply ceph.maintenance.noout

salt ’master_minion state.apply ceph.mds.auth
salt ’master_minion state.apply ceph.mds.key
salt ’mds*’ state.apply ceph.mds.keyring
salt ’mds*’ state.apply ceph.mds.pools
salt ’mds*’ state.apply ceph.mds.restart
salt ’mds*’ state.apply ceph.mds

salt ’master_minion state.apply ceph.mgr.auth
salt ’master_minion state.apply ceph.mgr.key
salt ’mgr*’ state.apply ceph.mgr.keyring
salt ’mgr*’ state.apply ceph.mgr.restart
salt ’mgr*’ state.apply ceph.mgr

salt ’mgr*’ state.apply ceph.mines

salt ’master_minion’ state.apply ceph.mon.key
salt ’mon*’ state.apply ceph.mon.restart
salt ’mon*’ state.apply ceph.mon

salt ’master_minion’ state.apply ceph.monitoring

salt ’master_minion state.apply ceph.noout.set
salt ’master_minion state.apply ceph.noout.unset

salt ’master_minion state.apply ceph.openattic.auth
salt ’master_minion state.apply ceph.openattic.key
salt ’openattic*’ state.apply ceph.openattic.keyring
salt ’openattic*’ state.apply ceph.openattic.oaconfig
salt ’openattic*’ state.apply ceph.openattic

salt ’*’ state.apply ceph.processes

salt ’*’ state.apply ceph.refresh

salt ’master_minion state.apply ceph.remove.ganesha
salt ’master_minion state.apply ceph.remove.igw.auth
salt ’master_minion state.apply ceph.remove.mds
salt ’master_minion state.apply ceph.remove.mon
salt ’master_minion state.apply ceph.remove.openattic
salt ’master_minion state.apply ceph.remove.rgw
salt ’master_minion state.apply
salt ’master_minion state.apply

salt ’*’ state.apply ceph.repo

salt ’*’ state.apply ceph.rescind.admin
salt ’*’ state.apply ceph.rescind.client-cephfs
salt ’*’ state.apply ceph.rescind.client-iscsi
salt ’*’ state.apply ceph.rescind.client-nfs
salt ’*’ state.apply ceph.rescind.client-radosgw
salt ’*’ state.apply ceph.rescind.ganesha
salt ’*’ state.apply ceph.rescind.igw.keyring
salt ’*’ state.apply ceph.rescind.igw.lrbd
salt ’*’ state.apply ceph.rescind.igw.sysconfig
salt ’*’ state.apply ceph.rescind.igw
salt ’*’ state.apply ceph.rescind.master
salt ’*’ state.apply ceph.rescind.mds.keyring
salt ’*’ state.apply ceph.rescind.mds
salt ’*’ state.apply ceph.rescind.mgr.keyring
salt ’*’ state.apply ceph.rescind.mgr
salt ’*’ state.apply ceph.rescind.mon
salt ’*’ state.apply ceph.rescind.openattic.keyring
salt ’*’ state.apply ceph.rescind.openattic
salt ’*’ state.apply ceph.rescind.rgw.keyring
salt ’*’ state.apply ceph.rescind.rgw
salt ’*’ state.apply
salt ’*’ state.apply
salt ’*’ state.apply
salt ’*’ state.apply ceph.rescind

salt ’master_minion state.apply ceph.reset

salt ’master_minion state.apply ceph.salt-api

salt ’*’ state.apply ceph.sync

salt ’*’ state.apply ceph.upgrade

salt ’master_minion’ state.apply ceph.warning.noout

salt ’master_minion*’ state.apply ceph.warning

Salt execution modules have different uses. Some rely on the side effect of executing multiple steps that have complex conditionals not suitable for an sls file. Others perform queries and return structured output.

One distinction with modules is that these run on a minion. As a result, the scope of pillar data is restricted to the minion itself.

Like states, the Salt targets are listed as pseudo hostnames. The master_minion target is intended to only run on the Salt master. Targets labeled as role* are expected to only run on minions with that role.

salt ’*’ advise.reboot
salt ’*’ advise.generic

salt ’*’ cephdisks.list

salt ’master_minion’ cephimages.list

salt ’*’ cephinspector.get_minion_public_network
salt ’*’ cephinspector.get_minion_cluster_network
salt ’*’ cephinspector.get_ceph_disks_yml
salt ’*’ cephinspector.get_keyring
salt ’*’ cephinspector.inspect

salt ’*’ cephprocesses.check
salt ’*’ cephprocesses.wait

salt ’*’ fs.btrfs_get_mountpoints_of_subvol subvol=’@/var/lib/ceph’
salt ’*’ fs.btrfs_get_default_subvol path=’/var/lib/ceph’
salt ’*’ fs.btrfs_subvol_exists subvol=’@/var/lib/ceph’
salt ’*’ fs.btrfs_create_subvol subvol=’@/var/lib/ceph’
salt ’*’ fs.btrfs_mount_subvol subvol=’@/var/lib/ceph’


salt ’*’ fs.get_attrs path=’/var/lib/ceph’
salt ’*’ fs.add_attrs path=’/var/lib/ceph’ attrs=’C’

[rec=True|False] [omit=’/var/lib/ceph/osd’]

salt ’*’ fs.remove_attrs path=’/var/lib/ceph’ attrs=’C’

[rec=True|False] [omit=’/var/lib/ceph/osd’]

salt ’*’ fs.set_attrs path=’/var/lib/ceph’ attrs=’C’

[rec=True|False] [omit=’/var/lib/ceph/osd’]

salt ’*’ fs.get_mountpoint_opts mountpoint=’/var/lib/ceph’
salt ’*’ fs.get_mountpoint path=’/var/lib/ceph’
salt ’*’ fs.get_mount_info path=’/var/lib/ceph’
salt ’*’ fs.get_uuid dev_path=’/dev/sda1’
salt ’*’ fs.get_device_info mountpoint=’/’
salt ’*’ fs.instantiate_btrfs_subvolume subvol=’@/var/lib/ceph’


salt ’*’ fs.migrate_path_to_btrfs_subvolume subvol=’@/var/lib/ceph’


salt ’*’ fs.inspect_path path=’/var/lib/ceph’

salt ’ganehsa*’ ganesha.configurations
salt ’ganesha*’ ganesha.get_exports_info

salt ’*’ kernel.replace

salt ’master_minion’ keyring.file
salt ’master_minion’ keyring.secret

salt ’master_minion’ mon.list

salt ’*’
salt ’*’ multi.ping_cmd

salt ’storage*’ osd.paths
salt ’storage*’ osd.devices
salt ’storage*’ osd.pairs
salt ’storage*’ osd.configured
salt ’storage*’ osd.list
salt ’storage*’ osd.rescinded
salt ’storage*’ osd.ids
salt ’storage*’ osd.tree
salt ’storage*’ osd.down
salt ’storage*’ osd.zero_weight
salt ’storage*’ osd.restore_weight
salt ’storage*’ osd.readlink
salt ’storage*’ osd.partition
salt ’storage*’ osd.split_partition
salt ’storage*’ osd.remove
salt ’storage*’ osd.is_empty
salt ’storage*’ osd.terminate
salt ’storage*’ osd.unmount
salt ’storage*’ osd.wipe
salt ’storage*’ osd.destroy
salt ’storage*’ osd.is_partitioned
salt ’storage*’ osd.deploy
salt ’storage*’ osd.redeploy
salt ’storage*’ osd.is_prepared
salt ’storage*’ osd.prepare
salt ’storage*’ osd.is_activated
salt ’storage*’ osd.activate
salt ’storage*’ osd.detect
salt ’storage*’ osd.is_incorrect
salt ’storage*’ osd.partitions
salt ’storage*’ osd.retain
salt ’storage*’

salt ’*’ packagemanager.up
salt ’*’ packagemanager.dup
salt ’*’ packagemanager.patch
salt ’*’ packagemanager.migrate

salt ’*’ proposal.generate
salt ’*’ proposal.test

salt ’master_minion’ purge.configuration
salt ’master_minion’ purge.roles
salt ’master_minion’ purge.proposals
salt ’master_minion’ purge.default

salt ’*’ retry.cmd
salt ’*’ retry.pkill

salt ’rgw*’ rgw.configuration
salt ’rgw*’ rgw.configurations
salt ’rgw*’ rgw.users
salt ’rgw*’ rgw.add_users
salt ’rgw*’ rgw.create_buckets
salt ’rgw*’ rgw.access_key
salt ’rgw*’ rgw.secret_key

salt ’master_minion’ wait.out
salt ’master_minion’ wait.until

salt ’*’

Salt runners have different uses as well. Some can be utilities with user friendly output. Others serve as conditionals inside of orchestrations. Lastly, some execute complex operations on the Salt master that are difficult to express in an orchestration.

Since runners execute in the context of the Salt master, these environments have access to all pillar data for all minions. However, file operations are restricted to the Salt master user such as salt.

All runners have a help function that lists the commands available with a brief description and example use. For example,


The full list of commands is below:

salt-run advise.salt_run
salt-run advise.salt_upgrade
salt-run advise.networks

salt-run benchmark.baseline
salt-run benchmark.rbd
salt-run benchmark.cephfs

salt-run cephprocesses.check [results=True|False]
salt-run cephprocesses.mon
salt-run cephprocesses.wait

salt-run disengage.check

salt-run filequeue.queues
salt-run filequeue.enqueue
salt-run filequeue.dequeue
salt-run filequeue.pop
salt-run filequeue.list
salt-run filequeue.items
salt-run filequeue.empty
salt-run filequeue.check
salt-run filequeue.remove
salt-run filequeue.vacate

salt-run fs.inspect_var
salt-run fs.create_var
salt-run fs.migrate_var
salt-run fs.correct_var_attrs

salt-run minions.ready
salt-run minions.message

salt-run net.get_cpu_count
salt-run net.jumbo_ping
salt-run net.iperf

salt-run orderednodes.unique

salt-run populate.proposals
salt-run populate.engulf_existing_cluster

salt-run proposal.test
salt-run proposal.peek
salt-run proposal.populate

salt-run push.proposal
salt-run push.convert

salt-run ready.check

salt-run remove.osd

salt-run rescinded.ids
salt-run rescinded.osds

salt-run select.minions
salt-run select.one_minion
salt-run select.public_addresses
salt-run select.attr
salt-run select.from



salt-run ui_ganesha.get_hosts
salt-run ui_ganesha.get_fsals_available
salt-run ui_ganesha.get_exports
salt-run ui_ganesha.save_exports
salt-run ui_ganesha.deploy_exports
salt-run ui_ganesha.status_exports
salt-run ui_ganesha.stop_exports

salt-run ui_iscsi.populate
salt-run ui_iscsi.config
salt-run ui_iscsi.interfaces
salt-run ui_iscsi.images
salt-run ui_iscsi.status
salt-run ui_iscsi.deploy
salt-run ui_iscsi.undeploy

salt-run ui_rgw.credentials
salt-run ui_rgw.endpoints
salt-run ui_rgw.token

salt-run validate.pillars
salt-run validate.pillar
salt-run validate.setup
salt-run validate.prep
salt-run validate.discovery
salt-run validate.deploy
salt-run validate.saltapi


Eric Jackson <>