Chef Backend to Automate HA

[edit on GitHub]

This page explains the procedure to migrate the existing Chef Backend data to the newly deployed Chef Automate HA. This migration involves two steps:

• Back up the data from an existing Chef Backend via knife-ec-backup.
• Restore the backed-up data to the newly deployed Chef Automate HA environment via knife-ec-restore.

Take backup using the knife-ec-backup utility and move the backup folder to the newly deployed Chef Server. Later, restore using the same utility. The backup migrates all the cookbooks, users, data-bags, policies, and organizations.

knife-ec-backup utility backups and restores the data in an Enterprise Chef Server installation, preserving the data in an intermediate, editable text format. It is similar to the knife download and knife upload commands and uses the same underlying libraries. It includes workarounds for unsupported objects by the tools and various Server API deficiencies. The goal is to improve knife download, knife upload, and the Chef Infra Server API to deprecate the tool.

Backup the Existing Chef Backend Data

1. Execute below command to install Habitat

      curl https://raw.githubusercontent.com/habitat-sh/habitat/master/components/hab/install.sh \ | sudo bash

2. Execute the below command to install the habitat package for knife-ec-backup

    hab pkg install chef/knife-ec-backup

3. Execute the below command to generate a knife tidy server report to examine the stale node, data etc. In this command:

• pivotal is the name of the user
• path of pivotal is the path where the user’s pem file is stored.
• node-threshold NUM_DAYS is the maximum number of days since the last checking before a node is considered stale.

    hab pkg exec chef/knife-ec-backup knife tidy server report --node-threshold 60 -s <chef server URL> -u <pivotal> -k <path of pivotal>

For Example:

    hab pkg exec chef/knife-ec-backup knife tidy server report --node-threshold 60 -s https://chef.io -u pivotal -k /etc/opscode/pivotal.pem

4. Execute the below command to initiate a backup of your Chef Server data. In this command:

• with-user-sql is required to handle user passwords and ensure user-specific association groups that are not duplicate.

• --with-key-sql is to handle cases where customers have users with multiple pem keys associated with their user or clients. The current chef-server API only dumps the default key. Sometimes, users will generate and assign additional keys to give additional users access to an account but still be able to lock them out later without removing everyone’s access.

    hab pkg exec chef/knife-ec-backup knife ec backup backup_$(date '+%Y%m%d%H%M%s') --webui-key /etc/opscode/webui_priv.pem -s <chef server

For example:

    hab pkg exec chef/knife-ec-backup knife ec backup backup_$(date '+%Y%m%d%H%M%s') --webui-key /etc/opscode/webui_priv.pem -s https://chef.io`.

• Execute the below command to clean unused data from reports. This is an optional steps

    hab pkg exec chef/knife-ec-backup knife tidy server clean --backup-path /path/to/an-ec-backup

5. Execute the below command to copy the backup directory to the Automate HA Chef Server.

     scp -i /path/to/key backup\_$(date '+%Y%m%d%H%M%s') user@host:/home/user

Restore Backed Up Data to Chef Automate HA

• Execute the below command to install the habitat package for knife-ec-backup

    hab pkg install chef/knife-ec-backup

• Execute the below command to restore the backup.

    hab pkg exec chef/knife-ec-backup knife ec restore /home/centos/backup\_2021061013191623331154 -yes --concurrency 1 --webui-key /hab/svc/automate-cs-oc-erchef/data/webui\_priv.pem --purge -c /hab/pkgs/chef/chef-server-ctl/*/*/omnibus-ctl/spec/fixtures/pivotal.rb

In place Migration (Chef Backend to Automate HA)

As part of this scenario, customer will migrate from chef-backend (5 machines) to Automate HA in-place, i.e. Automate HA will be deployed in those 5 machines only where Chef-backend is running. One extra bastion node will be required which will be managing the deployment of Automate HA on the chef backend infrastructure.

1. ssh to all the backend nodes of chef-backend and run

    chef-backend-ctl stop

2. ssh to all frontend nodes of chef-backend and run

    chef-server-ctl stop

3. Create one bastion machine under the same network space.

4. ssh to bastion machine and download chef-automate cli

https://packages.chef.io/files/current/latest/chef-automate-cli/chef-automate_linux_amd64.zip

5. Extract downloaded zip file

6. Create a airgap bundle using command

    ./chef-automate airgap bundle create 

7. Geneate config.toml file using command

    ./chef-automate init-config-ha existing_infra 

8. Edit config.toml and add following things

• update the intance_count
• fqdn : load balance url, which points to frondend node.
• keys : ssh username and private keys
• Make sure to provide Chef backend’s frontend server IPs for Automate HA Chef Automate and Chef Server.
• Make sure to provide Chef backend’s backend server IPs for Automate HA Postgres and OpenSearch machines.
• Sample configuration, please modify according to your needs.

[architecture.existing_infra]
secrets_key_file = "/hab/a2_deploy_workspace/secrets.key"
secrets_store_file = "/hab/a2_deploy_workspace/secrets.json"
architecture = "existing_nodes"
workspace_path = "/hab/a2_deploy_workspace"
ssh_user = "myusername"
ssh_key_file = "~/.ssh/mykey.pem"
sudo_password = ""

# DON'T MODIFY THE BELOW LINE (backup_mount)
backup_mount = "/mnt/automate_backups"

[automate.config]
# admin_password = ""
# automate load balancer fqdn IP or path
fqdn = "chef.example.com"
instance_count = "2"
# teams_port = ""
config_file = "configs/automate.toml"

[chef_server.config]
instance_count = "2"

[opensearch.config]
instance_count = "3"

[postgresql.config]
instance_count = "3"

[existing_infra.config]
automate_private_ips = ["10.0.1.0","10.0.2.0"]
chef_server_private_ips = ["10.0.1.0","10.0.2.0"]
opensearch_private_ips = ["10.0.3.0","10.0.4.0","10.0.5.0"]
postgresql_private_ips = ["10.0.3.0","10.0.4.0","10.0.5.0"]

9. Deploy using

./chef-automate deploy config.toml <airgapped bundle name>

10. Clean up the old packages from chef-backend (like Elasticsearch and postgres)