CloudShell's OOB Orchestration
Every CloudShell installation includes out of the box workflows. These reflect some common workflows we see across many of our customers that we’ve decided to integrate as default behavior. The OOB setup and teardown processes handle App deployment and startup, connectivity, App discovery and installation. The OOB Save and Restore processes are used for saving the sandbox state and restoring it as a new sandbox. The setup and teardown OOB scripts are included as part of the default blueprint template as of CloudShell 7.1, while the Save and Restore OOB scripts are included starting with CloudShell 9.0.
In this article:
Setup and Teardown Orchestration
The following diagram describes the OOB setup and teardown flow:
These OOB setup and teardown scripts can be found in the Scripts – Blueprint management page. You can review their source code in the cloudshell-orch-sandbox repository.
As of CloudShell 8.1, the default setup and teardown logic moved to a python package called cloudshell-orch-core for ease of use. The default blueprint template includes a reference to the cloudshell-orch-core package using the requirments.txt mechanism, which is supported for orchestration scripts.
Here is the implementation of the OOB setup script:
As you can see, to use the default orchestration logic, we instantiated the DefaultSetupWorkflow class and registered the sandbox to use the default behavior. Starting in CloudShell 8.1, sandbox setup is divided into 4 stages: preparation, provisioning, connectivity and configuration. It’s possible to disable the default implementation of each stage by setting enable_stageName=False, as illustrated in this example:
The OOB setup and teardown scripts can easily be customized or extended. Click here for an example on how to customize the app configuration order in the setup stage, or see other samples to learn how to extend the OOB orchestration scripts.
Extending the OOB Setup Orchestration Scripts
Setup script logic is divided into 4 stages – Preparation, Provisioning, Connectivity and Configuration. The OOB setup already includes some default logic in each of the stages as depicted in the diagram above. However, the OOB setup can easily be extended. Each Setup stage has a specific logic functionality.
- Preparation is empty in the default Setup script. This is the place to enter any code that logically has to be executed before Setup logic is initiated.
- Provisioning deploys and discovers all apps.
- Connectivity connects all layer 1/layer 2/subnet connections, powers on and refreshes IPs on deployed apps.
- Configuration applies any additional configuration on deployed apps
Each stage has an interim
on__[stage]_ended step which allows the execution of any code that has to run between stages. Note that all the functions you add to a stage (using
add_to_configuration, for example) run in parallel, while
on__[stage]_ended functions run sequentially.
You can extend the OOB setup and teardown scripts by adding additional steps, or controlling the order of execution. In this section we will focus on the setup script, for examples about how to extend the teardown, see Extending the OOB Teardown Orchestration Scripts below. An example for extending the OOB Setup script can be calling additional commands to validate Apps or resource states, launching additional orchestration, or controlling the order in which the sandbox is provisioned.
Create a copy of the appropriate script, (see below for extension options), and upload the updated version separately into CloudShell Portal as a Setup script. DO NOT remove any step in the setup workflow. However, you can add your own steps or change the order of execution.
Make sure not to name your extended script ‘setup’ but give it a more specific name. The name ‘setup’ is a reserved name, which may cause unexpected behavior when used on a setup script.
You can extend the Setup script to implement the required logic in one of the setup stages: preparation, provisioning, connectivity and configuration or as a post stage for validation. Make sure to add a requirements.txt file that will include the cloudshell-orch-core package. For example, adding some logic to the configuration stage can be made in the following way:
The workflow helper supports the following extension methods for setup orchestration:
Each of the following methods gets a custom function and list of components to use in the function. For example, executing some custom logic to validate resource configuration:
Note that all methods of the OOB setup logic in the same stage are executed in parallel. The custom function should get arrays of the sandbox and its components as inputs. It’s recommended to use this function template as a starting point:
Here is an implementation example of custom configuration logic for a 3 tier application where each type of App is configured consecutively while passing some global inputs and configuration parameters between the Apps:
Here is another implementation that shows a scenario where some physical devices need to be loaded while few applications are deployed:
You can extend the OOB Teardown script to execute custom steps prior to the out-of-the-box teardown orchestration, or to execute custom steps in parallel to the OOB teardown. This is done using the following extension methods, which are included in the workflow property in the Sandbox class:
Each of the above methods gets a custom function and list of components to use in the function. All steps configured using the before_teardown_started method will be executed in a sequential manner, and all steps configured using the add_to_teardown method will be executed in parallel.
Here is an example of how to execute a command on a resource prior to the default teardown orchestration, note that a requirements.txt file containing cloudshell-orch-core should be attached to the script:
Make sure to follow these steps when implementing a custom teardown orchestration:
Create a copy of the appropriate script, (see below for extension options), and upload the updated version separately into CloudShell Portal as a Teardown script. DO NOT remove steps from the teardown workflow. However, you can add your own steps or change the order of execution.
Save and Restore Orchestration
Note that these orchestration scripts apply to customers who have purchased the Save and Restore paid add-on. Contact your account manager to obtain a license.
Starting with CloudShell 9.0, Save and Restore scripts are provided to support the capability to save and restore sandboxes. They reside in a python package called cloudshell-orch-core. The OOB default blueprint template includes these orchestration scripts and a reference to the cloudshell-orch-core package (required by these scripts) using the requirements.txt mechanism. Here is the implementation of the OOB Save script:
By running the
execute_save method on a sandbox, the script will call a server logic that will create a saved sandbox. For details about the saving process, see the CloudShell Help.
Extending the OOB Save Orchestration Script
You can extend the OOB Save script to execute custom steps before or after the default sandbox save process takes place.
To do this, simply add your custom code before or after the line that executes the Save operation. For example, a Save orchestration script that sends a simple notification email when the Save operation completes:
Extending the OOB Restore Orchestration Script
You can also extend the OOB Restore script to execute custom functionality at any point during the default sandbox restore process. The Restore script is a part of the sandbox setup process, and actually replaces the setup. Out of the box, the setup and restore logic are identical. However, if you customized the Setup script and you want the same customized script to be launched when restoring a sandbox, you should customize the Restore script as well, as the Restore script is the one that is being launched in a restored sandbox’s setup phase. It is also possible to customize the Restore script to have a different logic than the Setup script, to create a logic that is relevant only for restored sandboxes. For detailed explanations on how to extend the script’s stages and use its extension methods, see the Setup and Teardown Orchestration section above.
Note that CloudShell does not support executing the Setup command directly via the Restore script using APIs, such as ExecuteEnvironmentCommand or EnqueueEnvironmentCommand.
For example, a Restore script that writes a message to the Output console before the Restore workflow operation (to extend the workflow operation itself, use the above extension methods in the Extending the OOB Setup Orchestration Scripts section above):
As you can see, to use the default orchestration logic, we instantiated the DefaultSetupWorkflow class and registered the sandbox to use the default Setup orchestration behavior. Starting in CloudShell 8.1, sandbox setup is divided into 4 stages: preparation, provisioning, connectivity and configuration. It’s possible to disable the default implementation of each stage by setting enable_stageName=False, as illustrated in this example: