by Nathan Bennett
At the beginning of October, VMware acquired the company SaltStack. The VMware blog post about the purchase indicated that the reason for the acquisition was “to bolster Software Configuration Management and Infrastructure Automation”. With products like VMware Cloud Foundation, vRealize Automation, and vRealize Operations, you might find yourself questioning this acquisition based on their claim. This is where we were back in November when an additional VMware blog post came out showing how vRealize Automation can deploy Salt-Open from scratch. This led me to dig deeper, finding a use case for SaltStack Enterprise, and configuring a deployment using vRealize Automation. The goal of this use case was to go from nothing deployed to the SaltStack Master to deploying Minions that are configured with one blueprint. We are only using only one outside resource besides vRealization Automation (vRA), which is a repository server (we used GitLab) that held the SaltStack files needed for configuring the Minions. All of this is ZERO touch. What this means is when clicking deploys from vRealize Automation, the solution would build the SaltStack infrastructure, and SaltStack would get the application up and running. After a week or two working in SaltStack, not only is the entire deployment automated, but we have integrated some self-healing solutions to show how Salt components can work together to fix applications after deployment. Before getting into the process, we should cover some fundamentals that got us there.
SaltStack Components
SaltStack is run with a Master/Minion architecture. This means that the Master has all the needed configuration to make sure the Minions are running following their state configuration. The state files are stored on the Master as well as any other files needed to create functions for the minions as shown in Figure 1 below.
figure 1
Looking at the Master/Minions themselves, we can breakdown each component starting with Salt Grains. Salt Grains are properties that are stored on the Minion. This is where the first challenge appears. The properties (Salt Grains) on the Minions are there, but we need those properties on the Master. This is where the Salt Mine function comes in. The Salt Mine is a function that you can write that pulls the Salt Grains and stores it in the Pillar, which is stored on the Master. The Salt Pillar itself is a tree-like structure of data defined on the Salt Master and can be passed through to the Minions. This tree-like data structure has multiple different configuration solutions which make the solution extremely versatile. We used the Salt Pillar to pull Mine information, store secret data, and hold the configuration for Salt Beacons. Salt Beacons are notification functions utilized to send data directly to the event bus on the Salt Master. This makes sure that the functions are recorded on the Salt Master and can be utilized, like all other notifications on the event bus, with the Salt Reactor. Finally, the Salt Reactor will dictate what the Salt Master will do depending on that event bus. The event bus captures all the different communications and notations happening on the Salt Master/Minion. With the Salt Reactor, you can automate a reaction based on those events.
Bring it Together
With an understanding of the different components that are used for SaltStack, let’s see what can be done when pulling them all together and combining vRealize Automation in the solution.
Figure 2 illustrates a look at the blueprint:
figure 2
This blueprint deploys the full stack for SaltStack Enterprise utilizing a centOS 7 template for the Master and Ubuntu18 for all the Minions. This allows Salt Enterprise to have the configuration needed, while still deploying configuration to the Minions and setting them up to communicate with the Master. On top of the SaltStack solution, we deployed an NSX-T Load Balancer for the front-end web servers with vRealize Automation. This allows the servers for SaltStack via Cloud-Init to be configured. Once Cloud-Init configures the Master and Minions, it will clone the needed repository to the Master and perform all the needed configurations. The first thing the Master will need to do is accept the Minion Key. Minions use Key Authorizations to create a communicative link to the Master. Before anything can be pushed from the Master to the Minion, the Key must be accepted in the Master. This process was automated with a piece of code in the Reactor to auto-accept Keys. Figure 3 is a snippet of the code:
figure 3
This references the name of the Minion and will auto-accept the Key if the name starts with “salt”. This is very basic, but it allows the key to be authenticated. This is the only first step to create the authorization for the minion to talk to the master. The big configuration comes from the event bus, which then tells the Reactor to call the state configuration to be run. This loop repeats for the different events that are stipulated in the Reactor during the communication between the bus and the Reactor.
This will automatically deploy the OS configuration to the Minions and configure the application on the servers.
Self-Healing
The next step brings us back to Salt Beacons. Now that you have seen how the event bus and the Reactor can be paired to deploy solutions via its codebase, you also see how Salt Beacons can be extremely powerful when utilizing Salt. Utilizing two different Beacons within the codebase allows us to watch and be notified of changes: one for services and another for a WordPress Configuration file change. The file change notifies the event bus of the change, which will kick off the Reactor to run the state once again on the server Figure 4 illustrates the event bus responding to a Beacon:
figure 4
You can see the apache2 service being stopped and Salt Master restarting the service.
This is an awesome way to see the changes that are done within your configuration. They will also react if moved outside of the state configuration. Other products will do this based on a job that runs on a time sequence or, if it has an agent, running it on a polling interval that operates in a time sequence. Both create a great amount of network traffic. By utilizing a Salt Beacon, you only create traffic when something is broken or drifts outside of the state.
Conclusion
SaltStack utilizes Intelligent Automation that can configure your infrastructure based on the Salt configuration. This is utilized by the different components that SaltStack has in its toolbelt. The SaltStack pulls the properties of the Minions and stores them within the Master, which can then be injected into the state configuration. The state file is now dynamic based on the Grains on those Minions, which end up moving to the Pillar on the Master. After your infrastructure is deployed, Salt Beacon will watch your configuration and notify the event bus when there are issues. This information is utilized by the Reactor to fix issues ranging from restarting the service to watching configuration files.
I hope this helps show how these two tools can deploy and configure your applications and create a self-healing solution. If you are interested in learning more about SaltStack or seeing more, Sterling is here to help you understand what is possible for your environment. Contact us directly by emailing connect@sterling.com.