We were able to corroborate the results above with our disk usage snapshot taken a week before and did not find suspicious disk usage, which would have prompted us to check for large files. The command used was sudo salt salt-minion-01 age. Our assumption was that this server could have been used to store files looted from other systems from the network. We wanted to examine the possibility of locally-stored exfiltrated data by first querying the disk usage within the breached server. We made use of Salt Runners, using the command sudo salt-run manage.status, as shown below. To begin our analysis, we wanted to determine whether our compromised system was still up or had been formatted. What follows is the discovery and analysis of the malicious actor’s actions within the JBoss server (salt-minion), as collected using Salt (salt-master). Fortunately, our effective and quick response ensured that the intrusion was stopped before significant damage could be done. The server was being used by the adversary to launch scans within the internal network according to our SIEM logs. On March 12th, 2019, our Intrusion Detection team discovered unauthorized access on our JBoss production server. Intrusion scenario - Ubuntu Linux server breach (salt-minion) What follows is a hypothetical scenario that can be used to inform the queries we make to the minion in order to demonstrate a bit of the capabilities of Salt. Module refers to execution modules and function refers to the functions that can exist within each module. In the command above, “section” can be replaced with a minion ID to refer to a specific minion or can be replaced with “*” to mean all the minions. The way this is done is by making use of salt commands which take the form below: Threat hunters can make significant use of SaltOpen to interrogate minions in order to support their hunting activities. You should have a similar result to the following: To ensure that you can issue commands to the minion, the following command can be run on the master: The output should be a key that matches on both hosts.

Within the minion: sudo salt-call –local key.finger Within the master: sudo salt-key –finger salt-minion-01 Now when we list the keys using ‘sudo salt-key -L’, we should be able to see the accepted key (minion), as shown below:Īt this point, the master and minion are connected, and the keys can be verified by issuing the following commands respectively: To accept the key above, we’ll issue the command sudo salt-key –accept=’salt-minion-01’. We then head back to the master and issue this command: sudo salt-key -L. We then copy the master.pub key above, feed it into the master_finger section within the file /etc/salt/minion in the salt-minion and restart the salt-minion service.

Now we can print the master-key fingerprint by running the command This is done by editing /etc/hosts and including the line ‘192.168.100.29 salt’. Next, we need to edit the hosts file to contain a DNS record for the salt master. This is done by editing the file /etc/salt/minion_id and adding salt-minion-01 as our minion ID. We then also edit the salt minion ID to hold a descriptive name of the salt minion. Our master is located at 192.168.100.29, so we feed that into the file. In our case, we edit the file to include the line ‘master: location’ (where location is where the master is located). This can be done by editing the minion config file found at /etc/salt/minion.

We also need to configure the minion to be able to find the master. The screenshot below shows the result of running the command above. This is done by editing the file /etc/salt/master to add the line ‘interface: 192.168.100.30’, where the IP address is the location of the salt minion.

To do this, we edit the salt master config file to contain the location of the salt minion. The following screenshot shows the result of running the command above. Once you have installed the salt-master on your master and salt-minion on your minions, you can start them both using the following commands respectively. In our case, we’ll be running the master and minion on Ubuntu 18.04 systems. Installation of Salt Open is a pretty straightforward process a good guide to it can be found here. Runners: These are master-side convenience applications which are executed by the salt-run command.Returners: These manage arbitrary return locations.Render modules: These make it possible to render information to the Salt state system.Grains: These detect static information about a system.State modules: These make up the back end for the Salt configuration management system.Execution modules: These represent functions that are available for direct execution from the remote execution engine.These modules determine the actions that the Salt user intends to carry out. Salt is made up of six modules that can influence its functionality and management.

Saltstack osquery