The recent SolarWinds breach presents a range of critical security issues for government agencies, healthcare organizations, government agencies and major companies in other industry verticals.

SolarWinds produces a variety of popular IT infrastructure monitoring solutions and the breach it suffered resulted in compromised files being introduced into updates of these monitoring solutions.

These monitoring solutions were used by perhaps 18,000 companies, with one of the most high-profile being FireEye – where use of the compromised SolarWinds software lead to the exfiltration of FireEye’s red team tools.

Given the popularity of the SolarWinds software, many organizations are now scrambling to determine if they too may have suffered a breach or other security issue, due to the use of compromised SolarWinds software.

While responding to an issue like this, it is important to consider that response is needed in two ways. The first set of responses needs to focus on determining whether or not use of the compromised SolarWinds software leads to any security issues within your own organization and taking measures to mitigate the potential for any issues to arise as a result of using the compromised software.

Assuming the compromised SolarWinds software was found to be running within your environment or previously running in your environment, some recommended remediation steps would include the following:

1. Assume all accounts used by SolarWinds for monitoring are now compromised. Monitoring software will typically use service accounts to log into servers, network appliances, and other IT infrastructure.  Any of the accounts that SolarWinds had access to should now be considered compromised and treated accordingly.
2. Look for SolarWinds monitoring account usage. Assuming that proper logging existed prior to this incident, logs should be searched to see if any of the accounts used by SolarWinds were used to attempt to access systems not normally monitored by SolarWinds and if any account usage patterns are suspicious.
3. Look for C&C traffic associated with the attack. The compromised SolarWinds software is associated with some specific IP addresses and communications with some specific domains. Communications to those IPs and domains should be blocked and alerts set up in a SIEM to ensure that any attempts to communicate with those destinations immediately flags a system for further investigation and remediation. 
4. Search for SolarWinds IOCs and other malware or potential indicators of compromise. Checks should be initiated to see if any systems within your environment appear to be compromised and any systems on which an IOC is found flagged for further investigation and remediation.
5. Take a snapshot or some other form of backup that can be used for later forensics if needed. While eliminating the compromised software is critical, before patching or a rebuilding of the SolarWinds systems occurs, it may be beneficial for a snapshot or backup to be taken so forensics can be done on the SolarWinds environment at a later point in time if needed. 
6. Eliminate the compromised software. SolarWinds now makes a patched version of their software available that eliminates the compromised component.  While at a minimum this patched version should be applied, if possible, it would be better to consider building a clean SolarWinds system from scratch as the compromised SolarWinds software may have led to the compromise of other software components within the server that SolarWinds was installed on.  The server should be considered compromised as well. 
7. Network segmentation. If not already in place, this recent security issue is a perfect illustration of why network segmentation of the internal network is critical. No system within a network should have unfettered access to everything else on the network.

While network segmentation will not prevent issues like this from happening, it will go a long way towards mitigating such issues as will limit what other resources within the environment that a compromised system can be used to access.

While far from comprehensive, the above remediations and mitigations give an overview of some of the major steps an organization can take to begin to get a handle on this issue within their own organization. As mentioned above, however, proper response to this breach requires two different types of responses.

Remediating the issue within your own organization is essential, but one needs to keep in mind that SolarWinds products are used by thousands of companies around the world.

• What other FireEye situations are out there that we do not yet know about?  
• What business partners or software vendors may have been using a compromised version of SolarWinds?
• How many of these third-parties are going to be found to have suffered issues similar to FireEye?
• Do we have any SolarWinds using business partners that host PHI that have had data exfiltrated in a manner similar to the red team tools?
• Have any companies we buy software from suffered supply chain compromises due to their use of SolarWinds where we now have to worry about the PACS system we just bought being used as a vector to attack us?
• Have other security vendors been compromised where now they have to worry about the tools we rely on to protect us no longer being effective?

While I hope the answer to all of these questions is no, as security professionals we need to prepare for the worst even as we hope for the best. 

This leads us to the important question of how we address these risks for which the following controls considerations are recommended:

1. Third-party risk management. A robust 3rd party risk management process is essential to mitigate the risks of business partners and risks associated with supply chains.  It is also important to remember to evaluate the 3rd party risk management processes of your business partners and vendors as a part of your own assessment process.  An issue with a business partner of your business partner may still become an issue for you. 
2. Zero Trust. Microsegmentation and other zero trust strategies are becoming increasingly critical.  Whether public facing or internal, every system should be treated as if it can be compromised.  Limiting what the system has access to goes a long way towards ensuring that any security incident remains contained. 
3. Defense in depth. Whether it is due to the compromise of a security vendor or some other reason, the reality is that controls do sometimes fail.  A defense in depth approach is critical to ensure that even if a control in the environment fails there are other controls that can work to mitigate the issue. 
4. Remember we are all in this together. Healthcare organizations need to be increasingly transparent and share threat intelligence and information about the security strategies that work and don’t work for them.  Security is essential for patient safety and we need to work together to ensure that we are keeping our systems and patients safe.

While breaches and compromises such as these are never a good thing, it’s important to keep in mind that every such incident is an opportunity to learn and to use the lessons learned to further improve the security within our environments.  Let’s all take the time to reevaluate our strategies in light of recent events and ensure that we are protected from such attack vectors going forward.

Christopher Frenz is Information Security Officer and AVP of IT Security at Mount Sinai South Nassau