Ransomware cyber response - Lessons from the trenches

" On a long enough timeline, the survival rate of an organization against a dedicated adversary drops to zero. "

7 years back, around this time, on a long night in the middle of nowhere, I encountered a curious malware sample. Something didn't feel right about it, and thankfully my schedule was wide open.

As I went through the sample, I was able to glean couple of things – it encrypted files, left a backdoor (classic “sethc”) and required manual intervention for execution. It also tried to reach out to a C2 IP (184.107.251.146). Ergo, with some effort – I was able to deduce its tactics and surmise its nature. 

It was my first brush with “LeChiffre”, a malware that offered “Ransomware as a Service” or RaaS capabilities. I was young then, and could not have comprehended the myriad scale of RaaS industry, that would later unfold in front of my eyes.

Back then, ransomware (or "Scareware", its fore-bearer) used to be a type of an opportunistic software, typically run by a ragtag gang of cyber criminals. Organized criminal gangs were yet to realize the immense potential of RaaS from a financial and operational risk standpoint. Over a period of time, RaaS would evolve into a vast enterprise with functions akin to the heads of a hydra – ransomware gangs, exploit brokers, forum owners, initial access brokers, chat support operators, ransomware developers, infrastructure providers and so on. Law enforcement may slay one head, another one will take it's place and assume dominance.

Now, as I look back, 75 plus "cyber response incidents" wiser – that was the moment that actually defined my strategy towards ransomware incidents in particular. Through this post, I would like to share some key lessons that might help you improve your cyber security posture, and preparedness against a ransomware incident.

Don’t Panic!

“ Don't panic, because everything is probably all right, and if it's not, panicking will make it worse. ” – Emily Barr

Ransomware incidents have the power to bring an enterprise to a grinding halt. The double, and triple extortion tactics are squarely aimed to impact where it hurts the most – reputation and regulators. However, know this – You are not the first it has happened to, neither you will be the last. However, if you play your cards right, you will become percipient and resilient.

Enterprise wide panic does nothing to solve an ongoing crisis. 

A single source of truth.

“ Experiment is the sole source of truth. It alone can teach us something new; it alone can give us certainty.” – Henri Poincare

I cannot overstate the importance of log availability and coverage during the course of a ransomware incident. Attackers love to wipe them to impede cyber response and to deter RCA. A centralized log repository acts as a single source of truth from a monitoring, detection and response standpoint.

Ensure your critical assets are correctly configured to send logs to a centralized repository or a SIEM, and they are sending the correct telemetry which can provide early warnings against an impending attack.

Your source of truth is the backbone of your security operations, and preparedness.

Seat-belts first.

“ Superman don't need no seat belt. ” – Muhammad Ali

For the lesser mortals, seat-belts first.

Ensure your environments are configured with “seatbelts first” mindset, it hurts to get hacked later on. I have observed production applications deployed on environments with key security settings disabled – as they were observed to interfere with the application functionality. Instead of fixing the issues with the application behavior via developer or vendor route, the security defenses were disabled in the interest of going live.

This provides good opportunities for an attacker to target vulnerable environments; and to their owners, an opportunity to learn costly lessons later on.

When in doubt, seat-belts first.

You cannot protect what you cannot see.

“ The power of visibility can never be underestimated. ” – Margaret Cho

Countless times, I have observed an internet exposed RDP, for a server configured for “Rob”. Rob was a project admin with administrator privileges, and his server was enabled with an “Any-Any” firewall rule. The server was not integrated with SIEM as it was "test environment" and was not protected by the anti-virus as it interfered with testing. Rob used to access the server through Anydesk or RDP.

Few remember when Rob left the organization, fewer know about that exposed, poorly secured server.

The RCA report will reveal that the attackers knew about that server and leveraged RDP to pivot inside, unobtrusively.

You may laugh now, but I know your deepest fears. Ultimately, you cannot protect what you cannot see.

Shields up!

“ Shall we raise our shields, Captain? ” – Pavel Chekov

While C.I.A principles are good for compliance, I would argue environment compartmentalization is a better strategy for all practical reasons. Containerize your applications, test applications on containers before they are deployed on production. Implement segmentation (and micro segmentation), actually enable payload & TLS inspection on firewalls, stop turning secure defaults off.

Backups are a different ball game altogether, ensure they are protected adequately and tested religiously. They are the lynch-pin for a successful recovery operation against a ransomware attack.

Your sysadmin’s machine is probably the most insecure machine in the network, an access to it exposes everything. Leverage PIM or MFA (hardware tokens, authenticator apps) for authentication and escalation of privileges across environment. Implement principles of least privileges and stop having exceptions for special user groups – Security is an onus for everyone.

Funnel the aforementioned telemetry to SIEM and tune it with at least MITRE ATT&CK use cases. Have some trained eyes to action anomalies and make your environment a cold and unwelcome place for adversaries.

Keep your shields up.

Practice, practice and practice!

“ Amateurs practice till they get it right, professionals practice till they cant get it wrong. ” – Anonymous

Offensive exercises such as red teaming may help identify attack avenues that you might have not considered as part of your existing threat model. Cyber drills may identify response paradigms and preparedness of your organization against a cyber attack.

Cyber drills or ransomware simulations may help uncover pain areas can induce panic and reduce operative effectiveness during the course of a ransomware incident – such as miscommunication, stakeholder accountability, regulator reporting & compliance measures, availability of technical owners and vendors, vendor support clauses, communication channels, PR strategy and communication, decision making process etc.

Practice till it becomes second nature.

Epilogue

“ The winner of the game is the player who makes the next-to-last mistake. ” – Tartakower

Ransomware, or any other cyber attack for that matter is a perpetual game of cat and mouse. The attackers will keep hunting for the right opportunity, while the defenders face the ever looming "goalkeepers paradox". In the grand scheme of things, reduction of opportunities against an invisible, impending attacker helps protects against known-unknowns, and potentially unknown-unknowns.

Cyber, just like any other discipline, is susceptible to the usual debates on the technicalities of implementation, nuances of operations, verbiage of policy, and stratagems of future. But when the curtain falls in the event of an incident; exeunt omnes – only the principles and lessons learnt remain, forged by experience, decisiveness and preparedness of an organization.

Happy holidays!

Thanks to Dasarath S and Vivek Gupta for proof reading. This was cross posted on my personal blog as well. Added inputs from backup standpoint as rightely pointed out by Vikram Jeet Singh. 

 This was cross posted on my linkedin blog as well.

 

Some perspectives on the rise of ransomware attacks

Crime as a Service has evolved into Ransomware as a Service (“RaaS”). The rise of ransomware attacks on companies and the way they are escalating both in terms of scale and tactics is something that was in the making for quite some time. I wished to document my own thoughts on why it has been the case.  

 

"Ransomware"​ by Stratageme.com is licensed with CC BY-NC-SA 2.0.

Previously, RaaS was a one stop shop of techniques – the threat actors had to scout, breach, infiltrate, spread, exfiltrate and extort previously. The RaaS scene has evolved into groups that offer specialisation into each of these skill areas. The RaaS players are focused on ensuring their ransomware is fast, has good after encryption support and has low detection instead of doing all these above-mentioned steps. For instance, nowadays, ransomware actors today buy initial access from “Initial Access Brokers” (“IAB”) – pentesters who have already broken into target networks or buy RDP access of compromised networks for as low as USD 5. The IAB Affiliates (or in some cases RaaS groups) use native utilities/ or leverage Living off the Land Binaries (Loll bins) to stay under the radar for extended periods of time. Scouts offer their services on forums like XSS/ dread/ exploit/ raidforums et al. Affiliates with low activity see their reputation go down or their access is shut by RaaS operators.

Since this is effectively a well-oiled industry now, it has led to formation of cartels. The Maze cartel consists of LockBit, Ryuk, Conti, Egregor, Suncrypt, Ragnar Locker to name a few. The RaaS operators also closely coordinate to share techniques and infrastructure. BlackMatter has tried to incorporate techniques from fellow ransomware threat actors LockBit and DarkSide. Groups typically close shop when they attract too much heat or their infrastructure is blown – they then wait, change names and emerge later. Infra wise, for instance once an IP is blacklisted, it becomes very difficult to whitelist it, hence the IP will be circulated from one gang to another since no good enterprise is going to touch it with a 10 feet pole. This makes threat intelligence an increasingly important and viable solution in identifying threats pre-emptively.

This brings us to the other side of the table. The core question – why is this happening and what makes if profitable business for cyber attackers. For this shady business to sustain, you need pseudo-anonymity as a key pillar. Let’s not confuse it with anonymity. Anonymity is when you can’t tell if X or Y was a threat actor. Pseudo-anonymity is when you know X was a threat actor but you don’t know who are the people behind X. This helps create a certain brand, an idea, an agency. Considering honour among thieves, it makes you a good anchor point for like-minded associates.

 “And ideas, are bulletproof”

 We know actors, Darkside, REvil, multiple APTs. We know their brand. Their affiliates and customers (ahem.. targets) know it too.

Two, rise of crypto and pseudo-anonymity of crypto transactions have eased the way these gents do business. They don’t have to leverage wire transfers to unknown remote countries. A wallet is fine, payment made is funnelled through mixers to make it harder for forensics analysis. By the way, Crypto (here currency, not cryptography) is not anonymous. Anyone can look crypto addresses, wallets, their balances. That’s how these groups are profiled. True anonymity won’t divulge information like this.

Three, generally bad security posture of organizations. Security has been historically seen as an expense, with undetermined ROI, part of IT operations. Whatever that doesn’t makes money for an organization, automatically gets low priority. Business operations enabled by IT take precedence and security becomes an afterthought. Since ROI for typical dilemma is, if what we are securing is less expensive than the measures to secure it, then there is no point in securing it. These unsecured assets / avenues pileup and collectively become a pile of things too hard to secure. They might even become obvious and things then get swept under the rug. Then all it requires is exploiting one vulnerability, and threat actors are in. Historic analysis of any breach will tell that 99 percent only 1 vulnerability was exploited to gain access to networks.  

Then comes a black swan event. A sophisticated adversary chains vulns to compromise at scale. SolarWinds compromise is a great example. Kaseya, is also a good example of this.

Four, one more enabler is the low complexity required to execute these attacks. Plethora of open-source ransomware are present in GitHub. Plenty of attack frameworks are available for free. Take for instance Pneuma.

https://github.com/preludeorg/pneuma 

This was released months back and is already cutting edge. It is also free.

Cobalt strike is available for purchase at low prices, it’s cracked versions are already available for free for quite some time. 10 years back, this would have required arcane knowledge of c2, comms, infra, automation for scaling, evasion and what not.

Today it requires a double click (figuratively speaking).

Finally, the dilemma of known knowns, known unknowns and unknown unknowns. I have yet to see a firm that has full view of its assets. If you don’t know what to protect, then you won’t know about it when it gets hacked. Ultimately, you can’t catch what you can’t see.

There are plenty of other minor enablers as well, but they are subsets of the above-mentioned ones.

This can be stopped by being proactive in your defence strategy, leveraging threat intelligence and having visibility of your assets. And additionally, by ensuring your reactive defence strategy is well practiced till it becomes second nature. In any case, this is not the last cyber-attack we have seen, given the risk, skill to reward ratio of executing these. I’d expect more escalations and more sophisticated hacks down the road – and we have just earned front-row seats.

How I got myself a capable laptop

It all started with my old (and very hated) HP Pavilion notebook (i5, 12 GB RAM, 500 GB HDD) almost dying on me. I wanted to get a new laptop, the only reason I stuck with HP for so many years was that I got it as a gift and I wanted to squeeze every drop of use I could get from it.

Well, let's get a new one then, and I wrote down what I needed -

Must have

1. Good, tactile, backlit keyboard
2. HDMI, not micro HDMI
3. Screen less than 13 inches
4. Good battery life

 Should have

1. i5-i7 would do, AMD Ryzen as well
2. Should be portable
3. Easy to open, repair and upgrade
4. USB 3.0
5. RAM 8 GB or more
6. 256 GB SSD or more

 Nice to have

1. Should support extra battery
2. SIM card slot
3. Swivel support
4. Graphics card
5. MIL-STD-810G
6. Fingerprint sensor for easy login

My options were quite limited considering what I needed would be automatically expensive - I was looking at spending at least INR 75000-100000 (USD ~1000-1300) to get a new one. That too a base model. 

I didn't mind buying a used one, if it served my purpose and was in good condition. I reached out to my connects in hardware segment and asked for their advice. 

A used Ferrari is always a Ferrari, a new ALTO will never match it.

Point well noted.

They referred me to leased laptop distributors, which typically have inventories of laptops which are leased to corporate for 2-3 years and then brought back once the contract is over. Since they are used, people are less inclined to buy them, but their configurations are top notch as compared to their retail consumer segment counterparts and they are built to repair. These laptops are then dismantled and their parts flood the after sales market. The distributors are more than happy if their laptops are sold before they are dismantled.

After having friendly chitchat with a lot of distributors, I finally narrowed my options to Lenovo X250 and an HP EliteBook. The keyboards were nice and tactile and the form factor was small. At one of the distributors, from a heap of laptops, I picked 2 and I asked the person if I can open it. He said why not, and he opened it for me. Both were in good condition, sporting 256 GB SSDs, 8 GB RAM, i5 5th gen processors and were costing INR 14000 (~USD 190), a far cry from new ones, but workable configuration. I asked about warranty and after a bit of negotiation, he agreed for a 1-year repair warranty for INR 2500 (~USD 34). Windows 10 pro was provided for free.

I was about to settle it for X250 one (as it had more ports, was smaller and checked almost everything I needed), one of associates waltzed in and said, "we just got a shipment of some new stock". I asked if I could take a look and they pointed me to next door.

From a heap of X260s I picked 3 - one with no battery and i7 6gen, one with an extra 6 cell battery with i5 6gen and one had 1 TB HDD. I asked if I could swap parts, and they said we don’t care, it’s all the same for us.

I took the extra battery and plugged it into i7 one. CPUZ said it had Skylake i7 6600u and Samsung 8 GB built in. It had 256 GB SSD and a working WWAN module (SIM module) as well. Single memory slot (DDR4, 260 pin SODIMM) but was easy to open and clean. After playing it with for 1 hour, post testing all the ports, modules, running some stress tests, and haggling a bit, I went home with a deal at INR 17000 (~USD 230) with 1-year warranty from distributor, Windows 10 pro bundled.

Then I did some research and checked the maximum RAM it supported - 16GB, 2133MHz DDR4, non-parity. Probably, enough for what I do. 2133 MHz is a bit hard to get by, so a better option was to buy 2666 MHz one since it will run automatically at 2133 Mhz. I did some research (read: going through Reddit threads, Lenovo forums) and found that one user was able to successfully upgrade it with 32 GB of RAM (M471A4G43MB1, costs around INR 27000/ ~USD 370 even more expensive than the laptop). Post upgrading to latest BIOS, I decided to take the risk and got myself a cheaper one (ADATA AD4S2666732G19, 32 GB RAM, 2666 MHz, INR 9000/ ~USD 122) from one of the distributors.

Went back home, disabled internal battery from BIOS, unscrewed & pried back cover and disconnected battery cable. Swapped out 8GB one with 32 GB one. Connected battery cable, power cable and was met with POST screen. Assembled everything back again and ran memtest86 and windows memory diagnostics. Everything was squeaky clean :). Hardened everything, installed virtual box, migrated my VMs, installed emulators and voila, my new system is ready.

I have been using X260 since last 6 months as my primary laptop with the following configuration which runs multiple VMs simultaneously, is used for maintaining remote infrastructure, occasional retro gaming/ emulation and occasional writing :  

1. Tactile backlit keyboard
   
2. 6th Gen Intel Core i7-6600U Processor, Turbo Boost 2.0 (3.4GHz)
3. 32 GB memory (ADATA AD4S2666732G19)
4. 12.5" HD (1366 x 768) IPS
5. 256 GB Samsung SSD
6. 3 Cell internal + 6 cell external battery
7. SIM card slot (WWAN)
8. 3 USB 3.0 ports (Superspeed)
9. 1 HDMI/ 1 Mini DisplayPort
10. 4-in-1 Card Reader (MMC, SD, SDHC, SDXC)
11. Intel I219 Gigabit LAN & Dual Band Wireless-AC 8260, with Bluetooth® 4.1
12. MIL-STD-810G compliant
13. Weighs around 1.5 KG
14. Bundled Windows 10 Pro

Total Cost - INR 26000 / ~USD 352

Lessons learnt –

1. Research, hunt and haggle
2. Be very specific about your requirements
3. Technology evolves every day, see what fits your needs on a long-term basis

Assessing a cyber security candidate

I typically assess a senior cyber security candidate across 7 basic domains for a technical interview, before I actually jump into security. Sometimes, a candidate is so good in these domains that asking questions about security becomes an afterthought. A dipstick feedback of fundamentals actually helps me understand where the candidate is coming from and if he can actually leverage his technical skills in real security engagements. Since these domains are exhaustive, assessment of fundamentals will depend on the previous experience a candidate is having. For instance, I would not expect a college grad to know complete ins and outs of active directory but would expect him to know programming, scripting, linux, VM and algorithms. A SOC guy should know network security, pcap analysis, protocols, BPF/ filters, elementary scripting. An experienced pentester is supposed to know almost all of the mentioned domains. At the end of the day, YMMV.

Tools are not going to make you a hacker, always remember what Gray Fox said -

"Only a fool trusts his life to a weapon"

These domains form the bread and butter basics of any good computer security candidate and enable him to understand the cross functional world from the point of an architect, an operations analyst, an incident responder, a developer, a packer mangler or simply as an adviser.

kinda like this. No Wait! Robyn Beck/AFP/Getty Images

Depending upon the feedback of this article, I may share some good to have domains as well.

Nevertheless, here are the domains :

1. OS Fundamentals / Software - This is a big one, without these, your attack vectors typically fall flat. Windows and Linux are mandatory. Can you setup a working environment for your own security setup from scratch? Comfortable with VMs? Docker? Jailed environments?
2. Network/ Network Security - Routers, switches, load balancers et al, be it software of hardware. Are the concepts clear? Considering a lot of the hardware is now virtualized/ customized and is being offered as a service by big providers and every now and then an attack/ exploit emerges that leverages misconfigurations in these systems/ services - these things are important. Can you understand a Pcap? Do you understand routing ? If there is one thing studying Phenoelit early on taught me, was to understand network and routing properly. 
3. Active Directory/ LDAP - I simply can't overstate the importance of AD/ LDAP when it comes to security. Considering how they function as the backbone for enterprise, you are bound to encounter these. Having good fundamentals around these gives you a good headstart when you actually pentest these environments.
4. Servers/ Web services/ APIs - Servers and web application basics, how they work, are deployed and do you know how to secure them? Fundamentals are important here. You may be able to find a bug in an application, but in case you can't fix the application per se, can you secure or advise correctly about securing the environment itself? How are application headers used? Do you know how to interact with an API? Can you create your own? Do you operate any website/ webservice? How do you scale it?
5. Programming/ Scripting - Any one programming or scripting language you are comfortable with - python/ ruby/ bash/ powershell et al. Doesn't matter what it is. Can you read code? Can you comprehend patterns? Can you write pseudocode? Do you have fundamental understanding of algorithms?
6. Hardware - Good to have knowledge of hardware basics, you should be comfortable with atleast setting up platforms like raspberry pi, beaglebone et al. Can you identify pinouts on an unknown board? Can you read technical manuals? What is your portable platform of choice? Can you setup your own VPN environment on a raspberry pi and hook it up with your test laptop?
7. Architecture/ Tooling - A typical question starts like this : create a full fledged network for 100 people with everything included, LAN, WAN web services, email et al. Now design for 1000 people. Now for 10 K people. Now let's break it systematically - how will you break it? What attack vectors? What if Burpsuite is not available? Can you leverage curl? How can we improve it?

These domains are absolute essentials, platforms and tools may make you a bug hunter, but a knowledge of these will make you a better one.
There you go, if you know these, you already have a healthy background into computer security basics and I wish you best of luck.

This was crossposted at Fruxlabs Team blog.

The Rescure Cyber Threat Intelligence Project - Sensor Update

We have massively upgraded our sensor detection, logging and monitoring capabilities at rescure.me - we detected around 350K attacks in last 24 hours which are then funneled and curated as feeds by our co-relation system. This included removing code cruft, updating data pipelines, a new ELK stack which can monitor multiple sensors at once. Feeds have been optimized as well and the stack has been migrated to new high performance servers.

Countless hours and personal funds have gone in to maintain this, special thanks to Fruxlabs Crack Team for being there. w00t!

In case you wish to collaborate in terms of sensors/ feeds/ research, please do reach us out at support@fruxlabs.com.