Backup a Linux machine with LVM Snapshots and rdiff-backup

Here is the completed script I wrote on Episode 461. Make sure you check out the full episode for details on how to make this work for you.

And of course, here is the episode:

NEMS – Nagios Enterprise Monitoring Server for Raspberry Pi

NEMS – Nagios Enterprise Monitoring Server for Raspberry Pi

Nagios doesn’t need a big fancy supercomputer to offer exceptional enterprise monitoring of network assets and resources, so our temptation is to re-purpose older servers to perform this reasonably lightweight task.

If you ask me, that’s not only overkill, but the attempt to save money by reusing older hardware will actually cost more due to higher electricity usage vs. a tiny Raspberry Pi 3 Microcomputer… which ironically may in fact have more modern system specifications than that old beast of a server you’ve been using.

It can be a wee bit daunting to setup a Raspberry Pi Nagios server from scratch, and there were no projects I found which were actively maintained at a level adequate for professional use. So I decided to start a new project–called NEMS: Nagios Enterprise Monitoring Server (for Raspberry Pi).

This project uses freely available applications such as Debian Linux, Nagios Core and a variety of other goodies, and I too release a fully ready-to-use image for you to use within yours or your customer’s network environments.

If you like NEMS, please donate: donate.category5.tv

The Out-Of-The-Box NEMS Experience:
NEMS 1.0 Bash MOTDNEMS 1.0 Main Menu ScreenNEMS 1.0 NConf (Configure Nagios) ScreenNEMS 1.0 Nagios 3.5.1 Tactical OverviewNEMS 1.0 Nagios Overview of ServerNEMS 1.0 NagVis (Reports) Opening ScreenNEMS 1.0 Server Overview in NagVis

System Requirements

  • Raspberry Pi 3 Micro Computer
  • 4GB+ Micro SD Card
  • Network Connection (will obtain IP address via DHCP)

Instructions

  • Purchase and assemble your Raspberry Pi 3.
  • Download the most current version of NEMS below.
  • Unzip and “Burn” the image to your Micro SD card.
  • Boot your NEMS server.
  • Login to NEMS console by either connecting a monitor and keyboard, or using ssh (Username: pi / Password: raspberry)
  • Type: sudo raspi-config
  • Expand the filesystem and exit raspi-config.
  • Reboot your Pi and enjoy NEMS when it comes online (see below for further details).

Buy The Needed Hardware

Raspberry Pi 3 Nagios ServerRaspberry Pi 3 are very affordable, and using our Micro SD image, you simply buy the device, “burn” the image to the Micro SD card, and boot it up.

Here’s our link to buy the device you’ll need, complete with the Micro SD card, a power adapter, a good solid case, and more: http://shop.category5.tv/?product=raspberry-pi-3-1-2ghz-1gb-complete-starter-kit

Please buy it through that link, or let me know if you need a customized link to a different model. We get a small percentage of the sale, and it helps to make it possible to offer this as a free download.

About NEMS

The goal with NEMS is to provide a free, full-featured, up to date drop in Nagios Enterprise Monitoring Server for Raspberry Pi 3. It has all the bells and whistles while being optimized for solid stability and fast performance.

Based (with big thanks to Ryan Siegel) on NagiosPi, with many of the settings reflective of the NagiosPi Wheezy distro (which is now obsolete).

Download NEMS

NEMS 1.0 Raspberry Pi Image

NEMS 1.0 Image File (zipped) – 861 MB

Unzip and “burn” to 4 GB+ Raspberry Pi 3 Micro SD Card.

Configuring Your NEMS Server:
Browse to: http://nems/nconf (or http://IPADDRESS/nconf)
Make changes to the config as needed for your environment (eg., add a host, service, edit a setting)
Click: “Generate Nagios Config”
If there are no errors, you will have the option to Deploy the new Nagios Config.

Using Your NEMS Server:
Browse to: http://nems  (or http://IPADDRESS)
Choose Nagios for a customized Nagios system (easy), or use the Reports to create sophisticated reports and custom dashboards (advanced).
To add hosts or configure your environment, use the built in “Configure Nagios” feature.

Passwords:
NOTE: You should change these defaults when setting up your NEMS server.
Nagios – U: nagiosadmin / P: nagiosadmin
Reports – U: admin / P: admin

Changelog:

Version 1.0 – May 8 2016
– Initial release. Built and tested on Raspberry Pi 3. Based on Raspbian Jessie. Inspired by NagiosPi, which in April 2016 was still running on the old Raspbian Wheezy. I started this new distro since NagiosPi seems to be out of date, and I want to have an easy drop-in Nagios img for the Raspberry Pi. Figured I’d share it with the world while I’m at it since there are probably others (possibly less tech savvy) who might want the same thing. I decided to leave most of the settings the same as NagiosPi (eg., usernames, passwords) so those coming from that distro can seamlessly transition, or so if NagiosPi wants to use our build to bring things up to date, they may do so with minimal effort.
– This initial build is using default repositories in a lot of cases and is meant to be rock-solid, not bleeding edge (eg., Nagios 3.5.1 instead of Nagios 4.1.1.
– Using the rpi-4.4.y Linux kernel tree (Currently 4.4.7-v7+ #876 SMP), firmware updated to 1e84c2891c1853a3628aed59c06de0315d13c4f1. Use rpi-update to check for upgrades, if needed.
– Includes rpi-update tool – an easier way to update the firmware on the Raspberry Pi – See https://github.com/Hexxeh/rpi-update
– On-board Bluetooth disabled due to potential stability issues. Use rpi-update to check for kernel updates and see if it is fixed, and then edit /boot/config.txt to re-enable. Until they fix it, use USB Bluetooth dongle if needed.
– Installed and configured: mysql-server mysql-client phpmyadmin apache2 nagios3 nagios-nrpe-plugin
– To keep things consistent for those coming from NagiosPi, I have used the same passwords. MySQL is: User: root Pass: nagiosadmin
– Installed w3m web browser to allow local testing in terminal: w3m localhost/phpmyadmin
– Manually installed NConf 1.3.0-0 “Final”, an Enterprise Nagios configuration tool. This tool was broken on NagiosPi’s instructions due to a missing symlink at /var/www/nconf, so I fixed that in my version. Access NConf via the “Configure Nagios” link on the main menu.
– Includes NagVis 1.7 – See https://sourceforge.net/projects/nagvis/?source=navbar – want to do 1.9 but not until out of beta.
– Built and integrated the first version of our menu system, which includes the first version of a custom Nagios skin to begin integrating a more modern interface. Menu accessible at http://nems/ (or http://IPADDRESS if that doesn’t work for you)
– Added a nice little MOTD with http://www.mewbies.com/how_to_customize_your_console_login_message_tutorial.htm
– Added a simple cronjob to check our web site for the currently available version and warn you if yours is out of date.

Known Issues:

  • None at this time. If you encounter an issue, please report it in the comments below so I can work on a fix (or please post your fix to help others and possibly to have it integrated into the next build).

TO DO:
– Improve the mobile responsive layout for tablets and smartphones (It’s really rudimentary at the moment due to the nature of Nagios’ interface… frames? C’mon now.)
– Investigate getting NEMS working on Raspberry Pi 2 devices – at present it probably breaks (untested)
– Upgrade Nagios to Nagios Core 4
– Upgrade NagVis to 1.9
– Create a migration / config export tool to ease migration to newer versions of NEMS as they release, and also to allow you to backup or import your config easily in case your MicroSD card fails.
– I’m open to suggestions! Please post your comments at http://baldnerd.com/nems

Support What We Do:
This project is a part of something much bigger than itself, and we’re all volunteers. Please see our Patreon page for information about our network.
– Please support us by simply purchasing your Raspberry Pi at https://cat5.tv/pi
– We have some support links on the NEMS menu, such as buying from Amazon using our partner link. Please use these every time you use those stores. A small percentage of your purchase will go toward our projects.
– Your donations are VERY MUCH appreciated – https://donate.category5.tv – Please consider how many hours (and hours) of work this project has saved you, and how much you’ll save on hardware and even electrical costs as you consider contributing
– Our network also has a Patreon page – Please consider becoming a patron – https://patreon.com/Category5

Make it so mountpoint can’t be written to if not mounted.

Have you ever accidentally saved files to a Linux mountpoint when the drive wasn’t mounted, and then couldn’t mount the drive thereafter? Or worse, had a backup run when the backup drive wasn’t mounted, only to fill your filesystem and crash the server?

These problems can be avoided by simply making your mountpoint immutable! What this means is, your mountpoint (the folder itself) cannot be written to. However, even as an immutable folder, it can be mounted to, and the filesystem of the mounted drive then controls the permissions of the folders therein.

It’s a simple Linux command. We’ll pretend our mountpoint is simply /mountpoint. Here’s all you have to do:

Brilliant! And oh, so simple.

Here’s a sample of what happens when I do this as root. Note that ‘mymountpoint’ is setup for me in my /etc/fstab file so it normally auto-mounts.

Enjoy that little tidbit!

As a side note, you might want to also get a notification if your drive isn’t mounted… so you could use the mountpoint command to send you an email if there’s a problem. Just add something like this to your backup script:

That simply checks if /mountpoint is a mounted mountpoint. If yes, it does nothing. If no, it will send you an email.

-Robbie

Convert video to several JPG images on Linux without ffmpeg.

I admit… I do love PHP in the command line. Does that make me a bad person? 😉

Here’s a tiny little script that I wrote to create many JPG screenshots of a video file. I use this each week to create a bunch of stills from our broadcast so I can use them as thumbnails and so-on. I didn’t want it to depend on ffmpeg since I don’t have that on any of my modern systems.

It requires just three packages: mplayer mediainfo php-5

Save it as whatever.php and run it like this: php whatever.php file.wmv

It will create a folder called file-Screenshots/ and will save one picture per 10 seconds for any video source. Just change “file.wmv” to the name of your video. Include the path if it’s not in the current folder.

Hope it helps you out.

-Robbie

Find the version number of all WordPress installations on your Linux server.

I have a lot of customers running WordPress on our shared hosting servers, and sometimes they neglect to update their WordPress installs. [Rolls Eyes]

I need to know which of these sites are using an obsolete version of WordPress so I may contact the customer and warn them that they need to update their software.

So here’s a helpful little Linux command I whipped up and ran as root to go through my /home folder searching for all WordPress versions. I only had to run it as root because I am checking through all users’ folders, not just my own. If you only want to check your own user, you don’t need root access.

I ran this command from my /home folder on the Linux server:

find . -name 'version.php' -exec grep '$wp_version =' {} /dev/null \; > /tmp/wordpress-versions.log

Breakdown:

  • find . -name ‘version.php’
    Search through the current folder, recursively, for any file named version.php. This is where WordPress stores the WordPress version number.
  • -exec
    Execute a command with each found item.
  • grep ‘$wp_version =’ {}
    Look within the found version.php file(s) in a loop for the term $wp_version = and output the result.
  • /dev/null
    Trick grep into thinking there is a second file, forcing it to precede the output with the filename provided by find
  • \;
    Close the find command.
  • > /tmp/wordpress-versions.log
    Save the results to a log file in /tmp. You can tail -f this file while scanning, or simply open or cat it when you’re done. Leave this portion out of the command if you’d rather have it output directly to your screen.

Memory leak in Zimbra 8.0.6 webmail

I’ve had a suspicion that since the Zimbra 8.0.6 update, something’s been wonky with Zimbra’s webmail client, so I decided to perform a very simple test: open Zimbra Webmail and leave it running.

Here is the outcome of that test.

Normal Operation for one business day.
This is how I operate day after day in my normal office environment.

Running:

  • Zimbra 8.0.6_GA_5922
  • Chromium Version 32.0.1700.123 Debian 7.4 (248368)

Memory Usage at Application Launch:

  • Browser window with Zimbra webmail client
    Thursday 8:47am – 139.5MB
  • Browser window with Google
    Thursday 8:55am – 45.92MB

Memory Usage ~ 24 Hours Later:
I left both browser windows running overnight. Here is where the memory usage stands…

  • Browser window with Zimbra webmail client
    Friday 8:28am – 564.3MB – 304% Increase in 24 Hours
  • Browser window with Google
    Friday 8:29am – 60.78MB – 32% Increase in 24 Hours

Memory Usage ~4 Days Later:
I left both browser windows running over the weekend.  Here is where the memory usage stands…

  • Browser window with Zimbra webmail client
    Monday 10:32am – 1.6 GB – 1,046% Increase in ~96 Hours
  • Browser window with Google
    Monday 10:33am – 60.53MB – 31% Increase in ~96 Hours

Since Zimbra cut the Evolution Connector from its product line, and the Zimbra Desktop software is still only available for a 32-bit platform, this leaves Zimbra operation on Linux sorely lacking. What has VMWare done?!  Hopefully Telligent can fix it.

-Robbie

*** UPDATE March 24, 2014 ***
We’ve ruled out Chrome by itself as the issue since it is only the window containing the Zimbra webmail client that shows any increase in memory usage.

To rule out browser extensions, I will run my next test with all Chrome extensions disabled.
Test 2 – Disable all Chrome Extensions and re-test.

Memory Usage After Weekend:

  • Browser window with Zimbra webmail client
    Friday 5:06pm – 106.3 MB

Memory Usage After Weekend:

  • Browser window with Zimbra webmail client
    Monday 8:37am – 1.7 GB

*** UPDATE April 15, 2014 ***
This has been added as a bug report for 88031 – https://bugzilla.zimbra.com/show_bug.cgi?id=88031

Preventing rsync from doubling–or even tripling–your S3 fees.

Using rsync to upload files to Amazon S3 over s3fs?  You might be paying double–or even triple–the S3 fees.

I was observing the file upload progress on the transcoder server this morning, curious how it was moving along, and I noticed something: the currently uploading file had an odd name.

My file, CAT5TV-265-Writing-Without-Distractions-With-Free-Software-HD.m4v was being uploaded as .CAT5TV-265-Writing-Without-Distractions-With-Free-Software-HD.m4v.f100q3.

I use rsync to upload the files to the S3 folder over S3FS on Debian, because it offers good bandwidth control.  I can restrict how much of our upstream bandwidth is dedicated to the upload and prevent it from slowing down our other services.

Noticing the filename this morning, and understanding the way rsync works, I know the random filename gets renamed the instant the upload is complete.

In a normal disk-to-disk operation, or when rsync’ing over something such as SSH, that’s fine, because a mv this that doesn’t use any resources, and certainly doesn’t cost anything: it’s a simple rename operation. So why did my antennae go up this morning? Because I also know how S3FS works.

A rename operation over S3FS means the file is first downloaded to a file in /tmp, renamed, and then re-uploaded.  So what rsync is effectively doing is:

  1. Uploading the file to S3 with a random filename, with bandwidth restrictions.
  2. Downloading the file to /tmp with no bandwidth restrictions.
  3. Renaming the /tmp file.
  4. Re-uploading the file to S3 with no bandwidth restrictions.
  5. Deleting the temp files.

Fortunately, this is 2013 and not 2002.  The developers of rsync realized at some point that direct uploading may be desired in some cases.  I don’t think they had S3FS in mind, but it certainly fits the bill.

The option is –inplace.

Here is what the manpage says about —inplace:

This option changes how rsync transfers a file when its data needs to be updated: instead of the default method of creating a new copy of the file and moving it into place when it is complete, rsync instead writes the update data directly  to  the destination file.

It’s that simple!  Adding –inplace to your rsync command will cut your Amazon S3 transfer fees by as much as 2/3 for future rsync transactions!

I’m glad I caught this before the transcoders transferred all 314 episodes of Category5 Technology TV to S3.  I just saved us a boatload of cash.

Happy coding!

– Robbie

Running phpcs against many domains to test PHP5 Compatibility.

Running a shared hosting service (or otherwise having a ton of web sites hosted on the same server) can pose challenges when it comes to upgrading.  What’s going to happen if you upgrade something to do with the web server, and it breaks a bunch of sites?

That’s what I ran into this week.

For security reasons, we needed to knock PHP4 off our Apache server and force all users onto PHP5.

But a quick test showed us that this broke a number of older sites (especially sites running on old code for things like OS Commerce or Joomla).

I can’t possibly scan through billions of lines of client code to see if their site will work or break, nor can I click every link and test everything after upgrading them to PHP5.

So automation takes over, and we look at PHP_CodeSniffer with the PHPCompatibility standard installed.

Making it work was a bit of a pain in the first place, and you’ll need some know-how to get it to go.  There are inconsistencies in the documentation and even some incorrect instruction on getting it running.  However, a good place to start is http://techblog.wimgodden.be…..

Running the command on a specific folder (eg. phpcs –extensions=php –standard=PHP53Compat /home/myuser/domains/mydomain.com/public_html) works great.  But as soon as you decide to try to run it through many, many domains, it craps out.  Literally just hangs.  But usually not until it’s been running for a few hours, so what a waste of time.

So I wrote a quick script to help with this issue.  It (in its existing form – feel free to mash it up to suit your needs) first generates a list of all public_html and private_html folders recursive to your /home folder.  It then runs phpcs against everything it finds, but does it one site at a time (so no hanging).

I suggest you run phpcs against one domain first to ensure that you have phpcs and the PHPCompatibility standard installed and configured correctly.  Once you’ve successfully tested it, then use this script to automate the scanning process.

You can run the script from anywhere, but it must have a tmp and results folder within the current folder.

Eg.:
mkdir /scanphp
cd /scanphp
mkdir tmp
mkdir results

And then place the PHP file in /scanphp and run it like this:
php myfile.php (or whatever you ended up calling it)

Remember, this script is to be run through a terminal session, not in a browser.

See what we’re doing there?  Easy breezy, and solves the problem when having to run phpcs against a massive number of domains.

Let me know if it helped!

– Robbie

Walk-in Wifi Responder

Had a thought this morning that wifi could be used to do some pretty rad stuff… like detecting when I get home by noticing my iPod touch.

Since most of us carry wifi-enabled devices with us at all times, and most of us have them set to auto-connect once in range of our routers, I thought, why not use that data?  It could be as simple as logging coming and going, or as sophisticated as automatically turning on my favorite music when I walk in the door.  Or even adjusting the thermostat when I arrive home to save energy when nobody is around.

As a very brief proof of concept I whipped out a simple algorithm in PHP which can be run from any Linux computer on your network.

Usage:  php wifi-check.php –device=devicename

My thinking is to put something like this in a looping script and let it run every so many seconds or something, calling particular functions if the device is detected as active vs inactive.

I’d welcome your thoughts in the comment section below.  What practical things could this be used for?