Next/Previous episode buttons when viewing the show notes page for any episode. – Done
Photo gallery re-write with grid-style gallery layouts.
RSS feeds to begin stating the age of an episode (eg., ’1 Week Ago’ or ’3 Years Ago’). – Done
Add Google-indexable breadcrumbs to Episode pages. – Done
Season 2 Episodes made available on-demand and in RSS feeds.
Season 1 Episodes made available on-demand and in RSS feeds.
Replace the main menu script with something more robust, to accommodate the growing list of menu items and introduce third-level navigation. – Done, sortof. Rather than replacing the menu, I recoded bits of the existing one to add new functionality. I think it works quite nicely.
Replace the jQuery scroller at the top of the home page with something a little more current. – Done, sortof. I decided that for the moment, I still like the scroller we have, and it’s lightweight, which is a plus. So instead I just improved it slightly. Buttons have been changed to our orange color to make them stand out more, too. I have begun coding a new “animated” version to replace this one eventually, but for now I’ve bought some time.
Make the giant header slideshow on season pages select a handful of random images from the season rather than the most recent ones.
Beginning with Episode 231 (February 21, 2012) we have been keeping a log of the Category5 TV chat room during the live broadcast, complete with timestamps.
These logs have been available to view for a while now via the show notes page for each episode since that time, but I’ve always wanted to take it a step further.
Effective immediately, logged-in registered viewers may enable the chat logs as their source for closed captioning.
This is a two-fold success. First, it adds a pretty neat feature to the on-demand video since logged-in viewers can now see what was said at the time during the live show, overlayed on the video, but secondly, it allowed me to generate an effective closed captioning system on our web site. I hope to eventually offer English (and electronic translations to other languages) captions, and this provides the back-end technology to do this.
To enable this feature on your account, login to Category5.TV and visit your profile page. Once their select “Chat Logs” as your source for closed captioning.
Make sure you’re logged in, as this is an exclusive feature to registered viewers.
Please offer your feedback below. What do you think of this feature?
Hope you’ve been enjoying the benefits of the new transcoders the past couple months, along with the absurdly fast distribution backend we added to the RSS feeds!
I’ve been feeling the waters now for a little over 2 months and am implementing some changes, effective immediately.
Our Goal: Every Episode for Free One of the goals at Category5 TV is to always offer all our episodes free of charge to the viewers, and in a very accessible way (or many ways to accommodate many different types of users).
The Problem With Our Goal
Our new distribution mechanism is pricey. Category5 TV stores over a terabyte of video “in the cloud”, and we have a very active community of happy downloaders. A lot of those downloaders like to grab every episode, especially people who have just found the show and want to see all the oldies. For every person who downloads the whole series, we pay for that download, and it has become substantial. Too substantial to maintain.
So, I’ve been contemplating… how do we do it? Do we start charging for downloads? Obviously not! That’s not in line with our goal to offer the video to you free of charge. Do we restrict viewers to only being able to download say, the last 5 episodes? Also a resounding “no!” as that would not be in line with our goal to give you the wealth of information that is available through Category5 TV.
And then it hit me.
It’s not the week-to-week downloads that cost us too much money in bandwidth fees… it’s the big downloads. The people who subscribe to the RSS feed (especially HD) and download all the available episodes. That’s about 90GB per user. And there are a lot of them.
So, my solution: effective immediately, the most recent 5 episodes of Category5 TV will be distributed through our fast, expensive CDN. This ensures the viewers who are watching week after week get the fastest, most reliable delivery service, even if they miss a few weeks and need to catch up.
All episodes from more than 5 weeks ago will still be available, but will be hosted on our slower network (which costs us substantially less, but is quite slow for distribution, and a little less reliable). This means those wanting to download all episodes will get them much more slowly, but they’ll get them none the less. This will save us several thousand dollars per year in bandwidth fees, which I think you can appreciate.
Potential Issue & Solution
Many users have their RSS aggregator setup to automatically download new files as they hit the RSS feed. Every week as a new episode is added, the 6th episode in the feed gets moved to the other distribution servers. This means that aggregator may download both the new episode (1st in the list) plus the newly moved episode (6th in the list), even though it has probably already been downloaded. The new URL makes the aggregator think it’s a new file.
The solution? Only subscribe to the most recent files. Update your feed URL, adding /fast to the end of the URL. For example, if you’re using the SD feed (http://rss.cat5.tv/sd), change your feed to http://rss.cat5.tv/sd/fast. This will make sure you only receive files off the fast CDN, and your aggregator will completely ignore anything moved onto the other distribution platform. You’ll receive the new episode every week, and the last of the most recent 5 episodes will simply fall off your list.
I am toying with the idea of offering optional premium accounts for a reasonable fee, which would allow a user to download the whole RSS feed directly from our super-fast CDN, and am even giving some thought to providing the series on Blu-Ray if there is a desire from the community to have such a thing. What are your thoughts on premium accounts, keeping in mind that we will always offer Category5 freely to the community? But those who offset the high expense of distribution get some added perks.
It costs us about $0.25 for each download of an HD episode of Category5. Would you pay $1 per month to offset that cost? $2 a month to more than pay for usage? What are your thoughts?
As you may have noticed, I had to slow down the transcoders.
Yes, they operate super fast and can transcode all our back episodes in a very short time as previously reported.
However… they cost a lot of money! Le gasp!
As you can imagine, Category5 TV back episodes take up a lot of space on the web. Over a terabyte as a matter of fact. So transcoding, uploading and distributing all 300+ episodes all in one fell swoop was a bad idea from a cost perspective.
So I slowed things down to a more manageable price point so I can space the cost out over multiple months.
As you know, I’ve worked hard to bring you the new transcoders the past while.
This meant introducing a whole new distribution model, with all new servers and a redeveloped distribution infrastructure for Category5 TV (yes, we outgrew our previous solutions).
We have a number of main distribution nodes, but there are three primary content delivery networks. We’ll call them CDN 1, CDN 2, and CDN 3.
During the initial stages of the transition, we placed our RSS feeds, Miro Internet TV and other “downloaders” such as the direct download links on CDN 1. It is our slowest, least reliable CDN, but it is also the cheapest for us to operate.
CDN 2 was not in use yet, and CDN 3 is our fastest, most reliable and most expensive system. CDN 3 was implemented as our on-demand distribution node. This means if you watched the show via our web site or a player embed, you were watching it through CDN 3.
Miro Internet TV in particular was experiencing some speed issues pulling form CDN 1, and the past two weeks one of our syndicate partners was also having trouble pulling the HD file down from the CDN 1 direct download links as provided by our weekly email.
So, I have now replicated everything onto CDN 2, which is much faster and more reliable than CDN 1, but not nearly as expensive as CDN 3 to operate.
I have moved all RSS feeds, Miro Internet TV feeds and direct download links onto CDN 2 and now CDN 1 becomes simply a redundant option for us to use should either of the other CDN’s become unavailable.
All this to say, you should now notice that your downloads are faster, and that you don’t experience timeouts when downloading the larger HD files from our services.
Thanks for your patience as we worked out the kinks.
As a computer and security specialist, I see a lot of viruses and malware. But more often than not, the removal of the malicious code from a computer system repairs the issue. A new ransomware application has popped up however that raises some real concern, because it in fact destroys your data in a seemingly unrecoverable way, and removal of the malware simply leaves your files in an inaccessible state with no chance at recovery.
What is it?
CryptoLocker leaves your files inaccessible and unrecoverable.
CryptoLocker is a new and cunning piece of ransomware discovered last month. Its spread is increasing, and we’re starting to see infections in a growing number of unrelated networks here in Ontario.
CryptoLocker needs to be taken very seriously, because it can result in the total and irreversible destruction of all your personal and company files.
What Does It Do?
CryptoLocker places itself on a Windows machine, easily circumventing even the best antivirus protection, at least at the time I write this. It appears to get in by way of an infected web site or possibly an infected email attachment masquerading as a seemingly legitimate file such as tracking data for a courier shipment, a money transfer or other fake electronic money transaction.
Once infected, the malware crawls through all mounted volumes (hard drives, network shares, USB drives, camera cards, etc.) for a variety of filetypes, mostly documents, spreadsheets, PDF files, pictures, etc., and encrypts them. This means the files on your own hard drive, your network mapped drives, and even cloud-based drives are encrypted (destroyed, made unreadable). Because the decryption key is not known, recovery is not an option.
Once the encryption process is complete, the software then launches an application window displaying a message that all your files have been locked, and you must pay the ransom ($300 is common right now) in order to recover your files.
Current, up to date antivirus tools detect the trojan and remove the malware software after the damage is done to your files. This results in the permanent inability to recover your files.
Perhaps the best way to explain the devastating effects of CryptoLocker is with a couple of fictitious scenarios:
A small business has a two-drive RAID mirror unit in their server as a form of backup. They have one extra drive, and the system features a removable tray caddy. This allows them to swap one of the hard drives each day and take it off-site.
One staff member was working on the system that morning and received an alert that their data had been encrypted after they opened a suspicious email attachment. They closed the alert and left the room.
The manager arrived an hour later and removed the second hard drive from the array, replacing it with the one they brought from home: their morning routine. The drive rebuilt based on the first drive, which now contains only encrypted data, and now all three drives are corrupt. All files are lost, including their backup.
A business office with a shared folder on the server uses that share for all their company data. Every workstation in the office has the share mounted to their Q: drive. This contains Excel spreadsheets, Word documents, PDF catalogues, product pictures and more.
The company feels this is a good way to manage their internal files since it gives all staff access to the files, is a RAID 1 mirrored drive (so if a hard drive crashes, they lose nothing) and it allows them to backup one single folder to the external backup drive on a nightly basis, resulting in the backup of all critical files.
One staff member is wrapping up their shift and quickly uses their computer to search for discounted tickets for an upcoming concert. They do a search in Google and start clicking on all the results to see which one offers the best deal, unconcerned about the fact that they do not recognize even one of the web sites as a reputable ticket source. Unbeknownst to the user, one of those sites is infected with CryptoLocker, which installs itself in the background while they search.
CryptoLocker silently goes through C: and corrupts every document, every spreadsheet, practically every personally-created file. It then finds the Q: drive and gets to work doing the same: corrupting all user files on the network share.
The following morning the user returns to work and finds an alert on their screen saying all files have been encrypted, and they immediately recognize it as being a virus of sorts. They run their virus scanner and it removes the infection without any problem. They go about their day.
All the while, other users on the network start to complain that they can’t access their Q: drive. IT has a look and finds that all files are corrupt and unreadable. They look at the backup drive connected to the server, and it too has been corrupted due to the previous night’s backup. All files are lost, including their backup.
What Can You Do?
If you have already been infected with CryptoLocker and do not have an unaffected backup, unfortunately there is nothing that can be done. It is not recommended that you pay the ransom, nor is there any guarantee that the hacker responsible will actually unlock your files if you do pay (some users have reported having paid the ransom and yet never got their files back).
So it all comes down to preventative measures: protecting yourself from this malware before you get infected.
Backup, backup, backup
I’m not just saying it three times for emphasis. I really mean it: you should have more than one backup solution in place.
Realistically the only true protection against the effects of CryptoLocker and similar viruses is to have a multi-tier backup system protecting the integrity of your files at all times.
Since the files on your drives and network are basically destroyed by CryptoLocker—possibly including your backup—the easiest, safest, and most assured way to recover from an infection should it occur, is by having a detached, unaffected copy of your files.
An off-site backup solution is likely the best option. It means your files are safely stored elsewhere, and if done right, they are stored incrementally. This means if you get an infection and CryptoLocker destroys all your files, and then your backup runs, your good backup does not get overwritten, as would be the case with both scenarios listed above. With an incremental backup, you can in fact restore from days gone by—from before the infection took place.
There are many off-site backup services out there, and I don’t want this to seem like a sales pitch—I genuinely just want you to be safe—so feel free to shop around. But all I ask is that you please include Positive E Solutions in your list of companies to check out. They have a very good, fully encrypted off-site backup service with hosting entirely in Canada. It can be used in conjunction with your existing backup infrastructure to leverage its effectiveness and further protect your critical data. It’s very affordable for either business or home use, and I can even let you try it for free for 30 days to see if it meets your needs. http://positiveesolutions.com/try-now.php
Enable Volume Shadow Copy
Volume Shadow Copy may help you recover from a CryptoLocker attack if it is enabled on the affected folder prior to the corruption taking place
Windows 7/Server 2008/Vista/Server 2003 have a feature called Volume Shadow Copy. It’s not to be mistaken for a backup, but it is a helpful tool in recovering from this type of infection: essentially a duplicate of the files found on volumes you specified to have shadowed. In the event of a CryptoLocker attack, your files are destroyed from their original locations, but the Volume Shadow Copy is untouched by the current incarnation of CryptoLocker, due likely to the special permissions required to write to the Volume Shadow Copy itself. Therefore, following the removal of CryptoLocker, you can right-click on the affected files or folders and revert to an earlier snapshot.
There are a ton of tutorials out there which teach how to enable Volume Shadow Copy, so I’ll avoid making this one of them. Activating Volume Shadow Copy helps reduce recovery time should a CryptoLocker infection take place.
It is a good idea, I think, to enable Volume Shadow Copy at the server level, directly on the volume containing your network share folders. In Scenario 2 above, this would be the RAID 1 which contains the contents of their Q: drives. That way, the shadow copy could be used to quickly restore to a previous set of files. If that doesn’t work, the backup can be used.
Update Flash and Java, But Disable Java in your Browser
I had a discussion with malware expert Adam Kujawa yesterday about CryptoLocker. He mentioned that Java and Flash are two of the main ways this virus is able to enter a Windows system. An unsuspecting user might conduct a search for something in Google, and click on a few links, and one of those web sites could be infected with the distribution mechanism to install CryptoLocker on your system. The recommendation is to disable Java from your web browser (only enabling it when needed), and absolutely keep both Java and Flash up to date.
Keep Your Antivirus / Anti-Malware Up To Date
The instant they release protection for this, you want to receive it. This is not a replacement for my backup suggestion above, but will save you some headaches.
Be Careful What You Click
We have received reports that CryptoLocker infections originated both from infected web sites and emails. It’s tough to ensure entire staff are cautious, but it’s still important for me to mention. If something appears suspect, don’t click it. If you receive an email you’re not expecting, don’t open it. If “your bank” sends you transaction details for a transaction you don’t remember making, don’t click the links. Just be careful what you click. These infections are able to circumvent the antivirus.
Mac and Linux Users
While CryptoLocker does not directly infect Mac or Linux machines at this time, these systems may have network-accessible file shares open to the network or a virtual machine. Therefore if a Windows computer on the network or a Windows virtual machine becomes infected with CryptoLocker, it is possible to lose the files hosted on your Mac or Linux computer (or NAS device).
Cloud Users Beware
CryptoLocker will crawl through and destroy personal files on cloud-based mapped drives such as Google Drive, PogoPlug or DropBox.
Having made his first international appearance during Episode 204 of Category5 TV Tuesday August 16, 2011, our studio mascot Space Fish, Major Tom passed on this day, Friday October 4, 2013. He spent 2 years, 1 month, 19 days with us in-studio.
Major Tom will always be remembered fondly for his colorful appearance. He also had a distinct talent for stinking up the studio despite our futile efforts to keep his habitat clean.
Major Tom’s final appearance on the live broadcast took place during Episode 283, Tuesday February 19, 2013.
Using rsync to upload files to Amazon S3 over s3fs? You might be paying double–or even triple–the S3 fees.
I was observing the file upload progress on the transcoder server this morning, curious how it was moving along, and I noticed something: the currently uploading file had an odd name.
My file, CAT5TV-265-Writing-Without-Distractions-With-Free-Software-HD.m4v was being uploaded as .CAT5TV-265-Writing-Without-Distractions-With-Free-Software-HD.m4v.f100q3.
I use rsync to upload the files to the S3 folder over S3FS on Debian, because it offers good bandwidth control. I can restrict how much of our upstream bandwidth is dedicated to the upload and prevent it from slowing down our other services.
Noticing the filename this morning, and understanding the way rsync works, I know the random filename gets renamed the instant the upload is complete.
In a normal disk-to-disk operation, or when rsync’ing over something such as SSH, that’s fine, because a mv this that doesn’t use any resources, and certainly doesn’t cost anything: it’s a simple rename operation. So why did my antennae go up this morning? Because I also know how S3FS works.
A rename operation over S3FS means the file is first downloaded to a file in /tmp, renamed, and then re-uploaded. So what rsync is effectively doing is:
Uploading the file to S3 with a random filename, with bandwidth restrictions.
Downloading the file to /tmp with no bandwidth restrictions.
Renaming the /tmp file.
Re-uploading the file to S3 with no bandwidth restrictions.
Deleting the temp files.
Fortunately, this is 2013 and not 2002. The developers of rsync realized at some point that direct uploading may be desired in some cases. I don’t think they had S3FS in mind, but it certainly fits the bill.
The option is –inplace.
Here is what the manpage says about –inplace:
This option changes how rsync transfers a file when its data needs to be updated: instead of the default method of creating a new copy of the file and moving it into place when it is complete, rsync instead writes the update data directly to the destination file.
It’s that simple! Adding –inplace to your rsync command will cut your Amazon S3 transfer fees by as much as 2/3 for future rsync transactions!
I’m glad I caught this before the transcoders transferred all 314 episodes of Category5 Technology TV to S3. I just saved us a boatload of cash.
Well, we’ve been on the new transcoders for one week now, and I’m excited to see the impact.
Last night was the first night where I was able to initiate an automated transcode of an episode shortly after we signed off the air.
There are still some things I need to work out. For example, I could not initiate the conversion until I had imported the photos, because the transcoder uses the episode’s image for the ID3 cover art on MP3 transcodes. So after I finished choosing and uploading the images for last night’s show, I fired the transcoder.
It was less than 8 minutes after I initiated the transcoder that the MP3 RSS feeds received the new episode. Just a little more than 48 minutes after initiating the transcoder, the Low Definition (LD) file completed. The show went up on the web site almost immediately after that (the files first get sync’d to our CDN and then added to the database, automatically).
All files (MP3, LD, SD, HD and WEBM) were complete in just 2 hours 31 minutes 4 seconds, including all distribution, even cross-uploading to Blip.TV (also automated now).
From 17 hours to only 2.5 hours. This thing is incredible.
And that means, on average, we’ll be able to transcode nearly 10 episodes per day — almost double the turnaround of our first week. That means the job which was estimated to take 72 days on our main server alone has been cut to only a day or two longer than one month. In just one month from now, all back episodes – six years worth of Category5 TV – will be transcoded.