Tuesday, November 08, 2016

Convert your CMS site to static in seconds with a simple linux command line

This little tip will hopefully be helpful to webmasters with deadbeat clients out there.  Scenario: you host site xyz.com for Joe Smith.  The site runs on ancient version of Joomla, which relies on an old, unsupported version of PHP. It's a pain to update, edit, and worst of all is a huge liability.  You might get paid to get the site reactivated, but you haven't heard from Joe in long enough that you suspect you won't.  You've decided to leave the site functional for another month.  Meanwhile, you have a webserver to migrate.  What is the easiest way to move the site without breaking it?

  1. Run an old version of PHP.  There are lots of problems with this option, which I don't think I need to go into.
  2. Convert it to a static
Imagine if you could have a working snapshot of the site and all of its resources. Images, javascript widgets, CSS, content, everything!  The site is a lot harder to edit (although not impossible), but you are no longer anchored to any PHP version (heck, you don't even need PHP at all), and there is no vulnerable CMS waiting to be compromised, trash your webserver and your reputation.  Brilliant!  But how to do this simply?

There are plugins for various popular CMS which claim to do this.  I have tried some and found mixed results.  In the end it was not what I hoped for in terms of simplicity or end result.

...Enter wget!  A very common linux command-line tool which can suck down websites.  Normally it only downloads content or a single file, but it turns out there are a bunch of argument permutations which can be applied in order to entirely and statically duplicate a living, breathing website. It goes something like this:

wget -P . -mpck --user-agent="" -e robots=off --wait 1 -E http://www.site.com/

The gist of this command is as follows:
We turn on mirror for recursion and time-stamping, get page requisites to get things besides just the content, such as script files, css, images, etc., continue which will resume a partial download of the site, and convert links which will convert absolute links so they work on a new or local site.  We also ignore robots.txt files which might restrict crawling, and wait 1 second so we don't hammer the other server too hard.

The result is a folder containing all of the files (images, html and scripts) required by the site, placed perfectly to create a 100% working, but static representation of the original site.

Much thanks to Jon Bickar for his post which accurately describes how to use wget this way.

Wednesday, April 06, 2016

Reaper, Linux, and the Behringer X-Air - Complete Studio Solution, Part 1

Introduction and Rationale

This is part one of a major effort to document my experiences with recreating my home studio, entirely using Linux.  Without getting into too many of the specifics, a few months ago I decided that I was unhappy with Windows' shenanigans - to the point that I was ready to make a serious attempt to leave it behind.  For most in this situation, the obvious choice is to switch to Mac OS.  With its proven track record, support, and options for multimedia production, it is naturally the first alternative to consider if your goal is to simply use something other than Windows.

For me the choice was not so simple. I despise Mac OS and, in general, the goals and philosophies put forth by Apple in an effort to ostensibly provide users with an "easy" working environment.  It does not help that I have also failed to find any aspect of the Mac OS UI intuitive, but I realize that this is a subjective matter.

With my IT background and user-control* favoring outlook, I decided on Linux.  I did this knowing that the road ahead would be paved with many bumps. The Linux audio world is a complex one, and while most (if not all) of the information to do what you want is available, it generally requires a lot of knowledge and a pioneering spirit to get there.  Hopefully this post will help you with the former, but the latter is something that you should expect to bring.

The spoiler that the impatient are probably hoping for is that, at the time of this writing, I have been able to achieve 99% of the functionality that I had with Windows.  The remaining 1% is functionality that I will either be able to achieve with a little more time investment, or functionality that I really didn't need that badly to begin with.

*I am a fan of the free software vision evangelized by Richard Stallman.  However, considering the availability of hardware and software, this project is by no means anything that would be considered "free" in his vision.  I have settled on a solution that consists of  the "best compromise" between what I found to be available, and the functionality I needed.  I would consider it to be significantly improved over the previous level of "freedom" I had when running Windows. 

Overview - Hardware and Software Components

Naturally there are some other major particulars involved with the setup I went forward with. Mainly:

    Behringer XR18
  • The relatively recently introduced Behringer X-Air, primarily due to its hardware flexibility and manufacturer-stated support for Linux (as well as Android, and basically ALL other major platforms).  In particular, I chose the XR18 based on the coinciding factors of cost, portability, number of channels, number of those equipped with XLR/TRS combo-jacks, and other features.  However, this project could presumably be used with any of the Behringer X-Air products with only minor modifications to the configurations, based on the specifics of the hardware.  As a USB audio device, I had concerns as to how this device would perform in terms of latency, but with sufficient tuning, I have found it to be a suitable even in situations involving DAW post-processed talent foldback. 

    Reaper - a powerful, low cost DAW that can run great on Linux
  • The Digital Audio Workstation software REAPER, which has been around for a decade, but is still something of a cult interest for whatever reason.  It is extremely powerful, flexible, and makes no assumptions about how you would like to approach multimedia production.  The cost is minimal ($60**) when compared to other DAWs of similar function (Logic, Protools).
**Granted, this is the "discounted" license, which most readers of this article will qualify for.

  • Reasonably powerful PC workstation
    • I won't spend a lot of time on this topic.  Reaper itself has very modest hardware requirements, which you can read about.  However, considering the layers required and the cost of PCs vs. performance and capacity, I do recommend that you overbuild your workstation to meet at least the following specifications:
      • Intel Core i5 quad core (64 bit) or equivalent processor
      • 8GB RAM
      • 1 Terabyte drive
    • In my case, the system I used has the following specifications:
      • Intel quad core Xeon W3565 @ 3.20GHz
      • 512 GB solid state boot drive
      • 4TB (2TB SSHD x2) w/ hardware RAID 1 data array
      • 16 GB RAM
Let's talk about some of the other key ingredients that will be necessary to create our complete studio solution.

Ubuntu 14.04 LTS, 64 bit

This is a very popular desktop-oriented Linux distribution that has "long term support", and was fairly current at the time I pursued this effort.  Certainly other distributions of Linux would be usable, but this is the one I will use as the basis for this project's examples.


In the world of Linux Audio, there are several layers of audio systems available in order to accomplish delivery of digital audio to and from your audio hardware.  However, if you want to do any serious audio production, JACK will be an important part of your complete breakfast.  There are two versions of JACK available, and in this project we will be using JACK 2.


So here we must present what will come as a fairly off-putting fact: REAPER doesn't ACTUALLY run natively on Linux,  If you want to run it on Linux, you must use the WINE Windows Compatibility layer. This is essentially a translator between Linux and Windows applications. It was off-putting enough for me that I resisted this experiment for years.  While I have found many simple Windows programs able to run well on WINE, to me there were many concerns, including performance, architecture support, reliability, and many other things.  But have no fear.  If you have all of your ducks in order, REAPER does run on Wine, and it runs very well.  


Since REAPER uses the Windows-based ASIO audio system, and Linux will use JACK, we need a bridge to allow these two to exchange audio.  WineASIO provides a seamless bridge that satisfies this need.

Other Supporting Linux Packages 

In support of the above, we will need to install several other components on our target Linux machine in order to configure and otherwise facilitate your most common needs.
  • ALSA - This is the lowest level for sound interfaces to connect to applications and other audio layers, at least it has been for many years. It is how Linux interfaces to your audio devices.  ALSA actually provides all of the basic functionality one needs to use an audio device, but there are other audio systems which can enhance this functionality, such as JACK and Pulseaudio. In this project, these systems will sit on top of ALSA, and for the purposes of following discussion, we will infer that connections to the hardware device itself are done through ALSA.
  • Pulseaudio - In the context of a strictly recording-studio oriented computer, this component is technically optional. However, in my circumstance, I use my computer for daily tasks such as web browsing and VOIP communications, and I did not wish to force all applications, such as browsers, simple audio players, headsets, webcams, etc. that require the use of audio to interface with JACK.  JACK, while very low latency and highly configurable, is not a friendly beast, and does not outwardly provide multiple applications the ability to coexist on a machine with a single main audio device, or multiple audio devices serving a single application.  This is a strength of Pulseaudio (although I certainly have a wishlist of how it could better serve these endeavors).
  • pulseaudio-module-jack - We will be using JACK to connect directly to our audio interface (the Behringer XR18).  Unfortunately, this is at odds with Pulseaudio, which, in the most common configuration (as in a stock Ubuntu distribution), also expects to connect directly to the hardware.  This cannot be.  The jack module allows Pulseaudio to instead connect to the audio interface via JACK (which connects to the audio interface via ALSA. As I warned you, the Linux audio world is complex).
  • pavucontrol - Pulseaudio is a powerful way to manage applications and audio devices, but to use its strengths, we need a way to control it.  Pavucontrol is a graphical user interface which allows us to decide how audio and devices should be connected.  Again, this is technically optional if you are not planning to use Pulseaudio for convenience applications.
  • qjackctl, and cadence - As previously mentioned, JACK is sort of an unfriendly beast, and we will need these tools to help wrangle it to serve our needs.
  • linux-lowlatency - When working with audio devices in a recording studio environment, you will inevitably need the lowest latency possible.  The default kernel in a desktop distribution is designed to balance performance for multitasking.  This package is an easy way to replace the default kernel with one that is well suited to live audio production.
  • Behringer X-Air-Edit - This is a program provided by Behringer to control the XR18 (or other X-Air interface).  It is available as a native Linux executable. It will be the equivalent of a patch panel within the X-Air hardware itself.
This concludes part one .  In the next installment, we will get into the details of each of these components, and how they will individually serve our needs.

Friday, January 01, 2016

Windows 10 and hardware incompatibility

A few months ago, I replaced my laptop and my desktop.  The move was motivated by a need to reinstall the OS (Windows 8) due to slowness and broken system components.  My desktop was particularly nasty - the Windows update mechanism broke, causing it to take 3 hours to boot up.  I spent the 2 months leading up deftly avoiding reboots, lest I lose a half-day of productivity.  Reinstalling the OS is somewhat traumatic, as I have lots of configuration to do, so I decided that I might as well update my hardware as well.  I also decided 2 other things:
  1. Try to switch away from Windows dependency as much as possible.
  2. For things that require Windows, try Windows 10, since it will eventually be the only Windows choice anyway.


On both my laptop and desktop, I am running Ubuntu 14.04 LTS - and I have to say, I am doing remarkably well.  To my amazement, I can still do all of my mixing in Reaper, on Wine.   The MOTU 896 FW interface is another story - Apparently MOTU is not friendly to Linux, so I have gotten rid of it and gotten a Focusrite Sapphire Pro 40 instead.  

There are a few things I need Windows for - such as Quickbooks, Corel Draw, Sketchup, Vegas, and some apps I need to run for development work... But for now I am doing okay by running these things in a Windows 7 Virtualbox VM.

Windows 10

I have also set up dual boot to Windows 10 (on demand, not the default) on my desktop and laptop.  I wish I could report as much success with my Windows 10 tests.  Unfortunately, I have had no joy, due to the following issues thus far:
  • Touchpad driver in my new HP Spectre x360 is buggy (jumpy and misses clicks). No fix found (Note: this may seem minor, but stuff like this really pisses me off, and reinforces the fact that paying top dollar for a laptop doesn't mean everything is going to work right, or even better than a cheap laptop).
  • Behringer X-Air audio driver is buggy (audio interface shows up intermittently, and sometimes stops responding in the middle of a recording). 
  • MOTU 896 doesn't work on Windows 10 either.

All told, these issues are significant the point where I will have to roll back to Windows 7 or 8 until things settle out a little more. 

Sweetwater.com has published this list - sort of a "state of Windows 10 compatibility" for various recording hardware.  As of now, it's rather bleak - with some pretty notable entries not being compatible at this time.  But I suppose if you are a "real recording engineer", you only use a Mac anyway, right? (Bleahhhh...)

It is pretty clear that, despite Microsoft's big push to roll out Windows 10, there are hardware vendors who are just not ready.  If you are on the fence and depend on a lot on certain hardware, make extra sure that everything works perfectly before you plan to leap.  However, since that's probably a lot of trial and error, I would just hold out as long as possible, until the hardware vendors get a chance to catch up.

Bonus tip:  You can take advantage of Microsoft's free Windows 10 upgrade offer without having to switch! 

I may go into more detail in another post, but here are the steps in a nutshell:

  1. Temporarily install a blank hard drive in place of your existing hard drive.
  2. Install a fresh, stock Windows 7 or 8 onto it (must be the exact version that came with your computer), and activate using your product key.
  3. Upgrade to Windows 10  - this will create a NEW product key associated with your hardware fingerprint, and stored on Microsoft's servers.  If you ever want to install Windows 10 on this computer in the future, it will use the same product registration automatically. 
  4. Remove the drive and replace with your original.  You don't need to keep Windows 10 installed on the spare drive, but it may be handy.
  5. Switch to Windows 10 whenever YOU want - even after the free upgrade deadline.