Valentine’s Day 2014: Wood Carving

This year’s Valentine’s day video is going to be a little short, I’m afraid. Over the past three weeks or so, I’ve been playing with several ideas. My main goal was to have decent, full HD video for my video, since I found the video quality in my previous videos somewhat lacking. Since I didn’t have much luck finding a decent video camera to use, I decided that a still camera would have to do. Raspberry Pi in hand, I set about creating a time-lapse rig using my dad’s DSLR camera.

Raspberry Pi

IMAG0054

It turns out that there’s a great camera control library called gphoto2, which will not only download videos from a variety of cameras, but is also capable of controlling the shutter on many Canon and some Nikon cameras. Once I had that installed, creating time-lapse videos was a fairly simple matter of one command: watch -n 10 gphoto2 –capture-image. This captures one picture every 10 seconds. My first test, watching an ice-cube melt, went pretty well, despite the fact that the tripod was a bit shaky.

In the end, I didn’t have time to build a power pack for the Raspberry Pi, so I couldn’t film without a power outlet, which is inconvenient to say the least. So, in the end, I decided to record a time-lapse video of my carving a piece of wood. Bare in mind, this is the first thing I have carved EVER. So it’s not that great, but the video is cool, at least.  It took me about an hour and 15 minutes to carve out the drawing I made.

The final obstacle was that Adobe Premiere decided that it couldn’t possibly load any video format that I threw at it, so unfortunately I couldn’t edit the video at all. so there’s a single picture at the bottom of this post showing the final result.

So, without further ado, here’s your moment of zen:


Finished Carving

Which Distro? An Introduction to Picking “a Linux”

Every few days, someone on the Linux users group on Facebook posts a question that goes something like this: “I’m new to Linux. Can you recommend a good distribution for blah?” Where blah is usually something like gaming, media, or learning Linux. Like many people who are new to Linux, when I was first exploring the Linux world, I tried out a lot of distros. Eventually, I came to the conclusion that it doesn’t matter which distro you choose. I’ll explain a bit more about what a distro is, what kinds there are, and why it does and doesn’t matter, below.

What is a Distribution?

It’s hidden right in the name, but it’s not immediately obvious: a distribution of Linux is the Linux kernel, packaged up and distributed in a usable form. Linux, by itself, is just a kernel: the part of an operating system that manages the hardware, and provides interfaces for the various pieces of software and hardware that make the computer usable. If you were to just boot the kernel all by itself, the machine would start and do nothing. It would spin the fans and hard disks faster or slower, handle new device connections, and maybe even accept user input, but the computer wouldn’t be usable, because the software that interacts with the user (you) is not part of the kernel – no text terminal, no on screen menus or mouse pointer, no nothing. Once the kernel is running, it spawns other programs that handle user interaction. This is a basic model of operating systems.

So, apart from some basic patches and alterations that the makers of Linux distros might make to their specific release of the kernel, the underlying kernel is basically the same from distro to distro. The difference is in the software they install around it, and that’s also one of the reasons Linux is more customizable than any other popular operating system today. For example, there are quite a few different graphical interfaces that support Linux: KDE*, Gnome, Unity, XFCE – these are all visually different interfaces that function differently. And that’s just a very, very small portion of the desktop environments available. So two different distros with the exact same kernel can look and behave very differently on the surface. This is, to a large extent, how various Linux distros differ: the packages included with the base system, and their initial configurations. Ubuntu and Kubuntu, for example, are two distinct distros, yet they are mostly the same except that Ubuntu ships with the Unity desktop environment, and Kubuntu ships with KDE. One could easily uninstall Unity from Ubuntu and install KDE.

Major Differences: Package Management

Arguably the most fundamental distinction between various distros is the way they facilitate software installation. On a base Linux system with no package manager, the way you install packages is by copying the executable binary and it’s shared objects to the proper locations. This isn’t the easiest or most convenient thing to do, as it doesn’t allow you to easily keep track of what software package is installed where, or what version it is. So most Linux distributions ship with a package manager like apt (Debian-based), yum (Red Hat), portage (Gentoo), or pacman (Arch). These package managers will not only install packages from a central repository – all of which has been checked so that it is compatible with all the other software in the repository – but they will install all the dependencies as well. Everyone who has ever tried to install a package from source on a freshly installed system will tell you that this is a great relief, and saves hours of hunting online for tarballs**.

Which package manager you choose is largely a matter of preference, and this is the basis on which I think you should make your decision. Debian-based distributions come with “apt”, which is – from my biased perspective – a reliable, relatively easy to use package manager. Distribution upgrades (i.e. major upgrades) can be done without nuking the system and starting over, it has support for multiple architectures on the same system (e.g. installing 32-bit packages on a 64-bit machine), and is pretty painless. Even the most frustrating problems can sometimes be solved by throwing around a bunch of apt commands semi-brainlessly.

Red-Hat/Fedora-like distributions (like RHEL, Fedora, OpenSuSE, CentOS, and Oracle Linux) use “yum”, which installs rpm files. The last time I used yum was years ago, so my opinion isn’t worth much in this regard. I’ve heard that you have to nuke a Fedora system in order to do a distribution upgrade – that is, you have to do major upgrades by wiping the system and starting over – but I don’t know if that’s the case anymore. If you’re a beginner, I’d recommend using one of the other options, but this is my wholly biased opinion; take it with a grain of salt (and maybe a trial of Fedora on a VM).

Gentoo (and probably some other distros based on Gentoo) use “portage”. Portage is pretty freaking cool, in that it compiles every single package from scratch. It’s a somewhat agonizing experience (although not as much so on today’s faster machines), especially if you want to install a huge software package like KDE. But the benefit of doing things this way is that every binary on your system is optimized specifically for the box sitting in front of you (or under your desk, or wherever it is you have the thing). It’s more useful if you actually know what you’re doing, and can manipulate the various compiler flags effectively, but I’m sure there’s some speed-up even if you don’t entirely know what’s going on under the hood. Gentoo is my favorite distro for learning the ins and outs of Linux, and if you’re a first-timer and really want to dive into Linux and get a good head start, I can’t recommend enough that you take the time to do a full, manual, Gentoo install. Just… uh… don’t be discouraged if you fail the first time. Or the second. You’ll learn a TON, trust me.

My experience with other package managers like pacman is minimal. I used Arch for a while, and it’s a very nice distro. It’s something of the best of both worlds between Gentoo and more user-friendly distros like Debian.

Smaller Distros

The Internet is replete with smaller distros with funny names, and there are too many to mention. Most of them are offshoots of one of the main distributions I’ve described above, with various configuration changes. There are some medium size distributions as well (Linux Mint, Puppy Linux, etc) which tend to do a pretty decent job, and are sometimes designed for very specific situations Puppy Linux and Damn Small Linux, for example, are designed to be very small and light weight, and are especially useful for booting from a CD or USB key to do system repairs. Linux Mint, in particular, is a refreshing spin on Ubuntu. I tend not to trust the really small distros though (the ones you’ve never heard of with websites straight out of the 90’s), because I’m dubious as to whether they’ll continue to be supported in the future, and whether they’ve been tested thoroughly. There are probably good ones out there, I just don’t shop around too much anymore.

Choices

In many ways, it all comes down to choices, and the number of them you want to make. If what you want is a plug-and-play operating system that isn’t Windows or Mac OS X, go with Ubuntu, Linux Mint, Fedora, or a similar distro that has a one-and-done type install: you pop the disk in the drive, set up your language, time zone, and login credentials, and away you go. These distros have default packages that support most of your day-to-day needs, and it’s fairly easy to install components that aren’t pre-installed. They work on most of the common hardware out of the box, and they have a lot of online support options.

If, on the other hand, you want to make the choices yourself, choose a distro like Gentoo, Arch, or Debian. Gentoo and Arch, in particular, don’t even choose a default desktop environment for you, so you can choose any configuration you want right from the beginning without having to undo someone else’s work. One time, I installed Gentoo only to realize that I had disabled the kernel configuration for my hard drive controller, so the system couldn’t boot: that’s how much control you have. Debian has some base packages that install a very minimal system, as well as some options that will install a lot of common packages for you. It’s more immediately usable than the other two, but allows you to install a minimal system if you want.

At the far end of the spectrum of choices is LFS: Linux From Scratch. You compile the kernel from scratch, and gradually start loading things on the disk until you have a working operating system. I’ve never done this, but it’s always been in the back of my mind. You can find resources for doing that here: http://www.linuxfromscratch.org/

Stability

One last thing I want to mention is stability. Stability is probably the other most important dimension of a distro, and I would be remiss if I didn’t talk about it just a bit. If you’re cycling through different distros exploring the Linux world, you might not care too much about stability. Honestly, if you play around with things enough, you’re going to wreck your distro no matter how stable it is. But if you’re looking installing Linux on a machine you care about, stability is very important.

Because the distro packagers are usually on the same team (or are the same people) as the people who maintain their distro’s package repositories, their attitudes and values contribute to how stable the resulting collection of packages will be. Debian, for example, is know for being fairly conservative and FLOSS (Free Libre Open Source Software) fanatical, which makes for a very stable, very reliable system, and makes it a bit harder to install proprietary software (not much harder, though.) Ubuntu, on the other hand, is less gun-shy, and uses more up to date packages at the expense of a slightly increased probability that their packages will have unresolved bugs. It’s worth doing some research to find out the attitude of a perspective distribution’s repository maintainers before making your final choice.

Stability is the main reason I forsook Ubuntu years ago, and now only use Debian. Ubuntu is the only operating system I have ever installed (aside from Windows) which has crashed during or right after installation***. It’s a great distro that is paving the way for lots of innovation and publicity in the Linux and Open-Source world, and it has become a stepping stone (at the very least) for new users, but I don’t like the way they do choose their packages, and the default packages that are installed. And, if I’m honest, even though it’s a small and easily fixable issue, Unity absolutely drives me up the wall.

Conclusion

Hopefully this will help you choose the distro that’s right for you. You should play around with a few of them and read up on them (Wikipedia is a great place to do this) before picking the one you intend to use for ever and ever… and always know that you can change your mind at any time. As you learn to use Linux, you’ll likely realize that you wish you had done certain things differently during your installation, so you’ll likely be itching to re-install after a while anyway.

If you’re curious, as you might already have guessed, I install Debian on everything I get my hands on: my desktop, my parents computers, Raspberry Pis – hell, I’d install it on my toaster if I could. After administrating around 20 Debian machines during my two years as a SysAdmin, I’ve come to appreciate its elegant simplicity and robustness, and I wouldn’t replace it with Ubuntu if you paid me. But that’s just my opinion; I encourage you to draw your own.

*It has been pointed out – and rightly so – that the K Desktop Environment (formerly referred to as simply “KDE”) is now properly called “KDE SC”, for “KDE Software Compilation”. For simplicity, however, and since it is still referred to in the Debian repository and popularly as KDE, I’ve left the incorrect acronym as is.

**A member of the Linux Facebook group pointed out that newcomers to Linux might not know what a “tarball” is. Tarball is slang for an archive compressed using the unix tar utility, usually with the extensions .tar, .tar.gz, or .tar.bz2. Source code for many open source packages come packaged in a tar archive.

***It’s true, I’ve had many a Gentoo installation crash on me on or before startup, but that was always because I had done something stupid, and was entirely my fault. The same opportunity doesn’t really exist in Ubuntu; I’ve had installations crash once, and succeed after installing again with the same options for no discernible reason.

Maker Faire Was Awesome (Formerly Untitled, Because I’m Silly)

Update: Yes, I forgot to title this post. I’m silly. It’s been a weird sort of day, please forgive me.

You may have noticed that I missed a blog post yesterday. I can explain… I totally forgot. I managed to do everything else on my list in a timely, orderly, disciplined manner, but the blog post fell off the edge of the Earth. In the end, I decided that it was better to get to sleep on time so I could go running this morning than to stay up and derail my entire week. So that’s what I did.

Yesterday, as planned, I went to Montreal’s very first Mini Maker Faire. It was everything I had expected. I spent nearly three hours wandering around the medium sized tent that had at least six different kinds of 3D printer, ham radio enthusiasts, makers, crafters, hackers, developers, and spectators. The $6 entrance fee seems a measly offering compared to all the awesome I witnessed there, and the best part was that I could stay as long as I wanted, walking around and absorbing it all.

I started reading Make magazine around seven years ago, and I’ve been a subscriber for maybe 5 years. Every year, as I become more and more adventurous, the idea of travelling to one of the great Maker Faires around the world has grown more and more appealing in my mind. One of these days, I’ll certainly go, and I might even go to more than one (especially since New York is only a train ride away.) Even so, it’s wonderful to be able to attend a miniature version of the fabled Maker Faire right here in my own town.

But that’s not the most exhilarating part. For me, the most exciting thing about the mini Maker Faire was seeing all the people who make things in Montreal and the surrounding areas. To find out that all these exciting things are going on around me is inspiring, and makes me want to get making! Before I got to Montreal, I was the only person I knew who had even heard of Make Magazine. I would tell people about it, and how exciting it was, but few people really understood what I was so excited about. At mini Maker Faire, I told people that I had a Cupcake, and they totally knew what I was talking about*!

After about two hours and 30 minutes, when I had seen every booth, and talked to most of the people behind them at length about what they were doing, and how they were doing it, I walked out to see the second level, and reached my saturation point. My brain started turning off; there had been too much to see and experience, and I had taken in about as much of it as I could. It was time to go home.

I wish I had taken pictures, but honestly I was too enthralled by what was going on around me. I’m always afraid of getting distracted by taking too many pictures, and ending up with the tourist’s dilemma of experiencing an exciting event through the lens of a camera. All in all, I would totally do it again tomorrow. And who knows, maybe one day I’ll be on the other side of one of those booths.

*It’s a 3D printer made by Makerbot called a Cupcake CNC. Not the dessert.

How to Heat Your Livingroom with Computers

It’s time I explained what all this cluster computer stuff is about. It’s kind of an insane project, but bare with me. Some of you may have played the Wikipedia game “Seven Steps to Jesus.” Basically, it’s a special case of the Wikipedia game Wikirace, where you try to get from one article to another as quickly as possible. In Seven Steps to Jesus, you try to get to the Wikipedia article about Jesus in seven clicks or less. Why Jesus? Why seven? Who knows, who cares. But the point is, that it’s fun for about ten minutes.

This ridiculous game wandered into my mind the other day, and I started thinking, “I wonder how you could create a computer program that would play Seven Steps to Jesus…” The answer depends. If you give the program access to the entire Wikipedia database, then the task is trivial. It’s very simple and quite fast to spit out the minimum number of steps from one article to another. But what if you don’t give the program access to the entire Wikipedia database? What if it has to go to an article, and choose which link to click just like humans do: with “intuition?”*

As you might have guessed, the task becomes more complicated. Now we’re talking about machine learning (FUN!) I started to think about how you could train an AI to assign a “relatedness” value to every given article on Wikipedia by giving it access to the Wikipedia database, and having it traverse the links from one article to another. If you’ve taken a class on AI, you know that eventually, this relatedness value will converge to the shortest path. Basically, this program will do nothing useful… except heat my living room.

Except! Except that I’m going to train an RL agent in parallel. That’s the only thing that might be novel about this (other than the fact that I’m introducing the “Seven Steps to Jesus” task to the AI world.) Ordinarily, you would want the agent to learn sequentially, because if the links change, you want the agent to learn with the changes. But in this case, I really don’t give a damn about sequential updates. Also, this task is stationary (the distance between any two articles in my downloaded Wikipedia database will never change,) so updating sequentially doesn’t matter all that much.

So what you should get from this, is that this project is a HUGE waste of time. But it’s fun, and I’m learning about graph databases, and RMI, and I got to build and run a computing cluster. Maybe there’s a real task for which this approach is suited. I’m not sure though. Usually in RL, you have to run a trial many times in order to test an approach, so there’s really no point in distributing the actual processing. In other words, if you’re going to run 100 trials of your schmancy new algorithm, you might as well just run one trial on five different machines until you finish, rather than splitting up the computation (which adds overhead,) into five different parts.

The point is, I’m having fun. Leave me alone.

Discipline Week Update: Today was day four of Discipline Week, and so far so good. I’ve been trying to avoid napping, because I want to really embrace this 7am to 11pm schedule I’ve got going, but today I really needed a nap. I ended up sleeping for maybe an hour and a half, which is really too much, but we’ll see how things go tomorrow. I’ll write a more detailed post tomorrow about how Discipline Week is going, but I thought I’d let you know that it’s still a thing, and it’s going well!

*Yes, there are algorithms that do this quickly, but you’re still missing the point: the point is, this will be fun. Fun, I tell you, FUN!

My Livingroom Is A Supercomputer

As a Computer Science Student, I like the idea of being able to run my code on the biggest, baddest computers there are. Unfortunately, most of my code doesn’t need more than my laptop. In fact, many of the programs I write would run just about as well on my phone, or an Apple II. However, sometimes I do write programs that require gobs of raw computing power. Last night, I conceived an idea for just such a program. Before I even started programming, I realized that my only hope for finishing computation on the large dataset I wanted to process (all the links on Wikipedia, many times) was to spread the computation across multiple computers. I needed a cluster. So I built one.

A cluster (short for computing cluster) for those who don’t know, is basically a bunch of computers connected via a network. You could think of the Internet as a computing cluster, but the important difference between the Internet and a useful computing cluster is that a useful cluster works toward a unified goal. As you might have guessed, part of the beauty of a cluster is that the computers involved don’t have to be all that powerful by themselves to be useful. Because my family doesn’t usually get rid of computers (precisely for occasions such as this,) we had three old laptops lying around, all of which I confiscated for my cluster. Add my laptop to that, and I have a cluster of six cores. It’s not a whole lot, but it may just be enough. I also have an old switch (what most people would call a “router”) that I saved from the trash during high school, which I’m using to connect the computers together. After installing Debian Linux, and puppet to manage the configurations on all the machines, I’m almost ready to run cluster computations!

In order to run a cluster, you need a server to distribute the data you want to process, and clients to actually do the processing. In this case, I’m talking about the server and client software, not machines. So I have to write a series of programs that will split up my data and process it on each of the cluster computers. It’s simple enough when you use Java’s RMI (Remote Method Invocation.) The server maintains a collection of the processed data, which is updated whenever the clients finish processing. Once the client has finished its data, it sends back its results and requests more data.

This strategy has a lot of advantages, including the fact that I don’t have to store the entire Wikipedia database on more than one machine. Since the data is sent in chunks, I can store it all on one of the two computers in my cluster that can handle that much data. Also, the clients can work asynchronously. Since the computers don’t have to wait for each other before getting new data to work on, my laptop can crunch along at 2.53GHz per core, processing as much data as it wants, while the poor little AMD Athlon slogs through whatever data it can handle.

My livingroom table, which currently houses four computers and a switch. Delightful.

My cluster is called “End of the World,” or EotW for short. I was trying to think of a theme for the hostnames of the computers in the network, and I settled on character’s from Haruki Murakami’s novels. Since there’s a “place” in his book Hard Boiled Wonderland and the End of the World called “End of the World,” I figured that would be fitting. The nodes are named toru, watanabe, and sumire.

How much did all this cost? $20 for a power strip. Even if I didn’t have all the laptops, I could have waited until the morning, gone to the recycling center, and picked up a bunch of computers and a switch for free. That’s what I love about cluster computing: with just a little recycled hardware, you can create a pretty fast computer. It’s true, my tiny computing cluster barely rivals a new, mid-line desktop computer with an Intel Xeon processor, but it’s still faster than just my laptop. And hopefully, with all those computers working full tilt for a few days, I’ll be able to crunch through my data*.

 

*As I said before, I’ll discuss what I’m trying to do at another time. Trying to explain it here would make this post too long, and I want some tangible results before I start talking about it.

Geek Tip: Volume Controls in wmii

wmii is awesome. wmii will change your life. wmii will change the way you think about working with a computer. It will help you stay focused, it will teach you more about how your computer works, and encourage you to solve problems on a scientific level, rather than a “maybe if I do this it will make the magical elves in my computer do what I want them to” level. It will make the time you spend on your computer more enjoyable, less cluttered, and more efficient. At least, it has done all these things for me. wmii is a window manager (layman’s terms: a desktop) that is minimal, functional, and efficient. It’s one step above the terminal, and doesn’t provide anything you don’t really need. But the fact that it’s highly customizable, keyboard controlled, and very efficient makes it ideal for computer geeks like me. One thing I’ve been pondering about lately, is how to make the volume keys on my keyboard work. I figured there would be an easy solution, and there is.

First, if your keyboard has dedicated volume keys, then they’re called “XF86AudioRaiseVolume” and “XF86AudioLowerVolume” unless something has gone wrong. More generally, to figure out what wmii (and other X11 applications) call the keys you want, run xev from the terminal, press the key in question, and look for the line that says (keysym 0xsomething, Something). In this case, the Something is the name of your key. Next, edit your wmiirc (usually in ~/.wmii-3.5/wmiirc, or you’ll have to copy it there from /etc/X11/wmii-3.5/wmiirc,) and find the section that starts with #Key Bindings. You’ll see a few lines that have the syntax

Key Something

command

You should enter your keys in exactly that format. For the volume keys, I wrote this:

Key XF86AudioRaiseVolume

amixer -c 0 set Master 2%+

Key XF86AudioLowerVolume

amixer -c 0 set Master 2%-

Press Mod-a and select wmiirc to reset wmii’s configuration file. Done. Enjoy your volume keys.

Tip: you can also use wmiirc to – besides changing just about anything related to wmii’s functionality – change your desktop background to something that isn’t just gray. Install feh (not necessary, but it makes things easy,) comment out the line in wmiirc that starts with “xsetroot “, and add “feh –bg-scale /path/to/background/image &” Done.

Super Quick Mini Post: Removing Weird Files in Linux

For the past few months, there has been a file in my home folder called ???. Deleting it with a file manager like Nautilus or Dolphin didn’t work, and “rm \?\?\?” didn’t work either. I finally deleted it, and in this Super Quick Mini Post, I will tell you how.

The reason “rm \?\?\?” didn’t work is that the file isn’t actually called ???; it’s name contains characters that are not contained in the terminal’s font, so it doesn’t know how to display them. So the easy way to delete them is with regular expressions. All you have to do is use sed to make rm ignore all files with normal characters. In my case, because there were no normal characters in the filename, this was easy. All I had to do was “rm $(ls | sed -e ‘s/[^a-zA-Z].*//’)” less the enclosing double quotes. If you’re familiar with regular expressions, you can probably see what this does; it’s fairly straightforward. In English, it says delete the following: list the directory, and remove all characters in lines that start with any letter between a and z or between A and Z. If you just run the commands inside the brackets, you’ll get a bunch of lines of nothing (which won’t do anything to rm) and the file you’re looking for. If the weird file contains alphanumeric characters, then you’ll probably have to modify the regular expression to suit the file exactly. So if your file is called “doof?s”, then you can do this “rm $(ls | sed -e ‘s/doof.s//’)” but that will delete any files starting with “doof” and ending with “s”.

I’ll write another post soon explaining fun things like where I’ve been for the past week, but I wanted to get this post out there for people experiencing this bizarre problem, and to show how useful and amazing regular expressions are. They’re one of the lesser known tools that would make many people’s computational lives so much simpler.

Note: Windows will not let you name files with characters it can’t hondle. However, I don’t believe it will prevent you from writing a program that accidentally creates files with unacceptable names, it can’t prevent people from naming files weirdly on removable media, and I’m almost positive that it can’t delete a file named using weird characters. That’s not a criticism of Windows, but simply an attempt to prevent people from saying “SEE! LINUX IS STOOPYD! U can’t do that in WINDOWS!! NERD!”