BASH – Get Size of Directory in "ls -l" Output
(The above photo is "flowring cabbage regular", by abundantlyabove29 (C) via Flickr).
UPDATE: I've written a newer and much better version of this script. You can find it at: http://www.shellperson.net/show-directory-size-with-ls/
I found a function like this a long time ago in some forum, but I remember it not working with filenames that include spaces. There's almost certainly a better and more-efficient way of doing this, but here's draft one.
This will output the same data as "ls -lh", in almost the exact same format, except directories will show the size of the sum of their contents (du -sh) instead of the 4.0K of space the directory "file" takes up. The only noticeable difference in format is that there is a slightly larger gap between the size and the date columns.
#!/bin/bash ## mimic "ls -l" except display size of sum of directory's contents for x in *; do echo -e "$(ls -lL | grep -w "$x" | cut -d ' ' -f 1-4) $(du -sh "$x" | cut -d ' ' -f 1)" "\t" "$(ls -lL | grep -w "$x" | cut -d ' ' -f 5-20 | sed s/^\ *// | cut -d ' ' -f 2-20)"; done
It's hard to enter code in a presentable way in wordpress - and this code doesn't appear perfectly. Note that the multiple-space gap that delimits (-d) the second "cut" command is a tab. You can add a tab at the command line by using CTRL+v followed by TAB. The code does work if you use the copy-to-clipboard button on the code block.
I saved this as "lsd", made it executable, and put it in a directory in my $PATH (~/bin).
Installing MPD & MPC in Ubuntu 9.10 Karmic Koala
(The above photo is "more flowers", by Abbie F (C) via Flickr).
EDIT: After running the setup described herein for several days, I've uninstalled PulseAudio and use ALSA instead. While PulseAudio may be a theoretically better system, its implementation in 9.10 is perfectly awful. Not only was MPD/MPC constantly crashing, but sometimes I'd lose sound system-wide. Also once Pulse is gone, you can roll back to Adobe Flash 9 (with sound!), which is infinitely better on my machine than 9.10's implementation of Flash 10.
The installation wasn't as straight-forward as I'd hoped, so maybe these notes will help others. This is specifically for Ubuntu 9.10 Karmic Koala (I'm running the 32 bit / i386 desktop edition), running the default pulse audio.
Install from official sources:
sudo apt-get install mpd mpc
Symlink your music folders to MPD's default watch folder:
sudo ln -s /path/to/music /var/lib/mpd/music
Update your MPD database. The new version of MPD has changed some things about the "mpd --update-db" command, so you should use:
mpc update
At this point I could play music, I just couldn't change the volume. I got the following error:
error: ACK [52@0] {setvol} problems setting volume
The problem was that MPD was trying to use ALSA instead of pulse audio. Edit the MPD configuration file to enable pulse audio. Find the Audio Output section of the file and add or un-comment the following lines:
audio_output {
type "pulse"
name "My MPD PulseAudio Output"
}
I did follow the instructions at http://ubuntuforums.org/showthread.php?t=1298789 regarding the PulseAudio Preferences (paprefs), but I don't know if it's necessary.
Finally, add the user "mpd" to the PulseAudio groups in order to give it permission to use PulseAudio:
sudo usermod -aG pulse,pulse-access mpd
Restart MPD:
sudo /etc/init.d/mpd restart
That's what worked for me.
Easily Change Text Color with BASH
Here's a script to easily add color to other BASH scripts. The only trick to it is that it uses a character that I don't know how to put on the web.
NB - If you select the "view plain" link at the top of the container surrounding the script, it will open a new link with the plain text of the script. You should use this plain text. Not only does it preserve the special character the script uses to change colors, it also preserves other characters that WordPress changes (note line 5, between "cat" and "EOF"). If you don't use the "view plain" link and try to cut, but instead paste the script directly from this page, it won't work. Also, if you need to create the special character I use to make it display different colors, you can make it in gnome-terminal by pressing "ctrl+v" followed by "ctrl+[".
#!/bin/bash
## Specify color to write in using arguments
function --help {
cat << EOF
ERROR: $0 requires a color argument.
USAGE: Changes the color of the text piped into it.
These color arguments are availabe:
ARGUMENT SHORTCUT
white ------ w
red ------ r
green ------ g
yellow ------ y
blue ------ b
purple ------ p
teal ------ t
bold ------ bb
The "bold" argument will modify any color.
Use a max of 2 arguments (one color and bold).
EOF
}
function bold {
# make the color bold
BOLD=1\;
}
function white {
COLOR=1
}
function red {
COLOR=31
}
function green {
COLOR=32
}
function yellow {
COLOR=33
}
function blue {
COLOR=34
}
function purple {
COLOR=35
}
function teal {
COLOR=36
}
## shortcuts
function bb {
bold
}
function w {
white
}
function r {
red
}
function g {
green
}
function y {
yellow
}
function b {
blue
}
function p {
purple
}
function t {
teal
}
## Execution
if [ "$#" = 0 ]
then
--help
fi
while (($#));
do
$1
shift
done
echo -n "["$BOLD""$COLOR"m"
cat
echo -n "[0m"
Here's a link to an example of what it looks like in action. Note that the input must be piped into the script.
Quickly Check Gmail with BASH
A very simple script to check if I have any new gmail:
#!/bin/bash ## Quickly checks if I have new gmail echo -e "Checking for new messages... \c" atomlines=`wget -T 3 -t 1 -q --secure-protocol=TLSv1 \ --no-check-certificate \ --user=USERNAME --password=PASSWORD \ https://mail.google.com/mail/feed/atom -O - \ | wc -l` echo -e "\r\c" [ $atomlines -gt "8" ] \ && echo -e " You have new gmail. \c" \ || echo -e " No new gmail. \c"
Just replace USERNAME and PASSWORD with your information (caution - your password is stored in plain text). I use colors on mine, but I'm not sure I can correctly post the requisite special character in WordPress. Here's what it looks like in gedit.
I can make that special character in gnome-terminal by pressing CTRL+v and then CTRL+[
There's probably an easier way to use colors, I just don't know it.
In case it's not clear from the script, this just downloads the atom feed (an XML file) and checks if it's more than eight lines long. If it is, then you have unread mail. (At least on my account, when my inbox has no unread items, the atom feed is eight lines long). The number may need to be adjusted if gmail changes the format of their feeds.
ssh Shortcuts in Bash
(The above photo is "Good and Evil", by Abbie F (C) via Flickr).
This is the bash script I use to ssh into my fileserver (without having to type the whole IP address, etc). It works for simply logging in with ssh, or for sending a command only. If you want to send a series of commands, you'll need to separate each command with \; or else it (a lone semi-colon) will be interpreted as an instruction to run the command on the local machine after the first command is sent via ssh (you're 'escaping' the semi-colon, so it gets sent through ssh as a regular character, instead of its typical role of separating commands). This method works for me, but I'd love to learn a better way if there is one.
I name the script something short and put it in my path. I also use it in conjunction with password-less login. I know it's not such a big deal to write out:
ssh user@ipaddress "command -arguments"
but it saves time if you're constantly ssh'ing to that machine.
Here it is in a more-legible format, that shouldn't be copied and pasted (because of the way WordPress changes quotation marks):
#!/bin/bash
SSH_ARGUMENTS=`while (($#)) ; do echo -n "$1 " ; shift ; done ; echo`
ssh administrator@192.168.0.202 "$SSH_ARGUMENTS"
And here it is in a way that's easier to copy and paste (but harder to read, in my opinion):
#!/bin/bash SSH_ARGUMENTS=`while (($#)) ; do echo -n "$1 " ; shift ; done ; echo` ssh administrator@192.168.0.202 "$SSH_ARGUMENTS"
Also, here's a slight variation that's useful if you have multiple machines with a sequential naming scheme, where you can specify the machine number:
#!/bin/bash ## for example: comp 4 shutdown -r now ## note: in my setup, the username correlates with the computer's number WORKSTATION_NUMBER=$1 shift SSH_ARGUMENTS=`while (($#)) ; do echo -n "$1 " ; shift ; done ; echo` ssh $WORKSTATION_NUMBER@192.168.0.10$WORKSTATION_NUMBER "$SSH_ARGUMENTS"
Edit: I suppose anyone who knows anything about BASH would know that the entire SSH_ARGUMENTS line could be removed and the variable $SSH_ARGUMENTS replaced with $*
...every day learning a little bit more.
System Clock in a VirtualBox Debian Guest
By default, the system clock in Debian is set to UTC (Universal Time), and then adjusted to your local time, based on your time zone and daylight savings time. This can be a problem if you are running a Debian VirtualBox guest machine, because VirtualBox sets the virtual machine's system clock to local time when the machine is started. Because Debian expects the system clock to be UTC, the time zone adjustment results in the wrong local time (unless you live near Greenwich?).
You can fix the problem by editing the the file "/etc/default/rcS" (you'll need to be root). Simply change the line "UTC=yes" to "UTC=no" and save the file. After rebooting, Debian will treat the system clock as local time (which it is) instead of UTC, and won't adjust it for time zones.
Using a VirtualBox Machine in Both Linux and Windows
It isn't difficult to use a VirtualBox virtual machine in both Linux and Windows, but it wasn't immediately obvious to me, so perhaps it will be helpful. The virtual machine consists of two primary parts; the virtual hard drive (named yourvirtualmachinename.vdi), and an XML file which contains the settings for the virtual machine (named yourvirtualmachinename.xml). Both of these files are contained within the ".VirtualBox" folder in your home directory ("/home/username/.VirtualBox" in Linux, and probably "C:\Documents and Settings\username\.VirtualBox" in Windows XP). The leading period "." in ".VirtualBox" indicates that it's a hidden folder in Linux. The .vdi hard disk file will be in the "HardDisks" subdirectory, while the .xml config file will be in the "Machines" subdirectory, in another subdirectory with the name of your virtual machine. Unfortunately, the .xml configuration file will not work in a new host with a different operating system, so you can't simply register the old file with the new host.
When migrating the virtual machine to another computer (or just another operating system), you will only need to copy the .vdi hard disk file. Put it in the "HardDisk" subdirectory on the new host and select "New..." from the "Machine" menu. Give the new virtual machine a name (it doesn't matter what), set the memory to the same level as you used on the old host (I doubt it matters if it's the same, but that's what I did - 512 MB), and choose "Existing" in the Virtual Hard Disk selection screen. That will open the "Virtual Media Manager". Select "Add" and then choose the .vdi virtual disk you copied from your other host system.
Once the 'new' machine has been created, adjust the settings to your liking. I enable both sound and USB, and set up a shared folder. In both Linux and Windows I use my Desktop as the shared folder (and name it, surprisingly, "Desktop"). Inside my Windows XP guest virtual machine, I've mapped the shared "Desktop" folder to drive letter X: with the command net use x: \\vboxsrv\Desktop (at the command prompt), and then set my Windows desktop to "X:\", so my virtual machine desktop is the same as the host machine's desktop. This is done by changing the registry key named "Desktop" (found at "HKEY_CURRENT_USER > Software > Microsoft > Windows > CurrentVersion > Explorer > User Shell Folders") to the value "x:". Once the net use command and the registry change have been done once, it will continue to work in both Linux and Windows (and presumably Mac?) as long as the shared folder is named "Desktop".
All this entire process is doing is creating a new "yourvirtualmachinename.xml" file for the new host, but using the same virtual hard disk. I haven't tried it, but I imagine that you could use the same virtual hard disk file in both operating systems in a dual boot environment. I'm assuming the best way of doing this would be to put the file on an NTFS partition, as most Linux distributions now have native NTFS read-write capabilities.
Here are the software versions I was working with:
Debian Sid (a.k.a. Unstable)
Windows XP Professional SP2
Sun xVM VirtualBox 2.1.4
Moving from Ubuntu to Debian
I get the impression that most people are migrating the other direction, from Debian to Ubuntu. Or actually, from lots of other Linux distributions to Ubuntu (but Debian is significant because it's Ubuntu's parent distribution).
I have only good things to say about Ubuntu -- it's the distribution that allowed me to switch to Linux. It's the first distribution to correctly configure my wireless card during installation (a task that both Windows XP and Vista fail at), which allowed me to get my questions answered online. I started using Ubuntu Dapper Drake, and have since installed each new release. I even convinced my Mom to switch to Ubuntu from Windows. However, since installing Debian Unstable (Sid) a few months ago, I've come to prefer Debian.
Ubuntu is still a relatively young distribution, which, combined with the aggressive new release schedule (every six months), means that it's changing very quickly. In general, I think this is a good thing. Each release becomes increasingly user friendly. But, it turns out that I don't really need a distribution that is that user friendly. I'm a pretty quick learner, and I've gotten used to editing configuration text files instead of clicking buttons in an Options menu. I think I became acclimated to how a Linux system works, and Ubuntu in particular, around the Fiesty Fawn release (maybe it was Gutsy Gibbon). Ubuntu does a fair amount of hand-holding for inexperienced Linux users, but once I got enough experience, I found that I could work more quickly and more precisely without many of the GUI tools. As I installed newer releases, I found myself disabling many of the new features. As I was setting up and customizing the new Hardy Heron release, I noticed that the xorg.conf configuration file was no longer in use, and I no longer knew how to set things up the way I liked them. (The xorg.conf file may be an odd thing to miss, as it is often the first truly confusing thing a new user comes across). So, hoping to find something more like I was used to (Fiesty Fawn), I tried Debian.
Debian was exactly what I was looking for. It had the same underlying system I learned on (specifically the APT system, which is awesome), but in less-shiny packaging. I would describe it as a more transparent distribution than Ubuntu. Debian doesn't hide as many of the available options (which is often the root of complexity) as Ubuntu, and consequently it's probably not as user-friendly. While maybe daunting to a new user, it was refreshing for me.
I first installed Debian Stable (Etch) (stable sounds good, right? at least it sounds better than unstable). Debian Stable is just that. The trade-off, though, is that the software is not very cutting-edge. Since I was already spoiled by always having new releases of Ubuntu, I knew I would have to upgrade to Unstable or go back to Ubuntu. During the upgrade process, I discovered that Ubuntu's popularity is very nice when it comes to finding answers online. It's not quite as easy to find the answer to a Debian question, but I survived. Unstable seems to run software that's just as new as the new Ubuntu release, if not newer. And, despite its name, I have had no problems at all with instability -- everything's worked great. So now I have a system that works in similar ways as my old Ubuntu system, with newer or just-as-new software, but that preserves a bit more of the classic linux structure. I'm pleased.
Another benefit of Debian is that it seems easier to have a minimalist installation. I felt like Ubuntu was getting slower with each new release, perhaps because it increasingly attempted to be everything to everyone. That definitely has its place, and I can see how it's helpful for Windows converts, but it's not for me. Also, although I'll miss the excitement of a new Ubuntu release every six months, I might actually get a bit more done if I'm not installing a new release twice a year.
Create a Debian Sid Virtual Machine with VMWare Tools
This post tries to describe how to create a Debian Sid (Unstable) virtual machine (including VMWare Tools) inside a Windows XP SP2 host. I used EasyVMX.com, Debian Linux, and VMWare Server (all free).
This tutorial assumes that VMWare Server (free) is already installed. VMWare player (free) can also be used, and it's a little easier in my opinion, but Server is what I was using.
First you need to have a blank virtual machine to install to. EasyVMX (easyvmx.com) is one of many ways to do this. I chose EasyVMX 2.0. Most of the settings will just depend on your preferences, but here are some of the options I chose.
- I gave the VM 512 MB RAM. Debian will work fine with less, if you don't have it to spare.
- I used a bridged network device (which gives the VM its own IP address).
- I created two CD drives, one for actual CDs, the other for mounting .ISO images.
- I gave the VM 6 GB hard drive space. This seems to be plenty for me. You could probably do 4.7 GB and get it to fit on a DVD. Once I tried 3 GB and it didn't allow enough space to upgrade to Sid.
Download the zipped VM file from the download link. Extract it into whatever directory you want. I chose "c:virtual machines".
Log into VMWare Server using Firefox (or whatever browser). There's a link in the Start Menu, or you can just go to https://[yourcomputername]:8333/ui/# (replace [yourcomputername] with your computer name). The first time I tried logging in with Firefox, it wouldn't accept the self-signed certificate, so you have to create an exception. Just follow the link at the bottom of the access restriction page to add the exception. Once at the login screen, enter your Windows login name and password.
If you haven't already downloaded the Debian installation .ISO, you can get it from http://www.debian.org/CD/ -- I used the 'Stable' image. Move the .ISO into the same directory as the virtual machine files you unzipped.
In the VMWare Server admin page, under the "Summary" tab, there's an "Add Datastore" link on the right-hand side. Use this to specify the directory where you put the VM files (use the "Local Datastore" section). Then select the "Virtual Machine" tab and click the "Add Virtual Machine to Inventory" link. Select your virtual machine (the .VMX file) from the Datastore that you added earlier. Once it's added, select the VM from the "Inventory" list on the left-hand side of the screen.
In the "Hardware" section, select the dropdown for CD/DVD Drive 2 and choose "Edit". Use the following settings:
- Host Media
- Connect at Power On
- ISO Image (browse to the location of the Debian Stable installation .ISO)
This is optional, but to install an easy link to your VM from the desktop, select "Generate Virtual Machine Shortcut" from the "Commands" section on the right-hand side, then click the link just above the "Enter" button on the dialogue that follows. Click OK to allow Firefox to add the shortcut to your Desktop.
Power-on the VM (you can just press the play button at the top of the management console). The VM will try to boot from the virtual hard drive, but since it's blank it won't find any boot instructions. It will then check the virtual CD/DVD drive for bootable media, where it will find the Debian installation .ISO and boot from it. Press "Enter" to start the installation.
Most of the installation options will vary according to your preference, but here were my choices:
- Language: English
- Region: United States
- Keymap: American English
- Hostname: sidvm
- Domain Name: home
- Partition Disks: Guided -- Use Entire Disk
- Partitioning Scheme: All files in one partition
- I accepted the suggested sizes for the / partition and the SWAP partition
- Time Zone: Eastern
- Use Network Mirror?: No
- Use popularity-contest?: No
- Software Selection: Desktop Environment, Laptop, and Standard System
- Video Modes: 1024x768, 800x600, and 640x480
- Install GRUB Boot Loader?: Yes
I think the installation took me about 1 hour. Now you have a functioning Debian Stable (Etch) virtual machine. I'm not a huge fan of Debian Stable (well, I am a fan, but I prefer using Unstable for more up-to-date software).
To upgrade to Unstable (Sid), you'll need to change the APT repositories to use Sid instead of Etch. From a terminal, login as root and open the sources.list file:
su
gedit /etc/apt/sources.list
Here is the sources.list file I used. Save the file and close gedit. Now to update the system from Stable to Unstable, you'll need to update the package lists and then do a dist-upgrade. I'd do a regular upgrade afterwards, but I don't know if it's necessary. I think you'll then have to do another dist-upgrade to catch a few packages that didn't get upgraded on the first pass. Here are the commands (from that same root terminal):
apt-get update
apt-get dist-upgrade
apt-get upgrade
apt-get dist-upgrade
During the upgrade process it will ask you a few questions. The default responses should work fine. Once it's done upgrading, reboot the computer (this is important). Now you have a functioning Debian Unstable virtual machine. The only thing missing is VMWare Tools (just to make everything run more smoothly). You'll need to install two packages before you can install VMWare Tools: "build-essential" and the kernel headers. The "build-essential" package contains tools for compiling programs (like gcc - the GNU C# Compiler). Without "build-essential", you will get the following error:
Setup is unable to find the "gcc" program on your machine. Please make sure it is installed. Do you want to specify the location of this program by hand?
The kernel headers are used for compiling the VMWare Tools modules. Without the headers, you will get the following message:
What is the location of the directory of C header files that match your running kernel? [/usr/src/linux/include]
The path "/usr/src/linux/include" is not an existing directory.
Here are the commands:
apt-get install build-essential
apt-get install linux-headers-`uname -r`
Note that those are back-ticks surrounding the phrase "uname -r" -- they are not single quotes. `uname -r` specifies your current kernel version. When I installed VMWare Tools, my system wanted to use gcc-4.3 as the C# Compiler -- unfortunately, the VMWare Tools installation script wanted to use gcc-4.1 This is the error it gave me:
Your kernel was built with "gcc" version "4.1.3", while you are trying to use "/usr/bin/gcc" version "4.3.3". This configuration is not recommended and VMware Tools may crash if you'll continue. Please try to use exactly same compiler as one used for building your kernel. Do you want to go with compiler "/usr/bin/gcc" version "4.3.3" anyway?
To fix this, in a root terminal, you'll need to tell Debian to use the older version. It should already be installed, but if not you can "apt-get install gcc-4.1".
cd /usr/bin
cp gcc gcc.backup
rm gcc
ln -s gcc-4.1 gcc
If you want to reverse this in the future, you can:
cd /usr/bin
rm gcc
cp gcc.backup gcc
Once those packages are installed and the correct C# compiler is being used, you'll need to go back to the VMWare Server management console in Firefox on the host OS. On the right-hand side, in the "Status" section, click on the link "Install VMWare Tools...". This will mount a virtual CD in the Debian guest VM. In my case it mounted it at /media/cdrom1. Back in the Debian VM, in a root terminal, type the following commands:
cd /tmp
tar zxpf /media/cdrom1/VMWareTools-2.0.0-1222956.tar.gz
cd vmware-tools-distrib/
./vmware-install.pl
Note that the specific version number may be different on your system. This will start the VMWare Tools installation script. The installation script will finish and then launch the configuration script. The configuration script will compile the necessary modules. I used the default settings throughout, except I set my display size to [12] (1024x768). The script will tell you that it "failed" when "Mounting HGFS Shares". This is normal behavior and it will also happen at startup. As far as I know, the HGFS Shares (a method of sharing files between host and guest systems) only work with VMWare Workstation (not free) and perhaps other enterprise (not free) releases. Reboot the system. You now should have a working Debian Sid virtual machine with VMWare Tools installed.
Note - If you want to share files between the host and guest, the easiest way I know is to share a folder in Windows XP and then open it in the Debian guest from "Network" in the "Places" menu. While doing this I did encounter an odd Windows behavior (read: bug) that prevented sharing. When I tried to access the share from the Windows machine, I got the following error:
serversharename is not accessible. You might not have permission to use this network resource. Contact the administrator of this server to find out if you have access permissions.
Not enough server storage is available to process this command.
I found the solution at http://www.pchell.com/support/notenoughserverstorage.shtml -- in short, I had to add a DWORD registry key named "IRPStackSize" to "HKEY_LOCAL_MACHINE -> SYSTEM -> CurrentControlSet -> Services -> Lanmanserver -> Parameters " and set it to "25".
UPDATE: I tried out VirtualBox (free) from Sun and vastly prefer it. In my experience it's considerably faster and easier to use.