Linux Recipes

Note: I use Debian as my normal distro, so all “recipes” will be in reference to that, unless otherwise noted.  Most Debian “recipes” will also work in Ubuntu.

Create a large file for a disk image quickly

The key is the seek parameter to dd. This allows you to quickly seek to the end of the file.

This command quickly creates a 2GB image:

dd if=/dev/zero of=file.img bs=512 count=0 seek=$((63*245*255))

Another way would be:

dd if=/dev/zero of=file.img bs=1 count=0 seek=2G

These methods create what people refer to as a file having “holes” in it. The physical file is not truly as large as is reported by ls -l. This is similar to files used by a virtual machine that “grow” as they are used. Files with “holes” are only supported on certain file systems, ext3 and 4 being two of them (that I know of).

Associate a flat file with a loop device and mount it

You can normally mount a flat file with:

mount -o loop ./file.img /mnt/<mountpoint>

Where <mountpoint> is the name of an empty folder under /mnt.

This assumes that the file itself is a “partition”. If not, you can use -o loop,offset=N to offset to the start of the partition on the volume.

However, I like to use the /dev/loop devices to mount flat files, because you can access them more like physical disks:

losetup /dev/loop0 /path/to/file.img
mount /dev/loop0 /mnt/<mountpoint>

.. or ..

cfdisk /dev/loop0
to partition the disk like a physical disk volume.

use losetup -d /dev/loop0 to dissociate the loop0 (or loopN) with the file.

use kpartx to map partitions to dev; (i.e. /dev/loop0p1, /dev/loop0p2)

modprobe dm-mod
kpartx -a -v file.img
mkfs.msdos /dev/mapper/loop0p1 #(or whatever)

If you don’t have the dm-mod driver or can’t use kpartx for some reason, you can always use the following:

shell:~# fdisk -lu test.img 

Disk test.img: 104 MB, 104857600 bytes
255 heads, 63 sectors/track, 12 cylinders, total 204800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
test.img1              63       16064        8001   83  Linux
test.img2           16065       32129        8032+  83  Linux
test.img3           32130       48194        8032+  83  Linux

shell:~# mount -o offset=$((512*32130)) test.img mnt

Note that the number of bytes per sector is 512, units are displayed in sectors, and the starting sector offset of the third partition is 32130. Hence I used offset=$((512*32130)). You can mount the other two partitions on this disk by using $((512*63)) or $((512*16065)), respectively.

Getting Debian working on Beagleboard

Nightly built debian kernels here.

Install-me.sh script (i.e. here), generated for each kernel, has code on installing kernel.

Key point — for kernels >= 2.6.36, need to use ttyO2 to refer to Beagleboard Rev C4’s serial port.

See here for more info

Install unstable packages with apt-get

apt-get install package/unstable

Downgrade a package

  1. Make sure a lower package is available. Run apt-cache showpkg <pkgname>>. Towards the end of the output look for something similar to the following:

    Provides:
    3.12.11-3 -
    3.12.8-1+squeeze1 -

     

  2. Now, downgrade using:

    aptitude install <pkgname>=3.12.8-1+squeeze1  ## (or whatever version number is lower).
    1. You may be asked to replace certain other packages that depend on the one you are downgrading. Aptitude should come up with several solutions, and you can choose to accept/reject the current one by specifying Y/n/q.

Download and untar a file together on the fly

Recently, I wanted to download the source code for Google Chromium web browser, but didn’t want to download and then untar; since it was so huge I didn’t want both the tar and the untarred source on my disk. Remembering that a tar file is sequential (that’s the nature of tapes), I learned how to download and untar simultaneously:

 wget -qO- http://chromium-browser-source.commondatastorage.googleapis.com/chromium.r102647.tgz | tar xz

HERE IS Documents

You can specify a document’s contents directly on the command line using Bash/sh.

cat > file.txt <<EOF
Line One
Line Two
Line Three
EOF

Below example ripped directly from here:

This is very useful because variables are evaluated during this operation. Here is a way to transfer a file using ftp from a shell script:

# Usage:
#       ftpfile machine file
# set -x
SOURCE=$1
FILE=$2
BFILE=`basename $FILE`
ftp -n $SOURCE <<EndFTP
ascii
user anonymous $USER@`hostname`
get $FILE /tmp/$BFILE
EndFTP

Trouble connecting to XRDP

This hint helped me.

In essence, I was out of X sessions for whatever reason. They weren’t being cleaned up.

/etc/init.d/xrdp stop
cd /tmp/.X11-unix 
rm -rf *
cd /tmp
ls -a | grep lock #Should show a bunch of .X##-lock files.
rm -rf .X*-lock

Adding includes or library paths to a “configure” script for compiling

I had a user account on a system and couldn’t install libraries for a compiliation. So, how do you specify a path for other libraries?

I struggled with this for a while and finally found the answer here.

env CFLAGS="-I/path/to/libs/root/usr/include -I/path/to/libs/root/usr/lib/glib-2.0/include -I/path/to/libs/root/usr/include/glib-2.0" LDFLAGS="-L/path/to/libs/root/usr/lib -L/path/to/libs/root/lib -lglib-2.0" make 2>&1 | tee out.txt

Note that -I and -L are specified multiple times for multiple folders. Also, if you want to specify that that particular version of glib-2.0 is used, then you need to specify -lglib-2.0 after the library folder declaration.

It also works if you export those two variables. Not sure why I never tried that before, since I knew about the CFLAGS and LDFLAGS variables that are present in most configure scripts:

export CFLAGS="-I/path/to/libs/root/usr/include -I/path/to/libs/root/usr/lib/glib-2.0/include -I/path/to/libs/root/usr/include/glib-2.0"

export LDFLAGS="-L/path/to/libs/root/usr/lib -L/path/to/libs/root/lib -lglib-2.0"

Shell script to download and extract Debian packages into an alternate folder

This script will download Debian packages, along with their dependencies, and extract them into a folder. You need to set the environment variables at the top of the package to set your architecture, etc.

Shell script to rip a CD

Requires cdparanoia and lame packages installed.

Save the below text into a file called ripcd and call chmod +x ripcd.

To run script, call ripcd <optional track prefix>; where <optional track prefix> is a word to use before each track number.

ex: ripcd "The Orchestra" will rip tracks: “The Orchestra 001”, “The Orchestra 002”, etc.

#!/bin/bash

if [ "$1" = "" ]; then
        OUTNAME="Track ";
else
        OUTNAME="$1 ";
fi;

cdparanoia -sQ 2>&1 | 
        awk -F'.' '/[0-9]+/{print $1}' | 
        egrep '^(W)*[0-9]+' | 

        while read i; do
                TRACKNO=000$i
                LEN=${#TRACKNO};
                TRACK=Track ${TRACKNO:$((LEN-3)):3};
                echo "Ripping $OUTNAME$TRACK ...";
                cdparanoia -q $i cdda.wav
                echo "MP3 Encoding $OUTNAME$TRACK ...";
                lame --quiet -b 256 cdda.wav "$OUTNAME$TRACK.mp3"
                rm cdda.wav
        done

Force the screen to blank

xset dpms force off

Source

MVS Unix System Services (USS)

I decided to put this under Linux Hints and tips, so I could find again it easily; even though USS isn’t really Linux.

Access an MVS dataset from the USS shell

cat "//'DATASET.NAME.HERE'"

GREPping an MVS file in the USS shell

USS didn’t like grep abc "//'MVS.DATASET'", so this worked:

cat  "//'MVS.DATASET'" | grep "LOOKFOR"

You can grep multiple patterns at once with the -e command. This is in contrast to Linux which uses -e to denote an “extended” regular expression. This was a stumbling block for me at first until I read the MAN page on OMVS.

cat  "//'MVS.DATASET'" | grep -e"LOOKFOR1" -e"LOOKFOR2" -e"LOOKFOR3" > output_to_file.txt

Forking the above command into the background (using an ampersand at the end of the line) seemed to make it run faster. Or that may have just been my imagination. In either case I’ll probably do that again next time.

Note: you can use regular expressions with the grep command. So the following should work (theoretically, I’d think it would speed up this search, too, since grep should know how many characters from the beginning of the line we are looking for):

cat  "//'MVS.DATASET'" | grep "^.....ABC" > output_to_file.txt

What is the best way to find a list of several strings within a large text file?

From here:

cat "//'MVS.DATASET'" | grep -F -f patterns.txt > output.txt

Where patterns.txt contains a newline-separated list of pattern expressions.

Debian Chroot

Socat

socat unix-listen:/dev/adbsock,fork exec:'/bin/bash -li',pty,setsid,stderr,sigint,sighup

Add the Ubuntu Keyring On a Debian System

From here:

wget http://archive.ubuntu.com/ubuntu/project/ubuntu-archive-keyring.gpg -O- | sudo apt-key add -
sudo apt-key update
sudo apt-key net-update

I used wget to download the ubuntu-archive-keyring.gpg file directly, and copied it to /usr/share/keyrings/ on my Debian system, so that I could validate Ubuntu packages for debootstrap (so that Debootstrap didn’t complain about unvalidated packages).

Debootstrap Ubuntu on Debian

Not much different than a regular Debian debootstrap, but putting here for future reference. List of Ubuntu Release Names. At the time of this writing, Trusty is the current Ubuntu release.

debootstrap --variant=buildd trusty . http://archive.ubuntu.com/ubuntu/

Disabling Services in a Chroot

From here:

Replacing /sbin/initctl with a symlink to /bin/true is a fairly standard
way to disable services in a chroot.

Configure WLAN to use WPA Supplicant

From here:

auto wlan0
iface wlan0 inet dhcp
    wpa-ssid mynetworkname
    wpa-psk mysecretpassphrase

Copy an ISO image directly to a USB flash drive

For Debian


~# apt-get install -y genisoimage

~# geteltorito -o usbimage.img acdromordvdimage.iso

~# cat usbimage.img > /dev/sdX

Notes

Concise explanation of HERE IS documents

Using cp to copy a sequential data or PDS member into a z/OS UNIX file