Latest
Happy People at LinuxFest Northwest
For the first time in 9 years I decided to take my camera to the after-party. I found a lot of happy people. If you like the picture enough, let me know. None of these have been through post, so I could clean up your portrait a bit. Great to see you all there! I look forward to 2014!
Backups: Using `find` Across a Panalopy of Directories
I love using the find command. In DOS, find is like grep. In Linux, find is the most powerful recursive DOS dir /s or Linux ls -r command you could ever put your saddle on.
One of the things you can do with find is to avoid directories, using the -prune switch. Like so:
find /usr/local -type d -a \( -name jre1.6.0_38 -prune -o -type d -print \)
Yeah, put your bike helmet on if you keep reading. That spat out a ton of gook. But was I lying? Well, grep out everything but what we should have pruned:
find /usr/local -type d -a \( -name jre1.6.0_38 -prune -o -type d -print \) | grep jre1.6
What if you have a series of subdirectories you want to include, but you cannot write enough -prune switches for them? This is a problem I frequently have. For instance, how do you exclude all your Firefox Cache directories, especially if you have multiple profiles? Great question.
I’d first use find to find all the directories I
do want to backup:
find /home/jed -maxdepth 4 -type d > /tmp/dirlist
Then you grep out things you really don’t want:
egrep -i "/cache|/Trash" /tmp/dirlist > /tmp/avoid
Then parse it into things you do want to find to avoid:
cat /tmp/avoid | while read F ; do echo " -path $F -o " ; done > /tmp/avoid2 ; echo "-path ./asdf" >> /tmp/avoid2
Now we can refresh our list of directories to descend:
find . -xdev -depth -type d \( `cat /tmp/avoid2` \) -prune -o -print
If we want to turn that right into files, modify the last print statement to find files:
find . -xdev -depth -type d \( `cat /tmp/avoid2` \) -prune -o -type f -print
Now if you want to find the files more recently created than your last backup in /home/backup/monday.tgz, try this:
find . -xdev -depth -type d \( `cat /tmp/avoid2` \) -prune -o -type f -newer /home/backup/monday.tgz -print
Is that enough to make you cry? Chin up, think of all the disk space you’re saving, and how much faster a specific backup can occur. This means you can run backups every 15 minutes.
Backups: Sorting Through a Restore
When you need to be rough with your data–change a bunch of files at once…and you might not do it right the first time. Or you need to recover something that grew a few bad sectors and you only have a bits of your file left. Do you have to restore ALL your work? How do you see where the changes are?
Let’s step thru a partial restore using tar and diff. For example, say I have a code directory that I’ve damaged with a regular expression. My work is in /home/work, my backups are in /home/backups.
I “untar” the backup in a temporary directory next to the work directory like so:
$ cd /home/ $ mkdir restore $ cd restore $ tar xzf /home/backup/monday.tgz $ cd /home
With the two directory trees next to each other, finding the differences is easy. The -r switch for diff tell us to search an entire directory tree, and the -q switch tells diff to display a brief description of just file names. (Diff will work on binary files, so this command will also work on graphics and audio files, just don’t forget the -q).
$ diff -qr /home/work /home/restore Files /home/work/lib/BugCatcherHelper.java and /home/restore/lib/BugCatcherHelper.java differ Files /home/work/lib/BugCatcher.java and /home/restore/lib/BugCatcher.java differ
This example is obviously contrived, and often what source control should be used for, but not everyone uses source control.
Linux Photography: Basic Darktable Tutorial
I know why my first few minutes with Darktable seemed so frustrating–they were all me scrubbing this modal interface looking for things I thought all should be in a menubar. But there are no menubars. While DT has quite a bit of keyboard shortcuts (not discussed today) Those were no help because you have to study the Settings dialog Shortcuts tab…no quick to get started with when you’re used to mundane office software. Darktable is as different from GIMP as GIMP is from Photoshop. Is DT similar to PhotoShop? You tell me.
- select photos in light table mode
- select an action group “basic”

- reset a module using the furthest right “standby” icon
- spot selects are often a rectangle

- turn down stars of photos you don’t want

- export selected group with the Export button at the bottom of the column. This will batch-export all the photos on your light-table screen.
Certainly this is nothing like the GIMP. You do have to scrub the interfaces to find all the gritty little features, but it is more batch-oriented and possibly a cleaner work-flow.
Backups: one quick file backup alias
When you have a file you need to edit and you have the foresight to think, “whoa, make a copy before I destroy…” you often copy hulk.txt to hulk.txt.old (that’s using the minimum of keystrokes:
cp hul[tab][tab] hul[tab][tab].old[enter].
Well, a week later, what do you rename your next .old file? .old2? No time to put this folder into revision control? Thought so. You can inspect that last modified time on your file with stat. Experiment with this first:
echo `stat hulk.txt | awk '/Modify:/ {print $2}'`
(*snrk* did I just get you use use Awk? OMG!)
So how does that help…more precisely, you’re asking how do I add that to a backup file name? One of many ways, and I will show you the method with least typing: use an in-place shell exansion.
cp hulk.txt .hulk.txt.`stat hulk.txt | awk '/Modify:/ {print $2}'`
STOP. What wee character did I just sneek into that filename? Hold on, first write it up in an alias so you can reuse it:
alias bu="cp hulk.txt .hulk.txt.\`stat hulk.txt | awk '/Modify:/ {print $2}'\`"
Right, the backslashes (or ‘hacks’ as I nic them) keep your statement from actually evaluating the command as soon as it’s defined. The backtick is the same as saying “bash -e …stuff...”. Anyhow, now type bu and you can backup hulk.txt again. Now type ‘ls’ and see where your backup is.
No file? And no error? Oh, right the period before name hides it (sneeky). This means the next time we accidentally do a “rm *” (which often appears when you say “rm * .old” — Computer, stop, replay with magnification: rm__*__.old ). You need a good-old-fasioned:
ls -a
It’s hiding. Let’s finish up here with your alias, properly written:
alias bu="\`cp $1 .$1.\`stat $1 | awk '/Modify:/ {print $2}'\`"
Can we do it without that crazy awk? Sure:
alias bu="\`cp $1 .$1.\`stat $1 --printf %Y '\`"
Now go make a backup…right now!
Lightroom vs Darktable [Tutorial Geek]
Here is a more in-depth comparison of features and processes available in Darktable and Lightroom.
Linux Photo Processing
At LinuxFest Northwest 2013 April 28/29, I will be giving a talk comparing the GIMP and Darktable. These are two very powerful photo manipulation tools.
I am particularly looking at two programs that provide a strong post-processing capability. You use them for different purposes and how they are used is quit different as well. There will be a few more posts on each of these. (What about digiKam? Honestly, I don’t know anyone who uses it, so it didn’t immediately come to mind. For all-around photo-management, digiKam is certainly worthwhile. I won’t speak against it.)
The GNU Image Manipulation Program (GIMP)
Largely, the GIMP is what many people might think of a Photoshop for Linux. Many would strongly disagree–commonly what you hear is this: Gimp is nothing like Photoshop. I think people will agree on this: if you need retouching, layer compositing, text, and pixel-pencil drawing, your choices are pretty likely going to be some version of Photoshop or the GIMP. If you dont want to pay for a copy of Photoshop but want to produce layered screen graphics or high-res graphics for printing, here’s your tool. (And while GIMP has some vector tools for pathings, it is not a vector drawing program–see Inkscape for that).
Darktable
If you have used Lightroom, (a semi-pro and above level raw photo organizer and post-process workflow program), Darktable shall fill an analagous role. Darktable is has no intentions of being a drawing program. Color control is Darktables primary focus. It’s internals operate on color as 32-bit floating point values, which is mighty accurate. However, this means it wants a 64-bit computer with at least 2 gigs of ram. WIth it you can run through a batch of photos imported from your SD card, pick a few 4- and 5-star photos, isolate that set, apply color correction and “make snapshots” of them to jpg or png images.
Next
I look forward to writing out a few examples comparing and constrasting how GIMP and Darktable are used. Linux and digital photography are getting along quite well these days, and I look forward to helping you get a leg up on these two programs!
Backups: using tar and find
If you are familiar with zip files, they are the DOS version of tar files (tar = Tape Archive). The tar utility is totally intended for storing backups. A quick way to backup your home directory is:
cd /home ; tar -cvf home-jed.tar ./jed
You might see that command grab a whole lot of stuff you don’t want to keep, including all your Firefox cache files and your Trash files. Also that archive is uncompressed. Lets get it compressed as much as we can, first, that’s easy:
tar -cvjf home-jed.tbz2 ./jed
Next, we can build a list of files we want to backup using find. Please don’t try and avoid the find command, once you begin to understand it, life in Linux really can improve. On our first try, we will pair it down with fgrep (simple grep) to exclude our Firefox .Cache directory.
cd home find jed/.mozilla/firefox \ | fgrep -v '.default/Cache' \ > /tmp/jed.txt
And following that, avoiding our trash can:
find jed/.mozilla/firefox \ | fgrep -v '.default/Cache' \ | fgrep -v '.local/share/Trash' \ > /tmp/jed.txt
Now think about why we want to use pipe operators in that second find command. Would it be easier as two commands both appending to /tmp/jed.txt? (Think about the overlap and duplication that results.)
If we wanted to use that file to guide tar, we change our tar command like so:
tar cvjf ./jed.tbz2 -T /tmp/jed.txt
In order to make regular backups a regularity, we need to make them pertinent and economical (of time and of space). We often do not want to back up ephemeral files that are byproducts of our work. If you program, you will have ready examples on your own drive: .a, .o, .out, .class code files often do not need to be kept if you make them several times a day.
Consider the example below. With it we can backup the substantive slice of our code tree to another drive on our system. We avoid the ephemeral files. We also chose to backup our code separately from the rest of our home directory. By doing this we can schedule code tree backups every hour, and schedule our home tree backups just once a day.
#!/bin/bash
function CodeSnap() {
local now=`date +%Y-%m-%d.%H%M`
local arcnom="/mnt/backup/code.$now.tbz2"
local flist="/tmp/code.$now.txt"
find ~/code -type f -a\
\( -name '*.xml' \
-o -name '*.java' \
-o -name '*.properties' \
-o -name '*.php' \
-o -name '*.pl' \
-o -name '*.conf' \
-o -name '*.pm' \
-o -name '*.c' \
-o -name '*.h' \
-o -name '*sh' \
-o -name '[Mm]ake*' \
\) > $flist
find ~/Documents -type f -a\
\( -name '*.php' \
-o -name '*.pl' \
-o -name '*.conf' \
-o -name '*.pm' \
\) >> $flist
tar vcjf $arcnom -T $flist
}
##
## Copyright (C) 2013, Jed Reynolds
## Free for non commercial use.
##
CodeSnap
Questions? I hope! You just saw a full strength, professional level bash script. If you don’t have questions, show me your script.
Backups: outline
Here’s some basic programs and techniques I’ll be covering about backups.
- tar
- rsync
- find
- date
- how to write “now” using date
- how to find files newer than your last backup
- all this will be done in bash






















































