Backups: using tar and find

If you are familiar with zip files, they are the DOS version of tar files (tar = Tape Archive). The tar utility is totally intended for storing backups. A quick way to backup your home directory is:

cd /home ; tar -cvf home-jed.tar ./jed

You might see that command grab a whole lot of stuff you don’t want to keep, including all your Firefox cache files and your Trash files. Also that archive is uncompressed. Lets get it compressed as much as we can, first, that’s easy:

tar -cvjf home-jed.tbz2 ./jed

Next, we can build a list of files we want to backup using find. Please don’t try and avoid the find command, once you begin to understand it, life in Linux really can improve. On our first try, we will pair it down with fgrep (simple grep) to exclude our Firefox .Cache directory.

cd home
find jed/.mozilla/firefox \
| fgrep -v '.default/Cache' \
> /tmp/jed.txt

And following that, avoiding our trash can:

find jed/.mozilla/firefox \
| fgrep -v '.default/Cache' \
| fgrep -v '.local/share/Trash' \
> /tmp/jed.txt

Now think about why we want to use pipe operators in that second find command. Would it be easier as two commands both appending to /tmp/jed.txt? (Think about the overlap and duplication that results.)

If we wanted to use that file to guide tar, we change our tar command like so:

tar cvjf ./jed.tbz2 -T /tmp/jed.txt

In order to make regular backups a regularity, we need to make them pertinent and economical (of time and of space). We often do not want to back up ephemeral files that are byproducts of our work. If you program, you will have ready examples on your own drive: .a, .o, .out, .class code files often do not need to be kept if you make them several times a day.

Consider the example below. With it we can backup the substantive slice of our code tree to another drive on our system. We avoid the ephemeral files. We also chose to backup our code separately from the rest of our home directory. By doing this we can schedule code tree backups every hour, and schedule our home tree backups just once a day.

function CodeSnap() {
    local now=`date +%Y-%m-%d.%H%M`
    local arcnom="/mnt/backup/code.$now.tbz2"
    local flist="/tmp/code.$now.txt"
    find ~/code -type f -a\
      \( -name '*.xml' \
      -o -name '*.java' \
      -o -name '*.properties' \
      -o -name '*.php' \
      -o -name '*.pl' \
      -o -name '*.conf' \
      -o -name '*.pm' \
      -o -name '*.c' \
      -o -name '*.h' \
      -o -name '*sh' \
      -o -name '[Mm]ake*' \
      \) > $flist
    find ~/Documents -type f -a\
      \( -name '*.php' \
      -o -name '*.pl' \
      -o -name '*.conf' \
      -o -name '*.pm' \
      \) >> $flist

    tar vcjf $arcnom -T $flist
## Copyright (C) 2013, Jed Reynolds
## Free for non commercial use.

Questions? I hope! You just saw a full strength, professional level bash script. If you don’t have questions, show me your script.

Another Great Linuxfest Northwest

What an impressive LinuxFest. It felt really pro this year with big printed badges and registration. I would have been. Happy to have picked up my badge friday night if I knew what the crowd might have been Sat morning.

The robot room was definitely Liam’s favorite. Thank you BAIRS! I really dug the competition robot. Seeing BPD’d bomb squad robot was quite a privilege for us as well. That was very generous for BPD to spend a few hours at Fest. Would have been cool of coulda got some footage.

I loved being able to poke into the Linux Action Room and it was a great opportunity to get to present on a live stream. Thanks, Chris, Bryan and Jeremy! Opening up the stream to some guest intro takes was a hoot!

The OpenStreetMap presentation was really neat and I would dig doing another bike ride map collection next year, now that I know how to better be prepared. I need to see how I can better use my BB to collect waypoints. It was fun taking a spin with Isaac, anyhow.

I’m grateful there were some people interested in bike rides for fest but I need to start that conversation earlier. But crazy ideas popped up:
– a display of how to charge mobile devices by bicycle
– ANT+ linux drivers that collect cycling telemetry
– getting penguin bicycle pennants printed to sell at the raffle booth
– any bike vendors or mfrs using open source?

I’d once againt like to heartily express my thanks, appreciation and gratitude to the LinuxFest Northwest organizers! It was a great fest and the best sunday of all the two day LFNW events yet!

Apache, MySQL, LinuxFest Northwest and Bikes

LinuxFest Northwest is coming soon on April 30-May 1. I often present on MySQL and was considering doing an Apache talk again. However, these are broad topics and I’d like to split them up into tracks. Who wants to also talk about MySQL or Apache? Time is short.

As a father of two action hero kids, I’ve brought Liam to Fest for two years now, and he’s enjoyed the robotics and computer lab. I encourage and hope to meet fellow parents coming on Sat who might have preschool or elementary age kids who might want to meet up. Little kids need a break, get bored easily, and having a soccer ball or frisbee or bicycle about might help them have some fun. Liam and I will be biking, if a small ride during lunch sounds good, biking around Squalicum Creek park/Birchwood is an option.

Friday and Saturday nights are often good fun. If you want to ride your bike to the pub and back to the Hampton, the route is pretty low traffic. Riding from the Hampton to BTC is a cinch, too! Ask, I shall post google maps links! (I’d also be happy to lead that little ride.)

Really stretching your legs on Sunday might feel great, specially after sitting thru so many talks, right? There’s growing interest in a Sunday Penguin bike ride after lunch. I think a ride up to Hovander Homestead would be a refreshing change from a weekend of lectures, no?
Meet at bike racks in front of G building. Food? I can do hot dogs on a propane grille!

Please rattle my cage and tell me you want to present and go for a Sunday ride!