thingelstad

Jamie Thingelstad's personal website

Month: November 2011 (page 1 of 2)

Our Family – November 2011

Thanks to Angie for taking the photo!

Count 404′s in group of access logs

I found myself needing to see all of the 404 errors in the access logs for all virtual hosts on my web server. I put all of my logs for a given application (in this case WordPress) in one place (/srv/www/wordpress/logs/$host-access.log). Logrotate kicks in to keep them segmented and compress them by day.

A bunch of Unix magic later…

zgrep " 404 " *-access.log* | \
cut -d " " -f 1,7 | \
sed s/\:.*\ /\ / | \
sed s/\-access.*\ /\ / | \
sort | \
uniq -c | \
sort -n -r | \
head -20

zgrep is just grep that handles both normal and gzipped files. Pipe that into cut to pull out just the data we want. The two sed commands pull out data that would mess up the aggregation (the IP address of the requester and part of the filename). Sort puts prepares the stream for uniq to do the counting. Then do a numeric sort in reverse and show the top 20 404′s in all log files.

Output looks like

    380 thingelstad.com /wp-content/uploads/2011/09/cropped-20090816-101826-0200.jpg
    301 thingelstad.com /wp-content/uploads/2009/06/Peppa-Pig-Cold-Winter-Day-DVD-Cover.jpg
    300 thingelstad.com /wp-content/thingelstad/uploads/2011/10/Halloween-2011-1000x750.jpg
    264 thingelstad.com /wp-content/uploads/2007/12/guitar-hero-iii-cover-image.jpg
    130 thingelstad.com /apple-touch-icon.png
    129 thingelstad.com /apple-touch-icon-precomposed.png
    121 thingelstad.com /wp-content/uploads/import/o_nintendo-ds-lite.jpg
    114 thingelstad.com /wp-content/thingelstad/uploads/2011/10/Crusty-Tofu-1000x750.jpg

Of course the next step would be to further the pipe into a curl --head command to see which 404′s are still problematic. That just makes me smile. :-)

As an aside, sort combined with uniq -c has to be one of the most deceptively powerful yet simple set of commands out there. I’m amazed at how often they give me exactly what I’m looking for.

Tammy’s Favorite Picture of Me

Tammy went digging deep into our Lightroom library looking for photos for our new exercise room. She told me she had found her “favorite picture” of me. I was very curious what picture it was. I like it. This is from the finish of the 2004 Chequamegon Fat 40. The kid in the background is out of place though, a little Photoshop could deal with that.

Move to Linode

This website (and all the other websites I run) should all be running much faster now. This weekend I flipped everything from my previous VPS at Slicehost to a new instance at Linode.

There is a ridiculous amount I could write about the move, and I’ll try to share what I think is most helpful to others. In general I can say that the Linode hosts are a lot faster than the Slicehost instance I had. Doing basic Linux stuff was 2-3 times faster on Linode than on Slicehost.

My new setup is also a lot faster due to how I deployed WordPress and MediaWiki. I’m now running everything on nginx instead of Apache. I’m also serving all my PHP out of php5-cgi instead of mod_php. Perhaps even more importantly I got all of my wiki and blog instances running under one PHP install for MediaWiki and WordPress. As a result, the APC module for PHP can do it’s job right. I’m now getting 99.9% PHP cache hits.

With all that said I fully expect I may have a thing or two not working right now. If you see anything broken a comment or email would be great.

Bring on the Snow

Good Commies Hi-Res Cover Art

Four of my friends from the B-Squad Softball team are in a band called the Good Commies. I was finally able to add some of these wonderful tracks to my iTunes collection. I insist on high resolution cover art so I fired up the scanner. Here are the files for anyone else who is looking for them. I couldn’t find anything other than thumbnails online.

iTunes Match Updates Ratings and Play Counts

Apple launched iTunes Match today and having all my music in iTunes, with three different computers, an iPhone 4S, iPad 2 and three Apple TVs I figured I could benefit from using iTunes Match for my music. I enabled it and it is still in the process of doing its first update. I was very curious about how iTunes Match dealt with certain metadata.

I’m a big user of Smart Playlists and many of them rely on song ratings, play counts and last played dates. I also often have Smart Playlists that reference other Smart Playlists. Some quick learnings from an hour of playing around.

  1. iTunes Match synchronizes ratings within 10-20 seconds between devices. I was playing a matched track on my iPad 2 and adjusted the rating. Within 20 seconds iTunes on my MacBook Pro reflected the new rating. I changed the rating in iTunes and within another 20 seconds it was updated on the iPad 2 without any action being taken.
  2. The play count for a track is updated promptly after the track is played.
  3. Unfortunately the last played date is not updated. This is annoying for me since I routinely use last play date to manage Smart Playlists.
  4. Smart Playlists that use other Smart Playlists as part of their criteria are not available in iTunes match. Simpler Smart Playlists are fine in iTunes Match.

Garage Reclaimed

While doing our remodel we lost the use of our garage. We used it for storage and the contractors had a big dumpster in the driveway. For most of the summer there was also a small lumber yard in the driveway and a portable toilet. So we parked on the street all summer. Best time to do it but an inconvenience.

This last weekend we got the garage cleared back out and I was irrationally excited to park both cars in there again.

20111109-232936.jpg

Typing Big Green Egg on iPhone

Have a Big Green Egg? Have an iPhone with iOS 5 on it? Then you absolutely need to go here

Settings – General – Keyboard – Shortcuts

Then create this shortcut to save your thumbs significant effort.

20111109-232413.jpg

Coffee is Oxygen?

The other day the coffee maker in the office was out of commission. From this note you can see the situation was dire.

20111109-231935.jpg

Older posts

© 2014 thingelstad

Theme by Anders NorenUp ↑