2005s
-
We are completing the 3rd phase of our landscaping project and they are installing a stone path with stairs in the back. This is totally smooth and hardly impacts us at all since it’s outside. They should finish this coming week and so far it looks great!
-
On Tuesday we had a new furnace and air conditioner installed. Our old one was going and we wanted to put in a higher efficiency unit. This went well and is now done after a couple of follow-up visits to make sure everything was done. We have a 96.2% efficient furnace and a 15+ SEER air conditioner now.
-
We had new shower doors put in the master bathroom. We now have the glass-style doors like nicer hotels have. This is a nice upgrade and completed on time.
-
We are having a new roof put on as well. This one started out very poorly with bad communication with this contractor and the crew not showing up for the first two days. Things are turning better though as of today. More communication and it looks like they should (weather permitting) finish by Wednesday. I really underestimated what a headache this one would be, it’s a huge mess.
-
The roof is in preparation for the installation of solar heat next. We wanted to get the roof redone before doing the solar projects (we are also considering solar electric) so we wouldn’t have to redo it all in a couple of years when it will be a lot harder. The solar heat will go on in mid-December but they are prepping the roof now with the appropriate pipes and moving things around.
Salut and Ravello
We stopped by two new restaurants yesterday and were happy with both. We had lunch at Salut in Minneapolis, at the 50th and France area. Salut is a fairly new French bistro. Don’t let the French part scare you off – they have an extremely approachable menu. It’s owned by the same people that own Manny’s (my favorite steakhouse!), Chino Lationa, Good Earth, etc. so you can rest assured that it’s going to be at least passable.
We entered the restaurant and I was shocked by it’s size. Much bigger inside than I expected from the outside. The menu was very good. Tammy and I had a glass of the Riesling. Wine at lunch? Sure, we’re on vacation! It was great. I had the steak frites for my lunch and Tammy had the steak sandwich. My tenderloin was amazing, and the frites were equally impressive. Highly recommend this place.
We then had dinner with our friends Jim and Suzie at Ravello. We ended up here because of a recommendation from a friend at the office, and really liked it. It’s in Long Lake so a bit of a drive unless you live in the western suburbs. They have a nice wine menu, and the lamb shank was delicious.
300 Gallons!
Yesterday we had Brian Broughten, the founder and owner of Advanced Aquarium to our house to get started on a bid to build a custom aquarium. He was here for about 90 minutes of good geek fun regarding aquarium stuff. Those that have known me for a long time know that I had a saltwater tank for the first half of the 1990s. In fact, my “Early Days” site still has the web page about it. I really like aquariums but I stopped for two reasons. First, I was a (relatively) poor student and couldn’t afford to do it right. I never had good lighting, and everything was DIY. Nothing against DIY, but that also meant somewhat crappy solutions for many things. The second issue was that I got sick of lugging buckets of water all the time. Now that we live in a house, I could have it plumbed straight in and not have to worry about pals and hoses.
The funny thing about the visit from Advanced Aquarium was that I used to shop at Brian’s store all the time. Broughten used to run a store out of his basement. My friend Mike and I would make somewhat regular trips there for stuff, but mostly to pick Brian’s brain and admire new fish and other creatures. When he came over I confirmed that that was in fact the same guy, and he actually vaguely remembered me as well.
Anyway, in a couple of weeks I’ll get a quote for the tank we worked out. I started getting so excited just talking about it. It would be 8 feet long, 2 feet deep and 30 inches tall – a whopping 300 gallons! All filtration and reserve water systems would be in the utility room. Out of sight, and sound. It would have dual pumps moving 3,000 gallons per hour through the reef. And it would have those magnificent metal halide lights that I could never afford before.
I’m trying not to get too excited, but I’m eagerly awaiting the quote.
RC Helicopter
My friend Chris, who is constantly finding new things to get into and get everyone he knows excited in, recently decided to turn his OCD towards RC helicopters.
In recent years helicopters have gotten much more approachable. Battery technology has evolved and you can now get a simple, ready-to-fly electric helicopter for a reasonable price. I picked up a E-Sky Lama 2 just like his from eBay a couple of weeks ago and have been having a blast!
First off, this helicopter is a toy more than a true RC helicopter. It’s small, about a foot long, and isn’t strong enough to fly outside unless it’s absolutely still. It is best suited for a large open-area in your house. It also can only carry it’s own weight. The first question most people have asked is “Can you strap a camera on it?” It’s surprising to me how many people ask this. Secondly, no, you need a bigger bird to do that with. (Chris just ordered a T-Rex, which can carry cargo and so much more. We’ll see how long I can hold out before upgrading.)
Even though it is a toy, it is a tremendous amount of fun after you learn to fly it. It took me about three 10 minute sessions before I could really hover with stability and not run into a variety of stuff. Now, after a few more sessions I have good control and am starting to make some maneuvers. My hope is to get to the point where I can race my father-in-law (he got one of these for Christmas from yours truly).
Photo Storage Article Published
I sent my Digital Photography Storage Explosion article to our editorial team at MarketWatch to see if they would be interested in an article version of it for our site. They said it looked interesting so I wrote up a much smaller version and it was published today. I got a byline! 🙂
This version is much abbreviated and less “research paper” like. If you find it interesting, check out the original, unabridged version.
Mazie Sleeps Through Night!
For the first time in the 196 days since she was born, Mazie slept through the night last night. She enjoyed restfulness, with minimal disturbances, from 7:00 pm to 6:10 am. Her mother was extremely excited but realistic that this is not a pattern yet. Her father was proud but thought she did this all the time. News at 11!
Mazie at 6 Months
Mazie turned 6 months old on Dec. 14th, and she had her 6-month Doctor appointment on Monday. She’s doing just great. She’s just growing faster and faster. She’s essentially 75th percentile for weight, height and head circumference. She’s mastered sitting up now and loves to sit on the couch or on the floor with tons of toys all around her banging things and having a blast. She’s also started real food now. She only had rice cereal, aside from milk of course, until she was over the 6-month mark. She’s started oatmeal now and really likes it. We’ve tried carrots which she was a bit suspect of, and at this point she’s decided that bananas are not for her. Whenever she gets a mouthful of bananas she gets a really funny look on her face.
I can tell she’s super excited about Christmas, or perhaps I’m just more excited about Christmas because of her. She’s going to love digging into all her toys. 🙂
I 💚 Hobbies
I’ve had fun this week getting caught up on some hobbies, restarting others, and considering picking up some more. I revisited my telescope for the first time in a while this week. I’ve been wanting an equatorial wedge, so I got that and played around with the whole rig some more. The wedge will allow me to get into astrophotography. While I still don’t feel terribly comfortable with my scope (I always feel like I’m going to break something), I’m much more comfortable than I was before. I took it outside last night and played around with the fairly clear, but incredibly light polluted, sky from our driveway.
I’ve also gotten a little fascinated with binoculars. Canon makes these amazing image stabilized binocs that would be great for nights out with the telescope (particularly if someone comes along) and for checking out birds. I admit that I’ve developed this weird birding hobby. I can’t wait for next summer with the new bird “sanctuary” we got setup this fall. It’s going to be crazy.
Then there is shortwave radio. This has interested me for years, but I’ve never taken the plunge. The place in town that sells my telescope stuff sells shortwave stuff (it’s actually more of a radio star, called Radio City after all). I was playing with this Yaesu VR-5000 radio there and thinking that would be a lot of fun to learn about as well. I could put a big antennae on the roof and tune into Europe, Asia and the Middle East.
Add all this to photography, anything and everything computer related, digital video – oh yeah, and stuff like cycling and hiking. I need more time in the day.
Celebrity Spotting: Al Franken
Okay,so maybe this is the Minnesota version of a celebrity spotting. It’s not Madonna, or Bono, or any other single-named person (really, how pompous do you have to be to only have one name?).
Today when Tammy, Mazie and I were going into the Galleria for some shopping, we were passing in front of the entrance of Barnes & Noble and there is Al Franken getting out of a black Lincoln town car. Apparently he was in town for a book signing event so a lot of people had this “Celebrity Spotting”.
Anyway, we didn’t gawk or anything. Didn’t want to make Al feel awkward. 🙂
The "Not-So-Open" WRT54G
This past week I was in Radio Shack and saw they were running a great deal on the Linksys WRT54G wireless router (double rebates!). I’ve had a D-Link 624 for a long time and its worked without any issues. However, I’d read at length about the WRT54G and it’s open platform. Linksys built the WRT54G on a Linux platform and made the firmware available via open source. There is some debate wether that was intentional or not, but the GPL forced them to make it available. The result was awesome 3rd party firmware like Sveasoft and DD-WRT.
So, I picked up one of these cheap units at Radio Shack and today I sat down to play with it.
Then everything went south…
About 20 days ago Linksys (now owned by Cisco) started selling the WRT54G v5. All the previous versions of this ran Linux, but now they are using VxWorks. Unfortunately, this means no 3rd party firmware. This seems like such a bad idea, particularly since this router became the success it is because of 3rd party firmware. Luckily there is a new model, WRT54GL that is the v4 box with Linux.
Buyer beware if you are looking to get one of these.
Home Vacation
I’m taking this week off, using up some of my remaining vacation time before it vanishes. Tammy and I are not feeling adventurous enough to go on a big trip with Mazie, so we are staying home for the week. I’ve picked a couple of projects to work on (like I need more of them!) and spending time relaxing and hanging out with Tammy and Mazie. We have some babysitting arranged so we can head out and have some outside fun as well.
I’m feeling really pumped about what I’m working on at the office lately (read: change the world), and it’s hard to disconnect. But, I know it’s healthy and I will come back after a week with even more focus and clearer vision.
'tis the Season
The holiday season is now officially upon us. We were at Southdale today and Mazie got her first picture with Santa! She didn’t have a lot of things on her list, but I clearly heard Santa mentioning something about the Teletubbies.
She had a great time today going around to different places and getting carried around by Mom and Dad. Such a cutie!
Santa’s little stand there had several print options, along with the option of getting a CD with the “Hi-Res Digital Photo” on it. I asked what resolution it was and none of Santa’s little helpers knew. They were shooting with a Canon Digital Rebel, so I asked if it was just native resolution of the Rebel. They figured yes. $26 later I have a CD with a 3.15 megapixel image on it. Now, that may be “Hi-Res” to some, but it’s practically a camera-phone in my book. Grrr…
Miss... TiVo... So... Much...
We just had our roof redone, and of course they had to rip the DirecTV satelite dish off the roof. Last night, for the first time in 6 years (at least), I had to actually sit down and watch a TV show at the time it was on and with commercials! It was stunning.
Technology is interesting. It starts as a nice plus, and faster than you can imagine turns into a must have. I wanted to catch this show and I realized I only knew (and this was vague) that it was on Monday night. I didn’t know what channel, and wasn’t sure what time. I had to go to TVGuide.com (I felt like I needed a shower afterwards) and look up when it was on.
Then, had to rush to the TV and watch it at the time the network wants me to. Man, this whole thing just blew.
Thankfully the dish gets put back up tomorrow morning, and TiVo will spark back to life and make our world a much happier place again. Whew!
Restaurant: Naar Grille
Tammy and I went to our first movie since Mazie was born today. We saw Walk the Line if you are interested (and yes, it was very good). We also went to a new restaurant for lunch before the movie. We tried out Naar Grille. They opened up about 2 weeks ago. If location really matters I worry for them. They are not in an ideal location, there is no major road with people zipping by. The food however was very good. We went on Sunday which we did not know was buffet only, but it was quiet tasty.
The food is billed as Mediterranean fare. The gyros were very good, and everything looked tasty. I’d encourage people to give it a try - particularly if you like this cuisine.
Google Analytics - Overload?
I was really excited to see the release of Google Analytics this week. I’ve been looking for a better way to track site usage and Weblog Expert just isn’t cutting it anymore. I would suggest that it’s essentially impossible to do site analytics using log files anymore. Spiders, robots, spammers, etc. make your log files into total garbage. Even with significant filtering, you still get mountains of trash.
I was moments away from plunking down “real money” on Visistat to solve this problem, and then Google Analytics came along free of charge.
I’ve been impressed with the data display and am really excited about it, but Google (for the first time I can recall) seems to be completely overwhelmed with traffic. It’s Sunday morning and my reports are still showing Thursday morning data. They target a 24 hour turnaround in report data, but this is a bit crazy. This is what they are telling customers.
Waiting for Data
Analytics has been successfully installed and data is being gathered now. The demand for Google Analytics surpassed even our highest expectations and as a result some customers may temporarily experience report-update delays. All data continues to be collected and no data has been lost. We are currently adding resources to ensure high-quality service. We apologize for any inconvenience.
I’m eagerly waiting for them to deal with the load. Maybe they need to add a few thousand more servers . However, if they can’t get this thing working soon I think they may end up with a real PR problem. Especially considering the target audience, web builders!
Terabytes of Disneyworld
A couple of years ago when Tammy and I visited Disneyworld I was struck by the pervasive use of digital cameras and digital video equipment. I started to wonder, as I watched everyone around me merrily clicking away megabytes of storage, how much data is all this? This train of thought simply would not leave my head, and I ended up doing some analysis on my own digital photography behavior, and ultimately trying to answer the question: How much storage does Disneyworld use everyday?
For this and a wonderful analysis of the storage issues with digital photography, read my article Digital Photograpy Storage Explosion.
Digital Photography Storage Explosion
This is also avaiable in PDF format.
You can also download the data used for analysis.
Abstract
Digital cameras have spread at an amazing rate. There are almost no film cameras being developed anymore, having been relegated to antique status. In fact, at the most recent photographer’s expo, for the first time in the history of the event, no 35mm film cameras were introduced. Canon, Nikon and others are only focused in the digital realm. Even for high-end photography, digital has now become the “king of the hill”.
While digital photography has huge benefits, it comes with a data management challenge that is an iceberg we are just now seeing the top of on the horizon. The average home computer user is ill-equipped to deal with these challenges, and even worse, most do not even realize the problem exists. This data problem ranges from photo organization, to storage and disaster recovery. These issues, particularly the last two, have the potential to be the Achilles heal of digital photography.
This document looks deeper into the trends regarding the storage requirements of digital photos, and some of the ancillary challenges that need to be addressed.
The Problem
I got my very first digital camera, a Fuji MX-700, in June of 1999 and instantly fell in love with it. That camera awoke me to the wonders of digital photography. Instant gratification after taking the picture and the ability to easily share images over the Internet were just the beginning. I had for years owned the most popular film camera ever made, the Pentax K-1000. It collected dust for years, and I recently released it to a better owner through an eBay auction.
With its very small sensor and file sizes that camera couldn’t compare to the quality of film. Many people scoffed at the idea that digital would be a real format for photography. The same fate fell on the first MP3 players. The failure in this logic is not accounting for the rapid pace that digital technology advances at. It was, and continues to be, only a matter of time before digital mediums eclipse their historical “analog” counterparts.
| Manufacturer | Model | Sensor Size |
|---|---|---|
| Fuji Film | MX-700 | 1.5 million |
| Fuji Film | MX-2700 | 2.3 |
| Toshiba | PDR-M70 | 3.37 |
| Canon | PowerShot G2 | 4.0 |
| Canon | PowerShot G3 | 4.0 |
| Canon | Digital Rebel | 6.3 |
| Canon | S500 | 5.0 |
| Canon | EOS 20D | 8.2 |
Every photo I’ve taken since that day in 1999 has been digital.
While much about this transition has been wonderful, and I certainly would not return to film, there are challenges. In the period from June 9, 1999 to October 27, 2005, a span of 2,332 days, I’ve taken over 20,000 pictures. That represents over 8 pictures a day for several years, a frequency that is unfathomable in film photography. This collection now represents nearly 50 gigabytes of storage that is more precious than any collection of spreadsheets or word processing documents I can imagine. Vacation memories, the birth of a child, weddings are just some examples of this immensely important data. Figure 1 shows the storage requirement of these photos over time.
Figure 1
The explosion of storage required to accommodate increasing numbers of photos at higher resolutions is a genuine challenge. This data management problem was, until recently, inconceivable to the average person and was only an issue that organizations with specialized technical staff had to deal with. The idea of backing up 50 GB of data on anything other than a spare hard drive is preposterous; optical media is too small (including DVD) and tape is cost prohibitive, hard to use and prone to failure. Even the hard drive option has only recently become an option with the advent of USB2 and Firewire, before that there really was no option.
The amazing thing about this storage issue is the rate at which it is growing. It took 299 days for me to collect my first gigabyte of photos. I didn’t reach the five gigabyte point until 1,144 days of photography. That is approximately half of the time I’ve had a digital camera. In the 1,188 days following that I’ve added another 41 gigabytes of photos, all while continuing to accelerate the storage needs. Figure 2 shows the storage size of photos taken by year (only showing years with complete data).
Figure 2
This may sound huge, but luckily this technical problem has been helped by hard drive manufacturers. Figure 3 below compares the growth of my digital photos to Moore’s Law1 and Kryder’s Law2. Both of these laws relate to the growth rates of technology. Moore’s Law, reduced, states that computing power doubles every 18 months. Kryder’s Law is a similar law for storage, but doubling even faster at 13 months.
Figure 3 - Plotted on logarithmic scale.
It is interesting to note that both Moore’s and Kryder’s growth rates were well ahead of me for nearly the entire first year of photography. Since then, I’ve steadily outpaced Moore’s and am knocking aggressively on the top-end of Kryder’s growth rate.
The casual observer could stop reading here and think there is no problem. The growth of storage (theoretically at least) is growing at the same pace as digital photography. However, that isn’t the entire story. If we make the assumption that the average person consumes technology at the average rate (dictated by Moore’s Law) what we find is that on average, people have to buy more technology than they would have historically purchased to deal with storing all this data. In the past the default hard drive options at Dell.com would be fine, but as digital photography explodes the average user will be buying more sophisticated systems that come with a variety of technical challenges. This is a hidden, and mandatory cost and still doesn’t resolve the issue of backing up this data.
Digital Behavior
Digital photography introduces people to an entirely new way of using photos, with one of the biggest implication being how many pictures you actually take.
Take a moment to look at Figure 4 below showing the number of photos taken each month with a 6-month moving average.
Figure 4
You can see a clear growth in the tendency to simply take more and more photos. I have been taking 36% more pictures every year since starting with digital photography. Compounding that problem is the growth of megapixels, and with it file-sizes, in digital cameras. The storage space required for the increasingly growing collection of photos is growing geometrically.
Figure 5 is interesting for other reasons as it identifies how events like vacation (July 2002, July 2004) affect these trends, as well as major life events like the birth of a child (June 2005). Going to Europe in July 2004 now carries with it a permanent issue of keeping 4.5 gigabytes of data around forever. That is notably more challenging than putting some prints in a photo album and placing it on the shelf. One is an active process of sophisticated data management, the other a passive matter of organizing physical documents. As we continue to use digital technology to capture events in our life we may begin to look at the digital cost of those events. Going on vacation will cost $2,000 and have a data load of 3GB, my childs school play will have a data load of 1.5GB, etc. All the while being additive of course, we are never removing old content.
Figure 5
Let’s take a moment to look at our digital photography behavior. This is somewhat tangential to the issue at hand, but is interesting nonetheless and provides some depth to the trend of taking more photos over time. Figure 6 below shows the number of digital photos taken each month during the sample period.
Figure 6
It is clear that the summer months represent big picture taking times. December also reflects a high volume of pictures, no doubt due to the Christmas holiday. Figures 7 and 8 show photographic activity by day and hour.
Figure 7
Figure 8
The results are not terribly surprising. The majority of photos are taken on the weekends and there are very few photos taken while sleeping. It is a little surprising that the afternoon is higher than the early evening hours.
Terabytes of Disney World
So clearly this is a growing mountain of data, and this is just one families archive of photos. The impact of this on a large scale, with hundreds of millions of photographers throughout the world is stunning. On a recent vacation to Disney World I was particularly taken with the pervasiveness of digital photography and digital video 3 equipment. I started to wonder what the storage requirement of the thousands of vacationers visiting Disney World everyday was.
I contacted Disney to find out how many visitors on average attend Disney World each day but they do not disclose that number.
Date: Fri, 21 Oct 2005 09:16:30 -0700 To: Subject: Re: Request for General Information about Walt Disney World Resort From: Walt Disney World Guest Mail Reply-To: Walt Disney World Guest Mail
Dear Jamie,
Thank you for contacting the Walt Disney World Resort.
We appreciate your interest in obtaining information about guest attendance. Regrettably, this type of specific information is proprietary and cannot be released. We are sorry that we cannot be of more assistance, and we appreciate your understanding.
If you have questions or need further assistance, feel free to contact us.
Sincerely,
Ronda WDW Online Communications
Denied the true information, I did find some estimates of attendance on the web4. Using this estimate I concluded that Disney World averaged 38,462 visitors a day. According to at least one survey 54% of Americans are using digital cameras5. It seems straightforward to me that the average visitor to Disney World is also more likely to have a digital camera, but we’ll use the average anyway.
The next challenge is determining the distribution of camera types. I used a standard bell curve distribution from 2 megapixel to 8 megapixel (shown on the right), and also assumed that everyone is taking a JPEG file. The last bit of information needed is an estimate of how many pictures each one of these digitally equipped visitors would take at Disney World. I estimated 100 pictures, which I would suggest is high, but realistic given that you are on vacation, at a special place and have a high desire to capture “life’s moments”. For comparison, I took 241 pictures in two days at Disney World, averaging 120 pictures a day.
Given these assumptions, the attendees to Disney World generate 3,869 GB, or 3.78 terabytes, of data every single day. An equally interesting way to look at this is bandwidth. If you used the average cable modem available in the US today, and transmitted data at the maximum rate every minute that Disney World was open 6, you would have to have 420 cable modems running in parallel (839 Mbps) to handle the data.
Challenges
The challenges posed by dealing with such a large volume of data for both the individual, and the technology world as a whole, are significant. When you add to the equation that these are your memories, some of the most precious things you have, the criticality of the problem becomes clear.
Backing up data of this size is not easy. My solution thus far has been a plan of massive redundancy. I have four computers in my house that have large hard drives and every night an automated task runs to replicate any new images onto all of the backup points. This provides great redundancy and protection, however it’s expensive using an additional 200 GB of storage for redundant copies, and growing every day. Realistically this solution will not be viable in a couple of years without further investment in high-end technology.
An additional problem with this is that it is all in one physical location so I’m not protected from catastrophic failure (e.g., fire burns down my house). It would be great to leverage a broadband connection to backup to a service provider over the Internet, however the pricing models for this make it unreasonable, costing hundreds of dollars a month. Alternatively, some technically sophisticated families can adopt a buddy system mirroring their photos between each other using backup programs with FTP options. This is a nice option, with relatively low costs but is hard to setup and issues like encryption must be addressed to insure privacy.
In the meantime, the best solution for offsite storage seems to be a removable USB or Firewire hard drive pair that you rotate to a safe deposit box. This is cumbersome and requires manual intervention which will only work for the most diligent of people.
We may think that the storage requirement problem is easily solved by Kryder’s law and the hard drive manufacturers; there are further challenges that will make storage requirements grow even faster.
The Raw Problem
Almost all digital photography is done straight to a JPEG file. This is an easy format that is usable on nearly every computer that exists, but may be a bad decision for pictures that you really care about. Most cameras support a file format called RAW which captures the photo in a different way, allowing you much more flexibility in the future. To keep it simple, a RAW file can be considered analogous to a negative 7, with shooting to JPEG being similar to a Polaroid. You can then digitally develop that RAW file and improve the picture significantly. However, this comes at a cost - storage. It should be assumed that in the relatively near future camera manufacturers will migrate away from JPEG as consumers look for higher quality and more post-processing capability.
Raw files are compressed, but not as much as JPEG files. The resulting file is approximately 300% larger than a comparable JPEG file. Figure 9 shows the theoretical effect of using RAW photography as opposed to JPEG.
Figure 9 - Plotted on logarithmic scale.
You can see the problem right away! We break above Kryder’s law immediately and stay well above it. At the end of our sample period our storage needs have tipled from 46 GB to 143 GB. Let’s look at this growth on a linear scale in Figure 10, as opposed to the logarithmic scale in Figure 9. The huge disparity in storage needs versus growth are profound.
Figure 10
While it is appealing to do photography only in RAW, the storage implications make it very challenging.
Other People’s
Another interesting phenomenon in digital photography is the impact of other people pictures. I have a couple thousand pictures that others have taken that I keep in my collection. Note, those pictures are excluded from all the analysis in this document. These photos further increase my growth rates.
I call this trend “group photography”. This is very typical with extended family outings or holidays. Several people with digital cameras will take pictures throughout the day and then one person will collect the photos from everyone. They will then synchronize the timestamps of the photos and distribute the entire collection of photos to everyone. This is a wonderful thing, but amplifies even further the massive storage issues related to digital photography. Now that trip to Disney World results in several hundred photos instead of the smaller hundred that you took by yourself.
Conclusions
I believe that I am seeing the challenges of this problem sooner than the average person. I may take more pictures, and I’ve been doing digital photography for a long-time. However, it is clear that the industry needs to provide both software and services to help the average consumer with this problem, or run the risk of huge issues as people start losing more photos and precious memories and digital photography experiences a customer backlash. It is only a matter of time before large numbers of people start running out of hard drive space, or have a virus delete their entire summer vacation. Given the huge storage sizes it is unlikely that people will have backups around. After all, how many people can really honestly say they practice good data management processes even with small files?
I already know several people that print all their digital pictures for this very reason. They simply do not trust their computer, or their own skills at managing the data, and print everything for archival. This seems cumbersome to me, but I understand the comfort that having those photos in storage would provide. Far too many people have been clobbered by an Internet virus and lost data. They never want to lose their memories.
There is a rather simple suggestion to make this problem better and that is editing. Most people tend to keep every digital photo they ever take, never deleting anything. I do not like the idea of deleting photos unless they are plainly out-of-focus or otherwise bad. I suggest this problem should be solved by the technology that created it.
There are several things that companies like Canon and Nikon should be doing to assure this problem is managed. Digital camera makers should be pushing the burgeoning online backup service market to lower it’s prices for large amounts of storage. There are hundreds of companies that will backup data for you to a managed service over a broadband connection. The problem is cost. These services still operate in terms of megabytes, not gigabytes. They all would charge several hundred dollars a month for this service.
Once solving the backup problem with cheap, reliable service provider model we need to address the software problem. There are no photo management programs that do well managing 20,000 photos. Programs like iPhoto and Picasa became unusable with very large photo collections. Additionally, the metadata that you attach to photos for indexing and searching needs to be self-contained in the photo files themselves, like the EXIF data that the camera embeds. The industry cannot try to lock people in to specific software packages by storing metadata in proprietary formats.
There are independent software developers working on the software problem, but it is not moving fast enough. The managed services market has held it’s prices unreasonably high even while storage costs continue to decline. The is an opportunity for services like Flickr to provide options to help these problems, but informed customers will be hesitant to turn over such important data to a service like that.
In the meantime, the most cost-effective solution is to leverage a “buddy system” with others that have broadband connections. Swap hard drives and use software to backup encrypted copies of your digital photo collection to their network. As is too often the case though, this solution is only viable for the technically sophisticated user with their own infrastructure.
1: For more information on Moore’s Law see Wikipedia at http://en.wikipedia.org/wiki/Moore%27s_Law.
2: For more information on Kryder’s Law see Wikipedia at http://en.wikipedia.org/wiki/Kryder%27s_law.
3: This document doesn’t attempt to address the storage issues related to digital video, however, it is worth noting that they are orders of magnitude bigger than digital photography.
4: Article at http://www.wdisneyw.co.uk/attend.html.
5: Article at http://www.letsgodigital.org/en/news/articles/story_4267.html.
6: Disney World Magic Kingdom is open from 9:00 am to 8:00 pm.
7: For more information on the RAW format see http://www.luminous-landscape.com/tutorials/understanding-series/u-raw-files.shtml.
Construction Zone
Our house has been a construction zone this last week, and will hopefully start to wind down after this next week. We scheduled a number of projects to occur right now.
I don’t know how people put up with remodeling their entire house. I’d be a basket case.
San Jose Trip
I spent last weekend in the San Jose area for my friend Scott’s wedding. Scott and I met when we were freshman in college. We hit it off right away sharing many techie, geek hobbies and a general interest in how things work (Scott was a EE major). Many a night were spent jawing on and on about computers, cars and numerous other topics. Scott moved to San Jose a number of years ago so we don’t see each other as often anymore. As is typical with guys we are not real active pen pals, so it was great to spend some time with him and be a part of his big day!
I flew into San Jose airport with a sense of deja vu. The last time I had flown into that airport was for my friend Phil’s wedding. Seems SJC is exclusively reserved for wedding use in my book. :-) I landed on Friday around noon with enough time to drop my junk off at the hotel and then go meet the guys for a groom’s day out.
We went to Lemans Karting in Fremont (where Scott and Cara live) and had a blast! 15 of us suited up, got our helmets on, and headed out on the track. The karts hit about 25 mph, but the scale sure makes it seem like a lot faster. I was really surprised by how much force it took to steer them through the hairpin turns. All of us had pretty sore hands and arms afterwards. I did pretty poorly in the first heat placing 9th out of 10 with a best lap of 26.013 seconds. However, I improved greatly in the second heat placing 3rd out of 10 with a best lap of 24.230 seconds. After that we went to City Beach for drinks, appetizers and dinner. Good food, good conversation and a really fun bunch of guys.
On Saturday morning, the big day, Scott and I rode with his friend Roger up to Club Sportiva in San Francisco. Cara arranged for Scott to have a Ferrari 308GTSi to drive for the day and I was accompanying him up to go get it. As an aside, Roger drove a Toyota Prius and I sure found that thing cool. It was a “shiny pebble” moment extraordinaire!
Club Sportiva is a pretty interesting concept. If you have a penchant for exotic and interesting cars, but lack the wallet to own them, you should check it out. We picked up the “Magnum Ferrari” and Scott drove us out on 280 and then up to Skyline drive to stretch out the car a bit. Very nice ride. One of the funnier moments was when a guy in a Ferrari 550 Maranello pulled up next to us, blipped his horn to say “Hi”, clicked his paddle shifters down twice and disappeared in a puff.
We stopped midway on Skyline Drive to have lunch at Alice’s Restaurant. It’s a little unclear if it is the Alice’s Restaurant. It does not appear to be “a half-a-mile from the railroad tracks”. On their menu they specify that it was created around the same time as the song, and the owner’s name was Alice. Good enough for me.
The wedding itself was in Half Moon Bay. Photos were on the beach with the waves pounding in and the ceremony itself up the coast shortly looking out on the ocean. It was a small, casual wedding. A very nice time with lots of laughter and joy. Congrats to Scott and Cara!
Baby Stuff
A couple of weeks ago Tammy, Maze and I all loaded into the minivan and headed on down to Iowa City. Our friends Dan and Melanie had just had their first baby, Henrik. And as a kicker our other friends Lisa and Eric had just bought a house and are expecting their first. It seems everyone is doing this baby thing!
It was great to see Dan and Mel and little Henrik. It was a little like stepping in a not-so-way-back machine because I had almost no real recollection of Maze being that small! I held him for a while and he felt light as a feather as he slept away happily in my arms. Wonderful little baby really.
The other thing that was funny was I realized how quickly I had become an “old hat” at this whole Dad thing. In a short 4 months it seems I’ve gotten pretty comfortable with it all. It was fun to see Dan in the same space I was in when Maze was born.
Dan got a blog running right before Henrik showed up and has been commenting on the experiences. It’s a fun read. (Unfortunately, you can’t comment on it, which makes me sad.)
Music Everywhere!
A few months ago I bought the Roku M1000 MP3 player for Mazie’s room. I really liked it, even wrote on here about how great it was. Well, I liked it so much I decided to buy two more. Not the M1000, but the cheaper M500. Roku was running a sale, I think moving them out to make room for new models. I picked up two to put the whole music collection in the bedroom and a yet-to-be-determined location.
Install was easy as can be. Hardest part was finding a power outlet for it. (Note to self, need to redo power in bedroom!) Still like these units, even though I’m ultimately left a little frustrated with UPnP servers.