Wednesday, August 06, 2008
Technology observations from my job search
For all that job searching is a pain in the butt, it gives you a chance to see what companies are doing and where their "pain points" are. Some observations from my search:
1. Companies are drowning in data, particularly small ones that can't afford to build a fancy glass house for racks of servers running Oracle. There's a golden opportunity here.
2. A few players are going after this opportunity, but they seem focused on the top of the market. This is probably reasonable from a business perspective, but my impression is that bowie-knives.com needs high performance stuff as much as Google.
3. The DB market may well end up splitting in two: high-performance query answering and search, and highly reliable archival and storage. The latter interferes with the former, and several places where I was interviewing definitely want high-performance query answering. (I'm building one of these puppies for my new company.)
4. There is lots of other movement going on in databases. With the end of Moore's Law, and data and the need for high-speed data processing growing quickly, new db architectures are appearing. Several companies are building various forms of "database appliances", and some new hardware stuff may help here.
5. Programming is about to get a whole lot harder as computer hardware changes intrude on programming in a way they haven't for at least 20 years. This is good for us old fogies who like hard "edge condition" problems, but Joe Java, who lives in a world where abstractions hide all the fun stuff, may have a hard time adjusting.
I saw this in a couple of companies: a bunch of smart young guys, steeped in the latest languages and alphabet-soup "skills", wondering how to get a factor of 100 performance speedup in their complex application without buying 100x more hardware. It can be done, but commodity approaches probably won't do it.
1. Companies are drowning in data, particularly small ones that can't afford to build a fancy glass house for racks of servers running Oracle. There's a golden opportunity here.
2. A few players are going after this opportunity, but they seem focused on the top of the market. This is probably reasonable from a business perspective, but my impression is that bowie-knives.com needs high performance stuff as much as Google.
3. The DB market may well end up splitting in two: high-performance query answering and search, and highly reliable archival and storage. The latter interferes with the former, and several places where I was interviewing definitely want high-performance query answering. (I'm building one of these puppies for my new company.)
4. There is lots of other movement going on in databases. With the end of Moore's Law, and data and the need for high-speed data processing growing quickly, new db architectures are appearing. Several companies are building various forms of "database appliances", and some new hardware stuff may help here.
5. Programming is about to get a whole lot harder as computer hardware changes intrude on programming in a way they haven't for at least 20 years. This is good for us old fogies who like hard "edge condition" problems, but Joe Java, who lives in a world where abstractions hide all the fun stuff, may have a hard time adjusting.
I saw this in a couple of companies: a bunch of smart young guys, steeped in the latest languages and alphabet-soup "skills", wondering how to get a factor of 100 performance speedup in their complex application without buying 100x more hardware. It can be done, but commodity approaches probably won't do it.
# posted by Foobarista : 9:21 PM
