Thursday, March 17, 2011

The re-emergence of DIY vs Big Organizations

Wow! Limor (of Adafruit Industries) is on the cover of Wired. Wicked. Congrats, I am filled with envy. =o) Even if she does look a little bit Photoshop'd. =o/ But, nice homage!

It's great to see someone from the DIY community get such recognition. I haven't read the issue yet, but extrapolating from some of the phrases on the cover, I imagine it is a celebration of the re-emergence of independent tinkering.

I have actually put quite a bit of thought toward this topic having recently jumped back and forth between the DIY hobby culture, serious academic research, and massively funded commercial product development. I've had the fortune to observe people trying to make new and interesting things at extremely different scales...from $100 budgets to $100,000,000 budgets.

One thing that I find very consistent: good ideas come from anywhere. The biggest factor in predicting where good work will come from is "how much does this person actually care about what they are working on?" In fact, big budgets and a sense of entitlement can actually hinder the emergence of interesting ideas. Having the *expectation* to do really great work can lead people or organizations to develop tunnel vision on "big" ideas, and miss out on smaller ideas that end up having a lot of impact or dismiss seemingly silly approaches that actually end up working.

There's a really great TEDx talk by Simon Sinek that touches on this. He actually brings up a number of great points in his talk, but the one I want to highlight here is his anecdote about Samuel Pierpont Langley vs. the Wright Brothers in pursuit of powered flight. Langley represented the exceptionally well funded professional research organization, and the Wright Brothers were the scrappy passionate pair of DIY'ers. Today, we now know the Wright brothers as the ones who created the first airplane and most have never heard of Langley. Big investment is not a very strong predictor of valuable output. But, an individual's willingness to continue working on the same problem with very little to no pay... is a good predictor.

The great thing about the hacker community is that, generally, most of them fall into the later category. Independent developers and hobbists care enough to spend their own money to work on the projects they believe in. As a result, I'm finding that the delta in the quality of ideas from a well funded research group, and the independent community (in aggregate) is getting smaller and smaller by the month. Increasingly, the best "hobby projects" surpass the quality level of "true research" work in the same area. This startling lack of contrast (or sometimes inversion) becomes laughably evident when I am reviewing academic/scientific work submitted for publication on a project that uses Kinect, and then the newest Kinect hack pops up on Engadget that simply beats it hands down.

Now, I could simply make a kurmudgenly claim that the quality of professional/research/academic work has gone down. But, I actually don't think that's true. In my opinion, what is happening is that the quality of independent projects are getting better.... fast. Which, I think resonates with this observation of a "DIY Revolution".

But, why is this re-emergence happening now? Wasn't is just a few years ago people were lamenting about how "black boxed" consumer products had gotten, and that the good old days where you could open up a product and futz with the innards in a meaningful way were gone? What's changed to cause this apparent re-birth?

I have a theory.

My Theory about the Re-Emergence of the DIY community:
In the 90's and early 2000's, Moore's law was absolute king. The primary deciding factor in purchasing an electronic product was simply how fast it was. This meant an intense focus on tighter and tighter integration of components and all the functionality was disappearing into tiny little black chips that could not be accessed nor modified by mere mortals. But now, people barely talk about raw "megahertz" or "megabytes" anymore. General purpose computers have gotten "fast enough". We now want specialized kinds of computers: one that fits in our pocket, plays games in 3D, one shaped like a tablet, one that goes in our car, one that can go under water, or get strapped your snowboard and not break. We have reached a surplus in computing power that makes it affordable to build (and buy) devices for smaller and smaller needs. Our imagination for what to do with computing has simply not kept up with Moore's Law. So, we find more uses for more modest amounts of computing power. But, what does this have to do with the DIY community?

A byproduct of having such an immense surplus in computing, is that the tools you can buy within a hobbist budget have also gotten exponentially better in just the past 3-4 years, while the improvement in professional tools have been more modest. The difference in capability between the electronics workbench of a professional engineer and a hobby engineer is getting really really small. Kinect is an overwhelming example of this. The cost of a high quality depth camera dropped nearly 2 orders of magnitude overnight. As a result, hobbists are out pacing many professionals in the same domain simply due to sheer parallelism. Perhaps not as dramatically, but this is happening with nearly all genres of electronic and scientific equipment. One day, maybe we'll see backyard DIY electron beam drilling for nano-machining.

When it is no longer about who has the most resources, it's about who has the best ideas. Then, it becomes a pure numbers game:

Take 10,000 professional engineers vs. 1 million hobbists with roughly equivalent tools. Which group will make progress faster? Now, consider that you have to pay the 10,000 engineers $100K/year to motivate them work, and the 1 million hobbists are working for the love of it. Does that change your answer? Even if it doesn't, you have to concede that there does exist a ratio which will make the output of these two groups equal. It's merely a matter of time.

If you follow me through this argument, which I won't claim to be bullet proof but it explains the trends we are observing quite nicely, then this has an interesting implication on organizations that are currently funding big research groups. When it's simply a matter of who has the best ideas, it's tough to try to employ enough people to get good coverage. You could try to spend a lot of energy on trying to find the "best" people, but that's about as challenging as predicting the stock market. Some inventors simply go "dry" of good ideas and end up not providing a good lifetime return on investment (I fully expect this to happen to me someday. I just hope it happens later rather than sooner.)

So to me, this suggest 3 options for big exploratory organizations:

1. Start tackling more resource intensive problems - things that fundamentally cannot be done today for a few thousand dollars, but at some basic level requires materials, tools, energy, computation, space, manpower that is impossible to obtain at a hobby level. The LHC and space programs are good examples of this. Even if the end goal may be of debatable near term economic value, there is a high probability that unexpected derivative technologies/projects will bring commercial/educational benefits elsewhere.

2. Empower everyone within your organization to do exploratory work. The tools are cheap and "research groups" have no monopoly on good ideas. It's hard to know where lightning will strike, so make sure you encourage it anywhere and hope you haven't missed a spot.

3. Partner with the outside developer community. There is plenty of precedence where using the resources you have to channel the creative power of the masses through the platforms you control can bring a tremendous amount of value if done in an organized manner. It is the rocket fuel that powers companies like Facebook, Twitter, and Groupon to go from non-existence to dominating entities in less than 3 years. The same can absolutely happen with traditional physical electronics and other consumer goods. It simply requires treating your customers as potential partners, rather than assuming they are all potential predators.


Vermine said...

Excellent synthesis.

This empowerment of the individual is not confined to electronics and tools, it's true for a lot of domains ranging from politics to psychology too, not necesseraly involving hardware tools.

Numerous studies have shown that a panel of "average" passionate brains are better at solving problems and coming up with creative solutions than the single "genius" brain of an expert.

Internet and the way it makes ressources available to everyone is behind this.

Elites have always feared what they call "horizontalism" and were keen on preserving a vertical, hierarchic, structures... It's understandable, it's their death we're witnessing.

Kdansky said...

Add another factor: Sharing of information has gotten easier than making a pot of tee. The internet offers a huge wealth of information that your hobby project can profit from. The fundamental (usually expensive) problems can be solved for virtually free.

Examples from a CS point of view: Data storage. When twenty years ago a single person was unable to store or search through a big amount of data, nowadays you can just learn some basic SQL and set up a free PostgreSQL on your home PC.

We don't need to re-invent the wheel any more, and we can actually find solutions very easily.

J. Peterson said...

My favorite example of the power of individual accomplishment is Einstein. In 1905, when he published some of his most significant works (including e=m*c^2 and work leading to his Nobel prize) it was just a "hobby". His day job was at the Swiss patent office.

Vermine said...

@J. Peterson

His hobby was also spying on Poincarré... Just sayin'

Which also further proves that horizontalism and open sources are a great improvement morally wise, no need for this pride and selfish bullshit anymore.
Plus, things get so complicate dand consit in such specialized niches, that accomplishing astonishing things all alone doesn't see very productive, let alone, possible.

It's also tied with God's image but that's another big subject.
The fact the modern world killed the lonely patriarcal figure has IMO a lot to do with the will to build (surrender to?)collaboration, rather than being the lone hero who greedily want his pockets full and only his.

We start to begin that happiness doesn't happen in the individualistic form.

Mike Hord said...

False dichotomy: it's not necessarily 10,000 engineers versus 1,000,000 hobbyists, because a lot of us engineers are hobbyists as well.

My observation has been that the best engineers- the ones who consistently do the best work and the biggest smiles- also tend to do lots of stuff on the side.

Vermine said...

@Mike Hord

False beef.

Doesn't invalidate the point that the "average passionate joe" can way more easily benefit from what the passionate engineers will contribute in the forums etc...

Vermine said...

Also, in order to MAKE and PRODUCE things, it shows there's often less nedd to be an expert.
It might seem unfair but the point is that the good and interesting stuff that the experts will come up with after some years of studies will be directly digested by the average passionate guy.

They just cream the good stuff. They go directly to what works, the cool hacks.

Unknown said...

Does anyone know of a websites designed to support the efforts of DIY'ers? I'm thinking of a site that provides tools for developing ideas, sharing those ideas with other and tools for "publishing" both one's ideas and background. ideally the site would support my development of my ideas and help me connect with other who share similar interests and a desire to collaborate. Any links to such sites would be appreciated.



Vermine said...

@ Aldo

Yeah there are plenty.

Just one example on the top of my mind related to game design:

You submit ideas and people able to develop it ponder if they want to do it or not.

Chris Gammell said...

@Mike Hord

That's the exact reason I try and recruit engineers that do stuff outside of work. Passion is hard to fake.

Serge Illustration said...

Am I the only one who is disturbed by the spelling of curmudgeonly or rather "kurmudgenly" as our author puts it? Wired, c'mon.

Nathan said...

I happen to know, for a fact, that this is happening already in the HTPC community. I'm a member of TeamXBMC and an incredible number of hardware companies have been approaching us about providing software for their platforms, rather than using proprietary in-house software, simply because our software beats theirs hands down.

Sébastien 'Cb' Kuntz said...

That's also happening with Virtual Reality, we're building VR systems for 1000€ or 500€ and having a lot of fun with that creating immersive experiences! (

Vermine said...

Look at that:

Do-it-yourself quantum spooky action...

Bryan Buckley said...

Space, the final frontier!

Seriously. The big companies and government should be working on the big things, like space and nanotechnology, etc.

Unknown said...

I can't help but think of the rise of open source projects reading this. Linux is the example that comes up first in my mind, but others come to mind as examples of the hobbists over the professional. Linux started as a hobby, and now is worked on by professionals and hobbists alike. This example has many parallels in DIY, as people share ideas they have in their hobbies. DIY is frequently open itself: just look at the plans referred to on this blog over the years, and you can find improvements to them as well as a result of them being shared. "Given enough eyeballs, all bugs are shallow" is not just for software.

Unknown said...

This will lead to a new economy of peer to peer exchange free from capitalism!

E.Jacob said...

I agree with Kdansky above.

Although Moores law and the sheer number of hobbyists plays a large role, the effect of the internet is what changed the game.

I have been hacking/making/DIYing for many years and I imagine many people were. Now with the internet we have a community, we are in some sense becoming more like the large company. We share resources and previous work. Now our numbers count for something.

Mark Whybird said...

HEy Johnny, have you read Makers, by Corey Doctrow? Based on this post and what I have seen of your work, I suspect you'll enjoy aspects of it (even if, like me, you sometimes find Doctrow annoying).

TimothyB said...

One additional thing for consideration...

COMPENSATE people for ideas that pan out. Not just in a little way, either. If there is a jackpot at the end of every rainbow that actually reaches the ground, that creates a lot of motivation for people to innovate and create on their own time. Make a real contract with your employees, binding on all concerned, that this will happen and is not discretionary, and give them a substantial piece of the pie once it comes out of the oven.

TimothyB said...
This comment has been removed by the author.