After taking a few years off of writing code, I am pleased to see how much the Internet culture has changed software development for the better. (At least for my better :). Software development is getting really cool again and empowering small teams to make slick products.

As one example of how things have changed, let's look at my experience from engineering school back in the day. (The day before the WWW or browsers.)

I have always had a love for video games and computer graphics so I took a few classes on the subject in college. Our textbook was thicker than a phonebook and our semester project was to build the most basic drawing program you ever saw. We worked alone and wrote in C and X-Windows.

The skills we learned focused on the algorithms for drawing lines, simple curves and basic shapes like a circle. I took this class twice and one time we did the 2D drawing program. The second time we did a simple wire-frame 3D viewer was a final project (think about Battlezone).

Contrast my experience with someone taking a graphics class today. Thanks to OOP and common APIs like OpenGL (probably written by programmers that took classes with me :), today's graphics students dont write simple algorithms; they write entire games for their class projects.

The lessons here?

Students today get to do code that is a lot more fun because they get a lot more done with each line of code. Compare my 3D wireframe viewer to the full fledged games by these students.

The reason for this is largely based on the power of object-oriented programming and common APIs - both of which are the result of building on the work of programmers before them. Now software development has always built on the work of others before them (I certainly did not write in assembler nor did I have to write a windowing system like X11) but the industry today has matured to a point where we are really seeing results.

a new era - elements of the change

There have been paradigm shifts in software development before so this is nothing new but changes over the past 10 years have lead to a new plateau today. Web services is a specific term, but for lack of a beter one, I think of these factors collectively as "web services" including:

• OOP & design patterns
• Iterative development models like Agile, Scrum and XP
• The web services model of modular interchangeable blocks
• LAMP
• Open-source software
• Advertising money and Google
• Software as a Service model

Many of these ideas, like design patterns, have actually been around for decades but today they are all working together to hit a tipping point of critical mass. Taken together these ideas are changing the way software is built and I am very excited about it.

OOP & design patterns

Object oriented programming didnt take off until the 1990's but as soon as I learned about it, I immediately knew I loved it. Our brains deconstruct the physical world into objects or patterns anyway so writing software this way was just a totally natural thing to me, far more intuitive than the procedural programming I was trained with.

But more than being intuitive, object orientated programming (OOP) really encourages encapsulation or the idea of a black box and that idea and practice has a huge effect. With an object, I dont need to know how it works inside, I just need to know what it does and how I can use it. This allows a division of labor among developers but it also allows reuse of code and rewriting (or refactoring) old code for new purposes. It is hard to overstate the importance of OOP.

Related to OOP is the idea of design patterns. These patterns are structures that are repeated in many programs or situations. For instance, all cars have engines even if there are many kinds of engines. Combined with OOP, one can write a toolbox of pattern-based objects and then re-use them for future programs. This saves a ton of time and really allows us to re-use code over and over.

Iterative development models like Agile, Scrum and XP

When I was trained in software, we used the waterfall model. The idea of the waterfall is that each work activity (planning, requirements, testing, development) is broken into its own project pieces and scheduled. As the project progresses from one step to another, there is a handoff from one functional group to another, say marketing to requirements to development, etc.

Most traditional projects use the waterfall model and this method makes a lot of sense for physical items, like building a house. First you start with a lot of work defining the requirements, then you estimate the tasks needed, the time it will take, the resources and the cost and create a schedule. Then you build the product.

While conceptually simple, the waterfall model is problematic for software. Much of the stress, overhead and shear waste of software development, ie all the stuff that is not fun, stems from the waterfall model. There are several reasons for this but the main problem is that software is much more iterative than other products and it needs a more flexible process.

One of the major problems with software development in general is that people (just about everyone involved) are really bad at visualizing programs. People are excellent at reacting to a program that you show them but until you show them something to react to they just cannot tell you what they want.

Unfortunately the waterfall model spends a lot of time on upfront overhead and as result you dont get a product out that customers can react to until the end of the process. Given the difficulties of visualizing and planning, the software that eventually comes out is often a poor match with what customers actually wanted but at that point there is little time, money or resources to make big changes.

A lot of products are shipped in an unsatisfying state or they are delayed. It is just very hard for the waterfall process to hit a fast moving target like customer demands. Developers are often asked to endure Herculean "crunch" times. Managers have to resort to CYA-politics and everyone feels a lot of the stress. This is the kind of stuff that call "unfun" software development and at the end of the day, there is just a lot of wasted time and a lot of failed projects.

The good (if not bizarre) news is that software is not a physical product so it does not have to follow the same rules as products from the physical world. Software is a part of the invisible world of the mind and that allows us to do things that could never be done with physical goods. Things like iterative development.

If you built a house, you really want clear plans up front and an idea of the whole system so that you can be sure the foundation can support the roof. Software however allows something amazing - you can actually build the house room by room. You can even build the roof first and the basement last! Such is the idea (and promise) behind the iterative development models.

This idea takes some getting used to but it really is awesome for programmers because it works well with smaller teams, addresses communications issues throughout the process, and most of all, it allows customers to see the product sooner. Instead of showing the whole house at the end, you get to show the customer each room at a time. If they dont like it, you rebuild the room until they do but you dont have to rebuild the whole house.

Iterative development models are still catching on but they have at least penetrated the buzzword world of many companies and will only get stronger as more people learn about them.

The web services model of modular interchangeable blocks

If you take the idea of small teams, iterative process, OOP, and combine it with the idea that the "network is the computer", you get this idea of web services. Instead of providing a huge monolithic program like MS Word, you build small, indivisble building blocks of features and then combine them into usable applications.

Again, this is an idea that works in the software world even if it makes no sense in the physical world. But imaging building a car with web services. Instead of buying the car from a car company, you build your own car by assembling pieces from many car specialists. One company would provide the engine, another the body, another the wheels, etc. The Internet allows all those pieces to work together and you get a working car at the end.

Web services is a totally cool idea that is just starting to get going. It also reflects a totally new mindset in software where you share instead of hoard. Google is an excellent example of this but Amazon and others are on the same bandwagon. Look at all the companies include google search, google ads and google maps in their own products - for free. You dont pay Google for the code they wrote.

Web services is not a new concept in software but rather another application of encapsulation but the results are awesome. One nerd can assemble blocks from a host of other groups and put together a major product all alone or in a small team. This will empower more visionaries and result in more great products for us to use.

The idea also works within companies themselves. Just look at Amazon - to use users it is a single website but Amazon has around 2,000 developers divided into 200 project teams all working simultaneously and largely independently on providing us cool services. The idea of trying to managing that many projects at once gives me heartburn but it is totally doable with the web services model.

LAMP

LAMP stands for Linux-Apache-MySQL-PHP but what it means is a standard structure and set of tools for building web applications. Not only are languages like PHP and Javascript easy to use, they are extremely powerful. This has opened to door to many new programmers who build their own tools faster and product code for others to share.

Sharing is key because a second element of LAMP is that it is free. You dont need to purchase a $1,000 development IDE from Microsoft before you can begin. If you have an idea and some programming skills, you can download and use all the LAMP tools for free and see where your idea can take you.

While the LAMP platform represents less development license revenue to companies like Microsoft, it is enabling a huge amount of creativity and a sense of community. By blowing away the cost barrier to entry, amateur and experienced programmers alike are able to get into the software development game.

Open-source software and Sourceforge

Much like the sharing-mentality of web services, open-source software has created a huge community of volunteer programmers. The theory is that products get better with more eyes looking at the code and that seems to be the case. OSS is an amazing adventure in massively distributed projects that produce awesome software which is often free.

There are a lot of philosophical issues debated about OSS but my takeaway is that it has really spurred innovation. Pick just about any topic you want and you can probably find someone working on code like that who would love your help or feedback. I continue to be amazed at the amount and quality of free software that is out there and how much time it saves others.

Advertising money and Google

Another huge factor of the new software model is advertising. Suddenly everyone is spending more time on YouTube than TV and the Old World has taken notice of the Internet. Money is to be made and Google is showing everyone that it is BIG money.

Google is interesting because it is a company that set out to solve one technical problem but ended up funding that work by making money in totally different way. In fact, most people I talk to really dont understand how Google makes money; to them, Google makes search.

But advertising is a totally different business model from traditional software, like Microsoft, which again has allowed companies to thrive and spurred innovation. While Google does search, just look at all the other nifty software pieces they have built and shared and then look at all the cool things their competitors have done to keep up. The end result is good for consumers.

Software as a Service model

For a while now, SAAS has been one of the most exciting trends for me personally because it lets the user focus on the tasks they want to do and puts the annyoing details on the nerds who run the servers.

You want to write a document or work on accounting or draw a picture. With server-based products, you just log into the program and work; the people that run the server take care of the software and versions and updates and most of all, backing up your data. You focus on what you are good at and let the provider do the same.

In contrast, personal computers require you the person to do everything including installation, maintenance, backups, and recovery. Not only do you have to master the program but you have to master the system that runs the program. *ugh* And if you dont, you either have trouble or you have to pay someone else to do it for you. This is a big expense for companies and a big roadblock for normal users.

Even though people like to think they want the power of a personal computer, very few people really want to worry about virus' or patches or any of the millions of details it takes to maintain modern software. Better to lease the software than put up with the headaches of ownership.

SAAS is small today but it is growing and most of all, the value-proposition is in line with what users want.

Closing

Well that was a bit longer than I intended but I think you get the point. A number of technologies and ideas today are working synergistically to create some awesome effect. After being a bit burned out on traditional software, I have to say that I am totally psyched about the direction Internet software is taking us.

The whole industry is becoming more module-oriented, more system-oriented. The development process is more natural, more fun and most of all empowering. Individuals and small teams with an idea have more power than ever before to produce great things. It is totally exciting.