Effective Java #2: Class Builders

I’ve added a new page in the Joshua Block Tribute section of my blog:

Class Builders

Bob-the-Builder-tv-01

“Can we build it. Yes we can!” (image source: starpulse.com)

Class builders help you replace complex and unreadable constructor parameter lists with something more intuitive.  While it may add slightly to the runtime of your application due to more function calls, the tradeoff in readability and maintainability is often worth it.

You could even make the argument that replacing a gazillion constructor parameters for some optionally-used builder function calls improves performance.

Or … perhaps builders just help you not pull your hair out when reading code. I’ll let you decide.

Effective Java #1: Static Factory Methods

I’ve added a new page in the Joshua Block Tribute section of my blog:

Static Factory Methods

I-love-Lucy

This famous “I Love Lucy” episode depicts what you might experience if you make creating objects into a crazy mess. (image credit: Desilu Productions/Paramount, source: cornerstone-ct.com)

Using static factory methods can help keep the creation of objects from turning into an assembly line of hilarity.  That means:

  1. less code needed to use your masterpiece API,
  2. an easier time figuring out how to get your API to create the right objects.

Welding Code Together With Git Branches

That's not me in the photo, but ... I've been there.  Ironworker Mike Kulp, 43, climbs a column before connecting a perimeter beam to the 54th floor. Here Kulp is working approximately 825 feet above street level.

That’s not me in the photo … but I’ve been there. Ironworker Mike Kulp, 43, climbs a column before connecting a perimeter beam to the 54th floor. Here Kulp is working approximately 825 feet above street level. (related blog: newyorknatives.com; image credit: Barbara Johnston)

I was an ironworker for a brief period of time at Local 86.  I loved structural ironworking.  And to this day I still have my tools.  If that work paid what my current job does, I’d be doing that for a living. I miss it.

Oddly enough there were some valuable lessons I learned on that job that could serve as an analogy for Git repository setups.

One of those lessons was that the integrity of the foundation of any project can determine whether or not it ends up being a successful one.   In other words, you get in the end what you set yourself up for – good or bad.

That lesson could apply to your mobile projects this way: If you can identify what the “foundational pieces” are for your project and spend some time shoring those up, the effort can help keep petty repository management issues at bay.  A bad repository setup can halt a project for a few days, so it’s important.

Git has a some great features that are great for doing this when your project relies on multiple repositories.  This blog is about one of them:

Subtree Merging: the ability to merge the code from one repository’s into a subdirectory in another repository’s.

Subtree Merging allows you to approach the construction of your mobile applications (or any software) somewhat like putting a skyscraper together … one section at a time.

Foundational pieces are first fixed into place.  For Git that means the basic repository with defined remotes.

Once that work is done, the rest of the building can be built up around that, making modifications and customizations as you go with an engineering and building process.  But the foundations will still remain the same.  For Git that’s the merging of changes in child repositories into your main one (super repository).

This blog assumes you have just a basic familiarity with git.  Perhaps you’ve cloned a repository or two and made some commits.  Even if you have a lot of experience with Git you might find this blog posting a handy reference.

The Setup

Our goal is to create an Android application.  We’ll start off with no repositories on our local computer and progress from there.  Nothing cloned, checked out, etc.

There will be four repositories that we’re going to work with:

  1. MyApp: A repository that will contain the Eclipse project for our Android application.   
  2. SecretSauce: A repository that will contain your employeer’s proprietary reusable library code.
  3. Twanoh, an open source repository that contains some useful functionality and utility code that will help shorten our coding time.  Located at http://code.google.com/p/twanoh/
  4. SuperRepo: a super-repository that allows you to manage the other three repositories without pulling your hair out. More on that later, below.

I also have a starter project generated by Eclipse with a default generated Android application.  I exported it to a tarball called MyApp.tgz. This will be imported into the MyApp repository once it’s created.

And of course to but this guide into a real-world context, there are some basic rules of engagement for the repositories we work with on this project.

  1. MyApp, SecretSauce, and SuperRepo don’t exist yet.  We have to create them.  These would represent proprietary code repositories that your company might control or have internal access too.
  2. Since Twanoh is an open source library, it will be read only.  None of the code in the other repositories can be inserted into Twanoh.
  3. Importing code into SuperRepo from any repository is O.K.
  4. Copying code between any other repository is not allowed.  It will just create un-necessarily duplicated code.

Caveat:

I’m working on a Mac (OSX Mountain Lion), but all the examples here are on the command line so you will have to adjust your commands to whatever system you’re working on.  I assume since Windows is so awesome, that it will be no problem at all for Windows users to mentally follow along and translate what I’m doing there to suit their own needs.

(the great thing about this being my blog as opposed to a published book: I get to be a smartass)

Where was I? Oh, right … the setup.  I’m going to start at the very beginning of a project setup so you can follow along.  Not that you’re dumb.  I just don’t like examples that gloss over too many details. I think you should be able to retrace my steps from the beginning in case you get lost or need to cross-check your own work.

I.

Repository Setup

14_37_WTC-Site---March-2006

Step 1. Lay a carefully constructed foundation for your project so everything else can fit into place. (image: WTC, March 2006)

Environment Preliminaries

It’s helpful to start with a clean slate.  So, the first thing I’m going to do is adjust my Unix prompt so that it’s easier for you to read (you’re welcome):

$ export PS1="\W $ "
~ $

That simplifies the prompt to show the directory I’m in.  Next, I’m going to create a special directory to do all my work in:

~ $ sudo mkdir -p /usr/local/blog-examples/code-branches
~ $

Then – export a shell variable to make it easy to navigate there.  That will make it clear what directory you are supposed to be in as you follow along.

~ $ export PROJ=/usr/local/blog-examples/code-branches
~ $ cd $PROJ
code-branches $ pwd
/usr/local/blog-examples/code-branches
code-branches $

Use git-init to Create Repositories

Now I’m ready to create the repositories I need.  Curious about the switches I passed to ‘git init’ ? Good. Go read about that.  I’ll wait.

Create a directory where I can store all my Git bare repositories (see Git – Setting Up the Server).  One for each repository.

$ cd $PROJ
code-branches $ pwd
/usr/local/blog-examples/code-branches
code-branches $ mkdir -p bare/MyApp; mkdir -p bare/SecretSauce; mkdir -p bare/SuperRepo
code-branches $ ls bare
MyApp     SecretSauce     SuperRepo
code-branches $

… and now to create the repository for MyApp using git-init:

code-branches $ cd $PROJ/bare/MyApp; git init --bare --shared=group
Initialized empty shared Git repository in /usr/local/blog-examples/code-branches/bare/MyApp/
MyApp $

… for SecretSauce

MyApp $ cd $PROJ/bare/SecretSauce; git init --bare --shared=group
Initialized empty shared Git repository in /usr/local/blog-examples/code-branches/bare/SecretSauce/
SecretSauce $

… for SuperRepo

SecretSauce $ cd $PROJ/bare/SuperRepo; git init --bare --shared=group
Initialized empty shared Git repository in /usr/local/blog-examples/code-branches/bare/SuperRepo/
SuperRepo $

Use git-clone to Clone Repositories

Bare repositories are not used for working copies of your repository.  Bare repositories are where you’ll push all your changes.  To do actual work, we need to clone all the repositories we need into working copies.

Create a directory for those and clone the repositories using git-clone:

SuperRepo $ cd $PROJ; mkdir -p working
code-branches $ cd $PROJ/working; git clone ../bare/Myapp; git clone ../bare/SecretSauce; git clone ../bare/SuperRepo
Cloning into 'Myapp'...
warning: You appear to have cloned an empty repository.
done.
Cloning into 'SecretSauce'...
warning: You appear to have cloned an empty repository.
done.
Cloning into 'SuperRepo'...
warning: You appear to have cloned an empty repository.
done.
working $ ls
Myapp SecretSauce SuperRepo
working $

Notice I didn’t clone the Twanoh project yet.  For our purposes, since we’re not modifying Twanoh, we don’t need to clone it.  But we will pull in branches from that project later.

Import the MyApp Eclipse Project into the MyApp repository

I’ve stored the exported Eclipse project in a file called MyApp.tgz and copied it to the $PROJ subdirectory.  I will now import it into the MyApp repository

working $ cd $PROJ; tar -C $PROJ/working -xzvf MyApp.tgz
x ./MyApp/
x ./MyApp/.classpath
x ./MyApp/.project
x ./MyApp/.settings/
x ./MyApp/AndroidManifest.xml
x ./MyApp/assets/
x ./MyApp/bin/
x ./MyApp/gen/
x ./MyApp/libs/
x ./MyApp/proguard-project.txt
x ./MyApp/project.properties
x ./MyApp/res/
x ./MyApp/src/
x ./MyApp/src_secretsauce/
x ./MyApp/src_twanoh/
x ./MyApp/src_twanoh/README
x ./MyApp/src_secretsauce/README
x ./MyApp/src/schilling/
x ./MyApp/src/schilling/richard/
x ./MyApp/src/schilling/richard/myapp/
x ./MyApp/src/schilling/richard/myapp/MyAppActivity.java
x ./MyApp/res/drawable-hdpi/
x ./MyApp/res/drawable-ldpi/
x ./MyApp/res/drawable-mdpi/
x ./MyApp/res/drawable-xhdpi/
x ./MyApp/res/layout/
x ./MyApp/res/menu/
x ./MyApp/res/values/
x ./MyApp/res/values/strings.xml
x ./MyApp/res/values/styles.xml
x ./MyApp/res/menu/main.xml
x ./MyApp/res/layout/main.xml
x ./MyApp/res/drawable-xhdpi/ic_launcher.png
x ./MyApp/res/drawable-mdpi/ic_launcher.png
x ./MyApp/res/drawable-hdpi/ic_launcher.png
x ./MyApp/libs/android-support-v4.jar
x ./MyApp/gen/schilling/
x ./MyApp/gen/schilling/richard/
x ./MyApp/gen/schilling/richard/myapp/
x ./MyApp/gen/schilling/richard/myapp/BuildConfig.java
x ./MyApp/gen/schilling/richard/myapp/R.java
x ./MyApp/bin/classes/
x ./MyApp/.settings/org.eclipse.jdt.core.prefs
code-branches $

I’ll also add  .gitignore and a README file to MyApp before I do my first checkin

code-branches $ cd $PROJ/working/MyApp
MyApp $ echo "Android project for MyApp" > README
MyApp $ echo ".DS_Store" > .gitignore
MyApp $ echo "bin" >> .gitignore
MyApp $ echo "gen" >> .gitignore
MyApp $

First Checkins

Now that MyApp has some code in it it’s possible to check it into the MyApp repository

MyApp $ git add .; git commit -m"Initial checkin - MyApp README"; git push origin master
[master (root-commit) 66812d1] Initial checkin - MyApp README
 17 files changed, 182 insertions(+)
 create mode 100644 .classpath
 create mode 100644 .gitignore
 create mode 100644 .project
 create mode 100644 .settings/org.eclipse.jdt.core.prefs
 create mode 100644 AndroidManifest.xml
 create mode 100644 README
 create mode 100644 libs/android-support-v4.jar
 create mode 100644 proguard-project.txt
 create mode 100644 project.properties
 create mode 100644 res/drawable-hdpi/ic_launcher.png
 create mode 100644 res/drawable-mdpi/ic_launcher.png
 create mode 100644 res/drawable-xhdpi/ic_launcher.png
 create mode 100644 res/layout/main.xml
 create mode 100644 res/menu/main.xml
 create mode 100644 res/values/strings.xml
 create mode 100644 res/values/styles.xml
 create mode 100644 src/schilling/richard/myapp/MyAppActivity.java
Counting objects: 32, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (21/21), done.
Writing objects: 100% (32/32), 358.44 KiB, done.
Total 32 (delta 0), reused 0 (delta 0)
To /usr/local/blog-examples/code-branches/working/../bare/Myapp
 * [new branch] master -> master
MyApp $

And for SuperRepo

MyApp $ cd $PROJ/working/SuperRepo
SuperRepo $ echo "Super repository for MyApp" > README
SuperRepo $ git add .; git commit -m"Initial checkin - SuperRepo README."; git push origin master
[master (root-commit) bef905b] Initial checkin - SuperRepo README.
 1 file changed, 1 insertion(+)
 create mode 100644 README
Counting objects: 3, done.
Writing objects: 100% (3/3), 265 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To /usr/local/blog-examples/code-branches/working/../bare/SuperRepo
 * [new branch] master -> master
SuperRepo $

And finally SecretSauce

SuperRepo $ cd $PROJ/working/SecretSauce
SecretSauce $ echo "Secret Sauce. CONFIDENTIAL. Do not distribute." > README
SecretSauce $ git add .; git commit -m"Initial checkin - SecretSauce README"; git push origin master
[master (root-commit) 09bc950] Initial checkin - SecretSauce README
 1 file changed, 1 insertion(+)
 create mode 100644 README
Counting objects: 3, done.
Writing objects: 100% (3/3), 284 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To /usr/local/blog-examples/code-branches/working/../bare/SecretSauce
 * [new branch] master -> master
SecretSauce $

Take a Break ….

There is a lot of detail in this blog so far.  Take a break.  Come back in a bit … cause this is where things start to get interesting.

II.

Stack Up The Project

14_37_2010_04_27-WTC-Site-Overview---Credit-Joe-Woolhead

Step 2. With a project foundation built, you can now safely add things on top of it. (image: WTC, May 2010)

Review

So, here’s what we’ve got so far:

  • We set up our shell to make things easier by: creating a directory to work in, setting our command prompt to show the current directory, and export an environment variable $PROJ to help us navigate there easily.
  • Our bare repositories are setup using git-init in $PROJ/bare.  They are $PROJ/bare/SuperRepo, $PROJ/bare/MyApp, and $PROJ/bare/SecretSauce.
  • We’ve cloned those three repositories into the $PROJ/working subdirectory and added files to them.

Link SuperRepo To the Other Repositories

If you haven’t done so, you might want to read up on git remotes and the  git-remote, and git-fetch commands.

It may not be immediately clear to you by reading that documentation that one working clone repository can point to multiple remotes at the same time with no ill side effects.  That is, our working copy of SuperRepo can point to many remote repositories.  You can completely flip between working copies/branches from those remote repositories without ever having code from them collide.  It’s as if you can flip between repositories very quickly using git-checkout.

(I don’t cover git-checkout in detail in this blog, but it will be used).

For example, let’s add a remote reference in SuperRepo to MyApp, SecretSauce and Twanoh:

SecretSauce $ cd $PROJ/working/SuperRepo
SuperRepo $ git remote -v add myapp ../../bare/MyApp
SuperRepo $ git remote -v add secretsauce ../../bare/SecretSauce
SuperRepo $ git remote -v add twanoh https://code.google.com/p/twanoh/
SuperRepo $ git remote -v rename origin superrepo
SuperRepo $ git remote -v
myapp ../../bare/MyApp (fetch)
myapp ../../bare/MyApp (push)
secretsauce ../../bare/SecretSauce (fetch)
secretsauce ../../bare/SecretSauce (push)
superrepo /usr/local/blog-examples/code-branches/working/../bare/SuperRepo (fetch)
superrepo /usr/local/blog-examples/code-branches/working/../bare/SuperRepo (push)
twanoh https://code.google.com/p/twanoh/ (fetch)
twanoh https://code.google.com/p/twanoh/ (push)
SuperRepo $

I also renamed the remote origin to superrepo so it’s easy to be reminded what it points to.

Now we can fetch branches for all the child repositories:

SuperRepo $ git fetch --all
Fetching superrepo
Fetching myapp
warning: no common commits
remote: Counting objects: 32, done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 32 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (32/32), done.
From ../../bare/MyApp
 * [new branch] master -> myapp/master
Fetching secretsauce
warning: no common commits
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../../bare/SecretSauce
 * [new branch] master -> secretsauce/master
Fetching twanoh
warning: no common commits
remote: Counting objects: 539, done.
remote: Finding sources: 100% (539/539), done.
remote: Total 539 (delta 322)
Receiving objects: 100% (539/539), 688.41 KiB, done.
Resolving deltas: 100% (322/322), done.
From https://code.google.com/p/twanoh
 * [new branch] javamail_import -> twanoh/javamail_import
 * [new branch] master -> twanoh/master
SuperRepo $

A quick peek at the branches we now have tell us that remote twanoh has a couple branches.  But the other child repositories myapp and secretsauce don’t.  That makes sense because we haven’t added any code to those two yet.

SuperRepo $ git branch -a
* master
 remotes/myapp/master
 remotes/secretsauce/master
 remotes/superrepo/master
 remotes/twanoh/javamail_import
 remotes/twanoh/master
SuperRepo $

Current Repository State

The image below shows the current state of the repository.  The various remotes are shown, and their history graphs are disconnected. This is correct, because we haven’t done any merges yet.

Screen Shot 2013-04-07 at 9.31.44 PM

The SuperRepo working clone, which shows remotes for SecretSauce, MyApp, and Twanoh. Click to enlarge.

Use git-merge & git-read-tree To Import Code

The next step is to import code into SuperRepo from the child repositories MyApp, SecretSauce, and Twanoh. First myapp, since that’s our main application

SuperRepo $ cd $PROJ/working/SuperRepo
SuperRepo $ git merge -s ours --no-commit myapp/master
Automatic merge went well; stopped before committing as requested
SuperRepo $ git read-tree --prefix=myapp -u myapp/master
SuperRepo $ git commit -m"Import MyApp tree."
[master 9cb4e70] Import MyApp tree.
SuperRepo $

read-tree uses the –prefix switch to tell Git that my app code needs to be imported to a subdirectory within SuperRepo.  It’s automatically created for you.

SuperRepo $ ls
README myapp
SuperRepo $

And now we doe the same with SecretSauce, but we tell read-tree to put the code from that repository into the directory myapp/secretsauce.

SuperRepo $ git merge -s ours --no-commit secretsauce/master
Automatic merge went well; stopped before committing as requested
SuperRepo $ git read-tree --prefix=myapp/secretsauce -u secretsauce/master
SuperRepo $ git commit -m"Import Secret Sauce tree."
[master ce552ae] Import Secret Sauce tree.
SuperRepo $

And finally Twanoh.  As I mentioned above, it’s not necessary to have cloned Twanoh anywhere because we’re not making changes to it.

SuperRepo $ git merge -s ours --no-commit twanoh/master
Automatic merge went well; stopped before committing as requested
SuperRepo $ git read-tree --prefix=myapp/twanoh -u twanoh/master
SuperRepo $ git commit -m"Import twanoh tree."
[master d0761cd] Import twanoh tree.
SuperRepo $

The last step is to push all the changes up to the superrepo repository

SuperRepo $ git push superrepo master
Counting objects: 583, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (192/192), done.
Writing objects: 100% (582/582), 1.02 MiB, done.
Total 582 (delta 325), reused 539 (delta 322)
To /usr/local/blog-examples/code-branches/working/../bare/SuperRepo
 bef905b..d0761cd master -> master
SuperRepo $

Now the Git graphical interface shows we’ve merged all the files into SuperRepo

Git's history show how all child repositories are merged into SuperRepo.

Git’s history show how all child repositories are merged into SuperRepo. Click to enlarge.

III.

Merging Updates

14_61_Edited-for-Web-20121016-IMG_7928

Step 3. With your project infrastructure topped out, customize and modify at will. Incorporating changes at any given point in time is essential until the project is complete. (Image: WTC, November 2012)

The setup we have so far could be considered the “Hello World” setup of a git repository. It contains a Hello World app that we can build on top of, and several child repositories that are brought into the mix – our toolkit so to speak.

The important thing about this setup is now we can use a stable process to incorporate other people’s work without having to re-tool our own project.  Whenever a change happens to MyApp, SecretSauce, or Twanoh, we can easily import the changes.

Make Modifications

Modify MyApp.

SuperRepo $ cd $PROJ/working/MyApp
MyApp $ echo "More README content for MyApp." >> README
MyApp $ git commit -a -m"Added to MyApp README."
[master 7c0e3d5] Added to MyApp README.
 1 file changed, 1 insertion(+)
MyApp $ git push origin master
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 325 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
To /usr/local/blog-examples/code-branches/working/../bare/Myapp
 66812d1..7c0e3d5 master -> master
MyApp $

And SecretSauce.

MyApp $ cd $PROJ/working/SecretSauce
SecretSauce $ echo "More README content for SecretSauce." >> README
SecretSauce $ git commit -a -m"Added to SecretSauce README."
[master a4f9863] Added to SecretSauce README.
 1 file changed, 1 insertion(+)
SecretSauce $ git push origin master
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 335 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To /usr/local/blog-examples/code-branches/working/../bare/SecretSauce
 09bc950..a4f9863 master -> master
SecretSauce $

We’ve simulated some changes, albeit simple, to both the application (MyApp) and a library it depends on (SecretSauce).  Those changes were made and pushed to their respective bare repositories.  SuperRepo doesn’t know about them yet so we have to tell SuperRepo to fetch the changes.

SecretSauce $ cd $PROJ/working/SuperRepo
SuperRepo $ git fetch -v myapp master
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../../bare/MyApp
 * branch master -> FETCH_HEAD
SuperRepo $ git fetch -v secretsauce master
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
From ../../bare/SecretSauce
 * branch master -> FETCH_HEAD
SuperRepo $

And for the final step. We use git-pull to merge the changes we just fetched into the project.  First MyApp.

SuperRepo $ git pull -s subtree --no-edit myapp master
From ../../bare/MyApp
 * branch master -> FETCH_HEAD
Merge made by the 'subtree' strategy.
 myapp/README | 1 +
 1 file changed, 1 insertion(+)
SuperRepo $

And then SecretSauce.

SuperRepo $ git pull -s subtree --no-edit secretsauce master
From ../../bare/SecretSauce
 * branch master -> FETCH_HEAD
Merge made by the 'subtree' strategy.
 myapp/secretsauce/README | 1 +
 1 file changed, 1 insertion(+)
SuperRepo $

Git has automatically kept track of the directory that each remote branch copied into with read-tree.  So, it’s able to properly update your project.

Here’s the final state of your repository.

The Git Gui now shows our changes merged into our projects repository, SuperRepo.

The Git Gui now shows our changes merged into our projects repository, SuperRepo. Click to enlarge.

Project Resourcing the Technical

“I Sing The Body Electric” was one of Bradbury’s better pieces of writing.  Sometimes, when I apply it to my techno-business trade of choice (bringing to market usable mobile products), out of my imagination comes titles like the one on this blog posting.  “I Sing the Body Electric” has a ring to it that is right up there with

rb1

Ray Bradbury. (source: afflictor.com.  See also his official site)

trip the light fantastic

Bradbury’s book, “I Sing The Body Electric,” gets at mysticism and being involved with things that don’t always have concrete explanations.  Trip the light fantastic touches on moving in a nimble way through a musical composition – or, as I like to say it, it connotes something profanely artistic that is born of improvisational genius.

Somehow both concepts are fitting to technical project management.  Grip your trousers tight, folks, because this blog just got a bit more artistic.

It’s common for people to try and deal with technical projects like the building of a mobile application as if they are fixed  things.  Contracts are signed, people are hired, and schedules are set.  Then, often times the expectations for all involved are said to be set in stone and inflexible.  At least, that’s what the engineers are often led to believe.  But then life always intervenes once a project gathers momentum.

A project begins to take seemingly unpredictable twists and turns.  Stress mounts, and suddenly the work can only seem to get done at great personal sacrifice. That sacrifice can mean unbearable amounts of overtime, or even bosses demanding people finish the projects the way it was originally conceived, and “on time and as expected,” at the risk of people losing their jobs.

The Bradbury Connection

As our pre-concieved notions of how a project should be executed give way to the realities of every day life, people seem perplexed.  In this way managing a software development effort can be like Bradbury’s story. It becomes a perplexing journey on an experience that is hard to explain.

How do we deal with that, especially when we know that project after project may carry the same “do or die” theme?   We can either clutch to our rigid project-day-one ideas and hope that someday we find that one project that “is executed perfectly.”  At least you get the luxury of blaming failure on a “bad project.”  Or we can the trip the light fantastic and move through the project in an appropriately improvised way; adapting to ever changing circumstances – new requirements, changes in staff, under bidding, and technical failures beyond our control, just to name few.

So, how do we perform well on our teams in that environment?  Having some concrete tools to improvise with can certainly help. And that’s where this blog posting comes in.  I submit to you that resourcing can set up a project to start off on the right foot, and on a day to day basis help everyone on a team adjust to the unpredictable nature that is inherent in working on projects that involve people with wildly different technical skills.

Resourcing

Resourcing 
[ree-sawrs-ing, -sohrs-ing, -zawrs-ing, -zohrs-ing, ri-sawrs-ing, -sohrs-ing, -zawrs-ing, -zohrs-ing]
Verb.
1. The act of assigning one or more resources to a particular task for a period of time.
2. Causing people, equipment, or to be organized to accomplish a particular task or goal.
3. Removing un-necessary resources from a group, team or resource pool that is assigned to a task.
4. Arranging one or more sources of supply, support, or aid, especially one(s) that can be readily drawn upon when needed; expedient
5. Assigning tactical assets to accomplish an immediate mission.
See also: Resource, wikipedia  

That’s my definition.  If you type “resourcing” into google.com, you’ll actually see a definition that Google provides.  My definition is pretty similar to that one.  But, if you look in the dictionary, there’s a chance you won’t find the term “resourcing”   So, I made one up based on how I relate to the problem of pulling together resources to complete a project.

I think this is also a common term to use in graphic design houses by project coordinators who are trying to assemble in-house people to work on specific client projects.  One multimedia design house I worked at actually had a “resourcing meeting” every week where the project coordinators had to fight over which engineer was working on which project.  It was a fun and entertaining mess to watch – mostly because it was my time that people were often fighting over.

You might enjoy that notion too, right up until the day you realize the project “managers” contributed to some factor that led you to working 70 hour weeks to compensate for some mis-calculation.  Under bidding, an incomplete schedule, inability to find enough people to help, faulty computers, or even insisting people on the team commit to tasks they are not technically prepared to handle.  These are but a few examples, but they are also the inspiration for this blog, because frequently, when it came to engineering, the project coordinators got resourcing wrong.  More on the bad side effects of a botched resourcing job later, below.

The Problem Resourcing Solves

You’ve got a project to do. Your team members may be paid by the hour, salaried, fixed priced, or somewhere in between. Whatever – that’s gets at how the project is expensed, but not how it’s executed on the day to day and has nothing to do with the project’s completion date.  Regardless of what the billing structure of your contract is and how you pay people, you always need people and resources aligned to do the work.

Assigning people to your project and coordinating their availability is a challenge that exists on any project.  How well this is done can determine whether or not a project succeeds.  Resourcing helps solve this problem.

Resourcing As A Tactical Project Strategy

Resourcing can be thought of in a number of ways:

  • “aligning the stars”: putting together all the things that a project needs to be successful.
  • tactical project planning: deciding what resources for a project are in play, when they are to be used, and where, etc.

You can pick any phrase that has to do with picking the right people to get today’s job done.  The job at hand. The one thing you’re going to get your team focused on to make money this week.

If you have work you need to get done this week, just ask yourself the who, what, where, when, (but not why) of the week’s work.  I say don’t ask why, because  I assume you’ve already answered that question.  If you need a reason, here’s a “why”: to make money.

Right – resourcing is also about doing your work with what you have in a profitable way.  I don’t mention this in the definition I have above.  But, it’s important to understand that resourcing is about finding the right mix of people and things to accomplish a project that you can afford to work on. There’s a balance  between capability and profit you can aim to achieve.

However you define profit, or “getting ahead” is up to you.  But, the idea with resourcing as a tactical strategy is to find the people and things you need to execute your project without going broke.  For example,you might choose to delegate everything – that is, resource your team so that you don’t have to do any work at all yourself because you’re busy on higher priority tasks.  And, you might be willing to do it regardless of cost or financial profit.  While not necessarily realistic, that’s still “getting ahead” in one way, and a perfectly valid way of looking at resourcing.

Depending on your role your boss might require you to watch the bottom line, but running a profitable project is another discussion I’ll leave for a different blog entry. For this blog, resourcing is just about getting your people and equipment together.

Some Basic Resourcing Components

So, let’s say you run a consulting firm and you’ve landed a project.  Your sales staff is really proud.  You have all the people, equipment and office space to execute.  Everyone’s motivated and ready to start.

Now what?  You tell everyone to show up and work … do “their magic” right?  No.  You have to give everyone a sense that there’s some organization going on.  Being too hands off can quickly slip you into the mysterious realm of being an irresponsible project manager.  This is where resourcing can help.

When it comes to resourcing, I believe the project size doesn’t matter – it can be a big project or a little one.  In fact, it may be better to try out the recommendations in this blog on a small project first.  In some ways, practicing good resourcing on a small project is more important than on a big project.  If you have a problem that you run into on a small project, a bigger project will just magnify that problem.  And that can be embarrassing.

Component 1: Scheduling

gantt-snapshot1

A gantt chart that identifies specific people, but assumes that nobody has sick time or vacation time. Is that realistic? (source: conferencebasics.com)

Chances are, the client wants to see a schedule, and have you itemize what you’ve brought together to get the job done.  So, a schedule is a critical component of resourcing.

There are a number of ways to represent a schedule: a simple list will do.  But you can also use calendars, gantt charts, dashboards, and so on.  Here are some helpful links:

The thing about a schedule is this: whatever form you put the schedule into, it must be changeable.  There is one guarantee that you have on any project – the schedule will change.  In the military, it’s often said that the first casualty of a war is the plan. So, why would we expect our civilian project plans to be somehow magically concrete? I think it’s dangerous to think that way.  A project will change right under your feet and you might not even notice it … unless you know what you’re looking for.

One of the key things to look for is the time your team puts into updating the project schedule: too much time spent on scheduling can cause chaos, but not enough time can lead to a disorganized mess.  So, getting this balance right can certainly help.  Here are the basic things involved in scheduling that will probably be useful to you.  You may have other things, but this could work at a minimum:

  1. Who is working on the project and when?
  2. Have you factored in vacation and sick time?
  3. Have you given the customer a reasonable expectation of timelines?

The only other thing I’ll say about scheduling is that if it doesn’t change on any given day, and all the project updates you see indicate “no changes” then that might be a sign that someone is not paying attention to the factors that can affect the schedule.  I’m constantly amazed at how often I’ve seen Technical Project Managers do this, despite the emphasis placed on knowing when a project is scheduled to be completed.

It’s just plain hard to see what adjustments to the project need to be made to keep it on time if you don’t spend enough time keeping the schedule updated.  Daily updates and adjustments are a good place to start.

Some Gantt Chart Alternatives

The simplest form of a project schedule is just some text on a note pad.  Here’s one I created for a fictional project where musicians and Siddhārtha combine forces to create a mobile app:

20130401-blog

Sometimes simple, handwritten Charts are the best. They need to be in a form that can be updated daily throughout a project as conditions change.

This took me only about 5 minutes to make up.  I made sure I added some markings to indicate who has sick leave available, as well as floating holidays.  All of which these people have worked hard for and deserve. In the right hand margin I indicate there is a potential to slide the project deadline by 16 total days: 12 sick days and 4 floating holidays.  Naturally, it would be best in this case to set the client expectation to receive the final app on the 1st of August.

However you get the information onto paper is irrelevant.  The key is to indicate indicate who is available, their specialty/role on the project and when they will be available to do the work.    And most importantly, to be able to visualize what would happen to the project if any combination of people become unavailable for a period of time.

If you’re into fancy graphics, then these might appeal to you as well:

PERT-Chart1

Pert Chart (source: savethemoneyinthemiddle.com). See also cmu.edu.

C16A1F1AE8AEA19217AE13B2F20FA1C6636B39B6_lis

Pareto Chart (source: brighthubpm.com)

Screen Shot 2013-04-01 at 5.28.43 PM

Work Breakdown Structure (source: brighthubpm.com)

Screen Shot 2013-04-01 at 5.32.31 PM

Cause and Effect Charts, also known as fishbone charts. (source: brighthubpm.com)

Cause and Effect charts are some of my favorite.  They’re also called fishbone charts.  You can put any kind of data on them that identifies factors which impact a goal.  They’re often used to identify issues related to problems, but you can easily see how they can be used to identify factors of successful or completed project.

Component 2: Computing Power

What computer equipment is available to use? Do you have to set up a VPN or other infrastructure to meet client needs?  Is your build system already overloaded?  The list goes on.  But I think you get the point.  The idea here is to constantly be aware of whether or not the computing equipment you have is overloaded.  Here are some factors you might add to the mix:

  1. How many computers are available?
  2. Do you already have a requisition request for a larger build system?  Your engineers might be telling you you’re on the verge of having an infrastructure melt down if there are a few of these.
  3. How long does it take to roll out all your builds?  Particularly important if you have a monolithic build system that is used to roll all of your company’s builds.  It’s good to have one machine to roll all your builds, but is it overloaded or not?  On an Android project, for example, it’s not uncommon to have a build system that contains terabytes of code which can include the entire Android source code base.
  4. Are your engineers fighting with system setups and slow internet connections?

One way to tell if you’re under powered in the hardware department (no snickering, ladies) is to take note of how much time your engineers in particular deal with system setup issues.  A basic acid test can be this: take metrics about your environment such as processing power, disk space, and RAM available on your machines.  Run some metrics to see how taxed that equipment is.

In other words, size matters when it comes to getting things done.  Alright – fine, snicker away.

Component 3: Identify your Mavens

It could be said that your team is right for any project they are technically prepared to execute.  And everyone likes to think they’ve hired perfect engineers – rockstars if you will.  But, are you aware of who is an expert at implementing applications with specific technologies?

Can you identify all the technologies that your team has built into applications before?   You may have a group that is well versed in writing apps for a particular platform (e.g. Android or iPhone devices), but have you identified all the features of those platforms that will be involved in you your final deliverable? How many people on your crew have actually delivered an application using those features?

For example, if you’re creating an app that uses GPS and Near Field Communciations, do you actually have people on your team who have built those features into any application? Pick any feature of a mobile device – the question I am asking still applies.

I’m not suggesting that failure to have implemented something like GPS into an mobile app is some sort of deficiency.  It just means your team mates haven’t gotten around to it.  So, what I’m saying is that it’s healthy to understand what specific technologies (esp. mobile features) your team mates have mastered in the past, and give them a safe place and the time to master things they haven’t yet.

Here’s a simple list of examples for Android 4.0, which it stands to reason nobody on your team has yet mastered:

  • HTC’s 3D API.
  • vsync timing, triple buffering, reduced touch latency, and CPU input boost
  •  hardware-accelerated 2D renderer
  • Renderscript Compute
  • Quick Settings
  • Application Tray
  • Dedicated  User Spaces
  • Daydream – yes, that’s an actual feature
  • Lock Screen Widgets
  • Native RTL support.
  • etc …

That’s a very basic list of enhancements to Android 4.0. I’ve just scratched the surface.  If your team is going to master those items, then it just stands to reason that they need time to actually write applications that utilize them.  Ideally, they would do that in a non-customer facing laboratory before customers ask for those features to be put into their apps.

And most importantly, do other people in your company or working group know where those strengths in your team lie?  One way to keep tabs on that is to itemize, feature by feature, things each member of your team has built into an application.

This is where an in office research lab comes in handy.  In terms of money, the effort to establish this can be limited to the commitment of individuals on your team. Especially if you already have a development group set up.

For example, I’ve always had a job that requires me to spend 20% of my time (in some cases more), learning about the state of the art of my field and applying it to real world problems.  And I’ve used that time diligently to try new things out, like Near Field Communications.  The beauty is, it’s not necessary to wait around until a client demands that you build an application with those things.

You can start a side-project with a somewhat real-world application that is designed to build some experience with “the new thing.”  The idea here is to align your team with projects that utilize technologies they’ve already mastered as engineers.  But it’s also necessary to give them all time and a safe environment expand the list of “mastered technologies”.

My suggestion is, quite simply, to create a laboratory type environment that lets you make mistakes at a time where there is no time pressure.  If you do that, for each technology you will ruled out all the things that don’t work when baking it into a product.  The only thing left will be the “right way” or best practices, and your rate of coding production during billable projects will increase dramatically.

This is also why it’s useful to make conference attendance (e.g. Google I/O) a mandatory event every year.

Component 4: Have a Gameplan To Deal With Busses

I call this component the “hit by the buss” component.  I encourage you to use your imagination to get in touch with what would happen if one of your co-workers got hit by a bus, or gets ill for a long period of time. Go ahead … cry.  You’ll miss that person.  As an aside, I’ll suggest that if you aren’t moved by that prospect on a human level chances are you despise or have objectified your team mates – the same people you also happen to spend a significant part of your life with.  I’ll let you talk that over with your shrink on your own time.

Where was I?

The obvious human tragedy is a big enough thing to deal with, but everyone will still be counting on your mission being completed.  Again, I turn to how the military has found a way to deal with these kinds of things.  On any particular mission, a certain amount of people can end up missing or (through no fault of their own) not able to help the team complete its mission.  And yet missions still get completed. Seemingly on time and in the most unpredictable of circumstances. There’s a lesson or three there that we can learn when it comes to tactical project planning.

One of the most effective ways to deal with this eventuality is to require your team to spend time time sharing knowledge.  And here’s an acid test to determine whether or not that’s been done properly.  On any given day, can employee “B” sit down in the chair and do the work that employee “A” has already started?

I like to ensure that this can happen by using six simple work lifestyle choices.  This may work for you as well:

  1. Code reviews on every code check in.  This involves a developer calling upon one of the people on his/her team to look over their shoulder while they explain in details what decisions they made and how those decisions were implemented.
  2. Documentation standards.  Don’t roll your eyes at this one yet. But it does work.  Technical specs, in-code documentation and comments are all “incomplete works.” They don’t create a running applicaton so they don’t have to be detailed. But are they all detailed enough to let everyone on a project understand what is being accomplished and where in the code it’s being accomplished? I’ll write more about this on a different blog entry.
  3. Daily engineering meetings.  People hate these, but as long as they are short and to the point and relate to the work that is going on, they’re never a waste of time.
  4. Collaborative working lifestyles. Do your employees talk shop a lot? It’s good to allow them time to do that on a daily basis. They don’t have to like each other, but they do have to respect each other enough to sit in the same room and talk through ideas.
  5. No hidden technicals.  If you have a team mate that always glosses over hidden details about their code or is too afraid of exposing their work that might be a sign that you’re not prepared to recover from a lost employee.
  6. Require people to be articulate.  It’s true that developers write software so that it can be read by another program (e.g. a compiler) and turned into an application.  But it could be said that that’s not the audience engineers write software for.  I submit that engineers write software for an audience and that audience is other engineers.  Does the in-line (code) documentation explain in correct and plain English (or whatever language you communicate in)?  Does the logic in the code reflect that?  When the logic of the code, the in-line documentation, and the technical spec are read by a human is it all consistent?  Basic communication skills here are key.

Those six items of the “hit by a bus” component may not seem like a resourcing problem, per-se, but the degree to which they  are used can definitely affect your options for putting people on your projects. So, while you may not be the person to solve those problems, it’s wise for you to at least make a mental checklist as to the state of things so you can make recommendations to your team mates.

Don’t Take My Word For It

I’ve tried to steer away from theoretical project management approaches and stick with some practical tools to help you stay on top of the work being done around you, regardless of your role.

So, let’s say what I’ve written here doesn’t appeal to you at all.  You get an initial schedule approved by your boss/client/whomever and then stick in your desk drawer, hoping that the project will “just work out.”  One thing you might consider, and this is what I mentioned about projects gone afoul above is this:

Clients will expect that any underestimation by the project manager will have to be made up in one way or another.  It’s usually through relentless and un-necessary overtime.  Especially when client expectations begin to fall out of alignment with the reality of your project’s work pace.

For example, the work of employees who can’t show up to do their job, regardless of the reason, will ultimately fall onto the shoulders of their team mates.  Or, at best, employees returning from a well earned vacation will be punished with unreasonable overtime. And for salaried employees that means working for free.

Burn out is right around the corner.  Talk about a morale buster.

Go ahead, just test that theory …

In Conclusion

In this blog posting I went over the basic concept of technical project resourcing.  And I discussed some things  that I’ve found work well when it comes to pulling together a working team to accomplish a task.  Your mileage will vary, but hopefully it will give you some constructive things to think about.

In short – it’s all about the setup. The moral of the story is that the more you have the right resourcing components in place, the better chance of success you’ll have and best of all, the least amount of variability you might see between planned completion dates with actual completion dates.

Notice that I didn’t discuss quality of the work that each individual contributor produces, rates of work, etc.  Those, too, are topics for a different blog.