Pre-Release Announcement: Pend Oreille

gema_01_img0091

Pend Oreille, WA was named after a French Canadian word. source: everyculture.com

I’m happy at long last to announce the pre-release version of a new Android library I wrote:

http://code.google.com/p/pend-oreille/

The release candidate is on the branch called “release-candidate“.

Being a release candidate means the code is far enough along to give it a preliminary try, but not yet ready to be used in production applications. It has passed some basic development smoke testing.  Being pre-release also means that the code and API are subject to change until it’s merged into the master branch for the official release.  You can also expect some things like javadocs and a guide to be missing. In other words – your mileage will vary.

The project is called Pend Oreille (pronounced pond-o-ray), in honor of a little area of Washington State by the same name.

The Pend Oreille library is very simple and small (under 20kb in size) and has a few basic features that I have always wanted to have handy, without always having to use DataOutputStream objects:

  • serialize Java primitives and primitive arrays to byte arrays.
  • deserialize java primitives and primitive arrays from byte arrays.
  • box and unbox primitive arrays
  • serialize an array of boolean values into a packed bit field.

Yep. That’s it.

EMail is Obsolete

200

source: giphy.com

I recently attended the WordPress Meetup Group in Seattle.  At that particular meeting Scott Berkun was giving a talk about his book, “The Year Without Pants,” which chronicles his roughly 18 month stint at WordPress.  Somewhere in his presentation he said something interesting:

WordPress employees don’t don’t use e-mail internally.

He pointed out that the only e-mail he utilized while at WordPress was to communicate to people outside the company. Everyone else used a combination of Instant Messaging, and phones routine work.

Interesting …

very interesting

That got me thinking.  Do we (and by that I mean me) really need e-mail at all? If I had to replace e-mail today what would I do? I mean, after all my e-mail archive can, when necessary, serve as a legal archive for contracts, promises, and agreements.

The short answer: I’m certain I can live without e-mail.

Here’s how:

  • docusign.com provides a document archive.
  • dropbox.com allows me to exchange documents with anyone who I really need it.
  • instant messaging for communications.  They can be archived just like e-mail can.

What do you think?

Messages don’t need to be more than 140 characters long anyway- cell phones and IM systems are now integrated enough.  So, here’s my farewell letter to e-mail:

September 25, 2013

Dear E-Mail,

I have been with you since before the dawn of the Interweb.  You first let me download NASA pictures in times before the first web browser was created.  I mean it was great – I could send an e-mail to NASA requesting a directory of files to download, and then send another e-mail to request a specific picture. When the picture came back I was able to save it on my computer and view it. That was cool! You gave me this first experience of it’s kind – and so willingly! I’ll never forget it.

Things have come a long way since then.  E-Mail has been the backbone of my communication capability for the past 20 years.  You have even helped me resolve some pretty huge debates.  But what I’ve come to realize is that my affair with you has come at a cost. A very high cost.

Every telemarketer in the world seems to think they can abuse you to send irrelevant e-mails and worst of all spam. I have to constantly log into you and sift through hundreds of messages (many not-so-short) just to get what I need. Perhaps for you it’s like some kind of exotic foreplay. But often lately it makes me feel like a dirty guy.

I’ve had a wakening of sorts, and have realized that it’s best that we start to part ways.  I will still need to keep you around until other people in the world catch up.  Just know that for the most part my life has changed and I’m going to be putting some distance between us.  Eventually we’ll grow so far apart we just won’t be an item any more.

Please don’t feel bad.  I wish you the best of luck and you will always know that for some you have an important purpose on this planet.  One which I am confident you will continue to find.

It’s not about you.  And it’s not because of you.  It’s not you.

It’s me.

(alright, fine – it really IS you)

Dial ‘M’ for Millisecond

Grace Kelly starred in "Dial M For Murder," a 1954 Hitchcock film.  (source: imdb.com)

Grace Kelly starred in “Dial M For Murder,” a 1954 Hitchcock film. (source: imdb.com)

There are times when, if you look close enough, milliseconds matter.  Functions in your mobile app that don’t run as fast as they could will incrementally steal processor cycles … harass the garbage collector … like some shadowy process stalker on the other end of the phone.

Given a specific algorithm you often have a choice.  One choice will keep runtime overhead in check as much as possible.  Another may add incremental amounts of runtime and memory usage with each use.  Next thing you know  …

You’ve dialed ‘M’ for murderous milliseconds.  Just don’t be surprised when the monster comes calling.

The problem is – how do you know that’s what you’ve done in the first place? How do you get to the processor cycle killing beast before it shows up during the QA cycle … laughing at you as it adds three, four, five … no six … seconds to the time it takes your app do to something like de-serialize binary data from a network connection.

One place that can help is to look at the functions in your app that get run repeatedly throughout it’s life.  Functions that perform data marshaling of Java primitives to and from byte arrays are pretty obvious.  I recently had some time to take a deep dive into a couple ways of doing this on Android, and I thought I would share some of the hard data.

So, here’s the problem I was looking at: reading and writing  Java primitives to/from byte arrays.  These kinds of functions are relied upon a LOT in many mobile applications making them good candidates for optimization.  Shaving even a few hundred milliseconds off a function that, say saves Double values to a byte array, could mean that the user is waiting several seconds less for the app to complete a task.  In other words, the faster you can get your primitives into and out of a byte array the more responsive your application is going to be.

So, I wrote a bit of code to compare two different ways of saving double values into a byte array.  In one version I used DataOutputStream:

    public static byte[] toByteArray(Double[] array) {
        ByteArrayOutputStream os = new ByteArrayOutputStream(array.length * 8);
        DataOutputStream dos = new DataOutputStream(os);

        try {
            for (int i = 0; i < array.length; i++) {
                if (array[i] == null)
                    dos.writeDouble(0);
                else
                    dos.writeDouble(array[i]);
            }
            dos.close();
        } catch (IOException e) {
            throw new PersistException(e);
        }

        return os.toByteArray();
    }

In another version I relied on straight byte manipulation using bitwise operators:

 
    public static final byte[] toBytes(Double[] array) {

        byte[] result = new byte[array.length * 8];
        for (int i = 0; i < array.length; i++) {
            if (array[i] != null) {
                toBytes(array[i], result, i * 8);
            }
        }

        return result;
    }

    public static final void toBytes(double value, byte[] dest, int start) {

        toBytes(Double.doubleToLongBits(value), dest, start);

    }

    public static final void toBytes(long value, byte[] dest, int start) {

        dest[start] = (byte) (value >>> 56);
        dest[start + 1] = (byte) (value >>> 48);
        dest[start + 2] = (byte) (value >>> 40);
        dest[start + 3] = (byte) (value >>> 32);
        dest[start + 4] = (byte) (value >>> 24);
        dest[start + 5] = (byte) (value >>> 16);
        dest[start + 6] = (byte) (value >>> 8);
        dest[start + 7] = (byte) value;

    }

The interesting thing about these two examples is this: If you look at the code for DataOutputStream you will notice that at its core, DataOutputStream essentially does the same thing as the function I wrote by hand.  You will also notice as well, that using DataOutputStream will potentially require more memory overall because the classes for DataOutputStream and ByteArrayOutputStream both need to be loaded into memory and they obviously have internal state that needs to be stored.

So, I calculated the time it took for each function to run:

 
        Double[] doubles = new Double[1000];
        for (int i = 0; i < 1000; i++) {
            doubles[i] = (double) i;
        }
        Debug.startMethodTracing("trace");

        long start = System.currentTimeMillis();
        byte[] resultBytes1 = toByteArray(doubles);
        long elapsed = System.currentTimeMillis() - start;
        Log.i("TRACE", "elapsed using buffers : " + elapsed + " ms to generate "
                + resultBytes1.length
                + " bytes");

        start = System.currentTimeMillis();
        byte[] resultBytes2 = toBytes(doubles);
        elapsed = System.currentTimeMillis() - start;
        Log.i("TRACE", "elapsed using my code : " + elapsed + " ms to generate "
                + resultBytes2.length + " bytes");

        Debug.stopMethodTracing();

I ran this several times to make sure the Dalvik optimizer had a chance to do its job:

 
09-19 11:02:07.308: I/TRACE(11146): elapsed using buffers : 328 ms to generate 8000 bytes
09-19 11:02:07.498: I/TRACE(11146): elapsed using my code : 189 ms to generate 8000 bytes
09-19 11:02:34.038: I/TRACE(11196): elapsed using buffers : 310 ms to generate 8000 bytes
09-19 11:02:34.238: I/TRACE(11196): elapsed using my code : 196 ms to generate 8000 bytes
09-19 11:02:59.778: I/TRACE(11226): elapsed using buffers : 289 ms to generate 8000 bytes
09-19 11:02:59.948: I/TRACE(11226): elapsed using my code : 166 ms to generate 8000 bytes

Conclusion

It could be said that if you take the lazy way out and stick with DataOutputStream, you’re making a choice to build a user experience delay right into your application.  A delay that is visible to the user.

Using DataOutputStream to serialize 1,000 doubles ran consistently about 120 milliseconds slower than using shift operators directly. 120 milliseconds isn’t a lot of time until you consider the compounding effect those 120 milliseconds will have.  1,000 doubles isn’t a lot of data – only 8,000 bytes to be exact.  You will only have to serialize the array of Doubles using DataOutputStream about three times (e.g.  24,000 bytes of data) before that 120 milliseconds becomes noticeable.

Introducing delays into your application that serve no purpose is generally a bad idea.

Take into account your application will be serializing/de-serializing megabytes – perhaps even gigabytes of data while it runs, and the additional runtime overhead DataOutputStream requires becomes that lurking monster in your app that will, pardon the phrase, byte you in the end.

I haven’t taken into account the increased in-memory requirement that DataOutputStream will require as well, nor the additional work you’re setting up the garbage collector to do.  In short, the number of objects instantiated and destroyed by DataOutputStream undoubtedly adds more stress on the garbage collector.  That, in turn, could actually begin to slow down your application as well compared to using the other method.