Reinventing the Wheel

I ran into this old post by Jeff Atwood, that makes a good point. Reinventing the wheel is most probably the best way to learn about “wheels”, and about the problems they designed to solve.
There is no better way to learn about anything than actually implementing it.

However, consider a project that, say, 3 engineers are working on, and each one of them reinvents a different “wheel”?
Here’s what will happen:

  • Engineer #1 will write a HTTP server
  • Engineer #2 will write a RoR like backend MVC framework
  • Engineer #3 will write a jQuery like dom abstraction JS library

You see where this is getting…
There are at least two main problems here:

  1. This project will probably be buggy, will take forever to develop, and will be impossible to maintain.
  2. The next engineer coming after you, is more likely to have experience with jQuery than in your home brewed fooQuery.js and will probably have more desire to gain expertise with jQuery over your amazing home brewed library.

Basically the more code we own, the more liability we have to maintain, test, debug, fix, improve, write documentation etc.
Given the limited resources we all have, the last thing we need is to own more code.

This is especially true when all the above problems are already solved with efficient, fully tested, fully documented open source projects we can use. for free. with communities and all the other good stuff that comes with open source projects.

This is exactly the reason why engineers should own or participate in pet projects.
A pet project is the perfect place where its not only okay to reinvent the wheel – it is desired — this is exactly the room to do crazy things, experiment, learn, and develop your professional skills.

Posted in Javascript | Leave a comment

Software Engineering Interview Questions

Edit, 6/26/2012:
This post is my personal rant about interview questions. If you got here looking for actual programming interview questions, I can recommend these books: Cracking the Coding Interview and Programming Interviews Exposed. These books helped many engineers getting their dream jobs.

It’s no secret I’ve been in a lot of interviews in the past couple of years, both as an interviewer and as a candidate. When it comes to tech interview questions, to simplify things, I can break it into 3 major types:

  1. Implementation questions
  2. Architecture/design questions
  3. “Problem solving” questions

The first type intends to check your experience with current industry standard tools. For instance, as a javascript engineer you will be asked about specific jQuery method, what problem it designed to solve, and how you can effectively use it, how a web browser works, or what is the HTTP protocol.

The second type intends to check your software design skills, and your ability to design a complex application. As a front end engineer, you will probably asked about MVC concepts, RESTful architecture, or design patterns.

The third type intends to check how well you handle with random problems you didn’t expect. However, in many cases, this part is a tricky, and mostly out of scope.
It’s not that I don’t think a good engineer should have strong “problem solving” skills. I do.
I just think most of these “problems” you are being asked about has nothing to do with what most of us actually do on a daily basis.

In my past interviews I was asked algorithmic questions, mathematical/geometry questions and physics questions. Hell, I’m surprised I wasn’t asked to solve rocket science problems. After all, my “problem solving skills” are being measured.
Continue reading »

Posted in Javascript | 5 Comments

Backbone.js Poller

Lets say you have a Backbone.js based messaging system on your application, and you want to stay synced with the server so you could alert your users when a new message arrives. You would like to call the server once every couple of seconds and ask for status updates.
Or maybe your model represents an on going background task, you want to poll for status updates.
You could start with something basic such as:

function poll(collection, delay) {
  setInterval(function(){
    collection.fetch({data: {foo: 'bar'}});
  }, delay);
}

This works, but has some flaws:

  1. You may end up running simultaneous requests hitting the browser’s allowed connection per host limit (typically 6, 2 on IE7, according to browserscope)
  2. Nothing prevents you from running the function multiple time with the same collection
  3. You can’t easily stop the process, and abort running requests
  4. It doesn’t handle cases where we want to stop polling a model on a condition (say, when it’s task is done)

This may results in unnecessary load on the server side

You could run the next request only upon successful fetch request, and write something like this:

function poll(collection, delay) {
  collection.fetch({
    data: {foo: 'bar'},
    success: function(){
      setTimeout(function(){
        poll(collection, delay);
      }, delay)
    }
  });
}

This solves #1 bit doesn’t help other issues.
It takes a little more than that, and starts smelling like a special helper is needed.
You could add a poll() and unpoll() methods to your base models or collections using inheritance or mixin.
I wouldn’t like to promote such approach since it breaks the model’s interface and introduces tight coupling between the model and application state.
This easily becomes a slippery way down to maintenance hell.
A better pattern would be to use an external helper that will keep our interfaces clean and our models stateless and decoupled.

I ended up writing a Backbone Poller, A polling helper utility for Backbone.js.
a basic usage for our messaging system will could like

var inbox = new InboxCollection();
var poller = Backbone.Poller.get(inbox, {
    delay: 2000, 
    data: {sort: 'date desc'}
});
poller.on('success', function(){
    // do something on every successful fetch
});
poller.start();

A more advanced usage for our long going offline task could look like

var options = {
    delay: 2000,
    // when this becomes false the poller will stop
    condition: function(model){
        return model.get('done') !== true; 
    }
};

var task = new OfflineTaskModel();
var poller = Backbone.Poller.get(task, options);
poller.on('complete', function(){
    // task is done, do something
});
task.set('command', 'some-long-procedure');
task.save({},{
    success: function(){
        poller.start();
        
    }
});

The code and examples are available under the MIT license in github.

Posted in Javascript | 4 Comments

Book Review: RESTful Web Services

 I’m working on a project that includes writing a client for a fairly sophisticated RESTful API.

To get some better understanding of both client and server RESTful philosophy I purchased and read RESTful Web Services by Leonard Richardson, Sam Ruby (O’Reilly Media, 2008).

I tend to agree with the foreword by David Heinemeier Hansson: “Every developer working with the Web needs to read this book.”

The web is RESTful and therefor, when working with the web one shall at least understand RESTful concepts.

This book is doing a great job describing to details the concepts of addressability, statelessness, connectedness, and the uniform interface, and clarifying why they are important fundamentals of the web.
It gives a great overview of HTTP, URIs, Design of a REST server and client.

If I had to take one thing from the book, I would take the importance of state management.
It is crucial to understand where the state of the application belongs to.
Saving the application state on the back-end breaks the stateless nature of a web app, and could end up with an application that is hard to maintain and scale.
Keeping the state in the client side (or even better – in the URI), and keeping a sane, loosely coupled design leads to scalable web application.

One thing I was not very impressed with are the code examples.
I skipped most of them, since I didn’t buy this book to learn how to implement a REST server using RoR or Django, but choosing XHTML as the document format is very odd, and using a <div> or a <span> to represent semantic data is even odder.
Calling HTML5 XHTML5 is annoying, but given this was published in 2008 I given it the benefit of the doubt.

Bottom line, this was a good read and I highly recommend it.
If you are looking for a book to give you some detail oriented overview of REST concepts, this book is for you.

Posted in Books | Leave a comment

Javascript Design Patterns: Strategy

For a quick assignment I was asked to write the following program:

In the language of your choosing build an application that, given an English language document as input, builds a data structure capturing the count of each word in the document.

Bonus:
Create an inverted index as well: a mapping of the words to absolute positions in the document.

As context, this is a foundational algorithm for search systems and information retrieval. Be sure to consider accuracy, complexity, and intent in the design of your program.

The last phrase was the one that got me thinking.
This sounded a little vague, so I started thinking what kind of design I can implement that would be flexible enough so I can apply the algorithms later.
This kind of program usually runs on the back end using Java/Python/etc, but obviously, the language of my choice was javascript.

I started by defining a master processor:

var Processor = function(doc) {
    this.doc = doc;
};
Processor.prototype.parse = function(parser, sorter) {
    data = parser.parse(this.doc);
    if (sorter && typeof sorter.sort === 'function') {
        return sorter.sort(data);
    }
    else {
        return data;
    }
};

This processor gets a document (in our simple example, a long string) and has parse method, which accepts a “parser” object that has a parse() method and an optional “sorter” object that has a sort() method.
As the names suggest, the former will parse the document, and the latter will sort the results.
This use of the Strategy Design Pattern will allow the application to be flexible for algorithm changes, in the future.

First, I defined an abstract Parser object.
This object is responsible to convert the input to a standard format, and apply an algorithm

var Parser = function(){};
/**
 * The implementation will return an object literal 
 * with the format {word: num, word: num ...}
 * 
 * @param str {String}
 * @returns {Object}
 */
Parser.prototype.parse = function(str){
    throw 'please extned abstract object';
};

/**
 * Convert an object literal into a sortable array
 * with the format [[word, num], [word, num], ...]
 * 
 * @param data {Object}
 * @returns {Array}
 */
Parser.prototype.convert = function(data) {
    var arr = [],
        data = data || {};
    for(var key in data) {
        if(data.hasOwnProperty(key)) {
            arr.push([key, data[key]]);
        }
    }
    return arr;
};

My next step was defining the parsers I was requested to write:
One for capturing the count of each word in the document:

var CountParser = function(arr) {
    this.arr = arr;
};
CountParser.prototype = new Parser; // extend the base parser
CountParser.prototype.parse = function(str) {
    var value, i, l, data = {},
        arr = str.split(/[\s]+/);
    for(i=0, l=arr.length; i<l; i++) {
        value = arr[i];
        data[value] = data[value] || 0;
        data[value]++;
    }
    // convert our object literal into a sortable array
    return this.convert(data); 
};

And another one for a mapping of the words to absolute positions in the document:

var IndexParser = function(arr) {
    this.arr = arr;
};
IndexParser.prototype = new Parser; 
IndexParser.prototype.parse = function(str) {
    var value, i, l, data = {},
        arr = str.split(/[\s]+/);
    for(i=0, l=arr.length; i<l; i++) {
        value = arr[i];
        if( ! data[value] ) {
            data[value] = i;
        }
    }
    return this.convert(data);
};

Next, I defined my Sorter object. It is an object that takes a comparator function as an argument and can later be ran agains arrays using the Array.sort method:

var Sotrer = function(comparator) {
    this.comparator = comparator;
};
Sotrer.prototype.sort = function(arr) {
    arr.sort(this.comparator);
    return arr;
};

Then, I implemented the two basic comparator algorithms: Ascending and Descending:

var ascCompare = function(a, b){
    return a[1] - b[1];
};
var descCompare = function(a, b){
    return b[1] - a[1];
};

The only thing left now is to run an example:

var doc = "Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium, totam rem aperiam, eaque ipsa quae ab illo inventore veritatis et quasi architecto beatae vitae dicta sunt explicabo. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit, sed quia consequuntur magni dolores eos qui ratione voluptatem sequi nesciunt. Neque porro quisquam est, qui dolorem ipsum quia dolor sit amet, consectetur, adipisci velit, sed quia non numquam eius modi tempora incidunt ut labore et dolore magnam aliquam quaerat voluptatem. Ut enim ad minima veniam, quis nostrum exercitationem ullam corporis suscipit laboriosam, nisi ut aliquid ex ea commodi consequatur? Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur, vel illum qui dolorem eum fugiat quo voluptas nulla pariatur?";
var processor = new Processor(doc);

var indexParser = new IndexParser(), 
    ascSorter = new Sotrer(ascCompare),
    countParser = new CountParser(),
    descSorter = new Sotrer(descCompare);
    
console.log(processor.parse(indexParser, ascSorter));
console.log(processor.parse(countParser, descSorter));

This design, based on strategy pattern, can later on take any parsing algorithm and any sorting algorithm with out the need for refactoring the core application.

Posted in Javascript | 5 Comments

Backbone.js URI Sync Adapter

I am working on a backbone based search page, and was trying to figure out what would be a simple way to have both a state model and a URI that represents that state of the page (the search string, search parameters, user preferences, etc.).

Thinking of mixing backbone routes and the application models seemed like an overkill, and would force me to handle everything through a route or an event.

After a short struggle, I realized that the best case would be to have a model that keeps me in model land, but has the representation in the URI, and came out with Backbone URI Storage.

The concept is very simple, and works like charm – use a standard backbone model, enjoy data binding, and simply hook the model in the URI.

var State = Backbone.Model.extend({
    sync: uriSync
});
var Search = Backbone.Model.extend({
    url: '/rest/v1/search/';
});

And now, I can update the application state when my search changes:

var state = new State({id: 'search-page'});
var search = new Search();
search.on('change', function(){
    state.save(search.toJSON());
});

A working code example can be found here.

Its as simple as that!

Posted in Javascript | Leave a comment

Backbone.js computed properties

A simple yet powerful way to create backbone model calculated fields or “macros”

For one of our applications, we needed a way to store “macros” in backbone models, that will return a calculated value, based on values we have in the model fields.

I came up with a quick solution that seems to be good enough so far.
the concept is, the same way you can set anything to a field value, a function in javascript IS a value, so nothing holds you from setting a function in a field:

var myModel = new Backbone.Model();
myModel.set('func', function(){
    return 'hello calculated field!';
}, {silent: true}); // bypass the change event

Later on we can get the value of this function with a call such as:

var value = myModel.get('func').call(myModel);

This works, but it is not very convenient.
I wanted an automated way to use calculated methods without any side effects, which will include the field results in the model signature.
JSON.stringify ignores functions so consider this:

    
var foo = {
    bar: function(){
        return 'bazz';
    }
};
JSON.stringify(foo); // prints "{}"

Based on that, the original backbone toJSON will just work as is, ignoring our macros.
However, if we wanted to have the values our macros returns in our signature, we want to make sure it is being called, returning the resut of the value rather than the function.

I came out with this:

var BaseModel = Backbone.Model.extend({
  get: function(attr) {
    var value = Backbone.Model.prototype.get.call(this, attr);
    return _.isFunction(value) ? value.call(this) : value;
  },
  toJSON: function() {
    var data = {};
    var json = Backbone.Model.prototype.toJSON.call(this);
    _.each(json, function(value, key) {
      data[key] = this.get(key);
    }, this);
    return data;
  }
});

As the code tells us, in our new get method we are returning values of calculated fields, and in our new toJSON method we are including computed properties by values.

This could be great, but we have to remember that model.toJSON() is what Backbone sends back to the server when we perform a save operation, so it may be a good idea, depending on the server implementation, to exclude all computed properties from the model signature:

toJSON: function() {
  var json = Backbone.Model.prototype.toJSON.apply(this, arguments);
  _.each(json, function (value, key) {
    if (typeof value == 'function') {
      delete json[key];
    }
  });
  return json;
}

An implementation could look like:

var MyModel = BaseModel.extend({
    defaults: {
        avg: function(){
            return (this.get('min') + this.get('max')) / 2;
        }
    }
});
var myModel = new MyModel({min: 1, max: 17});
var avg = myModel.get('avg'); // 9

One “drawback” we need to be aware of, is due to the nature of the implementation, you can not bind to change events on this field, since the value stays the function and what changes is the computed value it returns.

Simple? Powerful?

EDIT 3/14/2012:
If you do want to observe changes when a computed property value changes, you will need the system to be aware of which attributes are affecting it state.
This could be done like that:

var avg = function(){
    return (this.get('min') + this.get('max')) / 2;
};
avg.attributes = ['min', 'max'];
myModel.set('avg', avg);

Now we have a way to track which fields are impacting our computed property.
What’s left to do is observe changes on these fields, and fire a change event on our property when these happen.
We could attach the event listener on the “set” method, but that would mean we will need to keep track on when we unset, reset, etc and remove that listener.

A cleaner (yet a little less efficient) approach would be to listen to all change events, and figure out upon it if a related field had changed.
This could be done like this:

myModel.on('change', function(){
  var i, changedAttributes = this.changedAttributes() || [];
  _.each(this.attributes, function(value, key){
    if( _.isFunction(value) && _.isArray(value.attributes) ) {
      for(i in value.attributes) {
        if ( _.has(changedAttributes, value.attributes[i]) ) {
          this.trigger("change:"+key);
          return ;
        }
      }
    }
  }, this);
}, myModel);

We can add this to out initialize method, and get it automated, so our complete model will look like:

var BaseModel = Backbone.Model.extend({
  initialize: function(){
    this.on('change', function(){
      var i, changedAttributes = this.changedAttributes() || [];
      _.each(this.attributes, function(value, key){
        if( _.isFunction(value) && _.isArray(value.attributes) ) {
          for(i in value.attributes) {
            if ( _.has(changedAttributes, value.attributes[i]) ) {
              this.trigger("change:"+key);
              return ;
            }
          }
        }
      }, this);
    }, this);
  },
  get: function(attr) {
    var value = Backbone.Model.prototype.get.call(this, attr);
    return _.isFunction(value) ? value.call(this) : value;
  },
  toJSON: function() {
    var json = Backbone.Model.prototype.toJSON.apply(this, arguments);
    _.each(json, function (value, key) {
      if (typeof value == 'function') {
        delete json[key];
      }
    });
    return json;
  }
});

The question is: since a computed property is a macro/helper and not a “real” part of our data model, and they never do actually change, should we allow then to trigger change event?
I personally believe we shouldn’t use this approach, for it abstracts the true nature of the component. However, it can be useful.

Posted in Javascript | 7 Comments

Swedish Greys - a WordPress theme from Nordic Themepark.