In short: Backbone.js is bigger than you think.
Backbone.js advertises itself as 6.3 kilobytes, but that's a little misleading. Its dependencies can make it one of the largest libraries.
To compare, there are many libraries that have no dependencies and are smaller, and they do data binding to boot. Knockout.js is 14 KB, Angular.js comes in at 29, and Funnyface.js is just below the 10 kilobyte mark. Ember.js is larger, at 49 kilobytes. All of these libraries do data binding, so you don't have to code that yourself.
I think worrying about filesizes is often a premature optimization, but for how little Backbone does, it's surprising that it's so large.