Skip to main content

Hi there! I am a programmer and I'm very enthusiastic about OpenSource and OpenKnowledge.

twitter.com/koehr_in

github.com/nkoehring

alpha.app.net/koehr

koehr.in

Norman Köhring

Vuejs Reactivity From Scratch

8 min read

Vuejs Reactivity From Scratch

Vuejs is the star newcomer in the Javascript Framework world. People love how it makes complicated things very simple yet performant. One of the more exciting features is its seemingly magic reactivity. Plain data objects in components magically invoke a rerender when a property changes.

The button click invokes a function that just assigns a new value to a property. Still the template gets automagically rerendered. But we all know there is no fairydust involved, right? So how does it actually work?


The magic of getters and setters

With the ES5 standard JavaScript got lots of exciting new features. Some of them highly underrated and underused in my opinion. Getters and setters are an example. If you never heard of them, I would recommend you to read John Riesigs piece on them.

As soon as you know what getters and setters are: functions transparently called on every property access, you might already know where this goes. Boom! All the fairydust suddenly disappears.

Automatic getters and setters

Now that we at least in theory know how Vuejs realises the template data magic, let's build it ourselves for the sake of full understanding!

Abstract: A function that gets an object and returns one with the properties replaced by getters and setters that, on call, rerender a template. So far so good. If you are really impatient, you can find the final code in JSFiddle.

Let's start with a very simple approach:

The function iterates through all object keys and creates a new object with getters and setters in their place. It could also directly manipulate the original object:

I personally don't like to manipulate the existing object and prefer the first way.

Introducing: Object.defineProperty

Now before we go on with destroying our fantasies of fairydust computing, let's see if there is a more convenient way to what we've done for now. Here I introduce Object.defineProperty, which allows to set all possible attributes for the properties of an object. You can find a detailed description on MDN.

With this new knowlegde, the code can be made a bit more readable, by condensing everything into one call:

All those underscores where pretty annoying anyways. I generally suggest you to read more about Object.defineProperty. It extends the range of possibilities significantly!

Templating for the poor

To be able to rerender a component on data change, we should really introduce some components that can actually render and under the right circumstances rerender a template.

This code describes a very simple component, that has a data object and a render function. If this is called, it replaces the innerHTML of the given content element with the rendered output. Neat! Let's make the data reactive!

Reactive Component

As a start, it should be enough to simply make the data property reactive:

Yes, that seems to be good but it doesn't really update the template. Which becomes clear after a look at line 11-14: There is no render call ever. But reactive shouldn't know about component rendering, right? Let's try a more general approach with a callback:

Yeah, that works and so on but it looks like we slowly stumble away from elegance in our code. The changes in reactive() seem to be okay, but that function bind monstrosity in line 31 is something we better hide from our parents. Let's introduce a component factory before we get kicked out or end up in self hatred:

Cool! That works. The createComponent() function just does all the dirty work for us and returns a nice, reactive component, that is still just a simple object. If you have that code in a local setup and run something like component.data.name = 'Ada Lovelace', then it will automagically rerender the template to show 'Hello Ada Lovelace'.

Nested Data structures

All cool and hip stuff but what happens in the following scenario:

Setting deeper nested properties (line 44,45) doesn't work at all. The reason is that the reactivity only works on the first nesting level of the data object. Now you could say: Easy, we just set the whole object at once:

But this is not really what we strive for, isn't it? What we need is a way that makes all nested objects reactive in a recursive way. Surprisingly, this just needs a coupe of lines:

Only three lines (7-9) where added. They call reactive() on the given value in case it is an object. Now the nesting level doesn't matter anymore. REACTIVE ALL THE THINGS!!

Multiple Components

Considering that components are usually very gregarious, what happens if we find a friend for our component? Will it blend? Erm I mean, react?

It does! Hooray!

The attentive reader might have seen the change that sneaked into line 7: Because the type of array is object, an extra check has to be made here. Otherwise the array would be transformed to a plain object with keys 0, 1, etc.

But what happens now when we manipulate the Array directly?

Bummer! Setting the whole array works as expected but manipulating it doesn't trigger any change.

Reactive Arrays

As described in the caveats section of the Vuejs guide about list rendering, there are several …well caveats with array reactivity. It writes:

Due to limitations in JavaScript, Vue cannot detect the following changes to an array:
1. When you directly set an item with the index, e.g. vm.items[indexOfItem] = newValue
2. When you modify the length of the array, e.g. vm.items.length = newLength

Okay, fair enough. So what does happen in Vue to make Arrays reactive? Fairydust? Unfortunately yes. And this stuff is expensive! Nah, just kidding. Of course it is again no magic involved. I'm sorry my dear role-playing friends. What actually happens is that Arrays get their manipulating functions replaced by a wrapped version that notifies the component about changes. The source to this functionality is in core/observer/array.js.

Vuejs' approach is rather sophisticated but can be condensed down to something like what is seen in the first 24 lines here:

So this is a fairly big chunk to digest. The new function reactiveArray starts with creating a copy of the original array (Remember? I don't like manipulating the original object). Then, for each function in the list of manipulative array functions the original is saved which is then replaced by a wrapper function. This wrapper function simply calls the render callback additionally to the original array function.

Now also lipsumComponent.data.content is not set directly anymore but uses the overwritten push method. Setting it directly wouldn't work. Fixing that leads us to the last step:

Reactivity on set

For now the setter function didn't care about the value. If it would be a nested object, its children wouldn't be reactive. That means, if you set data.x to an object {foo: 1} and then change foo data.x.foo++, the template wouldn't rerender. This should be changed:

Instead of setting the plain value, reactive(value, callback) is called in line 49. This small change works only up to a certain point on its own though. The function has to decide what to do with non-objects or arrays, which happens now as a first step in reactive(). A plain non-object (remember: arrays are objects) simply gets returned as it is (line 30), arrays will be returned in their reactive version (line 31).

Conclusion

Congratulations! You made it this far or just skipped to read only the Conclusion, which is fine, I do that too sometimes.

In about 70 SLOC, we built a fully reactive component system. We made use of getters, setters and Object.defineProperty and learned, that I don't like to manipulate objects directly. Except for the last point, this should be valuable information that might become handy in future.

What else can be done you might ask? Vuejs' code is more sophisticated and handles some egde cases that I didn't mention for the sake of simplicity. For example if the yet to become reactive object has some getters and/or setters already, they would be overwritten by our simple solution. Vuejs' defineReactive uses Object.getOwnPropertyDescription to get a detailed information about the property it is going to wrap and incorporates existing getters and setters if applicable. It also ignores non-configurable (not meant to be changed at all) properties. How that works can be found in the source code.

Norman Köhring

The Magic 0xC2

3 min read

I built a web application with file upload functionality. Some Vue.js in the front and a CouchDB in the back. Everything should be pretty simple and straigt forward.

But…

When I uploaded image files, they somehow got mangled. The uploaded file was bigger than the original and the new "file format" was not readable by any means. I got intrigued. What is it, that happens to the files? The changes seemed very random but reproducible, so I created a few test files to see what exactly changes and when.

My first file looked like this:

0123456789
ABCDEFGHIJKLMNOPQRSTUVWXYZ
abcdefghijklmnopqrstuvwxyz

To my surprise, the file stayed the same! My curiosity grew. In the meantime I found a very intriguing pattern in uploads hexdump: C3 BF C3. It was everywhere. In another file, I found similar patterns with C2. So I wrote my next test file. This time a binary file:

00 01 02 03 04 05 06 07  08 09 10 11 12 13 14 15 |................|
16 17 18 19 20 21 22 23  24 25 26 27 28 29 30 31 |.... !"#$%&'()01|
32 33 34 35 36 37 38 39  40 41 42 43 44 45 46 47 |23456789@ABCDEFG|
48 49 50 51 52 53 54 55  56 57 58 59 60 61 62 63 |HIPQRSTUVWXY`abc|
64 65 66 67 68 69 70 71  72 73 74 75 76 77 78 79 |defghipqrstuvwxy|
80 81 82 83 84 85 86 87  88 89 90 91 92 93 94 95 |................|
96 97 98 99 a0 a1 a2 a3  a4 a5 a6 a7 a8 a9 aa ab |................|
ac ad ae af b0 b1 b2 b3  b4 b5 b6 b7 b8 b9 ba bb |................|
00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|

EDIT: As you probably already noticed, I counted up like in Base10 but it is actually Base16. So I skipped A-F until reaching A0. This might look weird but didn't affect the test.

The result after uploading was

00 01 02 03 04 05 06 07  08 09 10 11 12 13 14 15  |................|
16 17 18 19 20 21 22 23  24 25 26 27 28 29 30 31  |.... !"#$%&'()01|
32 33 34 35 36 37 38 39  40 41 42 43 44 45 46 47  |23456789@ABCDEFG|
48 49 50 51 52 53 54 55  56 57 58 59 60 61 62 63  |HIPQRSTUVWXY`abc|
64 65 66 67 68 69 70 71  72 73 74 75 76 77 78 79  |defghipqrstuvwxy|
c2 80 c2 81 c2 82 c2 83  c2 84 c2 85 c2 86 c2 87  |................|
c2 88 c2 89 c2 90 c2 91  c2 92 c2 93 c2 94 c2 95  |................|
c2 96 c2 97 c2 98 c2 99  c2 a0 c2 a1 c2 a2 c2 a3  |................|
c2 a4 c2 a5 c2 a6 c2 a7  c2 a8 c2 a9 c2 aa c2 ab  |................|
c2 ac c2 ad c2 ae c2 af  c2 b0 c2 b1 c2 b2 c2 b3  |................|
c2 b4 c2 b5 c2 b6 c2 b7  c2 b8 c2 b9 c2 ba c2 bb  |................|
00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

There it was again: The magic 0xC2!

So all bytes with a value higher than 0x79 got followed by a 0xC2. 0x79 is the ASCII code for y. This is at least what I thought. It actually is the other way around: All bytes with value 0x80 or higher got prefixed by a 0xC2! — there the scales fell from my eyes: UTF-8 encoding!

In UTF-8 all characters after 0x7F are at least two bytes long. They get prefixed with 0xC2 until 0xC2BF (which is the inverted question mark ¿), which is then followed by 0xC380. So what happened is, that on the way to the server, the file got encoded to UTF-8 ¯\_(ツ)_/¯

EDIT: Corrected some mistakes after some comments on Hackernews

Norman Köhring

Norman Köhring

Norman Köhring

Norman Köhring

So instead of advertising against bloated websites full of ads and trackers, let's change the browser! https://blog.chromium.org/2017/01/reload-reloaded-faster-and-leaner-page_26.html

Norman Köhring

Norman Köhring

Norman Köhring

Norman Köhring

Norman Köhring

Norman Köhring

Norman Köhring

Reading list for today contains: homomorphic , , , and !

Norman Köhring

Norman Köhring

It is just sad to read it. I'm seriously sad to be part of the male side sometimes. https://blog.jessfraz.com/post/this-industry-is-fucked

Norman Köhring

Oh my goodness! Looks like I need time off and lock myself! https://docs.rs/domafic/

Norman Köhring

Learning for today: If that one problem keeps staying despite all efforts, reconsider its source!

Norman Köhring

Probably the most complex speakable http://www.ithkuil.net and an interesting concept

Norman Köhring

Norman Köhring

Norman Köhring

Norman Köhring

That's what I hate so much about today's websites https://1-minute-modem.branchable.com

Norman Köhring

Norman Köhring

An evening of burgers, beer and number crunching awaits me!

Norman Köhring