Extensible Effects in Node.js, Part 2

By: Luke Westby April 8th, 2017

This part 2 in a 3 part series:


Hi friends! In the previous post we learned about using daggy to construct the Freer monad in JavaScript in order to make highly testable Node.js applications. We left off by constructing a Freer that could wrap another type and treat it like an AST. In its current state our Freer type provides us with map and chain functions for building up programs from this AST type that we can later interpret however we'd like. We can write interpreters for Tasks from data.task to do useful work at runtime, or we can interpret to a State monad to assert about the way our environment has changed during our tests.

As a refresher, here is the definition we have so far for Freer. There's a lot here so I definitely recommend that you take a look at the explanations from the previous post if needed.

const fmap = f => x => { if (x instanceof function) return compose(f, x) else if (typeof x.map === 'function') return x.map(f) else throw Error(`${x is not a Functor}`) } const Freer = daggy.taggedSum({ Pure: ['a'], Lift: ['f a', 'a -> b'], Chain: ['Freer f (Freer f a)'], }) Freer.prototype.map = function map(f) { return this.cata({ Pure: (x) => Freer.Pure(f(x)), Lift: (x, g) => Free.Lift(x, compose(f, g)), Chain: (x) => Free.Chain(fmap(fmap(f), x)) }) } Freer.prototype.chain = function chain(f) { return Freer.Chain(this.map(f)) } Freer.prototype.foldMap = function foldMap(f, point) { return this.cata({ Pure: a => point(a), Lift: (x, g) => f(x).map(g), Chain: x => x.foldMap(f, point).chain(y => y.foldMap(f, point)), }) } Freer.of = a => Freer.Pure(a) Freer.liftF = f => Freer.Lift(f, identity)

So this is pretty cool. But there are two problems with it when it comes to real-world usage. The first is chaining lots of code together can be tedious when our effects get really complicated. .chain() works fine when our code is short and simple, like

readFile('./in.txt') .chain(data => writeFile('./out.txt', data.toLowerCase())) .chain(() => readFile('./out.txt'))

but it will quickly get out of hand once we start introducing conditionals and additional business logic around our effect calls. The second problem is that our formulation for Freer doesn't allow us to do first-class error handling. If any one individual effect fails, it will result in a failure at the same position in the monad to which we interpreted, but this leaves no room for recovering at a custom location in our logic, reformatting and nesting errors for smart logging and reporting, or raising errors by choice. Let's tackle these one at a time.

Yielding Effects

In Haskell we can use do to express the chaining and binding of monadic values as a series of imperative-looking expressions.

do putStr "Hello" putStr " " putStr "world!" putStr "\n"

In JavaScript no such syntax is available. However, we can emulate it with a more general-purpose Generator and some clever calling code to run it. We will yield when we want an effect to run, and then proceed on with our surrounding logic. We can still use .chain() and .map() when it makes sense but at least this way we have options. The end result will look like this:

Freer.do(function* () { const data = yield readFile('./in.txt') const lowerCase = data.toLowerCase() // we can insert pure code wherever we want yield writeFile('./out.txt', lowerCase) const justWritten = yield readFile('./out.txt') return justWritten // as fun a bonus, we can also make return behave like the monadic return in Haskell })

Let's write Freer.do():

Freer.do = (generator) => { // Call the generator to get a reference to our iterator const iterator = generator() // Since the iterator will be held in closure we won't be able to reuse it if // we foldMap over the resulting Freer more than once. Since that's something // we should be able to do we'll cache the output of the iterator at each // step and then resuse those outputs const cached = {} // This function will connect each bit of code in between yields and return const step = (result, stage) => { // First, check if we already ran and if so just return what we got last time if (cached.hasOwnProperty(stage)) return cached[stage] // If the iterator has completed we want to treat the return keyword in JS // like the return function for monads in Haskell. This means checking if // the value returned was a Freer and if not wrapping it in one. I know this // isn't _technically_ the same as Haskell's return as returning something // that is already in a Freer would result in a Freer (Freer x) but it's // close enough for JS. if (result.done) { cached[stage] = result.value instanceof Freer ? result.value : Freer.of(result.value) } // If we aren't done we will take the Freer that was just yielded and chain // the next one onto it by calling .chain() and returing the next step given // the current value from the current Freer. else { cached[stage] = result.value.chain((a) => step(iterator.next(a), stage + 1)) } return cached[stage] } // And we kick things off by stepping in for the first time! return step(iterator.next(), 0) }

A special thank you to Brian Lonsdorf for his implementation of do in his repository freeky.

This makes for much cleaner code once we start introducing more complicated logic, if/else blocks, and so on. Great! So we solved a big problem of readability. But what do we do if something fails? In a generator it's common to use try/catch to deal with errors. How can we represent this behavior in Freer?

Fail, Or Else

Our current construction isn't sufficient to represent error situations. We need to add two additional constructors to Freer to be able to push failures and failure handlers onto the AST structure.

const Freer = daggy.taggedSum({ Pure: ['a'], Lift: ['f x', 'x -> a'], Join: ['Freer x (f (Freer x (f a)))'], Fail: ['x'], // This holds an error OrElse: ['Freer x (f a)', 'x -> Freer y (f b)'] // This holds a function that converts an error into a new Freer }) Freer.fail = Freer.Fail

We also need to update our instance functions to make sure not to do extra work if we've already failed, and to be able to handle that failure:

Object.assign(Freer.prototype, { map(f) { return this.cata({ Pure: (a) => Free.Pure(f(a)), Lift: (x, g) => Free.Lift(x, compose(f, g)), Join: (x) => Free.Join(fmap(fmap(f), x)), // If we've failed, we don't need to do anything. This will be a common pattern. Fail: (x) => Free.Fail(x), // When mapping over the OrElse case, we need both sides of the branching // structure it creates. We don't know until runtime what side we will need // so we map both over f. OrElse: (x, g) => Free.OrElse(x.map(f), compose(fmap(f), g)) }) }, chain(f) { // Again, we skip everything when we have a failure if (this instanceof Free.Fail) return this if (this instanceof Free.Pure) return f(this.a) return Free.Join(this.map(f)) }, orElse(f) { // If we have a failure already, we can immediately invoke the function f. // f has the signature x -> Freer y a, so calling it directly on the error // x gives us a new Freer. if (this instanceof Free.Fail) return f(this.x) // Otherwise we push the handler into the tree as an OrElse case. return Free.OrElse(this, f) }, // folMap now expects a fail argument, which will produce a failed version of // the interpretation monad in the same way that of lifts any other value into // the interpretation monad. foldMap(f, of, fail) { return this.cata({ Pure: (a) => of(a), Lift: (x, g) => f(x).map(g), Join: (x) => x.foldMap(f, of, fail).chain((y) => y.foldMap(f, of, fail)), Fail: (x) => fail(x), // foldMap also expects our interpretation monad to implement orElse now. // We get this for free with data.task's Task. OrElse: (x, g) => x.foldMap(f, of, fail).orElse((e) => g(e).foldMap(f, of, fail)) }) } })

These provide all the ingredients we need to catch errors, reformat them and throw them ourselves. We can now handle errors explicitly and check conditions to see if we should throw them.

readFile('./in.txt') .orElse(error => Freer.of(error.message)) .chain(data => writeFile('./out.txt', data.toLowerCase())) .chain(() => readFile('./out.txt')) .chain(data => data.length === 0 ? Freer.fail('No data') : Freer.of(data))

Now let's introduce error semantics into our makeshift do notation.

Freer.do = (generator) => { const iterator = generator() const cached = {} const step = (result, stage) => { if (cached.hasOwnProperty(stage)) return cached[stage] if (result.done) { cached[stage] = result.value instanceof Freer ? result.value : Freer.of(result.value) } else { cached[stage] = result.value .chain(a => { // Catch imperative errors throw by any logic in the current step // and convert them into Fail instances try { return step(iterator.next(a), stage + 1) } catch (e) { return Free.Fail(e) } }) .orElse(x => { // Catch any Fail instances that may have been emitted and give the // generator the opportunity to catch them by calling iterator.throw() try { return step(iterator.throw(x), stage + 1) } catch (e) { return Free.Fail(e) } }) } return cached[stage] } // Also guard the first step with a try/catch try { return step(iterator.next(), 0) } catch (e) { return Free.Fail(e) } }

Now we can use try/catch within our generators to emulate a more imperative-looking error-handling style

Freer.do(function* () { let data try { data = yield readFile('./in.txt') } catch (error) { data = error.message } const lowerCase = data.toLowerCase() yield writeFile('./out.txt', lowerCase) const justWritten = yield readFile('./out.txt') if (justWritten.length === 0) throw 'No data' return justWritten })

Good to go! Now we can write clear, easy-to-read imperative-looking code once our chaining gets more complicated. We can also deal with errors as a first class concept and even use JavaScript's built-in try/catch and throw within our generator functions to manipulated how errors flow through our programs.

That's all for Part 2. In Part 3 we'll talk about adding an explicit Applicative instance to our Freer construction to explain to our interpretation monad how to run things concurrently. To close it up we'll also demonstrate how to combine multiple interpreters together so we can split different effects up into modules. If you have any questions, please feel free to reach out on Twitter at @luke_dot_js.

Notable Clients

Tell us about your project

We’d love to help design & build your next big idea, or lend a hand on an existing project.

1765 N. Elston St.

Chicago, IL 60622