Elmish now supports RemoteDev time-travelling debugger

Screen Shot 2017-01-11 at 11.33.13 AM.png

Since day 1 elmish, developed at Prolucid for our front-end applications supported console logging of states and updates taking place in the application. You have to see it to appreciate how powerful of a feature that is and for a while it was enough. That is until ever-prolific¬†@forki decided he wants a full blown Elm-like debugger ūüôā

Thanks to RemoteDev tools and its developer Mihail we now have the experimental support in Elmish that makes the features like time-traveling and import/export available to apps targeting web and mobile!

For details see the README, the React sample and the React Native sample.

The Chrome and Firefox extensions¬†are easy to install, but the setup for mobile is a little-bit complicated, due to the fact that the communications are done over the network. It requires either a connection to a public cluster or running a local server, with everything that entails – making sure the routes are setup, HTTP/s traffic possible, ports are forwarded, etc. But it’s all very well documented in RemoteDev server repo.

Big Thanks to Steffen for pushing for the feature and Mihail and Alfonso for their support in making it possible!

Advertisements
Elmish now supports RemoteDev time-travelling debugger

Authoring Fable bindings: inheritance and variance in action.

JavaScript cheats. If you want to¬†pass some properties, there’s no need for a type to describe them – just return anything you want inside of curly braces and you are done. Of course there’s a cost, but that not what this post is about.

How does one achieve this level of freedom in order to integrate with JavaScript libraries out there when writing in a statically typed language РF#? Fable provides several mechanisms for transparent interop with the rest of JS world:

  • Record Types – going from typed¬†to untyped¬†is not a problem;
  • Discriminated unions with a special attribute¬†[<KeyValueList>].

Currently a typical way to create¬†new¬†fable bindings for a library starts with TypeScript definitions imported via ts2fable. ¬†The tool produces F# type definitions that closely resemble the OO roots of TypeScript definition, but are not particularly interesting to work with from a functional-first language. So the usual second step when¬†authoring a binding is a creation of a “helper” DSL – conversion of generated interfaces to KeyValueList discriminated unions and addition of factory functions to take lists of DU¬†instances that describe¬†various properties of the target.¬†Let’s take a look at an example, from React JSX intro:

Brief detour: JSX is template language that lets you mix¬†React elements described in XML/HTML-like syntax with JS6, it’s transpiled later into plain old JS. So in lines 2-4 we see a construction of h1element with implicit child element (“hello…” text) and an explicit className¬†and key attributes set. With regards to React, key¬†is unnecessary and may even be invalid, but this post is not about React and the attribute is¬†perfect to discuss the types involved.

We’ll skip the actual JS definition of the h1,¬†just keep in mind that we are setting a couple of properties and let’s¬†dive into TypeScript definition for each:

We can see that TypeScript introduced an interface hierarchy to describe something that would be a runtime composition in JS – object is just an array, indexed by property name. Leaving aside for now the interesting use of¬†type intersection, let’s take a look at what F# binding we get from ts2fable:

Unsurprisingly we get¬†a couple of interfaces Props¬†and HTMLAttributes, unified in HTMLProp. Let us consider how we would use the interfaces like that… we’d need an object of type HTMLProp¬†that we’d call an interface method on, but it’s only useful for callbacks. The callbacks are contravariant¬†-as long as it receives¬†the “biggest” type¬†we are type-safe.

But how would we construct an object in type-safe manner using these definitions? The answer is we can’t. Constructors need a covariant list of properties.

Let’s see what the idiomatic F# DSL offers on top of the imported definitions:

Here we see¬†the DUs¬†representing the properties from the imported interfaces and the “factory” functions to facilitate assembly of the element objects. We also see a marker interface IProp that doesn’t exist anywhere in the previously seen hierarchy. Why do we need the marker interfaces? To answer that question, let’s see the usage:

We pass HTMLAttr.ClassName and Props.Key as the list elements and a child text element.

What is interesting is that here we have a typed list with instances coming from different types. In JS we just bundle anything we like with {}¬†, in¬†TypeScript we can do the same, but there are also “intersection types” that allow one to express mixing of various properties together, in F# we need the¬†artificial marker interface to relate the unrelated types, so that our list could be constructed.

What we see taking place is transition from interface definitions usable by contravariant callbacks to a covariant list of properties that can be passed into the createElement and it required the unifying interface type to make it possible.

Unlike the fairly simple React hierarchy, which if you squint looks like a match for the original¬†inheritance tree, ReactNative is more involving and requires a lot of unifying interfaces¬†– one for each¬†valid combination of properties for a given element and requires careful thought and consideration. It’s a work in progress. On the other hand what¬†JS can only check at runtime, F# can check at compile time, as well as provide a nice IntelliSense experience!

Authoring Fable bindings: inheritance and variance in action.

Cross-platform UIs with F# and Fable

If you are small vendor and your primary focus hasn’t been in designing UIs, entering the field today presents you with too many choices.¬†However, if you’d like to use existing expertise and develop new ones in way that would accommodate broad range of platforms: Web, Mobile, Windows and, ideally, OSX and linux – the choices shrink dramatically.

We have some WPF expertise in house, but due¬†to¬†premature demise of Silverlight it has become a non-transferrable skill. Xamarin seemed like a way forward, but it would fragment our investment into Mobile/Desktop and despite Xamarin’s accomplishments (and they are impressive) it remains a fragile niche and having tried it we decided to keep looking.

React (and its Native derivative) with “learn once, write anywhere” approach seems like a promising direction but it has one (big) problem – JavaScript. Having built statically-verifiable code the weak¬†and¬†dynamic nature of JS leaves the language entirely unattractive. On the other hand JavaScript as ecosystem is like a¬†lab full of¬†petri dishes, rapidly blossoming and quickly killing off an infinite stream of ideas. It’s great of course that it’s happening, but trying to figure out the minimal viable combination of tools and libs… the fatigue sets in rather quickly.

Over the past couple of years we at Prolucid have been building up our F# skills developing backend systems and looking at¬†Elm with its beautiful implementation of¬†“model view update” architecture and Fable¬†with its amazing capabilities I¬†realized we may have our way forward.

Building hardware and low-level device software I¬†expect we’ll be dropping into native quite a bit, but doing native in the tools that were designed for it seems like an excellent idea anyway. At the same time we’ll be able to reuse tools, core logic and models across platforms and tiers.

What was lacking is F# implementation of Elm’s dispatch loop, similar to what Redux does, but w/o all the overhead seemingly designed to overcome the language shortcomings. There’s an implementation in fable-virtualdom from Tomas Jansson, but it’s tied at the moment with virtual dom management.

To that end an early build of Elmish is now available on npm. Elmish is a small library that has been designed following Elm’s concepts and terminology and it should work with React, ReactNative, virtualdom and any other DOM/rendering framework.

There are samples for React (Counter, TodoMVC) and React Native.

With exception of F# syntax and explicit dispatch¬†function being passed around one could follow Elm’s documentation when studying this approach to writing a UI, but hopefully I’ll get some time to write some docs¬†in the next few weeks.

Many thanks to Fable contributors for the help with the ecosystem and for making F# a competitive language to write cross-platform UIs in!

Cross-platform UIs with F# and Fable

Protoshell and Thriftshell update

As part of FsShelter development I had to implement the multilang serilizers for Storm, originally building against then current Storm 0.10.0.

I’ve started with Thrift, thinking that since its already a part of Storm runtime it would make the adoption easier compared to protobuf and given similar ¬†characteristics the downgrade in performance would not be noticeable.¬†After some testing it turned out that Thrift performed nearly at the speed of JSON (better with some payloads, worse with others), which might require some explanation.

Unlike monolithic protobuf, Thrift has a pluggable model for pretty much everything. So when people say Thrift they should qualify at least two¬†things: Transport and Protocol. Thirft looks comparable to protobuf only when Compact protocol is used. Compact however has a caveat, it doesn’t work with streaming transports, unless you implement custom framing logic to achieve something similar to protobuf’s¬†ParseDelimitedFrom functionality. And Storm is all about streaming, which is why I’m deprecating the support for Thrift.¬†Unless someone wants to maintain it I’ll be removing Thrift support from the future releases of FsShelter.

Protoshell on the other hand gets an update – Storm 1.0 has been released and some packages have been renamed. The new 1.0.1 release of Protoshell is now available and has been tested to work with latest Storm, so now FsShelter can benefit from massive performance improvements made in Storm.

FsShelter does not require a new build to benefit from this release. All one needs to start running FsShelter components against Storm 1.0.1 is the new server-side serializer implementation, which can be referenced directly from github as a paket dependency and included with the topology for deployment.

Protoshell and Thriftshell update

An easy way to try FsShelter

Thanks to docker, trying something out w/o having to figure out all the dependencies and pollute your system has become really easy. This is how I started with Storm and this is the way we now help others to try FsShelter as well Рfsshelter-samples container.

The container includes¬†an installation of Storm, Mono, F# and a pre-built clone of FsShelter repo. Original¬†build of¬†Mono (4.2.1) caused processes¬†to crash¬†now and then and was an interesting study in how Storm deals with failures and what it means for processing guarantees. Current version (4.2.3) runs solid and may deprive you from witnessing¬†Storm restarting all the components… you may have to crash them yourself¬†ūüôā

An easy way to try FsShelter

FsShelter: a Storm shell for F#

About a year ago Prolucid¬†adopted Apache Storm as our platform of choice for event stream processing¬†and F# as our language of choice for all of our “cloud” development.

FsStorm was an essential part that let us iterate, scale and deliver quickly, but even from the earliest days it was obvious that the developer experience could be improved. Unfortunately, it meant a complete rewrite of FsStorm:

  • FsStorm DSL is a really thin layer on top of Nimbus API model:
    • has explicit IDs when describing components in a topology
    • uses strings in all the names
    • matching of inputs/outputs is not guaranteed
  • FsStorm uses Json AST as it‚Äôs public API:
    • messages, tuples,¬†configuration
    • serialization mechanism¬†is hard-baked into the API

We’ve worked around some of the problems, usually by writing more code.

It actually makes sense that Storm itself doesn’t care about the type of the tuples/fields. It runs on JVM, which is very much typed, and it relies on sub-class polymorphism to make things¬†tick. However the public API for¬†the tuples looks like an afterthought in every language. But we figured, there is this “compiler” that can do “type checking”¬†for us, let’s make it work!¬†Maybe we can even make it faster if we replace Json with Protobuf?

Coming up with the new DSL that would allow the components to consume and emit tuples of various (static) types on multiple streams was an interesting experience and led to some strange places.¬†A lot has been written on F# DSLs, but none of that applied directly. Can I use “just functions”? Do I need a type provider? A computation expression? A compiler as a service?

logo

After a few false starts I found the¬†desired paradigm that could be expressed in F# succinctly. As it usually happens, once I gave up on certain notions (building “any purpose” graph from a single source in this case), the result was pretty simple.¬†And so, after a few weeks of journey and discovery, we are releasing FsShelter: a way to program¬†Storm with F# in a statically typed fashion.

Many thanks to Tomas Petricek, Scott Wlaschin, Andrew Cherry and Erik Tsarpalis, without them FsShelter wouldn’t have been possible.

FsShelter is currently in beta and any feedback is welcome and appreciated.

FsShelter: a Storm shell for F#

Real-time analytics with Apache Storm – now in F#

Over the past several month I’ve been¬†prototyping various aspects of ¬†an IoT platform – or more specifically, exploring the concerns of “soft” real-time handling of communications with potentially hundreds of thousands of devices.

Up to this point, being in .NET ecosystem I’ve been building distributed solutions with a most excellent lightweight ESB – MassTransit, but for IoT we wanted to be a little closer to the wire. Starting with the clean slate and having¬†discovered¬†Apache Storm and¬†Nathan’s presentation¬†I realized that it addresses exactly the challenges we have.

It appears to be the ultimate reactive microservices platform¬†for lambda¬†architecture:¬†it is fairly simple, fault tolerant overall, yet¬†embracing fire-n-forget and “let it fail” on the component level.

While Storm favours JDK for development, has extensive component¬†support for Java developers and heavily optimizes for JRE components execution, it also¬†supports “shell” components via its multilang protocol. Which is what, unlike Spark makes it interesting for a .NET developer.

Looking for a .NET library to implement Storm components there’s the¬†Microsoft’s implementation¬†– unfortunately components in C# end up looking rather verbose and it¬†happens to work exclusively with HDInsight/Azure,¬†which is a deal breaker for us, as we want our customers to be able to run it anywhere. Fortunately though, further search revealed recently open-sourced FsStorm announced on Faisal’s blog¬†and¬†I liked it at first sight:¬†concise F# syntax for components and the¬†DSL for defining topologies makes¬†authoring with it a simple and enjoyable process.

The FsStorm components could be just a couple of lines of F#, mostly statically verified, have clear lifecycle and easy to grasp concurrency story. And with F# enjoying 1st class support on Mono, we are able to run Storm components effectively on both dev Windows boxes and distributed Linux clusters while capitalizing on productivity and the wealth of .NET ecosystem.

It is now available under FsStorm umbrella as a NuGet package, with CI, a gitter chatroom and a bit of documentation.

While¬†still in its early days, with significant changes¬†on the horizon¬†– something I want¬†to tackle soon is static¬†schema definitions for streams and pluggable serialization with Protobuf by default, I believe it is ready¬†for production, so go forth and “fork me on GitHub”!

Real-time analytics with Apache Storm – now in F#