Category: Development

Building a proof of concept – off the ground in a few weeks

Many iterative-development thinkers have a notion of an “iteration zero” at the beginning of a project, that does not involve much software development but rather understanding the problem, choosing technology, choosing a set of features for a first major release, and so on. That work is often described as what happens at the beginning of a project, the beginning of ongoing work.

Having worked with many customers in the early stages of a project, we see a place for a small project-before-the-project. Not necessarily a first iteration to start ongoing work; but rather work before decision has even been made about when or whether to do a full project, to commit to ongoing work.

A customer comes to us, eager to show the potential of a project, but not yet in a position to commit to a long-term effort, nor eager to create a pile of non-software artifacts. Rather, they want to quickly show that something could work, how it could work, have some code in hand to serve as a working rough prototype. These needs will not be met by starting an effort that will take many months and many dollars to yield the first working result.

This is the problem we solve with a proof-of-concept engagement: One short pass through the whole development cycle. Such a project is a limited time and scope effort to understand a problem domain (just a little bit) and then write some software (just a little bit) that demonstrates the value of a customer’s idea. Such a project looks something like the description below, of course the details vary.

(A proof of concept effort is very workable for a greenfield project, not applicable inside a system that has an existing substantial code base. For those, we have another kind of initial engagement, to get an existing system up and running and survey its code base.)

Scope

  1. Meet with customer domain experts, typically for half the day for a couple of days. Understand enough to do initial, relevant work.
  2. Sketch (with a mix of code, drawing tools, and paper) what some of the most critical screens will look like.
  3. Verify with customer domain experts that the ideas we have captured are the most relevant to their vision.
  4. Implement: expand the most important aspects from mere static screens or drawings. Create working prototype code.
  5. Integrate working screens with other static mockup screens.
  6. Present the working prototype software to the customer, including source code.
  7. Assist the customer with installation of the working prototype in their environment, to enable easy internal demonstrations.
  8. Present a video demonstrating the working system, we have found this valuable especially when the prototype needs to be shown to a potentially wide audience around the customer organization.
  9. Discuss future steps with the customer.

The bulk of this work is understanding the problem and creating working code. A proof of concept effort is not about creating another long requirements document. It is not about working through all the details. It is about code that demonstrates a working implementation of the essence of the customer vision, demonstrating that if implemented it would solve a problem worth solving.

Technology

To get the maximum benefit from a short effort, we use technologies our team has extensive current experience with. Typically that will be (as of early 2017) Angular or React on the front-end, Node or Java on the backend. We work with numerous other technologies for production efforts, but some of them are less amenable to shipping a working result within days of project start.

Team

To delivery a working result in a a short time, the team on this kind of project consists of:

  • Two core, highly experienced developers (developer / trainers)
  • Assistance from other developers
  • Assistance from a designer
  • Assistance from a project manager

Schedule

The work happens over a 1-2 week period. Scheduling can be tough, because the developers involved will also be developers who have the deepest mastery of the technologies to be used – developers who also teach our classes, lead project teams. We work with customers to choose the right start date to do this successfully.

Location

Typically this work happens at our headquarters; with the meetings conducted with the usual remote meeting technology, or occasionally with a customer expert visiting. It is also possible to send a team to a customer site for this work – although with that variation, there is less opportunity for additional Oasis Digital team members to jump in.

Price

We charge a fixed, all-inclusive price for a fast proof-of-concept effort. Since the investment is known up front, there is no risk of exceeding an agreed budget, missing an estimate, and so on.

Where to go next

After a completed proof of concept project, our customer has working prototype software; but it is not a prototype in the sense of very poorly, hastily built code; because it is built by our instructor/developers, it is of surprisingly good quality, ready to form a starting point for a more substantial development effort.

If a development effort is warranted (sometimes the thing you learn from a working prototype, is that the idea is not worthy of major investment after all!) a customer in-house team could pick up a working prototype source code and run with it; or of course we at Oasis Digital can be involved in ongoing work.

Sounds good, how do I buy this?

This is just a blog post; there is no “buy now” button. Contact us to talk about your project, if it is suitable for proof of concept project, we can send a proposal.

 

Fast Angular 2 compile cycle with TSC

There are numerous choices for tooling in a TypeScript Angular 2 project. This list is no doubt incomplete:

  • SystemJS with JSPM
  • SystemJS without JSPM
  • Webpack
  • Angular-CLI (which uses System and Broccoli and other things behind the scenes)

(Warning – this content has been made obsolete by progress in Angular 2 and related tooling.)

At Oasis Digital, we have worked with all of these variations, and experienced trouble at least with all of them but have been around long enough to have trouble with. There is no clear winner yet, there are only trade-offs.

Speed matters. Specifically, speed of the development cycle, of how long it takes to compile/recompile an application and reload it in the browser during development. (I’m less concerned with the speed of a “production build” because those happen much less often and usually without a human actively waiting.)

As of May 2016 (these things change rapidly) we have found the following yields the very fastest results for the development cycle:

  • Bundle libraries using JSPM/System-Bundler or WebPack, or use bundles shipped by the Angular 2 team.
  • Set up tsconfig to use “module”: “system” and “outFile”: “somebundlenamehere.js”. TypeScript both compiles your code, and produces a single-file bundle of the results. This is much faster than having typescript emit numerous files and then bundling them with another tool.
  • Run tsc in your IDE; don’t bother to “watch” from the command line or by other means – better for compile errors to land in the IDE. There is no benefit to simultaneously compiling in your IDE and in a concurrent “watch” – omitting that (for example, by moving away from Webpack) roughly cuts in half the total CPU effort spent on each development cycle.

To demonstrate how little work is typically needed to set this up, see the following tiny “fork” of the angular.io “quickstart”. This single commit shows the slightly less configuration needed with the “module”: “system” approach.

https://github.com/kylecordes/quickstart/commit/3cf3041dd80ee8701a3cffb0cbe6264784017a2b

Of course, there is no benefit to this change in the quickstart application – the tutorial adds only a handful of components. But we have confirmed the performance lead with other teams developing on Angular 2 “at scale”. This approach is, as of May 2016, the fastest compile-reload cycle for large Angular 2 applications.

It might not last long; there are many efforts underway, including the official angular-cli, to build better development tooling. Moreover, Hot Module Reloading might make speed up the development cycle enough that the clean-reload time isn’t important.

AngularJS Testing Frameworks – Mark Katerberg – Angular Lunch

At our September 9, 2015 Angular Lunch, Mark Katerberg compared the different testing frameworks with AngularJS 1.x. He shows and explains the differences between Jasmine, Mocha and associated libraries, and how they compare for Angular application unit testing.

If you don’t have time to watch it, here is my super-high-level summary:

  • Karma+Jasmine is a solid, very usable default. It is also what we use here at Oasis Digital frequently.
  • Mocha and its associated libraries are more popular for server-side testing, and have certain advantages if you are willing to learn and adopt them.
  • QUnit is most appealing to those already familiar with an older generation of testing tools, but also has completely acceptable features for modern unit testing.

But don’t take my word for it. Watch the talk video, Mark has compared all of these and more.

Transcript

We have transcribed the talk about to text, provided below. This is a rough, first-draft transcription, so any errors are probably from that process.

Continue reading AngularJS Testing Frameworks – Mark Katerberg – Angular Lunch

Introduction to Event Sourcing and Command-Query Responsibility Segregation

The concepts of event sourcing (ES) and command-query responsibility segregation (CQRS) have been around for quite a while. They are getting more and more attention in the Java community, though they seem to have been much more popular over on the .NET side.

I have spent the last few months implementing a system with such an architecture at Oasis Digital. While working on it, we aimed for flexibility and the ability to make the right trade-offs and choose our own tools. Some interesting solutions came up in the process, things that would probably be impossible with existing frameworks like Axon.

I am going to write more about some of these pieces in the future. Before we get there, let’s start with a brief introduction to event sourcing and command-query responsibility segregation.

CQS

The main idea of command-query separation (CQS) is that all operations are either:

  • commands, changing state of the system,
  • queries, getting some information from the system.

Either one or the other, never both. For example, if a command changes anything in the system, it should not be used to read its state. Mutation-free read access should always be possible. Asking a system to change something to read a value seems plain wrong, and queries inadvertently changing state are very confusing, surprising and leading to bugs at best.

These principles can be applied on all levels. Uncle Bob in “Clean Code” calls it one of the main principles of good function design. The pattern is also applicable to system design.

CQRS in System Architecture

Command-query responsibility segregation (CQRS) is a pattern in system architecture inspired by CQS. It divides the system in two distinct parts, separating the components used for writing (executing commands) from those for querying. In such a system we can find two kinds of requests corresponding to these two models.

AC stands for Autonomous Component

Firstly, there are commands – ordering the system do something or change its state. A piece of business logic updates the domain model (or rejects the command) and lets the client know that the change has been accepted (or not).

The business logic is often implemented using domain-driven design, but it may just as well be transaction script or any other applicable technique. Actually, we ended up using a mix of these two in one area of the system.

Whenever anything changes, the domain model somehow notifies everyone – for example by publishing domain events. These events are received by view models, which update their own representation of the state, separate from the domain model.

This leads us to the second kind of requests: queries, getting information from the system without changing its state. These only use the view models, not the domain.

What is especially powerful about this pattern is the separation of concerns.

The domain (write) side is all about the business. It does not care about queries, all the different ways to show this information to the users, performance, optimal storage for these purposes and all that.

On the other hand, the query (read) side is all about the read access. Its main purpose is making queries fast and convenient.

While the domain is only implemented once, there can be multiple view models over the same data. They often use different databases. They can even be using different technologies – all kinds of NoSQL, normalized and denormalized SQL, in-memory representations, Lucene indexes, OLAP cubes etc.

The read models are also free to use different languages, if they make anything easier. There are no obstacles to implementing the domain model in Scala, but doing view models in Clojure or even SQL stored procedures.

Overall, the view models are very good (and safe) candidates for innovation.

Unlike the domain model, code quality in view models does not have to be perfect. They’re all about reshaping data and moving it around to make reads convenient. Some shortcuts and dirty tricks may be acceptable, as long as they don’t make the whole thing unmaintainable.

Read models are often denormalized, prepared to answer concrete questions in optimal way. It can even be as extreme as having a precomputed set answers to every query that the system will handle, stored in some trivial key-value way.

We often call the view models “projections” – because they “project” the domain events to a particular model, keeping only as much information as is necessary and in optimal shape for queries they serve.

Note that all the domain logic is only implemented in the domain (write) model. It’s done once and in the right place. Even if a value is directly derived (calculated), as soon as it has a business meaning or is calculated with some business logic, it belongs in the domain model.

Event Sourcing

Another pattern commonly used with CQRS is event sourcing.

In short, it means that all data in the system is stored in the form of events, in an event log. An event is a piece of information stating that something has occurred (user created, name changed, shipping address added, order submitted, order delivered). They’re always in past tense, saying that something has happened.

The events never change. You can never delete or update them. If something has happened, it’s happened. If it was a mistake, it can be corrected with a complementary action generating a new “inverse” event, but there’s no going back and saying it has not happened.

Combining ES and CQRS

Event sourcing and command-query responsibility segregation perfectly fit each other. It’s a powerful synergy effect: each of them becomes more powerful thanks to the combination.

When a command comes in, the domain model calculates the new state of the system and possibly emits some new events (which are the only way the changes are persisted). When another command comes in for the same logical area, the domain model is restored from the past events, and responds to the new command by generating some new events. The events represent concrete changes that have business meanings. Technically in a sense they are “deltas” (or “diffs”) of the system state.

The view models simply handle these events, picking up only these they are interested in, and updating their state to support future queries.

Benefits

There are many benefits to using this approach, let me just point out a few of them.

CQRS naturally leads to task-based UIs. Every action represents some very concrete business event. There can be a huge difference between changing someone’s name, changing their shipping address, or making them a gold customer. If end user wants to change name, they get a small form that has exactly what is needed for that operation. If they make a customer gold – that’s another action in another form.

Contrast this with the traditional CRUD, spreadsheet-like systems. They have no such operations as “change name” or “change customer status to gold”. All the users can do is “change user”. Implementing logic that changes something when a particular field changes all of a sudden becomes harder. Validation is a nightmare. Auditing and simply seeing when something happened and what the users did, is impossible without adding more layers of complexity on top.

It is harder for the users, too – the use cases have to be implemented in their minds, knowing what to change when to achieve a desired effect.

Related to the above, and a reason why CQRS is often used with domain-driven design, is that the shape of the system naturally maps to business contexts and use cases. Commands correspond to concrete user intents, and queries are designed to answer concrete questions. Once again, it’s the exact opposite of a glorified spreadsheet-like DB frontend.

It’s applicable on higher level, too: Different areas of the business (bounded contexts in the DDD lingo) can be implemented as separate models. For example, a warehousing context can represent a book in a completely different way than a sales context. One may be interested in its size, weight and number on stock. The other – in author, genre, cover image, publisher description etc.

Event sourcing also makes reporting a lot easier. By definition it never loses information. Maybe yesterday you did not need to know how often users add and remove items from the shopping cart, and only cared about the orders they eventually submitted. But today business wants to trace this information, so maybe they can discover items that clients wanted but changed their mind. It may be worthwhile to tempt them with these items in future ads.

Answering such a change in a “traditional” system would be a nontrivial effort. With ES+CQRS chances are that all it will take is just another straightforward projection of data that is already there, and immediately answer questions about the past!

Finally, another obvious benefit is performance. Since the view models are separate, they can have very different schema. Avoid joins, keep data denormalized, answer many questions in linear or even constant time. Read-only access is much easier to scale, as is the write side which doesn’t care about queries anymore.

Costs

ES+CQRS is not without cost, and is not the best approach to all systems. More about this in a future post.

More Resources

Like I said in the beginning, these ideas have been around for quite a few years. There is a huge number of resources available online, in books and at conferences. This post has merely scratched the surface and is only meant as (yet another) humble introduction to the topic.

Here’s a few links to the masters:

Learn TypeScript Now, Prepare for Angular 2.0

Like most organizations building client-side web applications (SPAs), here at Oasis Digital we have most commonly written code in plain old JavaScript, i.e. ES5, with help from linters, unit tests, etc. Lots and lots of code is out there, written in ES5 plus AngularJS and other frameworks.

There are obvious weaknesses to this approach, or rather gains that are set aside by not embracing any of the numerous newer innovations. We have dallied with CoffeeScript, worked on minor projects with ClojureScript, and so on. There are a lot of great languages out there beyond the JavaScript, but statistically JavaScript/ES5 is the winner.

(For a bit of “inside baseball”: we at Oasis Digital are generally quite willing to use unpopular, better tools. So why ES5? Because of our secondary line of business, training. When trying to train someone to use a new JavaScript library or framework, the most likely common ground we will have with them is baseline ES5.)

However, the era of mostly writing JavaScript code as it will execute in the browser is ending rapidly. Innovation of the JavaScript language is running far ahead of what browser vendors will keep up with; and even to the extent browser vendors keep up, large projects will inevitably have to support today’s browser versions (with the JavaScript version embedded therein) for years to come. A build process with compilation to JavaScript will be part of most SPA projects in the future.

So the question is, which post-JavaScript language to pick up? My personal favorite is ClojureScript, though I also see much merit in type-inference-equipped functional languages like Elm. For mass use though, I suspect the answer is simply a JavaScript successor. This is where it gets tricky. ES2015 (wow, I miss the shorter “ES6”) is the standard, and it will eventually land in all the major browsers. It will “win” in terms of mass adoption. Today is a good day to start moving your server side (NodeJS) code, test code, and as much as possible all of your JavaScript development to ES2015 or even a bit beyond – we’ve already done so. Babel works very well. Traceur works very well.

typescript2But there is at least one major exception. Angular 2.0 (which is coming soon, who knows how soon?) is built in TypeScript. You can go look right now, the source code in progress is sitting there in GitHub, and it consists mostly of TypeScript files. Most of the examples out there (including those we have published) use TypeScript. It will certainly be possible to write Angular 2 applications using plain JavaScript or any other language that compiles to JavaScript, but the first class citizen will be TypeScript.

Therefore, if you are using Angular today, and want to upgrade to Angular 2 someday, start learning TypeScript. Start using TypeScript. Start using TypeScript in some of your AngularJS 1.x work (we do).

Even aside from the obvious Angular motivation above, type systems in general (and TypeScript in particular) can be very useful. They enable a much more helpful editing/IDE experience. Numerous bugs can be caught at compile time. The incidence of runtime errors tends to be much lower. Unit testing can carry less of the load when using a typed language, so that whatever amount of effort you devote to unit testing, a greater share of it can go toward important functionality versus more trivial tests.

Here are resources to get started.

We will follow up later with examples of moving Angular 1.x code to TypeScript; we might even add a class offering to teach TypeScript for AngularJS developers. Stay tuned.

Rapid Prototyping

Here at Oasis Digital we work on many kinds of projects. We start new projects in a “green field”. We revive old projects where former developers are no longer around. We teach classes open to the public, we travel to our customers to teach classes privately. We attack entire projects. We supply brainpower to existing projects.

Among the most fun work: to rapidly create a working-prototype implementation of a new idea.

In projects like this, it is often important to deliver quickly. To make that possible, the right goals and constraints are important. I have listed some of the possible goals and constraints below. Some of these conflict with each other! This is not a checklist of things to always do, but rather a list of ideas to consider.

  • Reduce the scope of the working-prototype as far as feasible, but no further.
  • Choose a team small enough to minimize coordination, but maximum parallel productivity.
  • Choose a programming language and other tools we know best.
  • Choose a different language and/or toolset, more well-suited to creating maximum functionality with little code for the problem at hand – even if it is not among those where we have the most experience.
  • Support only the most important browser or two, omitting legacy browser support.
  • Implement a wide swath of features, but very shallowly.
  • Choose the most vital/informative parts of the intended software system, and implement those in a working prototype manner, leaving the less interesting parts as static mockups.
  • Start with a very plain visual appearance, to make it obvious it is “just” a prototype.
  • Start with a more polished visual appearance, perhaps using an off-the-shelf template, to show the level of polish that will be possible in a more complete project.
  • Even if the system may eventually be a complex, multi-tier widely distributed system, implement the prototype as client-side only system; no server-side data storage at all. Often enough data can be stored in “HTML 5 local storage” to make it clear whether the project is actually a good idea and should proceed past a mere prototype.
  • Implement the prototype using something that approximates (but simplifies) the intended long-term architecture, because the prototype may need to demonstrate that it is a suitable architecture.

The final appealing thing about rapid prototype projects: they are straightforward to propose and price, often with a simple statement of work and fixed/firm price.

Preparing for Angular 2.0

Here at Oasis Digital, we have been excited about the possibilities for Angular 2.0 ever since its (somewhat contentious) public debut at the Fall 2014 European Angular conference. ripThat was the conference with the infamous “tombstone” slides, and the frequently misunderstood explanation by the Angular team members that they did not have a migration strategy for Angular 1-to-2 yet. Many commenters at the time seemed to misunderstand the yet part as some assertion that there would be no migration strategy at all. Fortunately, the agitation around that initial announcement has subdued, work has continued, progress is being made.

Though there has been quite a lot of progress on Angular 2, which we have been following (much of the development is being conducted in the open) our recommendation now is that if you don’t have a reason to work with A2 well before its release, you’re probably better off ignoring it a bit longer. As I write this in August 2015, the Angular 2 alpha versions are still rather rough, particularly in the experience of getting started.

But at Oasis Digital we do have a compelling reason to start using new versions before they are ready. We must do this so that we have significant experience when the time comes to start teaching our students (for our Angular class) how to use the new version, and even for the teaching we already include in the class about how to prepare for Angular 2.

One of our primary trainers, Paul Spears, has been leading the charge. Here are links to a couple of repositories he has published, showing a minimal project to get up and running with Angular 2, loaded using JSPM:

There is quite a lot to explain about what is going on in those repositories, here are a few of the most important bits.

We expect that many teams (perhaps most) will concurrently upgrade their development/build tools to the current generation of ES6/2015 module-centric build systems. The leader in the space appears to be JSPM, so we have chosen that for this work. It is also possible (and we will likely publish example this way in the future) to simply load Angular 2 along with its required dependencies using the script tags from downloaded files or from a CDN, of course.

The JSPM configuration may look quite alien compared to the familiarities of Grunt or Gulp. This is because of the above-mentioned module loader approach, and because JSPM already understands many things that must be explained to Grunt or Gulp by the user or a plugin.

These examples are set up for development, loading numerous separate tiny libraries using numerous HTTP requests; but JSPM and similar tools also have an excellent solution for production asset preparation, using essentially the same configuration information to package all of the JavaScript into a file or two.

These repositories show working examples as of right now, early August 2015. Angular 2 has been changing quite substantially from one alpha to the next. Moving up to the next version will likely require adjustments, as it has in previous versions. This is definitely alpha software.

The config.js file looks quite scary, but much of the contents there are generated by JSPM tooling. With some adjustment, it is possible to omit the hairiest block of information therein from source control, and instead treat it as a build artifact. As you look at the contents of that file, remember that you will not have to write the bulk of it.

 

The Cobbler’s Kids

The old saying about the cobbler’s kids that have no shoes is as true in concept now as it was a millenia ago. At Oasis Digital, where we find great solutions to increase the success of our customers, we have been so focused on those projects that we find ourselves lacking our marketing shoes. Our marketing efforts have been tertiary at best.

 

A few years ago we realized the need to develop our brand, a few months ago we started moving forward, and a few weeks ago we sat down and started the process with our team. I asked them to articulate what we are really good at, what they were proud of. The list was broad, and spoke to their understanding of the value they bring to our clients. Here is the list:

  • We don’t just build what the customer describes, we address needs in a well-thought out manner. We creatively solve problems, not just execute.Fists-shoes-216x300
  • What we build can last (longevity). and scale
  • Our teams communicate and collaborate
  • Challenges/problems are shared in the team
  • Our tech stack stays fresh
  • We teach about what we use
  • High quality, comprehensive design documents that hit the mark the first time.
  • Continuous learning/continuous improvement professionally
  • We can bridge the gap between techies and lay people
  • Our team can communicate effectively
  • Always look to improve, solve the problem with less code
  • We push for regular delivery, to finish items, not have long-standing open work items

This is the first bit, what we do well. The further question is how do we do these things. Very often, how you do things communicates your value to your customers. For example, people find our training very valuable because it is intensely workshop based, and taught by an actual practitioner of the craft, not a professional teacher.

  • We build high quality software
  • We empower our customers to prosper
  • We leave customers and projects better than we found them

We are a values based organization, so the “what” and the “how” are not the full answer. When you answer the “why” you find the values that motivate your work. The answer I came to was two-fold:

  • We love to solve problems
  • We love to help people

The wording will change over time but our core motivator is solving hard problems that have value. We really enjoy that process. Adding value, helping people and businesses to be successful, those are solid values that we believe will resonate with the right kinds of customers.

I love the powerful words pushed forward by the team. They are full of meaning and demonstrate the consistency through the team of our values.

love, empower, build, communicate, help, teach, improve, finish, learning, effective, collaborate

I cannot wait to have these ideas permeate our web presence and our marketing communications. If we can push this to conclusion I believe we will finally have a nice pair of shoes!

Task Based User Interfaces

Here at Oasis Digital, the bulk of our work is on complex business data systems. checkThese projects sometimes involve a so-called task-based user interface. Briefly, this is an interface where the operations available to the user are presented in terms of the problem domain, rather than in terms of editing data. Other names for this idea include “task-focused user interface” and “inductive user interface”.

Origins

I was unable to determine the specific research/practice origins of task-based UI, but it is most commonly seen as arriving from three main directions. First, the inductive user interface guidelines published by Microsoft around 2001 lay out a detailed description and motivation. Second, the domain driven design (DDD) and command query responsibility separation (CQRS) communities often described task-based UI as suitable for use in front of the system built with those principles. Third, simply looking at mass-market software there are numerous examples.

CRUD

Task-based UI is the opposite of CRUD UI, a convenient foil. Many software systems have numerous features which are readily described as CRUD: create, read, update, delete. Some developers describe their work as primarily creating CRUD systems, with either pride or disdain. When analyzing a proposed feature or proposed software project, we sometimes evaluate the extent to which the project is “just CRUD” versus behavioral functionality. Brought to the user interface, CRUD typically means screen after screen in which users can make an entity of some kind, find an existing one, update one, or remove one. In other words, a minimal veneer over basic database operations.

Simple CRUD user interfaces have the advantage that they are generally easy to create, and amenable to common tooling. Many development environments offer wizards or libraries to ease such creation, and this dates back to at least early 1990s, and probably much farther than that.

Therefore, CRUD has a lot of appeal when first creating a new application. Unfortunately, a system built primarily around CRUD tends to make behavior beyond minor validation unduly difficult to implement. If you find yourself squeezing business logic into an “on change” or “on click” event handler somewhere, you are probably experiencing this difficulty. As systems grow in complexity this bad fit can consume vast amounts of time and money.

There is a way out, and that is a task-based user interface, one which presents the user with options and actions which fit the problem domain of the software, rather than those which appear to manipulate data in trivial ways.

Change of Address Example

A common example to explain task-based UI is a change of address operation in a system which has subscribers, members, or some other class of people to whom things might be mailed etc. The CRUD approach to change of address is simple: the user searches for the appropriate user record, views that record, moves the cursor to the address fields, types over those values, and clicks “save”. Very convenient, very easy to program.

Unfortunately, down the line if there is nontrivial behavior in the system, it can be extremely difficult to determine what such an edit means. Was the address corrected because it had been mis-entered in the past? Was the ZIP Code changed because the post office changed the ZIP code assigned to that physical location? Or does this edit represent the person moving to a different place of residence? If the latter, how might that affect them in terms of the problem domain?

To avoid this, a task-based user interface would not only present address fields which can be edited. Rather, the user would see information about a person (including their address) then have options (buttons or whatever else is suitable in the visual UI guidelines at hand) with names like “correct mis-entered address”, “subscriber moved to another residence”, etc. Such explicit actions make it possible to capture the underlying domain-level meaning, rather than just edits to data fields. A downstream system receiving this information could then respond appropriately.

Choose Your Words Wisely

Task-based user interface is therefore to a considerable extent about words: the words which describe actions the user can take. If a system merely uses a task-based user interface on top of a CRUD-centric back-end implementation, then the words can at least be chosen well in the user interface. If the system uses a domain driven approach internally, then some of the well-chosen words in the user interface may become part of the ubiquitous language used throughout the implementation.

I encourage developers to consider a task-based approach even if the underlying system is not particularly domain driven.

Add, Edit, Delete, +, –

Speaking of words, having built numerous user interfaces, I very often still “default” to wire framing with button labels like “add” and “delete”. Such things are fine for an early initial wireframe, but task-based design principles suggest reconsidering whether a more specific set of operations is suitable.

An example came up at work today with a list of employees, and the initial proposal was a “delete” button next to each one. But one does not merely delete an employee: at even the most heartless of companies there will be quite a lot more process involved in removing an employee from a list of employees. A system which manages the employees then, should probably have some sort of process flow around that removal, and it should use words other than “delete”.

A Final Quip

If the user wanted Excel, they would use Excel. If they are using an application which is instead specific to a problem domain, then it should have features, both internally and at the user interface level, relevant to that problem domain.

Links

http://en.wikipedia.org/wiki/Task-focused_interface
http://msdn.microsoft.com/en-us/library/ms997506.aspx
http://codebetter.com/iancooper/2011/07/15/why-crud-might-be-what-they-want-but-may-not-be-what-they-need/
https://cqrs.wordpress.com/documents/task-based-ui/
http://stackoverflow.com/questions/12255874/what-is-an-example-of-a-task-based-ui
http://codebetter.com/gregyoung/2010/02/16/cqrs-task-based-uis-event-sourcing-agh/

Data Services for AngularJS Applications

I wrote recently about data/API services for complex “single page” JavaScript-heavy web applications. Everything there applies to AngularJS as well as other frameworks (Ember, Knockout, React, etc.), but there are some particulars to keep in mind for ease of interaction between an AngularJS-largeAngularJS web application and a data service. This topic is also asked about in almost every “Angular Boot Camp” class I teach, so by posting this I have an online resource to point at.

Use RESTful URLs and HTTP Methods

By following RESTful patterns rather than inventing ad-hoc patterns for a server API shape, your code will fit in more easily with the rest of the software industry. This reduces the friction in bringing on more developers, in integrating your application with others, and so on. A RESTful server API will operate easily with AngularJS’s built-in mechanisms for working with a server, whether you use plain $http, $resource, or Restangular. For example, all of those already assume that the HTTP status codes indicate success and failure, so do not make up your own way of encoding failure within HTTP “200” success.

Avoid dogma in REST design, though. Most of the time, you can make a server API shape reasonably RESTful with approximately the same implementation effort as something ad hoc. If you have a case where a RESTful design will require substantially more work than something else, consider an “escape hatch”: a portion of your API that isn’t particularly RESTful, hidden behind a generic POSTable endpoint.

Make Data Binding Easy

I assume that by choosing AngularJS, developers aim to leverage its strengths rather than fight it. That means embracing “HTML enhanced”, data binding, and other banner AngularJS features, rather than manually manipulating the DOM (which immediately fails code review here), mostly passing information around using bindings and $watch rather than events, etc.

Therefore, if possible make your data service return data in a shape already suitable for binding in your AngularJS application. Done well, this can reduce the amount of JavaScript code you must write to populate a complex screen to almost nothing. By complex, I mean a screen which contains the modern equivalent of numerous master detail relationships, hundreds of fields, etc.

Of course, JavaScript is a powerful and convenient language for data structure manipulation, so in a sense it does not matter what shape of data is returned from a server; it is always possible to process that data into a shape suitable for data binding. Hence this guideline is oriented toward ease and speed of development when the server code is being written fresh.

(As an aside, I am not 100% sold on data binding; there are other approaches, like that taken by React, which have significant advantages. But if you are using AngularJS, embrace its strengths. Use data binding pervasively.)

Make Commands / Saves Easy Also

Following along from the above, a conveniently and idiomatically crafted AngularJS application will typically yield JavaScript data structures suitable for PUT or POSTing back to a data backend. Make your server accept PUT or POST of updated or new data in the same object “shape” as it returned data. This way, the data can make a round-trip from the server, through data bindings, to be manipulated by a user, then back to the server; all with very little code to write.

This applies to commands (in the CQRS sense) also. If a user is working with the screen via data binding, then that data binding can often yield a data structure ready to send to a server with a couple of lines of code. If you find it necessary to write extensive lines of code to formulate a server interaction, perhaps either your data binding or server should be adjusted stat.

Consider NodeJS on the Server Side, for Code Sharing

Working with complex client-side web development, you will almost certainly use NodeJS in your development toolchain, even if you are not using NodeJS for your data services.

But NodeJS is worthy of consideration for this key reason: it enables code sharing between the front end and back end. With some care in crafting modules, chunks of functionality which must be implemented on both sides of the client/server API (such as validation, but also many kinds of domain logic) can be reused across both, ensuring consistent implementation and eliminating duplication.

There are numerous other platforms worthy of consideration for other reasons. Stay tuned here (and follow me on twitter), I will have more to write about that.

Guard Your Innovation Budget

If your organization is in the process of adopting AngularJS for the first time, consider not simultaneously adopting a new server-side technology. As humans we each have a certain ability to absorb change, and doubly so for organizations comprised of many humans working together. You will probably get better results if you switch to a new client-side development toolkit in a different year then you switch to a new server side development toolkit. I think of this as having an “innovation budget” which can either be concentrated on one layer at a time, or diluted across more than one layer.

Help Wanted

If anyone knows of convenient source code examples already online of doing these things well versus poorly, please contact me – I would love to add links. Given sufficient time I may create the examples, but with the pace of change in our industry these tools could be obsolete before that happens!

 

Data/API Servers/Services for Single Page Web Applications

A Client needs a Server, or at least a Service

Over the last few years the team at Oasis Digital has created various complex “single page” web applications, using AngularJS, KnockoutJS, and other tools. These applications’ assets and code are be statically hosted (cheaply on  CDN if needed), but of course each application needs a backend data service (possibly comprised of smaller or even “micro” services internally) to provide data and carry the results of user operations to permanent storage.

Context and Background

Our work context is typically data- and rule-centric line-of-business applications, hosted in a company data center or on virtual/cloud or dedicated hardware, or occasionally a (more cloudy) PaaS like Heroku; so advice here is for that context. “Backend as a Service” solutions are appealing, but I don’t have any experience with those.

The systems we build most often store data in a traditional RDBMS like PostgreSQL or MS SQL Server, but data service needs are similar with a NoSQL or other non-RDBMS data store. (Among the many topics I should write about: making effective, justified use of polyglot persistence with CQRS and related ideas.)

We have also worked extensively with multi-tier desktop applications, which have essentially the same data service needs, albeit with different data serialization formats. As a result, we have worked on and thought about data services extensively.

Building a Data API Service

For convenient, rapid, efficient development of robust data/API services, your tool set should have as many as possible of the following:

  1. A server-side programming language / platform, with its runtime environment, native or VM.
  2. A way of routing requests (hopefully with a RESTful pattern matching approach).
  3. Automatic unmarshaling of incoming data into data structures native to the programming language. If you find yourself writing code that takes apart JSON “by hand”, run away.
  4. Similarly, automatic marshaling of ordinary data structures into JSON. If you see code which uses string concatenation to build JSON, it should either be to meet some specific needs for extra marshalling performance of a huge data structure, or shouldn’t be there at all.Database_icon_simple
  5. Analogous support for other data formats. Successful systems live a long time. If you create one, someone will want to talk to it in a situation where JSON isn’t a good fit. A few years from now, you might think of JSON the way we think of XML now. Therefore, stay away from tools which are too deeply married to JSON or any other single data format.
  6. If you’re using a relational database, your data server toolkit should make it quite easy to talk to that database. Depending on the complexity of the relationship between your data services and the underlying data store, either an object relational mapper (ORM) or perhaps a table/query mapper is suitable. If you find yourself working with the low-level database API to pluck fields out, you are looking at a long investment with little payoff.
  7. Good support for a wide variety of database types (relational and otherwise). This reduces the risks from future database support requirements.
  8. A reasonable error handling system. Things will go wrong. When they do, an appropriate response should flow back to the client code, while a fully detailed explanation should land in a log or somewhere else suitable – ideally without re-inventing this on each project or for every API entry point.
  9. Depending on application needs, some way of maintaining a persistent connection (SSE, websocket, or fallback) to stream back changing information.
  10. A declarative way to specify security roles needed for subsets of your API (RESTful or otherwise).
  11. Monitoring / metrics.
  12. Scalability.
  13. Efficiency, so you are less likely to need to scale, and so that if you must scale, the cost isn’t awful.
  14. Rapid development supported by good tooling. Edit-compile-run cycles of a few seconds.
  15. A pony.

Is That All?

This is quite a checklist; but a toolset lacking these things means that a successful, growing project will probably need to reinvent many of them – shop carefully. In later posts, I’ll write more about particular technology stacks.

 

Taking CSS Seriously

At Oasis Digital, we have spent much of our effort over the last several years creating complex “single page” web applications. There is much to say about that, but today I’m writing about one specific sliver: styling the application pages with CSS. To do good work at scale when using CSS to style an application (versus a one-off “webpage”), the only path to success is to take CSS seriously as a language, and study how to use it in an idiomatic, semantic, maintainable way.

Unfortunately, thinking about CSS in this way is considered far into “advanced” CSS territory, and these topics received almost no attention from the countless online and off-line resources to learn CSS.

What does it mean to take CSS seriously? Briefly, it means learning the technology and patterns of use in depth, by studying what others have done and thinking about why and how to solve problems.

Idiomatic CSS

CSS written by experts tends to follow many idioms. These are patterns that have been found useful again and again: a consistent order of selectors, consistent use of white space and other formatting, and so on. There is a popularly cited online resource explaining some of the most common idioms, and A List Apart (which you should probably read extensively) has a busy topic area around CSS.

Robust CSS

css-is-awesomeDo you always use the same browser, with your window the same size, with the same settings? Then your CSS is probably not robust at all. It is quite easy to accidentally style in such a way that the slightest disruption in the layout causes unexpected unpleasant results.

The way to robustness is exposure to ongoing change. Use multiple browsers, and switch often. Adjust your font size every day up or down, for a good reason or no reason. Change the width and height of your browser window, don’t just always leave it maximized at whatever very common screen size you happen to use. Do you work on a PC? Try a Mac sometimes. You work on a Mac? Your CSS-styled pages probably change in some minor but irritating way if you haven’t looked on a PC.

Good CSS is robust. It is specified sufficiently but not over-specified. A particular anti-pattern to avoid: layout some elements; observe their width on your particular browser and settings; set the width of an element that is supposed to contain them to a hard-coded number based on that single observed width; then watch what happens when some minor difference in font or other matter makes something a little bit wider than you expected.

Appropriate CSS

A hammer is a great tool, but not every problem is a nail. Grid systems are wonderful, but not for every element on every page. To take CSS seriously, we must learn when and where to use grid systems, ad hoc floats, that shiny new flexbox, and even an old-fashioned table.

Layered CSS

Many projects today start with a CSS framework like Bootstrap or Foundation. Working in this context requires a keen awareness of how your CSS interacts with that provided by the framework. It is common to see CSS which randomly and arbitrarily overrides rules set by such a framework, with no regard for what sense those framework provided rules made. The typical result: layer upon layer of overrides, trying to fix a resulting problem by adding layer upon layer of of more overrides.

The key to winning this battle of layers is to stop fighting. To use the delete key extensively, trimming away excess CSS to fix problems rather than adding more. Adjust variables provided by your framework, instead of ad hoc overwriting its calculated class attributes. Understand exactly what layers are needed and why.

Semantic CSS

“Semantic” means “pertaining to the meanings of words”. But the problem many of us have around semantics isn’t so much of wrong meaning, but of applying the wrong names to the right meaning.

It seems everyone is taught, on their CSS journey, about the evil of the in-line style attribute. As a result, we see in the wild many CSS classes like this:

.float-right { float: right; }

Thus allowing the CSS user to type class=”float-right” instead of style=”float: right;”.

This of course misses the point of the admonition against in-line style completely. It is not the literal use of the style attribute that is the problem, it is that the idea behind CSS is to mark up content with classes which say something about what the content is, then use CSS separately to apply a visual layout and so on.

If you can look at the name of a CSS class and know exactly the style settings it contains, it is probably not a very good name. But if you can look at a piece of content, and tell what CSS class you should apply based on what kind of content it is, you probably have a very good name.

Did I mention, don’t actually write CSS itself?

CSS is lacking critical facilities for abstraction and removal of duplication. There are persuasive arguments floating around that CSS is too broken to ever be fixed; and I would not argue against that. But given that we are mostly stuck with CSS for many types of applications, it is necessary to use a layer on top of CSS. The usual contenders are Sass and LESS. This means that all of these goals mostly apply to the Sass or LESS you write, rather than to the CSS generated from Sass or LESS.

People who have studied both usually conclude that Sass is a better choice than LESS, and indeed the former is what we most often use.

(A helpful tip: if you are using a CSS framework like Bootstrap or Foundation, use a version of those which is supplied using whichever language about CSS you have chose. For example, if you use Sass, use the Bootstrap Sass version – don’t use the normal Bootstrap LESS or the bare Bootstrap CSS files!)

Maintainable CSS

The net result of all of the above is maintainable CSS. CSS which you can keep adding to, in an incremental way, as the application you are styling continues to grow and improve. CSS which you or someone else can understand a year later. CSS which you can be proud of when someone looks under the hood.

Balanced CSS

CSS is not ideal; it has gradually accumulated features since the emergence of the web and therefore has legacy concerns, among other problems. The CSS to solve any particular problem might not be able to meet all of these design qualities at the same time. Therefore, good quality CSS strikes a balance: semantic enough, robust enough, idiomatic enough, and so on.

Rapidly Written CSS

Some of the ideas here sound like they could take a long time to code – but good CSS is faster to complete – bad CSS burns up far more hours than good, in the quest to create a finished system.

Of course, mastering tools can take a long time. The payoff is that a master usually creates code (CSS or otherwise) with high “first time quality”.

</rant>

At Oasis Digital, we are not perfect. We do not yet do all of these things perfectly all the time; there is sometimes a tradeoff between delivery speed and polish. But we do take CSS seriously, and we work hard to move in the right direction along all these axes.