Software Product Quality at Oasis Digital

A long-studied topic

Decades ago, business guru Philip Crosby famously defined quality as “conformance to requirements”. This definition seems useful in software development only to the extent every aspect of the software has been comprehensively understood and written down – rarely the case in real projects.

Fewer decades ago, software and consulting guru Gerald Weinberg slightly less famously wrote that “Quality is value to some person” – an insight more applicable to our context here at Oasis Digital, consultants and developers of custom software products. (Incidentally, to gain dense insight into software development and other topics in well-written tidy packages, read Weinberg’s books.)

Still, to point out that quality is whatever someone (typically a paying customer) says it is, doesn’t help all that much with a problem we face regularly.

I want a high quality software product, but what does that mean?

At Oasis Digital, customers often come to us with a vision, or partial written requirements, for a software product or system. Around this essential kernel, there are numerous possibly-implied desires or requirements, many related to quality.

What should an organization (or person) want, if they generally want a good, high quality product? We think of the answer to this as the implied requirements for high-quality software.Recently the team here started gathering a list of quality attributes (special thanks to Paul Spears for kicking this off on a whiteboard for all to see). Here is our checklist of desirable attributes, requirements, meta-requirements, and other aspects of quality or otherwise “good” software.

Quality checklist

The software we work on most often has both server-side and web/tablet/mobile UI, so our checklist contains a somewhat broad mix of topic areas.

Functionality

  1. The software works “on the happy path”; it has all the specified desired features.
  2. The software handles numerous potential error conditions well; it fails gently, and visibly. It recovers, or fails clearly if it can’t.
  3. The software implements a workflow at least as friendly to users as envisioned; ideally even more so.
  4. The software augments, rather than consumes, human mental bandwidth while using it.
  5. The features are generally composable where appropriate. That is, when a pair or set of features are more valuable when used together, they can be used together and work as expected.
  6. The software conforms to legal or regulatory requirements to which the subject; achieving this often requires cooperation among developers, customer representatives, and sometimes experts in compliance. In some projects this may be a minor aspect, while in others it is a primary defining motivator.

Support and operations

  1. The software captures logs of events that go wrong (and generally also of things that go right); it does so in a manner suitable for aggregation and analysis, with generally well considered log levels, a machine-readable log format, and so on.
  2. The software has features suitable to help with support efforts; for example it shows what version of the software is in use, helpful error data is exposed (in log as well as, where possible, on screen) rather than discarded, etc.
  3. The software is operations-friendly. It has switches, features, or other attributes helpful for operations teams responsible for keeping the software working.
  4. The software does not forget facts to which it has been exposed; where technically feasible, it has an append-only, log-structured view of the world. This both supports debugging efforts and future not yet known requirements.

Appearance and behavior (UI/UX)

  1. The user interface conforms to some design system, its layout and appearance are not completely subjective and ad hoc, but follow some understood and well considered guidelines for appearance, layout, etc.
  2. Unless the software is either very large (with a large budget), or is an art piece (such as a game), it does not “go its own way” with an ad hoc design.
  3. The user interface is aesthetically pleasing, in the subjective sense. (To whom?)
  4. The user interface further has suitable animations or dynamic style behavior, suitable for its design system.
  5. The user interface is themeable; at least its colors, and possibly other aspects, can be adjusted to fit in coherently with other software that uses some defined color palette. The user interface code should therefore use appropriate color theme variables or similar mechanism, not be hardcoded to match a design system or ad hoc requests.
  6. The user interface is responsive; it makes reasonably good use of a wide range of screen sizes. It is not a fixed size for a single screen size, unless its target (embedded) deployment environment is similarly strictly limited.
  7. The user interface does not suffer the “keyhole problem”; when presenting the user with a significant amount of data, it makes good use of the display to show the user many options and useful context.  http://www.aristeia.com/TKP/draftPaper.pdf
  8. To the extent of the user interface presents data in tabular form, the tables presents numeric and text content with suitable alignment.
  9. The user interface features the variable contents (data) and more prominently than fixed labeling; a well-chosen design system generally will achieve this goal out-of-the-box.

Operating / human environment

(As of this writing, most of the software we work on has a web user interface, and that shows in this checklist.)

  1. The software supports all current browsers, and possibly (depending on target deployment environment) one or more obsolete browsers as needed.
  2. The software has good accessibility characteristics, including testing with a screen reader or similar assistive technology.
  3. The user interface visually scales well in response to user font size overrides; it does not attempt to block the user from changing the font size, and its layout remains usable across a range of font sizes.
  4. The user interface contrast levels (as supported by the design system) are high enough to pass accessibility testing.
  5. Color is used effectively to maximize the speed of comprehension; but no information is ever presented only in the form of color, so that the software remains workable for users who don’t perceive color fully.
  6. The software is reasonably compatible with its platforms internationalization capabilities; and if needed, has been (or can be) suitably localized.

Performance and throughput

  1. The software has been tested, and works acceptably, with a realistic date volume. It is often necessary to obtain or generate test data of configurable size to verify this need has been met.
  2. Performance characteristics in error handling characteristics have been considered jointly; so that an occasional error does not completely halt the throughput capabilities of the software. It is possible to move past or set aside a failure case and continue meeting throughput expectations in the case of occasional error.

Security

  1. The software is built on a platform or framework choices which have reasonably well considered security characteristics; the software cooperates with this platform in such a way to generally inherit those characteristics.

(Security could fill books, not one section of a single blog post. For a software product applied primarily to an internal, benign audience, the above is probably sufficient; but for software deployed to the open Internet, or in other cases where hostile actors are expected, appropriate much more substantial security design and implementation is needed.)

Development Process

  1. Intentionally chosen, considered process appropriate for the project
  2. Regular demos or other progress presentations to stakeholders
  3. Regular code review, before (not only after) code enters the mainline of development
  4. (of course many books could be and have been written on development process!)

Internal characteristics

It’s possible to write software which externally does everything it is required to do, but internally is a shambles. Some thinkers imagine that this is the timely and inexpensive way to create software. We have not found that to be the case. Rather, to achieve external quality without overwhelming cost, internal quality is vital. We strive to create reasonably good internal quality without being explicitly asked to do so. Internal quality characteristics often include:

  1. Consistent code style, applied automatically
  2. Linting, applied automatically
  3. Internal and cross project code reuse – general avoidance of duplication
  4. Architectural consistency across portions of a system
  5. Consistent use of suitable platform features; don’t reinvent the wheel, don’t blindly apply techniques from one platform to another

Making sense of Quality for a customer project

This list is long (and could grow much longer). Achieving these things may consume substantial time and effort. At the same time, software projects often arrive at our door already under schedule pressure. To manage this conundrum, we work with customers to consider this list as a default; a list of things that probably should be done, but which a customer might choose to skip some items for schedule or budget needs. For each aspect of quality, a certain amount of minimum attention is needed (and automatically applied by a high-quality software team), but beyond that there is a range of possibility subject to customer priority.