Loopback is a powerful Node.js API framework built on top of Express that comes with a lot of functionality in-the-box. Recently, I gave a talk about creating APIs with Loopback in the context of building Angular web apps. In that talk I created a vanilla Loopback API using the Loopback CLI and connected the resulting API to the Angular Tour of Heroes demo application. While the CLI allows for easy configuration of Loopback’s JSON files via simple command-line operations, there are times when you need to write code to expand the functionality of your API, especially when the backing storage is not an off-the-shelf database but a custom enterprise API. In cases like this, the best solution is often to write your own connector. Also, many developers would like to use TypeScript with Loopback. While Loopback 4 will use TypeScript by default, version 4 has not yet been released. Although Loopback 3 does not use TypeScript, any Loopback 3 project can be converted to use TypeScript today. In this article I will explain how to convert any Loopback 3 project to TypeScript and also how you can expand your API’s capabilities by creating your own connector.

Background and Motivation

TypeScript

By default, Loopback projects are configured with JSON files and coded in JavaScript by default. While motivating TypeScript over JavaScript is beyond the scope of this post, there are many persuasive arguments for using types as a first line of defense against bugs. As a first-order approximation, TypeScript is merely typed JavaScript, and TypeScript readily transpiles into JavaScript. Thus, as we will see, TypeScript can afford type safety in any Loopback project with minimal headache.

Architecture of Loopback

Briefly, Loopback represents groups of data abstractly as models that interact with backing storage via connectors. Incidentally, a configured instance of a connector is called a data source. In Loopback, a model represents the schema of one instance of a certain kind of data, and a connector enables any number of models to interact with backing storage. There are many connectors available, such as MongoDB, MySQL, and PostgreSQL, which are installed as NPM packages.

The idea behind this separation of concerns is that you can describe the shapes of – and relationships between – your data separately from describing how to retrieve or update that data. For instance, if one were building an API to represent a hospital, a “physician” model could be created that contains properties such as specialty, years practiced, and gender, and a separate “patient” model containing, for example, properties for a patient’s age, gender, address, and phone number. Then, each model could be connected to backing storage via a connector.

Furthermore, the connector does not have to be the same for each model: physicians could be stored on a MySQL database and patients stored on a MongoDB instance, for example. The relationship between each patient and a physician can then be handled fully inside Loopback by constructing what are called relations.

For more information about the architecture and typical usage of Loopback, see my Loopback talk.

Loopback Connectors

Since Loopback has many connectors available as npm packages for different kinds of storage, the model-connector architecture works very well when the backing storage is an off-the-shelf instance of, for example, Postgres, MongoDB, or even Elasticsearch. However, when your model must interact with a custom API, you are largely left with the following three options:

1. Use the Loopback REST connector
2. Write custom code directly inside the model
3. Create a custom connector

The first option only works if the API is sufficiently RESTful, and the second results in code that is not shared between models. Thus, the best way to enable your Loopback API to interact with non-RESTful APIs is often to write your own connector.

When Should TypeScript be Compiled?

When converting a JavaScript project to TypeScript, one can generally choose to either run the project in ts-node – the TypeScript version of Node – or compile the project with the TypeScript compiler and run the resulting JavaScript output with the standard version of Node. Although the ts-node option avoids the need to explicitly compile each time the source code is changed during development, it also implies using ts-node in production, which we generally avoid in favor of Node itself.

Thus, I will assume that our goal is to compile from TypeScript source rather than running the TypeScript project in-place. The end result will be a server directory containing TypeScript source files and JSON configuration files, and a build directory that contains the compiled JavaScript files along with the same JSON configuration files. To do this, we will use the TypeScript compiler and an npm CLI utility to copy Loopback’s configuration files to the build directory.

Unfortunately the Loopback CLI will not work on the TypeScript project. However, the CLI can still be useful by performing actions on a test Loopback project, checking how the changes affected the JSON configuration files, and performing the same changes on the TypeScript project by hand. We have found that, after using Loopback enough, it can be faster to perform actions, such as creating models, by hand rather than using the CLI.

Converting a Loopback Project to TypeScript

When creating a new project with the Loopback CLI, a JavaScript project is created by default. These steps assume the project is a fresh CLI-generated project, however the general approach applies to any Loopback project.

To convert a Loopback CLI-generated project to TypeScript, we can take the following steps:

1. Create a ‘build’ directory in the Loopback project’s root level for the output JS and JSON files
2. Run ‘npm i –save-dev typescript’ in the project to install TypeScript as a development dependency
3. Create a ‘tsconfig.json’ file in the root level with “outDir” set to “build/server” and “include” containing an entry “server/**/*.ts.” An example ‘tsconfig.json’ file:

{
"compilerOptions": {
    "target": "es6",
    "module": "commonjs",
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "outDir": "build/server",
    "sourceMap": true,
    "noImplicitAny": true
  },
"include": [
    "server/**/*.ts"
  ]
}

4. Run ‘npm i –save-dev @types/node’ to install the TypeScript types for Node.js
5. Rename all .js files to .ts and fill in types. As a tip, setting “module.exports = value” in TypeScript can be achieved with “export = value”.

export = function enableAuthentication(server: any) {
  // enable authentication
  server.enableAuth();
};

6. Run ‘npm i –save-dev copyfiles’ which installs an npm CLI utility that copies files
7. In ‘package.json’:

1. Edit “main” to point to “build/server/server.js”
2. Add a “compile” script that performs “tsc && copyfiles \”server/**/*.json\” build/server -u 1” to copy the JSON configuration files and preserve the directory structure

8. Run ‘npm start’ to start your API server!

{
  "name": "loopback_ts",
  "version": "1.0.0"
  "main": "build/server/server.js"
  "engines": {
    "node": ">=4"
  },
  "scripts": {
    "lint": "eslint .",
    "start": "node .",
    "compile": "tsc && copyfiles \"server/**/*.json\" build/server -u 1",
    "posttest": "npm run lint && nsp check"
  },
...

build
  server
    boot
    component-config.json
    config.json
    datasources.json
    middleware.development.json
    middleware.json
    model-config.json
    server.js
    server.js.map
node_modules
server
...

Now the project has been converted to TypeScript! Next steps include:

1. Configuring linting
2. Adding a “clean” script to clean the build directory using, for example, rimraf or rimraf-standalone for “rm -rf” cross-platform compatibility, and
3. Setting up a directory for the client application to live in

These are left as an exercise for the reader or a follow-up article.

Writing and Using a Custom Connector

Using an existing connector generally involves installing the connector with “npm install,” adding the connector as a data source (either by hand in the JSON or via the Loopback CLI), and using the data source with a model (again, either by hand or via the CLI). This works because Loopback looks for connectors in the node_modules directory, where npm packages are installed. Thus, there are generally two ways to incorporate a custom connector into a Loopback project: publish your connector with the prefix “loopback-connector-” in the name as a repository, for example on GitHub, and install it with “npm install,” or place the code inside your Loopback project and use a JavaScript hook to instantiate the connector as a datasource in code. Here we describe the latter option since in general we would not like to have to publish every custom connector that we write.

The following two code snippets show the boilerplate TypeScript code that is required to create a new, custom model and connect the connector to a model. In the connector code, when Loopback creates a new Data Source from a connector, it calls the connector’s exported “initialize” function by passing a Data Source object and a callback function. The initialize function creates a new instance of the connector and initializes pointers in the Data Source and connector objects to point to each other. The constructor of the connector initializes any properties of the Data Source that were passed as properties when the Data Source was created.

export class MyConnector {
  dataSource: any;
  propertyName: string;

  constructor(settings: any) {
    // Initialize properties here:
    this.propertyName = settings.properties.propertyName;
  }

  // Implement connector methods here (see Table 1)
}

export function initialize(dataSource: any, callback: Function) {
  const connector = new MyConnector(dataSource.settings);

  dataSource.connector = connector;
  connector.dataSource = dataSource;

  callback();
}

import * as loopback from 'loopback';

import * as MyConnector from 'path/to/the/connector';

const myDataSource = (loopback as any).createDataSource('dataSourceName', {
  connector: MyConnector,
  properties: {
    propertyName: 'Hello, World!'
  }
});

export = function (myModel: any) {

  // Connect model to data source
  myModel.attachTo(myDataSource);

};

Supporting PersistedModel in the Connector

After creating the boilerplate code, the logic of the connector must be implemented as methods in the connector’s class. Since the most common use case of Loopback models is the PersistedModel, which generally represents any model that is persisted in a backing data storage, we focus on using custom connectors with a model that declares PersistedModel as its base class.

As the Loopback documentation explains, the PersistedModel is the base class for most built-in models, and the vast majority of Loopback model use-cases rely on the PersistedModel as a base class. The PersistedModel provides standard create, read, update, and delete (CRUD) operations and exposes REST endpoints for them. Since we are creating a custom connector, the connector must implement methods that the PersistedModel’s CRUD operations use.

After the Data Source is attached to the PersistedModel, specific methods in the connector are called to create, retrieve, update, or delete data based on the source PersistedModel endpoint. Table 1 shows which connector methods are called for which PersistedModel endpoints. As we can see, only a few connector methods support a wide variety of endpoints.

Table 1: Connector methods that must be implemented to support the given PersistedModel endpoints

To implement the connector methods, the parameters of these methods must be discovered. This is left as an exercise for the reader, however an easy way is to declare several parameters, log them to the console, call the associated endpoint(s), and observe the console output. In general, data is passed first, followed by an authorization object and a callback function.

Disabling Remote Methods

Finally, if any PersistedModel endpoints are not needed, they can be disabled using “disableRemoteMethodByName” as shown in the code snippet below. This particular snippet disables all but the immutable endpoints of a PersistedModel. The only caveat to using this JavaScript hook is that any endpoints that are not static methods of the model’s class belong to the model’s prototype and must be declared as such with “prototype.methodName”.

export = function (myModel: any) {

  // Connect model to data source
  myModel.attachTo(myDataSource);

  // Disable mutable and unimplemented endpoints
  myModel.disableRemoteMethodByName('createChangeStream');
  myModel.disableRemoteMethodByName('upsert');
  myModel.disableRemoteMethodByName('updateAll');
  myModel.disableRemoteMethodByName('upsertWithWhere');
  myModel.disableRemoteMethodByName('create');
  myModel.disableRemoteMethodByName('replaceOrCreate');
  myModel.disableRemoteMethodByName('replaceById');
  myModel.disableRemoteMethodByName('deleteById');
  myModel.disableRemoteMethodByName('count');
  myModel.disableRemoteMethodByName('prototype.updateAttributes');

};

Conclusion

Although Loopback 4 will use TypeScript natively, it has not been released at the time of writing this article, and many would like to use TypeScript with Loopback 3 today. While Loopback 3 makes it very easy to create APIs based on off-the-shelf databases via JavaScript and connectors that are available as NPM packages, it is typically not clear how to convert a Loopback 3 project to TypeScript or create custom, unpublished connectors that will only be used within local projects. As we have seen, it is indeed possible to convert a Loopback CLI project to TypeScript, create a custom connector locally, and attach this connector to a PersistedModel. After copying short bits of boilerplate, only 6 methods have to be implemented to support 16 endpoints. Finally, since a connector can then be reused across many models, a sufficiently general connector can be reused within one project or across many projects.