Ditto

Ditto

Status: Stable but dormant. The was last pushed in September 2024 and sees little activity today; the library is published on npm as .

When dealing with data integration, translating JSON from external formats into an internal representation is a constant chore: every new data source arrives with its own field names and nesting. Ditto solves this declaratively. Instead of writing a bespoke transformation script per source, you write a mapping file that describes the output shape, and Ditto produces JSON from the input data to match it.

Ditto parses the mapping file (see ) and runs three stages (_preMap, _map, _postMap), where the output of each stage is fed as input to the next:

How to use Ditto

Ditto exposes a class that can be instantiated with a mapping file and an optional plugins list. Call unify with the document you want transformed:

const Ditto = require('json-ditto');
// This is valid JSON mapping file that maps to the mapping rules belowconst myCustomMappings = require('./myMappingFile');
// Create a new mapper that will always use the "myCustomMappings" fileconst myCustomMapper = new Ditto(myCustomMappings);
// Call the unify function that will read the "documentToBeUnified" and transforms it via the mapping filemyCustomMapper.unify(documentToBeUnified).then((result) => {    .....});

Alternatively, create a default instance and pass both the document and the mappings to unify itself (its signature is unify(document, mappings)):

const Ditto = require('json-ditto');
const myCustomMappings = require('./myMappingFile');
// Pass the document and the mapping file to unify directlynew Ditto().unify(documentToBeUnified, myCustomMappings).then((result) => {    .....});

Built-in plugins are available out of the box. Extra plugins can be registered either through the addPlugins(plugins) method or by passing them to the constructor directly via new Ditto(myCustomMappings, myCustomPlugins). Added plugins extend, and never overwrite, the default set.

Plugins: transformations in the mapping file

The heart of Ditto's expressiveness is plugins: plain JavaScript functions invoked from the mapping file with the @ symbol. A plugin is nothing more than a function that takes 0+ arguments:

'use strict';
const _ = require('lodash');
module.exports = function aggregateExperience(experience) {
    let totalExperience = 0;
    _.each(experience.values, function(experience){        if (!!experience.experienceDuration) totalExperience += experience.experienceDuration    })
    return totalExperience === 0 ? null : totalExperience;};

Arguments in the mapping file are separated by | and can be paths into the input document, hard-coded values, or defaults:

• @concatName(firstName|lastName): call concatName with the values extracted from the firstName and lastName paths in the input file
• @concatName(firstName|>>this_is_my_last_name): the >> prefix passes a hard-coded string instead of a path lookup
• @concatName(firstName|lastName|*>>default): the * prefix marks a default value, used when the other lookups come up empty

If a plugin might receive a variable number of arguments, the JavaScript arguments keyword handles it cleanly:

function concatName() {	return _.flatten(_.values(arguments)).join(' ').replace(/\s\s+/g,' ').trim();}

Functions in functions

A plugin's value can be fed into another plugin as a parameter by nesting the call inside the arguments. One catch: parameters of inner function calls are separated by a comma , rather than the usual |:

"type"   : "@getLinkType(value|@getLinkService(value,service))","value"  : "@cleanURI(value|@getLinkType(value,@getLinkService(value,service)))",

Plugins activation

Plugins are activated in the /ditto/plugins/plugins.js file by adding the plugin name (matching the .js filename of the definition) to the plugins array; each one is required and exported for the main mapping function:

'use strict';
module.exports = {	aggregateExperience              : require('./aggregateExperience'),	assignIds                        : require('./assignIds'),	assignUniqueIds                  : require('./assignUniqueIds')	....

The full list of built-in plugins lives in the , and the complete mapping-rule syntax is documented in the .