Tutorial: development server and automation-builder

When you run the @wranggle/automation-builder server locally, it will serve your automations to the Wranggle browser extension, such that they show up and can be run from its popup menu. The latest version of your automation is remembered/cached by the extension for a duration you specify, so you only need to run the automation-builder server while your automations are under active development.

With @wranggle/automation-builder included as a devDependency, you write your automation as a normal node module, using javascript import/require statements as normal, editing and saving files in your own IDE as normal.

Project setup

Since the process of setting up a Wranggle automation is mostly identical to setting up a bundled javascript package, we'll start with a partially-written package to save time.

Exercise files

1. Download the tarball: wranggle-dev-server-tutorial.tar

2. Expand it: tar xf wranggle-dev-server-tutorial.tar and cd into the project

3. Open the directory in your favorite IDE or editor.

@wranggle/automation-builder

Make sure you have node.js v6.9.0 or higher installed:

  node --version

In the expanded exercise file directory, download project dependencies:

# eg:
  yarn install
# or:
  npm install 

Ensure that @wranggle/automation-builder is a devDependency and is up to date:

# eg:
  yarn add --dev @wranggle/automation-builder
# or with npm:
  npm install --save-dev @wranggle/automation-builder

To verify it's working, run from the command line:

  node_modules/.bin/wranggle version

Run the build/bundling system

You can use any build/bundling system with automation-builder. This example uses Webpack — the Kahba tutorial uses Browserify.

In a command line shell, start the build system, watching for changes:

# eg:
  yarn watch
# or:
  npm run watch 

Webpack will watch for changes and write out a bundle to the dist directory.

Localhost server

You configure your automation script in wranggle-automation.json. The config offers some options for the local development server. Let's take a look inside.

wranggle-automation.json

The name and description field values here will be visible to the user in the extension popup when the user is selecting an automation to run. The "code" is used as an id for the automation and must be unique to all of your automations.

You can set a cache duration here as well, telling the Wranggle browser extension how long you want to keep your automation available even when the dev server isn't running. To only offer this automation while the dev server is running, set it to 0. To keep it around longer, set a value in milliseconds. So for 2 days, we would update it to:
"cache": 172800000,. Note: currently the Wranggle browser extension caches the automation script itself only when the automation starts.

wranggle-automation.json also tells the dev server where it can find our bundled automation script, set in our project to dist/, corresponding to Webpack's output directory.

There are some other options but our example project won't touch on any.

Start automation-builder server

Our project's package.json includes a script shortcut "server" for running node_modules/.bin/wranggle server

In a command line shell, start the server with:

# eg:
  yarn server
# or:
  npm run server 

With no automation paths specified by the shell command, automation-builder will by default look in the current working directory, identifying automations by the presence of a wranggle-*.json file. Finding wranggle-automation.json in our example project, it knows to serve our bundled automation script to the Wranggle browser extension. We'll look inside this json file later.

Ensure your browser accepts the localhost certificate for https

Wranggle makes all of its API-related requests over https. Unless we tell the browser otherwise, it will deny Wranggle's background requests to localhost because it uses a self-signed certificate.

Accept the certificate in your browser. You should see a text/json response.

Runtime

Let's first run an automation and then look at the Wranggle-related APIs it uses.

Running an example

Our automation doesn't do anything yet. In src/dev-server-tutorial.js replace the entire file contents with:

import insertScreenshotThumbnail from './tutorial-support/insert-screenshot-thumbnail.js';
const ScreenshotSessionStoreKey = 'randomScreenshot';

class DevServerTutorial {
  constructor(runner) {
    const { kahba, prefab } = runner; // each of these has its own tutorial so we only touch on them here
    const { async, asyncjs, bluebird, jquery, $, lodash, _ } = runner; // some references and aliases to popular libs

    kahba.waterfall([
      runner.visit('https://en.wikipedia.org/wiki/Special:Random'),

      () => {
        prefab.reportText(`Navigated to ${ global.location.href }`);
        return runner.captureVisibleTab({ sessionStore: ScreenshotSessionStoreKey });
      },

      screenshotStats => prefab.reportText(`Screenshot: ${ JSON.stringify(screenshotStats, null, 2) }`),

      runner.visit('https://en.wikipedia.org/wiki/Screenshot'),

      () => runner.sessionStore.get(ScreenshotSessionStoreKey).then(dataUri => {
        prefab.reportText(`Inserting our thumbnail into the article`);
        insertScreenshotThumbnail(dataUri);
        prefab.reportText('DevServerTutorial finished!');
      }),

    ], err => {
      err && prefab.reportText(`DevServerTutorial hit an error, yikes: ${ err.userMessage || err.message || 'Unknown Error' }`);
      runner.stop();
    });
  }
}
global.WranggleAutomationScript = DevServerTutorial; // we declare our automation script's entry point here

Try it out

This automation navigates to Wikipedia's Special:Random page which issues a redirect to a random wiki page. Our automation then captures an image of the visible tab, visits the "Screenshot" wiki article and inserts a thumbnail that, when clicked, will open the full image.

If your watch script is still running, the source code will be bundled after you save.

If your server script is still running, this automation will be listed in the Wranggle extension's popup. Unless you changed its name in wranggle-automation.json, you'll see it listed as Tutorial - Dev Server.

Go ahead and run it: open the Wranggle popup in a new browser tab, find and select your automation, and then start it. Look for the inserted thumbnail in the content page near the top of the article. Clicking the thumbnail will open the full-sized image capture in a new tab.

Automation Runner

The runner object offers methods that communicate with the Wranggle browser extension (such as runner.captureVisibleTab), and it also carries references to other APIs, most importantly the Kahba API, which gives us the ability to write navigating automations using its multi-page control flow methods. We'll only touch on Kahba in this tutorial — it has its own in-depth tutorial.

The runner.prefab reference lets us create quick-and-dirty user interfaces for the browser extension's popup panel, convenient when you don't want to create your own custom UI for the popup. We'll only glance at the Prefab API here.

Although you can import/require any npm library into your automation script, the runner object exposes some popular ones, such as Lodash, jQuery, and some others that the browser extension happens to have handy.

Our example uses one of the automation runner's data stores: a session store for temporary data that's held only for the duration of the start-stop automation session. The runner also offers a long-term runner.durableStore to persist data that your automation needs in future sessions.

Calling runner.stop() stops the automation, similar to the user pressing the big Stop button in the Wranggle popup.

All imports/requires are handled by your bundler (eg Webpack, Browserify, Rollup, etc) at compile time. The Wranggle browser extension itself does not have its own import/require mechanism so we need to declare our entry point, set here on global.WranggleAutomationScript. It should point to either a function(runner) {} or a class containing constructor(runner) {}.

Additional Code Explanations

runner.captureVisibleTab delegates to the browser's captureVisibleTab method, creating an image as a data-uri. By passing in { sessionStore: ScreenshotSessionStoreKey } as an option, we instruct Wranggle to persist the result in the sessionStore for the duration of the automation's start/stop session. We use this image data after navigating to another wiki article, fetching it back from the session store and inserting it into the DOM.

The insertScreenshotThumbnail function (in the last step) is imported from a local project file: src/tutorial-support/insert-screenshot-thumbnail.js. It doesn't use any Wranggle-specific APIs but serves as an example of using imports/requires in an automation.

Although this approach to DOM modification, using Mustache templates and jQuery to append content, is easy to write, be careful about what you insert into the DOM if the information is sensitive. A malicious page could potentially look for and report such insertions. For tips and security notes on other approaches, see the Wranggle Guide on Working With The DOM.

Finished!

Remember to shut down the dev server and bundler.

Return to the list of tutorials.