Using External Language Tools

Many languages nowadays have existing code completion and linting/analysis libraries and tools available. Python has its Jedi, Go has GoCode, TypeScript has TSServer, and so on. Tools like these can be integrated into Cloud9 to add code completion support or error and warning markers.

📘

Building pure JavaScript analyzers instead?

Not all language plugins are created equal. Some are written fully in JavaScript. See Language Handlers for the details on the APIs for writing JavaScript-based analyzers that run fully in the browser. The present page is about tools not written in JavaScript and/or not running in the browser.

Background: Code Completion and Analysis in the Cloud

As Cloud9 is an online IDE, it has to deal with the challenge of getting the source code from the client to the server, before some analyzer tool that runs there can have a look at it. Cloud9 provides the worker_util.execAnalysis() API that takes care of this. This function efficiently transfers any unsaved changes of the currently open file to the server, runs an analysis tool, and then gets the results back.

To conserve bandwidth and improve performance, execAnalysis() doesn't send the whole unsaved document to the server, but only the latest changes. For this it uses the Operational Transformation protocol also used for real-time collaboration in Cloud9. It also makes sure that there are no race conditions when sending the request from the worker to the server, even though the user can continue to edit the document in the UI thread.

Adding a Linter/Analyzer

Linters and analyzers can produce a list of errors given a source file. For example, php has a built-in linter that can be invoked from the command-line:

$ php -l myfile.php

Parse error: parse error, expecting `','' or `';'' in - on line 1
Errors parsing -

Many linters can also operate on stdin instead of an actual file:

$ echo '<?php echo "hello" world; ?>' | php -l

Parse error: parse error, expecting `','' or `';'' in - on line 1
Errors parsing -

Cloud9 plugins can invoke a linter like this in the workspace to show errors in the editor, using the worker_util.execAnalysis() API:

JavaScript

var workerUtil = require("plugins/c9.ide.language/worker_util");

handler.analyze = function(docValue, ast, callback) {
    var markers = [];
    workerUtil.execAnalysis(
        "php",
        {
            mode: "stdin",
            args: ["-l"],
            maxCallInterval: 1200,
        },
        function(err, stdout, stderr) {
            if (err && err.code !== 255) return callback(err);
            
            // Parse each line of output and create marker objects
            (stdout + stderr).split("\n").forEach(function parseLine(line) {
                var match = line.match(/(.*?) in (?:.*?) on line (\d+)/);
                if (!match)
                    return;
                var message = match[1];
                var row = match[2];
                markers.push({
                    pos: { sl: parseInt(row, 10) - 1 },
                    message: message,
                    level: message.match(/error/) ? "error": "warning"
                });
            });
            
            callback(null, markers);
        }
    );
};

The execAnalysis() function will take the current file and any unsaved content and sends it to the workspace. The function makes sure that only the latest changes are sent over the connection to minimize latency.

📘

Using Temporary Files with Analysis Tools

worker_util.execAnalysis() uses stdin for passing the current file by default, since it may have unsaved content. Some tools don't like stdin very much and prefer a temporary file. Use the mode: "tempfile" option to pass a temporary file instead, invoking the analyzer tool as follows:

    "php",
    {
        mode: "tempfile",
        args: ["-l", "$FILE"],
    }

Adding a Code Completer

Code completion tools such as Jedi or GoCode provide code completion given an input source file and a cursor location. The worker_util.execAnalysis() function makes it possible to invoke such a tool from a language handler.

Below is an example implementation of base_handler.complete() using execAnalysis(). It calls a tool conveniently named analyzer-tool on the workspace and passes the current document over stdin along with the current row and column:

handler.complete = function(doc, ast, pos, options, callback) {
    workerUtil.execAnalysis(
        "analyzer-tool",
        {
            mode: "stdin",
            json: true,
            args: [
                "completions", "--row", pos.row, "--column", pos.column, "--path", path
            ]
        },
        function onResult(err, stdoutJSON, stderrJSON) {
            if (err) return callback(err);
            
            if (typeof stdoutJSON !== "object")
                return callback(new Error("Couldn't parse analyzer-tool output: " + stdoutJSON));

            callback(null, stdoutJSON);
        }
    );
};

👍

Use the Installer to Install Dependencies

You can use the Cloud9 installer to install third-party dependencies like linters, code completion tools, and analyzers.

Reducing Code Completer Latency

To reduce latency of a server-side completer, it can be beneficial to use the predictNextCompletion() API to predict completions early, and the noDoc feature to make sure documentation is only transferred to the client when needed. See Customizing Code Completers.

Reference Implementations

For a full example of a code completer and linter using the techniques above, see also the Python plugin in the Full Example Language Handlers page. In addition, the Go plugin also serves as a useful, more minimal reference.