{"__v":41,"_id":"551bd2c50d209f21006cad92","category":{"__v":9,"_id":"54d5635632d98b0d00384b01","pages":["54d5635732d98b0d00384b10","54d5635732d98b0d00384b11","55104b632dd9010d0019e144","55104c802dd9010d0019e146","551bd2c50d209f21006cad92","552e6fcd5d4baa0d00783691","552e73705d4baa0d00783699","567be431ee052b1700dbda49","567ef29af9ffa20d008782ff","56a8dd2470a9440d00ef5fe4"],"project":"54d53c7b23010a0d001aca0c","version":"54d5635532d98b0d00384afb","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2015-02-07T00:03:52.618Z","from_sync":false,"order":11,"slug":"language-tooling","title":"Language Tooling"},"parentDoc":null,"project":"54d53c7b23010a0d001aca0c","user":"54d5c32a7e05890d006f150d","version":{"__v":10,"_id":"54d5635532d98b0d00384afb","forked_from":"54d53c7c23010a0d001aca0f","project":"54d53c7b23010a0d001aca0c","createdAt":"2015-02-07T00:59:01.934Z","releaseDate":"2015-02-07T00:59:01.934Z","categories":["54d5635632d98b0d00384afc","54d5635632d98b0d00384afd","54d5635632d98b0d00384afe","54d5635632d98b0d00384aff","54d5635632d98b0d00384b00","54d5635632d98b0d00384b01","54d5635632d98b0d00384b02","54d652097e05890d006f153e","54dd1315ca1e5219007e9daa","54e21e2b22de1c230094b147","54e68e62a43fe13500db3879","54fa1d3fe7a0ba2f00306309","551c453a23a1ee190034d19a","551df586e52a0b23000c62b6","551f39be6886f8230055f02a","55a6720751457325000e4d97"],"is_deprecated":false,"is_hidden":false,"is_beta":true,"is_stable":true,"codename":"","version_clean":"0.1.0","version":"0.1"},"updates":[],"next":{"pages":[],"description":""},"createdAt":"2015-04-01T11:13:09.938Z","link_external":false,"link_url":"","githubsync":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":2,"body":"Many languages nowadays have existing code completion and linting/analysis libraries and tools available. Python has its [Jedi](https://github.com/davidhalter/jedi), Go has [GoCode](https://github.com/nsf/gocode/), TypeScript has TSServer, and so on. Tools like these can be integrated into Cloud9 to add code completion support or error and warning markers.\n[block:callout]\n{\n  \"type\": \"info\",\n  \"title\": \"Building pure JavaScript analyzers instead?\",\n  \"body\": \"Not all language plugins are created equal. Some are written fully in JavaScript. See [Language Handlers](doc:language-handlers) for the details on the APIs for writing JavaScript-based analyzers that run fully in the browser. The present page is about tools not written in JavaScript and/or not running in the browser.\"\n}\n[/block]\n# Background: Code Completion and Analysis in the Cloud\n\nAs Cloud9 is an online IDE, it has to deal with the challenge of getting the source code from the client to the server, before some analyzer tool that runs there can have a look at it. Cloud9 provides  the [worker_util.execAnalysis()](https://apidoc.c9.io/c9v3/#!/api/language.worker_util-method-execAnalysis) API that takes care of this. This function efficiently transfers any unsaved changes of the currently open file to the server, runs an analysis tool, and then gets the results back.\n\nTo conserve bandwidth and improve performance, `execAnalysis()` doesn't send the whole unsaved document to the server, but only the latest changes. For this it uses the [Operational Transformation](https://en.wikipedia.org/wiki/Operational_transformation) protocol also used for real-time collaboration in Cloud9. It also makes sure that there are no race conditions when sending the request from the worker to the server, even though the user can continue to edit the document in the UI thread.\n\n# Adding a Linter/Analyzer\n\nLinters and analyzers can produce a list of errors given a source file. For example, php has a built-in linter that can be invoked from the command-line:\n\n```\n$ php -l myfile.php\n\nParse error: parse error, expecting `','' or `';'' in - on line 1\nErrors parsing -\n```\n\nMany linters can also operate on `stdin` instead of an actual file:\n\n```\n$ echo '<?php echo \"hello\" world; ?>' | php -l\n\nParse error: parse error, expecting `','' or `';'' in - on line 1\nErrors parsing -\n```\n\nCloud9 plugins can invoke a linter like this in the workspace to show errors in the editor, using the  [worker_util.execAnalysis()](https://apidoc.c9.io/c9v3/#!/api/language.worker_util-method-execAnalysis) API:\n\n```javascript\nvar workerUtil = require(\"plugins/c9.ide.language/worker_util\");\n\nhandler.analyze = function(docValue, ast, callback) {\n    var markers = [];\n    workerUtil.execAnalysis(\n        \"php\",\n        {\n            mode: \"stdin\",\n            args: [\"-l\"],\n            maxCallInterval: 1200,\n        },\n        function(err, stdout, stderr) {\n            if (err && err.code !== 255) return callback(err);\n            \n            // Parse each line of output and create marker objects\n            (stdout + stderr).split(\"\\n\").forEach(function parseLine(line) {\n                var match = line.match(/(.*?) in (?:.*?) on line (\\d+)/);\n                if (!match)\n                    return;\n                var message = match[1];\n                var row = match[2];\n                markers.push({\n                    pos: { sl: parseInt(row, 10) - 1 },\n                    message: message,\n                    level: message.match(/error/) ? \"error\": \"warning\"\n                });\n            });\n            \n            callback(null, markers);\n        }\n    );\n};\n```\n\nThe `execAnalysis()` function will take the current file and any unsaved content and sends it to the workspace. The function makes sure that only the latest changes are sent over the connection to minimize latency.\n[block:callout]\n{\n  \"type\": \"info\",\n  \"title\": \"Using Temporary Files with Analysis Tools\",\n  \"body\": \"[worker_util.execAnalysis()](https://apidoc.c9.io/c9v3/#!/api/language.worker_util-method-execAnalysis) uses `stdin` for passing the current file by default, since it may have unsaved content. Some tools don't like `stdin` very much and prefer a temporary file. Use the `mode: \\\"tempfile\\\"` option to pass a temporary file instead, invoking the analyzer tool as follows:\\n\\n```\\n    \\\"php\\\",\\n    {\\n        mode: \\\"tempfile\\\",\\n        args: [\\\"-l\\\", \\\"$FILE\\\"],\\n    }\\n```\"\n}\n[/block]\n# Adding a Code Completer\n\nCode completion tools such as [Jedi](https://github.com/davidhalter/jedi) or [GoCode](https://github.com/nsf/gocode/) provide code completion given an input source file and a cursor location. The [worker_util.execAnalysis()](https://apidoc.c9.io/c9v3/#!/api/language.worker_util-method-execAnalysis) function makes it possible to invoke such a tool from a language handler. \n\nBelow is an example implementation of [base_handler.complete()](https://apidoc.c9.io/c9v3/#!/api/language.base_handler-method-complete) using `execAnalysis()`. It calls a tool conveniently named `analyzer-tool` on the workspace and passes the current document over `stdin` along with the current row and column:\n\n```\nhandler.complete = function(doc, ast, pos, options, callback) {\n    workerUtil.execAnalysis(\n        \"analyzer-tool\",\n        {\n            mode: \"stdin\",\n            json: true,\n            args: [\n                \"completions\", \"--row\", pos.row, \"--column\", pos.column, \"--path\", path\n            ]\n        },\n        function onResult(err, stdoutJSON, stderrJSON) {\n            if (err) return callback(err);\n            \n            if (typeof stdoutJSON !== \"object\")\n                return callback(new Error(\"Couldn't parse analyzer-tool output: \" + stdoutJSON));\n\n            callback(null, stdoutJSON);\n        }\n    );\n};\n```\n[block:callout]\n{\n  \"type\": \"success\",\n  \"title\": \"Use the Installer to Install Dependencies\",\n  \"body\": \"You can use the [Cloud9 installer](doc:adding-an-installer) to install third-party dependencies like linters, code completion tools, and analyzers.\"\n}\n[/block]\n## Reducing Code Completer Latency\n\nTo reduce latency of a server-side completer, it can be beneficial to use the `predictNextCompletion()` API to predict completions early, and the `noDoc` feature to make sure documentation is only transferred to the client when needed. See [Customizing Code Completers](doc:customizing-code-completers).\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"Reference Implementations\"\n}\n[/block]\nFor a full example of a code completer and linter using the techniques above, see also the Python plugin in the [Full Example Language Handlers](doc:complete-examples#section-python) page. In addition, the Go plugin also serves as a useful, more minimal reference.","excerpt":"","slug":"existing-tools","type":"basic","title":"Using External Language Tools"}

Using External Language Tools


Many languages nowadays have existing code completion and linting/analysis libraries and tools available. Python has its [Jedi](https://github.com/davidhalter/jedi), Go has [GoCode](https://github.com/nsf/gocode/), TypeScript has TSServer, and so on. Tools like these can be integrated into Cloud9 to add code completion support or error and warning markers. [block:callout] { "type": "info", "title": "Building pure JavaScript analyzers instead?", "body": "Not all language plugins are created equal. Some are written fully in JavaScript. See [Language Handlers](doc:language-handlers) for the details on the APIs for writing JavaScript-based analyzers that run fully in the browser. The present page is about tools not written in JavaScript and/or not running in the browser." } [/block] # Background: Code Completion and Analysis in the Cloud As Cloud9 is an online IDE, it has to deal with the challenge of getting the source code from the client to the server, before some analyzer tool that runs there can have a look at it. Cloud9 provides the [worker_util.execAnalysis()](https://apidoc.c9.io/c9v3/#!/api/language.worker_util-method-execAnalysis) API that takes care of this. This function efficiently transfers any unsaved changes of the currently open file to the server, runs an analysis tool, and then gets the results back. To conserve bandwidth and improve performance, `execAnalysis()` doesn't send the whole unsaved document to the server, but only the latest changes. For this it uses the [Operational Transformation](https://en.wikipedia.org/wiki/Operational_transformation) protocol also used for real-time collaboration in Cloud9. It also makes sure that there are no race conditions when sending the request from the worker to the server, even though the user can continue to edit the document in the UI thread. # Adding a Linter/Analyzer Linters and analyzers can produce a list of errors given a source file. For example, php has a built-in linter that can be invoked from the command-line: ``` $ php -l myfile.php Parse error: parse error, expecting `','' or `';'' in - on line 1 Errors parsing - ``` Many linters can also operate on `stdin` instead of an actual file: ``` $ echo '<?php echo "hello" world; ?>' | php -l Parse error: parse error, expecting `','' or `';'' in - on line 1 Errors parsing - ``` Cloud9 plugins can invoke a linter like this in the workspace to show errors in the editor, using the [worker_util.execAnalysis()](https://apidoc.c9.io/c9v3/#!/api/language.worker_util-method-execAnalysis) API: ```javascript var workerUtil = require("plugins/c9.ide.language/worker_util"); handler.analyze = function(docValue, ast, callback) { var markers = []; workerUtil.execAnalysis( "php", { mode: "stdin", args: ["-l"], maxCallInterval: 1200, }, function(err, stdout, stderr) { if (err && err.code !== 255) return callback(err); // Parse each line of output and create marker objects (stdout + stderr).split("\n").forEach(function parseLine(line) { var match = line.match(/(.*?) in (?:.*?) on line (\d+)/); if (!match) return; var message = match[1]; var row = match[2]; markers.push({ pos: { sl: parseInt(row, 10) - 1 }, message: message, level: message.match(/error/) ? "error": "warning" }); }); callback(null, markers); } ); }; ``` The `execAnalysis()` function will take the current file and any unsaved content and sends it to the workspace. The function makes sure that only the latest changes are sent over the connection to minimize latency. [block:callout] { "type": "info", "title": "Using Temporary Files with Analysis Tools", "body": "[worker_util.execAnalysis()](https://apidoc.c9.io/c9v3/#!/api/language.worker_util-method-execAnalysis) uses `stdin` for passing the current file by default, since it may have unsaved content. Some tools don't like `stdin` very much and prefer a temporary file. Use the `mode: \"tempfile\"` option to pass a temporary file instead, invoking the analyzer tool as follows:\n\n```\n \"php\",\n {\n mode: \"tempfile\",\n args: [\"-l\", \"$FILE\"],\n }\n```" } [/block] # Adding a Code Completer Code completion tools such as [Jedi](https://github.com/davidhalter/jedi) or [GoCode](https://github.com/nsf/gocode/) provide code completion given an input source file and a cursor location. The [worker_util.execAnalysis()](https://apidoc.c9.io/c9v3/#!/api/language.worker_util-method-execAnalysis) function makes it possible to invoke such a tool from a language handler. Below is an example implementation of [base_handler.complete()](https://apidoc.c9.io/c9v3/#!/api/language.base_handler-method-complete) using `execAnalysis()`. It calls a tool conveniently named `analyzer-tool` on the workspace and passes the current document over `stdin` along with the current row and column: ``` handler.complete = function(doc, ast, pos, options, callback) { workerUtil.execAnalysis( "analyzer-tool", { mode: "stdin", json: true, args: [ "completions", "--row", pos.row, "--column", pos.column, "--path", path ] }, function onResult(err, stdoutJSON, stderrJSON) { if (err) return callback(err); if (typeof stdoutJSON !== "object") return callback(new Error("Couldn't parse analyzer-tool output: " + stdoutJSON)); callback(null, stdoutJSON); } ); }; ``` [block:callout] { "type": "success", "title": "Use the Installer to Install Dependencies", "body": "You can use the [Cloud9 installer](doc:adding-an-installer) to install third-party dependencies like linters, code completion tools, and analyzers." } [/block] ## Reducing Code Completer Latency To reduce latency of a server-side completer, it can be beneficial to use the `predictNextCompletion()` API to predict completions early, and the `noDoc` feature to make sure documentation is only transferred to the client when needed. See [Customizing Code Completers](doc:customizing-code-completers). [block:api-header] { "type": "basic", "title": "Reference Implementations" } [/block] For a full example of a code completer and linter using the techniques above, see also the Python plugin in the [Full Example Language Handlers](doc:complete-examples#section-python) page. In addition, the Go plugin also serves as a useful, more minimal reference.