Javascript completions in Emacs using CEDET and Firefox/mozrepl

Tagged:  

OK, let's just directly start with setting this up, because if I talk about CEDET too much, I'm afraid you'll just stop reading. So I will describe how you can get nice completions for Javascript coding from a running Firefox, using CEDET and mozrepl. And yes, this little tutorial will also deal with the actual CEDET installation, because contrary to what you've might read elsewhere, it really isn't that hard. You will need at least Emacs 23, though - we do not support Emacs22 anymore.

So, first step: Install the CEDET development version.

  • Get the development code from CEDET. Our primary repository can be accessed through Bazaar:
    bzr checkout bzr://cedet.bzr.sourceforge.net/bzrroot/cedet/code/trunk cedet
    

    But there's also a git mirror at

    git://git.randomsample.de/cedet.git
    

    (can also be accessed via http)

  • Enter the toplevel cedet directory and call 'make'.
  • Open your init file (.emacs) and add the following lines:
    ;; Load CEDET
    ;; This should be near the top of your init file, so that this can
    ;; really replace the CEDET that ships with Emacs proper.
    (load-file "/home/foo/cedet/cedet-devel-load.el")
    
    ;; Add further minor-modes to be enabled by semantic-mode.
    ;; See doc-string of `semantic-default-submodes' for other things
    ;; you can use here.
    (add-to-list 'semantic-default-submodes 'global-semantic-idle-summary-mode)
    (add-to-list 'semantic-default-submodes 'global-semantic-idle-completions-mode)
    
    ;; Enable Semantic
    (semantic-mode 1)
    
    
  • Restart Emacs.

Congratulations, you've just installed CEDET. If you got problems with these steps, please report them in the comments or better through the CEDET-devel mailing list. We just completely switched to a new development branch for better merging with Emacs, so it might be entirely possible we have missed something.

Everything's working so far? Then let's turn to the Javascript part.

  • Install the mozrepl extension from

    https://addons.mozilla.org/firefox/addon/mozrepl/

  • Restart Firefox and start Mozrepl from its menu entry under "Tools".
  • You should now be able to connect to port 4242 (try "telnet localhost 4242" or "nc localhost 4242", which should show you the mozrepl prompt).
  • Now let's turn to Emacs. Open a file "test.html" and copy&paste the following contents:
    <!doctype html>
    <html>
    <head>
      <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.5/jquery.min.js"></script>
      <script src="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8/jquery-ui.min.js"></script>
    </head>
    <body>
    This is just a test.
    </body>
    </html>
    

    As you can see, this is basically a HTML5 file that loads the jQuery framework from googleapis.com.

  • Save the file and do
    M-x semanticdb-mozrepl-activate RET
    

    Then, enter the URL "file:///<FULL PATH>/test.html". CEDET's Semantic will then open a connection to mozrepl and load the file in Firefox. After this is done, Emacs should say "Finished activating mozrepl database for <URL>", and Firefox should show you the above test.html file.

  • Now create a new file "foo.js". Emacs should automatically switch to javascript-mode (if you configured Emacs to use another mode, it might not work). Now simply enter
    $.a
    

    and wait for a second or two. Semantic should then start to give you completions on the jQuery object '$' through the so called "ghost text" style.

    You can cycle through the completions by pressing TAB and choose one with RET. If you've never dealt with jQuery, just type

    document.get
    

    and wait for a second, and more familiar stuff should appear.

OK, if this is working so far, you are actually finished with the basic setup! Everything what follows now is dealing with the different types of completion interfaces you can use.

What you've already seen is the so called "idle completion" from Semantic, which by default uses this special kind of "ghost text" which isn't too intrusive so that it should not interfere with typing. However, you're probably more used to the typical popup-tooltips, which are also available. But first, let me say right off the bat that all completion mechanisms in CEDET are pretty basic. CEDET was always intended to be more of a framework for application developers, and luckily there are now at least two very nice completion frameworks which support CEDET, but have to be installed separately. I'll deal with those two in a minute, but first let's look at the ones you can use right away:

  • M-x semantic-ia-complete-symbol-menu: This will display a menu with the possible completions. You can choose one by pressing up/down and RET. This is an older demonstration function for completion in Semantic.

  • M-x semantic-complete-analyze-inline: This is the newer and more modern interface for doing completions in CEDET. You can define different completion styles by doing
    M-x customize-variable RET semantic-complete-inline-analyzer-displayor-class RET
    

    In this screenshot, you see the tooltip display of the completions. In the same way, you can configure the display style for idle completion, which you've already witnessed above, through 'semantic-complete-inline-analyzer-idle-displayor-class' (Yeah, I know... TAB is your friend when calling customize on those things.).

If you are unhappy with these types of completions, you might want to take a look at the separate completion packages which are available. The two most popular are auto-complete and company-mode:

  • auto-complete is available at

    http://cx4a.org/software/auto-complete/

    It has a lot of features and is pretty snappy. It supports CEDET's Semantic and I happen to like it a lot.

    In the screenshot you could see further documentation for the 'ajax' function if mozrepl could supply any (possible for other languages like Emacs Lisp, though). However, its default configuration might be a bit "too much" for you. For example, you might want to restrict the number of backends it asks for completions. If you just set

      (setq-default ac-sources '(ac-source-semantic-raw))
    

    it will only query Semantic for completions and nothing else. Also, have a look at the variables "ac-delay" and "ac-auto-start", which define when auto-completion starts; the default behavior might be too intrusive for you, especially since getting completions from Semantic might not be fast enough for fluid typing (for large C++ projects it surely isn't).

  • company-mode is available via GNU ELPA (built into Emacs24, just do M-x list-packages), or from

    http://nschum.de/src/emacs/company-mode/

    In my opinion it is easier to configure and its defaults are less intrusive than auto-complete, but it also has less features. On Emacs24, it doesn't play well with an activated header-line (if you don't know what this is, you're not using them). Otherwise, it is a really nice framework for completions and works with CEDET out of the box. You just have to configure "company-semantic-modes", for example

    (setq company-semantic-modes '(c-mode c++-mode js-mode jde-mode java-mode emacs-lisp-mode))
    

    and make sure the buffer-local 'company-backend' includes 'company-semantic'. Please see the documentation for details. You can see an example in the first screenshot on this page.

OK, I think you have enough to get started. Please let me know how this stuff works for you. It is a pretty new thing and I'm not sure how stable this actually is; communicating with mozrepl is a bit of a challenge, but at least for me, it works pretty well so far.

Now that we've dealt with the interesting part, a few words on how it works, which should also highlight the problems this approach has.

Without the mozrepl connection, CEDET cannot help you much with Javascript. Yes, we have a grammar and it happily parses function definitions, but since Javascript is so incredibly dynamic and doesn't have proper classes, you cannot do much with static parsing. Since lately I have to deal more with Javascript to investigate all the new features like Webworkers, File API and so on, I first did what I guess most people do: install Firebug and use the console and try things in a REPL-like fashion. So I found myself hopping from Emacs to Firefox and back. I dimly remembered this little extension called mozrepl, which allows you to connect to a running Firefox session, and there's also a Emacs comint mode for it. It works OK, but it lacks the completion mechanisms from the Firebug console, so I did the next best thing and hooked it into CEDET.

CEDET has the concept of an "omniscience database". For example, Emacs itself is such a database when you're coding Emacs Lisp. As soon as you evaluate your functions, Emacs knows about them, including documentation, file location, and so on. For (client-side) Javascript, the Browser essentially is what Emacs is for Emacs Lisp: the framework in which the code runs. Through mozrepl, we can ask the Browser's Javascript engine for details on objects and the DOM.

There's one thing though: if you're switching tabs, the context in mozrepl will switch, too. There is however a built-in solution to this problem, but it is not activated by default since I'm not sure yet how well it works. The semanticdb-mozrepl backend can detect if you have switched to another tab and switch back automatically if it needs to query mozrepl. The variable that controls this behavior is "semanticdb-mozrepl-switch-tabs"; just see its docstring.

OK, I think this will suffice for now. Please let me know how this works for you, either in the comments or ask question on the CEDET-devel mailing list (which BTW is also available via Gmane as gmane.emacs.cedet).

Installed (emacs24). Configured. And... it fails. Mozrepl disconnects all the time and asks for URL to reconnect (in mini-buffer). I have to hit return to reconnect and again... again.... again.... It disconnects always on some code that has no complete candidates. Such as 'if' statement etc. Then it cannot connect automatically without user interaction. So, semantic mozrepl is cool but not usable for me at the moment :(
me too,why?
I also got this problem and don't know why
I've been trying to setup a working JavaScript development environment using Emacs for quite some time, and this looks very promising, but whenever I try to compile CEDET (via make or using Emacs, from a fresh bzr checkout) I get the following: In toplevel form: dframe.el:422:19:Warning: reference to free variable `x-display-pixel-width' dframe.el:422:61:Warning: reference to free variable `terminal' dframe.el:423:19:Warning: reference to free variable `x-display-pixel-height' In end of data: dframe.el:999:1:Warning: the following functions are not known to be defined: declare-function, &optional, x-display-pixel-width, x-display-pixel-height, mouse-set-point, popup-menu Wrote /Users/cesarolea/workspace/cedet/lisp/speedbar/dframe.elc > speedbar.elc In toplevel form: speedbar.el:130:1:Error: Symbol's value as variable is void: x-display-pixel-width make[2]: *** [speedbar.elc] Error 1 make[1]: *** [speedbar] Error 2 make: *** [compile] Error 2 And then it stops. I really don't know where to go from there. Any hints?
I was thinking if this will autocomplete anything that is loaded into the browser rather than just DOM stuff. So if I want to statically use autocomplete in project, Is it enough to create a single html with all the JS included and connect this tab of FF to the Emacs through mozilla repl. Thanks Arun
yep
Yes, that should be enough.
Is there any chance to get semantic running with js2-mode?
yep
It's probably not too much work, but I had the impression that this mode was more or less abandoned.
Generating autoloads for texi.el...done
Saving file /export/profile/dotemacs.d/cedet.git/lisp/cedet/srecode/loaddefs.el...
Wrote /export/profile/dotemacs.d/cedet.git/lisp/cedet/srecode/loaddefs.el
(No changes need to be saved)
make[1]: Leaving directory `/export/profile/dotemacs.d/cedet.git/lisp/cedet/srecode'
Creating Makefiles using EDE.
Debug on Error enabled globally
...
...snip a load of lisp for brevity...
  command-line-1(("-l" "cedet-remove-builtin.el" "--eval" "(setq cedet-bootstrap-in-progress t ede-project-directories t)" "-f" "toggle-debug-on-error" "-l" "cedet-devel-load
.el" "--eval" "(progn (global-ede-mode) (find-file \"/export/profile/dotemacs.d/cedet.git/lisp/Project.ede\") (ede-proj-regenerate) (find-file \"/export/profile/dotemacs.d/ce
det.git/doc/texi/Project.ede\") (ede-proj-regenerate))"))
  command-line()
  normal-top-level()

make: *** [lisp/cedet/Makefile] Error 255
Which is unfortunate. I've been wanting to try CEDET for years but every time I want to try it out I inevitably stumble on a land-mine which makes getting it up and running an epic. I applaud the nice simple .init example and the addition of a cedet-devel-load which I assume is trying to make the default case (running out of tree snapshot against system emacs) integrate more nicely. However I suggest the team might want to invest some time in ensuring the tip-of-tree never breaks so potential users don't end up scratching their heads. By all means keep on posting about CEDET, one day I hope to be enlightened by what it offers ;-)
It seems the timestamps from the Makefiles are older than the Project.ede files. Please use

make touch-makefiles
make clean-all

and try again. And I agree that we need to streamline these things, but it is a bit difficult here because it all depends on the timestamps the files have after they are checked out. How did you get the source code (through git or bzr)?

git. I tried bzr and it built fine. However next problem:

  (when (featurep package) (error "%s is already loaded.  Removing CEDET now would be unwise." (symbol-name package)))
  (while --cl-dolist-temp-- (setq package (car --cl-dolist-temp--)) (when (featurep package) (error "%s is already loaded.  Removing CEDET now would be unwise." (symbol-name package))) (setq --cl-dolist-temp-- (cdr --cl-dolist-temp--)))
  (let ((--cl-dolist-temp-- cedet-remove-builtin-package-list) package) (while --cl-dolist-temp-- (setq package (car --cl-dolist-temp--)) (when (featurep package) (error "%s is already loaded.  Removing CEDET now would be unwise." (symbol-name package))) (setq --cl-dolist-temp-- (cdr --cl-dolist-temp--))))
  (catch (quote --cl-block-nil--) (let ((--cl-dolist-temp-- cedet-remove-builtin-package-list) package) (while --cl-dolist-temp-- (setq package (car --cl-dolist-temp--)) (when (featurep package) (error "%s is already loaded.  Removing CEDET now would be unwise." (symbol-name package))) (setq --cl-dolist-temp-- (cdr --cl-dolist-temp--)))))
  (cl-block-wrapper (catch (quote --cl-block-nil--) (let ((--cl-dolist-temp-- cedet-remove-builtin-package-list) package) (while --cl-dolist-temp-- (setq package (car --cl-dolist-temp--)) (when (featurep package) (error "%s is already loaded.  Removing CEDET now would be unwise." (symbol-name package))) (setq --cl-dolist-temp-- (cdr --cl-dolist-temp--))))))
  (block nil (let ((--cl-dolist-temp-- cedet-remove-builtin-package-list) package) (while --cl-dolist-temp-- (setq package (car --cl-dolist-temp--)) (when (featurep package) (error "%s is already loaded.  Removing CEDET now would be unwise." (symbol-name package))) (setq --cl-dolist-temp-- (cdr --cl-dolist-temp--)))))
  (dolist (package cedet-remove-builtin-package-list) (when (featurep package) (error "%s is already loaded.  Removing CEDET now would be unwise." (symbol-name package))))
  cedet-remove-builtin()
  eval-buffer(#> nil "/home/ajb/.emacs.d/cedet.bzr/cedet-remove-builtin.el" nil t)  ; Reading at buffer position 2274
  load-with-code-conversion("/home/ajb/.emacs.d/cedet.bzr/cedet-remove-builtin.el" "/home/ajb/.emacs.d/cedet.bzr/cedet-remove-builtin.el" nil nil)
  load("/home/ajb/.emacs.d/cedet.bzr/cedet-remove-builtin.el" nil nil t)
  load-file("/home/ajb/.emacs.d/cedet.bzr/cedet-remove-builtin.el")
  (if (boundp (quote cedet-bootstrap-in-progress)) nil (load-file (expand-file-name "cedet-remove-builtin.el" CEDETDIR)))
  (unless (boundp (quote cedet-bootstrap-in-progress)) (load-file (expand-file-name "cedet-remove-builtin.el" CEDETDIR)))
  (let ((CEDETDIR (file-name-directory (or load-file-name (buffer-file-name))))) (unless (boundp (quote cedet-bootstrap-in-progress)) (load-file (expand-file-name "cedet-remove-builtin.el" CEDETDIR))) (add-to-list (quote load-path) CEDETDIR) (add-to-list (quote load-path) (expand-file-name "lisp/cedet" CEDETDIR)) (add-to-list (quote load-path) (expand-file-name "lisp/eieio" CEDETDIR)) (add-to-list (quote load-path) (expand-file-name "lisp/common" CEDETDIR)) (add-to-list (quote load-path) (expand-file-name "lisp/speedbar" CEDETDIR)) (require (quote eieio)) (require (quote ede)) (unless (boundp (quote cedet-bootstrap-in-progress)) (load-file (expand-file-name "lisp/eieio/loaddefs.el" CEDETDIR)) (load-file (expand-file-name "lisp/speedbar/loaddefs.el" CEDETDIR)) (load-file (expand-file-name "lisp/cedet/loaddefs.el" CEDETDIR)) (load-file (expand-file-name "lisp/cedet/ede/loaddefs.el" CEDETDIR)) (load-file (expand-file-name "lisp/cedet/cogre/loaddefs.el" CEDETDIR)) (load-file (expand-file-name "lisp/cedet/srecode/loaddefs.el" CEDETDIR)) (load-file (expand-file-name "lisp/cedet/semantic/loaddefs.el" CEDETDIR)) (setq Info-directory-list (cons (expand-file-name "doc/info" CEDETDIR) Info-default-directory-list))))
  eval-buffer(# nil "/home/ajb/.emacs.d/cedet.bzr/cedet-devel-load.el" nil t)  ; Reading at buffer position 2709
  load-with-code-conversion("/home/ajb/.emacs.d/cedet.bzr/cedet-devel-load.el" "/home/ajb/.emacs.d/cedet.bzr/cedet-devel-load.el" nil nil)
  load("/home/ajb/.emacs.d/cedet.bzr/cedet-devel-load.el" nil nil t)
  load-file("/home/ajb/.emacs.d/cedet.bzr/cedet-devel-load.el")
  eval((load-file "/home/ajb/.emacs.d/cedet.bzr/cedet-devel-load.el") nil)
  eval-last-sexp-1(nil)
  eval-last-sexp(nil)
  call-interactively(eval-last-sexp nil nil)
  recursive-edit()
  debug(error (error "No kbd macro has been defined"))
  call-last-kbd-macro(1)
  call-interactively(call-last-kbd-macro nil nil)
Which I assume is a problem clashing with the CEDET already in Emacs 24 (I'm running out of the Emacs bzr tree).
Some parts of the Emacs-internal CEDET are already activated. You have to put the `load-file' command that loads 'cedet-devel-load.el' at the top of your init file (or at least near the top), like the comment over it says. Ideally, it should be the first thing that gets loaded when Emacs starts up.
Ahh yes, I shall re-start Emacs (it usually stays up for days at a time on my work machine).
I've just push a change so that the 'touch-makefiles' rule will now be called by default.
thanks, that fixed the build but I got another issue. Am subscribing to the mailing list now ;-)
Great post, it's nice to see that CEDET is progressing. Thanks to posts like this I hope we'll get to understand a bit more how to leverage all its potential. I'm a happy eieio user myself, but semantic, bovinator, and others go over my head atm. Btw, I think you forgot some elisp on auto-complete section, "If you just set" ...
There was one tiny '"' missing in a 'pre' tag. :-)