To view the examples in this notebook as intended, you need both R and Python language servers up and running.
Magics support is implemented with kernel-specific definitions. The idea is to:
By default, a small number of rules for extractors and overrides is provided, showcased below on the example of IPython kernel and rpy2 integration. The user will be able to specify custom overrides and extractors in the settings.
The extraction is performed by implementations of IForeignCodeExtractor extractors, with RegExpForeignCodeExtractor provided as a simple regular expression-based implementation for the default rules. These rules and this extractor are not designed to be comprehensive, but rather to provide a nice initial experience to the majority of users. More advanced use-cases will require a custom implementation of the IForeignCodeExtractor interface.
With custom implementation of extractors it is easily possible to extract HTML code from within Python (or any other language) strings or CSS from within HTML for analysis by the LSP server, thus the implementation of extractors is not limited for the use with cell or line magics.
The overrides are implemented with regular expressions defined in objects obeying IMagicOverride interface.
In the future the "included batteries" may be moved out to separate extensions i.e. jupyterlab-lsp-ipython for default IPython magics support and jupyterlab-lsp-rpy2 for rpy2 support.
Shell assignments are masked by default so that no "Invalid syntax" message is shown (regardless of the linter used):
!ls
%%python magic¶from statistics import mean
x = mean([1, 2, 3])
The content of script magics with aliases defining their language is moved to separate virtual document for LSP linting, completion and all other features; this is then transparently mapped to the top level virtual document and then back to the notebook:
%%python
# x is not defined in this scope, should be underlined
x
y = 1
# y is not defined in this scope
y
Consecutive calls to the %%python magic will result in separate namespaces. In the extension codebase this is referred as the magic being standalone.
%%python
# y is not defined here (separate namespace from the fist %%python magic)
y
Following IPython magics are treated as standalone:
%%python, %%python2, %%python3, %%pypy - use python server if available%%perl - uses perl server if available%%ruby - uses ruby server if available%%html --isolated - uses html server if available%%sh - uses sh server if availableFollowing IPython magics are treated as parts of a continious virtual document (not standalone):
%%js, %%javascript - use javascript server if available%%html (when without --isolated switch) - uses html server if available%%latex - uses latex server if available%%markdown - uses markdown server if available%%timeit and all other built-in cell magics¶%%timeit reads from the global namespace but it does not write to it:
%%timeit
# note: x is accessible from global namespace and hence not underlied her:
y = x
y
Contrary to the %%python magic, the %%timeit is masked out for the LSP server rather than moved to a separate virtual document. This means that the above example seems to work perfectly only because no linting is active in %%timeit magic:
%%timeit
this_is_undefined_but_wont_be_highlighted
The user can change this behaviour in settings.
By default the name of the magic, will be kept and presented in the LSP document, to prevent a false positive of "defined but unused" on magic imports or magic function/class definitions. For example:
from jupyter_helpers.selective_import import skip_on_import
from jupyter_helpers.selective_import import skip_all_below
%skip_all_below
%skip_all_below was transformed into skip_all_below() with the built-in regular expression, thus preventing the LSP linter from wrongly falgging the import as unused.
All of this will be made customizable to the user by the modification of regular expressions set in the settings.
We could restrict the matches of the default override to only known IPython magics in the potential future jupyterlab-lsp-ipython client extension, allowing the user to disable this feature, but as for now the names are not validated in any way.
Both cell and line magics are linted:
# notice the undeline after 1 ("Trailing whitespaces is superfluous")
%R test = 1
%%R
# again, some trailing spaces added for visibility
test2 = data.frame()
x = test
x = t
%%R
# try <tab> after labell - it should complete to ggplot2::labeller,
# and then, offer you a signature suggestion upon '(' insertion.
ggplot2::labell
Line magics and the "host" (here Python code) can be interspersed within a single cell:
undefined_python_variable
%R invalid R syntax
valid_python = 'syntax'
%R valid_r = c('syntax')