|
|
|
|
<?xml version="1.0"?>
|
|
|
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
|
|
|
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
|
|
|
|
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
|
|
|
<head>
|
|
|
|
|
<meta name="generator"
|
|
|
|
|
content="HTML Tidy for Windows (vers 1st August 2002), see www.w3.org" />
|
|
|
|
|
<meta name="generator" content="SciTE" />
|
|
|
|
|
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
|
|
|
|
|
|
|
|
|
|
<title>Lexilla Documentation</title>
|
|
|
|
|
|
|
|
|
|
<style type="text/css">
|
|
|
|
|
<!--
|
|
|
|
|
/*<![CDATA[*/
|
|
|
|
|
CODE { font-weight: bold; font-family: Menlo,Consolas,Bitstream Vera Sans Mono,Courier New,monospace; }
|
|
|
|
|
A:visited { color: blue; }
|
|
|
|
|
A:hover { text-decoration: underline ! important; }
|
|
|
|
|
A.message { text-decoration: none; font-weight: bold; font-family: Menlo,Consolas,Bitstream Vera Sans Mono,Courier New,monospace; }
|
|
|
|
|
A.seealso { text-decoration: none; font-weight: bold; font-family: Menlo,Consolas,Bitstream Vera Sans Mono,Courier New,monospace; }
|
|
|
|
|
A.toc { text-decoration: none; }
|
|
|
|
|
A.jump { text-decoration: none; }
|
|
|
|
|
LI.message { text-decoration: none; font-weight: bold; font-family: Menlo,Consolas,Bitstream Vera Sans Mono,Courier New,monospace; }
|
|
|
|
|
H2 { background: #E0EAFF; }
|
|
|
|
|
|
|
|
|
|
table {
|
|
|
|
|
border: 0px;
|
|
|
|
|
border-collapse: collapse;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
div.console {
|
|
|
|
|
font-family: Menlo,Consolas,Bitstream Vera Sans Mono,Courier New,monospace;
|
|
|
|
|
color: #008000;
|
|
|
|
|
font-weight: bold;
|
|
|
|
|
background: #F7FCF7;
|
|
|
|
|
border: 1px solid #C0D7C0;
|
|
|
|
|
margin: 0.3em 3em;
|
|
|
|
|
padding: 0.3em 0.6em;
|
|
|
|
|
}
|
|
|
|
|
span.console {
|
|
|
|
|
font-family: Menlo,Consolas,Bitstream Vera Sans Mono,Courier New,monospace;
|
|
|
|
|
color: #008000;
|
|
|
|
|
font-weight: bold;
|
|
|
|
|
background: #F7FCF7;
|
|
|
|
|
border: 1px solid #C0D7C0;
|
|
|
|
|
margin: 0.1em 0em;
|
|
|
|
|
padding: 0.1em 0.3em;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
.name {
|
|
|
|
|
color: #B08000;
|
|
|
|
|
}
|
|
|
|
|
/*]]>*/
|
|
|
|
|
-->
|
|
|
|
|
</style>
|
|
|
|
|
</head>
|
|
|
|
|
|
|
|
|
|
<body bgcolor="#FFFFFF" text="#000000">
|
|
|
|
|
<table bgcolor="#000000" width="100%" cellspacing="0" cellpadding="0" border="0"
|
|
|
|
|
summary="Banner">
|
|
|
|
|
<tr>
|
|
|
|
|
<td><img src="SciTEIco.png" border="3" height="64" width="64" alt="Lexilla icon" /></td>
|
|
|
|
|
|
|
|
|
|
<td><a href="index.html"
|
|
|
|
|
style="color:white;text-decoration:none;font-size:200%">Lexilla</a></td>
|
|
|
|
|
</tr>
|
|
|
|
|
</table>
|
|
|
|
|
|
|
|
|
|
<h1>Lexilla Documentation</h1>
|
|
|
|
|
|
|
|
|
|
<p>Last edited 21 April 2021 NH</p>
|
|
|
|
|
|
|
|
|
|
<h2>Introduction</h2>
|
|
|
|
|
|
|
|
|
|
<p>Lexilla is a library containing lexers for use with Scintilla. It can be either a static library
|
|
|
|
|
that is linked into an application or a shared library that is loaded at runtime.</p>
|
|
|
|
|
|
|
|
|
|
<p>Lexilla does not interact with the display so there is no need to compile it for a
|
|
|
|
|
particular GUI toolkit. Therefore there can be a common library shared by applications using
|
|
|
|
|
different GUI toolkits. In some circumstances there may need to be both 32-bit and 64-bit versions
|
|
|
|
|
on one system to match different applications.</p>
|
|
|
|
|
|
|
|
|
|
<p>Different extensions are commonly used for shared libraries: .so on Linux, .dylib on macOS, and .DLL on Windows.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<h2>The Lexilla protocol</h2>
|
|
|
|
|
|
|
|
|
|
<p>A set of functions is defined by Lexilla for use by applications. Libraries that provide these functions
|
|
|
|
|
can be used as a replacement for Lexilla or to add new lexers beyond those provided by Lexilla.</p>
|
|
|
|
|
|
|
|
|
|
<p>The Lexilla protocol is a superset of the external lexer protocol and defines these functions that may be exported from a shared library:<br />
|
|
|
|
|
<code>int <span class="name">GetLexerCount</span>()</code><br />
|
|
|
|
|
<code>void <span class="name">GetLexerName</span>(unsigned int index, char *name, int buflength)</code><br />
|
|
|
|
|
<code>LexerFactoryFunction <span class="name">GetLexerFactory</span>(unsigned int index)</code><br />
|
|
|
|
|
<code>ILexer5 *<span class="name">CreateLexer</span>(const char *name)</code><br />
|
|
|
|
|
<code>const char *<span class="name">LexerNameFromID</span>(int identifier)</code><br />
|
|
|
|
|
<code>const char *<span class="name">GetLibraryPropertyNames</span>()</code><br />
|
|
|
|
|
<code>void <span class="name">SetLibraryProperty</span>(const char *key, const char *value)</code><br />
|
|
|
|
|
<code>const char *<span class="name">GetNameSpace</span>()</code>
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p><span class="name">ILexer5</span> is defined by Scintilla in include/ILexer.h as the interface provided by lexers which is called by Scintilla.
|
|
|
|
|
Many clients do not actually need to call methods on <span class="name">ILexer5</span> - they just take the return from CreateLexer and plug it
|
|
|
|
|
straight into Scintilla so it can be treated as a machine pointer (void *).
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p><span class="name">LexerFactoryFunction</span> is defined as a function that takes no arguments and returns an <span class="name">ILexer5</span> *:
|
|
|
|
|
<code>ILexer5 *(*LexerFactoryFunction)()</code> but this can be ignored by most client code.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>The Lexilla protocol is a superset of the earlier external lexer protocol that defined the first 3 functions
|
|
|
|
|
(<span class="name">GetLexerCount</span>, <span class="name">GetLexerName</span>, <span class="name">GetLexerFactory</span>)
|
|
|
|
|
so Lexilla can be loaded by applications that support that protocol.
|
|
|
|
|
<span class="name">GetLexerFactory</span> will rarely be used now as it is easier to call <span class="name">CreateLexer</span>.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p><span class="name">CreateLexer</span> is the main call that will create a lexer for a particular language. The returned lexer can then be
|
|
|
|
|
set as the current lexer in Scintilla by calling
|
|
|
|
|
<a class="seealso" href="ScintillaDoc.html#SCI_SETILEXER">SCI_SETILEXER</a>.</p>
|
|
|
|
|
|
|
|
|
|
<p><span class="name">LexerNameFromID</span> is an optional function that returns the name for a lexer identifier.
|
|
|
|
|
<code>LexerNameFromID(SCLEX_CPP) → "cpp"</code>.
|
|
|
|
|
This is a temporary affordance to make it easier to convert applications to using Lexilla.
|
|
|
|
|
Applications should move to using lexer names instead of IDs.
|
|
|
|
|
This function is deprecated, showing warnings with some compilers, and will be removed in a future version of Lexilla.</p>
|
|
|
|
|
|
|
|
|
|
<p><span class="name">SetLibraryProperty</span> and <span class="name">GetLibraryPropertyNames</span>
|
|
|
|
|
are optional functions that can be
|
|
|
|
|
defined if a library requires initialisation before calling other methods.
|
|
|
|
|
For example, a lexer library that reads language definitions from XML files may require that the
|
|
|
|
|
directory containing these files be set before a call to CreateLexer.
|
|
|
|
|
<code>SetLibraryProperty("definitions.directory", "/usr/share/xeditor/language-definitions")</code>
|
|
|
|
|
If a library implements SetLibraryProperty then it may also provide a set of valid property names with
|
|
|
|
|
GetLibraryPropertyNames that can then be used by the application to define configuration file property
|
|
|
|
|
names or user interface elements for options dialogs.</p>
|
|
|
|
|
|
|
|
|
|
<p><span class="name">GetNameSpace</span> is an optional function that returns a namespace string
|
|
|
|
|
that can be used to disambiguate lexers with the same name from different providers.
|
|
|
|
|
If Lexilla and XMLLexers both provide a "cpp" lexer than a request for "cpp" may be satisfied by either but "xmllexers.cpp"
|
|
|
|
|
unambiguously refers to the "cpp" lexer from XMLLexers.</p>
|
|
|
|
|
|
|
|
|
|
<h2>Building Lexilla</h2>
|
|
|
|
|
|
|
|
|
|
<p>Before using Lexilla it must be built or downloaded.</p>
|
|
|
|
|
<p>Lexilla requires some headers from Scintilla to build and expects a directory named
|
|
|
|
|
"scintilla" containing a copy of Scintilla 5+ to be a peer of the Lexilla top level
|
|
|
|
|
directory conventionally called "lexilla".</p>
|
|
|
|
|
|
|
|
|
|
<div>To build Lexilla, in the lexilla/src directory, run make (for gcc or clang)</div>
|
|
|
|
|
<div class="console">make</div>
|
|
|
|
|
<div>or nmake for MSVC</div>
|
|
|
|
|
<div class="console">nmake -f lexilla.mak</div>
|
|
|
|
|
<br />
|
|
|
|
|
|
|
|
|
|
<div>After building Lexilla, its test suite can be run with make/nmake in the lexilla/test directory. For gcc or clang</div>
|
|
|
|
|
<div class="console">make test</div>
|
|
|
|
|
or for MSVC<br />
|
|
|
|
|
<div class="console">nmake -f testlexers.mak test</div>
|
|
|
|
|
<div>Each test case should show "<code>Lexing ...</code>" and errors will display a diagnostic, commonly showing
|
|
|
|
|
a difference between the actual and expected result:<br>
|
|
|
|
|
<code>C:\u\hg\lexilla\test\examples\python\x.py:1: is different</code>
|
|
|
|
|
</div>
|
|
|
|
|
|
|
|
|
|
<p>There are also RunTest.sh / RunTest.bat scripts in the scripts directory to build Lexilla and then build and run the tests.
|
|
|
|
|
These both use gcc/clang, not MSVC.</p>
|
|
|
|
|
|
|
|
|
|
<p>There are Microsoft Visual C++ and Xcode projects that can be used to build Lexilla.
|
|
|
|
|
For Visual C++: src/Lexilla.vcxproj. For Xcode: src/Lexilla/Lexilla.xcodeproj.
|
|
|
|
|
There is also test/TestLexers.vcxproj to build the tests with Visual C++.</p>
|
|
|
|
|
|
|
|
|
|
<h2>Using Lexilla</h2>
|
|
|
|
|
|
|
|
|
|
<p>Definitions for using Lexilla from C and C++ are included in lexilla/include/Lexilla.h.
|
|
|
|
|
For C++, scintilla/include/ILexer.h should be included before Lexilla.h as the
|
|
|
|
|
<code>ILexer5</code> type is used.
|
|
|
|
|
For C, ILexer.h should not be included as C does not understand it and from C,
|
|
|
|
|
<code>void*</code> is used instead of <code>ILexer5*</code>.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>For many applications the main Lexilla operations are loading the Lexilla library, creating a
|
|
|
|
|
lexer and using that lexer in Scintilla.
|
|
|
|
|
Applications need to define the location (or locations) they expect
|
|
|
|
|
to find Lexilla or libraries that support the Lexilla protocol.
|
|
|
|
|
They also need to define how they request particular lexers, perhaps with a mapping from
|
|
|
|
|
file extensions to lexer names.</p>
|
|
|
|
|
|
|
|
|
|
<h3 id="CheckLexilla">From C - CheckLexilla</h3>
|
|
|
|
|
|
|
|
|
|
<p>An example C program for accessing Lexilla is provided in lexilla/examples/CheckLexilla.
|
|
|
|
|
Build with <span class="console">make</span> and run with <span class="console">make check</span>.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<h3>From C++ - LexillaAccess</h3>
|
|
|
|
|
|
|
|
|
|
<p>A C++ module, LexillaAccess.cxx / LexillaAccess.h is provided in lexilla/access.
|
|
|
|
|
This can either be compiled into the application when it is sufficient
|
|
|
|
|
or the source code can be copied into the application and customized when the application has additional requirements
|
|
|
|
|
(such as checking code signatures).
|
|
|
|
|
SciTE uses LexillaAccess.</p>
|
|
|
|
|
|
|
|
|
|
<p>LexillaAccess supports loading multiple shared libraries implementing the Lexilla protocol at one time.</p>
|
|
|
|
|
|
|
|
|
|
<h3>From Qt</h3>
|
|
|
|
|
|
|
|
|
|
<p>For Qt, use either LexillaAccess from above or Qt's QLibrary class. With 'Call' defined to call Scintilla APIs.<br />
|
|
|
|
|
<code>
|
|
|
|
|
#if _WIN32<br />
|
|
|
|
|
typedef void *(__stdcall *CreateLexerFn)(const char *name);<br />
|
|
|
|
|
#else<br />
|
|
|
|
|
typedef void *(*CreateLexerFn)(const char *name);<br />
|
|
|
|
|
#endif<br />
|
|
|
|
|
QFunctionPointer fn = QLibrary::resolve("lexilla", "CreateLexer");<br />
|
|
|
|
|
void *lexCpp = ((CreateLexerFn)fn)("cpp");<br />
|
|
|
|
|
Call(SCI_SETILEXER, 0, (sptr_t)(void *)lexCpp);<br />
|
|
|
|
|
</code></p>
|
|
|
|
|
|
|
|
|
|
<p>Applications may discover the set of lexers provided by a library by first calling
|
|
|
|
|
<span class="name">GetLexerCount</span> to find the number of lexers implemented in the library then looping over calling
|
|
|
|
|
<span class="name">GetLexerName</span> with integers 0 to <code>GetLexerCount()-1</code>.</p>
|
|
|
|
|
|
|
|
|
|
<p>Applications may set properties on a library by calling <span class="name">SetLibraryProperty</span> if provided.
|
|
|
|
|
This may be needed for initialisation so should before calling <span class="name">GetLexerCount</span> or <span class="name">CreateLexer</span>.
|
|
|
|
|
A set of property names may be available from <span class="name">GetLibraryPropertyNames</span> if provided.
|
|
|
|
|
It returns a string pointer where the string contains a list of property names separated by '\n'.
|
|
|
|
|
It is up to applications to define how properties are defined and persisted in its user interface
|
|
|
|
|
and configuration files.</p>
|
|
|
|
|
|
|
|
|
|
<h2>Modifying or adding lexers</h2>
|
|
|
|
|
|
|
|
|
|
<p>Lexilla can be modified or a new library created that can be used to replace or augment Lexilla.</p>
|
|
|
|
|
|
|
|
|
|
<p>Lexer libraries that provide the same functions as Lexilla may provide lexers for use by Scintilla,
|
|
|
|
|
augmenting or replacing those provided by Lexilla.
|
|
|
|
|
To allow initialisation of lexer libraries, a <code>SetLibraryProperty(const char *key, const char *value)</code>
|
|
|
|
|
may optionally be implemented. For example, a lexer library that uses XML based lexer definitions may
|
|
|
|
|
be provided with a directory to search for such definitions.
|
|
|
|
|
Lexer libraries should ignore any properties that they do not understand.
|
|
|
|
|
The set of properties supported by a lexer library is specified as a '\n' separated list of property names by
|
|
|
|
|
an optional <code>const char *GetLibraryPropertyNames()</code> function.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
<p>Lexilla and its contained lexers can be tested with the TestLexers program in lexilla/test.
|
|
|
|
|
Read lexilla/test/README for information on building and using TestLexers.</p>
|
|
|
|
|
|
|
|
|
|
<p>An example of a simple lexer housed in a shared library that is compatible with the
|
|
|
|
|
Lexilla protocol can be found in lexilla/examples/SimpleLexer. It is implemented in C++.
|
|
|
|
|
Build with <span class="console">make</span> and check by running <a href="#CheckLexilla">CheckLexilla</a> against it with
|
|
|
|
|
<span class="console">make check</span>.
|
|
|
|
|
</p>
|
|
|
|
|
|
|
|
|
|
</body>
|
|
|
|
|
</html>
|
|
|
|
|
|
