Using Highlight With Pandoc
by Tristano Ajmone, MIT License (MIT). Last edited: Apr 16, 2017.
“Highlight PP-Macros” is a set of macros to add cross-platform in-document access to Highlight within markdown source files via PP (pandoc preprocessor). It allows to fetch an exteranl source file, highlight its syntax, and inject the resulting html into the preprocessed document. It also provides a macro for importing an Highlight theme as an inline CSS stylesheet.
"Highlight.pp" v1.2 (2017-04-16)
CONTENTS
Introduction
The solution proposed here relies on three tools to achieve integration of Highlight in pandoc source files:
- Pandoc
- PP (a generic pandoc preprocessor)
- The
Highlight.pp
macros file found in this folder
Pandoc
Pandoc — aka “the universal markup converter” — is a well known command line tool for converting documents from one format to another, and especially from/to various markdown flavors, including “pandoc markdown” (an extended markdown variant introduced by pandoc itself). This very document was written in markdown and converted to html5 using pandoc and a custom template.
Pandoc is widely used for academic papers and technical documentation, so it comes with a built-in syntax highlighter. By default, pandoc doesn’t support third party syntax highlighters, and the supported languages are hardcoded into pandoc and not extensible by the end user without recompiling pandoc.1
Furthermore, pandoc doesn’t provide any means to import the source code from an external file — every time the source file changes, you have to manually copy-&-paste its contents.
PP
Highlight PP-Macros leverages PP, a generic preprocessor “written with pandoc in mind”. PP adds to the pandoc workflow the power of user definable macros and a set of built-in macros for cross-platform file I/O operations and conditional branching — and many other great features not cover here.
Since one of PP’s goals was to introduce literate programming functionality to the pandoc workflow, it provides a solid base for handling external files, invoking third party tools, and perform operations on blocks of source code.
Highlight Macros
The file “Highlight.pp
” contains the macros for integrating Highlight in pandoc workflow. It was taken from the “Pandoc Goodies” project, this file being one of the various macros modules that make up the PP-Macros Library subproject hosted therein:
If you intend using Highlight pp-macros, you are strongly adviced to check regularly the above link for updated versions of “Highlight.pp
”. The contents of this folder are intended mainly as a starting example of how to integrate Highlight into pandoc workflow, and they may not reflect the latest release of the macros found on the Pandoc Goodies project.
How It Works
Currently, the Highlight PP-Macros module works only with html output. It works on all OS that support PP.
There are two macros available:
!HighlightFile(FILE)(LANG)[(OPTIONS)]
— imports and syntax-highlights and external file into the document as a raw html<pre><code>
block.!HighlightInlineTheme(THEME)
— retrives a Highlight theme and injects its CSS version into the document.
The !HighlightFile
Macro
The !HighlightFile
macro takes two mandatory parameters and an optional third one:
FILE
— the name of the external source code file.LANG
— the language of the source code.OPTIONS
— additional command line options for invoking Highlight.
The LANG
parameter is mandatory even if the source code file has an extension known to Highlight: the macro needs LANG
in order to write its value as the class of the <code>
tag in the enclosing <pre><code>
block (these tags are written by the macro, not by Highlight). For example, !HighlightFile(somefile.py)(python)
will produce:
<pre class="hl"><code class="python">
By default, this macro invokes Highlight with the following basic options:
--fragment
— omit document header and footer, get just the highlighted code.--syntax=<LANG>
— set language of source code toLANG
parameter.--no-trailing-nl
— omit trailing newline.--validate-input
— strip BOM (if present).
… followed by the literal string value of the OPTIONS
parameter (if any), which must consist of options compatible with the above invocation settings, and keeping in mind that Highlight’s output will be enclosed in <pre><code>
tags.
Since this macro outputs raw html blocks, pandoc’s built-it highlighter will ignore them. Furthermore, pandoc will still highlight any fenced (or backticked) source code block within the markdown document (ie: unless it’s explicitly invoked with the --no-highlight
options). This allows using Highlight side by side with pandoc’s highlighter, and even benefit from dual syntax color themes (this might require some CSS tweaking, or custom stylesheets).
The !HighlightInlineTheme
Macro
The macro !HighlightInlineTheme
offers a quick solution for theming Highlight code without having to first export a theme to CSS (via Highlight) and then import it back into the final document as an external CSS file v(ia pandoc): the macro imports a Highlight theme directly into the document, as an inline CSS stylesheet.
It takes a single mandatory parameter:
THEME
— the Hihglight theme name to import.
It invokes Highlight with the following options:
--print-style
— print stylesheet only.--style=<THEME>
— select theTHEME
colour style (theme).--stdout
— print output result to STDOUT.
The macros encloses Highlight’s CSS output within <style type="text/css">
tags.
This macro can be invoked anywhere in the document and will apply to all Highlight code blocks (one theme per document). The imported theme will not affect code blocks highlighted by pandoc.
Pandoc’s built-in highlighting styles might or might not interfere with imported Highlight themes, depending on the style’s definition ( pandoc’s default style, pygments
, usually doesn’t interfere because it doesn’t set a background color). In some cases, slight adjustments to Highlight’s CSS are required to preserve styling integrity.
Example Files
build-example.bat
— The batch script to build the example file.example.md
— The example source file (markdown + PP macros).example-preprocessed.md
— The PP-preprocessed versionexample.md
.example.html
— The html conversion ofexample-preprocessed.md
via pandoc.example.pb
— The external source code file example (PureBASIC).Highlight.pp
— The required PP macros file.
Ensure that pandoc, PP and Highlight are available on the system PATH
, then execute the build-example.bat
script.
The script issues two commands:
pp -import Highlight.pp example.md > example-preprocessed.md
pandoc example-preprocessed.md -o example.html
The first line invokes pp
and tells it to -import
the contents of Highlight.pp
(so that the macros herein defined are loaded in the current environment) and then process the file example.md
. Since PP outputs to STDOUT
, we redirect (>
) its output to example-preprocessed.md
.
If you open example-preprocessed.md
in a text editor you’ll see how the macros inside example.md
were expanded.
The second line invokes pandoc
and tells it to convert example-preprocessed.md
and write the output (-o
) to the example.html
file (pandoc is smart enough to deduce the desired input and output formats from the files’ extensions).
In a real work scenario, you would carry out the above task with a single command line, by piping (|
) PP’s output directly to pandoc instead of writing it to the example-preprocessed.md
file:
pp -import Highlight.pp example.md | pandoc -f markdown -o example.html
For learning purposes, build-example.bat
writes to disk the intermediate preprocessed markdown file, so that you may examine how the macros are expanded by PP before feeding them to pandoc.
External Links
- http://pandoc.org
- http://cdsoft.fr/pp/
- https://github.com/tajmone/pandoc-goodies/tree/master/pp#highlight
This is the state of things with pandoc v1; with the upcoming release of pandoc version 2 this will change (see #3334) as dynamic loading of syntax definitions will be supported via the new built-in highlighter. Even so, using Highlight with pandoc will still be possible and desirable because of the features offered by Highlight plugins, or because of syntax definitions not available for pandoc.↩