# Live Documentation in the Notebook

Live Documentation in the Notebook

Conversion of HTML (output by Sphinx or docutils) to Sage worksheet txt file.

This takes an HTML document, i.e., Sage documentation, and returns it in the editable format (notebook worksheet format with evaluable examples). It also returns a string representing the CSS link for the document. The HTML parser is setup to return only the body of the HTML documentation page and to re-format Sage examples and type-setting.

This module contains three classes:

- `sagenb.notebook.docHTMLProcessor.genericHTMLProcessor`: gathers all the common methods of the other two classes.
- `sagenb.notebook.docHTMLProcessor.SphinxHTMLProcessor`: translates HTML file generated by Sphinx into a worksheet text file
- `sagenb.notebook.docHTMLProcessor.docutilsHTMLProcessor`: translates HTML file generated by docutils `rst2html` command into a worksheet text file

Note

This extension of htmllib.HTMLParser was partly inspired by Mark Pilgrim’s ‘Dive Into Python’ examples.

AUTHORS:

- Dorian Raymer (2006): first version
- William Stein (2007-06-10): rewrite to work with twisted Sage notebook
- Mike Hansen (2008-09-27): Rewrite to work with Sphinx HTML documentation
- Sebastien Labbe (2011-01-15): Added a new class named docutilsHTMLProcessor used for translating the html output of the rst2html docutils command run on a rst file into worksheet text file. Also added a new class named genericHTMLProcessor which gathers the common methods of both docutilsHTMLProcessor and SphinxHTMLProcessor classes. Added lots of doctests to make its coverage 100% doctested.

EXAMPLES:

Process the output of docutils `rst2html` command:

    sage: rst = ""
    sage: rst += "Additions in Sage\n"
    sage: rst += "-----------------\n"
    sage: rst += "\n"
    sage: rst += "Let's do easy computations with Sage::\n"
    sage: rst += "\n"
    sage: rst += "    s" + "age: 4 + 3\n"
    sage: rst += "    7\n"
    sage: rst += "    s" + "age: 1 - 2\n"
    sage: rst += "    -1\n"
    sage: rst += "\n"
    sage: rst += "Let's do `x^2`::\n"
    sage: rst += "\n"
    sage: rst += "    s" + "age: x^2\n"
    sage: rst += "    x^2\n"
    sage: from docutils.core import publish_string
    sage: html = publish_string(rst, writer_name='html')
    sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
    sage: p = docutilsHTMLProcessor()
    sage: txt = p.process_doc_html(html)
    sage: len(txt)
    191
    sage: print(txt)
    <h1 class="title">Additions in Sage</h1>
    <BLANKLINE>
    <BLANKLINE>
    <BLANKLINE>
    <p>Let's do easy computations with Sage:</p>
    <BLANKLINE>
    {{{id=0|
    4 + 3
    ///
    7
    }}}
    <BLANKLINE>
    {{{id=1|
    1 - 2
    ///
    -1
    }}}
    <BLANKLINE>
    <p>Let's do $x^2$:</p>
    <BLANKLINE>
    {{{id=2|
    x^2
    ///
    x^2
    }}}
    <BLANKLINE>
    <BLANKLINE>

WARNING:

> Input strings must be unicode.

class `sagenb.notebook.docHTMLProcessor.``SphinxHTMLProcessor`

  Bases: `sagenb.notebook.docHTMLProcessor.genericHTMLProcessor`

  `end_div`()

    Once we end the highlighted div, convert all of the pieces to cells.

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings\n']
        sage: p.keep_data = True
        sage: attrs = [('class', 'highlight')]
        sage: p.start_div(attrs)
        sage: p.start_pre([])
        sage: sprompt = 'sa' + 'ge' + ': '    # to avoid problems with doctest script
        sage: p.handle_data('%s4+4\n8\n%sx^2\nx^2\n' % (sprompt, sprompt))
        sage: p.end_pre()
        sage: p.end_div()
        sage: print(p.all_pieces)
        a lot of stuff done bunch of tmp strings
        {{{id=0|
        4+4
        ///
        8
        }}}
        <BLANKLINE>
        {{{id=1|
        x^2
        ///
        x^2
        }}}
        sage: p.temp_pieces
        []
        sage: p.in_highlight_div
        False

        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: attrs = [('class', 'something-else')]
        sage: p.start_div(attrs)
        sage: p.handle_data('some data')
        sage: p.end_div()
        sage: print(p.all_pieces)
        a lot of stuff done
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<div class="something-else">', 'some data', u'</div>']
        sage: p.in_highlight_div
        False

  `end_form`()

    Ignore all of the pieces since we started the form.

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.end_form()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        []

  `end_pre`()

    Ignore tag </pre> when inside highligh div.

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.in_highlight_div = True
        sage: p.end_pre()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']

        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.in_highlight_div = False
        sage: p.end_pre()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'</pre>']

  `end_span`()

    Ignore all spans that occur within highlighted blocks

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.in_highlight_div = True
        sage: p.end_span()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']

        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.in_highlight_div = False
        sage: p.end_span()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'</span>']

  `false_positive_input_output_cell`(cell_piece)

    Return the untouched html string of a false positive input output cell.

    A false positive input-output cell come from a block of code which doesn’t start with the sage prompt string ‘sage:’ or the Python prompt ‘>>>’.

    INPUT:

    - `cell_piece` - string, a cell piece

    OUPUT:

    > string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: s = "sage -rst2html -h"
        sage: print(p.false_positive_input_output_cell(s))
        <div class="highlight"><pre>
        sage -rst2html -h
        </pre></div>

  `reset`()

    Initialize necessary variables. Called by `HTMLParser.__init__()`.

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: d = SphinxHTMLProcessor()    #indirect doctest
        sage: d.keep_data
        False
        sage: d.in_highlight_div
        False
        sage: d.temp_pieces
        []
        sage: d.all_pieces
        u''
        sage: d.cellcount
        0

  `start_div`(attrs)

    Find out if we are starting a highlighted div.

    Once we hit the <div> tag in a highlighted block, hand of all of the pieces we’ve encountered so far and ignore the tag.

    INPUT:

    - `attrs` - list of tuple

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: attrs = [('class', 'highlight')]
        sage: p.start_div(attrs)
        sage: p.all_pieces
        u'a lot of stuff done bunch of tmp strings'
        sage: p.temp_pieces
        []
        sage: p.in_highlight_div
        True

        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: attrs = [('class', 'something-else')]
        sage: p.start_div(attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<div class="something-else">']
        sage: p.in_highlight_div
        False

  `start_form`(attrs)

    Hand of everything we’ve accumulated so far.

    Forms are ignored.

    INPUT:

    - `attrs` - list of tuple

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: attrs = []
        sage: p.start_form(attrs)
        sage: p.all_pieces
        u'a lot of stuff done bunch of tmp strings'
        sage: p.temp_pieces
        []

  `start_pre`(attrs)

    Ignore tag <pre> when inside highligh div.

    INPUT:

    - `attrs` - list of tuple

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.in_highlight_div = True
        sage: attrs = []
        sage: p.start_pre(attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.in_highlight_div = False
        sage: attrs = []
        sage: p.start_pre(attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<pre>']

  `start_span`(attrs)

    Ignore all spans that occur within highlighted blocks

    INPUT:

    - `attrs` - list of tuple

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.in_highlight_div = True
        sage: attrs = []
        sage: p.start_span(attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: p = SphinxHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.in_highlight_div = False
        sage: attrs = []
        sage: p.start_span(attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<span>']

class `sagenb.notebook.docHTMLProcessor.``docutilsHTMLProcessor`

  Bases: `sagenb.notebook.docHTMLProcessor.genericHTMLProcessor`

  Translates output of the docutils parser rst2html into notebook text.

  EXAMPLES:

      sage: rst = ""
      sage: rst += "Additions in Sage\n"
      sage: rst += "-----------------\n"
      sage: rst += "\n"
      sage: rst += "Let's do easy computations with Sage::\n"
      sage: rst += "\n"
      sage: rst += "    s" + "age: 4 + 3\n"
      sage: rst += "    7\n"
      sage: rst += "    s" + "age: 1 - 2\n"
      sage: rst += "    -1\n"
      sage: rst += "\n"
      sage: rst += "Let's do `x^2`::\n"
      sage: rst += "\n"
      sage: rst += "    s" + "age: x^2\n"
      sage: rst += "    x^2\n"
      sage: from docutils.core import publish_string
      sage: html = publish_string(rst, writer_name='html')
      sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
      sage: p = docutilsHTMLProcessor()
      sage: txt = p.process_doc_html(html)
      sage: len(txt)
      191
      sage: print(txt)
      <h1 class="title">Additions in Sage</h1>
      <BLANKLINE>
      <BLANKLINE>
      <BLANKLINE>
      <p>Let's do easy computations with Sage:</p>
      <BLANKLINE>
      {{{id=0|
      4 + 3
      ///
      7
      }}}
      <BLANKLINE>
      {{{id=1|
      1 - 2
      ///
      -1
      }}}
      <BLANKLINE>
      <p>Let's do $x^2$:</p>
      <BLANKLINE>
      {{{id=2|
      x^2
      ///
      x^2
      }}}
      <BLANKLINE>
      <BLANKLINE>

  `end_cite`()

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.start_cite([])
        sage: p.handle_data('x^2')
        sage: p.end_cite()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings',  u'$', 'x^2',  u'$']

  `end_div`()

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.end_div()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']

  `end_pre`()

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: attrs = [('class', 'literal-block')]
        sage: p.start_pre(attrs)
        sage: sprompt = 'sa' + 'ge' + ': '    # to avoid problems with doctest script
        sage: p.handle_data('%s4+4\n8\n%sx^2\nx^2\n' % (sprompt, sprompt))
        sage: p.end_pre()
        sage: print(p.all_pieces)
        a lot of stuff done bunch of tmp strings
        {{{id=0|
        4+4
        ///
        8
        }}}
        <BLANKLINE>
        {{{id=1|
        x^2
        ///
        x^2
        }}}
        sage: p.temp_pieces
        []
        sage: p.in_pre_litteral_block
        False

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: attrs = [('class', 'something-else')]
        sage: p.start_pre(attrs)
        sage: p.handle_data('some data')
        sage: p.end_pre()
        sage: print(p.all_pieces)
        a lot of stuff done
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<pre class="something-else">', 'some data', u'</pre>']
        sage: p.in_pre_litteral_block
        False

  `end_script`()

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.end_script()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data
        True

  `false_positive_input_output_cell`(cell_piece)

    Return the untouched html string of a false positive input output cell.

    A false positive input-output cell come from a block of code which doesn’t start with the sage prompt string ‘sage:’ or the Python prompt ‘>>>’.

    INPUT:

    - `cell_piece` - string, a cell piece

    OUPUT:

    > string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: s = "sage -rst2html -h"
        sage: print(p.false_positive_input_output_cell(s))
        <pre class="literal-block">
        sage -rst2html -h
        </pre>

  `reset`()

    Initialize necessary variables. Called by `HTMLParser.__init__()`.

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: d = docutilsHTMLProcessor()    #indirect doctest
        sage: d.keep_data
        False
        sage: d.in_pre_litteral_block
        False
        sage: d.in_div_footer_block
        False
        sage: d.temp_pieces
        []
        sage: d.all_pieces
        u''
        sage: d.cellcount
        0

  `start_cite`(attrs)

    INPUT:

    - `attrs` - list of tuple

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: attrs = []
        sage: p.start_cite(attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings',  u'$']

  `start_div`(attrs)

    INPUT:

    - `attrs` - list of tuple

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: attrs = [('class', 'document'), ('id', 'title')]
        sage: p.start_div(attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']

  `start_pre`(attrs)

    INPUT:

    - `attrs` - list of tuple

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: attrs = [('class', 'literal-block')]
        sage: p.start_pre(attrs)
        sage: p.all_pieces
        u'a lot of stuff done bunch of tmp strings'
        sage: p.temp_pieces
        []
        sage: p.in_pre_litteral_block
        True

        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: attrs = [('class', 'something-else')]
        sage: p.start_pre(attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<pre class="something-else">']
        sage: p.in_pre_litteral_block
        False

  `start_script`(attrs)

    INPUT:

    - `attrs` - list of tuple

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: attrs = [('type', 'text/x-mathjax-config')]
        sage: p.start_script(attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data
        False

class `sagenb.notebook.docHTMLProcessor.``genericHTMLProcessor`

  Bases: `HTMLParser.HTMLParser`

  This class gathers the methods that are common to both classes `sagenb.notebook.SphinxHTMLProcessor` and `sagenb.notebook.docutilsHTMLProcessor` .

  `end_body`()

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.end_body()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']

  `end_html`()

    INPUT:

    - `data` - string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.end_html()
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings']

  `get_cellcount`()

    Return the current cell count and increment it by one.

    OUTPUT:

    - an int

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: d = docutilsHTMLProcessor()
        sage: d.get_cellcount()
        0
        sage: d.get_cellcount()
        1

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: d = SphinxHTMLProcessor()
        sage: d.get_cellcount()
        0
        sage: d.get_cellcount()
        1

  `hand_off_temp_pieces`(piece_type)

    To separate the documentation’s content from the Sage examples, everything is split into one of two cell types. This method puts the current `self.temp_pieces` into `self.all_pieces`.

    INPUT:

    - `piece_type` - a string; indicates the type of and how to process the current `self.temp_pieces`. It can be one of the following:
      - `"to_doc_pieces"` - put temp_pieces in all_pieces
      - `"ignore"` - delete temp_pieces
      - `"to_cell_pieces"` - translate temp_pieces into cells and put it in all_pieces

    EXAMPLES:

    Move temporary pieces to all pieces:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.hand_off_temp_pieces('to_doc_pieces')
        sage: p.all_pieces
        u'a lot of stuff done bunch of tmp strings'
        sage: p.temp_pieces
        []

    Ignore temporary pieces:

        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.hand_off_temp_pieces('ignore')
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        []

    Translate temporary pieces (starting with sage prompt) into cells:

        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['sage'+': 4+4\n', '8\n', 'sage'+': 9-4\n', '5\n']
        sage: p.hand_off_temp_pieces('to_cell_pieces')
        sage: print(p.all_pieces)
        a lot of stuff done
        {{{id=0|
        4+4
        ///
        8
        }}}
        <BLANKLINE>
        {{{id=1|
        9-4
        ///
        5
        }}}
        sage: p.temp_pieces
        []

    Translate temporary pieces (not starting with sage prompt) into cells:

        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.hand_off_temp_pieces('to_cell_pieces')
        sage: print(p.all_pieces)
        a lot of stuff done <pre class="literal-block">
        bunch of tmp strings
        </pre>
        sage: p.temp_pieces
        []

  `handle_charref`(ref)

    INPUT:

    - `ref` - string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.handle_charref('160')
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'&#160;']

  `handle_comment`(data)

    INPUT:

    - `data` - string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.handle_comment('important comment')
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<!--important comment-->']

  `handle_data`(data)

    INPUT:

    - `data` - string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.handle_data('some important data')
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', 'some important data']

  `handle_decl`(text)

    INPUT:

    - `data` - string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.handle_decl('declaration')
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<!declaration>']

  `handle_endtag`(tag)

    introduced when replacing SGMLParser by HTMLParser

  `handle_entityref`(ref)

    INPUT:

    - `ref` - string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.handle_entityref('160')
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'&160']

  `handle_pi`(text)

    Handle processing instructions

    INPUT:

    - `text` - string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.handle_pi('instructions')
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<?instructions>']

  `handle_starttag`(tag, attrs)

    introduced when replacing SGMLParser by HTMLParser

  `process_cell_input_output`(cell_piece)

    Process and return a `cell_piece`.

    All divs with CSS class=”highlight” (if generated with Sphinx) or class=”literal-block” (if generated with docutils) contain code examples. They include

    - Models of how the function works. These begin with, e.g., ‘INPUT:’ and are re-styled as divs with class=”usage_model”.
    - Actual Sage input and output. These begin with ‘sage:’. The input and output are separated according to the Notebook edit format.

    INPUT:

    - `cell_piece` - a string; a cell piece

    OUTPUT:

    - a string; the processed cell piece

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: s = "s" + "age: 4 + 4\n8"    # avoid the doctest script to parse "sage:"
        sage: p.process_cell_input_output(s)
        u'\n{{{id=0|\n4 + 4\n///\n8\n}}}\n\n'
        sage: print(p.process_cell_input_output(s))
        {{{id=1|
        4 + 4
        ///
        8
        }}}

        sage: s = "age: 4 + 4\n8"
        sage: print(p.process_cell_input_output(s))
        <pre class="literal-block">
        age: 4 + 4
        8
        </pre>

        sage: s = '&gt;'*3 + " 4 + 4\n8"
        sage: print(p.process_cell_input_output(s))
        {{{id=2|
        4 + 4
        ///
        8
        }}}

        sage: s = "s" + "age: 4 + 4\n8\ns" + "age: 2 + 2\n4"
        sage: print(p.process_cell_input_output(s))
        {{{id=3|
        4 + 4
        ///
        8
        }}}
        <BLANKLINE>
        {{{id=4|
        2 + 2
        ///
        4
        }}}

  `process_doc_html`(doc_in)

    Returns processed HTML input as HTML output. This is the only method that needs to be called externally.

    INPUT:

    - `doc_in` - a string containing properly formed HTML

    OUTPUT:

    - a string; the processed HTML

    EXAMPLES:

        sage: rst = ""
        sage: rst += "Title\n"
        sage: rst += "-----\n"
        sage: rst += "n"
        sage: rst += "Some text\n"
        sage: from docutils.core import publish_string
        sage: html = publish_string(rst, writer_name='html')
        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: txt = p.process_doc_html(html)
        sage: len(txt)
        51
        sage: txt
        u'<h1 class="title">Title</h1>\n\n<p>nSome text</p>\n\n\n\n'

  `start_body`(attrs)

    Set `self.keep_data` to True upon finding the opening body tag.

    INPUT:

    - `attrs` - a string:string dictionary containing the element’s attributes

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import SphinxHTMLProcessor
        sage: d = SphinxHTMLProcessor()
        sage: d.keep_data
        False
        sage: d.start_body(None)
        sage: d.keep_data
        True

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: d = docutilsHTMLProcessor()
        sage: d.keep_data
        False
        sage: d.start_body(None)
        sage: d.keep_data
        True

  `unknown_endtag`(tag)

    INPUT:

    - `tag` - string

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: p.unknown_endtag('head')
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'</head>']

  `unknown_starttag`(tag, attrs)

    INPUT:

    - `tag` - string
    - `attrs` - list of tuples

    EXAMPLES:

        sage: from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor
        sage: p = docutilsHTMLProcessor()
        sage: p.all_pieces = 'a lot of stuff done '
        sage: p.temp_pieces = ['bunch ', 'of ', 'tmp ', 'strings']
        sage: p.keep_data = True
        sage: tag = 'style'
        sage: attrs = [('type', 'text/css')]
        sage: p.unknown_starttag(tag, attrs)
        sage: p.all_pieces
        'a lot of stuff done '
        sage: p.temp_pieces
        ['bunch ', 'of ', 'tmp ', 'strings', u'<style type="text/css">']