Package lxml :: Package html :: Module clean |
[hide private]
[frames] | no frames]

# Module clean

source code

A cleanup tool for HTML.

Removes unwanted tags and content. See the Cleaner class for details.

Classes | [hide private]
| unicode str(object='') -> string
| Cleaner Instances cleans the document of each of the possible offending elements. The cleaning is controlled by attributes; you can override attributes in a subclass, or set them in the constructor.

Functions | [hide private]
character |
unichr(i) Return a string of one character with ordinal i; 0 <= i < 256. |
_is_image_dataurl(...) search(string[, pos[, endpos]]) --> match object or None. Scan through string looking for a match, and return a corresponding match object instance. Return None if no position in the string matches. | source code
_is_possibly_malicious_scheme(...) search(string[, pos[, endpos]]) --> match object or None. Scan through string looking for a match, and return a corresponding match object instance. Return None if no position in the string matches. | source code
_is_javascript_scheme(s) | source code
_substitute_whitespace(...) sub(repl, string[, count = 0]) --> newstring Return the string obtained by replacing the leftmost non-overlapping occurrences of pattern in string by the replacement repl. | source code
clean_html(...) | source code
autolink(el, link_regexes=_link_regexes, avoid_elements=_avoid_elements, avoid_hosts=_avoid_hosts, avoid_classes=_avoid_classes) Turn any URLs into links. | source code
_link_text(text, link_regexes, avoid_hosts, factory) | source code
autolink_html(html) Turn any URLs into links. | source code
word_break(el, max_width=40, avoid_elements=_avoid_word_break_elements, avoid_classes=_avoid_word_break_classes, break_character=unichr(0x200b)) Breaks any long words found in the body of the text (not attributes). | source code
word_break_html(html) | source code
_break_text(text, max_width, break_character) | source code
_insert_break(word, width, break_character) | source code

Variables | [hide private]
| basestring = `str, bytes`
| _css_javascript_re = `re.compile(r'`(?is)`expression\s`*`\(.`*?`\)')`
| _css_import_re = `re.compile(r'`(?i)`@\s`*`import')`
| _conditional_comment_re = `re.compile(r'`(?is)`\[if`[`\s\n\r`]``+`.`*?`\]`...``
| _find_styled_elements = `descendant-or-self::*[@style]`
| _find_external_links = `descendant-or-self::a [normalize-space`...``
| clean = `Cleaner()`
| _link_regexes = ``[`re.compile(r'`(?i)``(?P<``body``>`https`?`://`(?P<``host``>``[``...``
| _avoid_elements = ``[``'``textarea``'``,` `'``pre``'``,` `'``code``'``,` `'``head``'``,` `'``select``'``...``
| _avoid_hosts = ``[`re.compile(r'`(?i)`^localhost')`,` re.compile(r'`(?``...``
| _avoid_classes = ``[``'``nolink``'``]``
| _avoid_word_break_elements = ``[``'``pre``'``,` `'``textarea``'``,` `'``code``'``]``
| _avoid_word_break_classes = ``[``'``nobreak``'``]``
| _break_prefer_re = `re.compile(r'`(?i)``[``^`a`-`z`]`')`
| __package__ = `None` hash(x)
| __test__ = ``{``}``

Function Details | [hide private]

### autolink(el, link_regexes=_link_regexes, avoid_elements=_avoid_elements, avoid_hosts=_avoid_hosts, avoid_classes=_avoid_classes) | source code
Turn any URLs into links. It will search for links identified by the given regular expressions (by default mailto and http(s) links). It won't link text in an element in avoid_elements, or an element with a class in avoid_classes. It won't link to anything with a host that matches one of the regular expressions in avoid_hosts (default localhost and 127.0.0.1). If you pass in an element, the element's tail will not be substituted, only the contents of the element.

### autolink_html(html) | source code
Turn any URLs into links. It will search for links identified by the given regular expressions (by default mailto and http(s) links). It won't link text in an element in avoid_elements, or an element with a class in avoid_classes. It won't link to anything with a host that matches one of the regular expressions in avoid_hosts (default localhost and 127.0.0.1). If you pass in an element, the element's tail will not be substituted, only the contents of the element.

### word_break(el, max_width=40, avoid_elements=_avoid_word_break_elements, avoid_classes=_avoid_word_break_classes, break_character=unichr(0x200b)) | source code
Breaks any long words found in the body of the text (not attributes). Doesn't effect any of the tags in avoid_elements, by default `<textarea>` and `<pre>` Breaks words by inserting &#8203;, which is a unicode character for Zero Width Space character. This generally takes up no space in rendering, but does copy as a space, and in monospace contexts usually takes up space. See http://www.cs.tut.fi/~jkorpela/html/nobr.html for a discussion

Variables Details | [hide private]

  ### _conditional_comment_re Value:
      re.compile(r'(?is)\[if[\s\n\r]+.*?\][\s\n\r]*>')

  ### _find_external_links Value:
      descendant-or-self::a  [normalize-space(@href) and substring(normalize
      -space(@href),1,1) != '#'] |descendant-or-self::x:a[normalize-space(@h
      ref) and substring(normalize-space(@href),1,1) != '#']

  ### _link_regexes Value:
      [re.compile(r'(?i)(?P<body>https?://(?P<host>[a-z0-9\._-]+)(?:/[/-_\.,
      a-z0-9%&\?;=~]*)?(?:\([/-_\.,a-z0-9%&\?;=~]*\))?)'),
       re.compile(r'(?i)mailto:(?P<body>[a-z0-9\._-]+@(?P<host>[a-z0-9_\.-]+
      [a-z]))')]

  ### _avoid_elements Value:
      ['textarea', 'pre', 'code', 'head', 'select', 'a']

  ### _avoid_hosts Value:
      [re.compile(r'(?i)^localhost'),
       re.compile(r'(?i)\bexample\.(?:com|org|net)$'),
       re.compile(r'^127\.0\.0\.1$')]