Building Resilient Systems on AWS: Learn how to design and implement a resilient, highly available, fault-tolerant infrastructure on AWS.

Python html5lib Skipped Elements

By David Walsh on August 8, 2013

I've been working on some interesting python stuff at Mozilla and one task recently called for called for rending a page and then finding elements with a URL attribute value (like img[src] or a[href]) and ensuring they become absolute URLs. One problem I encountered when using html5lib was that LINK and IMG elements were being skipped when I tokenized the HTML. After browsing through the html5lib source code, I found a variable called voidElements which included both LINK and IMAGE:

voidElements = frozenset((
    "base",
    "command",
    "event-source",
    "link",
    "meta",
    "hr",
    "br",
    "img",
    "embed",
    "param",
    "area",
    "col",
    "input",
    "source"
))

When I commented out those two elements, they were found upon next run of my routine, meaning their presence in the set were causing me problems. Here's how I skirted the issue:

new_void_set = set()
for item in html5lib_constants.voidElements:
	new_void_set.add(item)
new_void_set.remove('link')
new_void_set.remove('img')
html5lib_constants.voidElements = frozenset(new_void_set)

Since voidElements is a frozenset, I couldn't simply remove LINK and IMG, so I needed to create a new frozenset without those elements. Let me know if there's a more python-ish way of creating this frozen set. In an event, delving into the deep recesses of html5lib paid off and I accomplished the goal!

Recent Features

By David WalshSeptember 19, 2011
Introducing MooTools Templated
One major problem with creating UI components with the MooTools JavaScript framework is that there isn't a great way of allowing customization of template and ease of node creation. As of today, there are two ways of creating: new Element Madness The first way to create UI-driven...
By David WalshMay 7, 2012
Detect DOM Node Insertions with JavaScript and CSS Animations
I work with an awesome cast of developers at Mozilla, and one of them in Daniel Buchner. Daniel's shared with me an awesome strategy for detecting when nodes have been injected into a parent node without using the deprecated DOM Events API.

Incredible Demos

By David WalshApril 8, 2010
Create a Trailing Mouse Cursor Effect Using MooTools
Remember the old days of DHTML and effects that were an achievement to create but had absolutely no value? Well, a trailing mouse cursor script is sorta like that. And I'm sorta the type of guy that creates effects just because I can.
By David WalshSeptember 19, 2011
Introducing MooTools Templated
One major problem with creating UI components with the MooTools JavaScript framework is that there isn't a great way of allowing customization of template and ease of node creation. As of today, there are two ways of creating: new Element Madness The first way to create UI-driven...

Discussion

Darren Hickling
You could use list comprehension to filter the elements instead; something like:

html5lib_constants.voidElements = frozenset([e for e in html5lib_constants.voidElements if e not in [“link”, “img”]])

Sounds like it would be useful to add void element overriding as a feature to the library in the future, though.