Popular:
 Building Resilient Systems on AWS: Learn how to design and implement a resilient, highly available, fault-tolerant infrastructure on AWS.

Python html5lib Skipped Elements

By  on  

I've been working on some interesting python stuff at Mozilla and one task recently called for called for rending a page and then finding elements with a URL attribute value (like img[src] or a[href]) and ensuring they become absolute URLs.  One problem I encountered when using html5lib was that LINK and IMG elements were being skipped when I tokenized the HTML.  After browsing through the html5lib source code, I found a variable called voidElements which included both LINK and IMAGE:

voidElements = frozenset((
    "base",
    "command",
    "event-source",
    "link",
    "meta",
    "hr",
    "br",
    "img",
    "embed",
    "param",
    "area",
    "col",
    "input",
    "source"
))

When I commented out those two elements, they were found upon next run of my routine, meaning their presence in the set were causing me problems.  Here's how I skirted the issue:

new_void_set = set()
for item in html5lib_constants.voidElements:
	new_void_set.add(item)
new_void_set.remove('link')
new_void_set.remove('img')
html5lib_constants.voidElements = frozenset(new_void_set)

Since voidElements is a frozenset, I couldn't simply remove LINK and IMG, so I needed to create a new frozenset without those elements.  Let me know if there's a more python-ish way of creating this frozen set.  In an event, delving into the deep recesses of html5lib paid off and I accomplished the goal!

Request Metrics real user monitoring
Request Metrics real user monitoring
Request Metrics real user monitoring
Request Metrics real user monitoring

Recent Features

  • By
    I’m an Impostor

    This is the hardest thing I've ever had to write, much less admit to myself.  I've written resignation letters from jobs I've loved, I've ended relationships, I've failed at a host of tasks, and let myself down in my life.  All of those feelings were very...

  • By
    Welcome to My New Office

    My first professional web development was at a small print shop where I sat in a windowless cubical all day. I suffered that boxed in environment for almost five years before I was able to find a remote job where I worked from home. The first...

Incredible Demos

  • By
    MooTools & Printing – Creating a Links Table of Contents

    One detail we sometimes forget when considering print for websites is that the user cannot see the URLs of links when the page prints. While showing link URLs isn't always important, some websites could greatly benefit from doing so. This tutorial will show you...

  • By
    Animated AJAX Record Deletion Using MooTools

    I'm a huge fan of WordPress' method of individual article deletion. You click the delete link, the menu item animates red, and the item disappears. Here's how to achieve that functionality with MooTools JavaScript. The PHP - Content & Header The following snippet goes at the...

Discussion

  1. Darren Hickling

    You could use list comprehension to filter the elements instead; something like:

    html5lib_constants.voidElements = frozenset([e for e in html5lib_constants.voidElements if e not in [“link”, “img”]])

    Sounds like it would be useful to add void element overriding as a feature to the library in the future, though.

Wrap your code in <pre class="{language}"></pre> tags, link to a GitHub gist, JSFiddle fiddle, or CodePen pen to embed!