w3lib

Python library of web-related functions such as extract base url from HTML snippets, translate entites on HTML strings, convert raw HTTP headers to dicts and vice-versa, construct HTTP auth header, converting HTML pages to unicode, etc.