ElementTree - Revision history

Michael Murtaugh: /* Absolutizing Href's and Src's (in complete... still buggy) */

2014-10-06T14:56:32Z

Absolutizing Href's and Src's (in complete... still buggy)

← Older revision		Revision as of 16:56, 6 October 2014
Line 117:		Line 117:
	</source>		</source>

	== Absolutizing Href's and Src's (~~in complete~~... still buggy) ==		== Absolutizing Href's and Src's (incomplete... still buggy) ==

	<source lang="python">		<source lang="python">

Michael Murtaugh: /* Absolutizing Href's and Src's (in complete... still buggy) */

2014-10-06T14:56:17Z

Absolutizing Href's and Src's (in complete... still buggy)

← Older revision		Revision as of 16:56, 6 October 2014
Line 120:		Line 120:

	<source lang="python">		<source lang="python">
			import html5lib
	from urlparse import urljoin		from urlparse import urljoin

			t = html5lib.parse(r.text, treebuilder="etree", namespaceHTMLElements=False)

			def myurljoin (base, href):
			if href.startswith("//"):
			href = urlparse(base).scheme+":"+href
			return urljoin(base, href)

	for link in t.findall(".//*[@href]"):		for link in t.findall(".//*[@href]"):

Michael Murtaugh: /* Getting at the parent */

2014-10-06T14:53:47Z

Getting at the parent

@@ Line 115: / Line 115: @@
 for parent, child in iterparent(tree):
      ... work on parent/child tuple
 </source>

Michael Murtaugh at 15:12, 27 May 2014

2014-05-27T15:12:54Z

@@ Line 98: / Line 98: @@
 print ET.tostring(x, method="text", encoding="utf-8")
 print ET.tostring(x)
 </source>

Michael Murtaugh at 12:44, 27 May 2014

2014-05-27T12:44:15Z

← Older revision		Revision as of 14:44, 27 May 2014
Line 90:		Line 90:

	Source: http://docs.python.org/2/library/xml.etree.elementtree.html#supported-xpath-syntax		Source: http://docs.python.org/2/library/xml.etree.elementtree.html#supported-xpath-syntax

			== ... and back to text ==

			<source lang="python">
			from xml.etree import ElementTree as ET

			print ET.tostring(x, method="text", encoding="utf-8")
			print ET.tostring(x)
			</source>

Michael Murtaugh: /* Reading HTML from "the wild" = */

2014-05-27T12:43:00Z

Reading HTML from "the wild" =

← Older revision		Revision as of 14:43, 27 May 2014
Line 22:		Line 22:
	</source>		</source>

	== Reading HTML from "the wild" ===		== Reading HTML from "the wild" ==

	Use [[html5lib]] to read HTML pages as element trees.		Use [[html5lib]] to read HTML pages as element trees.

Michael Murtaugh: /* Reading some text */

2014-05-27T12:42:38Z

Reading some text

← Older revision		Revision as of 14:42, 27 May 2014
Line 15:		Line 15:

	== Reading some text ==		== Reading some text ==
			The '''.fromstring()''' function can be used to convert HTML or other XML formatted text into an ElementTree.

	<source lang="python">		<source lang="python">
Line 20:		Line 21:
	t = ET.fromstring("<p>Hello world!</p>")		t = ET.fromstring("<p>Hello world!</p>")
	</source>		</source>


	== Reading HTML from "the wild" ===		== Reading HTML from "the wild" ===

Michael Murtaugh at 12:41, 27 May 2014

2014-05-27T12:41:27Z

← Older revision		Revision as of 14:41, 27 May 2014
Line 13:		Line 13:

	While more sophisticated (and faster) libraries for working with HTML/XML exist (namely lxml), python's standard ElementTree implementation is quite capable (and using lxml requires a separate C module to be compiled on your platform which can sometimes complicate installation / distribution of your script).		While more sophisticated (and faster) libraries for working with HTML/XML exist (namely lxml), python's standard ElementTree implementation is quite capable (and using lxml requires a separate C module to be compiled on your platform which can sometimes complicate installation / distribution of your script).

			== Reading some text ==

			<source lang="python">
			from xml.etree import ElementTree as ET
			t = ET.fromstring("<p>Hello world!</p>")
			</source>


			== Reading HTML from "the wild" ===

			Use [[html5lib]] to read HTML pages as element trees.

			<source lang="python">
			import html5lib
			from urllib2 import urlopen
			f = urlopen("http://en.wikipedia.org/wiki/Gunny_sack")
			t = html5lib.parse(f, namespaceHTMLElements=False, treebuilder="etree")
			t.find(".//h1")
			print t
			</source>

	== .iter() ==		== .iter() ==

Michael Murtaugh: /* .find(), .findall() = */

2014-05-27T12:38:21Z

.find(), .findall() =

← Older revision		Revision as of 14:38, 27 May 2014
Line 17:		Line 17:
	The iter() function "walks" over all the elements of a tree.		The iter() function "walks" over all the elements of a tree.

	== .find(), .findall() ===		== .find() and .findall() ==
	ElementTree supports a small subset of the (more extensive) xpath query language:		ElementTree supports a small subset of the (more extensive) xpath query language via the '''.find()''' and '''.findall()''' functions.

	{\|class="wikitable"		{\|class="wikitable"

Michael Murtaugh at 12:37, 27 May 2014

2014-05-27T12:37:50Z

← Older revision		Revision as of 14:37, 27 May 2014
Line 1:		Line 1:
	ElementTree is an interface to XML documents (so like SVG, and HTML when preprocessed with say [[html5lib]]).		ElementTree is an interface to XML documents (so like SVG, and HTML when preprocessed with say [[html5lib]]).

	~~It's kind of~~ mix between a list and a dictionary and gives ~~you~~ relatively easy access to both ~~read~~ (to say "scrape" data from ~~webpage~~) and ~~write~~ (to say alter an SVG or ~~webpage and~~ then ~~re-output it~~).		ET lets you treat documents like a mix between a list and a dictionary and gives relatively easy access to both ''reading'' (to say "scrape" data from webpages) and ''writing'' (to say alter an SVG or fuse together multiple webpages to then resave as text).

	The basic rules:		The basic rules:
Line 14:		Line 14:
	While more sophisticated (and faster) libraries for working with HTML/XML exist (namely lxml), python's standard ElementTree implementation is quite capable (and using lxml requires a separate C module to be compiled on your platform which can sometimes complicate installation / distribution of your script).		While more sophisticated (and faster) libraries for working with HTML/XML exist (namely lxml), python's standard ElementTree implementation is quite capable (and using lxml requires a separate C module to be compiled on your platform which can sometimes complicate installation / distribution of your script).

			== .iter() ==
			The iter() function "walks" over all the elements of a tree.

			== .find(), .findall() ===
	ElementTree supports a small subset of the (more extensive) xpath query language:		ElementTree supports a small subset of the (more extensive) xpath query language: