.NET Html Agility Pack: How to use malformed HTML just like it was well-formed XML...

!! Update 06/08/18 !! The home of Html Agility Pack has changed again (hopefully for the last time...). The project is now located at CodePlex, available here:
http://www.codeplex.com/Wiki/View.aspx?ProjectName=htmlagilitypack

Comments and questions are welcome!
Simon.

Here is an agile HTML parser that builds a read/write DOM and supports plain XPATH or XSLT. It is an assembly that allows you to parse "out of the web" HTML files. The parser is very tolerant with "real world" malformed HTML. The object model is very similar to what proposes System.Xml, but for HTML documents (or streams).

Sample applications:

• Page fixing or generation. You can fix a page the way you want, modify the DOM, add nodes, copy nodes, you name it.
• Web scanners. You can easily get to img/src or a/hrefs with a bunch XPATH queries.
• Web scrapers. You can easily scrap any existing web page into an RSS feed for example, with just an XSLT file serving as the binding. An example of this is provided.

There is no dependency on anything else than .Net's XPATH implementation. There is no dependency on Internet Explorer's dll or tidy or anything like that. There is also no adherence to XHTML or XML, although you can actually produce XML using the tool.

For example, here is how you would fix all hrefs in an HTML file:

HtmlDocument doc = new HtmlDocument();
doc.Load("file.htm");
foreach(HtmlNode link in doc.DocumentElement.SelectNodes("//a[@href"])
{
HtmlAttribute att = link["href"];
att.Value = FixLink(att);
}
doc.Save("file.htm");

You can still download the .NET Framework 1.1 version here, and for the .NET Framework 2.0 version, please go to CodePlex (link above)

Note: This page was previously located at http://blogs.msdn.com/smourier/archive/2003/06/04/8265.aspx