HTML Agility Pack - using XPath to get a single node - Object Reference not set to an instance of an object
See the question and my original answer on StackOverflowYou can't rely on a developper tools such as FireBug or Chrome, etc... to determine the XPATH for the nodes you're after, as the XPATH given by such tools correspond to the in memory HTML DOM while the Html Agility Pack only knows about the raw HTML sent back by the server.
What you need to do is look visually at what's sent back (or just do a view source). You'll see there is no TBODY element for example. So you want to find anything discriminant, and use XPATH axes for example. Also, your XPATH, even if it worked, would not be very resistant to changes in the document, so you need to find something more "stable" for the scraping to be more future-proof.
Here is a code that seems to work:
HtmlNode node = doc.DocumentNode.SelectSingleNode("//td[@class='dnTableCell']//a[text()='High']/../../td[3]");
This is what it does:
- find a TD element with a CLASS attribute set to 'dnTableCell'. The // token means the search is recursive in the XML hierarchy.
- find an A element that contains a text (inner text) equals to 'High'.
- navigate two parents up (we'll get to the closest TR element)
- select the 3rd TD element from there