Displaying Selected YouTube Data API Thumbnails On A Web Page Via ASP.NET Web Forms

Previously, I blogged about “Displaying Selected YouTube Video Thumbnails On An ASP.NET Web Forms Page,” when you know the video IDs of the thumbnails you want to hyperlink.

A reader recently asked me how to hyperlink YouTube video thumbnails based on searching for a keyword. I promised to address that, so here goes.

Interestingly enough, searching the YouTube Data API is accomplished in a REST-like manner quite similar to the methodology I used for shortening URLs in ASP.NET via the bit.ly API.

  • Form a simple request URL to the YouTube Data API that contains the appropriate search parameters;
  • Use a WebRequest to send that URL to Google, which returns an XML document with results;
  • Use WebResponse to dump that stream into an XmlDocument;
  • Use XPath and XmlNode‘s SelectNodes method to recursively get the thumbnails from each entry; and
  • Bind up a pile of Hyperlink controls, which are added dynamically to a Panel control.

Sounds more complicated than it actually is. Let’s do it.

Overview Of The YouTube Data API

First step in leveraging the YouTube Data API: Get a Youtube Data API developer key.

As of this writing, you can keyword search query the YouTube Data API without supplying a developer key and get a useful response. However, Google wants you to supply a key, and a time may come where you must supply one. So take the 30 seconds required to register a key.

With our developer key in hand, we can proceed to build a simple keyword query. In this case, we’re going to get the top 12 videos that meet the search criteria “Miss Maine.”

For our purposes, “top 12 videos” means the 12 videos YouTube deems most relevant to the exact search term “Miss Maine.” In other words, our search is going to work exactly as though we had typed “Miss Maine” (including the double quotes) into the search box at youtube.com; the results we get back should be the same as the first 12 results we would get via a default search on youtube.com. (For the usual vague reasons, this won’t always be the case, but the results sets will be similar.)

The YouTube Data API tells us that, to do a search query, we can GET a request to its servers, with our search parameters as querystring variables, and Google will return to us an XML document containing records that match our query.

Because we’re not requesting user-specific data, we don’t need to authenticate our requests; as previously mentioned, we don’t even need to send along our developer key. (But, again, being courteous and thorough, we will send it, as requested.)

So, the methodology we use to get our records is exactly the same as we used to shorten a URL via the bit.ly API; we’re going to create a URL containing all our parameters, then make a WebRequest to the YouTube Data API; we’ll get its returned XML via a WebResponse, and push that into an XmlDocument, from which we can extract the information we want.

As we build this solution, let’s keep some non-technical things in mind:

  • Many of the videos on YouTube aren’t very good, in the aesthetic, intellectual or civil senses. In other words, much of what is on YouTube is terrible.
  • A sizable number of YouTube videos are spam.
  • It’s de rigueur to keyword spam video descriptions, especially if the video is spam.
  • Trolling is mandatory for YouTube video comments, and you’ll probably be linking directly to that.

We can mitigate the damage, somewhat, by applying the smartSearch filter, being very specific with our search term, including specific words we don’t want returned in our results, restricting the categories in which we want to search … in other words, being as specific as we can about what we want to see.

However, I’ll bet a dollar to doughnuts that if you request any sizable result set on any generic search term, your results set is going to contain items you wish weren’t in there. That’s just the nature of the thing, and there’s little that can be done about it.

Querying The YouTube Data API

There are three distinct phases in our solution: query the API, get the information we need from its response, and put that information on the page. Therefore, we will use three distinct functions and subroutines to do the work.

First up, a function to submit our query. It accepts as arguments our developer key and a formatted querystring, and it puts the response into an XML document for us. If the request fails, it will report as much in a Label control, and return an empty XML document.

Function QueryYouTubeDataAPI(ByVal strAPIKey As String, ByVal strQuery As String) As XmlDocument
	'Returns empty XMLDocument on failure, results XML on success
	'This function requires your page to have a label control named lblStatus for error reporting

	'build URL to shorten method resource
	Dim strUri As New StringBuilder("https://gdata.youtube.com/feeds/api/videos?")
	strUri.Append("v=2")
	strUri.Append("&")
	strUri.Append(strQuery)
	strUri.Append("&key=")
	strUri.Append(Server.HtmlEncode(strAPIKey))
	strUri.Append("&prettyprint=true") 'adds line breaks & white space to response; useful for debugging

	Dim objRequest As HttpWebRequest
	Dim objResponse As WebResponse
	Dim objXML As New XmlDocument() 'This is the document the function will return

	Try
		'create request for shorten resource
		objRequest = WebRequest.Create(strUri.ToString)
		'since we are passing querystring variables, our method is get
		objRequest.Method = "GET"
		'act as though we are sending a form
		objRequest.ContentType = "application/x-www-form-urlencoded"
		'don't wait for a 100 Continue HTTP response
		objRequest.ServicePoint.Expect100Continue = False
		'since we are using get, we need not send a request body; set content-length to 0
		objRequest.ContentLength = 0

		'read the Data API response into XML document
		objResponse = objRequest.GetResponse()
		objXML.Load(objResponse.GetResponseStream())
	Catch ex As Exception
		lblStatus.Text = "Error querying YouTube Data API. Message: " & ex.Message
	End Try

	'send XML Document
	Return objXML
End Function
A note about error trapping: The error trapping I have used here does not check whether or not the YouTube Data API returns a sensible result. It only checks that we formed a request and the API responded.

If there is a problem with the query string you send to the YouTube Data API, or your request is somehow malformed, the API will return to you the 25 most popular videos for the day, as of the time of the request.

You can check to see if the YouTube Data API is seeing your request properly by looking for the /feed/link rel=’self’ node:

<feed xmlns='http://www.w3.org/2005/Atom' xmlns:app='http://www.w3.org/2007/app' xmlns:media='http://search.yahoo.com/mrss/' xmlns:openSearch='http://a9.com/-/spec/opensearch/1.1/' xmlns:gd='http://schemas.google.com/g/2005' xmlns:yt='http://gdata.youtube.com/schemas/2007' gd:etag='W/&quot;CkYNSH07cSp7I2A9WhVSGEg.&quot;'>
	<!-- ... previous nodes -->
	<link rel='self' type='application/atom+xml' href='https://gdata.youtube.com/feeds/api/videos?q=%22Miss+Maine%22&amp;start-index=1&amp;max-results=12&amp;duration=short&amp;safeSearch=strict&amp;orderby=relevance_lang_en&amp;v=2'/>
	<!-- additional nodes ... -->
</feed>

If the URL in that node doesn’t resemble the one you sent, most likely your querystring is malformed; but possibly, there are other errors in your request URL.

Creating The Query String

You’ll notice that I am not very specific about the query string argument passed to the previous function. That’s because I want to give you maximum flexibility, and the best way to do that is to allow for a free-form query string to be supplied.

So let’s look at how I’ll build my sample query string. I’ll do that, of course, with key-value pairs. That is, I will specify the parameter I want to send, and set it as equal to something.

First up, the search term. We’re going to use “Miss Maine.” So, the first part of our query string is:

q="Miss Maine"
Note that the YouTube Data API says all parameters should be URL encoded. Our function will take care of that for us when we create the WebRequest, so don’t bother encoding your query string in advance, or you’ll get errors. Just pass it to the function as natural text.

Technically, we can stop here and let the default API request variables take over. But I want to be more specific about my search results, so let’s add some more parameters.

Next, I’ll limit my responses to 12 records, by appending that key-value pair to my string:

q="Miss Maine"&max-results=12

I’d like only short (under 4 minutes) videos:

q="Miss Maine"&max-results=12&duration=short

To cull at least some offensive material, I’ll use strict safeSearch:

q="Miss Maine"&max-results=12&duration=short&safeSearch=strict

Finally, I’m going to specifically ask for English language videos that best match my search term. So my completed query string is:

q="Miss Maine"&max-results=12&duration=short&safeSearch=strict&orderby=relevance_lang_en
You may be wondering why I am not using the GData .NET Library to accomplish this search. That’s an excellent option in general, but it involves installing components on your local machine and Web server.

For many Web developers, adding DLLs to a Web server or installing assemblies on a workstation can be problematic, if not prohibited outright. So I want to use as many built-in resources as possible. However, if you can install the GData API, consider going that route.

Examining The XML Response

Assuming our request goes through — that is, our Web server actually delivers the request to YouTube, which in turn sends a response back — we now have an XML document in storage that contains the information we requested. (Or, in the event of a malformed request, the default results set previously described.)

You can see the XML document returned for my demo query string here: http://www.dougv.net/demos/youtube_data_api_thumbs/videos.xml

In our case, we want some specific data for each video:

  • Its title;
  • the default thumbnail; and
  • the URL to its player page on youtube.com.

We get back lots more useful information in the XML response: The name of the uploader, the run time of the video in seconds, its likes and dislikes count, the categories and keywords assigned to the video, the date and time it was uploaded, its view count, etc. But for this solution, we’ll only use the video’s title, thumbnail and URL.

Although the actual XML document contains much more data per entry, for our purposes, the relevant structure of the document looks like this (I’ve stripped out extraneous attributes):

<?xml version='1.0' encoding='UTF-8'?>
<feed>	
	<entry>
		<title></title>
		<link rel='alternate' />
		<media:group>
			<media:thumbnail />
		</media:group>
	</entry>
</feed>

In practice, the parts of an entry (that is, an individual record) that we want to use look like this:

<entry gd:etag='W/&quot;DU8AQ347eCp7I2A9WhRWEEQ.&quot;'>
	<title>Miss Maine USA 2011</title>
	<link rel='alternate' type='text/html' href='https://www.youtube.com/watch?v=STm4GUqwLVo&amp;feature=youtube_gdata'/>
	<media:group>
		<media:thumbnail url='http://i.ytimg.com/vi/STm4GUqwLVo/default.jpg' height='90' width='120' time='00:00:43.500' yt:name='default'/>
	</media:group>
</entry>

Traversing The XML Response Document

There are a number of ways in ASP.NET to traverse / select nodes from an XML document.

The way most Web resources will tell you to do it is with LINQ to XML. LINQ allows us to make a SQL-like query of the XML document, and receive in return a SQL-like recordset, which we can then use to ouput our data.

LINQ to XML is definitely the way to go if you’re going to work with a large XML document, or need to extract data in a very complicated way (e.g, get various node attributes; select nodes based on data contained in other nodes).

LINQ to XML would work here, too, but I think it’s a bit of overkill, considering we only need three values from 12 records. So I’m just going to use the built-in parsing functions that are part of XmlDocument, a class we have already encumbered. Also, I’m more used to using XPath than I am LINQ.

Another way is to create an XML Stylesheet (XSLT) which will “flattened out” the XML document into 12 nodes (each representing one record) with three attributes (each representing the title, URL and thumbnail for that record). I describe that methodology at “Using National Weather Service XML Feeds With ASP.NET, ADO.NET And XSL.”

But I am trying to limit the number of things you have to learn at one time to accomplish this task. In my estimation, it’s a bit easier to hack together an XPath expression than to write an XSLT, so this is how I’m proceeding.

We’ll need three XPath expressions that will give us, for each of the 12 videos returned:

  1. the inner text of title,
  2. the inner text of the link node that has the attribute ‘alternate’, and
  3. the url attribute of whichever media:thumbnail node that has the additional attribute of yt:name=’default’.

The XML returned by the YouTube Data API uses several namespaces (atom, media, yt, etc.). Therefore, we first have to make reference to those namespaces in order to traverse the response XML.

We do that by creating an XmlNamespaceManager, then adding the specification URLs to it:

'add namespaces so we can traverse this thing
Dim xmlNSM As New XmlNamespaceManager(xmlDoc.NameTable)
xmlNSM.AddNamespace("atom", "http://www.w3.org/2005/Atom")
xmlNSM.AddNamespace("media", "http://search.yahoo.com/mrss/")
xmlNSM.AddNamespace("yt", "http://gdata.youtube.com/schemas/2007")

(We got these URLs from the root (“feed”) node’s attributes. That’s where they always appear, any time an XML document references namespace(s).

An XML namespace is basically a description of what elements an XML document will have, the kinds of data each node will contain, and other useful technical information that helps a parser understand what it is looking at and how to consume it.

Now that we have the namespaces referenced, we can go ahead and create our XPath arguments. Here they are:

'let's get our entry node values
Dim xmlTitleNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/atom:title", xmlNSM)
Dim xmlURLNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:player", xmlNSM)
Dim xmlThumbNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:thumbnail[@yt:name='default']", xmlNSM)

Note that I use the namespace:node syntax to select the nodes I want. If I didn’t do that, the XML parser wouldn’t understand what I meant if I said simply, “feed”, for example. I have to let the parser know which namespace the element “feed” belongs to, so it can find it.

With our nodes on hand, we just need to iterate each XmlNodeList, get our values, and pass them to a subroutine that adds hyperlinked thumbnails to our page.

We can use a simple For loop here, because by design, the number of title, thumbnail and hyperlink nodes will be the same, and the nodes will be in the same order in all three XmlNodeLists. (That is, the first node in each XmlNodeList will be for the first record; the title, URL and thumbnail values will match up, because they were listed in order in the XML document.)

'For loop will iterate them; by definition, counts are the same for each XmlNodeList
For I = 0 To xmlTitleNodes.Count - 1
	strTitle = xmlTitleNodes.Item(I).InnerText
	strURL = xmlURLNodes.Item(I).Attributes("url").Value
	strThumb = xmlThumbNodes.Item(I).Attributes("url").Value

	CreateHyperlinkedThumb(strTitle, strURL, strThumb)
Next

Here’s what the entire subroutine looks like:

Sub MakeThumbnailLinks(ByVal strAPIKey As String, ByVal strQuery As String)
	'create XML document we will parse
	Dim xmlDoc As New XmlDocument()
	xmlDoc = QueryYouTubeDataAPI(strAPIKey, strQuery)



	'add namespaces so we can traverse this thing
	Dim xmlNSM As New XmlNamespaceManager(xmlDoc.NameTable)
	xmlNSM.AddNamespace("atom", "http://www.w3.org/2005/Atom")
	xmlNSM.AddNamespace("media", "http://search.yahoo.com/mrss/")
	xmlNSM.AddNamespace("yt", "http://gdata.youtube.com/schemas/2007")

	'let's get our entry node values
	Dim xmlTitleNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/atom:title", xmlNSM)
	Dim xmlURLNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:player", xmlNSM)
	Dim xmlThumbNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:thumbnail[@yt:name='default']", xmlNSM)

	'for debugging, we'll also get the url that represents what the Data API actually used
	Dim objNode As XmlNode = xmlDoc.SelectSingleNode("/atom:feed/atom:link[@rel='self']", xmlNSM)
	lblStatus.Text = "<strong>Request URL returned by YouTube Data API:</strong> " & Server.HtmlEncode(objNode.Attributes("href").Value)

	'strings to store values
	Dim strTitle As String
	Dim strURL As String
	Dim strThumb As String

	'looping variable
	Dim I As Integer

	'For loop will iterate them; by definition, counts are the same for each XmlNodeList
	For I = 0 To xmlTitleNodes.Count - 1
		strTitle = xmlTitleNodes.Item(I).InnerText
		strURL = xmlURLNodes.Item(I).Attributes("url").Value
		strThumb = xmlThumbNodes.Item(I).Attributes("url").Value

		CreateHyperlinkedThumb(strTitle, strURL, strThumb)
	Next
End Sub

Creating Thumbs From Results

The CreateHyperlinkedThumb subroutine takes the title, URL and thumbnail strings we got, applies them as properties to a Hyperlink control, and adds those controls to a Panel on our page.

Sub CreateHyperlinkedThumb(strTitle As String, strURL As String, strThumb As String)
	'create hyperlink
	Dim ctlLink As New HyperLink()

	'set values
	ctlLink.Text = strTitle
	ctlLink.ToolTip = strTitle
	ctlLink.NavigateUrl = strURL
	ctlLink.ImageUrl = strThumb
	ctlLink.CssClass = "margin-5"
	ctlLink.Target = "video"

	'add to panel
	pnlThumbs.Controls.Add(ctlLink)
End Sub

And with that, we’re done!

To invoke, we simply call the MakeThumbnailLinks subroutine, passing to it our YouTube Data API developer key and the querystring we built:

MakeThumbnailLinks("YOUTUBE_DATA_API_DEVELOPER_KEY", "q=""Miss Maine""&max-results=12&duration=short&safeSearch=strict&orderby=relevance_lang_en")

You can see a working demo here: http://www.dougv.net/demos/youtube_data_api_thumbs/Example2.aspx

This code on github: https://github.com/dougvdotcom/aspnet_youtube_api

All links in this post on delicious: http://delicious.com/dougvdotcom/displaying-selected-youtube-data-api-thumbnails-on-a-web-page-via-asp-net-web-forms

2 thoughts on “Displaying Selected YouTube Data API Thumbnails On A Web Page Via ASP.NET Web Forms

  1. Pingback: Displaying Selected Youtube Data API Video Thumbnails On An ASP.NET Web Forms Page

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current ye@r *