Previously, I blogged about “Displaying Selected YouTube Video Thumbnails On An ASP.NET Web Forms Page,” when you know the video IDs of the thumbnails you want to hyperlink.
A reader recently asked me how to hyperlink YouTube video thumbnails based on searching for a keyword. I promised to address that, so here goes.
- Form a simple request URL to the YouTube Data API that contains the appropriate search parameters;
- Use a WebRequest to send that URL to Google, which returns an XML document with results;
- Use WebResponse to dump that stream into an XmlDocument;
- Use XPath and XmlNode‘s SelectNodes method to recursively get the thumbnails from each entry; and
- Bind up a pile of Hyperlink controls, which are added dynamically to a Panel control.
Sounds more complicated than it actually is. Let’s do it.
Overview Of The YouTube Data API
First step in leveraging the YouTube Data API: Get a Youtube Data API developer key.
With our developer key in hand, we can proceed to build a simple keyword query. In this case, we’re going to get the top 12 videos that meet the search criteria “Miss Maine.”
For our purposes, “top 12 videos” means the 12 videos YouTube deems most relevant to the exact search term “Miss Maine.” In other words, our search is going to work exactly as though we had typed “Miss Maine” (including the double quotes) into the search box at youtube.com; the results we get back should be the same as the first 12 results we would get via a default search on youtube.com. (For the usual vague reasons, this won’t always be the case, but the results sets will be similar.)
The YouTube Data API tells us that, to do a search query, we can GET a request to its servers, with our search parameters as querystring variables, and Google will return to us an XML document containing records that match our query.
Because we’re not requesting user-specific data, we don’t need to authenticate our requests; as previously mentioned, we don’t even need to send along our developer key. (But, again, being courteous and thorough, we will send it, as requested.)
So, the methodology we use to get our records is exactly the same as we used to shorten a URL via the bit.ly API; we’re going to create a URL containing all our parameters, then make a WebRequest to the YouTube Data API; we’ll get its returned XML via a WebResponse, and push that into an XmlDocument, from which we can extract the information we want.
- Many of the videos on YouTube aren’t very good, in the aesthetic, intellectual or civil senses. In other words, much of what is on YouTube is terrible.
- A sizable number of YouTube videos are spam.
- It’s de rigueur to keyword spam video descriptions, especially if the video is spam.
- Trolling is mandatory for YouTube video comments, and you’ll probably be linking directly to that.
We can mitigate the damage, somewhat, by applying the smartSearch filter, being very specific with our search term, including specific words we don’t want returned in our results, restricting the categories in which we want to search … in other words, being as specific as we can about what we want to see.
However, I’ll bet a dollar to doughnuts that if you request any sizable result set on any generic search term, your results set is going to contain items you wish weren’t in there. That’s just the nature of the thing, and there’s little that can be done about it.
Querying The YouTube Data API
There are three distinct phases in our solution: query the API, get the information we need from its response, and put that information on the page. Therefore, we will use three distinct functions and subroutines to do the work.
First up, a function to submit our query. It accepts as arguments our developer key and a formatted querystring, and it puts the response into an XML document for us. If the request fails, it will report as much in a Label control, and return an empty XML document.
Function QueryYouTubeDataAPI(ByVal strAPIKey As String, ByVal strQuery As String) As XmlDocument 'Returns empty XMLDocument on failure, results XML on success 'This function requires your page to have a label control named lblStatus for error reporting 'build URL to shorten method resource Dim strUri As New StringBuilder("https://gdata.youtube.com/feeds/api/videos?") strUri.Append("v=2") strUri.Append("&") strUri.Append(strQuery) strUri.Append("&key=") strUri.Append(Server.HtmlEncode(strAPIKey)) strUri.Append("&prettyprint=true") 'adds line breaks & white space to response; useful for debugging Dim objRequest As HttpWebRequest Dim objResponse As WebResponse Dim objXML As New XmlDocument() 'This is the document the function will return Try 'create request for shorten resource objRequest = WebRequest.Create(strUri.ToString) 'since we are passing querystring variables, our method is get objRequest.Method = "GET" 'act as though we are sending a form objRequest.ContentType = "application/x-www-form-urlencoded" 'don't wait for a 100 Continue HTTP response objRequest.ServicePoint.Expect100Continue = False 'since we are using get, we need not send a request body; set content-length to 0 objRequest.ContentLength = 0 'read the Data API response into XML document objResponse = objRequest.GetResponse() objXML.Load(objResponse.GetResponseStream()) Catch ex As Exception lblStatus.Text = "Error querying YouTube Data API. Message: " & ex.Message End Try 'send XML Document Return objXML End Function
If there is a problem with the query string you send to the YouTube Data API, or your request is somehow malformed, the API will return to you the 25 most popular videos for the day, as of the time of the request.
You can check to see if the YouTube Data API is seeing your request properly by looking for the /feed/link rel=’self’ node:
<feed xmlns='http://www.w3.org/2005/Atom' xmlns:app='http://www.w3.org/2007/app' xmlns:media='http://search.yahoo.com/mrss/' xmlns:openSearch='http://a9.com/-/spec/opensearch/1.1/' xmlns:gd='http://schemas.google.com/g/2005' xmlns:yt='http://gdata.youtube.com/schemas/2007' gd:etag='W/"CkYNSH07cSp7I2A9WhVSGEg."'> <!-- ... previous nodes --> <link rel='self' type='application/atom+xml' href='https://gdata.youtube.com/feeds/api/videos?q=%22Miss+Maine%22&start-index=1&max-results=12&duration=short&safeSearch=strict&orderby=relevance_lang_en&v=2'/> <!-- additional nodes ... --> </feed>
If the URL in that node doesn’t resemble the one you sent, most likely your querystring is malformed; but possibly, there are other errors in your request URL.
Creating The Query String
You’ll notice that I am not very specific about the query string argument passed to the previous function. That’s because I want to give you maximum flexibility, and the best way to do that is to allow for a free-form query string to be supplied.
So let’s look at how I’ll build my sample query string. I’ll do that, of course, with key-value pairs. That is, I will specify the parameter I want to send, and set it as equal to something.
First up, the search term. We’re going to use “Miss Maine.” So, the first part of our query string is:
Technically, we can stop here and let the default API request variables take over. But I want to be more specific about my search results, so let’s add some more parameters.
Next, I’ll limit my responses to 12 records, by appending that key-value pair to my string:
I’d like only short (under 4 minutes) videos:
To cull at least some offensive material, I’ll use strict safeSearch:
Finally, I’m going to specifically ask for English language videos that best match my search term. So my completed query string is:
For many Web developers, adding DLLs to a Web server or installing assemblies on a workstation can be problematic, if not prohibited outright. So I want to use as many built-in resources as possible. However, if you can install the GData API, consider going that route.
Examining The XML Response
Assuming our request goes through — that is, our Web server actually delivers the request to YouTube, which in turn sends a response back — we now have an XML document in storage that contains the information we requested. (Or, in the event of a malformed request, the default results set previously described.)
You can see the XML document returned for my demo query string here: http://www.dougv.net/demos/youtube_data_api_thumbs/videos.xml
In our case, we want some specific data for each video:
- Its title;
- the default thumbnail; and
- the URL to its player page on youtube.com.
We get back lots more useful information in the XML response: The name of the uploader, the run time of the video in seconds, its likes and dislikes count, the categories and keywords assigned to the video, the date and time it was uploaded, its view count, etc. But for this solution, we’ll only use the video’s title, thumbnail and URL.
Although the actual XML document contains much more data per entry, for our purposes, the relevant structure of the document looks like this (I’ve stripped out extraneous attributes):
<?xml version='1.0' encoding='UTF-8'?> <feed> <entry> <title></title> <link rel='alternate' /> <media:group> <media:thumbnail /> </media:group> </entry> </feed>
In practice, the parts of an entry (that is, an individual record) that we want to use look like this:
<entry gd:etag='W/"DU8AQ347eCp7I2A9WhRWEEQ."'> <title>Miss Maine USA 2011</title> <link rel='alternate' type='text/html' href='https://www.youtube.com/watch?v=STm4GUqwLVo&feature=youtube_gdata'/> <media:group> <media:thumbnail url='http://i.ytimg.com/vi/STm4GUqwLVo/default.jpg' height='90' width='120' time='00:00:43.500' yt:name='default'/> </media:group> </entry>
Traversing The XML Response Document
The way most Web resources will tell you to do it is with LINQ to XML. LINQ allows us to make a SQL-like query of the XML document, and receive in return a SQL-like recordset, which we can then use to ouput our data.
LINQ to XML is definitely the way to go if you’re going to work with a large XML document, or need to extract data in a very complicated way (e.g, get various node attributes; select nodes based on data contained in other nodes).
LINQ to XML would work here, too, but I think it’s a bit of overkill, considering we only need three values from 12 records. So I’m just going to use the built-in parsing functions that are part of XmlDocument, a class we have already encumbered. Also, I’m more used to using XPath than I am LINQ.
Another way is to create an XML Stylesheet (XSLT) which will “flattened out” the XML document into 12 nodes (each representing one record) with three attributes (each representing the title, URL and thumbnail for that record). I describe that methodology at “Using National Weather Service XML Feeds With ASP.NET, ADO.NET And XSL.”
But I am trying to limit the number of things you have to learn at one time to accomplish this task. In my estimation, it’s a bit easier to hack together an XPath expression than to write an XSLT, so this is how I’m proceeding.
We’ll need three XPath expressions that will give us, for each of the 12 videos returned:
- the inner text of title,
- the inner text of the link node that has the attribute ‘alternate’, and
- the url attribute of whichever media:thumbnail node that has the additional attribute of yt:name=’default’.
The XML returned by the YouTube Data API uses several namespaces (atom, media, yt, etc.). Therefore, we first have to make reference to those namespaces in order to traverse the response XML.
We do that by creating an XmlNamespaceManager, then adding the specification URLs to it:
'add namespaces so we can traverse this thing Dim xmlNSM As New XmlNamespaceManager(xmlDoc.NameTable) xmlNSM.AddNamespace("atom", "http://www.w3.org/2005/Atom") xmlNSM.AddNamespace("media", "http://search.yahoo.com/mrss/") xmlNSM.AddNamespace("yt", "http://gdata.youtube.com/schemas/2007")
(We got these URLs from the root (“feed”) node’s attributes. That’s where they always appear, any time an XML document references namespace(s).
Now that we have the namespaces referenced, we can go ahead and create our XPath arguments. Here they are:
'let's get our entry node values Dim xmlTitleNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/atom:title", xmlNSM) Dim xmlURLNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:player", xmlNSM) Dim xmlThumbNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:thumbnail[@yt:name='default']", xmlNSM)
Note that I use the namespace:node syntax to select the nodes I want. If I didn’t do that, the XML parser wouldn’t understand what I meant if I said simply, “feed”, for example. I have to let the parser know which namespace the element “feed” belongs to, so it can find it.
With our nodes on hand, we just need to iterate each XmlNodeList, get our values, and pass them to a subroutine that adds hyperlinked thumbnails to our page.
We can use a simple For loop here, because by design, the number of title, thumbnail and hyperlink nodes will be the same, and the nodes will be in the same order in all three XmlNodeLists. (That is, the first node in each XmlNodeList will be for the first record; the title, URL and thumbnail values will match up, because they were listed in order in the XML document.)
'For loop will iterate them; by definition, counts are the same for each XmlNodeList For I = 0 To xmlTitleNodes.Count - 1 strTitle = xmlTitleNodes.Item(I).InnerText strURL = xmlURLNodes.Item(I).Attributes("url").Value strThumb = xmlThumbNodes.Item(I).Attributes("url").Value CreateHyperlinkedThumb(strTitle, strURL, strThumb) Next
Here’s what the entire subroutine looks like:
Sub MakeThumbnailLinks(ByVal strAPIKey As String, ByVal strQuery As String) 'create XML document we will parse Dim xmlDoc As New XmlDocument() xmlDoc = QueryYouTubeDataAPI(strAPIKey, strQuery) 'add namespaces so we can traverse this thing Dim xmlNSM As New XmlNamespaceManager(xmlDoc.NameTable) xmlNSM.AddNamespace("atom", "http://www.w3.org/2005/Atom") xmlNSM.AddNamespace("media", "http://search.yahoo.com/mrss/") xmlNSM.AddNamespace("yt", "http://gdata.youtube.com/schemas/2007") 'let's get our entry node values Dim xmlTitleNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/atom:title", xmlNSM) Dim xmlURLNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:player", xmlNSM) Dim xmlThumbNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:thumbnail[@yt:name='default']", xmlNSM) 'for debugging, we'll also get the url that represents what the Data API actually used Dim objNode As XmlNode = xmlDoc.SelectSingleNode("/atom:feed/atom:link[@rel='self']", xmlNSM) lblStatus.Text = "<strong>Request URL returned by YouTube Data API:</strong> " & Server.HtmlEncode(objNode.Attributes("href").Value) 'strings to store values Dim strTitle As String Dim strURL As String Dim strThumb As String 'looping variable Dim I As Integer 'For loop will iterate them; by definition, counts are the same for each XmlNodeList For I = 0 To xmlTitleNodes.Count - 1 strTitle = xmlTitleNodes.Item(I).InnerText strURL = xmlURLNodes.Item(I).Attributes("url").Value strThumb = xmlThumbNodes.Item(I).Attributes("url").Value CreateHyperlinkedThumb(strTitle, strURL, strThumb) Next End Sub
Creating Thumbs From Results
The CreateHyperlinkedThumb subroutine takes the title, URL and thumbnail strings we got, applies them as properties to a Hyperlink control, and adds those controls to a Panel on our page.
Sub CreateHyperlinkedThumb(strTitle As String, strURL As String, strThumb As String) 'create hyperlink Dim ctlLink As New HyperLink() 'set values ctlLink.Text = strTitle ctlLink.ToolTip = strTitle ctlLink.NavigateUrl = strURL ctlLink.ImageUrl = strThumb ctlLink.CssClass = "margin-5" ctlLink.Target = "video" 'add to panel pnlThumbs.Controls.Add(ctlLink) End Sub
And with that, we’re done!
To invoke, we simply call the MakeThumbnailLinks subroutine, passing to it our YouTube Data API developer key and the querystring we built:
MakeThumbnailLinks("YOUTUBE_DATA_API_DEVELOPER_KEY", "q=""Miss Maine""&max-results=12&duration=short&safeSearch=strict&orderby=relevance_lang_en")
You can see a working demo here: http://www.dougv.net/demos/youtube_data_api_thumbs/Example2.aspx
The code for this solution, as well as “Displaying Selected YouTube Video Thumbnails On An ASP.NET Web Forms Page,” can be downloaded here: Displaying Selected YouTube Data API Thumbnails On A Web Page Via ASP.NET Web Forms demo code
All links in this post on delicious: http://delicious.com/dougvdotcom/displaying-selected-youtube-data-api-thumbnails-on-a-web-page-via-asp-net-web-forms