dougv.com « Doug Vanderweide » Programming

The Basics Of Avoiding MySQL Injection Attacks In ASP.NET Web Forms

Doug Vanderweide — Tue, 08 May 2012 22:16:45 +0000

Received in my email today:

Hi

say your blog and thought you might help.

strsql = “SELECT StaffID, DesignationID, StaffName, Password, ShopID from staffT where StaffName =” & UserName.Text & ” AND Password =” & Password.Text & “”

from the string, the username.text and password.text are form controls. what is happening is there are passing null values regardless of what you input in the text boxes resulting in a system error.

“System Error Object reference not set to an instance of an object”

Am using Mysql as the database.

I’m always glad to answer such questions, especially when the questioner is flirting with disaster, as much as this questioner is.

A trained eye can immediately spot the problem with the SQL statement above, aside from the problem of NULL values tossing errors. Namely, it’s wide-open to SQL injection. (And an even keener eye will note that the values for user name and password aren’t delimited with single-quotes.)

So here’s my reply email to the questioner:

Your SQL statement has three problems.

It is wide open to injection attack. See http://www.unixwiz.net/techtips/sql-injection.html for examples.
As you noted, when nulls are passed in, the expression fails.
It appears you have not delimited your text inputs with single quotes.

Assuming you are using ASP.NET, and that your user names and passwords are only alphanumeric, the direct fix to your problem is this:

Dim strUser As String
If String.IsNullOrEmpty(UserName.Text) Then
	strUser = String.Empty
Else
	strUser = UserName.Text.Replace("'", "''")
End If

Dim strPass As String
If String.IsNullOrEmpty(Password.Text) Then
	strPass = String.Empty
Else
	strPass = Password.Text.Replace("'", "''")
End If

strsql = "SELECT StaffID, DesignationID, StaffName, Password, ShopID from staffT where StaffName = '" & strUser & "' AND Password = '" & strPass & "'"

That should get you going, but you should employ the following best practices fixes:

Use stored procedures with paramerterized input values. This serves to automatically sanitize your queries for data type.
Use RequiredFieldValidators and RegularExpressionValidators to ensure that you receive some sort of input for each field, and that each does not contain an effort to inject your code with SQL.
Impose some sort of control that limits multiple login attempts.
Better yet, use the built-in membership provider in ASP.NET, which can be used with MySQL, and has already resolved many potential attack vectors.

Hope this helps.

Additional Thoughts And Clarification

The guidance I gave the emailer, instructing him to escape single quotes, is a bare-minimum escaping sequence. As a practical matter, he should sanitize his inputs against additional MySQL sequences, such as double-dashes and semicolons, as well as reserved SQL statement words, such as DROP, ALTER, DELETE, etc. There’s an HTTPModule example over at the ASP.NET forums that does this automatically for an application.

Using parameterized queries / stored procedures to combat SQL injection is a primary recommendation from both Microsoft and MySQL (pdf).

In addition to securing data type, query parameters limit the ability of an attacker to inject SQL by fixing the form of the query implicitly. In other words, it’s harder for him to mangle a parameter than it is to mangle a string.

All links in this post on delicious: http://delicious.com/dougvdotcom/the-basics-of-avoiding-sql-injection-attacks-in-asp-net-web-forms

The numbers inside parentheses are relevance scores. Scoring is based, in order of priority, on title, category, content and tags. The higher the score, the more likely that post relates to this post.

Tags: coding standards, data types, elegance, hacking, regular expression, Windows Server

TEA Time: New England GiveCamp 2012 Recap

Doug Vanderweide — Tue, 08 May 2012 02:38:47 +0000

Last weekend I was in Cambridge, Mass. for New England GiveCamp 2012, the third of annual meet-ups that match technical and design people with nonprofit organizations that need their help.

The Charles River Esplanade is on the left. Hatch Memorial Shell and Teddy Ebersol's Red Sox Fields are in the foreground.

My cause was The Esplanade Association, an organization that cares for the Charles River Esplanade Park.

The Charles River Esplanade Park is the Boston-side green space along the river, from the Museum of Science to the Boston University Bridge. While it’s owned and managed by the state of Massachusetts, TEA (which has to be the coolest acronym possible for a Boston-based group) exists to organize people to help protect and care for the park.

Much of their work involves organizing volunteers to clean up the park several times each year. TEA also holds a number of programs in the park — yoga, Zumba, dances and the like — and runs several fund raising projects.

They came to GiveCamp, initially, looking for a way to better coordinate singing up groups and individuals for cleanup days.

Ticketing Turns To Overhaul

Currently, the way Jessica B. Pederson, TEA’s project manager, handles such reservations is by posting a schedule of cleanup days on her Web site, and asking people to email her if they are interested in participating. Then, via email exchanges back and forth, the details get worked out.

Even though the majority of these events are done through groups — so Jessica is in contact with a few group representatives, rather than scores of individuals — this proves an understandably arduous process that consumes a great deal of Jessica’s time, and it was something that she knew just screamed out to be automated.

You can’t swing a cat without hitting sample booking / ticketing system code on the Web, and there’s little practical difference between booking, say, a hotel room or buying a ticket to a play, and reserving a volunteer opportunity slot. Just provide the date, the number and types of tickets available, and a way for people to provide their contact information, and you’re done.

So I started by looking for ASP.NET-based ticketing solutions on the Web, since most of the Web developers who participate in GiveCamp are .NET folks. (Actually, I was side-hoping I could request Saurabh Moondhra and William Wade, the two ASP.NET developers I worked with during GiveCamp 2011, whom I knew would shine on such a project.)

And I was also thinking, in the back of my mind, it would be nice to address TEA’s Web site.

Don’t get me wrong; the site they had worked fine and contains a lot of good information presented in a sensible way. But it’s also stuck in 2005, and could use a cosmetic and functional overhaul.

And coincidentally, I am working on a WordPress site for a client right now that included the need for a basic calendaring and ticketing system. I had identified a plugin, Events Manager, for them that wasn’t quite right for that project; but it almost perfectly fit Jessica’s needs, especially if she was willing to upgrade to the “pro” version.

And then, by coincidence again, Kelley Muir — who coordinates the projects at GiveCamp — emailed me to say that she had asked this year’s nonprofits to identify possible second projects, and TEA said they’d like to have a content management system and general site redesign, too.

Jackpot! No need to reinvent the wheel; no need to even resize the wheel. We’re gonna put them in the Honda of the Web — good ol’ WordPress — and slap some 20-inch rims (Events Manager) on it.

Meet TEA

I talked with Jessica and Christopher Timmel, TEA’s communications director, on the phone on Wednesday, to go over their needs and expectations, and knew instantly that I had lucked out a second time.

I’m a big fan of green space. You can’t live in Augusta, Maine and not like trees and rivers, after all.

But as a simple matter of public policy, I don’t think parks and open spaces get enough credit or resources, in spite of being a huge factor in quality of life. Understandably, because trees and grass pretty much grow on their own, it’s easy to put government dollars and effort elsewhere.

So when I see a group like TEA, I’m already a huge fan of their work.

Even better was that Jessica and Chris had a very clear picture of what they actually wanted. Even more rarely, what they wanted was actually what they needed. (It’s strange, but often, clients pitch requirements that don’t resolve their problems, or do so in ways that only make the problems worse or create new problems. Not so with Chris and Jessica.)

Plus, they’re personable, sincere and nice people. No coddling, wrangling or placating was in my weekend forecast. Not that it’s been a problem for me in past GiveCamps; just that any time you can get a client that’s positive about its problems, you need to celebrate.

Meet The Team

So I asked Kelley to assign a designer and a couple PHP people (since WordPress skinning requires a little knowledge of PHP, and I figured we might need to adjust some of the Events Manager code via overrides) to the project.

That’s how Christina Yung and Jason Dufour were assigned to the project, and they were my third stroke of luck.

Jessica Pederson, program director for The Esplanade Association, and Christina Yung, graphic designer, running through the old esplanadeassociation.org Web site on Friday night of GiveCamp 2012.

Christina primarily works in print, signage and branding, but has recently expanded into Web design. Jason is primarily an ASP.NET developer, but has PHP experience, and very strong jQuery and CSS skills. As anyone who’s ever skinned a WordPress site knows, the hard part is getting the CSS right, and the things you can’t quite get right in CSS, you can usually fix in jQuery.

An even better perk is that Jason and Christina work together at their day jobs, which means there would be no need to worry about chemistry or communication. And the weekend bore out that both were take-initiative-and-ownership types, which fits my management style perfectly: I like to provide an area of responsibility and a goal, then let people figure out, and do, what needs to get done to achieve it.

So two times I was a GiveCamp project lead, and two times Kelley assigned my team members who were skilled, friendly people who knew what to do and got done what needed to get done.

The Project

Even when you have a superior platform (WordPress); the right tools inside that platform (Events Manager); a client with full buy-in, all collateral organized and ready, and authority to make decisions on the fly (Chris and Jessica); and a team that’s got the skills and drive to get the job done (Jason and Christina), putting a Web site together over a weekend is a frenzy.

There were a few points where I thought we wouldn’t make it.

In addition to the usual bumps in any project road — misunderstandings, mistakes, unforeseen complications and unintended consequences are the hallmarks of every Web project — there were many, many steps in getting the content right.

Because of the kinds of information TEA needs to communicate, this WordPress site wasn’t header-footer-sidebar-index-css-and-we’re-done.

They needed fairly complicated page flow, and the calendar itself mutated a couple times as it became clearer what events would need a registration component, and which events they simply wanted to let people know about. And their information required a few different page templates and a couple extra sidebars, to ensure it showed up properly.

Additionally, I had the objective to use as few plugins (aside from Events Manager) and as little code as possible (especially in the templates themselves), because I wanted to ensure whomever helps them with technical needs going forward can make heads and tails out of what we did. So that meant every error we made in terms of approach, taxonomy and content was magnified, since we couldn’t simply find a plugin or write some code to fix the issue.

In the end, we would up with only eight plugins, three of which are standard in all my WordPress installs:

Bluetrait Event Viewer (BTEV), which logs system events and is very useful for debugging, intrusion detection and the like;
Category Posts Widget, which — as its name suggests — allows you to show posts in a specific category as a sidebar widget;
Events Manager, a calendar and booking system I simply cannot flog hard enough;
Google XML Sitemaps, to help with site indexing;
HTML Page Sitemap, which automatically generates a sitemap based on wp_list_pages();
Simple Lightbox, for slideshow functionality (TEA has lots and lots of pictures);
Single Post Widget, which — as its name suggests — allows you to show a single post in a sidebar; and
WP Super Cache, to reduce server load and improve site responsiveness.

The Results

Jessica and Chris are happy with their new site. They’re going to take a week to finish migrating content, come up with a list of questions and tweaks, then present it to their board of directors for feedback.

But you don’t have to wait. Check out the before and after shots:

esplanadeassociation.org, before the redesign

esplanadeassociation.org, after the redesign

As I quipped at the GiveCamp closing presentations, it looks good because Christina made it that way; it works because Jason made it that way; and it’s not done because I made it that way.

The site needs to go live still; a fair amount of content remains to be migrated in, there are a few cosmetic tweaks needed, and we didn’t run a complete walkthrough / debugging of the site because all the elements are not in place.

But that is not to detract at all from what was accomplished. While I intend to help TEA go live with this site, really it can be finished by anyone with basic HTML / CSS knowledge. Anything short of complete disapproval of the skin won’t be difficult to accommodate, and that skin is not going to be rejected. Not by sane people, anyway.

So I hit for the cycle: Great project; great client; great team; great results. GiveCamp 2012 was a complete success.

A Round Of Applause

I’d be remiss if I didn’t note, once again, what a great experience GiveCamp is on the whole.

There’s just something reaffirming and enjoyable in being around capable, motivated, positive people who are in it for the love. Sincerely, you meet the nicest and best people at GiveCamp.

You get fed well. (Thank you Green Mountain Coffee Roasters, Naked Pizza, b.good catering, Whole Foods, Cape Cod Chips, Rip van Wafels and Cakes By Kelli.) You get all kinds of great swag (that’s generally true of any event hosted by Jim O’Neil, New England GiveCamp’s host and driving force, but is certainly true of GiveCamp). And you get to be in Cambridge, which is an amazing place.

I should also mention, after the disappointment of the Royal Sonesta hotel last year, that the best deal in town for a stay near MIT has to be the Hampton Inn on Monsignor O’Brien Highway. Even though I figure I paid about $30 per hour to sleep there (11 hours of sleep @ $340 for two nights), it’s very clean, the bed is awesome, the staff is friendly and efficient, it’s 100 percent geared toward serving business travelers (free WiFi, lightning-fast checkin and automatic checkout), the hot breakfast is actually quite palatable and it’s only a one-mile walk up Third Street from NERD, where GiveCamp is hosted.

So, everybody who’s taken part or helped make it happen, thanks. You done good. Real good. And if you couldn’t make it this year, GiveCamp 2013 is April 26-28; mark your calendars now!

All links in this post on delicious: http://delicious.com/dougvdotcom/tea-time-new-england-givecamp-2012-recap

Tags: coding standards, crowdsourcing, ethics, GiveCamp, graphic design, Jim O'Neil, marketing, Microsoft

Getting All ZIP Codes In A Given Radius From A Known Point / ZIP Code Via ASP.NET

Doug Vanderweide — Wed, 11 Apr 2012 14:02:01 +0000

Some time ago I wrote a PHP-MySQL based solution to getting all ZIP Codes in a given radius from a known point / ZIP Code. I’ve long intended to do an ASP.NET version of that post, and here it is.

I won’t bother revisiting the mechanics in detail. I do urge you to read the post on the PHP version of this solution, at least to familiarize yourself with the mechanics of what I am doing and the compromises I’ve taken in coming up with this solution.

I will note the following for the “get to the point” types:

The first thing we need is to procure a geocoded database table of ZIP Codes. There are several out there; the one I am using is the ZIP Code Database project, available at Sourceforge. You’ll need to figure out how to get their CSV file into your SQL Server database; BULK INSERT is an option, or you can script it.
The basic method I am going to use is to create a square. Specifically, I am going to:
- ask the end user for a starting ZIP Code, and a radius from that point from which he would like other ZIP Codes to come;
- create a square by selecting points North, South, East and West at the given distance from the starting point; then
- query the database for all points that fall within that square (or, in other words, all points with latitudes less than North, greater than South, less than East and greater than West).
I’m going to put my results in a GridView. However, you could easily just use a DataReader or DataTable to get the relevant records and do with them as you like.
The formulas I am using to compute longitude and latitude coordinates come from moveabletype.co.uk.

Step 1: Helper Functions

Before we get into the nitty-gritty, we need to create two helper functions: One that converts degrees into radians, and another that reverses the process. (In trigonometry, angles are calculated in radians; a radian is the ratio between the length of an arc and its radius. If an arc is as long as its radius, that’s 1^c).

First, to convert from degrees to radians:

Function Deg2Rad(ByVal sglDegrees As Single) As Single
	Return sglDegrees * (Math.PI / 180.0)
End Function

And to go from radians to degrees:

Function Rad2Deg(ByVal sglRadians As Single) As Single
	Return sglRadians * (180.0 / Math.PI)
End Function

Step 2: Functions To Plot Latitude And Longitude

Now that we can move back and forth between degrees and radians, we can calculate the latitude and longitude of points at a known distance and bearing from an initial geocoordinate.

Given a known starting point, expressed as lat1 and lon1; a known distance from that point, d; a known bearing from that point, b; and a known radius of the sphere over which we are travelling, r; we calculate a new geocoordinate, expressed as lat2 and lon2, thus:

lat2 = asin(sin(lat1) * cos(d/r) + cos(lat1) * sin(d/r) * cos(b))
lon2 = lon1 + atan2(sin(b) * sin(d/r) * cos(lat1), cos(d/r) - sin(lat1)*sin(lat2))

Note that we have to convert our latitude, longitude and bearing variables into radians when we make these calculations; and we need to convert the results from radians into degrees.

So here’s a function that accepts, as arguments, a starting latitude, radius, bearing and distance; and returns the latitude at the provided bearing and distance from that starting latitude.

Function CalculateLatitudeCoordinate(ByVal sglLat1 As Single, ByVal intRadius As Integer, ByVal intBearing As Integer, ByVal intDistance As Integer) As Single
	Return Math.Asin(Math.Sin(sglLat1) * Math.Cos(intDistance / intRadius) + Math.Cos(sglLat1) * Math.Sin(intDistance / intRadius) * Math.Cos(intBearing))
End Function

And here’s a function that accepts, as arguments, a starting latitude and longitude, and ending latitude, radius, bearing and distance; and returns a longitude at the provided bearing and distance from the starting coordinates.

Function CalculateLongitudeCoordinate(ByVal sglLon1 As Single, ByVal sglLat1 As Single, ByVal sglLat2 As Single, intRadius As Integer, ByVal intBearing As Integer, ByVal intDistance As Integer) As Single
	Return sglLon1 + Math.Atan2(Math.Sin(intBearing) * Math.Sin(intDistance / intRadius) * Math.Cos(sglLat1), Math.Cos(intDistance / intRadius) - Math.Sin(sglLat1) * Math.Sin(sglLat2))
End Function

Step 3: Create The Form

To implement, we need to create a form with a few elements:

a TextBox to accept an initial ZIP Code;
a DropDownList to provide the desired distance from that point, which will serve to create our search area;
a RequiredFieldValidator;
a RegularExpressionValidator, to ensure we get a five-digit number as our input;
a Button to submit;
a Label, to display messages;
a GridView, to display the ZIP Codes we get from the database; and
a SqlDataSource, against which we will bind the ZIP Codes

Select all ZIP Codes within 5 10 25 50 100 miles of ZIP Code

Note that my SqlDataSource has SelectParameters; I’ll use code behind to update those parameters with the coordinates I’ll figure out, also via code behind, and bind the GridView.

Although I am going to bind my data via codebehind, and therefore could have used a SqlDataReader, DataTable or the like as my GridView’s data source, I am using a SqlDataSource because I want to be able to page and sort my results, and I am too lazy to write code behind to do all that; those features are native to a GridView bound to a SqlDataSource.

If I didn’t want to page and sort, I’d just use a SqlDataReader and bind the GridView to that.

Step 4: Get The Initial Point

We need to get, from the database, the initial geocoordinates for the user-supplied ZIP Code. We do that with a stored procedure:

CREATE PROCEDURE [dbo].[sp_get_zip_code]
	@zip_code CHAR(5)
AS
BEGIN
	-- SET NOCOUNT ON added to prevent extra result sets from
	-- interfering with SELECT statements.
	SET NOCOUNT ON;

    -- Insert statements for procedure here
	SELECT *
	FROM zip_codes
	WHERE zipcode = @zip_code
END

This query should return to us the city name, state, latitude and longitude for the specified ZIP Code. As always, we want to catch any exceptions in making the query, and we want to make sure we get a record from the database (that is, we can find the starting ZIP Code in the database).

If we can’t get starting coordinates, we’ll report that.

Otherwise, we’ll output details about the starting point to our Label control, and invoke a (yet-to-be-written) subroutine to populate the GridView.

Sub GetInitialCoordinates() Handles btnZip.Click
	'This subroutine requires a Label control named lblStatus

	'Prepare to connect to db and execute stored procedure
	Dim objConn As New SqlConnection("YOUR CONNECTION STRING")
	Dim objCmd As New SqlCommand("sp_get_zip_code", objConn)
	objCmd.CommandType = CommandType.StoredProcedure

	'we need to supply the ZIP code as an input parameter to our stored procedure
	objCmd.Parameters.Add(New SqlParameter("zip_code", SqlDbType.Char, 5))
	objCmd.Parameters("zip_code").Value = tbZip.Text

	'sglMinLat = south, sglMaxLat = north, sglMinLon = west, sglMaxLon = east
	Dim sglMinLat As Single
	Dim sglMaxLat As Single
	Dim sglMinLon As Single
	Dim sglMaxLon As Single

	Try
		'open connection
		objConn.Open()
		'put results into datareader
		Dim objReader As SqlDataReader
		objReader = objCmd.ExecuteReader()
		If objReader.HasRows Then
			'if starting point found, calculate box points
			objReader.Read()
			sglMinLat = Rad2Deg(CalculateLatitudeCoordinate(Deg2Rad(objReader("latitude")), 3959, Deg2Rad(180), ddlDistance.SelectedValue))
			sglMaxLat = Rad2Deg(CalculateLatitudeCoordinate(Deg2Rad(objReader("latitude")), 3959, Deg2Rad(0), ddlDistance.SelectedValue))
			sglMinLon = Rad2Deg(CalculateLongitudeCoordinate(Deg2Rad(objReader("longitude")), Deg2Rad(objReader("latitude")), Deg2Rad(sglMinLat), 3959, Deg2Rad(270), ddlDistance.SelectedValue))
			sglMaxLon = Rad2Deg(CalculateLongitudeCoordinate(Deg2Rad(objReader("longitude")), Deg2Rad(objReader("latitude")), Deg2Rad(sglMinLat), 3959, Deg2Rad(90), ddlDistance.SelectedValue))

			'report starting point details to lblStatus
			Dim strOut As String
			strOut = "ZIP Code " & tbZip.Text & " is assigned to " & objReader("cityname") & ", " & objReader("statecode") & ".
"
			strOut &= "It is located at latitude " & objReader("latitude") & ", longitude " & objReader("longitude") & ".

"
			strOut &= "At a distance of " & ddlDistance.SelectedValue & " miles, the search box coordinates are:
"
			strOut &= "Maximum latitude (North): " & sglMaxLat & "
"
			strOut &= "Miniumum latitude (South): " & sglMinLat & "
"
			strOut &= "Maximum longitude (East): " & sglMaxLon & "
"
			strOut &= "Minimum longitude (West): " & sglMinLon & "
"
			lblStatus.Text = strOut

			'populate gridview
			PopulateGridView(sglMinLat, sglMaxLat, sglMinLon, sglMaxLon, objReader("latitude"), objReader("longitude"))
		Else
			'starting point not found
			lblStatus.Text = "Error retrieving initial ZIP Code coordinates: No record found for " & tbZip.Text & "."
		End If
		objConn.Close()
		objCmd.Dispose()
		objConn.Dispose()
	Catch ex As Exception
		'technical problem running the query
		lblStatus.Text = "Error executing database query for initial coordinates: " & ex.Message
	End Try
End Sub

Step 5: Bind The GridView

Lastly, we need to create a stored procedure that will accept the coordinates for our search box, and return all coordinates that fall within that box.

Notice that in addition to the North, South, East and West coordinates that form the search box, I am including the latitude and longitude of the ZIP Code that acts as our starting point, as well as the radius of the Earth.

That’s because I am going to calculate, on the fly, the distance from my starting point to the ZIP Codes returned by the query.

Given two known points, expressed as lat1, lon1, lat2 and lon2; and a known radius of the sphere on which they are located, expressed as r; the distance between those points is found via this formula:

distance = acos(sin(lat1) * sin(lat2) + cos(lat1) * cos(lat2) * cos(lon2 - lon1)) * r

Again, we have to convert our latitude and longitude coordinates from degrees to radians; but note that we do not need to convert the result of the expression from radians to degrees. I will, however, round that result down to a precision of 2.

CREATE PROCEDURE [dbo].[sp_get_zips_in_radius]

@maxlat DECIMAL(9, 6),
@minlat DECIMAL(9, 6),
@maxlon DECIMAL(9, 6),
@minlon DECIMAL(9, 6),
@startlat DECIMAL(9, 6),
@startlon DECIMAL(9, 6),
@radius INT

AS
SELECT zipcode, statecode, latitude, longitude, cityname,
ROUND(ACOS(SIN(RADIANS(@startlat)) * SIN(RADIANS(latitude)) + COS(RADIANS(@startlat)) * COS(RADIANS(latitude)) * COS(RADIANS(longitude) - RADIANS(@startlon))) * @radius, 2) AS distance
FROM zip_codes
WHERE latitude < @maxlat AND latitude > @minlat AND longitude < @maxlon AND longitude > @minlon
ORDER BY distance, zipcode, statecode, cityname

Now that we have the stored procedure ready to go, we just need to update the values of our SqlDataSource’s SelectParameters, then bind the GridView:

Sub PopulateGridView(ByVal sglMinLat As Single, ByVal sglMaxLat As Single, ByVal sglMinLon As Single, ByVal sglMaxLon As Single, ByVal sglStartLat As Single, ByVal sglStartLon As Single)
	sqlZip.SelectParameters("minlat").DefaultValue = sglMinLat
	sqlZip.SelectParameters("maxlat").DefaultValue = sglMaxLat
	sqlZip.SelectParameters("minlon").DefaultValue = sglMinLon
	sqlZip.SelectParameters("maxlon").DefaultValue = sglMaxLon
	sqlZip.SelectParameters("startlat").DefaultValue = sglStartLat
	sqlZip.SelectParameters("startlon").DefaultValue = sglStartLon

	gvZIP.DataBind()
End Sub

And that’s all there is to it. You can see a working demo here: http://dougv.net/demos/zip_code_distance

You can download the code here: https://www.dougv.com/wp-content/uploads/2012/04/zip_code_distance.zip

All links in this post on delicious: http://delicious.com/dougvdotcom/getting-all-zip-codes-in-a-given-radius-from-a-known-point-zip-code-via-asp-net

Tags: latitude / longitude, regular expression, ZIP Code database

How To Increment A Counter In MySQL Based On A Radio Button Click

Doug Vanderweide — Sat, 07 Apr 2012 00:42:06 +0000

Asked recently on Formspring:

how to increment count in database on clicking radio button

There are a few ways to go about this. I’ll demonstrate two: a traditional, PHP / MySQL only, postback approach, and a jQuery version that uses AJAX to asynchronously record and update the counts.

Just to be clear: In order to complete this solution, we have to use both JavaScript and a server-side scripting language. We use JavaScript to intercept the user clicking the radio button, but process the fact that the button was clicked on the server.

Also, for the purpose of this tutorial, I’ll assume that the radio button involved is part of a group. That is, we have several radio buttons, all with the same name, but different values, e.g.:


	Select a color:
	Red (0) |
	Green (0) |
	Blue (0) |
	Black (0)

Example 1: PHP / MySQL Postback

The easiest way to process a click is right on the page that contains the radio button, via a simple postback.

We begin with a MySQL table that will hold our count. It has two columns: color_name, which will contain unique values and thus can act as our table’s primary key; and color_count, the number of times that color has been clicked.

CREATE TABLE IF NOT EXISTS `colorcounter` (
  `colorname` varchar(5) NOT NULL,
  `colorcount` int(10) unsigned NOT NULL DEFAULT '0',
  PRIMARY KEY (`colorname`)
)

Next, we query the database for the current counts, which we’ll display in our form, later. We’ll dump the results set into an associative array, which will make outputting the counts in our form a little easier.

$link = mysql_connect('server', 'user', 'password') or die('Cannot connect to database server');
mysql_select_db('database') or die('Cannot select database');

// click-count increment routine will go here; more on that shortly ...

// get current color click counts
$rs = mysql_query("SELECT * FROM colorcounter") or die ('Cannot process SQL count totals query');
if(mysql_num_rows($rs) > 0) {
	while($row = mysql_fetch_array($rs)) {
		$count[$row['colorname']] = $row['colorcount'];
	}
}

We now need to set up our form to be ready to process the user click, and to display the click count records we previously received.

We set the form to submit to itself.
We add an onclick event to each radio button, instructing it to submit the form. (The logic that will process the click follows shortly; we’re just assuming that every time the form is submitted, we need to increment a counter.)
We display the current click counts after each radio button.


	Select a color:
	Red () |
	Green () |
	Blue () |
	Black ()

Finally, we need to add some code to the page that will see if the form has been submitted, and increment the appropriate count.

// if this is a postback ...
if(isset($_POST['color_name'])) {
	// create array of acceptable values
	$ok = array('red', 'green', 'blue', 'black');
	// if we have an acceptable value for color_name ...
	if(in_array($_POST['colorname'], $ok)) {
		// update the counter for that color
		$q = mysql_query("UPDATE colorcounter SET colorcount = colorcount + 1 WHERE colorname = '" . $_POST['color_name'] . "'") or die ("Error updating count for " . $_POST['color_name']);
	}
}

Note that this code block should appear before the code block that gets the current click counts (where I included the comment in the first block of PHP code), in order to ensure the counts are up-to-the-minute.

You can see a working demo here: http://www.dougv.com/demo/ajax_radio_clickcount/index.php

Example 2: Using AJAX

The previous example has a number of drawbacks, not the least of them being that each time you click on a radio button, the entire form posts back to the server.

That limits its usefulness quite a bit. We could work around this problem a few ways; perhaps making sure we repopulate our form fields following the postback, and only performing final processing when the submit button is clicked (by adding an onclick event to that button, which hands off the results to a JavaScript function or a different processing page).

That’s an awful lot of work for very little benefit. It makes way more sense to leverage AJAX, and update our click counts without having to post the form itself back to the server.

JavaScript — more specifically, jQuery — to the rescue!

I could have written a traditional AJAX solution here. However, it’s wordy to do that in a way that ensures browser comparability.

To me, it makes sense to encumber the jQuery library whenever you create an AJAX script, rather than reinventing the wheel with an old-school XMLHTTPRequest object that has to be written several ways to ensure it works in whatever browser you encounter.

To get our jQuery solution to work, we need to create two PHP “helper” pages: one that will update our click counts whenever a radio button is selected, and the other that will get the current click counts, both on initial page load and following a click.

The get_counts.php Helper Page

Let’s start by calling the helper page that gets our click counts get_counts.php. It works almost exactly like the code block that gets the counts in our PHP postback example, except this page is going to output our results in JSON.

We’re putting the results out that way because it’s very easy for jQuery to process data in JSON format.

A crash course in JSON: In the end, JSON is just a fancy way to create associative arrays in JavaScript. Or, more simply, it’s an easy way to supply a whole bunch of key-value pairs to a JavaScript function.

Or, to put it even another way, think of JSON as the JavaScript version of an XML file. It’s just a way to organize data into columns and rows.

JSON is actually a bit more complex than that, but the way we’re using it, JSON is just a recordset, exactly like we would get from a MySQL query. It’s just written differently.

Let’s look page will expect to see a querystring variable named “color,” which indicates which value needs to be incremented. The page code itself will be very similar to the processing code in our postback version.

$link = mysql_connect('server', 'user', 'password') or die('Cannot connect to database server');
mysql_select_db('database') or die('Cannot select database');

// get new count totals, pass as JSON
$rs = mysql_query("SELECT * FROM colorcounter") or die('Cannot get updated click counts');
if(mysql_num_rows($rs) > 0) {
	$out = "{ ";
	while($row = mysql_fetch_array($rs)) {
		$out .= "\"$row[colorname]\" : $row[colorcount], ";
	}
	$out = substr($out, 0, strlen($out) - 2);
	$out .= " }";

	header("Content-type: application/json");
	echo $out;
}

The increment_counter.php Helper Page

Now that we have a way to get the counts, we need a way to increment them. Once again, we make a PHP helper page — increment_counter.php — that works fundamentally the same as the code block that increments the click counts in our PHP postback page, except that it calls upon GET, instead of POST.

(Yes, you can use jQuery to make POST and GET AJAX requests. When I’m sending a single variable that just needs to meet a range, I prefer to use GET.)

$link = mysql_connect('server', 'user', 'password') or die('Cannot connect to database server');
mysql_select_db('database') or die('Cannot select database');

// if this is a postback ...
if(isset($_GET['color'])) {
	// create array of acceptable values
	$ok = array('red', 'green', 'blue', 'black');
	// if we have an acceptable value for color_name ...
	if(in_array($_GET['color'], $ok)) {
		// update the counter for that color
		$q = mysql_query("UPDATE colorcounter SET colorcount = colorcount + 1 WHERE colorname = '" . $_GET['color'] . "'") or die ("Error updating count for " . $_GET['color']);
	}
}

The Form And JavaScript

With our helper pages out of the way, we can proceed to work on the form and the jQuery JavaScript needed to make this work.

As usual, our first step is to procure and include a copy of jQuery on our page. Lately, I prefer to simply link to the version hosted by the Google Libraries API, but you can download jQuery directly and run it off your server if you prefer. Either way, call it somewhere in your page’s head section:

Next, I want to strip the form back to its basic components.


	Select a color:
	Red (0) |
	Green (0) |
	Blue (0) |
	Black (0)

Notice that I no longer have onclick events assigned to each radio button. That’s because I’ll wire up that event via $(document).ready.

More on that shortly. First, I want to declare a global variable, lastChecked. It will keep track of the value of the last radio button that was checked.

This will keep our page from recording multiple clicks on the same radio button, over and over again; clicks won’t be recorded unless, and until, a new value is clicked. In other words, if I just keep clicking Red, it will only be recorded once; if I then click Blue, that will be incremented, and clicking Red again will increment its value, once again.

var lastClicked = '';

I next want to write JavaScript functions for each of my helper pages. The first, getTotals(), will — as its name suggests — send an AJAX request to get the current click totals. It will then assign those totals to the labels after each radio button.

function getTotals() {
	// function to get click counts as JSON from helper page
	// expects get_count.php to be in same directory level

	$.ajax({
		type: "GET",
		url: "get_count.php",
		dataType: "json",
		error: function(xhr, status, msg) {
			alert("Failed to get click counts: " + msg);
		}
	})
	.done(function(data) {
		// loop through JSON variables, assign to count labels
		$.each(data, function(key, value) {
			var tmp = "#c_" + key;
			$(tmp).text(value);
		});
	});
}

I can now write the JavaScript function that will increment click counts. It’s named, quite cleverly, processClick, and takes as its argument the radio button (obj, for “object”) that was clicked.


function processClick(obj) {
	// function to increment click count via ajax
	// expects increment_count.php to be in same directory level

	if(lastClicked != obj.val()) { // don't count clicks on currently active radio button
		lastClicked = obj.val(); // set currently clicked radio button to this one

		var qs = "color=" + obj.val(); // set query string value

		$.ajax({
			type: "GET",
			url: "increment_count.php",
			data: qs,
			error: function(xhr, status, msg) {
				alert("Failed to process click count: " + msg);
			}
		})
		.done(function() {
			getTotals(); // update totals on successful processing
		});
	}
}

Finally, we want to use $(document).ready to initially populate the click total labels, and to assign processClick as the event handler for clicks on all the radio buttons on the page.


$(document).ready(function() {
	getTotals(); // get click totals on initial page load

	$(document).ready(function() {
		// add click incrementing event handler to all radio buttons on page
		$('input:radio').click(function() {
			processClick($(this));
		});
	});
});

No doubt some of you have noticed that I pass the entire radio button object to my processClick function, when all I really need to pass is its value. I see it as six of one, half-dozen of another, but if there’s a compelling argument against sending the entire object vs. its value, leave a comment and let’s discuss it. I’m always happy to clean up bad code.

You can see a working example here: http://www.dougv.com/demo/ajax_radio_clickcount/example2.htm

Notes And Code

A few notes about this project:

It’s not overly difficult for someone to click spam what I have here, either by simply clicking a radio button multiple times, or by calling the increment_count.php helper page multiple times.

A way around that would be to create single-use tokens for each iteration of the form, so that only one click per rendering of the form would be allowed.

You could do that by creating a session variable or cookie, if you’re using the PHP postback example; setting a Boolean in that session / cookie to be true the first time the form is submitted; and refusing to process the click if that session / cookie Boolean is true.

Or, for more security, you could add another MySQL table, which counts the number of times your form has been rendered. It would consist of an autoincremented primary key, which would act as an identifier for this instance of the form; and an tinyint column for recording a Boolean, which would indicate if the form had been submitted.

Every time your form is rendered, you would insert a new record in your form-rendering table, with a Boolean value of false indicating that the form has not yet been processed, then use mysql_insert_id to get the primary key value of the record you just created.

Using mcrypt, you would then create a single-use token by encrypting that record ID with a secret key only you know, and pass that to whatever method you are using to increment the counter.

That processing code would decrypt the token, make sure the record with that ID has a Boolean false as its “form has been submitted” value. If not, the code would not process the click; but if so, the code would process the click, then mark the form’s Boolean value as true.

An added benefit of this method is that it also inhibits forgeries and cross-site scripting attacks, since the only way to create legitimate tokens is via your secret key. (Theoretically, a hacker could crack your key, but use a sufficient passphrase and a good crypto algorithm, and the chances of anyone even taking a crack at it, nonetheless succeeding, are infinitesimal.)

Sounds more complex than it is. I’ve got demonstrating how to make a single-use token like this in Evernote blog column ideas. If significant interest is expressed, I’ll move it to the top of the list.

You can download the code in this post here: https://www.dougv.com/wp-content/uploads/2012/04/ajax_radio_clickcount.zip

All links in this post on delicious: http://delicious.com/dougvdotcom/how-to-increment-a-counter-in-mysql-based-on-a-radio-button-click

Tags: arrays, global constants, Google, radio buttons

Making A Simple WordPress Shortcode Plugin

Doug Vanderweide — Fri, 16 Mar 2012 19:42:53 +0000

If you’ve been reading this blog for any length of time, you’ve noticed my penchant for asides — brief digressions in which I explain a term, offer advice, or explain why I’m doing something a certain way.

Until yesterday, I was doing that via raw HTML, by adding a div tag and assigning it to the CSS class “aside,” which is defined in my theme’s style.css file:

This content will appear as an aside.

And that makes the text above look like this:

This content will appear as an aside.

Well, I’m just as lazy as the next guy, and just as careless, too. So sometimes I was forgetting to close that div, or was misspelling it, or otherwise making a general mess by typing a simple div tag.

Which got me to thinking: Why not make a WordPress shortcode plugin, to abbreviate and simplify this repetitive task?

Making your own shortcode is an excellent way to learn the basics of writing WordPress plugins. And once you get the hang of it, you’ll find WordPress plugin authoring isn’t all that hard to do, yet will make you infinitely more marketable as a Web developer.

An Overview Of WordPress Shortcodes

As the name implies, a WordPress shortcode is a way to use a simple, short directive to apply formatting, or append code, to a WordPress post or page.

More often than not, shortcodes are provided as part of a larger plugin solution.

For example, I use SyntaxHighlighter Evolved for code markup.

By typing a simple shortcode — [vbnet], [js], [php], etc. — I can quickly apply SyntaxHighlighter to my code samples. SyntaxHighlighter Evolved will also do me the favor of converting HTML entities, so I don’t have to do that by hand.

For example, this:

[php]
$foo = 'soup & sandwich';
echo $foo;
[/php]

Becomes this:

$foo = 'soup & sandwich';
echo $foo;

One thing a shortcode can’t do is prevent the WordPress visual editor from converting HTML entities. In other words, if you enter raw HTML in the HTML editor, then switch to the visual editor, it will probably mangle your code.

There are a number of plugins and strategies meant to prevent this from happening. I’ve tried a few and found them problematic. That’s why I pretty much edit text exclusively in the HTML editor. It’s just easier and more reliable.

Of course, if you don’t usually blog things that require special coding, sticking with the visual editor is fine. It handles shortcodes without incident, provided what you put inside the shortcode isn’t raw HTML.

Another example of how a shortcode can be used to add elements or affect style is WordPress’s built-in [caption] shortcode.

If you look at your post in the WordPress HTML editor, after uploading and adding a captioned photo, the code block that contains the photo will look something like this:

[caption id="attachment_4535" align="alignnone" width="585" caption="Blake Lively At The TIME 100 Gala, 26 April 2011. Photo credit: Flynet © 2011"][/caption]

As you can see, the [caption] shortcode takes a series of attribute arguments — id, align, width and caption — and wraps itself around the selected image.

When the post renders on the screen, the HTML produced looks like this:

Blake Lively At The TIME 100 Gala, 26 April 2011. Photo credit: Flynet © 2011

So, the [caption] shortcode:

outputs a containing div;
assigns the id of the caption as the id of that containing div;
adds the wp-caption CSS class to the div;
assigns the align attribute as a CSS class for that div;
sets the width of the image, plus 10 pixels, as the width of the caption container (to allow for 5 px padding left and right); and
places the value of the caption attribute into a paragraph element, which is made a child element of the caption div.

The final product looks like this:

Obviously, a WordPress shortcode is a very powerful way to render formulaic content. Any time you find yourself repetitively writing the same HTML or text, over and over again, you should consider making a shortcode, especially if hand-coding that pattern is time-consuming or accident-prone.

An Overview Of WordPress Plugins

A WordPress plugin is basically a way to intercept any of the many events that take place when working with WordPress, and to get WordPress to either alter how it goes about that task, add additional steps, or not do what it would normally do.

Usually, a plugin watches for when a post or page is created, and either adds some content, changes content somehow, or performs additional tasks related to the content being created or changed.

For example, there are plugins that will post a tweet any time you create a new post. Other plugins will optimize your post’s search engine indexing; figure out which previous posts are related to this post and hyperlink to them; automatically tag your post with relevant terms; etc.

Some plugins don’t deal with posts at all. There are plugins that extend and improve the built-in user management system; work with special post types; provide shopping carts or forums; or even fundamentally change the functionality of WordPress.

A plugin might leverage one event in the WordPress lifecycle, or several events. It might do one thing — such as our plugin — or scores of things.

Really, the sky is the limit. By design, WordPress is very extensible, which is a big part of the reason why it’s so popular.

“There’s a plugin for that” is pretty much cliche when it comes to WordPress. But in those cases when there isn’t a plugin for that, you can make one yourself.

Your First WordPress Plugin In Three Easy Steps

So let’s make a really basic WordPress shortcode plugin — one that will apply a specific style to the text it contains.

First, let’s put together the CSS for our [aside] shortcode. We’ll call that CSS style, appropriately, .aside:

div.aside {
	padding: 25px 50px;
	background: #efe;
	font-size: 12px !important;
	line-height: 150%;
	text-align: justify;
}

Put this CSS block in your theme’s style.css file.

Next, we need to fire up a text editor — Notepad or TextEdit will do fine; I use Notepad++ — and write the plugin that will provide our shortcode functionality.

Whatever text editor you use to write a plugin needs to save its files in ANSI encoding. In most plain-text editors, that’s the default encoding scheme.

If your text editor saves files in UTF-8, for example, when you activate a plugin, WordPress will report an error along the lines of:

The plugin generated X characters of unexpected output during activation. If you notice “headers already sent” messages, problems with syndication feeds or other issues, try deactivating or removing this plugin.

To fix that, set your text editor to save files in ANSI encoding, copy and paste your plugin code into new text files, replace your wrongly encoded plugin files with those new files, and reinstall the plugins.

According to the WordPress Codex, in order to create a plugin, we first need to provide a PHP comment block that contains some metadata: The name of our plugin, some author and version information, plus a description.

If we don’t include this metadata, we can’t manage it in the Plugins area of the WordPress admin section. So let’s do that first.

/*
Plugin Name: Aside Shortcode
Plugin URI: http://www.dougv.com/2012/03/16/making-a-simple-wordpress-shortcode-plugin/
Description: A simple plugin that adds an "aside" shortcode to WordPress
Version: 1.0
Author: Doug Vanderweide
Author URI: http://www.dougv.com
License: GPL3
*/

With that out of the way, we can actually create our shortcode. We do that with a simple PHP function.

Our function should be named something other than what we want to use as a shortcode. In other words, I want to use [aside] as my shortcode; so I don’t want to call this function aside. Instead, I’ll name it aside_shortcode.

function aside_shortcode($atts, $content = null) {
   return '' . $content . '';
}

Note the two arguments. I’ll discuss those shortly.

As you can see, our function simply adds a div tag, with the aside class, around the content contained inside our shortcode tags.

We now need to give our shortcode a name, and tell it to use this function when it finds those shortcode tags. That’s handled via the add_shortcode function, which is built into WordPress:

add_shortcode('aside', 'aside_shortcode');

So, here’s what our entire plugin PHP file looks like:

' . $content . '';
}

add_shortcode('aside', 'aside_shortcode');
?>

We just save this file with the name aside_shortcode.php.

Final step: Install and activate the plugin.

We can do that one of two ways:

Use FTP to create an aside_shortcut folder inside our wp-content/plugins folder; then upload our aside_shortcut.php file into that new folder.
Add aside_shortcut.php to a ZIP file; then use the Add New feature, under Plugins in the WordPress admin section, to upload and automatically create the appropriate directory.

With the plugin uploaded, we just need to go to the Plugins page in the WordPress admin section and activate it.

Congratulations! You’re officially a WordPress developer.

More On WordPress Shortcodes

I previously noted that aside_shortcode took two arguments: $attrs and $content.

(Technically, a shortcode function can take a third argument, $code, which is a reference to the shortcode name itself. But do me a favor and forget about that. There’s a reason for it, but it’s very specialized and almost always inapplicable.)

If I were writing a shortcode that had attributes / arguments — for example, such as this:

[caption id="attachment_4535" align="alignnone" width="585" caption="Blake Lively At The TIME 100 Gala, 26 April 2011. Photo credit: Flynet © 2011"]

– then, when I wrote my shortcode handler function, $attrs would be an associative array of all the attributes that were added to my shortcode:

$attrs['id'] = 'attachment_4535';
$attrs['align'] = 'alignnone';
$attrs['width'] = 585;
$attrs['caption'] = 'Blake Lively At The TIME 100 Gala, 26 April 2011. Photo credit: Flynet © 2011';

In our [aside] shortcode, we’re not using attributes / arguments, so $attrs is a zero-length array.

But suppose you want to write a shortcode that can apply any of several CSS classes to a selection of text. Let’s call that shortcode “styler,” and let’s suppose it takes an argument with the name “class,” which will be the CSS class you want to assign to the enclosed text, via a span tag.

We can change our shortcode function to handle that:

function styler_shortcode($atts, $content = null) {
   return '' . $content . '';
}

add_shortcode('styler', 'styler_shortcode');

So, if we had this in our post content:

Four score and [styler class="big"]seven years[/styler] ago, our fathers [styler class="small"]brought[/styler] forth on this [styler class="green"]continent[/styler] ...

it would render this HTML:

Four score and seven years ago, our fathers brought forth on this continent ...

Again, you’d just need to add the relevant CSS classes to your theme’s style.css file, and those styles would be applied to the rendered text.

Actually, you could write your plugin in a way that it either adds the relevant CSS to your WordPress installation’s head, or references a stylesheet specific to the plugin; or even provide an admin settings panel that lets end users define styles.

That’s more advanced stuff, so I won’t get into it here. Just know that again, the possibilities are endless.

There’s a security risk in using the $attrs array. Namely, we’re not sure the attributes sent are ones we expect or want. In other words, people can send arbitrary arguments to our shortcode; and that can contaminate the output.

As the Shortcode API documentation notes, you can protect against this by using the built-in shortcode_attrs() function, which sanitizes the $attrs array before processing. As this is an introduction, I won’t get into that. I’ll just caution you that if your blog is used by multiple authors, any time you create a shortcode that relies on attributes, you’ll want to sanitize them first.

By now, you’ve probably figured out that $content represents all the text that appears between opening and closing shortcode tags.

In other words, given this:

[caption id="attachment_4535" align="alignnone" width="585" caption="Blake Lively At The TIME 100 Gala, 26 April 2011. Photo credit: Flynet © 2011"][/caption]

$content is:

The default value of $content is null because in some cases, we may want to write a shortcode that simply renders some pre-determined content on the screen.

For example, suppose you have a couple standard lines of text you use to conclude a post:

Copyright © 2012 Doug Vanderweide.
I distribute code under the GNU GPL version 3.
I herd u liek mudkipz.

Since these lines are static, we can call them via a very simple shortcode. We’ll call that shortcode “kicker,” which will take an ID attribute of 1, 2 or 3, for whichever line we want to output; if no such attribute is specified, the third line will be used.

Thus, our shortcode function would be:

function kicker_shortcode($atts, $content = null) {
	switch($attrs['id']) {
		case 1:
			$out = 'Copyright © 2012 Doug Vanderweide.';
			break;
		case 2:
			$out = 'I distribute code under the GNU GPL version 3.';
			break;
		default:
			$out = 'I herd u liek mudkipz.';
	}
   return $out;
}

add_shortcode('kicker', 'kicker_shortcode');

After installation, we call that shortcode in our posts or pages thus:

Thanks for reading! [kicker id="1"]

As in, no need for content / containing shortcode tags. Wherever we put that single shortcode tag, there will be our output:

Thanks for reading! Copyright © 2012 Doug Vanderweide.

You can download the sample shortcode plugins here: http://www.dougv.com/wp-content/uploads/2012/03/wordpress_shortcodes.zip

I have added three separate plugin files to the download code, each representing one shortcode. But if you want to have a single plugin that contains multiple shortcodes, by all means, combine the PHP code into a single file.

In other words, you could strip out the PHP comments from two of these three plugin files, copy and paste the functions into one of them, change the plugin name of that file to be something more appropriate (e.g., “Style Shortcodes”), and install a that single plugin file. If you copy and paste correctly, all three shortcodes will be available in a single file.

All links in this post on delicious: http://delicious.com/dougvdotcom/making-a-simple-wordpress-shortcode-plugin

Tags: arrays, blogging, html entities, metadata, notepad++

Displaying Selected YouTube Data API Thumbnails On A Web Page Via ASP.NET Web Forms

Doug Vanderweide — Thu, 15 Mar 2012 23:37:41 +0000

Previously, I blogged about “Displaying Selected YouTube Video Thumbnails On An ASP.NET Web Forms Page,” when you know the video IDs of the thumbnails you want to hyperlink.

A reader recently asked me how to hyperlink YouTube video thumbnails based on searching for a keyword. I promised to address that, so here goes.

Interestingly enough, searching the YouTube Data API is accomplished in a REST-like manner quite similar to the methodology I used for shortening URLs in ASP.NET via the bit.ly API.

Form a simple request URL to the YouTube Data API that contains the appropriate search parameters;
Use a WebRequest to send that URL to Google, which returns an XML document with results;
Use WebResponse to dump that stream into an XmlDocument;
Use XPath and XmlNode‘s SelectNodes method to recursively get the thumbnails from each entry; and
Bind up a pile of Hyperlink controls, which are added dynamically to a Panel control.

Sounds more complicated than it actually is. Let’s do it.

Overview Of The YouTube Data API

First step in leveraging the YouTube Data API: Get a Youtube Data API developer key.

As of this writing, you can keyword search query the YouTube Data API without supplying a developer key and get a useful response. However, Google wants you to supply a key, and a time may come where you must supply one. So take the 30 seconds required to register a key.

With our developer key in hand, we can proceed to build a simple keyword query. In this case, we’re going to get the top 12 videos that meet the search criteria “Miss Maine.”

For our purposes, “top 12 videos” means the 12 videos YouTube deems most relevant to the exact search term “Miss Maine.” In other words, our search is going to work exactly as though we had typed “Miss Maine” (including the double quotes) into the search box at youtube.com; the results we get back should be the same as the first 12 results we would get via a default search on youtube.com. (For the usual vague reasons, this won’t always be the case, but the results sets will be similar.)

The YouTube Data API tells us that, to do a search query, we can GET a request to its servers, with our search parameters as querystring variables, and Google will return to us an XML document containing records that match our query.

Because we’re not requesting user-specific data, we don’t need to authenticate our requests; as previously mentioned, we don’t even need to send along our developer key. (But, again, being courteous and thorough, we will send it, as requested.)

So, the methodology we use to get our records is exactly the same as we used to shorten a URL via the bit.ly API; we’re going to create a URL containing all our parameters, then make a WebRequest to the YouTube Data API; we’ll get its returned XML via a WebResponse, and push that into an XmlDocument, from which we can extract the information we want.

As we build this solution, let’s keep some non-technical things in mind:

Many of the videos on YouTube aren’t very good, in the aesthetic, intellectual or civil senses. In other words, much of what is on YouTube is terrible.
A sizable number of YouTube videos are spam.
It’s de rigueur to keyword spam video descriptions, especially if the video is spam.
Trolling is mandatory for YouTube video comments, and you’ll probably be linking directly to that.

We can mitigate the damage, somewhat, by applying the smartSearch filter, being very specific with our search term, including specific words we don’t want returned in our results, restricting the categories in which we want to search … in other words, being as specific as we can about what we want to see.

However, I’ll bet a dollar to doughnuts that if you request any sizable result set on any generic search term, your results set is going to contain items you wish weren’t in there. That’s just the nature of the thing, and there’s little that can be done about it.

Querying The YouTube Data API

There are three distinct phases in our solution: query the API, get the information we need from its response, and put that information on the page. Therefore, we will use three distinct functions and subroutines to do the work.

First up, a function to submit our query. It accepts as arguments our developer key and a formatted querystring, and it puts the response into an XML document for us. If the request fails, it will report as much in a Label control, and return an empty XML document.

Function QueryYouTubeDataAPI(ByVal strAPIKey As String, ByVal strQuery As String) As XmlDocument
	'Returns empty XMLDocument on failure, results XML on success
	'This function requires your page to have a label control named lblStatus for error reporting

	'build URL to shorten method resource
	Dim strUri As New StringBuilder("https://gdata.youtube.com/feeds/api/videos?")
	strUri.Append("v=2")
	strUri.Append("&")
	strUri.Append(strQuery)
	strUri.Append("&key=")
	strUri.Append(Server.HtmlEncode(strAPIKey))
	strUri.Append("&prettyprint=true") 'adds line breaks & white space to response; useful for debugging

	Dim objRequest As HttpWebRequest
	Dim objResponse As WebResponse
	Dim objXML As New XmlDocument() 'This is the document the function will return

	Try
		'create request for shorten resource
		objRequest = WebRequest.Create(strUri.ToString)
		'since we are passing querystring variables, our method is get
		objRequest.Method = "GET"
		'act as though we are sending a form
		objRequest.ContentType = "application/x-www-form-urlencoded"
		'don't wait for a 100 Continue HTTP response
		objRequest.ServicePoint.Expect100Continue = False
		'since we are using get, we need not send a request body; set content-length to 0
		objRequest.ContentLength = 0

		'read the Data API response into XML document
		objResponse = objRequest.GetResponse()
		objXML.Load(objResponse.GetResponseStream())
	Catch ex As Exception
		lblStatus.Text = "Error querying YouTube Data API. Message: " & ex.Message
	End Try

	'send XML Document
	Return objXML
End Function

A note about error trapping: The error trapping I have used here does not check whether or not the YouTube Data API returns a sensible result. It only checks that we formed a request and the API responded.

If there is a problem with the query string you send to the YouTube Data API, or your request is somehow malformed, the API will return to you the 25 most popular videos for the day, as of the time of the request.

You can check to see if the YouTube Data API is seeing your request properly by looking for the /feed/link rel=’self’ node:

If the URL in that node doesn’t resemble the one you sent, most likely your querystring is malformed; but possibly, there are other errors in your request URL.

Creating The Query String

You’ll notice that I am not very specific about the query string argument passed to the previous function. That’s because I want to give you maximum flexibility, and the best way to do that is to allow for a free-form query string to be supplied.

So let’s look at how I’ll build my sample query string. I’ll do that, of course, with key-value pairs. That is, I will specify the parameter I want to send, and set it as equal to something.

First up, the search term. We’re going to use “Miss Maine.” So, the first part of our query string is:

q="Miss Maine"

Note that the YouTube Data API says all parameters should be URL encoded. Our function will take care of that for us when we create the WebRequest, so don’t bother encoding your query string in advance, or you’ll get errors. Just pass it to the function as natural text.

Technically, we can stop here and let the default API request variables take over. But I want to be more specific about my search results, so let’s add some more parameters.

Next, I’ll limit my responses to 12 records, by appending that key-value pair to my string:

q="Miss Maine"&max-results=12

I’d like only short (under 4 minutes) videos:

q="Miss Maine"&max-results=12&duration=short

To cull at least some offensive material, I’ll use strict safeSearch:

q="Miss Maine"&max-results=12&duration=short&safeSearch=strict

Finally, I’m going to specifically ask for English language videos that best match my search term. So my completed query string is:

q="Miss Maine"&max-results=12&duration=short&safeSearch=strict&orderby=relevance_lang_en

You may be wondering why I am not using the GData .NET Library to accomplish this search. That’s an excellent option in general, but it involves installing components on your local machine and Web server.

For many Web developers, adding DLLs to a Web server or installing assemblies on a workstation can be problematic, if not prohibited outright. So I want to use as many built-in resources as possible. However, if you can install the GData API, consider going that route.

Examining The XML Response

Assuming our request goes through — that is, our Web server actually delivers the request to YouTube, which in turn sends a response back — we now have an XML document in storage that contains the information we requested. (Or, in the event of a malformed request, the default results set previously described.)

You can see the XML document returned for my demo query string here: http://www.dougv.net/demos/youtube_data_api_thumbs/videos.xml

In our case, we want some specific data for each video:

Its title;
the default thumbnail; and
the URL to its player page on youtube.com.

We get back lots more useful information in the XML response: The name of the uploader, the run time of the video in seconds, its likes and dislikes count, the categories and keywords assigned to the video, the date and time it was uploaded, its view count, etc. But for this solution, we’ll only use the video’s title, thumbnail and URL.

Although the actual XML document contains much more data per entry, for our purposes, the relevant structure of the document looks like this (I’ve stripped out extraneous attributes):

In practice, the parts of an entry (that is, an individual record) that we want to use look like this:


	Miss Maine USA 2011

Traversing The XML Response Document

There are a number of ways in ASP.NET to traverse / select nodes from an XML document.

The way most Web resources will tell you to do it is with LINQ to XML. LINQ allows us to make a SQL-like query of the XML document, and receive in return a SQL-like recordset, which we can then use to ouput our data.

LINQ to XML is definitely the way to go if you’re going to work with a large XML document, or need to extract data in a very complicated way (e.g, get various node attributes; select nodes based on data contained in other nodes).

LINQ to XML would work here, too, but I think it’s a bit of overkill, considering we only need three values from 12 records. So I’m just going to use the built-in parsing functions that are part of XmlDocument, a class we have already encumbered. Also, I’m more used to using XPath than I am LINQ.

Another way is to create an XML Stylesheet (XSLT) which will “flattened out” the XML document into 12 nodes (each representing one record) with three attributes (each representing the title, URL and thumbnail for that record). I describe that methodology at “Using National Weather Service XML Feeds With ASP.NET, ADO.NET And XSL.”

But I am trying to limit the number of things you have to learn at one time to accomplish this task. In my estimation, it’s a bit easier to hack together an XPath expression than to write an XSLT, so this is how I’m proceeding.

We’ll need three XPath expressions that will give us, for each of the 12 videos returned:

the inner text of title,
the inner text of the link node that has the attribute ‘alternate’, and
the url attribute of whichever media:thumbnail node that has the additional attribute of yt:name=’default’.

The XML returned by the YouTube Data API uses several namespaces (atom, media, yt, etc.). Therefore, we first have to make reference to those namespaces in order to traverse the response XML.

We do that by creating an XmlNamespaceManager, then adding the specification URLs to it:

'add namespaces so we can traverse this thing
Dim xmlNSM As New XmlNamespaceManager(xmlDoc.NameTable)
xmlNSM.AddNamespace("atom", "http://www.w3.org/2005/Atom")
xmlNSM.AddNamespace("media", "http://search.yahoo.com/mrss/")
xmlNSM.AddNamespace("yt", "http://gdata.youtube.com/schemas/2007")

(We got these URLs from the root (“feed”) node’s attributes. That’s where they always appear, any time an XML document references namespace(s).

An XML namespace is basically a description of what elements an XML document will have, the kinds of data each node will contain, and other useful technical information that helps a parser understand what it is looking at and how to consume it.

Now that we have the namespaces referenced, we can go ahead and create our XPath arguments. Here they are:

'let's get our entry node values
Dim xmlTitleNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/atom:title", xmlNSM)
Dim xmlURLNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:player", xmlNSM)
Dim xmlThumbNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:thumbnail[@yt:name='default']", xmlNSM)

Note that I use the namespace:node syntax to select the nodes I want. If I didn’t do that, the XML parser wouldn’t understand what I meant if I said simply, “feed”, for example. I have to let the parser know which namespace the element “feed” belongs to, so it can find it.

With our nodes on hand, we just need to iterate each XmlNodeList, get our values, and pass them to a subroutine that adds hyperlinked thumbnails to our page.

We can use a simple For loop here, because by design, the number of title, thumbnail and hyperlink nodes will be the same, and the nodes will be in the same order in all three XmlNodeLists. (That is, the first node in each XmlNodeList will be for the first record; the title, URL and thumbnail values will match up, because they were listed in order in the XML document.)

'For loop will iterate them; by definition, counts are the same for each XmlNodeList
For I = 0 To xmlTitleNodes.Count - 1
	strTitle = xmlTitleNodes.Item(I).InnerText
	strURL = xmlURLNodes.Item(I).Attributes("url").Value
	strThumb = xmlThumbNodes.Item(I).Attributes("url").Value

	CreateHyperlinkedThumb(strTitle, strURL, strThumb)
Next

Here’s what the entire subroutine looks like:

Sub MakeThumbnailLinks(ByVal strAPIKey As String, ByVal strQuery As String)
	'create XML document we will parse
	Dim xmlDoc As New XmlDocument()
	xmlDoc = QueryYouTubeDataAPI(strAPIKey, strQuery)

	'add namespaces so we can traverse this thing
	Dim xmlNSM As New XmlNamespaceManager(xmlDoc.NameTable)
	xmlNSM.AddNamespace("atom", "http://www.w3.org/2005/Atom")
	xmlNSM.AddNamespace("media", "http://search.yahoo.com/mrss/")
	xmlNSM.AddNamespace("yt", "http://gdata.youtube.com/schemas/2007")

	'let's get our entry node values
	Dim xmlTitleNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/atom:title", xmlNSM)
	Dim xmlURLNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:player", xmlNSM)
	Dim xmlThumbNodes As XmlNodeList = xmlDoc.SelectNodes("/atom:feed/atom:entry/media:group/media:thumbnail[@yt:name='default']", xmlNSM)

	'for debugging, we'll also get the url that represents what the Data API actually used
	Dim objNode As XmlNode = xmlDoc.SelectSingleNode("/atom:feed/atom:link[@rel='self']", xmlNSM)
	lblStatus.Text = "Request URL returned by YouTube Data API: " & Server.HtmlEncode(objNode.Attributes("href").Value)

	'strings to store values
	Dim strTitle As String
	Dim strURL As String
	Dim strThumb As String

	'looping variable
	Dim I As Integer

	'For loop will iterate them; by definition, counts are the same for each XmlNodeList
	For I = 0 To xmlTitleNodes.Count - 1
		strTitle = xmlTitleNodes.Item(I).InnerText
		strURL = xmlURLNodes.Item(I).Attributes("url").Value
		strThumb = xmlThumbNodes.Item(I).Attributes("url").Value

		CreateHyperlinkedThumb(strTitle, strURL, strThumb)
	Next
End Sub

Creating Thumbs From Results

The CreateHyperlinkedThumb subroutine takes the title, URL and thumbnail strings we got, applies them as properties to a Hyperlink control, and adds those controls to a Panel on our page.

Sub CreateHyperlinkedThumb(strTitle As String, strURL As String, strThumb As String)
	'create hyperlink
	Dim ctlLink As New HyperLink()

	'set values
	ctlLink.Text = strTitle
	ctlLink.ToolTip = strTitle
	ctlLink.NavigateUrl = strURL
	ctlLink.ImageUrl = strThumb
	ctlLink.CssClass = "margin-5"
	ctlLink.Target = "video"

	'add to panel
	pnlThumbs.Controls.Add(ctlLink)
End Sub

And with that, we’re done!

To invoke, we simply call the MakeThumbnailLinks subroutine, passing to it our YouTube Data API developer key and the querystring we built:

MakeThumbnailLinks("YOUTUBE_DATA_API_DEVELOPER_KEY", "q=""Miss Maine""&max-results=12&duration=short&safeSearch=strict&orderby=relevance_lang_en")

You can see a working demo here: http://www.dougv.net/demos/youtube_data_api_thumbs/Example2.aspx

The code for this solution, as well as “Displaying Selected YouTube Video Thumbnails On An ASP.NET Web Forms Page,” can be downloaded here: http://www.dougv.com/wp-content/uploads/2012/03/youtube_data_api_thumbs.zip

All links in this post on delicious: http://delicious.com/dougvdotcom/displaying-selected-youtube-data-api-thumbnails-on-a-web-page-via-asp-net-web-forms

Tags: bit.ly, Google, Microsoft, REST, RSS, XPath, XSLT

Preloading Images With JavaScript

Doug Vanderweide — Mon, 12 Mar 2012 23:10:43 +0000

Anne Hathaway

In my recent travels through old blog posts, I’ve noticed a number of occasions where I’ve discussed how to preload images.

Almost all those examples are stupid or just plain wrong. For that, you have my apologies, and I aim to rectify those mistakes with this post.

First, why would we want to preload an image? Simply put, we intend to show it later on our Web page — either as a result of a mouseover, or a click, or some other sort of Document Object Model (DOM) event.

For example, maybe we want to mouseover a series of thumbnails, and show a larger version of that image in the same place.

Rather than making the end user wait for a new image to load as a result of doing something on a Web page, it makes sense to load the image we intend to show in advance, so it will display almost instantaneously as a result of an event.

I’ll first show why two of my previous methods for preloading images are wrong or dumb, then describe two correct ways to preload images: via basic JavaScript and via jQuery.

The lovely Anne Hathaway will be our model.

The Wrong Way: A Single img Tag And JavaScript src Loop

I previously favored preloading images by applying a single HTML img tag on a page, one without a src attribute and also hidden via CSS:

Then, I would create a preloadImages() function, which took as its arguments the paths to the images I wanted to preload. I would then loop through those arguments, setting the src attribute for hiddenImg to be the paths passed to the function:


function preloadImages() {
	var img = document.getElementById('hiddenImage');
	var i;

	for(i = 0; i < arguments.length; i++) {
		img.src = arguments[i];
	}
}

window.onload = function() {
	preloadImages('i/d01.jpg', 'i/d02.jpg', 'i/d03.jpg', 'i/d04.jpg', 'i/d05.jpg');
}

Unfortunately for me, this simply doesn’t work. And the reason why it doesn’t work is because, if I pass more than one image to the preloadImages() function, they simply don’t have enough time to load before the next image is called.

In short, the only image that actually preloads under this methodology is the last one in the arguments list. And that pretty much makes this approach useless.

You can see what I am talking about here: http://www.dougv.com/demo/js_preload_images/wrong.htm

The Dumb Way: Using HTML And CSS

Better in the technical sense, but just as bad in the practical sense, is using HTML and CSS alone to preload images.

In this case, we simply add, to the end of the page, a bunch of img tags that bring in the pictures we intend to display via JavaScript, based on a DOM event. We either place them inside a hidden div or assign those images to a CSS class that sets them to display: none.

You can see an example here: http://www.dougv.com/demo/js_preload_images/index.htm

While this is better than the wrong way, because at least all the images manage to get preloaded, this is dumb.

Suppose our client has disabled JavaScript. This approach doesn’t care; the images for our JavaScript-driven effects will still load in the background.

In other words, we encumber all the overhead of bringing in effect images for an effect that will not be seen. That’s wasteful. Maybe it’s no big deal, given that the average user has plenty of computing power and bandwidth to spare, but it’s inelegant. And by now, you know that little puts a burr in my butt more than wasteful code.

Example 1: Using JavaScript’s appendChild() Method

The proper way to preload images with JavsScript (and this time, I mean it) is to:

create an HTML div that is hidden;
dynamically create an img element for each image we want to preload;
assign each of those dynamically created elements a src attribute that is a path to an image we want to preload; then
append that dynamic img element to the hidden div.

So first, the HTML:

And next, the JavaScript function:

function preloadImages() {
	var hiddenDiv = document.getElementById('hiddenImages');
	var i;
	var img;

	for(i = 0; i < arguments.length; i++) {
		img = document.createElement('img');
		img.src = arguments[i];
		img.alt = '';
		hiddenDiv.appendChild(img);
	}
}

To invoke this, we simply call the function:

preloadImages('i/b01.jpg', 'i/b02.jpg', 'i/b03.jpg', 'i/b04.jpg', 'i/b05.jpg');

Witness this in action: http://www.dougv.com/demo/js_preload_images/example1.htm

I prefer to put the preloadImages() function in the head of the page, and call it just before the closing body tag.

That way, the page gets a chance to load all its elements, before we start calling the images that we intend to show later. In other words, the page gets a chance to at least start loading the images it’s supposed to be displaying at first glance, before it starts loading the images we’ll show based on clicks / mouseovers / whatever.

Example 2: Using jQuery And document.ready

Of course, no discussion about JavaScript these days is complete without showing a jQuery version. So here’s mine.

In our case, we can use the same basic methodology as in Example 1. However, there are a few changes:

Because jQuery has the .hide() method built-in, we’ll just hide each image as we create it. That means the div into which we are adding these images need not be hidden at all.
We need to call $(document).ready() sometime after we create the hiddenImages div. Otherwise, jQuery doesn’t see where it’s supposed to add the images it’s creating, and thus silently fails. So, for simplicity’s sake, I am adding both the preloadImages() function and my call to $(document).ready() just before the closing body tag.

So here’s our hiddenImages div (again, no need to hide it):

And here’s the JavaScript we place, sometime after it, to preload:

function preloadImages() {
	for(var i = 0; i < arguments.length; i++) {
		$('').attr({ src: arguments[i], alt: '' }).appendTo('#hiddenImages').hide();
	}
}

$(document).ready(preloadImages('i/c01.jpg', 'i/c02.jpg', 'i/c03.jpg', 'i/c04.jpg', 'i/c05.jpg'));

Working example: http://www.dougv.com/demo/js_preload_images/example2.htm

It’s worth noting that there are a number of image effect examples and plugins at the jQuery tutorial site.

So, whether you prefer to go the old-school JavaScript route, or to get newfangled with jQuery, the proper way to preload an image for DOM-event-based display is to create image elements via JavaScript, then add those elements to some other element on the page — either an element that’s already hidden, or by hiding the images as they are created.

You can download the source code here: http://www.dougv.com/wp-content/uploads/2012/03/js_preload_images.zip

All links in this post on delicious: http://delicious.com/dougvdotcom/preloading-images-with-javascript

Tags: arrays, coding standards, DOM images, elegance

Displaying Selected YouTube Video Thumbnails On An ASP.NET Web Forms Page

Doug Vanderweide — Thu, 08 Mar 2012 16:27:59 +0000

I received a kind email recently from a reader, thanking me for my article titled “Retaining Values In A Form Following PHP Postback And Clearing Form Values After Successful PHP Form Processing“.

He also asked how he could use the YouTube Data API to search for videos by keyword; display thumbnails of those videos on an ASP.NET page; then either hyperlink to those videos on YouTube, or display them on his page via the YouTube Player API.

I’ll address that specific question in an upcoming post. First, I want to show how to do what the questioner asks if you already know the video IDs of specific YouTube videos you want to show on a page.

If you know the video ID(s) for the YouTube videos you want to display on a page, you can call them directly from YouTube’s image servers, thanks to a predictable URL and naming scheme, and hotlinking.

As this thread at Stack Overflow notes, every YouTube video has at least three thumbnails, numbered 1.jpg to 3.jpg. These images are 120px by 60px and are roughly taken at the start, middle, and end of the video, respectively.

The file named default.jpg is the default thumbnail at 120px x 90px resolution. Two others — 0.jpg and hqdefault.jpg — are 480px x 360px. All three of these default images are copies of whichever of the three video thumbnails has been set as the default frame capture for the video.

Finally, a maximum resolution version, available as maxresdefault.jpg, is available if there is a high-definition (720p or 1080p) version of the video. It measures 1280px x 720px. When there isn’t a high definition version of the video, this appears as the generic YouTube thumbnail icon, at 120px x 90px resolution.

So, here are all the thumbnail images available for Lisa Hannigan’s “What’ll I Do” music video (hover over each image to see its name):

And here are the thumbnails for a decidedly low-res video for The Monkees’s song, “Last Train To Clarksville”:

ASP.NET Web Forms Code To Display And Hyperlink Known YouTube Video Thumbnails

Here’s some simple ASP.NET Web Forms code to add hyperlinked thumbnails for five different YouTube videos, the IDs of which we know, to a Panel control. (I’m assuming here that the Panel control is named pnlThumbs.)

Sub AddVideoThumbs()
	'create ArrayList of known video IDs
	Dim arrIDs As New ArrayList
	arrIDs.Add("STxXS5lLunE")
	arrIDs.Add("Mwrl6bWfvrc")
	arrIDs.Add("4nTo8rjo-lM")
	arrIDs.Add("eVVXtknZVf0")
	arrIDs.Add("pxg113O_SRI")

	'some temp string variables for our processing loop
	Dim strTemp As String
	Dim strLink As String
	Dim strSrc As String

	For Each strTemp In arrIDs
		strSrc = "http://img.YouTube.com/vi" & strTemp & "/default.jpg" 'path to default thumbnail
		strLink = "http://www.YouTube.com/watch?v=" & strTemp 'path to video on YouTube

		Dim objLink As New HyperLink 'create hyperlink
		objLink.NavigateUrl = strLink
		objLink.ImageUrl = strSrc
		objLink.Text = strTemp 'sets alt text to ensure XHTML compliance
		objLink.Target = "video" 'will open all videos in same, new window / tab
		objLink.CssClass = "margin-0-5" 'a specific-to-me CSS class to space out the thumbnails

		pnlThumbs.Controls.Add(objLink)
	Next
End Sub

You can see this in action here: http://www.dougv.net/demos/youtube_data_api_thumbs/Default.aspx

In an upcoming post, I will describe how to use the YouTube Data API to search for videos, display their thumbnails on a page, and hyperlink to those videos. I’ll also package all the demo code together for download at that time.

Update, 15 March 2012: The second part of this tutorial, and downloadable demo code for both examples, is available at Displaying Selected YouTube Data API Thumbnails On A Web Page Via ASP.NET Web Forms.”

All links in this post on delicious: http://delicious.com/dougvdotcom/displaying-selected-youtube-video-thumbnails-on-an-asp-net-web-forms-page

Tags: arrays, Google, hotlinking, Microsoft, Stack Overflow

Converting Latitude And Longitude Coordinates Between Decimal And Degrees, Minutes, Seconds

Doug Vanderweide — Wed, 07 Mar 2012 23:18:18 +0000

I received an email yesterday from a reader who needed help with implementing my blog post, “Calculating The Bearing And Compass Rose Direction Between Two Latitude / Longitude Coordinates In PHP.”

His problem: He has latitude and longitude coordinates in his database tables that are in degrees-minutes-seconds (DMS) format, rather than decimal format. In other words, his coordinates look more like 55° 45′ 06″ N, 37° 37′ 04″ E than 55.751667, 37.617778.

So, my reader needed a way to convert back and forth between the two.

Fortunately, that’s easily done with a couple PHP functions. And since there’s not a lot that comes up in a Google search about how to make those conversions in PHP (lots of standalone converters, some JavaScript code, little PHP), I’ll fill the void here.

I should also note this function can be used for my other PHP latitude / longitude post, “Getting All ZIP Codes In A Given Radius From A Known Point / ZIP Code Via PHP And MySQL.”

Converting From DMS To Decimal

The easier of the two conversions is going from degrees-minutes-seconds to decimal. That’s because just as in an hour of time, each degree has 60 minutes, and each minute has 60 seconds.

So, to convert from DMS to decimal, we simply start with the degrees; then add the minutes value divided by 60, and the seconds value divided by 3600 (60 * 60). If the coordinates are South (latitude) or West (longitude), we need to make the decimal coordinate negative; North and East coordinates are positive.

Thus, a North latitude or East longitude is converted as:

$decimal = $degrees + ($minutes / 60) + ($seconds / 3600);

A South latitude or West longitude is calculated as:

$decimal = ($degrees + ($minutes / 60) + ($seconds / 3600)) * -1;

Here’s a simple function that makes these conversions. It takes, as arguments, the degrees, minutes and seconds of either a latitude or longitude coordinate, as well as its direction (N, S, E or W), and returns a decimal response.

function DMS2Decimal($degrees = 0, $minutes = 0, $seconds = 0, $direction = 'n') {
	//converts DMS coordinates to decimal
	//returns false on bad inputs, decimal on success

	//direction must be n, s, e or w, case-insensitive
	$d = strtolower($direction);
	$ok = array('n', 's', 'e', 'w');

	//degrees must be integer between 0 and 180
	if(!is_numeric($degrees) || $degrees < 0 || $degrees > 180) {
		$decimal = false;
	}
	//minutes must be integer or float between 0 and 59
	elseif(!is_numeric($minutes) || $minutes < 0 || $minutes > 59) {
		$decimal = false;
	}
	//seconds must be integer or float between 0 and 59
	elseif(!is_numeric($seconds) || $seconds < 0 || $seconds > 59) {
		$decimal = false;
	}
	elseif(!in_array($d, $ok)) {
		$decimal = false;
	}
	else {
		//inputs clean, calculate
		$decimal = $degrees + ($minutes / 60) + ($seconds / 3600);

		//reverse for south or west coordinates; north is assumed
		if($d == 's' || $d == 'w') {
			$decimal *= -1;
		}
	}

	return $decimal;
}

You can test this function here: http://www.dougv.com/demo/php_lat_lon_conversion/dms2decimal.php

You call this function as you would any built-in PHP function. For example, assuming you have DMS coordinate values stored in the PHP variables $deg, $min, $sec and $dir, you could assign a value to a decimal equivalent variable, $dec, thus:

$dec = DMS2Decimal($deg, $min, $sec, $dir);

Converting Decimal To DMS

Going from decimal coordinates to degrees-minutes-seconds is a bit more complicated.

Set the DMS direction to South (negative latitude), West (negative longitude), North (positive latitude) or East (positive longitude);
set the decimal to its absolute (i.e., unsigned) value;
set degrees to be the floor of the decimal;
subtract degrees from decimal, to get its fractional portion;
multiply that by 3600, to get the total number of seconds;
divide the seconds by 60, to get the total number of minutes;
subtract the total number of minutes, times 60, from seconds; then
set seconds value to its floor.

To accomplish this task via a PHP function, we will pass to it six arguments: the decimal coordinate; variables for degrees, minutes, seconds and direction; and a Boolean indicating if this is a latitude or longitude coordinate.

Four of our six arguments — degrees, minutes, seconds and direction — will be passed by reference. What that means to a layman is, the values of those arguments will be set by the function, without the need to specifically set them via an evaluation.

In other words, in PHP, you usually set a variable value via a function like this:

$foo = myfunction($bar);

But we’ll do it this way:

myfunction($bar, $foo);

An example of that in practice follows the function code.

function DecimalToDMS($decimal, &$degrees, &$minutes, &$seconds, &$direction, $type = true) {
	//set default values for variables passed by reference
	$degrees = 0;
	$minutes = 0;
	$seconds = 0;
	$direction = 'X';

	//decimal must be integer or float no larger than 180;
	//type must be Boolean
	if(!is_numeric($decimal) || abs($decimal) > 180 || !is_bool($type)) {
		return false;
	}

	//inputs OK, proceed
	//type is latitude when true, longitude when false

	//set direction; north assumed
	if($type && $decimal < 0) {
		$direction = 'S';
	}
	elseif(!$type && $decimal < 0) {
		$direction = 'W';
	}
	elseif(!$type) {
		$direction = 'E';
	}
	else {
		$direction = 'N';
	}

	//get absolute value of decimal
	$d = abs($decimal);

	//get degrees
	$degrees = floor($d);

	//get seconds
	$seconds = ($d - $degrees) * 3600;

	//get minutes
	$minutes = floor($seconds / 60);

	//reset seconds
	$seconds = floor($seconds - ($minutes * 60));
}

You can see this in action here: http://www.dougv.com/demo/php_lat_lon_conversion/index.php

Let’s suppose you have the variables $deg, $min, $sec, $dir, for degrees, minutes, seconds and direction, respectively. You also have a decimal coordinate stored in $dec.

This is how you would invoke the function above to assign values to your DMS variables:

//if your decimal coordinate is a latitude:
DecimalToDMS($dec, $deg, $min, $sec, $dir, true);

//if your decimal coordinate is a longitude:
DecimalToDMS($dec, $deg, $min, $sec, $dir, false);

You can download the demo code here: http://www.dougv.com/wp-content/uploads/2012/03/php_lat_lon_conversion.zip

All links in this post on delicious: http://delicious.com/dougvdotcom/converting-latitude-and-longitude-coordinates-between-decimal-and-hours-minutes-seconds

Tags: converters, Google, latitude / longitude

Disable Windows Antivirus When Installing aspell English Dictionary

Doug Vanderweide — Wed, 29 Feb 2012 19:49:35 +0000

A lesson I learned the hard way today, while installing aspell support for Notepad++:

If you’re installing Kevin Atkison’s English dictionary for aspell, you need to disable your antivirus program (at least, if you’re using Avast, as I am).

If you don’t, the dictionary installer can’t write its unpacked files to disk and will fail silently. As in, it just plain closes, and Notepad++ will report something along the lines of “Aspell and/or dictionaries are missing.”

FYI.

Also, if you haven’t heard of Notepad++, you should check it out. It’s an open-source, GPL-licensed Win32 text editor. (It runs perfectly fine in Win64).

Highly extensible via plugins, translated to all kinds of languages, exceptionally powerful, with support for syntax highlighting in just about every programming language under the Sun and syntax checking for a fair number of them, too.

It’s pretty much the only tool I use any more for Web coding, even when writing ASP.NET Web Forms. (I still use Visual Studio for some Windows coding. But Notepad++ has completely replaced Dreamweaver.)

All links in this post on delicious: http://delicious.com/dougvdotcom/disable-windows-antivirus-when-installing-aspell-english-dictionary

Tags: developer tools, Microsoft, MSDN, notepad++, open source, Visual Studio

dougv.com « Doug Vanderweide » Programming

The Basics Of Avoiding MySQL Injection Attacks In ASP.NET Web Forms

Additional Thoughts And Clarification

Related Posts

TEA Time: New England GiveCamp 2012 Recap

Ticketing Turns To Overhaul

Meet TEA

Meet The Team

The Project

The Results

A Round Of Applause

Related Posts

Getting All ZIP Codes In A Given Radius From A Known Point / ZIP Code Via ASP.NET

Step 1: Helper Functions

Step 2: Functions To Plot Latitude And Longitude

Step 3: Create The Form

Step 4: Get The Initial Point

Step 5: Bind The GridView

Related Posts

How To Increment A Counter In MySQL Based On A Radio Button Click

Example 1: PHP / MySQL Postback

Example 2: Using AJAX

The get_counts.php Helper Page

The increment_counter.php Helper Page

The Form And JavaScript

Notes And Code

Related Posts

Making A Simple WordPress Shortcode Plugin

An Overview Of WordPress Shortcodes

An Overview Of WordPress Plugins

Your First WordPress Plugin In Three Easy Steps

More On WordPress Shortcodes

Related Posts

Displaying Selected YouTube Data API Thumbnails On A Web Page Via ASP.NET Web Forms

Overview Of The YouTube Data API

Querying The YouTube Data API

Creating The Query String

Examining The XML Response

Traversing The XML Response Document

Creating Thumbs From Results

Related Posts

Preloading Images With JavaScript

The Wrong Way: A Single img Tag And JavaScript src Loop

The Dumb Way: Using HTML And CSS

Example 1: Using JavaScript’s appendChild() Method

Example 2: Using jQuery And document.ready

Related Posts

Displaying Selected YouTube Video Thumbnails On An ASP.NET Web Forms Page

ASP.NET Web Forms Code To Display And Hyperlink Known YouTube Video Thumbnails

Related Posts

Converting Latitude And Longitude Coordinates Between Decimal And Degrees, Minutes, Seconds

Converting From DMS To Decimal

Converting Decimal To DMS

Related Posts

Disable Windows Antivirus When Installing aspell English Dictionary

Related Posts