Cesium the weekend!

lsugrid

Looked into Cesium, the WebGL JavaScript library for creating 3D globes, and 2.5D and 2D maps, today.

So far, so good. The tutorials are great, and documentation relatively well populated. Installation was painless. The major quirks so far have been due to funny interactions between Windows Chrome and DirectX, which I won’t pretend to understand, but apparently it makes it so outlineWidth is not supported, which means all polygon outlines are set permanently to 1px. Not a huge deal, but I’m guessing I’ll run into more technical stumbling blocks as I delve further in.

Here’s a quick example, which draws a 10 degree grid over the Earth, and places an LSU beacon above Tiger Stadium.

Note: must use WebGL-enabled browser, which means Chrome or newer version of Firefox/IE/etc. You can check if your browser is compatible here.

GeoNet Prime Answering Times

geonet_primetimes

ESRI runs a biannual contest to encourage participation on their help forum, GeoNet. Prizes go to the top ten point-getters. I found out during the last contest that it takes a great deal of time and persistence to keep up with the top of the pack (I ended up 5th). There are several tips for optimizing your effort (some of which I outlined, somewhat sarcastically, here).

One such tip is: get on GeoNet during times when there are the greatest number of fresh, unanswered questions. If you’ve spent any time on GeoNet, you have likely noticed that questions are generally asked during North American working hours, when people are struggling to get through their work-related GIS tasks. I wanted to put some better numbers to this idea, so I set about gathering the data myself.

All the information is there: each post has the date/time it was asked written right there in the posting. You could click on each post and record that date/time into an Excel and be done with it, but that would be awfully tedious. This is where screen scraping comes in. Screen scraping is the direct equivalent of having your computer control your web browser: click here, find this part of the HTML code, read it, and do something with it.Luckily, your computer doesn’t care if it has to spend all day doing the same thing over and over and over…

I chose to use Python, but you can do this in other languages, as well. Useful libraries to download are Requests and lxml. I use Requests for making the, you guessed it, “requests”, which are similar to typing a URL in the address bar of your browser. I use lxml for parsing and traversing the returned HTML code, which you can look at on any web page by pressing Ctrl+u (at least, in Chrome).

from lxml import html
import requests, time, csv

with open('C:/junk/geonet.csv', 'w') as csvfile: # create and/or open a CSV file
  csvWriter = csv.writer(csvfile, delimiter=" ", quoting=csv.QUOTE_MINIMAL) # writer
  dateList = []
  baseUrl = 'https://geonet.esri.com/content' # store the URL prefix

  for i in range(10): # loop through the first 10 'Content' pages
    page = requests.get(baseUrl + '?start=' + str(i*20)) # navigate to page
    tree = html.fromstring(page.text) # retrieve the HTML
    linkList = tree.iterlinks() # find all the links on the page
    threads = []

    for link in linkList: # loop through the links
      if link[2].startswith('/thread/'): # find those starting with "thread"
        threads.append(link[2]) # add the link to the list

    threadBase = 'https://geonet.esri.com' # store the URL prefix
    for thread in threads: # loop through the threads listed on the 'Content' page
      page = requests.get(threadBase + thread) # navigate to the correct thread page
      tree = html.fromstring(page.text) # retrieve the HTML
      dates = tree.find_class('j-post-author') # retrieve the date
      dateList.append(dates[0].text_content().strip()) # write to list
      csvWriter.writerow(dates[0].text_content().strip()) # write to CSV
      time.sleep(5) # wait 5s to give server a chance to handle someone else's requests

Anyhow, the graph at the start of this post shows pretty much what I expected: people on the East Coast get confused, then people on the West Coast get confused, then everyone goes home.

ArcGIS JS API: KML/Buffer

I’ve been tinkering around with the ArcGIS JS API. This example pulls a KMZ file from the Prince George Open Data Catalogue, displays it on a map, and requests/displays a 1km buffer around the features. The tricky part was installing/configuring a proxy page to allow for the convoluted buffer request.

Disclaimer: for a better example, see here for loading a KML and buffering the features.

Lego Area

legoCBC reported that Lego will spend 1 billion Danish kroner ($185.4 million CAD) to come up with a more sustainable material to replace acrylonitrile butadiene styrene. The figure that really caught my eye is that Lego produces more than 5,000,000 pieces per hour.

They do not get into what “a piece” means, so I’m going to assume that means the plain-old 2×4 block that we all know and love (keep in mind that the average piece is likely smaller than a 2×4, although I believe this is the most common piece). A 2×4 measures 15.8 x 31.8mm (from here). At 5,000,000 pieces per hour, that would be 2512.2 sq.m. Or, 60293 sq. m per day. Or, 22,006,872 sq. m per year. That’s 22 sq. km!

You can use this map to visualize it for yourself. Feel free to drag the circles to your city.

3-Letter Word Scatterplot

three_letters

Here‘s a D3-powered scatterplot showing the distribution of 3-letter words, at least, according to wordfind.com.

Instructions are simple: chose the starting letter in the dropdown list. Then, find individual words by the second letter (y-axis) and third letter (x-axis). Mouseover the point to convince yourself.

OpenWeatherMap IDW

idwOpenWeatherMap has a nice API that allows you to consume live weather data. I have made other simple examples here (v2.1) and here (v2.0 – doesn’t work), but you should check out the OWM API documentation and examples for more.

This example (v2.5) collects weather info from several stations across British Columbia and Alberta, interpolates temperature values between the stations using the Inverse Distance Weighting method, and displays both the stations (points) and interpolated surface (raster) as PaperJS objects.

 <!DOCTYPE html>
 <html>
 <head>
 <!---------------------
 Code by Darren Wiens
 dkwiens@gmail.com
 ----------------------->
 <style type="text/css">
 html { height: 100% }
 #map { height: 500px; width: 500px; }
 canvas { position: absolute; top: 10; left: 10; z-index: 1; pointer-events:none; }
 </style>
 <!-- Load the Paper.js library -->
 <script type="text/javascript" src="http://darrenwiens.net/scripts/paper.js"></script>
 <script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=false"></script>
 <script src="http://code.jquery.com/jquery-1.8.2.js"></script>
 <!-- Define inlined PaperScript associate it with myCanvas -->
 <script type="text/paperscript" canvas="myCanvas">
 var maxWidth = 500;
 var maxHeight = 500;
 var maxPoint = new Point(maxWidth,maxHeight);
var rect = new Path.Rectangle(0,0,maxWidth, maxHeight);
 rect.strokeColor = 'black';
 rect.strokeWidth = 3;
var map;
 var sampleCirs = new Group();
 var texts = new Group();
 var interpRaster = new Raster('myImg');
 var minTemp = 1000;
 var maxTemp = -1000;
function onFrame(event)
 {
}
function pointToLatLng(point)
 {
 var proj = map.getProjection();
 var bounds = map.getBounds();
 var ne = bounds.getNorthEast();
 var sw = bounds.getSouthWest();
 var neWorldXY = proj.fromLatLngToPoint(ne);
 var swWorldXY = proj.fromLatLngToPoint(sw);
 var curPixelX = point.x / Math.pow(2,map.getZoom());
 var curPixelY = point.y / Math.pow(2,map.getZoom());
 var curWorldX = curPixelX + swWorldXY.x;
 var curWorldY = curPixelY + neWorldXY.y;
 var curWorldPoint = new google.maps.Point(curWorldX,curWorldY);
 var curLatLng = proj.fromPointToLatLng(curWorldPoint);
 return curLatLng;
 }
function latLngToPoint(latLng)
 {
 var proj = map.getProjection();
 var calWorldPoint = proj.fromLatLngToPoint(latLng);
 var calPixelPointx = calWorldPoint.x * Math.pow(2,map.getZoom());
 var calPixelPointy = calWorldPoint.y * Math.pow(2,map.getZoom());
 var bounds = map.getBounds();
 var ne = bounds.getNorthEast();
 var sw = bounds.getSouthWest();
 var neWorldPoint = proj.fromLatLngToPoint(ne);
 var swWorldPoint = proj.fromLatLngToPoint(sw);
 var ePixelPoint = neWorldPoint.x * Math.pow(2,map.getZoom());
 var nPixelPoint = neWorldPoint.y * Math.pow(2,map.getZoom());
 var wPixelPoint = swWorldPoint.x * Math.pow(2,map.getZoom());
 var sPixelPoint = swWorldPoint.y * Math.pow(2,map.getZoom());
 var screenPixelX = calPixelPointx - wPixelPoint;
 var screenPixelY = calPixelPointy - nPixelPoint;
 var point = new Point(screenPixelX, screenPixelY);
 return point;
 }
function drawPoints(s)
 {
 childCount = 0;
 for (var station=0;station<s.length;station++) {
 if (s[station].last.hasOwnProperty('main')) {
 if (s[station].last.main.hasOwnProperty('temp')) {
 var lat = s[station].station.coord.lat;
 var lng = s[station].station.coord.lon;
 var newPoint = new Point(latLngToPoint(new google.maps.LatLng(lat, lng)));
 sampleCirs.addChild(new Path.Circle(newPoint,2));
 sampleCirs.children[childCount].value = s[station].last.main.temp - 273.15;
 sampleCirs.children[childCount].latlng = pointToLatLng(sampleCirs.children[childCount].position);
 if (sampleCirs.children[childCount].value < minTemp) {
 minTemp = sampleCirs.children[childCount].value;
 }
 if (sampleCirs.children[childCount].value > maxTemp) {
 maxTemp = sampleCirs.children[childCount].value;
 }
 sampleCirs.strokeColor = 'black';
 sampleCirs.fillColor = 'black';
 var text = new PointText(newPoint + new Point(3,-3));
 text.content = Math.round(sampleCirs.children[childCount].value);
 text.latlng = pointToLatLng(text.position);
 texts.addChild(text);
 childCount++;
 }
 }
 }
 drawRaster();
 }
function drawRaster()
 {
 interpRaster.size = new Size(100,100);
 interpRaster.fitBounds(view.bounds);
 for (var i=0;i<100;i++)
 {
 for (var j=0;j<100;j++)
 {
 var wj = 0;
 var wis = [];
 for (var k=0;k<sampleCirs.children.length;k++)
 {
 var dx = sampleCirs.children[k].position.x - (i * 5);
 var dy = sampleCirs.children[k].position.y - (j * 5);
 var dk = Math.sqrt(Math.pow(dx,2) + Math.pow(dy,2));
 var p = 3;
 var wj_inst = 1/(Math.pow(dk,p))
 wj += wj_inst;
 wis.push(wj_inst*sampleCirs.children[k].value);
 }
 var u = 0;
 for (var l=0;l<wis.length;l++)
 {
 u += wis[l]/wj;
 }
 var scale = (u-minTemp)/(maxTemp-minTemp);
 interpRaster.setPixel(i,j,new RgbColor(scale,0,1-scale,1));
 }
 }
 interpRaster.opacity = 0.7;
 interpRaster.latlng = map.getCenter();
 interpRaster.TLlatlng = pointToLatLng(interpRaster.bounds.topLeft);
 interpRaster.BRlatlng = pointToLatLng(interpRaster.bounds.bottomRight);
 interpRaster.moveBelow(sampleCirs);
 console.log("MAX: " + maxTemp);
 console.log("MIN: " + minTemp);
 }
function getData(s){
 drawPoints(s);
 }
$(document).ready(function() {
 var style = [
 { "elementType": "geometry.fill",
 "stylers": [ { "visibility": "off" } ]
 },{ "featureType":
 "road",
 "elementType": "geometry.fill",
 "stylers": [ { "visibility": "on" },
 { "color": "#000000" } ]
 },{ "featureType": "water",
 "elementType": "geometry.fill",
 "stylers": [ { "color": "#000000" } ]
 } ]
var myLatlng = new google.maps.LatLng(53.917, -122.75);
 var myOptions = {
 zoom: 5,
 center: myLatlng,
 mapTypeId: google.maps.MapTypeId.ROADMAP,
 disableDefaultUI: true
 }
 map = new google.maps.Map(document.getElementById("map"), myOptions);
 map.setOptions({styles: style});
google.maps.event.addListener(map, 'projection_changed', function() {
 $.getJSON('http://api.openweathermap.org/data/2.5/station/find?lat=53.917&lon=-122.75&cnt=30', getData); // API v.2.5
 });
google.maps.event.addListener(map, 'center_changed', function() {
 for (var i=0;i<sampleCirs.children.length;i++)
 {
 sampleCirs.children[i].position = latLngToPoint(sampleCirs.children[i].latlng);
 sampleCirs.children[i].latlng = pointToLatLng(sampleCirs.children[i].position);
 texts.children[i].position = latLngToPoint(texts.children[i].latlng);
 texts.children[i].latlng = pointToLatLng(texts.children[i].position);
 }
 interpRaster.position = latLngToPoint(interpRaster.latlng);
 interpRaster.latlng = pointToLatLng(interpRaster.position);
 });
google.maps.event.addListener(map, 'zoom_changed', function() {
 for (var i=0;i<sampleCirs.children.length;i++)
 {
 sampleCirs.children[i].position = latLngToPoint(sampleCirs.children[i].latlng);
 sampleCirs.children[i].latlng = pointToLatLng(sampleCirs.children[i].position);
 texts.children[i].position = sampleCirs.children[i].position + new Point(3,-3);
 texts.children[i].latlng = pointToLatLng(texts.children[i].position);
 }
 var tl = latLngToPoint(interpRaster.TLlatlng);
 var br = latLngToPoint(interpRaster.BRlatlng);
 var bounds = new Rectangle(tl,br);
interpRaster.fitBounds(bounds);
 interpRaster.TLlatlng = pointToLatLng(interpRaster.bounds.topLeft);
 interpRaster.BRlatlng = pointToLatLng(interpRaster.bounds.bottomRight);
 });
 });
</script>
 </head>
 <body>
 <canvas id="myCanvas" width="500" height="500"></canvas>
 <div id="map" width="500" height="500"></div>
 <img id="myImg"></img>
 </body>
 </html>