A Real Life Geoprocessing Service In Action (ArcGIS Server 10)

As I have mentioned before, I'm an oldschool GIS guy.  Because of this, I have a hard time embracing "secular" web mapping toolkits.  Its kind of like the old days.  Yes, you can use GIS to just make maps but it is really about the data and analytic capabilities.  So translated into the geoweb, yes there are all kinds of toolkits to make webmaps but very few that expose the richness of the data and provide real analytic capabilities.  That is why a recent project I was lucky enough to be apart of was so exciting.  Web enabling rich GIS capabilities via a simple user interface and user experience using ArcGIS Server's Geoprocessing service.

The application is really simple, basically allowing users to dynamically calculate the amount of in-place oil shale amounts.  We have kind of been calling it the oil shale calculator.  However, the science and methodology for assessing the oil shale resources in the first place was far more complex and something that I can't take credit for but would like to acknowledge those who did (Johnson, R.C., Brownfield, M.E., and Mercier, T.J., (U.S. Geological Survey Oil Shale Assessment Team)).  Have a look at it if you are interested.

But for the sake of this post, the result of this work (at least the part that was used for this application) was basically a raster with gallons of in-place oil shale resources per cell and a model implementing the zonal stats function for doing the dynamic calculations (nice work Tracey!).  Here is the geoprocessing model.

Pretty simple really.  Basically, two input parameters.  A user defined polygon and the raster containing oil shale values.  The output being a sum based on the zonal statistics function.  This was actually modeled after one of the AGS samples.  When implementing this model, we ran into what I consider a bug.  First, the literature suggests that you can store the output table in memory(ArcGIS 10 only).  However, we found this was not the case.  We were forced to use a physical file system location, basically the scratch disk (bad:  output table=in_memory\results, good: output table=%scratchworkspace%\oilshale.dbf).

The application itself was pretty simple as well.  Have a look.  One gotcha we ran into here was that basically the REST API endpoint for this service actually expects 2 input parameters, not just one as is suggested by its Services Directory page.  The missing parameter that is actually required is the zone field parameter.  We called ours id. You can view the page source of the app to see how we manually included this parameter but a preview is below:

Participating with OneGeology: Experience, Lessons Learned and Other Tidbits

I recently had the opportunity to "stand up" a WMS representing a 1:5,000,000 scale geologic map of North America for integration into a platform called OneGeology.




OneGeology, whos mission statement is to:

"Make web-accessible the best available geological map data worldwide at a scale of about 1: 1 million, as a geological survey contribution to the International Year of Planet Earth." (see: http://onegeology.org/what_is/mission.html)

 aims to:

• create dynamic digital geological map data for the world.
• make existing geological map data accessible in whatever digital format is available in each country. The target scale is 1:1 million but the project will be pragmatic and accept a range of scales and the best available data.
• transfer know-how to those who need it, adopting an approach that recognises that different nations have differing abilities to participate.
• the initiative is truly multilateral and multinational and will be carried out under the umbrella of several global organisations.   (see:  http://onegeology.org/what_is/objective.html)

The process for contributing to this worthy effort was pretty interesting and I wanted to share this experience a bit.

To begin with, the map we were submitting was actually a replacement for an existing geologic map of North America which only covered the southern portion.  The previous WMS was served using Map Server while our new one was being powered with ArcGIS Server 9.3.1.  This was quite the test for AGS given the outstanding performance of Map Server with regard to WMS serving.  In an effort to boost performance without doing a bunch of map authoring and data processing, we performed a couple of interesting tricks that I outlined in some detail in a previous post.

Basically, participation with the OneGeology platform can occur at several tiers, each basically correlated to functionality.  Our participation was of the first tier, that being simply contributing a WMS.  This experience was not trivial.  Even though WMS is technically a "standard," there is still enough wiggle in the spec that a specific implementation is needed to ensure interoperability.  The folks at OneGeology have done a outstanding job with this and have documented, in detail, requirements for how WMS services need to be configured for integration into their system.  Our specific instances are here:

http://certmapper.cr.usgs.gov/arcgis/rest/services/one_geology_wms/USGS_Geologic_Map_of_North_America/MapServer -- raster one

http://certmapper.cr.usgs.gov/arcgis/rest/services/one_geology_wms/USGS_Geologic_Map_of_North_America_GFI/MapServer -- vector one for getFeatureInfo requests only

And finally the portal.  It is a really nice application and serves as a great demonstration of "mashing up" distributed data services that originate throughout the globe.  It is really pretty astonishing, given the wide range of participating organizations.

OneGeology Portal Depicting Geologic Map of North America

Web 2.0 and the Geoweb Part 1: Web 2.0 Examples

As I have mentioned earlier, I am ramping up for the upcoming semester and am feverishly prepping. My course, "Introduction To the Geoweb" is part of the Master of Engineering/GIS offered at the University of Colorado at Denver.

In the coming weeks, I intend to share materials I will be presenting to my students to all of you in hopes of getting some constructive feedback from the experts--YOU! I won't be sharing everything, just some selected materials that I think can benefit from some "participatory lecture development" if there is such a thing. I will also be citing much of the work that many of you have contributed so your feedback is critical.

The collection of materials I intend to share is tentatively termed "Web2.0 and the Geoweb." My hope during this series of lectures is to expose students to well documented Web 2.0 patterns and examples as a foundation for further exploration of the Geoweb. Some have suggested that a detailed look at web architectures and http are warranted before this discussion. Rest assured, the students will be prepared for these more advanced concepts.

The primary source of my research related to Web 2.0 patterns and examples comes from Web 2.o Architectures by James Governor and others. It is a good piece, articulating web2.0 patterns in a formal way and has worked nicely for foundational discovery. I have also incorporated materials from several additional sources that have proven adequate.

The story I hope to weave reads something like this: the Geoweb (which I think I can fully embrace) has roots, much of which can be described using well document Web 2.0 patterns. These patterns are best described using real world examples which can then be articulated formally using a standard method of description. This framework then serves as the foundation for further discussions mapping these patterns and subsequent reference models and architectures to Geoweb concepts, some of which have been stewing for some time and others which have emerged recently.

With that said, part 1 of 4 (or maybe 5) slides follow. These are subject to change which goes without saying. Be aware that these are lecture slides so there is a fair amount of text. Students get a little "bent without bullets."

I will be posting my recorded lectures as well. Continue to Part 2: Web 2.0 Patterns.

Web 2.0 and the Geoweb Part 1: Web 2.0 Examples

View more presentations from Gregory L. Gunther.

ArcGIS Server Sample Flex Viewer: TwitterFeedWidget

I have been thinking about how Twitter could be used to enhance the capabilities/user experience of a web mapping system and came op with some decent stories. The first would be providing users with access to a feed dedicated to the application which contains information about it including updates, news, changes, downtimes, etc. The second would be users who wish to perform searches of Twitter content that may be related to the map and eventually, overlaying geolocated Tweets onto the map. These (and the fact that I just wanted to poke around the Twitter API) gave me enough of an excuse to start messing with the Twitter API. I ended up with a simple little widget that allows users to search for tweets within a specified geographic location, in this case, a circle. Nothing earth shattering but still maybe somewhat useful. Before giving you some of the nuts and bolts of this, I want to be sure and reference the materials I used:

• Flex Cookbook Entry: Display a Twitter Feed
  
• Flex Cookbook Entry: Accessing Resources CrossDomain
• Free Twitter Icons
• Custom Draw with Circle Tool
• Distance Calculations website
• Flex XML Syndication Tools
• Flex CoreLib Utility
  

Some Details on how it works.
It is really pretty simple. The user is able to draw a circle on the map and any tweets that originated from within that location based on the Twitter user's location property are returned as an ATOM Feed. The processes uses the Twitter API geocode parameter where a lat, long and radius of a circle are needed. The widget obtains the lat, long, and radius values from the circle tool. I added a coordinate transformation function to the Circle Tool component to convert the distance units (radius of the circle) to miles from decimal degrees. I used the Great Circle Distance Formula. The ATOM feed is parsed using the Flex XMLSyndication package and is validated using the Flex CoreLib Utility Package (see links above).

Proxy Server
As with any application making cross-domain asynchronous data requests, a proxy server is needed to "fool" the browser security settings. Flex has a nice framework for bypassing this. If the feed host contains a crossdomain.xml file that allows cross-domain access, you are good to go. However, Twitter locks this down (Twitter crossdomain.xml). I have included a proxy server implemented in Java within the package. It is NOT PRODUCTION quality but will get you started. There are other implementations that are easier out there. Have a look at the resources I sited for some other examples.

Known Issues and Limitations
Simply put, there are several limitations with this that make it, lets say, less than production quality. I hope to rectify these soon. These include:

• I haven't tested the feed results very well. The widget catches feeds that are empty which is good but the way I did this is kind of sloppy. I won't go into it here but if you have questions, I can elaborate.
• The coordinate transformation for the distance units (radius conversion from DD to miles) I am using is very rough but good enough for this use.
• It doesn't play very nicely with the map action and map navigation state of the Sample Flex Viewer (SFV). Given that the draw circle component wasn't designed for use with the SFV, it isn't integrated very well with the event driven model the SFV uses. Not to say it isn't a nice component because it is. Using it in the Sample Flex Viewer is just a bit of a stretch. There are alot of issues with this but from a use perspective, users have to keep clicking on the draw circle tool after each use and then state defaults back to the default map navigation (pan). Another problem is if the user trys to change the map action or map navigation state (by selecting another component that does so), during the middle of component use, bad things happen. One of the reasons for this is that I didn't want to make changes to the core by adding custom events, etc. However, I am hopeful that I can improve this without making changes to the core and will look into it.
• Improvement can be made to the tweet results listing (links to profile, user timeline, etc...).
  
• Alot of other testing needs to occur as well. Just running out of time right now.
  

Webmap of US Oil and Gas Production

The USGS Energy Resources Program recently upgraded the US Oil and Gas Production map service for use in their new web mapping platform called Envision which is based on the ArcGIS Server Sample Flex Viewer. The ERP is actively updating the system to deliver all legacy map services and has recently upgraded the US Oil and Gas Production map (By Laura R.N. Biewick USGS DDS 69-q)