Intermediate Python Workshop – What to cover?

Hello GIS Studio followers!  Wordpress tells me there are 200 of you?  Seriously?  Well let’s put that power to work.

Lets Talk

Lets Talk

I am creating a new half day workshop designed to cover intermediate python topics – as they relate to GIS tasks.

Here’s an outline of the topics so far

  • Setting up a code template with built in tools to get you up and coding faster
  • A peek inside a more powerful IDE editor and why you would want to use it.
  • Using GIT – what is it (revision control software) and how to set yours up
  • A look behind the scenes at task automation with Python – what powers a local government every evening for GIS cleanup
  • Advanced ODBC / arcpy data access module connection strings
  • How to use ST_Geometry in your queries  to run spatial queries without ArcMap
  • Zipping files in python
  • Logging to a text file or SQL database

What is missing?  What would you all like to get out of an intermediate class?

It will be lecture format half day – unless we generate a long list and then maybe we will shoot for an all day class!

Supercharge your GIS reporting with SQL Reporting, ST_Geometry and ReportLab for python

I had the great pleasure of attending the Washington GIS Conference – an annual event put on by WA URISA. It was a busy three days being a workshop presenter, a paper presenter, on the conference committee and president of the organization.

We had a really surprising turn out at our paper presentation. I thought we were presenting on a somewhat obscure topic and would have been happy with an audience of 10 people. Turns out about 35 or 40 people showed up and stayed for the whole thing! I know, I can’t believe it either. And people are specifically mentioning this in the feedback survey. I think there is some untapped need here.

Here’s a copy of the presentation in PDF format. Let us know if you have any questions.

http://gisstudio.files.wordpress.com/2013/05/reporting-using-gis-data-in-sql-server-final.pdf *

*note that some links to live reports will not work as they are on an internal server.

Python Yield – Make the firehose a little trickle of data

rice-yield-may-decline-by-10pc-1352410048-8056Yield!

Wow – I saw this three line script at gis.stackexchange.com that will list all data in a File Geodatabase and was impressed! Three lines! A thing of beauty! When I looked closer, I saw a yield tool that I wasn’t familiar with, so I set about learning what the heck it is.

Here is a useful explanation: http://stackoverflow.com/questions/231767/the-python-yield-keyword-explained of what is going on.

 

 

def listFcsInGDB():
    ''' set your arcpy.env.workspace to a gdb before calling '''
    for fds in arcpy.ListDatasets('','feature') + ['']:
        for fc in arcpy.ListFeatureClasses('','',fds):
            yield os.path.join(arcpy.env.workspace, fds, fc)

Turns out, it’s like a list, but just feeds your script one item at a time. You can go ahead and generate giant lists and not worry about overloading what ever you are doing. It’s great for this kind of setting where you don’t know how much data is going to be listed.

I haven’t personally used this yet, but wow, I hope I get too because I’ll feel like I moved up a notch on the ladder of Python. Let me know if you have success!

Using python to check data since that last time it was checked…

follow-upIn my last post on time, I mentioned that I wanted to be checking in on whether features in GIS data had been updated or not. If it had, I wanted to copy the data that had been modified and record it in an archive. Originally I thought I would just check in every 5 minutes, or 30 minutes, or whatever the client wanted. Then I started thinking about all the things that could go wrong. If it was every minute, and the process of archiving the data took longer than one minute, the script wouldn’t run again until the first process had finished and I could potentially miss the changes that happened during that slack time. Or what if it failed entirely and then I lost track of when the data had last been checked? Or… I’m sure my mind wandered off into all the things that could go wrong.

So I thought a better way to handle this was to record the date and time the the data had been checked in a little text file and then read it back the next time the script ran. The file acts as kind of safety net in case things don’t go as planned.

Now the script can be run as often or as little as needed and all the data should be caught.

Here’s how I did it…

Continue reading

What time is it? Converting time from UTC to what you want.

What Time Is itTick tock…. time is a real bummer now that Esri implemented UTC time being recorded in the time tracker settings. Now the create date and last edit date are recorded in a time zone that most of us don’t work in. ( I understand why, I just want to grumble a little bit.) So… that opened up a new opportunity to learn about time shuffling in python.

What I wanted to do was write a script that was going to be run every so often. Maybe every 5 minutes, maybe every 30. It needed to check for changed data in an SDE feature class and archive whatever changes had been made.

Originally I wanted the script to figure out what time it was five minutes ago, but in UTC time so that I can query the last edit and last modified date out of a feature class stored in SDE. I did a lot of reading about time formatting. The most straightforward explanation I could find was here: http://www.doughellmann.com/PyMOTW/datetime

The best thing I have to tell you is that there is a built in UTC converter in Python! Hooray!

It works like this -

Continue reading

An easy way to get the line number of your python code

Sometimes you just want to know where things are in the script when you are debugging.  Or where things came from.  You can pass that information or report it while debugging using this snippet.  Easy, simple.

import inspect

def lineno():
    """Returns the current line number in our program."""
    return inspect.currentframe().f_back.f_lineno

print "this is line", %lineno()
print  " the line number, %s, is in the middle of this sentence"%lineno()
i = type(lineno())
print i

results in :

this is line number 8
the line number, 10, is in the middle of this sentence
<type 'int'>

Note that the line number is type integer.

Thanks to the Danny Yoo and the Active State recipes for this easy tip.

Calc the lat/long of a point in python using new ArcPy tools

Calculating the latitude and longitude of a point that is not in a geographic coordinate system was a tricky thing for me to figure out one day.

My data was not in the “World Projection” (as ArcMap calls it) so when I finally figured out how to access the x,y coordinates of the point, they were not in Lat/Long.  So basically I reproject the file, extract the x,y coordinates, then reproject back to State Plane which is what I need.  Not too elegant, but it gets the job done.

Here’s how I accomplished it:

Continue reading