Make an FTP tool – or any tool – the easy way.

Perhaps this is something that is inherently understood by “coders”, but as a newbie code writer, I had to figure this out on my own.  It’s much more efficient to make your own tools and reference them over and over than it is to write the same code over and over in each little script you make.

I didn’t come to realize that I had things I wanted to do over and over until I had a library of scripts and started to see patterns.  Some of the ‘do-overs’ are things like FTP all the files in a folder to our FTP site, or email myself a note once a script has completed (or even if it has failed), or log events into a table or file as the script progresses.

This sort of thing was intimidating as I was reading python tutorials and the samples use things like double underscores and references to “self” in a def statement.  I still have no idea what those do (I”ll let you know if I figure it out!) but honestly I haven’t needed it yet.  It doesn’t have to be that complicated.  Continue reading

Zipping files in python

Every so often I run an automated process that copies all of our clients data to file geodatabases and then zips them up to post to the web. Previously I used PKZip as a command line zipper, calling it from my python script in a batch file. I decided to explore the possibility of zipping from within python. Turns out there is a module called zipfile that does just what I need. A fellow blogger got me started on the right path with a very helpful blog posting (here it is).

# Description: 
# run this after creating the gis data for the web script
#---------------------------------------------------------------------------
import sys, string, os, zipfile
#if available add compression
try:
    import zlib
    compressType = zipfile.ZIP_DEFLATED
except:
    compressType = zipfile.ZIP_STORED
#Variables...
shpPath = "N:/yourpath/datafiles"
#***********************************************************************
# Read the list of shapefiles and zip'em
# shapefiles have a prefix that determines what zip file the go in
# reads the files alphabetically, or this script wouldn't work
# if the zip file already exists, it will overwrite
#***********************************************************************
x = 0
currentZip = ''
try:
  itemList = os.listdir(shpPath)
  for i in itemList:
    fullpath = os.path.join(shpPath, i)
    shpName = i.split("_",1)  # the data naming convention for the data includes an underscore.  This reads the prefix and name into a list
    thisZip = shpName[0] # reads the prefix
    shp = shpName[1] #reads the name of the file
    zipname = 'ClientName_' + thisZip +'_shps.zip'  #add a new prefix (ClientName_) and groups all the like prefixes into one zip.  So... all the transportation data is in one zip file.
    if i <> zipname:
      if currentZip == thisZip:
        #add to zip archive
        zf = zipfile.ZipFile(shpPath + '/zips/'+ zipname, mode='a')
        try:
            print 'adding', i
            zf.write(fullpath, arcname='ClientName_'+ i, compress_type=compressType)   #this writes the zip file
            x+=1
            #arcname sets the name and strips the folder location from the saved path internal to the zip file
        finally:
            zf.close()
      else:
        #make a new archive and set the new currentZip variable
        currentZip = thisZip
        print '\ncreating new archive'
        zf = zipfile.ZipFile(shpPath + '/zips/'+ zipname, mode='w')
        try:
          print 'adding', i
          zf.write(fullpath, arcname='ClientName_'+i, compress_type=compressType)
          x+=1
          zf.close()
        except:
          print "Unexpected error:", sys.exc_info()[1]
    else:
      print 'cannot zip a zip file'
      pass
except:
  print "Unexpected error:", sys.exc_info()[1]