Tuesday, December 18, 2012

Using env.addOutputsToMap

The environmental variable addOutputsToMap prevents geoprocessing task results from being displayed in the TOC.  This environmental property by default is set to true, so every tool's result will display in the table of contents in ArcMap.

When developing python add-ins, you should utilize this tool to prevent sub-processes results from being added to the map.  A good example of this, is if your python add-in creates a table if it doesn't exist for logging purposes.  Your end user does not want to see this table, or know it exists, but if addOutputsToMap is set to true, the value will display in the table of contents.  Changing it to false would prevent the data from showing.

from arcpy import env
env.addOutputsToMap = false
#... perform GP task...
env.addOutputsToMap = true


Monday, December 17, 2012

Finding Distance Along A Line

The arcpy polyline geometry object has a very helpful method called positionAlongLine (value, {use_percentage}).  This function finds either a distance (in the feature's coordinate system) or the percentage from the line's starting location.  It then returns a PointGeometry object that can be used for further analysis, or that can be saved out as a feature class using the CopyFeatures tool.  If you use the optional use_percentages, false means it's a distance, true means it is a percentage with values between 1-100.  If a value is great than the length or over 100 % then the last point is used.  If values are less than 0 or less than 0% then the first point is used as well.

Assuming you have a line feature class, just create a search cursor and begin accessing your locations.  

fc = r"c:\temp\lines.shp" 
with da.SearchCursor(fc, ["SHAPE@"] as cursor:
    for row in cursor:
       print row[0].positionAlongLine(10, True)    
       print row[0].positionAlongLine(10, False)

Wednesday, December 12, 2012

Finding and Changing Max Value in Numpy Array

Today, I answered a forum question about finding and changing a numpy array.  You can use the max() and the where() in the numpy to search for values in your array and change them using loops.

You can download a very simple example here.

Here is the simple code:

import numpy
myarray = numpy.array([[  5.4,   7.5,   2.2,   8.5,   8.6,   7.5],
                       [  7.7,   3.5,   1.4,   9.6,   8.5,   5.5],
                       [  5.2,   6.1,   8.6,   6.7,   4.3 ,  6.8],
                       [  9.6,   4.5,   2.7,   3.6,   6.7,   4.5],
                       [  1.2,   2.3,   7.2,   6.3,   2.2,   2.0 ],
                       [  1.3,   2.0,  -99.0,    9.6,  -99.0,    1.2]
print myarray
maxValue = myarray.max()

itemindex = numpy.where(myarray >= maxValue)

for i in xrange(0, len(itemindex[0])):
    array1 = itemindex[0]
    array2 = itemindex[1]
    myarray[array1[i]][array2[i]] = 0

print myarray

Here we have the numpy array and find the array's maximum value.  Then using the where(), the location in the array are given back.  From there, the code then changes the values in the 2-D array from the maximum value back to zero.


Thursday, December 6, 2012

ArcPy Sample: Zipping a Shapefile

I've decided to post full code samples on github.  Here is my first sample to zip shapefiles.


Tuesday, December 4, 2012

AGS 10.1 Quirk with Windows Domain Security

Today I configured a test machine to use windows domain accounts at the GIS tier for security.  When I went to connect to the site using catalog, I noticed that if I included the '\' (backslash) in the user name it could not connect.  After removing the backslash, the connection worked.

This means that if your user is 'domainname\name' you would use just the 'name' in the AGS connection string.

Monday, December 3, 2012

Truncate Table, A New Way To Erase Rows

Truncate table is the 10.1 way to erase ALL the rows in a table or feature class.  It will ignore any sql query, so use this tool carefully.

Some Notes:

  • Supported data types are simple, meaning no terrains, topologies, etc..
  • Data must be unversioned to execute
  • After rows are removed, the data is unrecoverable, so make sure you know what you are doing before you go and use this.

This is a good tool for workflows where you need to clear out rows on some regular basis.

Usage Example:
import arcpy
from arcpy import env
wrksp = r"c:\temp\data.gdb"
env.workspace = wrksp
fcs = arcpy.ListFeatureClasses()
for fc in fcs:
del fcs


Thursday, November 29, 2012

Walk, new arcpy.da function as 10.1 SP1

Walk is a useful function that walks directories.  Though this is an easy function to create using recursion, it is now including as a core function at 10.1 SP1 for arcpy.

The help describes the function as such:
Generate data names in a catalog tree by walking the tree top-down or bottom-up. Each directory/workspace in the tree yields a tuple of three: dirpath, dirnames, and filenames. Python's os module includes an os.walk function that can be used to walk through a directory tree and find data. os.walk is file based and does not recognize database contents such as geodatabase feature classes, tables, or rasters. arcpy.da.Walk can be used to catalog data.

import arcpy
import os
workspace = "c:/data"
feature_classes = []
for dirpath, dirnames, filenames in arcpy.da.Walk(workspace,
    for filename in filenames:
        feature_classes.append(os.path.join(dirpath, filename))


Wednesday, November 21, 2012

Shift Those Polygons at 10.1

The arcpy team at Esri has a great blog post that shows how to shift polygons using the SHAPE@XY tag.
Here is the sample:

with arcpy.da.UpdateCursor(in_features, ['SHAPE@XY']) as cursor:
        for row in cursor:
            cursor.updateRow([[row[0][0] + (x_shift or 0),
                               row[0][1] + (y_shift or 0)]])

Here we have the shape column (column 0) and to access the X and Y property, you just edit the first and second objects in the X,Y pair.  The x_shift and y_shift are float values that will move the code in one direction or another.  

It's very simple sample, but it shows how you can easily move polygons.  In the past you had to adjust each vertex, but now this is done internally thanks to the new arcpy.da module at 10.1.

Tuesday, November 20, 2012

New Geometry Tags for Cursors

At 10.1 SP1, there are 3 new tags that can be used to represent geometry objects.  Those tags are:
The well-known binary (WKB) representation for OGC geometry. It provides a portable representation of a geometry value as a contiguous stream of bytes.  The well-known text (WKT) representation for OGC geometry. It provides a portable representation of a geometry value as a text string.  The esri JSON string representing the geometry.

Here is a simple usage example from the python window:

from arcpy import da
with da.SearchCursor("Counties", ["SHAPE@JSON", "SHAPE@WKT", "SHAPE@WKB"]) as rows:
     for r in rows:
         print r[0]
         print r[1]
         print r[2]
You'll see all three types print out.  The print out doesn't really mean anything except to show the various string formats that a coder can now export the geometry object into. It will also make exporting geometries to something like a TAB or Excel table a bit more practical.  I would stay away from csv, since many of these formats use the ',' (comma) to separate the points.


Monday, November 19, 2012

Using the Oracle Instant Client with ArcGIS Server

At 10.1 you can use the Oracle instant clients to connect to your oracle SDE instances.  It's easier than it seems.

  1. Download the appropriated instant client (for ArcGIS Server use x64 version)
  2. Create a folder on the local hard drive (write down the path)
  3. Unzip the instant client in the folder in step 2
  4. Assuming windows, edit the 'PATH' variable and add Step #2's path to the variable
  5. Restart ArcGIS service
  6. Create your direct connect connection:
  7. Register the connection with server
Now you can use the sde connection for any map document, geoprocessing service, etc... 

This workflow will be valid for desktop instant client setup, except it should be noted that the 64-bit version should be before the 32-bit reference path.


Thursday, November 15, 2012

Using the Constant PI in C++

In visual c++, you need to do the following in order to use the defined PI constant.

#include < math.h > 

double EuclidFunctions::AreaOfCircle(double radius){
return (pow(radius,2)) * M_PI;

where 'M_PI' is the constant value.


Tuesday, November 6, 2012

Go Vote!

If you live in the United States, please go vote.   You have no excuse.

Thank you

Friday, October 26, 2012

Creating a Dynamic Fishnet Tool - Python Addin

At 10.1, you can create python add-ins.  These add-ins are very much like add-ins in .NET, but they do not support GUI development.  This means that you can either just call a function after interacting with a map, or you can open an existing tool.

On the resources.arcgis.com page, one example given is the use of fishnet, but the grid sizes are static at 10x10.  This is a basic example,  much like the 'Hello World' examples in most programming languages, but what if you want user inputs into the fishnet tool?  Can it be done?  The answer is yes, but you need more add-in components to do it.

To use the code posted, you must first install the python add-in component for ArcGIS 10.1.  You can find that here.

After that is installed, create a new project, and add a toolbar, a tool, and two combo boxes.
When naming the ID and Class Name, do not use the same name.  This will lead to problem if you have to communicate between controls.
I called my controls the following:

  • toolbar - ID: analysisTB
  • combobox 1: ID: cboRows1, Class Name: cboRows
  • combobox 2: ID cboColumns1, Class Name: cboColumns
  • tool: ID: fishnetTool101 Class Name fishnetTool
The code behind will create multiple classes and events.  Erase the events that are not needed, so the control is not listening to events that are not needed. 

First code the combo box controls.  Here we will set the control values and store the values to a global variable rows and columns:

rows = 0
columns = 0
class cboClassColumns(object):
    """Implementation for fishnet_addin.cboColumns (ComboBox)"""
    def __init__(self):
        self.items = [i for i in xrange(1, 100)]
        self.editable = True
        self.enabled = True
        self.dropdownWidth = 'WWW'
        self.width = 'WWW'
    def onSelChange(self, selection):
        global columns
        columns = int(selection)
    def onEditChange(self, text):
        global columns
            columns = int(text)
            columns = 0
            pythonaddins.MessageBox("Please enter a valid integer", "Value Error")

class cboClassRow(object):
    """Implementation for fishnet_addin.cboRows (ComboBox)"""
    def __init__(self):
        self.items = [i for i in xrange(1, 100)]
        self.editable = True
        self.enabled = True
        self.dropdownWidth = 'WWW'
        self.width = 'WWW'
    def onSelChange(self, selection):
        global rows
        rows = int(selection)
    def onEditChange(self, text):
        global rows
            rows = int(text)
            rows = 0
            pythonaddins.MessageBox("Please enter a valid integer", "Value Error")

OnSelChange event sets the row or column value, and since I made my control editable, I validate the user's entry on the onEditChange event.

Next is the fishnet function.  This code goes behind button control in this example:

class fishnetClass(object):
    """Implementation for fishnet_addin.fishnetClassID (Tool)"""
    def __init__(self):
        self.enabled = True
        self.shape = "Rectangle"
        self.cursor = 3
    def onRectangle(self, rectangle_geometry):
        """ creates a temp fishnet polygon """
        extent = rectangle_geometry
        fishnet = None
        global rows
        global columns
        fshFC = r"in_memory\fishnet"
        if arcpy.Exists(fshFC):
            fishnet = arcpy.CreateFishnet_management(fshFC,
                            '%f %f' %(extent.XMin, extent.YMin),
                            '%f %f' %(extent.XMin, extent.YMax),
                            0, 0, int(rows), int(columns),
                            '%f %f' %(extent.XMax, extent.YMax),'NO_LABELS',
                            '%f %f %f %f' %(extent.XMin, extent.YMin, extent.XMax, extent.YMax),
            del extent
            pythonaddins.MessageBox("Cannot Run the Fishnet Tool", "Error Message")
        return fishnet

Run the makefile.py, and install the add in. Now you can create a dynamic fishnet tool using python.

Thursday, October 25, 2012

Using pythonaddins Messagebox

New 10.1, there is a module created for python add-ins.  It's called the pythonaddins module, and you can create a messagebox.  The messagebox object allows you to display messages to an end user.

The arcpy help can be found here.

So the messagebox can be defined as such:

MessageBox(message, title, {mb_type})

To use you just pass in a message and title as text.  The mb_type is the type of messagebox type value.  It supports the following types:

mb_type value
OK Only
Cancel/Try Again/Continue

Complete Example:
>>> import pythonaddins
>>> result = pythonaddins.MessageBox("Press Cancel", "TITLE", 1)
>>> print result

Simple Right!

Wednesday, October 24, 2012

Visual Studios 2010 - Configuring a 64-Bit C++ Project

I can say it better that Microsoft's help, so here is the link.

At 10.1 there will be support for 64-Bit Python Geoprocessing for the desktop software.  It's probably going to be a good idea to know how to do this because you'll now have to account for both 32 and 64 bit builds.


Tuesday, October 23, 2012

Easy C++ Project for Python Use

For this tutorial, I will be using Visual Studios 2010 (C++) and Python 2.7 with ctypes.

Let's assume your boss wants a program that performs addition, there are many ways to write this code, but for the sake of this this tutorial, you want to use the python and C++.
  • Open Visual Studios
  • Follow the steps in creating a project here.
  • Give the project a name called additioncpp
  • Create a header file called 'additioncpp.h' and enter in the following code:
      int sum(int, int);
  •  In the dllmain.cpp file, comment out everything but the '#include "stdafx.h" line
  • Now we are ready to write the sum() we defined in the additioncpp.h header file.  Since python like 'C', and not 'C++' we need to use exten "C" around the functions
      #include "stdafx.h"
      #define DLLEXPORT extern "C" __declspec(dllexport)

      DLLEXPORT int sum(int a, int b) {
          return a + b;

  • Compile the code and copy the .dll to a new folder
  • Create a new .py file and type in the following:
  • print 'start' 
    from ctypes import cdll
    mydll = cdll.LoadLibrary(r'PATHTODLL\additioncpp.dll')
    print mydll
    print mydll.sum
    print mydll.sum(1,2)
    print 'end'
  • You should see an output like this:
>>> <_funcptr 0x10ca47b0="0x10ca47b0" at="at" object="object">
>>> 3
>>> end


Monday, October 22, 2012

Exporting C++ Functions for Use in C

If you have functions in a dll written in C++ that you want to access from C, you should declare these functions with C linkage instead of C++ linkage. Unless otherwise specified, the C++ compiler uses C++ type-safe naming (also known as name decoration) and C++ calling conventions, which can be difficult to call from C.
To specify C linkage, specify extern "C" for your function declarations. For example:

   extern "C" __declspec( dllexport ) int MyFunc(long parm1);

 More help can be found here.

Friday, October 19, 2012

What is Extern "C" in C++?

If you are going to use ctypes in python, then you need to export your dll with a C-wrapper.  

You can find a great definition of extent "C" here.  In case you do not want to click on the link, here is what is said by 'Faisal Vali'.

extern "C" makes a function-name in C++ have 'C' linkage so that client C code can use your function using a 'C' compatible header file that contains just the declaration of your function. Your function definition is contained in a binary format that the client 'C' linker will then link to using the 'C' name.  Since C++ has overloading of function names and C does not, the C++ compiler cannot just use the function name as a unique id to link to, so it mangles the name by adding information about the arguments. A C compiler does not need to mangle the name since you can not overload function names in C. When you state that a function has extern "C" linkage in C++, the C++ compiler does not add argument/parameter type information to the name used for linkage.
I couldn't have said it better myself, so I won't

Thursday, October 18, 2012

Setting up C++ Project for Python/ArcGIS Use

This article will discuss setting up a Visual Studios 2010 project to create a DLL using C++ which can be called by python using ctypes.

 1. Open Visual Studios
 2. Create a new C++ Win32 Project
 3. Give it a name, example: simplepythoncpp
 4. Press next 
 5. Select ‘Dll’ for the type of project
 6. Here you’ll have a set of created header files (.h) and Source Files (.cpp)
 7. To use ArcObjects, you need to reference the com libraries.  To do this, you need to tell visual studios where these libraries are.  To set the location, do the follow:
      a. In the Solution explorer, right click and select properties
      b. Select ‘VC++ Directories’
      c. Click on ‘Library Directories’ 
      d. Select ‘Edit’
      e. Navigate to the ArcGIS 10.1 install directory and select the ‘com’ folder
 8. Staying in the properties, we need to enable ‘C++/CLI’ 
      a. Click on ‘Command Lin’
      b. Under ‘Additional Options’ type ‘/clr’
 9. Next we need to define some ‘Code Generation’ options
      a. Click on ‘Code Generation’
      b. Under ‘Basic Runtime Checks’ select ‘Default’
 10. Press ‘Apply’ button then ‘OK’
Let’s test our project changes.
 1. Open the ‘stdafx.h’ header file
 2. Leave the default #include statements, and type in the following:

#import "esriSystem.olb" raw_interfaces_only, raw_native_types, no_namespace, named_guids, exclude("OLE_COLOR", "OLE_HANDLE", "VARTYPE")

 3. Compile project
No errors should appear, so your project is setup correct.  
Where do you do from here? Well, all the libraries you need in this header file, and reference them in your .cpp file.  

Wednesday, October 17, 2012

c++ and python = woot!

Everything old is new, and so begin my venture back into the C++.  My focus now is using C++ libraries with ArcObjects, slightly different then what I did way back in the before times.  

Some tips:

  • Create a DLL project and ensure that the ArcGIS/com folder is mapped to the project properties -> 'VC++ Directories' -> 'Include Directories'
  • get comfortable with resources.arcgis.com and the Visual Studios help system
  • Experiment and push the limits
Enjoy the ride

Tuesday, October 9, 2012

ArcGIS 10.1 and matplotlib

Graphing at 10.1 is now easier because ArcGIS comes with matplotlib.  To me, this is an exciting enhancement that broadens what python developers can do when it comes to visualization of data in ArcGIS.

A simple example of this is the basic X/Y plot but instead of using random number, assume we have a table and the X/Y (distance/elevation) is in your table.


import arcpy
import os
import matplotlib.pyplot as plt
x = []
y = []

elevPNG = env.scratchFolder + os.sep + "elev.png"
fig = plt.figure()
table = r"C:\temp\scratch.gdb\data"
fields = ["FIRST_DIST", "FIRST_Z"]
with arcpy.da.SearchCursor(table, fields) as rows:
    for row in rows:
plt.plot(x,y, color='r')
plt.xlabel('Distance From Start Location')
plt.title('Landscape Profile')
fig.savefig(elevPNG, dpi=fig.dpi)

So what has happened, is that the x and y information is stored in a 1:1 fashion in list objects.  The figure is created and lists populated using a simple search cursor.  The line graph is shown using the plot() and labels are added to make the graph easier to read.

The results is something like this:
In this application, a user can quickly assess the terrain without examining each elevation point, and can determine if a better route is needed.

Friday, October 5, 2012

Archiving Made Easy at Python 2.7

At python 2.7, there is a great shutil tool called make_archive().  It can make either a .zip or .tar file, and eliminates the need to create a walking function to compress files.

The function supports the following compression types by default:
  • zip
  • bztar
  • gztar
  • tar
But if you have another format, you can register it using the register_archive_format().

compressFile = shutil.make_archive(r"c:\temp\mycompressfile", "zip", r"c:\temp\data")
Notice that the first parameter doesn't have the file extension.  The function will return the path to the newly created compressed file.

To uncompress the file, you will have to do that manually using zipfile or tarfile modules.


Thursday, September 27, 2012

Using Map() or Apply_async() For Multiprocessing

With the release of this new python article on the ArcGIS blogs.  I find it increasingly harder to find concrete/simple explanations of important functions that help make life easier for processing data in ArcGIS.
Though the crux of this article is to give guidance to when or when not to use multiprocessing with ArcGIS, it does use a variation on a method of multiprocessing different than I prefer.  So I decided to investigate the differences between map() and apply_async() on the Pool object.
Pool.apply_async() is also like Python's built-in apply(), except that the call returns immediately instead of waiting for the result. An ApplyResult object is returned. You call its get() method to retrieve the result of the function call. The get() method blocks until the function is completed. Apply_async() does not preserve the order of the results meaning that the codes return results will not be a 1:1 correlation between when it was called and when it finished.  In addition, apply_async() allows coders to call multiple functions instead of a single function.
Pool.map() is like the Python's built-in map() function.  Pool.map() applies the same function to many arguments. The results are returned in an order corresponding to the order of the arguments.

When it comes right down to it, you basically need to find the right tool for the right job.

Here are some resources to help you along.


Monday, September 24, 2012

Delete Rows Using the UpdateCursor - arcpy.da

The arcpy.da.UpdateCursor() provides the method to alter and update field values.  This quick post will show you how to delete a row.

All you need to do is specify the fields, here I take the default of * and the feature class.  You can reduce the number of rows by passing a query as well.

Code Sample Run From Python Window:

with da.UpdateCursor(r'C:\temp\scratch.gdb\floodsub', "*") as urows:
...     for row in urows:
...         urows.deleteRow()
...         break

This little sample just erases the first row in the feature class.  The 'with' statement automatically closes the cursor object, so it does not have to be manually deleted.  This ensures the locks are removed on success or on failure.


Friday, September 21, 2012

Uninstalling Time!

The funny things you see when upgrading versions of software:

Installers always were lousy estimators of time, but this one was way off!!!! I guess I need two cups of coffee for this one.


Friday, September 7, 2012

Article on 3rd Party Modules

The Esri python team wrote another good article on 3rd party modules, which is a good read for anyone who is new to python.

The article can be found here.


Tuesday, August 28, 2012

Tutorial - How to add built-in commands to your custom toolbar or menu

Esri posted a great blog post on building an add-in using python.  You can find the information here.


Friday, July 20, 2012

The Tile Package

What is it?
New at 10.1, tile packages are compressed files that contain a map document's cached data.  The tile package or .tpk is ideal for disconnected use and for sharing information to ArcGIS Online.
Why Should I Care?
It makes sharing cache easy, and you can create custom caches on the fly.  From a geoprocessing/application view of things, this means you can get your data out to mobile users without needing an air card.  It also produces an easy way to share cache from server to server or AGOL.

Other reasons include:
  • Improved rendering performance
  • Improved quality
  • Follows industry standards (AGOL, Google, Bing, etc..)

You can create a package using geoprocessing as follows:
import os
import arcpy
from arcpy import env
# Set environment settings
env.overwriteOutput = True
env.workspace = "C:/Tilepackages/"
# Loop through the workspace, find all the mxds and create a tile package using the same name as the mxd
for mxd in arcpy.ListFiles("*.mxd"):
    print "Packaging " + mxd
    arcpy.CreateMapTilePackage_management(mxd, "ONLINE",
                                       os.path.splitext(mxd)[0] + '.tpk',   "PNG8", "10")
Pretty easy to create.  It should be noted that your ArcMap Document's extent determines the area to be processed.  You can embed web service layers as well as local data.  The cached is fused together and it means you cannot query the base data as well.

Have packaging

Tuesday, July 17, 2012

Create a Map Service Layer (10.1)

At 10.1, you can use JSON to create maps and add map services to that layer.  This means you can create map service layer indirectly.  There is no direct way to create a map service layer in 10.1, but by using the ConvertWebMapToMapDocument() you can.  Information about the function can be found here.

ConvertWebMapToMapDocument() described in the help is as follows:

Converts a web map (in JSON format) that you intend to print or export to a map document. The map document can be further modified before finally being printed or exported.

You notice that the help does not directly mention the creation of the map service layer, but when you drill down into the supported JSON syntax, you can see how you can use this method to generate your map service layer.

A JSON map consists of the following:

   "mapOptions": {}, 
   "operationalLayers": [], 
   "baseMap": [], 
   "exportOptions": {}, 
   "layoutOptions": {} 

Using the JSON library and dictionaries, you can easily match this structure. As shown below:
import json
from arcpy import mapping
mapService = {} 
operationalLayer = [] 
layer = {} 
layer["id"] = str(webmapID) 
layer["url"] = str(url) 
layer["title"] = str(title) 
layer["opacity"] = str(opacity) 
layer["visibility"] = str(visibility) 
layer["minScale"] = str(minScale) 
layer["maxScale"]= str(maxScale) 
mapService["operationalLayers"] = operationalLayer 
jsonDump = json.dumps(mapService) 
result = mapping.ConvertWebMapToMapDocument(jsonDump)

Now we have a MapDocument object that has our map service layer in it.  Using the other rmapping module functions, loop through the TOC and save the layer to disk.

mxd = result.mapDocument
layers = mapping.ListLayers(mxd)
for layer in layers:

That's it, now we have create a map service layer saved to disk.  If you have to deal with security, you can add a token to your operational layer.


Monday, July 16, 2012

Constructing Polygons at 10.1

Polygon geometries are closes paths forms from a series of X,Y coordinates.  A polygon geometry object can be constructed from using an Array object containing the arcpy.Point(X,Y) values.

In this example, I will construct the extent of a feature class as a polygon to clip another datasource:
import arcpy
dsParcel = r"c:\temp\sample.gdb\Parcels"
dsLandUse = r"c:\temp\sample.gdb\LULC"

pnt = arcpy.Point()
array = arcpy.Array()
extentParcel = arcpy.Describe(dsParcel).extent
coords = [[extentParcel.XMin,extentParcel.YMin],[extentParcel.XMax,extentParcel.YMin],
outLU = r"c:\temp\sample.gdb\clipLULC"
#  Create Spatial Reference
sr = arcpy.SpatialReference()
sr.factoryCode = 4326
# Load the Location
for coord in coords:
    pnt.X = coord[0]
    pnt.Y = coord[1]
polygon = arcpy.Polygon(array,sr)


Notice how we didn't have the close the polygon!  The object does it for us.


Friday, July 13, 2012

Create a List of Geometries (10.1)

At 10.1, you can use geometries directly in geoprocessing functions like Clip or reading geometry properties.  Here is a tip to copy all the geometries into memory and use it as a list:

import arcpy
ds = r"c:\temp\sample.gdb\Parcels"
g = arcpy.Geometry()
geomList = arcpy.CopyFeatures_management(ds,g)
# Get the Area (Sq Ft)
area = 0.0
for geom in geomList:
    area += geom.area
print "Total Parcel Area: {0}".format(area)

The CopyFeatures() copies the results as a geometry object to a list and the area property is a standard Geometry object property.


Thursday, July 12, 2012

ArcGIS Python Window Color Scheme

If you are like me, I prefer dark backgrounds and white or bright text verse the standard white on black.

The python window color scheme can be changed by doing the following:

  1. Right click on the python window 
  2. Select Format
  3. Select 'Set Black Theme'
Now your background is black

Wednesday, July 11, 2012

ArcGIS 10 SP5 Released

Service Pack 5 has been released.

Get it while it's hot from here.

List of issues fixed here.


Tuesday, July 10, 2012

Insert Cursor (arcpy.da) at 10.1

The insert cursor, just like the old insert cursor object, is used to write a new row to an existing table.  When using the InserCursor object you must tell the cursor object what type of geometry you will be using.  So if your feature class is a point feature class, use "SHAPE@XY" but if it's a polygon or polyline, use "SHAPE@".  It should be noted that "SHAPE@" is valid for point features as well.

The InsertCursor has one function called insertRow(), which is a list object with field values in it.

Table Examples:

from arcpy import da
with da.InsertCursor(fc, ["name"]) as cursor:
   cursor.insertRow(["BOB SMITH"])

Feature Class Example (Point):

from arcpy import da
with da.InsertCursor(fc, ["name", "SHAPE@"]) as cursor:
   cursor.insertRow(["A Point",(-77, 34)])

Feature Class Example (Line):

from arcpy import da

array = arcpy.Array([arcpy.Point(-77, 34),
                     arcpy.Point(-77.1, 34.1),
                     arcpy.Point(-77.2, 34.2)])
polyline = arcpy.Polyline(array)

with da.InsertCursor(fc, ["name", "SHAPE@"]) as cursor:
   cursor.insertRow(["A Line", polyline])
Feature Class Example (Polygon):

from arcpy import da

array = arcpy.Array([arcpy.Point(-77, 34),
                     arcpy.Point(-77, 34.1),
                     arcpy.Point(-77.2, 34.1),
                     arcpy.Point(-77, 34)])
pg = arcpy.Polygon(array)

with da.InsertCursor(fc, ["name", "SHAPE@"]) as cursor:
   cursor.insertRow(["Polygon", pg])


Monday, July 9, 2012

Setup WING IDE for ArcPy 10.1

I've changed python IDE to WING.  It's a very nice IDE, but it took sometime getting use to from pydev's eclipse add on.

Here is the basic setup of WING IDE with PyLint:

1. Download easy_install: http://pypi.python.org/pypi/setuptools/#windows
2. Install easy_install
3. Go to the install location of Python 2.7.x and run 'easy_install pylint'
4. Install WING IDE
5. Start WING IDE
6. Select Tools -> PyLint
7. Right click on the PyLint window and select 'configure'
8. Edit the 'command =' to be 'command = C:\Python27\ArcGIS10.1\Scripts\pylint.bat'
9. Run pylint on a .py file

* if you get an error, try adding the full path to the python.exe in the pylint.bat file.


Tuesday, June 26, 2012

Introduction to the Update Cursor (arcpy.da) at 10.1

The UpdateCursor creates a read-write access to records returned from a feature class or table.
Returns an iterator of lists. The order of values in the list matches the order of fields specified by the field_names argument.  UpdateCursor objects can be iterated using a for loop and it supports with statements. Using a with statement will ensure that the database locks are removed.

You can use both an insert and update cursor at the same time if an edit session is opened.


from arcpy import da
fc = r"c:\temp\samples.gdb\nests"
fields = ('BirdPop', 'Rank')

with da.UpdateCursor(fc, fields) as cursor:
    for row in cursor:
        if (row[0] >= 0 and row[0] <= 10):
            row[1] = 1
        elif (row[0] > 10 and row[0] <= 20):
            row[1] = 2
        elif (row[0] > 20 and row[0]<= 30):
            row[1] = 3
        elif (row[0] > 20):
            row[1] = 4

In this simple example, you see that you can use conditional statements to change values within the 'with' statement.


Friday, June 22, 2012

ArcGIS API for Silveright 3.0 Released

I like Silverlight as a programming language, and now the ArcGIS API for Silveright is released.  You can check out the announcement here.


Wednesday, June 20, 2012

Working with BLOB data at 10.1 (arcpy.da)

Reading and writing BLOB data has always been an issue previous to 10.1, but now it's not a problem.  A reader pointed out that fact the other day, so I thought why not point this out to everyone.
From the help (source: resourcebeta.arcgis.com):
A BLOB is data stored as a long sequence of binary numbers. ArcGIS stores annotation and dimensions as BLOBs, and items such as images, multimedia, or bits of code can be stored in this type of field. You can use a cursor to load or view the contents of a BLOB field.
In Python, BLOB fields can accept strings, bytearray, and memoryviews. When reading BLOB fields, a memoryview object is returned.
So what does this mean to you.  We'll it's time to store those documents, pictures from vacation, and videos with geo tags in a spatial database.

Writing Blobs :
from arcpy import da

myFile = open(r"c:\temp\image.jpg",'rb').read()

with da.InsertCursor(r"c:\temp\demo.gdb\table",['blobFieldname','fileName']) as cursor:

Reading Blobs:

from arcpy import da
import os
with da.SearchCursor(r"c:\temp\demo.gdb\table",['blobFieldname','fileName']) as cursor:
   for row in cursor:
      binaryRep = row[0]
      fileName = row[1]
      # save to disk
      open(r"c:\saveFolder" + os.sep + fileName, 'wb').write(binaryRep.tobytes())
      del row
      del binaryRep
      del fileName

Writing and reading Blob fields involves using the python built in open().  This function can open up files in various modes, but ensure that the method used are the binary modes 'rb' (read binary) and 'wb' (write binary), or else this process will not work.  You can essentially open up any file as a Blob, so stick whatever you want in there, but realize that this can balloon the size of your database up greatly.  It does however open up a whole new host of ways of transporting file based data.  Off the top of my head, I can image how this could improve data transport for local parallel processing methods of spatial and non-spatial data.


Tuesday, June 19, 2012

SearchCursor (arcpy.da) 10.1 Continued

The new cursor objects contain a very helpful property called fields.  This property returns a tuple of field names defined in the field_names argument.  This means that if you pass "*" for all fields you can then get the associated field to the associated value.

Remember, cursor objects will ignore BLOB and Raster fields.

import arcpy
from arcpy import da
fc = r"c:\temp\demo_polygon.shp"
with da.SearchCursor(fc,"*") as rows:
   for field in rows.fields:
      print field

So in this example, the cursor will print all the fields being accessed.  It will also show the type of Shape field is being returned.  If all fields is used ("*"), then the SHAPE@XY property returned by default.


Monday, June 18, 2012

10.1 SearchCursor (arcpy.da)

The Search Cursor object creates read-only access to the records in a feature class or table.  It will return an iterator of tuples.  The order of values matches the order of fields mentioned by the field_names argument.

When comparing the old search cursor to the new search cursor, functionally they are basically the same, except the data access module is better, stronger, and faster. It's like the Six Million Dollar Man of upgrades in ArcPy.

Looking at the inputs:
SearchCursor (in_table, field_names, {where_clause}, {spatial_reference}, {explode_to_points}, {sql_clause})

The two biggest differences between arcpy.SearchCursor and arcpy.da.SearchCursor, is the field_names is required, and the optional parameter explode_to_points.

Paramter field_names can be "*", which means all fields, but it does exclude BLOB and raster fields.  This is required because you should only be returning what you need.  The other interesting aspect of the is the way you can obtain geometry and other field information using the @.

  • SHAPE@XY —A tuple of the feature's centroid x,y coordinates
  • SHAPE@TRUECENTROID —A tuple of the feature's true centroid x,y coordinates
  • SHAPE@X —A double of the feature's x-coordinate
  • SHAPE@Y —A double of the feature's y-coordinate
  • SHAPE@Z —A double of the feature's z-coordinate
  • SHAPE@M —A double of the feature's m-value
  • SHAPE@ —A geometry object for the feature
  • SHAPE@AREA —A double of the feature's area
  • SHAPE@LENGTH —A double of the feature's length
  • OID@ —The value of the ObjectID field

The parameter explode_to_points is a boolean value.  If you specify the value to true, it will explode multi-geometry values into there own rows.  This means if you have a 5 multipoint geometry object, it will have 5 rows instead of 1.

Example: Basic Use
import arcpy
fc = r"c:\temp\wells.shp"
fields = ["WELL_ID","TYPE","SHAPE@XY"]
with arcpy.da.SearchCursor(fc,fields) as cursor:
   for row in cursor:
      print "{0}, {1}, {2}".format(row[0],row[1],row[2])

So here we have just a simple read-only example where the user will pass in a feature class and the fields needed to query.  It will print the information out on the screen.  Notice the usage of the with statement.  This is great because it automatically cleans up the cursor object.  This means no more del statements and help reduce the number of cursor locks on data.


Friday, June 15, 2012

Introducing the Data Access Module (arcpy.da) at 10.1

The data access module also known as arcpy.da is used to manipulate and search data.  It allows control of an edit session, faster and better cursor support, function to convert to and from numpy arrays, and supports versioning, replicas, domains, and subtypes work flows.

The data access module has the following functions:

  • ExtendTable - Joins the content of a numpy array based on a common attribute field.  This is like a table join but more flexible.
  • FeatureClassToNumPyArray - this convert a feature class to a numpy structured array
  • ListDomains - Lists the attribute domains in a geodatabase
  • ListReplicas - Lists replicas in a work space
  • ListSubtypes - returns a dictionary of subtypes for a table or feature class
  • ListVersions - list the versions in the work space
  • NumPyToFeatureClass - converts a numpy array to a point feature class
  • NumPyToTable - converts a numpy array to a table
  • TableToNumPyArray - converts a table to numpy array

The data access classes:

  • Domain - the domain object properties that describes an attribute domain
  • Editor - editor class allows use of edit sessions and operations to manage database transactions.  This means rollbacks, and better error handling!
  • InsertCursor - adds new rows to a table or feature class
  • Replica - contains properties of a replica
  • SearchCursor - read only access to a table or feature class
  • UpdateCursor - allows read/write access to a table or feature class.  This is what you use to update or delete a row.
  • Version - contains properties that describe a version.

I plan on detailing these items in future posts.

Thursday, June 14, 2012

Random Number Generator (arcpy.env)

This is a tool that creates a random number based on a seed number and an algorithm selection.
Inputs are as follows:
  1. Seed—The seed is an integer value and is used to initiate the random number generator. The default value is 0.
  2. Random Generator Type—The random generator algorithm.

    • ACM599—ACM collected algorithm 599. This is the default.
    • MERSENNE_TWISTER—Mersenne Twister mt19937.
    • STANDARD_C—Standard C Rand

So to create a random number generation environmental variable you can do it two ways:

Option 1:

from arcpy import env
env.randomGenerator = "4 MERSENNE_TWISTER"

Option 2:

from arcpy import env
env.randomGenerator = arcpy.CreateRandomValueGenerator(4, "MERSENNE_TWISTER")

To produce a random number, the easiest way is to use the arcpy.CalculateValue() and get the results from that tool.

import arcpy
arcpy.env.randomGenerator = "4 MERSENNE_TWISTER"
result = arcpy.CalculateValue_management("arcgis.rand('normal 0.0 10.0')")
randomValue = float(result.getOutput(0))
print randomValue

The code should print a random number.

It should be noted that this environmental variable will effect any tool that uses random values.


Wednesday, June 13, 2012

env.ScratchGDB (ArcPy 10.1)

A new environmental parameter as 10.1 is env.scratchGDB.  This environmental parameter is guarantees to have a temporary scratch file geodatabase for your writing pleasure.  This means no more checking to see if a user created a file geodatabase for doing some work.
I personally think this is great.  It takes a common task and puts it into core.  Bravo ArcPy Development Team!
From this help:
Its primary purpose is for use by scripts and models as geoprocessing services, with the added focus of only pointing to a known geodatabase. When ArcGIS for Server executes a geoprocessing task, the scratch GDB is always available to write output to. This environment is also useful in authoring scripts and models for use on the desktop. Writing output to the scratch GDB will make your tool portable, because this location will always be available or created at execution time.

Some notes on the env.scratchGDB:

  1. Scratch GDB will exist if called and user will have write access to the file geodatabase
  2. Scratch GDB is read-only.  You can only change the location of the scratch GDB by changing the env.scratchWorkspace 
  3. If the env.scratchWorkspace is not set, then it defaults to the user's temporary directory

>>> import arcpy

>>> from arcpy import env
>>> env.scratchWorkspace = r"c:\temp"
>>> print env.scratchGDB


Tuesday, June 12, 2012

Introducing env.scratchFolder (ArcGIS 10.1)

The scratch folder is a new environmental variable that is used for the development of both published and non-published scripts.  It will point to a known folder that exists, which is very nice because at 10.0, env.scratchWorkspace or env.workspace could always be set to None or even worse set to a file geodatabase.
From the online help:
"The Scratch Folder is the location of a folder you can use to write file-based data, such as shapefiles, text files, and layer files. It is a read-only environment that is managed by ArcGIS. 
The Scratch Folder environment complements the scratch workspace environment. Its primary purpose is for use by scripts and models as geoprocessing services, with the added focus of pointing only to a known folder. When ArcGIS for Server executes a geoprocessing task, the Scratch Folder is always available to write output to. This environment is also useful in authoring scripts and models for use on the desktop. Writing output to the Scratch Folder will make your tool portable, because this location will always be available or created at execution time."
 Some important notes about env.scratchFolder:

  1. It will always exists
  2. It's read only property
  3. It will always be folder even if your scratch workspace is set to a file geodatabase
  4. If there is no Scratch Workspace set in your map document, or you're working in an environment where the Scratch Workspace has not been explicitly set, the Scratch Folder defaults to the current user's temporary files directory. This directory is typically at C:\Users\\AppData\Local\Temp on Windows 7 or C:\Documents and Settings\\Localsystem\Temp on Windows XP.

>>> import arcpy
>>> from arcpy import env
>>> env.scratchWorkspace = r"c:\temp\myfgdb.gdb"
>>> print env.scratchFolder


Monday, June 11, 2012

Using SQL to Get n number of Rows

When you query spatial data in ArcMap/Catalog/Python, etc.. you cannot limit the row search by default and you can only specify the where part of the sql statement: "SELECT * from < DATASET > WHERE" and then you can specify TYPE = '1'.

Let's assume you want to just get back n number of rows.  If you are using SDE, you could use ArcSDESQLExecute's methods to perform your sql, but you could also use a sub-query. 

Take your default statement that ArcMap allows you to perform on a layer: 

     "Select * from wgs.temp where "

Since we want to just get back n number of rows, use the OBJECTID field as a reference and the 'in' function:

    "Select * from wgs.temp where OBJECT in (select OBJECTID  from wgs.temp where rownum <= 5)"

This will return 5 rows in the data set.  If you want 10, 15, of n number of rows, replace the 5 with whatever value you want.  You can apply this to update and search cursors on SDE data only.


Thursday, June 7, 2012

Monday, June 4, 2012

Creating a Single Directory or Multiple Directories

When automating processes, you should be very aware of system directory structures and the directory structure that your processes need to function properly.

The OS module has a collection of directory tools to make or remove directories as well as checking if they exist or not.

The standard make a directory function is the os.mkdir()

>>> import os
>>> directories = r"c:\temp3\level1"
>>> os.mkdir(directories)

The other folder creation tool I find very helpful is the os.makedirs(), which will generate a several folders at once.

>>> import os
>>> directories = r"c:\temp3\level1\level2\level3\level4"
>>> os.makedirs(directories)


Thursday, May 31, 2012

Using Tkniter in ArcToolbox

If you download this script: http://resources.arcgis.com/gallery/file/geoprocessing/details?entryID=C2594BA8-1422-2418-A048-74E477699334 from the resource center and want to call it from a toolbox, you can do the following:

  1. Open ArcCatalog
  2. Create a new toolbox (something.tbx)
  3. Create a new script
  4. Point to the file from the download
  5. Uncheck both "Show command window when executing" and "Run python script in process"
  6. Press finish
Now the GUI form will show.  

(Note, I didn't write that script, but I'm using it as an example of showing a GUI form from ArcToolbox.  If you have issues with the code, please contact the developer of the script.)


Friday, May 25, 2012

Python Add-Ins for 10.1

I'm really exciting for the increasing integration of python inside of ArcGIS Desktop.  If you have done any desktop development at 10 then you will be familar with the Add-In wizard.  Check out the information here.

The types allowed: (source: webhelp.esri.com)

Add-In Type
A button is the simplest form of functionality that can be used to execute
some business logic when it is clicked. A button assumes that an action or a
series of actions will be executed. Buttons may exist on toolbars and menus.
A tool is very similar to a button. However, a tool requires user interaction
with the desktop application’s display first, and then based on that interaction,
executes some business logic. The Zoom In toolZoom In in ArcMap is a good
example—it requires that you click or drag a rectangle over a map before the
 display is redrawn, showing the map contents in greater detail for the specified
 area. Tools may exist on toolbars and tool palettes.
Combo Boxes
A combo box provides a set of choices, visually seen as a drop-down box, from
which a selection can be made and acted upon. A combo box can also be
enabled to allow an end user to add a value in the combo box. The Scale combo
box Scale Drop-down List in ArcMap is a good example of a combo box. When data with a known
coordinate system is added to ArcMap, the scale combo box is enabled giving
the user a set of predefined scales from which to choose. It also allows a
user to type a new scale not in the list of options, and the display is updated
based on the value added.
A menu is a container for buttons or other menus. Menus can be exposed by
adding them to an existing toolbar (see Creating an Add-In toolbar for more
information on creating your own toolbar) or an existing menu.
A toolbar is a container for buttons, tools, combo boxes, tool palettes, and menus.
Toolbars can be floating or docked in desktop applications, just like any system
toolbar within the application. Toolbars can also be activated so they are visible
when a desktop application is started.
Tool Palettes
A tool palette is a container for tools. Tool palettes can only be exposed by adding
 them to an existing toolbar (see Creating an Add-In toolbar for more information
on creating your own toolbar).
Application Extensions
Application extensions can be used in different ways to provide supplemental
functionality to ArcGIS Desktop:
1. Application extensions are used to coordinate activities between other components
—such as buttons and tools—within a containing add-in. For example, an extension
 may not activate a set of tools on a toolbar unless a specific set of layers exist in the map.
2. Application extensions are usually responsible for listening and responding to various
events exposed by the host application. For example, anytime a layer is added
 or removed, an event is triggered, and the extension responds by
automatically saving the map document.

Thursday, May 24, 2012

Feature Class as JSON (ArcPy 10.1)

I just learned a really easy way to display a feature class as JSON by using featureset object and the Describe().

fc = r"c:\temp\myData.shp"
featurSet = arcpy.FeatureSet()
desc = arcpy.Describe(featureSet)
print desc.pjson # so we can read it
#### desc.json also works. ####
del desc
del fc
del featureSet


  "displayFieldName" : "",
  "geometryType" : "esriGeometryPolygon",
  "spatialReference" : {
    "wkid" : 4326,
    "latestWkid" : 4326
  "fields" : [
      "name" : "FID",
      "type" : "esriFieldTypeOID",
      "alias" : "FID"
      "name" : "Id",
      "type" : "esriFieldTypeInteger",
      "alias" : "Id"
  "features" : [
      "attributes" : {
        "FID" : 0,
        "Id" : 0
      "geometry" : {
        "rings" : [


Wednesday, May 23, 2012

Python for ArcGIS 10.1

At 10.1, python developers will be using Python 2.7.  This is the last 2.x release of python before moving to the Python 3.x releases.  You can find out all about python 2.7 here.


Tuesday, May 22, 2012

Compressing File Geodatabases in a Folder

In a previous post, I showed how to uncompress a set of file geodatabases in a folder.  To compress a set of file geodatabases, using the CompressFileGeodatabase_management().  This script will use the List functions to list all the file geodatabases and then compress them.  When completed successfully it returns 'true' else it returns false.

import arcpy
from arcpy import env
if __name__ == '__main__':
        workspace = arcpy.GetParameterAsText(0)
        env.workspace = workspace
        fgdbs = arcpy.ListWorkspaces("*","FileGDB")
        for fgdb in fgdbs:
        env.workspace = None
        env.workspace = None


Monday, May 21, 2012

Checking Hard Drive Space Using Python

Have you ever needed to check how much space is left on a drive, but didn't know how, well now you will.  Python ctypes and os libraries allow a developer to check the HD space pretty easily.  If you have a windows system, you can use ctypes to leverage the kernal32.dll functions, and if you have the other operating systems, python has a built in function called os.statvfs().

import ctypes
import platform
import os

class HardDriveSpaceException(Exception):
    def __init__(self, value):
        self.parameter = value
    def __str__(self):
        return repr(self.parameter)

def get_free_space(folder, format="MB"):
        Return folder/drive free space 
    fConstants = {"GB": 1073741824,
                  "MB": 1048576,
                  "KB": 1024,
                  "B": 1
    if platform.system() == 'Windows':
        free_bytes = ctypes.c_ulonglong(0)
        ctypes.windll.kernel32.GetDiskFreeSpaceExW(ctypes.c_wchar_p(folder), None, None, ctypes.pointer(free_bytes))
        return (int(free_bytes.value/fConstants[format.upper()]), format)
        return (int(os.statvfs(folder).f_bfree*os.statvfs(folder).f_bsize/fConstants[format.upper()]), format)
if __name__ == "__main__":
        byteformat = "mb"
        size =  get_free_space(r"c:", byteformat)
        print size
        if size[0] < 10000000000:
            raise HardDriveSpaceException("Hard drive space limit reached, there is only %s %s space left." % (size[0],size[1]))
    except HardDriveSpaceException as e:
        print e

In this example, there is a function that checks your HD space, and a custom error. The custom error is raised if the numeric value is over a specific value.

I've tested the HD space function on Win7, Win7 x64, and Window Server 2008 R2 x64.  It works.  I don't have any OS's that aren't windows based, so let me know if it works on iOS or the other that some people use.

Some maybe wondering why would I want to do something like this.  Well, if you are creating content automatically, such as PDFs or any geospatial content, you want to ensure that you have the space to save it.