Support Python GUI Support in ArcGIS

Please support my idea on having Python GUI support in ArcGIS.

http://ideas.arcgis.com/ideaView?id=087E00000004SmHIAU

I want to get this idea to over 1000 points before the beginning of 2014, and I need your help!

Thank you

GPX to Feature Class

A handy tool created at 10.1, but refined at 10.2 is the GPX to Feature Class tool, which allows users to convert GPX files from GPS units to feature classes.  Waypoints and Tracks can then be extracted from the converted GPX file, and rendered as desired.

The ArcGIS online help for the GPX to Feature Class can be found here.

import arcpy
import os
if __name__ == "__main__":
    input_gpx = r"C:\data.gpx"
    convert_fc = arcpy.env.scratchGDB + os.sep + "gpxfc"
    gpx_fc = arcpy.GPXtoFeatures_conversion(Input_GPX_File=input_gpx, 
                                            Output_Feature_class=convert_fc
                                           )[0]
    track_points_lyr = arcpy.MakeFeatureLayer_management(in_features=convert_fc, 
                                                        out_layer="TRKLYR", 
                                                        where_clause="Type = 'TRKPT'", 
                                                        )[0]
    way_point_lyr = arcpy.MakeFeatureLayer_management(in_features=convert_fc, 
                                                      out_layer="WPT", 
                                                        where_clause="Type = 'WPT'", 
                                                        )[0]    

    if int(arcpy.GetCount_management(track_points_lyr)[0]) > 0:
        trkpt = arcpy.CopyFeatures_management(way_point_lyr,
                                              env.scratchGDB + os.sep + "trkpt")[0]
        trkLine = arcpy.PointsToLine_management(trkpt,
                                                env.scratchGDB + os.sep + "lines",
                                                "Name")[0]
    if int(arcpy.GetCount_management(way_point_lyr)[0]) > 0:
        waypts = arcpy.CopyFeatures_management(way_point_lyr,
                                               env.scratchGDB + os.sep + "WAYPTS")[0]
        

Here the code converted the GPX file to a feature class, then further refined the results from just a set of points to tracks and waypoints as mentioned earlier in this post.

There is a difference between 10.1 and 10.2 that should be noted.  In 10.2, more of the GPX properties will be converted where as in 10.1, I have noticed that only a small set of the data is actually pulled in from the GPX files.  So if you need all the GPX properties, ArcGIS 10.2 is probably what you need.  You can always parse the XML in the GPX manually using the minidom library in python if this doesn't suit you needs.


Enjoy

Geometry Objects Make Life Easier

Sometimes you just want to work with a geometry, but you do not always want to go through all the steps of creating a cursor object.  In the 10.x framework, you can output almost any geoprocessing tool to a arcpy.Geometry object.

To gain access to all feature's geometries in a feature class, just do the following:

import arcpy
from arcpy import env
fc = r"c:\temp\data.shp"
geoms = arcpy.CopyFeatures_management(fc, arcpy.Geometry())
for g in geoms:
   print g.extent

This sample allows anyone to directly access the geometries of the input feature class without having to use the Cursor objects.

The same idea can be applied to other functions to the analysis function as well:

import arcpy
from arcpy import env
fc = r"c:\temp\data.shp"
geom = arcpy.Buffer_analysis(fc, arcpy.Geometry(), "100 Feet", "FULL", "ROUND")[0]
print geom.extent

Here the buffer tool outputs a single geometry to the geom object and the extent is displayed.

Where this becomes really powerful is when you need to perform geometry operations on your data, and want to put the results back into that row.

import arcpy
from arcpy import env
fc = r"c:\temp\data.shp"
with arcpy.da.UpdateCursor(fc, ["SHAPE@"]) as urows:
   for urow in urows:
      geom = arcpy.Buffer_analysis(urow[0], arcpy.Geometry(), "100 Feet", "FULL", "ROUND")[0]
      row[0] = geom
      urows.updateRow(urow)
      del urow
      del geom

Assuming that the input is a polygon, this snippet shows how geometries can be used as inputs and outputs thus allowing for easy insertion back into the original row.

Hope this helps!

Happy Halloween!

Happy Halloween.  Everyone Have Fun And Be Safe!

Table to Excel (10.2 Arcpy)

Table to excel is a great tool for converting your spatial data into an excel spreadsheet.  In the past, before 10.2, you would have to use a 3rd party module like Python Excel in order to convert your data to and from ArcGIS tables to excel spreadsheets.

It should be noted that this only support the MS Excel 5.0/95 format (.xls) file type.

The syntax and use is pretty simple for this tool, it has 4 inputs, the table path (string), out path (string), use field alias (Boolean and optional), and use domain and sub-type description (Boolean and optional).  Let's dive into an example:

import arcpy
import os
if __name__ == '__main__':
    table = r"c:\temp\scratch.gdb\StarbucksAddresses"
    out_excel = r"c:\temp\starbucks.xls"
    if os.path.isfile(out_excel):
        os.remove(out_excel)
    arcpy.TableToExcel_conversion(table, out_excel)

All we have done here is taken the simplest example and convert my list of Starbucks locations and converted to an excel spreadsheet so I can now use it in excel.

Enjoy

ArcGIS Tool help link - http://resources.arcgis.com/en/help/main/10.2/index.html#/Table_To_Excel/001200000054000000/

Excel to Table (10.2) Tool

New at ArcGIS 10.2 is the Excel to Table tool.. This tool is found under the conversion toolbox, and it a quick way to easily convert your Excel Workbooks (.xlsx files) and MS Excel 5.0/95 Workbook (.xls files) into geodatabase tables.

• Supports xlsx and xls file extensions
• First row is considered the field names, and can be renamed so a valid table is generated
• It assumes that each field contains uniform data types and the data is not mixed

ExcelToTable() takes the following inputs:

1. Input_Excel_File - MS Office Excel file path 
2. Output_table - export table name and path
3. Sheet (optional) - Name of the sheet to export.  If none is provide the 1st sheet will be exported

Sample:

import arcpy
from arcpy import env
if __name__ == '__main__':
    env.overwriteOutput = True
    xls_file = r"c:\temp\somedata.xls"  # supports xlsx and xls
    dest_gdb = env.scratchGDB
    arcpy.ExcelToTable_conversion(xls_file,
                        os.path.join(dest_gdb, "sheet"))

Pretty easy.  The biggest advantage I see with this tool is if you are in an environment where you cannot install 3rd party modules like xlrd.

Enjoy

ArcGIS 10.2 Help Documentation

With the release of 10.2, it is important to know where you can find help.  The product documents can be found here (http://pro.arcgis.com/documentation/).

Enjoy

Creating a SQlite database via 10.2 ArcPy

At ArcGIS 10.2, there is not support for SQLite database, according to Wikipedia is described as follows:

is a relational database management system contained in a small (~350 KB) C programming library. In contrast to other database management systems, SQLite is not a separate process that is accessed from the client application, but an integral part of it.

To create a SQLite database, use the CreateSQLiteDatabase().

import arcpy
# Set local variables
sqlite_database_path = 'C:/Data/Counties.sqlite'
# Execute CreateSQLiteDatabase
arcpy.gp.CreateSQLiteDatabase(sqlite_database_path, "ST_GEOMETRY")

It's not too hard.  All file paths must end in '.sqlite' extension.  The database supports two spatial types: "ST_GEOMETRY" or "SPATIALITE".  ST_GEOMETRY is Esri's storage type, whereas SPATIALITE is the SpatiaLite geometry type.  ST_GEOMETRY is the default geometry storage type.

This database can be fully used with the python sqlite3 module that comes as a default with any python installation.  This means you can create a database using ArcPy, then perform all updates through native SQL instead of update/insert cursors.

Some notes:

• ":memory:" stored databases are not supported, they must be written to disk
• SQLite version 3.16.2

Enjoy

Hurricane RSS Feeds

The National Hurricane Center has created two very helpful feeds for those who want to know the latest GIS hurricane data:

• Atlantic Feed: http://www.nhc.noaa.gov/gis-at.xml
• Pacific Feed: http://www.nhc.noaa.gov/gis-ep.xml

Each feed contains a wealth of information that can be parsed out:

• Graphical Tropical Weather Outlook [shp]
• 120h Wind Speed Probabilities (34kt, 50kt, 64kt) [shp] [kml]
• Advisory Summary Information [xml]
• Advisory Forecast Track [shp] [kml]
• Advisory Cone of Uncertainty [shp] [kml]
• Advisory Watches & Warnings [shp] [kml]
• Advisory Wind Field [shp]
• Preliminary Best Track [shp] [kml] & Wind Swath [shp]
• Storm Surge (Probabilities, Exceedance) [shp] [kml]

shp means shapefile (in zip files) and kml is keyhole markup language (google format). 

This is a great improvement over the old way of trying to automate pulling data from the National Hurricane Center's website.  Hopefully next year they will improve these feeds by offering JSON or GeoJSON formats like the USGS has for new real time Earthquake data.

Always Tip Well Because...

Always tip that pizza driver well, because you never know if they are a GIS major or not.  Besides being funny, this is a very interesting ArcGIS Online map.  It shows concentrations of pizza consumption in the college town of Shippenburg, PA.  It also allows people in that town to see how much your neighbors are tipping.  Better tip well, you do not want to be the cheap guy on the block.

http://www.arcgis.com/home/item.html?id=f86568bc807b49cd86e437194ccd2bca

I wonder if the map creator is the driver and if now he will get more tips!

Enjoy

Python Add-Ins and Tkinter

ArcGIS for Desktop does not support any python GUIs out of the box, but let's say we want to have a form pop-up anyway. As shown below

One way to do this is to create a wx python instance at start up, which is create before the desktop python loop is created.  You would then reference the wx loop instead of the ArcGIS python loop.. but it's complicated...  You can also see tons of forum posts like this, that describe how the in process causes python to crash with GUIs: http://gis.stackexchange.com/questions/36848/crashing-arcgis-10-1-add-ins-using-multiprocessing

Let's assume though you want to use Tkinter because it's core, it comes with the python install.  Tkinter is the out of the box GUI that comes with python, and if you want to learn more about it, you can check it out here.  

Since ArcGIS for Desktop runs python add-ins 'in process'.  We cannot use multiprocessing or subprocessing to launch another instance python and display the code.  This is essentially the thing hindering python GUI development in the 10.x framework.  To get around this issue, code must be executed out of process.  Luckily for python people, we can create toolboxes and reference those toolboxes through the ImportToolbox().  Toolboxes allow you to run code 'out of process', which means multiprocessing!  Pretty sweet.

Let's take a look at the code to generate the form:

import Tkinter
from Tkinter import *
import multiprocessing
import sys, os
import arcpy
def show_form():

    root =Tk()
    root.title('Button')
    Label(text='I am a button').pack(pady=15)

    #button are labels that react to mouse and keyboard events.
    #in this case the button is packed to be at the bottom of the root or
    #toplevel widget.

    Button( text='Button') .pack(side=BOTTOM)
    root.mainloop()
    
if __name__ == "__main__":
    pythonExe = os.path.join(sys.exec_prefix, 'python.exe')
    multiprocessing.set_executable(pythonExe)
    multiprocessing.freeze_support = True
    jobs = []
    pool = multiprocessing.Pool(1)
    jobs.append(pool.apply_async(show_form,[]))
    pool.close()
    pool.join()
    del pool
    del jobs
    arcpy.SetParameterAsText(0, True)   

Now that we have the code to create the form, create a new python toolbox. Add this script and make sure you UNCHECK 'Run python script in process'

Next create your python add-in toolbar and add a button.  Copy the python script and toolbox from above into your 'install' folder.  Begin editing the python add-in script as follows:

import arcpy
import pythonaddins
import os
class btnShowForm(object):
    """Implementation for tkinterTester_addin.button (Button)"""
    def __init__(self):
        self.enabled = True
        self.checked = False
    def onClick(self):
        tbx = arcpy.ImportToolbox(os.path.join(os.path.dirname(__file__), "scripts.tbx"))
        tbx.showform() # name of script in toolbox is showform

Finally all you have to do is create the .esriaddin file. Just double click on the makeaddin.py and install the addin.

When you run the add-in from ArcMap, you should see a pop-up appear.  Pretty cool!  The one down side is that ArcPy needs to reload, so it can cause a slight delay in showing the form.


Enjoy

Also, if you want Python GUI support in ArcGIS for Desktop, please vote up this (http://ideas.arcgis.com/ideaView?id=087E00000004SmHIAU).

10.2 Map Document is Backwards Compatible with 10.1

Straight from the online help:

ArcGIS 10.1 and 10.2 map, globe, and scene documents are directly compatible with each other. You can open an ArcGIS 10.2 document in ArcGIS 10.1 without any special steps.

I thought this was interesting to note.

See what else is new with 10.2 here: http://resources.arcgis.com/en/help/main/10.2/#/What_s_new_in_ArcGIS_10_2/016w0000005s000000/

Enjoy

Spatial Data for Mars

The geo-geek in me thinks this is cool, and you should too.  Here is a link to grab all sorts of geo-spatial data for Mars.

https://isis.astrogeology.usgs.gov/IsisSupport/index.php?topic=1166.0

Enjoy

Convert time Object to datetime Objects

Here is a quick snippet to convert a time object to a datetime object.

import time
import datetime
gmt_time_object = time.gmtime()
dt = datetime.datetime.fromtimestamp(time.mktime(gmt_time_object))

Now you have a datetime object from a time object.
Enjoy

Merging Python Dictionaries

Sometimes you need to merge two dictionaries into one, and here is a pythonic way of doing it:

>>> x = {'key1': 34, 'key2': 35}
>>> y = {'key3':36', 'key4': 36}
>>> z = x.copy() 
>>> z.update(y)
>>> print z
{'key1': 34, 'key2': 35, 'key3': 36, 'key4': 36}

This method copies the x dictionary into a new set of memory.  This needs to be done because just assigning to a new variable say z = x will reference the same block of memory thus when you update q, you will update dictionary x as well, and we do not want that.  So copy() allocates a new set of memory blocks, puts the information into those blocks, then update() will add the additional keys from an existing dictionary.

It's very simple, and the pythonic way.  (The pythonic way is sort of like the force, but you can move things with your mind or alter thoughts, or do parlor tricks... so it's not like the Force at all)

Enjoy

Understanding the Density Tool Outputs

Straight from the Esri blog on analysis, is a great article talking about the density tool output.

It's worth a read, and it can be found here.

Enjoy

How to Create a Row with an Empty BLOB in ArcPy

Insert and Update cursor do now support None type in Blob fields, so what does one do?  If you try to set your row object to 'None' you get this error message:

expected contiguous memoryview. type: NoneType

Now let's assume you want to pass None, because for some record you do not have an object that you need to pass for that field.  Use the Memoryview object to solve this problem.  The documentation on the object can be found here.

The python document describe the object as such:

A memoryview object exposes the new C level buffer interface as a Python object which can then be passed around like any other object.

Now in our context, we must do the following:

row[0] = memoryview('')

That's it. Now we have something similar to 'None' for your blob field.

I also posted a quick tutorial on read/writing blob data using arcpy.da here.


Please support this idea of having GUI designer in python built in with python add-ins.

It can be found here. (direct link: http://ideas.arcgis.com/ideaView?id=087E00000004SmHIAU)

Enjoy

Working with numpy's Structured Array and numpy.dtype

In my previous post, I showed how to quickly get access data (2007/2010) into a file geodatabase without creating an ODBC connection in ArcCatalog/Map using pyODBC.  You might have noticed that I used numpy to create a table pretty easily, but you might be wondering what are the dtypes?

numpy.dtype are data type objects that describe how the bytes in a fixed-size block of memory are seen by the system.  So is the data a string, number, etc...  It describes the follow aspects of data:

1. Type of data (integer, float, object, ...)
2. size of data (number of bytes)
3. describes an array of data
4. names of fields in the record
5. if the data is a sub-array, it's shape

Getting the dtype formats for common python data types if fairly easy in python. The numpy.dtype() will return the proper format for almost any python data type:

>>> print numpy.dtype(str)
|S0

For array protocol type strings, there are various data types supported:

'b'	Boolean
'i'	(signed) integer
'u'	unsigned integer
'f'	floating-point
'c'	complex-floating point
'S', 'a'	string
'U'	unicode
'V'	anything (void)

(source: numpy help documents)

This allows you to specify thing like string length.
Example:

>>> dt = numpy.dtype('a25')  # 25-character string

After you know what your data types are, you will want to associate these types with the fields in the records.  I prefer to use the optional dictionary method where there are two keys: 'names' and 'formats'.  You would then pass this information to create your 'structured array'.

Example:

>>> dt = numpy.dtype({'names': ['r','g','b','a'],

     'formats': [numpy.uint8, numpy.uint8, numpy.uint8, numpy.uint8]})

>>> colors = numpy.zeros(5, dtype = dt)

>>> print colors

[(0, 0, 0, 0) (0, 0, 0, 0) (0, 0, 0, 0) (0, 0, 0, 0) (0, 0, 0, 0)]

There are many ways to declare dtypes for your data, and you can read them all here.
More on structured arrays in numpy can be found here.

Enjoy

Using pyODBC to Connect To Access with ArcPy

pyODBC allows users to access any data source with the proper ODBC driver installed on the system.  This is very convenient, and helpful because at 10.1 using ArcPy only you cannot create ODBC connection.

You can download pyODBC here.  Grab the correct python version and install it.

Once installed, let's try to access a 2010-2012 MS Access Database.

I create a dummy database called 'db_text.accdb' and create a table called 'Addresses' with 5 text fields: 

1. name
2. street
3. town
4. country
5. zipcode

After that I populated two rows with dummy data.  Once you have some dummy data, let's move forward.

import pyodbc

import arcpy

import numpy

arcpy.env.overwriteOutput = True

accb = r"C:\temp\db_test.accdb"
access_con_string = r"Driver={Microsoft Access Driver (*.mdb, *.accdb)};Dbq=%s" % accb
cnxn   = pyodbc.connect(access_con_string)
cursor = cnxn.cursor()
cursor.execute("select * from Addresses")
rows = cursor.fetchall()

dts = {'names': ('ID','name','street', 'town', 'country', 'zipcode'),
       'formats':(numpy.uint8, 'S255','S255','S255','S10','S10')}

array = numpy.rec.fromrecords(rows, dtype=dts)

arcpy.da.NumPyArrayToTable(array, r"c:\temp\scratch.gdb\outTable")

Here we connected to the MS Access database using the drivers installed on my local system. Next I performed a simple query to return all records from the table. With the new da.NumPyArrayToTable() at 10.1, I want to convert the pyodbc rows to a table that can be used in ArcMap. I convert the list of tuples (rows object) to a numpy.array object. After the conversion, I fire off the arcpy function and now I have a table on disk.

For a complete listing of 10.1 numpy functions, check out: http://resources.arcgis.com/en/help/main/10.1/index.html#/What_is_the_data_access_module/018w00000008000000/

There you can learn about extending tables, convert feature classes to numpy arrays, and rasters as well.

Enjoy

Dynamic Layers and Workspaces

An interesting and exciting aspect that I really like at 10.1 is the use of dynamic map services to display data from a work space on disk.

The exciting aspect is this:

• Dynamic Rendering
• Add 1:n layers to a map service

Where is gets interesting is this, in ArcPy, you cannot update symbology for feature classes.  So essentially if you wanted to add a feature class to a map document, you have to accept the default symbology, or apply the symbology from an already existing set of layer files (.lyr).  Not real dynamic, and at least I feel, a horrible choice.  

Leveraging the web map REST specifications, as a developer you could create a map object (mapping.MapDocument object) and export the display to PNG, PDF, JPG, etc...  The downside is that at 10.1, it appears you cannot save the MapDocument object to disk and view the dynamic layer in the TOC.  This will throw an error when you open the map document file.

So what is this good for you are probably thinking?  It's true you cannot view dynamic layers in 10.1, but the ability to use dynamic layers gives data driven pages the much needed shot in the arm to add layer on the fly from a web application.  The new printing services for ArcGIS for Server leverages this concept except the out of box tool only produces one export at a time, where as a data driven page tool would produce multiple exports from a single input.

To get this concept to work, you would have to do the following:

import arcpy

from arcpy import mapping

json = arcpy.GetParameterAsText(0)
results = mapping.ConvertWebMapToMapDocument(json)

mxd = results.mapDocument

mapping.ExportToPDF(mxd, r"c:\temp\export.pdf")

Here we created a new map object from a JSON string input and dumped it out to a PDF.  (JSON not included here)

ConvertWebMapToMapDocument() has some other very interesting functions/properties.  It can incorporate existing map document templates, and even extract Point, Lines and Polygon graphics to feature classes when passed as JSON to the function.

You can find more about ConvertWebMapToMapDocument here: http://resources.arcgis.com/en/help/main/10.1/index.html#//00s30000006n000000.  In addition to the function you can also find examples of web map JSON so you can quickly start playing around with the functions.


Please support this idea of having GUI designer in python built in with python add-ins.

It can be found here. (direct link: http://ideas.arcgis.com/ideaView?id=087E00000004SmHIAU)

Enjoy

Great Learning Python Resource

Hey Everyone,

The Internet is filled with great resources for learning programming, almost too much, but here is a great resource: http://learnpythonthehardway.org/book/

This site hosts a e-book called 'Learn Python The Hard Way' and I highly recommend if you are starting out programming, or want to just a refresher course on python that you take a look.

There are also courses on C, CLI, SQL, Ruby, and Regex.

You might learn something new, so give it a try.

As always, I encourage my reader to please support my idea on the ArcGIS Ideas Site to allow GUI integration with ArcPy.  Please vote it up here: http://ideas.arcgis.com/ideaView?id=087E00000004SmHIAU

Enjoy

Support GUI Design in ArcGIS for Desktop

Please support this idea of having GUI designer in python built in with python add-ins.

It can be found here: http://ideas.arcgis.com/ideaView?id=087E00000004SmHIAU

Thanks everyone.

Creating Data Frame Extents (ArcPy 10.1)

I read your comments, and thank you for posting them.  I was asked how do you clip all layers in a map document by the current data frame's extent.

In a previous post, I discussed how to create extent polygons for feature classes and individual features.  The same method applies for creating an extent polygon using data frames.

The data frame object has a whole host of properties and methods which can be found here.  To get the extent of a data frame, just reference the extent property to obtain the data frame's current extent.  Create the extent polygon or feature class, and clip by the extent geometry.

# Clip layers by extent of data frame (assuming in arcmap session)
import os
import arcpy
from arcpy import env
from arcpy import mapping
mxd = mapping.MapDocument("CURRENT")
df = mxd.activeDataFrame
extent = df.extent

array = arcpy.Array()
array.add(extent.lowerLeft)
array.add(extent.lowerRight)
array.add(extent.upperRight)
array.add(extent.upperLeft)
array.add(extent.lowerLeft)
polygon = arcpy.Polygon(array, df.spatialReference)
array.removeAll()
del array
for layer in mapping.ListLayers(mxd, data_frame=df):
    clipped_fc = env.scratchGDB + os.sep + layer.datasetName + "_clipped"
    arcpy.Clip_analysis(in_features=layer, 
                        clip_features=polygon,
                        out_feature_class=clipped_fc)
    del layer
del mxd
del df
del extent

This code is just a quick re-hash of old concepts applied to a new method. The extent object creates a polygon, which is used to clip the layers in that data frame.

ArcGIS for Developers

Today I noticed a new site from Esri, and it's called 'ArcGIS for Developers'.  This website consolidates the web, mobile, and introduces some new desktop APIs.

The one that really excites me is the Qt API because I'm a fan of C++.  I also introduces the new approach to open for Esri by using GitHub for code sharing.  I've only started playing, but I'm excited to see what comes.

Check it out here: http://developers.arcgis.com/en/

Happy coding.

The random Module (python)

The random module is pseudo-random number generator.  This means that's truly not random, but close enough.

You should seed, initialize the generator with either a value, or with the system time (default value).

Example:

>>> import random
>>> random.seed()
>>> print random.random()
0.884447776659
>>> print random.random()
0.379767969805
>>> print random.random()
0.580327390006

Random also has the ability to pick from ranges and collection of elements, like letter:

>>> random.randrange(start=0, stop=101, step=1)
52

>>> random.choice('abcdefghij')
'b'

Happy Randomness.

ArcGIS Server and Spatial Analyst

In ArcGIS 10.0, you needed to worry about path length for raster operations if the raster path plus file name is over 154 characters.  Now at 10.1, the character length has been increased to 254.  This means for most raster operations you will not have to alter the TEMP variable for the ArcGIS account.

If you do run into issues where spatial analyst or 3D analyst tools fail on server, but work on desktop in your model, try changing the ArcGIS account's TEMP environmental parameter to a different folder.

A good example would be c:\serverwrksp.  You also should make sure that the ArcGIS account has read/write access to that folder.


To shorten or change the path that server uses, you need to modify the ArcGIS account's TEMP variable.  Server should then honor the new destination path.  You might have to either log in and log off or reboot the box to get the OS to honor the new variable.

Here is how you change the variable in Windows:
To view or change environment variables:

1. Right-click My Computer, and then click Properties.
2. Click the Advanced tab.
3. Click Environment variables.
4. Click one the following options, for either a user or a system variable:
   • Click New to add a new variable name and value.
   • Click an existing variable, and then click Edit to change its name or value.
   • Click an existing variable, and then click Delete to remove it.

Please support my idea of having a python GUI by voting it up here.

Enjoy

Creating a List with n number of entries (Python)

Normally when you create a list object in python, you cannot specify the number of items in the object by default unlike say an array object in C# or C++.  I recently ran into a situation where I was using the arcpy.da cursor objects for table that could have n number of fields.  I wanted the row to be initialized with a default value, so I used the * operator on the list to multiply a single value into a multi-value list.

Example:

# Get a list of fields that are not BLOB, OBJECTID, RASTER, GEOMETRY or GUID
fields = [field.name for field in arcpy.ListFields(service_table) \
                  if not(field.type.upper() in ['BLOB', 'OID', 'RASTER','GEOMETRY','GUID'])
# create a row list with default values
row = [""]* len(fields)

This will create a list object called row, where each value by default is an empty string ("").

Enjoy

Query Feature Service By Object IDs (Python)

To query a feature service using object ids, you need to perform a POST.  For those who don't know the difference between a "GET" and "POST" here is a quick blurb based on the HTML specifications:

"POST" means that former means that form data is to be encoded (by a browser) into a URL while the "GET" means that the form data is to appear within a message body. 

Even simpler, "GET" is basically for just getting data where a "POST" may involve anything, like updating data, sending, or creating data.

Using python, you can perform both "GET" and "POST" methods, but since we need a "POST" to query by IDs, here is a simple example.  Please note that most feature services limit 1000 features being returned, so if you want to grab all of the features from a feature service, you'll have to perform multiple queries to get all the features back in JSON format.

Example:

import urllib2
import urllib
import urlparse
import httplib

def query_by_objectid(url, objectIDStart=0, objectIDEnd=1001):
    """ performs a POST operation where the query is called using
        the object id method.  If a feature service has more than
        1000 records, use this method to get a range of features
        from the feature service.

        Inputs:
           :url:  - string of feature service URL
           :objectIDStart: - integer of the start whole number
           :objectIDEnd: - end range whole number
        Returns:
           returns string JSON of query
    """
    url = url + '/query'
    start = int(objectIDStart)
    end = int(objectIDEnd)

    objectIDs = ",".join([str(x) for x in range(start, end)])
    headers = {"Content-type": "application/x-www-form-urlencoded",
               "Accept": "text/plain"}
    parameters = {'objectIds' : objectIDs,
                  'f' : 'json'}
    urlparams = urllib.urlencode(parameters)
    parts = urlparse.urlparse(url)
    h = httplib.HTTPConnection(parts.netloc)
    headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"}
    h.request('POST', parts.path, urlparams, headers)
    r = h.getresponse()
    return r.read()

if __name__ == "__main__":
    url = 'http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/TaxParcel/AssessorsLiveLayers/MapServer/1'
    print query_by_objectid(url=url,
                            objectIDStart=0,
                            objectIDEnd=5)

So we have a simple "POST" example. This sample should work with both ArcGIS Server 10.1 and ArcGIS Online feature services.

Please support my idea of having a python GUI by voting it up here.

Enjoy

Creating an Map Document From Python (10.1)

Previous posts, I have discussed ways to create map service layers from scratch, and now I'm going to show you how to create a blank map document using the 'ConvertWebMapToMapDocument' tool in ArcToolbox.  The beauty of this tool is that it doesn't have to be run on server, but can be run from desktop.  It uses JSON based on the ExportWebMap specification.

A JSON map consists of the following:

{ 
   "mapOptions": {}, 
   "operationalLayers": [], 
   "baseMap": [], 
   "exportOptions": {}, 
   "layoutOptions": {} 
}

import json
from arcpy import mapping
mapService = {} 
jsonDump = json.dumps(mapService) 
result = mapping.ConvertWebMapToMapDocument(jsonDump)
mxd = result.mapDocument
mxd.saveACopy(r"c:\temp\blank.mxd")

Now you can do whatever you need to do with your map document.

wx GUI Article for ArcGIS

Here is an interesting article on how to create an interactive GUI for ArcGIS 10.1.

Basically what it does is it creates the form before ArcMap/Catalog finishes loading therefore the core software doesn't prevent the mainloop() to fail.  At least that's my take on it. 

If you think custom GUIs should be part of the next major release, I suggest you go to the Esri Idea site and vote up this idea.  It will need YOUR support to get into the next release.

Happy GUI designing!

Convert Table or Feature class to CSV (10.1)

Sometimes you need to export data from a feature class or table to a CSV file.  CSV stands for comma separated values.  Wikipedia defines a CSV file as such:

A comma-separated values (CSV) file stores tabular data (numbers and text) in plain-text form. Plain text means that the file is a sequence of characters, with no data that has to be interpreted instead, as binary numbers. A CSV file consists of any number of records, separated by line breaks of some kind; each record consists of fields, separated by some other character or string, most commonly a literal comma or tab. Usually, all records have an identical sequence of fields.

To create a CSV file at 10.1, you need to strip out certain field types: Geometry, Blob and Raster because the file format only supports plain text.  This can be done by doing the following:

 fieldnames = [f.name for f in desc.fields if f.type not in ["Geometry", "Raster", "Blob"]]

Next you need to write the field names and rows to a file.  This is extremely easy with the CSV library in python.  Documentation about this standard library can be found here.

def get_rows(data_set, fields):
   with da.SearchCursor(data_set, fields) as cursor:
      for row in cursor:
         yield row
if __name__ == "__main__":
   data_set = arcpy.GetParameterAsText(0) # feature class/Table
   output = arcpy.GetParameterAsText(1) # csv file
   desc = arcpy.Describe(data_set)
   fieldnames = [f.name for f in desc.fields if f.type not in ["Geometry", "Raster", "Blob"]]
   rows = get_rows(data_set, fieldnames)
   with open(output,'wb') as out_file:
      out_writer = csv.writer(out_file)
      out_writer.writerow(fieldnames)
      for row in rows:
         out_writer.writerow(row)

Here we have a function called get_rows() which takes two parameters.  The first is the data_set, which can be a table or feature class.  The next is the fields.  At 10.1, you must define your fields unlike the 10.0 cursor objects.  The function uses the yield, which is a generator.  Basically the code only runs if the function is called in a loop (I know that's not 100% correct), but here is a better explanation.  Using the CSV module in python, we can then easily write out each row within the rows generator object.

FYI, this is written so you can put this code into a script and toolbox for ArcGIS 10.1.  Just add the imports.

Enjoy

10.1 SP 1 32 bit vs 64 bit

I ran across an interesting forum posting about the differences of 32 vs 64 bit python.

Check it out here: http://forums.arcgis.com/threads/70241-10.1-sp-1-and-32-64-bit-Python-versions

It's worth a good read to understand if you install the 64-bit background processor provided at SP 1 how it operates and works on your machine.

Remember if you do install the 64-bit version of python, you will have to install the 64-bit of all your extensions as well.

Enjoy

ArcPy 10.1 - Multiple Data Frames and Data Driven Pages

Data driven pages is a great way to automate map production over a given area, and it works right out of the box with a single data frame.  If you have 2 or more data frames though, you will need to control the movement of each data frame object by performing some sort of coding (ArcObjects or ArcPy).

A simple way to move multiple data frames is to:

1. Get a list of all data frames
2. Identify the parent data frame
3. Move the extents of the other data frame

Sounds simple?  Well it is.

To get started, get a reference to the map document

mxd = arcpy.mapping.MapDocument("CURRENT")

Change "CURRENT"  to a path if not run in ArcMap.  Then list all the data frames

dfs = arcpy.mapping.ListDataFrames(mxd)

Now how do you know what data frame is the parent data frame or what data frame is a child data frame? Luckily at 10.1, the data driven page object provides a property called dataFrame, which returns the DataFrame object.  At 10.0, you would either use names or indexing values to determine which is the parent or not, but this example is for 10.1.

ddp = mxd.dataDrivePages

parentDF = ddp.dataFrame

Next step is to pan the other data frames to the extent of the parent data frame's extent.

for pageNum in range(1, ddp.pageCount + 1):

   for df in dfs:

      ddp.currentPageID = pageNum

      row = ddp.pageRow

      extent = row.Shape.extent

      if (df.name == parentDF.name) and (df.scale == parentDF.scale):

         print 'I found the parent frame'

      else:

         df.panToExtent(extent)

   arcpy.mapping.ExportToPNG(mxd, r"C:\Project\OutPut\ParcelAtlas_" + str(pageNum) + ".png")

Here we have a bunch of code that loops through each data driven page area and then modifies each underlying data frame that isn't the parent.  After the data frames are changes, the code then exports the maps to PNG to the '...\output' folder with reference to the page number.

Enjoy