Using Tkniter in ArcToolbox

If you download this script: http://resources.arcgis.com/gallery/file/geoprocessing/details?entryID=C2594BA8-1422-2418-A048-74E477699334 from the resource center and want to call it from a toolbox, you can do the following:

1. Open ArcCatalog
2. Create a new toolbox (something.tbx)
3. Create a new script
4. Point to the file from the download
5. Uncheck both "Show command window when executing" and "Run python script in process"
6. Press finish

Now the GUI form will show.  

(Note, I didn't write that script, but I'm using it as an example of showing a GUI form from ArcToolbox.  If you have issues with the code, please contact the developer of the script.)

Enjoy

Python Add-Ins for 10.1

I'm really exciting for the increasing integration of python inside of ArcGIS Desktop.  If you have done any desktop development at 10 then you will be familar with the Add-In wizard.  Check out the information here.

The types allowed: (source: webhelp.esri.com)

Add-In Type	Description
Buttons	A button is the simplest form of functionality that can be used to execute some business logic when it is clicked. A button assumes that an action or a series of actions will be executed. Buttons may exist on toolbars and menus.
Tools	A tool is very similar to a button. However, a tool requires user interaction with the desktop application’s display first, and then based on that interaction, executes some business logic. The Zoom In tool in ArcMap is a good example—it requires that you click or drag a rectangle over a map before the  display is redrawn, showing the map contents in greater detail for the specified  area. Tools may exist on toolbars and tool palettes.
Combo Boxes	A combo box provides a set of choices, visually seen as a drop-down box, from which a selection can be made and acted upon. A combo box can also be enabled to allow an end user to add a value in the combo box. The Scale combo box  in ArcMap is a good example of a combo box. When data with a known coordinate system is added to ArcMap, the scale combo box is enabled giving the user a set of predefined scales from which to choose. It also allows a user to type a new scale not in the list of options, and the display is updated based on the value added.
Menus	A menu is a container for buttons or other menus. Menus can be exposed by adding them to an existing toolbar (see Creating an Add-In toolbar for more information on creating your own toolbar) or an existing menu.
Toolbars	A toolbar is a container for buttons, tools, combo boxes, tool palettes, and menus. Toolbars can be floating or docked in desktop applications, just like any system toolbar within the application. Toolbars can also be activated so they are visible when a desktop application is started.
Tool Palettes	A tool palette is a container for tools. Tool palettes can only be exposed by adding  them to an existing toolbar (see Creating an Add-In toolbar for more information on creating your own toolbar).
Application Extensions	Application extensions can be used in different ways to provide supplemental functionality to ArcGIS Desktop: 1. Application extensions are used to coordinate activities between other components —such as buttons and tools—within a containing add-in. For example, an extension  may not activate a set of tools on a toolbar unless a specific set of layers exist in the map. 2. Application extensions are usually responsible for listening and responding to various events exposed by the host application. For example, anytime a layer is added  or removed, an event is triggered, and the extension responds by automatically saving the map document.

Feature Class as JSON (ArcPy 10.1)

I just learned a really easy way to display a feature class as JSON by using featureset object and the Describe().

fc = r"c:\temp\myData.shp"
featurSet = arcpy.FeatureSet()
featureSet.load(fc)
desc = arcpy.Describe(featureSet)
print desc.pjson # so we can read it
#### desc.json also works. ####
del desc
del fc
del featureSet

Output


{
  "displayFieldName" : "",
  "geometryType" : "esriGeometryPolygon",
  "spatialReference" : {
    "wkid" : 4326,
    "latestWkid" : 4326
  },
  "fields" : [
    {
      "name" : "FID",
      "type" : "esriFieldTypeOID",
      "alias" : "FID"
    },
    {
      "name" : "Id",
      "type" : "esriFieldTypeInteger",
      "alias" : "Id"
    }
  ],
  "features" : [
    {
      "attributes" : {
        "FID" : 0,
        "Id" : 0
      },
      "geometry" : {
        "rings" : [
          [
            [
              -170.48758544651321,
              27.863404726470606
            ],
            [
              -167.28669772639196,
              27.956184080677076
            ],
            [
              -167.28669772639196,
              25.497531194206999
            ],
            [
              -170.53397512361644,
              25.327435711495355
            ],
            [
              -170.48758544651321,
              27.863404726470606
            ]
          ],
          [
            [
              -169.37423319603627,
              27.229412472726722
            ],
            [
              -169.28145384182986,
              25.961427965239068
            ],
            [
              -167.90522675443472,
              26.208839576456285
            ],
            [
              -167.93615320583692,
              27.337655052634318
            ],
            [
              -169.37423319603627,
              27.229412472726722
            ]
          ]
        ]
      }
    }
  ]
}


Enjoy

Python for ArcGIS 10.1

At 10.1, python developers will be using Python 2.7.  This is the last 2.x release of python before moving to the Python 3.x releases.  You can find out all about python 2.7 here.

Enjoy

Compressing File Geodatabases in a Folder

In a previous post, I showed how to uncompress a set of file geodatabases in a folder.  To compress a set of file geodatabases, using the CompressFileGeodatabase_management().  This script will use the List functions to list all the file geodatabases and then compress them.  When completed successfully it returns 'true' else it returns false.

import arcpy
from arcpy import env
if __name__ == '__main__':
    try:
        workspace = arcpy.GetParameterAsText(0)
        env.workspace = workspace
        fgdbs = arcpy.ListWorkspaces("*","FileGDB")
        for fgdb in fgdbs:
            arcpy.CompressFileGeodatabaseData_management(fgdb)
        env.workspace = None
        arcpy.SetParameterAsText(1,True)
    except:
        env.workspace = None
        arcpy.AddError(str(arcpy.GetMessages(2)))
        arcpy.SetParameterAsText(1,False)

Enjoy

Checking Hard Drive Space Using Python

Have you ever needed to check how much space is left on a drive, but didn't know how, well now you will.  Python ctypes and os libraries allow a developer to check the HD space pretty easily.  If you have a windows system, you can use ctypes to leverage the kernal32.dll functions, and if you have the other operating systems, python has a built in function called os.statvfs().


import ctypes
import platform
import os


class HardDriveSpaceException(Exception):
    def __init__(self, value):
        self.parameter = value
    def __str__(self):
        return repr(self.parameter)


def get_free_space(folder, format="MB"):
    """ 
        Return folder/drive free space 
    """
    fConstants = {"GB": 1073741824,
                  "MB": 1048576,
                  "KB": 1024,
                  "B": 1
                  }
    if platform.system() == 'Windows':
        free_bytes = ctypes.c_ulonglong(0)
        ctypes.windll.kernel32.GetDiskFreeSpaceExW(ctypes.c_wchar_p(folder), None, None, ctypes.pointer(free_bytes))
        return (int(free_bytes.value/fConstants[format.upper()]), format)
    else:
        return (int(os.statvfs(folder).f_bfree*os.statvfs(folder).f_bsize/fConstants[format.upper()]), format)
if __name__ == "__main__":
    try:
        byteformat = "mb"
        size =  get_free_space(r"c:", byteformat)
        print size
        if size[0] < 10000000000:
            raise HardDriveSpaceException("Hard drive space limit reached, there is only %s %s space left." % (size[0],size[1]))
    except HardDriveSpaceException as e:
        print e

In this example, there is a function that checks your HD space, and a custom error. The custom error is raised if the numeric value is over a specific value.

I've tested the HD space function on Win7, Win7 x64, and Window Server 2008 R2 x64.  It works.  I don't have any OS's that aren't windows based, so let me know if it works on iOS or the other that some people use.

Some maybe wondering why would I want to do something like this.  Well, if you are creating content automatically, such as PDFs or any geospatial content, you want to ensure that you have the space to save it.

Enjoy

Uncompressing Multiple File Geodatabases

Often when I receive spatial data from individuals on a DVD or CD, it is in the form of a file geodatabase and it's compressed.  If all the data is in a single file geodatabase, it is not a big problem, but when you get multiple file geodatabases, that's when I break out my python.

Here is a helpful little script that will try to uncompress all file geodatabases in a single folder.  It should be noted that if a file geodatabase is already uncompressed, it will just be ignored.

import arcpy
from arcpy import env
if __name__ == '__main__':
    try:
        workspace = arcpy.GetParameterAsText(0)
        env.workspace = workspace
        fgdbs = arcpy.ListWorkspaces("*","FileGDB")
        for fgdb in fgdbs:
            arcpy.UncompressFileGeodatabaseData_management(fgdb)
        env.workspace = None
        arcpy.SetParameterAsText(1,True)
    except:
        arcpy.AddError(str(arcpy.GetMessages(2)))
        arcpy.SetParameterAsText(1,False)
So what I have done is list all file geodatabases in a given workspace (folder), then I just use the standard UncompressFileGeodatabaseData_management().  When I'm done processing, I set my environmental workspace variable to None just to clean up some loose ends.  I could delete the object like fgdbs, but since they are self contained within the function, those variables should not persist beyond the life of the function.

Enjoy

Sorting a List of Lists

In 10.1, the new arcpy.da (new cursor functions) returns essentially a list of lists.  A way to sort these list of lists is to use sorted() which is a built in function of python.  More can be found here.

Here is a simple example, but I think it will get what you need.


>>> myresults = [
[1,"A",2],
[2,"Z",1],
[3,"J",0]
]
>>> print sorted(myresults, key=lambda x:x[0])
[[1, 'A', 2], [2, 'Z', 1], [3, 'J', 0]]
>>> print sorted(myresults, key=lambda x:x[1])
[[1, 'A', 2], [3, 'J', 0], [2, 'Z', 1]]
>>> print sorted(myresults, key=lambda x:x[2])
[[3, 'J', 0], [2, 'Z', 1], [1, 'A', 2]]

Here we have our list with 3 entries in each sub-list item. The lambda function tells the sorted() what to sort on. The numeric value is the position in the sub-list item inside the parent list.

Quicksort() using python

One method of sorting lists is to use the quicksort.  The elements in the list must have a strict weak order and the index of the array can be of any discrete type.


Follow these steps to implement:

1. Choose any element of the list to be the pivot.
2. Divide all other elements (except the pivot) into two partitions.
   • All elements less than the pivot must be in the first partition (lower half list).
   • All elements greater than the pivot must be in the second partition (upper half list).
3. Use recursion to sort both partitions.
4. Join the first sorted partition, the pivot, and the second sorted partition

The run time of quicksort ranges from O(n log n) with the best pivots, to O(n²) with the worst pivots, where n is the number of elements in the list.  (O is complexity)

def qsort(list):
    if not list:
        return []
    else:
        pivot = list[0]
        less = [x for x in list     if x <  pivot]
        more = [x for x in list[1:] if x >= pivot]
        return qsort(less) + [pivot] + qsort(more)

Example:
someList = [3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5, 8, 9, 7, 9, 3]
print qsort( someList )
[1, 1, 2, 3, 3, 3, 4, 5, 5, 5, 6, 7, 8, 9, 9, 9]


Enjoy the sort.

In the Esri's ArcGIS for SharePoint SDK, you can create buttons in the SharePoint ribbon.  In this example, I will talk about the OnHidden event.  This event fires when the Hide() is called on the floating window after it is displayed.  This can be used to signal a process that the event is finished, and now you can do some other process or show another form etc...

Here I just implemented ICommand, and kept everything default.  I created a custom event EventHandler called OnHidden that will get call when the floating window is hidden or closed by a user.

    [Export(typeof(ICommand))]
    [DisplayNameAttribute("This is a Demo")]
    [DefaultIcon("/DemoFun;component/Images/Demo.png")]
    public class cmdGenerateExcelTemplate : ICommand
    {
        public bool CanExecute(object parameter)
        {
            return true;
        }

        public event EventHandler CanExecuteChanged;
        public event EventHandler OnHidden;
     
        public void Execute(object parameter)
        {
         
            MapApplication.Current.ShowWindow("This is my control",
                 <controlobject>,
                false,
                null,
                OnHidden,
                WindowType.Floating);
            this.OnHidden += new EventHandler(cmdGenerateExcelTemplate_OnHidden);

        }

        void cmdGenerateExcelTemplate_OnHidden(object sender, EventArgs e)
        {
            MessageBox.Show("I'm hidden now!");
        }
That's It.  It's very easy to do, and helpful in some instances.

Enjoy

Using Static Methods on a Class

Static Methods are functions that can be called from a class without creating that object.  From python.org - "A static method does not receive an implicit first argument"

This means basically that it is not dependent on the class from within it resides, but it is an associated function to the overall class. 

Take the example below:

import arcpy
import os
class StaticExample(object):

    def __init__(self,*argv):
        '''
        Constructor
        '''
        pass
    @staticmethod
    def createSDEFromConnectionInfo(folderName,
                           fileName,
                           serverName,
                           serviceName,
                           databaseName,
                           username,
                           password,
                           versionName,
                           saveUserInfo,
                           saveVersionInfo,
                           authType,
                           overwriteExisting=True):
        """
            Static Method To Create an SDE Connection File
            Inputs:
                folderName - The folder path where the .sde file will be stored.
                fileName -  The name of the ArcSDE Connection file. Use the .sde file extension.
                serverName - The ArcSDE Server machine name.
                serviceName - The ArcSDE Service name or TCP port number.
                databaseName - For non-Oracle databases only. The DBMS database to connect to.
                username - Database username to connect with using Database Authentication.
                password - The database user password when using Database Authentication.
                versionName -  The Geodatabase version to connect to.
                saveUserInfo - SAVE_USERNAME —Save the username and password in the connection file.
                               DO_NOT_SAVE_USERNAME —Do not save the username and password in the file.
                               Every time you attempt to connect using the file you will be prompted for
                               the username and password.
                saveVersionInfo - SAVE_VERSION —Save the version name in the connection file.
                                  DO_NOT_SAVE_VERSION —Do not save the version name in the connection file.
                                  Without the version name being saved with the file you will be prompted
                                  for the version to connect to each time you access the connection file.
                authType - Either DATABASE_AUTH or OPERATING_SYSTEM_ATUH - forms of authentication
                overwriteExisting - Tells the function if it can overwrite the .sde file.
        """
        if overwriteExisting == True and \
           os.path.isfile(folderName + os.sep + fileName) == True:
            os.remove(folderName + os.sep + fileName)
        arcpy.CreateArcSDEConnectionFile_management (folderName, fileName,
                                                     serverName, serviceName,
                                                     databaseName, authType,
                                                     username, password,
                                                     saveUserInfo, versionName,
                                                     saveVersionInfo)

To declare your static method, use the descriptor tag @staticmethod.  This tells the class that this function does not need to know about itself.  Using the function is easy:

print StaticExample.createSDEFromConnectionInfo(parm1,param2,....)
>>> c:\temp\mysde.sde

It sometimes is helpful to have functions that are associated with a class be accessible without creating the class object. Some would say just put the function in the .py file outside the class, but then you lose the ability of inheritance on that function.

Happy Coding

Finding Duplicates (3 Ways)

In this post I will show some methods of finding duplicate values in an enterprise SDE database.
The general query is quite simple.  We will perform a SELECT on the table and return the values and the number of times it repeats within that table
First I will use cx_Oracle module to query a table and return both the count and field value that has duplicates:
import cx_Oracle
#.... connection information
#.... create cursor
columnName = "ClientName" 
table = "clients" 
sql = "SELECT %s, count(*) from %s GROUP BY %s HAVING count(*) > 1" % (columnName, table, columnName) 
print cursor.execute(sql).fetchall()
That will return a 2 column list of tuples where as a developer you can then parse that information as needed.

Let's assume you want to do it a more ArcPy way.  ArcSDESQLExecute class can be used to perform the same SQL statement.  The ArcSDESQLExecute class provides a means of executing SQL statements via an ArcSDE connection (source: webhelp.esri.com).

# Two ways to create the object, which also creates the connection to ArcSDE. 
# Using the first method, pass a set of strings containing the connection properties: 
# sdeConn = arcpy.ArcSDESQLExecute("gpserver3","5151","#","toolbox","toolbox") 
# Using the second method pass the path to a valid ArcSDE connection file 
# sdeConn = arcpy.ArcSDESQLExecute("data\Connection to GPSERVER3.sde")
sdeConn = arcpy.ArcSDESQLExecute(r"c:\temp\connectionFile.sde"

columnName = "ClientName" 

table = "clients" 

sql = "SELECT %s, count(*) from %s GROUP BY %s HAVING count(*) > 1" % (columnName, table, columnName) 

results = sdeConn.execute(sql)
for result in results:
   print result # List with [column value, # of duplicates]


You can always turns to ArcToolbox for help if you do not like using SQL. Using Summary Statistics you can get the count of the values in a specific field. It most likely will be slower for large data sets compared to the other two method mentioned above, but it can be used for any supported data set in ArcGIS.

Happy Duplicate Finding

ArcPy - List Columns for Table

In my previous post, I showed you how to use pure SQL in oracle to list the fields of a table.  This can be done using ArcPy as well.
ArcPy has a function called: ListFields (dataset, {wild_card}, {field_type}) where all you are required to pass is the dataset. The wild card and field type are limiters that can help slim down the results.
sdeFile = r"c:\temp\connectionFile.sde" 
fc = sdeFile + os.sep + "myFeatClass" 
fields = arcpy.ListFields(fc) 
for field in fields:
   print field.name

The code returns a field object from which you can access a whole set of properties. This method is preferred when examining spatial data because it provides the data in a format that the Arc system can read and understand.
Enjoy

Listing Oracle Columns Using Python

I've started coding a set of helpful functions for oracle and python using the cx_oracle library for python.  Just download the library and install the one associated with your python install.

In this example, I will list a set of columns in a table using python. First we need to connect to the oracle database
import cx_Oracle
username = "user" 
password = "*****" 
ip = "111.11.11.11" 
port = 1521 
SID = 'ORCL' 
table = 'mytablename' 
dsn_tns = cx_Oracle.makedsn(ip, port, SID) 
db = cx_Oracle.connect(user,passwword, dsn_tns)

All I did was create the entry like it would appear in the tnsname.ora file then pass that information along with the username and password to the database.
Great, now we are connected to the oracle database.  Now the database can be queried.

cursor = db.cursor()
results = cursor.execute("select * from %s where 1=0" % table)
print results.description


[('OBJECTID', <type 'cx_oracle.number'="">, 39, 22, 38, 0, 0), ('NEWNAME', <type 'cx_oracle.unicode'="">, 50, 100, 0, 0, 1)]

Now you do not have to use arcpy to list the fields in a table.  Some other ideas you could do with your enterprise database is: rename fields, create views, etc...

Enjoy