Unique Values from NumPy Array

Here is a quick example of how to get a unique value list from a numpy array.

from __future__ import print_function
import numpy as np

values = [['a', 'b', 'c'],
          ['e', 'b', 'd'],
          ['a', 'b', 'c']]
np_array = np.array(values)
print (np.unique(np_array[:,1], return_index=False))

# should see: ['b']

Enjoy!

Speed Up Polygon Grid Generation

There are many tools that generate regular uniform grids with geoprocessing, but I found a novel was of generating grids pretty quickly for large areas.  Rasters are fast and can be created on the fly easily using numpy.  To make the raster a feature class, ArcPy has a Raster to Polygon tool that should be available at all levels of desktop.

import arcpy
import numpy
import math
fc = r"c:\temp\somedata.shp" # assume projection is projected coordinate system not Lat/Long where measurements are in decimal degrees
width = 10000 # meters
height = 10000 # meters
#  Gets the feature classes' extent
#
desc = arcpy.Describe(fc)
extent = desc.extent
#   Set the Mask
#
arcpy.env.mask = fc
#   Calculate the number of columns
#

dist_x = math.sqrt(math.pow((extent.XMax - extent.XMin), 2))
dist_y = math.sqrt(math.pow((extent.YMax - extent.YMin),2))
col_x = int(dist_x / width)
col_y = int(dist_y / height)
if col_x % distanceMeters > 0:
   col_x += 1
if col_y % distanceMeters > 0:
  col_y += 1
#   Construct the structure numpy array with random values
#
numpyArray = (numpy.random.permutation(col_x * col_y) + 1).reshape(col_y, col_x)
#   Convert array to raster 
#
myData = arcpy.NumPyArrayToRaster(numpyArray,
                                  arcpy.Point(extent.XMin, extent.YMin),
                                  width, height)
#   Convert the raster to a polygon feature class
#
grid = r"%s\gridData" % arcpy.env.scratchGDB
grid = arcpy.RasterToPolygon_conversion(in_raster=myData,
                                            out_polygon_features=grid,
                                            simplify="NO_SIMPLIFY",
                                            raster_field="VALUE")[0]

The code takes the extent of the data and creates a grid. The mask environmental parameter essentially clips the raster to the shape of the feature class. This way you only gets cells within the given polygon area. If you wanted cells to fall outside the feature classes' shape, then just comment out that line.

So that is that.


If you haven't done so, check out ArcREST.
Also vote for my idea for a GUI builder for Python Add-Ins.

Working with numpy's Structured Array and numpy.dtype

In my previous post, I showed how to quickly get access data (2007/2010) into a file geodatabase without creating an ODBC connection in ArcCatalog/Map using pyODBC.  You might have noticed that I used numpy to create a table pretty easily, but you might be wondering what are the dtypes?

numpy.dtype are data type objects that describe how the bytes in a fixed-size block of memory are seen by the system.  So is the data a string, number, etc...  It describes the follow aspects of data:

1. Type of data (integer, float, object, ...)
2. size of data (number of bytes)
3. describes an array of data
4. names of fields in the record
5. if the data is a sub-array, it's shape

Getting the dtype formats for common python data types if fairly easy in python. The numpy.dtype() will return the proper format for almost any python data type:

>>> print numpy.dtype(str)
|S0

For array protocol type strings, there are various data types supported:

'b'	Boolean
'i'	(signed) integer
'u'	unsigned integer
'f'	floating-point
'c'	complex-floating point
'S', 'a'	string
'U'	unicode
'V'	anything (void)

(source: numpy help documents)

This allows you to specify thing like string length.
Example:

>>> dt = numpy.dtype('a25')  # 25-character string

After you know what your data types are, you will want to associate these types with the fields in the records.  I prefer to use the optional dictionary method where there are two keys: 'names' and 'formats'.  You would then pass this information to create your 'structured array'.

Example:

>>> dt = numpy.dtype({'names': ['r','g','b','a'],

     'formats': [numpy.uint8, numpy.uint8, numpy.uint8, numpy.uint8]})

>>> colors = numpy.zeros(5, dtype = dt)

>>> print colors

[(0, 0, 0, 0) (0, 0, 0, 0) (0, 0, 0, 0) (0, 0, 0, 0) (0, 0, 0, 0)]

There are many ways to declare dtypes for your data, and you can read them all here.
More on structured arrays in numpy can be found here.

Enjoy

Finding and Changing Max Value in Numpy Array

Today, I answered a forum question about finding and changing a numpy array.  You can use the max() and the where() in the numpy to search for values in your array and change them using loops.

You can download a very simple example here.

Here is the simple code:

import numpy
myarray = numpy.array([[  5.4,   7.5,   2.2,   8.5,   8.6,   7.5],
                       [  7.7,   3.5,   1.4,   9.6,   8.5,   5.5],
                       [  5.2,   6.1,   8.6,   6.7,   4.3 ,  6.8],
                       [  9.6,   4.5,   2.7,   3.6,   6.7,   4.5],
                       [  1.2,   2.3,   7.2,   6.3,   2.2,   2.0 ],
                       [  1.3,   2.0,  -99.0,    9.6,  -99.0,    1.2]
                       ]
                      )
print myarray
maxValue = myarray.max()

itemindex = numpy.where(myarray >= maxValue)

for i in xrange(0, len(itemindex[0])):
    array1 = itemindex[0]
    array2 = itemindex[1]
    myarray[array1[i]][array2[i]] = 0

print myarray

Here we have the numpy array and find the array's maximum value.  Then using the where(), the location in the array are given back.  From there, the code then changes the values in the 2-D array from the maximum value back to zero.

Enjoy