Long Running Scripts

Well, today I've hit a major road block, but I have a work around that should help many of you out there. Have you ever had a process that performs the same operation multiple times, but sometimes that operation fails. It's almost random. Take my situation, I have a clipping function that clips 100s of data sets, but it will randomly fail with a TopoEngine error. The help suggests that you check the geometries of your features, but it turns out, they are fine.

After much search, and questioning, I discovered that I need to delete the Geoprocessing object, then re-create it as such:


import arcgisscripting
from time import strftime
print str(strftime("%H:%M:%S"))
gp = arcgisscripting.create(9.3)
gp.overwriteoutput = True
print str(strftime("%H:%M:%S"))
del gp
print str(strftime("%H:%M:%S"))
gp = arcgisscripting.create(9.3)
gp.overwriteoutput = True
print str(strftime("%H:%M:%S"))


I did it twice, and measured the time. The first time you create the object, it will take the longest. On my machine, it takes 3 seconds to initialize, but after that, the process takes less than a second. Since the processes I am running already take 2-3 hours to complete a 1-2 minute increase is not too hard to live with.

I found that if I delete the geoprocessing object after I perform work on about 3 processes, I eliminate the error.

Enjoy

Listing Only Directories Using Python

Sometimes it's necessary to list directories, but when the os.listdir() method is called, the return list contains both folders and files. To fix this do the following:



dirs = [ name for name in os.listdir(--Directory--)) if os.path.isdir(os.path.join(str(--Directory--), name))]
for d in dirs:
d = str(d)
d = d.upper()
if d.find(".GDB") == -1:
outString = outString + str(d) + "; "



Methods like this are very helpful when you need to use tool validation. It allows you to quickly list all data in a directory, so users can select what they need.

You'll notice that the script has a find(). This is used because file geodatabases are essentially folders and therefore need to be excluded from the generated list.

Enjoy

Table to HTML table

Sometimes it's necessary to generate an html table for returned results. This tool basically takes any format that can support a search cursor, and generates an HTML file on your local disk. I was using this script in conjunction with some reporting tools to analyze AGS server logs.

Enjoy, and I hope it sparks some great GP ideas.



# ---------------------------------------------------------------------------
# TableToHtml.py
# ---------------------------------------------------------------------------

import sys, string, os, arcgisscripting
gp = arcgisscripting.create(9.3)

tablePath = gp.GetParameterAsText(0)
filePath = gp.GetParameterAsText(1)

outfile = open(filePath, "w")
fields = gp.ListFields(tablePath)

fieldNames = []
for field in fields:
if (field.type <> "Geometry" and field.type <> "BLOB"):
fieldNames.append(field.name)

outfile.write(" <table border=""1"">\n")

outfile.write(" <tbody><tr>\n")
for fieldName in fieldNames:
outfile.write(" <th>" + fieldName + "</th>\n")

outfile.write("</tr>\n")

cur = gp.SearchCursor(tablePath)
row = cur.Next()
while row:
outfile.write(" <tr>\n")
for fieldName in fieldNames:
outfile.write(" <td>" + str(row.getValue(fieldName)) + "</td>\n")

outfile.write("</tr>\n")

row = cur.Next()

del cur

outfile.write("</tbody></table>\n")

outfile.flush()
outfile.close()


ArcPad Tips and Thoughts

If anyone is developing for ArcPad 8.0, make sure you get service pack 1. This will correct a whole host of issues with the AXF files. Also, if you wish to connect to an AXF, here is an example:

dim objRSPS, pDS
Set objRSPS = Application.CreateAppObject("RecordSet")
Set pDS = Application.CreateAppObject("DataSource")
pDS.Open(pDSDataSource)
set objRSPS = pDS.OpenLayer("")
If objRSPS.RecordCount > 0 Then
objRSPS.MoveFirst
End If
...do other stuff...


So basically what is happening here, the objRSPS is creating a recordset object, then pDS is a data source object. pDS opens the data source via path name. objRSPS obtains the data set's records by referencing what layer you want from the axf database (axf is a compact sql server).

Note: the axf is picky, and it will tell you. If any domain values are invalid, it will catch it. So keep your databases clean if you are going to use this stuff.

Note 2: the shapefile is going away in arcpad, so get use to the AXF way of doing things.

Enjoy

Deprecation Plan for ArcGIS 9.3.1 and ArcGIS 9.4

The deprecation plans for ArcGIS 9.3.1 and ArcGIS 9.4 can be found here. It shows that VBA and VB6 are finally going away and there are OS support changes planned also. It's time for people to move on to new technologies, so start getting ready to transition now.

Important Links to Bookmark for Geoprocessing Tool Development

Here are some important links that I think any developer should have in there back pocket, if they are going to use, publish, or create GP tools.

This is about Feature and Record sets.

This is about the input and output types that are supported when publishing services.

Serialize Anything!

When you start with threading, especially in AGX 900 sdk, you are going to need to translate your objects into strings. To do this, just use the two functions below.



private byte[] getByteArrayWithObject( Object o )
{
/*

1) Create a new MemoryStream class with the CanWrite property set to true
(should be by default, using the default constructor).

2) Create a new instance of the BinaryFormatter class.

3) Pass the MemoryStream instance and your object to be serialized to the
Serialize method of the BinaryFormatter class.

4) Call the ToArray method on the MemoryStream class to get a byte array
with the serialized data.

*/


MemoryStream ms = new MemoryStream();
BinaryFormatter bf1 = new BinaryFormatter();
bf1.Serialize( ms, o );
return ms.ToArray();
}
private object getObjectWithByteArray( byte[] theByteArray )
{
MemoryStream ms = new MemoryStream( theByteArray );
BinaryFormatter bf1 = new BinaryFormatter();
ms.Position = 0;

return bf1.Deserialize( ms );
}



You can then translate your byte[] into a string using the built in Convert with .Net. Now you have values passed into the AGX BackGroundThread.

Enjoy