Wednesday, June 20, 2012

Working with BLOB data at 10.1 (arcpy.da)

Reading and writing BLOB data has always been an issue previous to 10.1, but now it's not a problem.  A reader pointed out that fact the other day, so I thought why not point this out to everyone.
From the help (source: resourcebeta.arcgis.com):
A BLOB is data stored as a long sequence of binary numbers. ArcGIS stores annotation and dimensions as BLOBs, and items such as images, multimedia, or bits of code can be stored in this type of field. You can use a cursor to load or view the contents of a BLOB field.
In Python, BLOB fields can accept strings, bytearray, and memoryviews. When reading BLOB fields, a memoryview object is returned.
So what does this mean to you.  We'll it's time to store those documents, pictures from vacation, and videos with geo tags in a spatial database.

Writing Blobs :
from arcpy import da

myFile = open(r"c:\temp\image.jpg",'rb').read()

with da.InsertCursor(r"c:\temp\demo.gdb\table",['blobFieldname','fileName']) as cursor:
   cursor.insertRow([myFile,'image.jpg'])


Reading Blobs:

from arcpy import da
import os
with da.SearchCursor(r"c:\temp\demo.gdb\table",['blobFieldname','fileName']) as cursor:
   for row in cursor:
      binaryRep = row[0]
      fileName = row[1]
      # save to disk
      open(r"c:\saveFolder" + os.sep + fileName, 'wb').write(binaryRep.tobytes())
      del row
      del binaryRep
      del fileName



Writing and reading Blob fields involves using the python built in open().  This function can open up files in various modes, but ensure that the method used are the binary modes 'rb' (read binary) and 'wb' (write binary), or else this process will not work.  You can essentially open up any file as a Blob, so stick whatever you want in there, but realize that this can balloon the size of your database up greatly.  It does however open up a whole new host of ways of transporting file based data.  Off the top of my head, I can image how this could improve data transport for local parallel processing methods of spatial and non-spatial data.

Enjoy

12 comments:

Anonymous said...

Hello,
Could You please help me with this script. I', inserting it (path and file names modification) to ArcGIS -> Geoprocessing -> Python but it isn't working....

Andrew said...

please example your issue better

Anonymous said...

Hello,

I have Multipoint feature type called 'single' in file geodatabase (called base.gdb in 'LIDAR' folder). In multipoint I have two fields ('Class' and 'Intensity') as a blob. I want to read values from that fields.
In ArcMap's Python Windows I put that code:

import arcpy
from arcpy import da
import os
with da.SearchCursor(r"c:\LIDAR\base.gdb",['Class','single']) as cursor:
for row in cursor:
binaryRep = row[0]
single = row[1]
# save to disk
open(r"c:\RESULT" + os.sep + single2, 'wb').write(binaryRep.tobytes())
del row
del binaryRep
del single

In results I get error:

Runtime error
Traceback (most recent call last):
File "", line 4, in
RuntimeError: cannot open 'c:\LIDAR\base.gdb'

Patch is good for sure.
Could you help me with that script so I could read my multipoint's attribute? Thanks in advance. I can't programm in Python so I'm like a baby in the woods with that :)

Andrew said...

You perform search cursors on a table not the database itself. Therefore you need to change the path from "c:\LIDAR\base.gdb" to r"c:\LIDAR\base.gdb\FeatureClassORTableName"

Anonymous said...

I did that and I'm getting that now:

Runtime error
Traceback (most recent call last):
File "", line 5, in
RuntimeError: A column was specified that does not exist.

Column name >Class< is also good

ed said...

hi, i've this query...

from arcpy import da
import os
with da.SearchCursor(r"c:\pruebas\map books.mdb\mapas",['DESCRIPCION','imageblob']) as cursor:
for row in cursor:
fileName = row[0]
binaryRep = row[1]
# save to disk
open(r"c:\pruebas\tmp" + os.sep + fileName, 'wb').write(binaryRep.tobytes())
del row
del binaryRep
del fileName

-------
but the file mapa.png or mapa.mxt is empty. Help please

Bryan said...

Hi Andrew! Great post! Does this same technique work with UpdateCursor? I'm trying to update existing records that already contain empty blob fields. Is this possible or is InsertCursor the only way to go? Thanks in advance

Andrew said...

You can use an updateCursor to insert you blob data into an existing row. You'll have to use a convert your data into a binary object I believe. So you can do something like obj = open(, 'rb').read() then row[0] = obj then you need to update your row using the updateRow().

I left a lot out, but that should get you on the right path.

Timothy Michael said...

Andrew,

Great post! I was linked here from an Esri forum post and your method to export worked perfectly.

Thanks,
Tim

Anonymous said...

Hi Andrew,

when running your modified code:
>>> from arcpy import da
... import os
... with da.SearchCursor(r"c:\GIS\BLOB\DATAB.mdb\export",['ANOTACE','test']) as cursor:
... for row in cursor:
... binaryRep = row[0]
... test = row[1]
... # save to disk
... open(r"c:\GIS\BLOB" + os.sep + test, 'wb').write(binaryRep.tobytes())
... del row
... del binaryRep
... del test

I am getting:

Runtime error
Traceback (most recent call last):
File "", line 4, in
RuntimeError: Too few parameters. Expected 1.

I have spent over 3 hours trying and searching for solution but with no success.

Do you have any idea what is wrong there?

Thank you,
Joseph
dachen84@yahoo.com

Andrew said...

I think the issue is that you are trying to use personal geodatabase. See if you get the same error if you put your data in a file geodatabase.

Anonymous said...

Super helpful! Glad I found this snippet. Thanks!!

-Jeff