Searching for User Content on AGOL Using ArcREST

ArcGIS Online (AGOL) and Portal organizations are great, but managing the whole organization's content can be tough.  Here is a brief example on how you could search an site (in our case AGOL) and look at what a given user has shared publicly.

Searching is fairly straight forward, and do not require any security logins to perform.  You can see this by just going to www.arcgis.com and typing something into the search box.  For this example though, I am going to use login credentials just in case I needed to perform an administrative task down the line.

if __name__ == "__main__":
    username = "MY USERNAME"
    password = "MY PASSWORD"
    url = "http://www.arcgis.com/sharing"
    proxy_url = None#"127.0.0.1" # for fiddler
    proxy_port = None#"8888" # for fiddler
    q2 = "owner:esri"
    securityHandler = arcrest.AGOLTokenSecurityHandler(username,
                                                       password,
                                                       proxy_port=proxy_port,
                                                       proxy_url=proxy_url)    
    admin = arcrest.manageagol.Administration(url=url, 
                                              securityHandler=securityHandler,
                                              proxy_url=proxy_url,
                                              proxy_port=proxy_port)    
    res = admin.query_without_creds( q=q2, start=1, num=1) # Want to find only public item on owner:esri
    items = []
    total = int(res['total'])
    steps = int(total / 100)
    if (total % 100) > 0:
        steps += 1
    step = 1
    while step <= steps:
        res = admin.query_without_creds( q=q2, start= 1 + ((step-1)*100), num=100)
        for r in res['results']:
            print r # do something with item!
            del r
        del res
        step += 1    

The item is returned from the query_without_creds() as a dictionary.  It contains information about each item that is unique to that item.

The script isn't doing much with the items that it finds, but it does show how you can loop through the owner's items (esri).  If you could administer the owner's item, you could change those public items to private.

To get a better sense of what you cannot and can query, and how the syntax works, I would check out this (http://doc.arcgis.com/en/arcgis-online/reference/search.htm).  It shows the advanced search syntax in great detail.

You can get ArcREST here (www.github.com/esri/ArcREST) if you do not have it.

Happy searching!

Using ArcREST to Export Hosted Feature Services

THIS POST IS FOR RELEASE ARCREST 2.0 AND WILL NOT WORK PERFECTLY FOR ARCREST RELEASE 3.0.

When you upload a feature class to ArcGIS Online (AGOL), you have the option to publish those feature classes. Now let's assume you have enabled editing on that data set. The question is how do you pull down the data so you can have an updated version in office? Well luckily for us, you have ArcREST to automate this task! ArcREST can be found here (www.github.com/Esri/ArcREST) if you do not have it.

This example will show how to automate the download of a hosted feature class as a shapefile to a centralized storage location.

The workflow is as follows:

1. Connect to the AGOL site
2. Export the item to a shapefile
3. Download the item to disk
4. Erase the exported item

import arcrest
import uuid
import os
import time
#   Inputs
#
itemID = "THE ITEM ID TO EXPORT"
username = "SOME USERNAME"
pw = "MY PASSWORD"
url = "http://www.arcgis.com/sharing"
filePath = r"c:\temp"
#   Logic
#
#  Create security handler
shAGOL = arcrest.AGOLTokenSecurityHandler(username, pw)
#  Connect to AGOL
org = arcrest.manageagol.Administration(url=url, securityHandler=shAGOL)
# Grab the user's content (items)
content = org.content  
usercontent = content.usercontent(username=username)
#  Create a export item with a random name using UUID
#  and export it to a zipfile
#
result =  usercontent.exportItem(title="%s" % uuid.uuid4().get_hex(),
                                     itemId=itemID,
                                     exportFormat=exportDataAs,
                                     exportParameters=None)

exportedItemId = result['exportItemId']
jobId = result['jobId']
exportItem = content.item(itemId=exportedItemId)
#   Ensure the item is finished exporting before downloading
#
status =  usercontent.status(itemId=exportedItemId, jobId=jobId, jobType="export")
    
while status['status'].lower() == "processing":
    time.sleep(3)
    status =  usercontent.status(itemId=exportedItemId,
                                 jobId=jobId,
                                 jobType="export")
filePath = exportItem.itemData(f="json", savePath=filePath)
#   Erase the exported item to clean up
#   AGOL
#
usercontent.deleteItems(items=exportItem.id)
del exportItem
print 'finishe!'

So what we have is a quick little script that downloads a file to the temp folder.  The key thing to look at is the while part of the code.  When perform actions like publish, generateFeatures, export, and createService, they are performed asynchronously on the system.  This means that you need to check if they are finished before performing further actions on the items.  The status method returns a dictionary of values, and you need to ensure that the value is not equal to 'processing', or any further actions like itemData() will return invalid information or data.  In the case of the itemData() you will download an empty zip file.

Enjoy and happy coding.