Showing posts with label ArcGIS Server. Show all posts
Showing posts with label ArcGIS Server. Show all posts

Friday, May 22, 2015

ArcREST - New Geoprocessing Model

Today ArcREST has a geoprocessing (GP) update.  The changes are minor, but the results are big.  The GP objects has been streamline so all the properties are the same.

Example of GPObject in Action:

import arcrest
# convert a feature class to a GPFeatureRecordSet
gpfrs = arcrest.ags.GPFeatureRecordSetLayer.fromFeatureClass(r"c:\temp\grid.gdb\sample", paramName="parameterName on GP tool")
The above code shows converting a feature class into a GPFeatureRecordSet object that can be given to a task that asks for that type of parameter.

Working with GP services is easy as well.

Example:

import arcrest
if __name__ == "__main__":
    sh = arcrest.AGSTokenSecurityHandler(username="user",
                                         password="password",
                                         token_url="http://site:6080/arcgis/admin/generateToken"
                                         )
    url = "http://site:6080/arcgis/rest/services/gp/scriptmv/GPServer"
    gp = arcrest.ags.GPService(url=url,
                               securityHandler=sh)
    for task in gp.tasks:
        if task.name.lower() == "":
            # submit the job
            job = task.submitJob(inputs=None)
            # wait till the job finishes
            while job.jobStatus != "esriJobSucceeded": pass
            # get the job results
            results = job.results # do something here.

Here we accessed a GP service and found a tool we wanted to run.  Since we are running the process async, submitJob() was used.  The function has additional options, like submitting the result via POST verse GET, etc...

Enjoy

Wednesday, April 22, 2015

ArcREST Basics - Authentication

When starting working with any new package, there is always a learning curve, and I am trying to make ArcREST easy and fun to use.   So here is my first of many ArcREST posts on how to use this package to meet your Esri REST API use cases.

This post will focus on the most basic thing, authentication.  Authentication is the most basic idea to understand when working with ArcGIS Online, ArcGIS Server, or ArcGIS Portal.  It identifies who you are, and what you are allowed to do.  Not all operations require authentication, but it's good to understand the token based security model and how it applies to you ArcGIS stack.

A quick blurb from the ArcGIS help describes what token security does:
ArcGIS Server provides a proprietary token-based authentication mechanism where users can authenticate themselves by providing a token instead of a user name and password. An ArcGIS token is a string of encrypted information that contains the user's name, the token expiration time, and some proprietary information. To obtain a token, a user provides a valid user name and password. ArcGIS Server verifies the supplied credentials and issues a token. The user presents this token whenever accessing a secured resource.
Before ArcREST, I you probably had to write this function out a lot:


import urllib
import urllib2
import httplib
import time
import json
import contextlib

def submit_request(request):
    """ Returns the response from an HTTP request in json format."""
    with contextlib.closing(urllib2.urlopen(request)) as response:
        job_info = json.load(response)
        return job_info

def get_token(portal_url, username, password):
    """ Returns an authentication token for use in ArcGIS Online."""

    # Set the username and password parameters before
    #  getting the token. 
    #
    params = {"username": username,
              "password": password,
              "referer": "http://www.arcgis.com",
              "f": "json"}

    token_url = "{}/generateToken".format(portal_url)
    request = urllib2.Request(token_url, urllib.urlencode(params))
    token_response = submit_request(request)
    if "token" in token_response:
        print("Getting token...")
        token = token_response.get("token")
        return token
    else:
        # Request for token must be made through HTTPS.
        #
        if "error" in token_response:
            error_mess = token_response.get("error", {}).get("message")
            if "This request needs to be made over https." in error_mess:
                token_url = token_url.replace("http://", "https://")
                token = get_token(token_url, username, password)
                return token
            else:
                raise Exception("Portal error: {} ".format(error_mess))

It is lots of code just to gain access.  As a user/developer you have not even begun to work on your actual workflow of what you need.

So there has to be an easier way? 

Well there is!  ArcREST supports three main forms of token based security in the form of classes.

  • PortalTokenSecurityHandler - this allows access to ArcGIS Portal Site
  • AGOLTokenSecurityHandler - this generates a token for ArcGIS Online
  • AGSTokenSecurityHandler - this generates a token for ArcGIS Server Sites
Why should I use this? 
  1. allows for cleaner code
  2. pass username/password once in your code
  3. built in proxy support 
  4. handles token expiration automatically
#3 is a biggie, at least in my eyes.  For long running tasks, there is a chance, depending on your expiration time, that your token might fail you because it is too old.  The SecurityHandler classes automatically handle the expiration for you, and regenerate new tokens as needed.

Examples of creating Securityhandler objects:

1. Creating an ArcGIS Server Security Handler
import arcrest
if __name__ == "__main__":
    token_url = "http://mysite.com:6080/arcgis/admin/generateToken"
    username = "username"
    password = "password"
    sh = arcrest.AGSTokenSecurityHandler(username=username,
                                         password=password,
                                         token_url=token_url)

2. Creating an ArcGIS Online Security Handler
import arcrest
if __name__ == "__main__":
    username = "< username >"
    pw = "< password >"
    proxy_url = None
    proxy_port = None
    sh = arcrest.AGOLTokenSecurityHandler(username, password=pw)

3. Creating an ArcGIS for Portal Security Handler
import arcrest
if __name__ == "__main__":
    username = "< username >"
    pw = "< password >"
    tokenUrl = "https://mysite.com/portal/sharing/rest/generateToken"
    org_url = "https://mysite.com/portal/sharing/rest"
    sh = arcrest.PortalTokenSecurityHandler(username=username, 
                                            password=pw, 
                                            org_url=org_url, 
                                            token_url=tokenUrl)

In each example, it shows how to generate the token handlers.  You then can pass the security handlers objects onto other functions like arcrest.manageorg.Administration(), which will allow administrators to perform operations to manage a portal or AGOL site.

Hope this helps!


Please download ArcREST here (http://www.github.com/Esri/ArcREST)
If you use ArcREST, help us out and post comments in the issues sections.  We are always trying to make it better.  If you think you can make it better, fork it and submit a pull request!


Tuesday, February 26, 2013

ArcGIS Server and Spatial Analyst

In ArcGIS 10.0, you needed to worry about path length for raster operations if the raster path plus file name is over 154 characters.  Now at 10.1, the character length has been increased to 254.  This means for most raster operations you will not have to alter the TEMP variable for the ArcGIS account.

If you do run into issues where spatial analyst or 3D analyst tools fail on server, but work on desktop in your model, try changing the ArcGIS account's TEMP environmental parameter to a different folder.

A good example would be c:\serverwrksp.  You also should make sure that the ArcGIS account has read/write access to that folder.


To shorten or change the path that server uses, you need to modify the ArcGIS account's TEMP variable.  Server should then honor the new destination path.  You might have to either log in and log off or reboot the box to get the OS to honor the new variable.

Here is how you change the variable in Windows:
To view or change environment variables:
  1. Right-click My Computer, and then click Properties.
  2. Click the Advanced tab.
  3. Click Environment variables.
  4. Click one the following options, for either a user or a system variable:
    • Click New to add a new variable name and value.
    • Click an existing variable, and then click Edit to change its name or value.
    • Click an existing variable, and then click Delete to remove it.
Please support my idea of having a python GUI by voting it up here.

Enjoy

Thursday, February 21, 2013

Query Feature Service By Object IDs (Python)

To query a feature service using object ids, you need to perform a POST.  For those who don't know the difference between a "GET" and "POST" here is a quick blurb based on the HTML specifications:
"POST" means that former means that form data is to be encoded (by a browser) into a URL while the "GET" means that the form data is to appear within a message body. 

Even simpler, "GET" is basically for just getting data where a "POST" may involve anything, like updating data, sending, or creating data.

Using python, you can perform both "GET" and "POST" methods, but since we need a "POST" to query by IDs, here is a simple example.  Please note that most feature services limit 1000 features being returned, so if you want to grab all of the features from a feature service, you'll have to perform multiple queries to get all the features back in JSON format.

Example:


import urllib2
import urllib
import urlparse
import httplib

def query_by_objectid(url, objectIDStart=0, objectIDEnd=1001):
    """ performs a POST operation where the query is called using
        the object id method.  If a feature service has more than
        1000 records, use this method to get a range of features
        from the feature service.

        Inputs:
           :url:  - string of feature service URL
           :objectIDStart: - integer of the start whole number
           :objectIDEnd: - end range whole number
        Returns:
           returns string JSON of query
    """
    url = url + '/query'
    start = int(objectIDStart)
    end = int(objectIDEnd)

    objectIDs = ",".join([str(x) for x in range(start, end)])
    headers = {"Content-type": "application/x-www-form-urlencoded",
               "Accept": "text/plain"}
    parameters = {'objectIds' : objectIDs,
                  'f' : 'json'}
    urlparams = urllib.urlencode(parameters)
    parts = urlparse.urlparse(url)
    h = httplib.HTTPConnection(parts.netloc)
    headers = {"Content-type": "application/x-www-form-urlencoded", "Accept": "text/plain"}
    h.request('POST', parts.path, urlparams, headers)
    r = h.getresponse()
    return r.read()

if __name__ == "__main__":
    url = 'http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/TaxParcel/AssessorsLiveLayers/MapServer/1'
    print query_by_objectid(url=url,
                            objectIDStart=0,
                            objectIDEnd=5)

So we have a simple "POST" example. This sample should work with both ArcGIS Server 10.1 and ArcGIS Online feature services.

Please support my idea of having a python GUI by voting it up here.

Enjoy