Overview

The Address Cleanse API is a web service that can be used to perform address validation, deduplication and suppression checks using a variety of data types.

It follows RESTful principles - accepting HTTP requests and returning JSON response documents. Unless otherwise stated all methods will return HTTP status 200 on success and a 4xx or 5xx status when an error occurs.

A Hopewiser online Address Cleanse Account is needed.

Click here for more detailed Address Cleanse API reference documentation

Quick Start

Create a configuration

  1. 1. Authentication
  2. 2. Job configuration

1. Authentication

All Address Cleanse API methods require a security token for authorization. This token is generated using the web interface at https://www.hopewiser.com/online-address-cleanse/?api=true

To generate a token go to https://www.hopewiser.com/online-address-cleanse/?api=true in a browser. Create an account if needed and then log in. Click on your username in the top right hand corner of the screen to open up a menu. On that menu select "Export API auth token", a window will apeear containing the token text. Store that text for use by your application.

The token should be sent in a HTTP header of the format:

Authorization: Bearer $TOKEN_VALUE

This should be added to all HTTP requests made to the API.

2. Job configuration

The recommended way to create a configuration for the Address Cleanse API is to configure the desired settings using the web interface at https://www.hopewiser.com/online-address-cleanse/?api=true.

Once a configuration has been created it can be exported for use with your client application.

Go to https://www.hopewiser.com/online-address-cleanse/?api=true in a browser. Log in and follow the instructions to upload a file and set up the run as desired.

Once the job is ready to run, click on the cog icon in the top right corner to bring up the file properties. There should be a button labelled "Export settings for API". Select that and a window will appear containing the JSON text describing your settings. Save that text for use by your application, it will be needed by the /api/v1/run method.

This only needs to be done once, the same settings can be used for any number of input files provided the files are all of the same format. i.e. Have the same delimiter, number of columns etc.

Upload an input file

  1. 1. Begin
  2. 2. Chunk
  3. 3. End

1. Begin

The first step in processing a file is to upload it. In order to handle large files the upload is split in to 3 steps.

First call the begin method, giving it a size and filePrefix (name).

/api/v1/files/upload/begin?size=XXX&filePrefix=NAME

Parameters:

Name Description Example Value
size The file size in bytes 16384
filePrefix A string that will become part of the filename on the server CustomerAddresses

Response:

{
  "fileId":"CustomerAddresses2984614951664834951.dat",
  "chunkUrl":"https://filesvr.hopewiser.com/filesvr/upload/chunk"
}

2. Chunk

Then call the chunk method using the chunkUrl and fileId returned by /begin and an idx indicating which chunk this is, starting from zero. The file data should be sent in the body of the HTTP POST request as raw binary data bytes.

https://filesvr.hopewiser.com/filesvr/upload/chunk?fileId=XXX&idx=0
https://filesvr.hopewiser.com/filesvr/upload/chunk?fileId=XXX&idx=1

etc.

Parameters:

Name Description Example Value
fileId ID returned by /begin CustomerAddresses1234
idx Index number for the chunk 0

It is recommended the chunks be no larger than 10 MiB especially if your application is browser based. Call this method repeatedly for each part of the file you upload, incrementing the idx parameter each time.

3. End

Once the file data is completely uploaded, call the end method with the fileId returned by /begin and optionally a hash field containing the sha256 hash of the file. If a hash is provided it will be compared with the file data stored on the server to guarantee the file's integrity.

/api/v1/files/upload/end?fileId=XXX&hash=YYY

Parameters:

Name Description Example Value
fileId ID returned by /begin CustomerAddresses1234
hash SHA256 hash of the file (optional) d785d959746f4180c8c6ef885d9abaccb5a43f004c718a906271c3de014eb69b

Retrieve a list of uploaded files

A list of uploaded files and their metadata can be retrieved using the following method. This method takes no parameters and returns a JSON array containing a description of each uploaded file.

/api/v1/files

Response:

[
  {
    "id": 147,
    "cols": 6,
    "rowCount": 597,
    "delimiter": ",",
    "hasHeader": true,
    "useQuotes": true,
    "creationDate": 1510939445,
    "inputFileDeleteDate": 1510939445,
    "lastUpdate": 1510939445
  }
]

The dates in the response are given using Unix time, which is the number of seconds that have elapsed since January 1, 1970 (midnight UTC/GMT), not counting leap seconds (in ISO 8601: 1970-01-01T00:00:00Z).

1. Start a cleanse job

To start a cleanse process on an uploaded file call the following method, giving it the inputFileId which was returned as the fileId response in the /upload methods, and a name to identify this job. The request body should contain the JSON settings data previously generated from the web interface and exported using "Export settings for API".

/api/v1/run?inputFileId=CustomerAddresses1234&name=CleanseRun1

Parameters:

Name Description Example Value
inputFileId ID returned by /begin CustomerAddresses1234
name A name to identify this run CleanseRun1

Body:

{
  "inputFileDesc": {
    "cols": 7,
    "delimiter": "|",
    "hasHeader": true,
    "useQuotes": true
  },
  "inputFields": [
  {
    "name": "Add Line 1",
    "mappingId": 6,
    "mappingOrder": 9
  },
  {
    "name": "Add Line 2",
    "mappingId": 6,
    "mappingOrder": 10
  },
  {
    "name": "Add Line 3",
    "mappingId": 6,
    "mappingOrder": 11
  },
  {
    "name": "Add Line 4",
    "mappingId": 6,
    "mappingOrder": 12
  },
  {
    "name": "Add Line 5",
    "mappingId": 6,
    "mappingOrder": 13
  },
  {
    "name": "City",
    "mappingId": 6,
    "mappingOrder": 14
  },
  {
    "name": "Country",
    "mappingId": 0,
    "mappingOrder": 15
  }
  }],
  "outputFields": [{
    "groupId": 1
  },
  {
    "groupId": 8
  },
  {
    "groupId": 2
  },
  {
    "groupId": 3
  },
  {
    "groupId": 4
  }],
  "dedupe": {
    "duplicationLevel": 2,
    "dedupeSeparateFile": false
  },
  "suppress": {
    "suppressDeceased": true,
    "suppressGoneaways": true,
    "suppressPreferenceServices": true
  }
}

Response:

{
  "runId": 49
}

2. Get the status of a job

Depending on the size of the file and the options selected the run could take only a few seconds or several minutes. To determine the current status of the job you can poll using the following method, giving it the runId returned from /run. This will return a block of JSON text containing the current status of the job. The returned HTTP status code will also change. While the job is running the returned HTTP status will be 404, once the job is complete it will become 200.

/api/v1/run/status?runId=49

Parameter:

Name Description Example Value
runId ID returned by /run 49
{
    "name": "CleanseRun1",
    "statusCode": 6,
    "statusDesc": "Processing",
    "stage": {
        "stageCount": 7,
        "currentStage": 5,
        "stageName": "Post processing",
        "stageStatus": "Postprocess",
        "stageProgress": 0.14285714285714286
    }
}

The stage part of the response is optional, it will only be there while the job is being processed. Once the cleanse part of the job is complete the response will only contain the name, statusCode and statusDesc fields.

Possible status codes

Value Description
0 Uploading
1 Uploaded
2 QueuedForTest
3 ProcessingTest
4 FinishedTest
5 QueuedForResults
6 ProcessingResults
7 AwaitingPayment
8 Paid
9 Error
10 Deleted

The number and values of the stages in the stage object are subject to change. The field names are fixed, but their values may be updated at any time as the software is upgraded.

For example, there are currently 7 stages, but that number may increase or decrease in future. If it does change the stageCount will update to reflect this.

1. Statistics

Once a job is complete various statistics and metadata is available to download. These can be viewed without payment. To download the results file however, it must first be payed for. All the results methods take as a parameter the runId returned by the /run method to identify which job you are interested in.

To get the number of matches for the various categories of checks made by Address Cleanse (address matches, de-duplications and suppressions):

/api/v1/results/statistics?runId=49

Parameter:

Name Description Example Value
runId ID returned by /run 49

Response:

{
    "recordCount": 76,
    "codingStats": [
        {
            "name": "Matched",
            "description": "A correct address",
            "count": 11
        },
        {
            "name": "UnmatchedPremise",
            "description": "The premise, e.g. house number, was not found",
            "count": 1
        },
        {
            "name": "UnmatchedStreet",
            "description": "The thoroughfare, e.g. street, was not found",
            "count": 1
        },
        {
            "name": "UnmatchedTown",
            "description": "The town was not found",
            "count": 62
        },
        {
            "name": "Unmatched",
            "description": "No parts of the address could be verified",
            "count": 1
        }
    ],
    "dedupeStats": [
        {
            "name": "Unique",
            "description": "The record is unique",
            "count": 98
        },
        {
            "name": "Parent",
            "description": "The first record that is a duplicate based on the level selected",
            "count": 1
        },
        {
            "name": "Child",
            "description": "The record is a duplicate of a parent record",
            "count": 1
        }
    ],
    "suppressionRemovedStats": [
        {
            "name": "Goneaway",
            "description": "People who have moved",
            "typeName": "DBS",
            "typeDescription": "Purifi",
            "count": 11
        },
        {
            "name": "Goneaway",
            "description": "People who have moved",
            "typeName": "DCR",
            "typeDescription": "DisConnect",
            "count": 3
        },
        {
            "name": "Goneaway",
            "description": "People who have moved",
            "typeName": "GAS",
            "typeDescription": "Goneaway Suppression",
            "count": 1
        },
        {
            "name": "Goneaway",
            "description": "People who have moved",
            "typeName": "RMG",
            "typeDescription": "Re-Movers",
            "count": 7
        },
        {
            "name": "Preference",
            "description": "People listed on Mailing Preference Service",
            "typeName": "MPS",
            "typeDescription": "Mailing Preference Service",
            "count": 1
        }
    ],
    "suppressionFlaggedStats": []
}

2. Costs

To see a breakdown of the various charges incurred for the run:

/api/v1/results/costs?runId=49

Parameter:

Name Description Example Value
runId ID returned by /run 49

Response:

[
    {
        "name": "Address Validation",
        "netcost": 3072,
        "cost": 3610
    },
    {
        "name": "De-duplication",
        "netcost": 43,
        "cost": 50
    },
    {
        "name": "Telephone Preference Service",
        "netcost": 7,
        "cost": 8
    },
    {
        "name": "Goneaways",
        "netcost": 299,
        "cost": 351
    },
    {
        "name": "Preference Services",
        "netcost": 1,
        "cost": 1
    }
]

3. Pay

To allow the file to be downloaded use the payment method.

/api/v1/results/pay?runId=49

Parameter:

Name Description Example Value
runId ID returned by /run 49

If you have not set up an agreement with Hopewiser to allow automatic payments, this will return a URL you must manually visit in a browser to complete your payment information.

If you have set up such an agreement then you must still call the payment API method, but no further action is needed.

Response:

{
  "invoiceUrl": "http://www.hopewiser.com/payment/xxxx"
}
{
  "invoiceId": "196"
}

4. Download results file

Finally, to retrieve the results, call the following method, which will return a url from which a zip file containing results and statistics can be downloaded.

/api/v1/results/file?runId=49

Parameter:

Name Description Example Value
runId ID returned by /run 49

Response:

{
  "downloadUrl": "https://filesvr.hopewiser.com/filesvr/download?downloadName=AddressCleanseResults.zip&fileId=CleanseRun1234.dat"
}

Version

Retrieve the version of Address Cleanse software

/api/v1/version

Response:

{
    "major": 1,
    "minor": 1,
    "patch": 0,
    "description": "1.1.0"
}