Get our latest book on "Top 10 artificial intelligence myths."
Download

Last updated

March 4, 2021

Table of contents

Uploading files and writing data via Impira Write API

This guide will walk you through:

  • Uploading files and writing data via Impira Write API
  • Inserting and updating data
  • Deleting files or data
  • Using integrations (Amazon S3 and Dropbox)

We strive to offer every function in the UI through our API so you can mix-and-match the right combination of UI and API for your use. This guide is divided into multiple sections to walk you through each core feature of the API.

See Impira Read API to read files or data at a given path, or to query data via our Impira Query Language (IQL) API.

Find your organization name

Your organization's name will appear in the API end-point used to query your data. It'll be in your browser's URL bar, after `/o/`.

``` language="js"
https://app.impira.com/o/<YOUR_ORG_NAME>/collections
```

Create an API token

In order to call the API, you’ll need to create an API token (which can only be created by account owners).

  1. Click on the gear icon in the top-right corner. This will take you to the User Access page.

  1. Choose API from the top-left navigation menu.

  1.  Click the + symbol on the right side of the page and give your token a name, like "Demo."


  1.  Copy your token and keep it safe and private as you would a password.

API URL structure

The Impira API uses a common URL structure:

``` language="js"
https://app.impira.com/o/<YOUR_ORG_NAME>/api/v2/<path>
```

  • `<path>` specifies the resource you want to read or write to

The API supports different operations through the standard HTTP verbs GET, POST, PATCH, and DELETE. We’ll walk through how to use each of these verbs in depth in the sections that follow.

In most cases:

  • GET allows you to retrieve information about a collection, file, or other resource
  • POST allows you to upload files and insert new records
  • PATCH allows you to update existing records
  • DELETE allows you to delete resources

The following table walks through the different possible values of `<path>`:

You can use each of the GET, POST, PATCH, and DELETE verbs to read and write files and data at each of these paths.

Uploading files with POST

Uploading files directly into Impira is a breeze. Impira supports two methods to write files through the API:

  1. Submitting a POST request with a file URL and other metadata.
  2. Directly uploading a file as multipart/form-data through a POST request.

Uploading files with URLs

You can upload a file to a given resource (e.g., a collection) by submitting a POST request to the appropriate path. The request takes a JSON object which includes one or more entries to insert. Each entry includes the following information:

  • The File object contains the path to the file and optional metadata fields:
  • File.path is a URL to the file. The file must be accessible to Impira’s cloud-based servers. You can use an HTTP or HTTPS URL, or if your data is public, you can use an Amazon S3 URL.
  • File.name is an optional name for the file which appears in the UI. This name doesn't have to be unique. If you omit the name, the file will appear with an empty name in the UI.
  • If you're uploading to a collection, you can only specify values in the File field. If you'd like to specify values for other fields, you must do that in a follow up PATCH request (see below). You cannot specify ML model (inferred) fields through the API.
Example request

``` language="js"
curl -X POST "https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/collection/Applications" \
  --header "X-Access-Token: YOUR_TOKEN_HERE" \
  --header "Content-Type: application/json" \
  --data '
{
    "data": [
      {
        "File": {
        "path": "https://assets-global.website-files.com/5e3898dff5077817ea80d71c/6047b3733b9cbd22251c7ef8_Screen%20Shot%202021-03-09%20at%2012.42.02%20PM.png",
        "name": "application form 10"
      }
    },
    ...
  ]
}'
```

Example response

``` language="js"
{
  "uids": [
    "927a834f5f2036e6"
  ]
}
```

Uploading files directly

If you have a file on your local machine or server and want to upload it directly to Impira, you can simply upload it with a multi-part file form submission. The direct file upload API supports up to 10 MB per request, which you can spread across one or more files.

For each file you'd like to upload, you can submit a file form field. You can optionally specify a data field for each file, which is a JSON encoded value that contains any record values to include alongside the file object. Just like URL-based upload, you can only specify fields in the File object (e.g., name). You can edit any follow-up fields in a PATCH request.


Example request

```language=“js”
curl -X POST "https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/collection/Applications" \
  --header "X-Access-Token: YOUR_TOKEN_HERE" \
  --form "file=@local_file.pdf"
```

Example response

``` language="js"
{
  "uids": [
    "8a4dc36aa82bf85b"
  ]
}
```

Example request

``` language=“js”
curl -X POST https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/collection/Applications \
  --header 'X-Access-Token: YOUR_TOKEN_HERE' \
  --form 'file=@local_file1.pdf' \
  --form 'data={"File": {"name": "my_file_1.pdf"}}' \
  --form 'file=@local_file2.pdf' \
  --form 'data={"File": {"name": "my_file_2.pdf"}}' \
```

Example response

``` language="js"
{
  "uids": [
    "cb713868a5155ad8",
    "f1df198b1ebdef98"
  ]
}
```

Inserting data with POST

You can insert data into a dataset the same way as uploading a file. Unlike writing to /files or a collection, you don't need to include the File object when writing to a dataset. Instead, just specify the fields you want to write.

NOTE: These fields must exist on the dataset. You'll receive an error if you try to write data referencing fields that don't exist.


Example request

``` language=“js”
curl -X POST "https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/dc/337dd57d0c5948f9" \
  --header "X-Access-Token: YOUR_TOKEN_HERE" \
  --header "Content-Type: application/json" \
  --data '
{
  "data": [
  {
      "Name": "Rhyley Green"
      "Hired": true,
    },
    {
      "Name": "Ammar Duffy"
      "Hired": false,
    }
    ...
  ]
}'
```

Example response

``` language="js"
{
  "uids": [
      "360fe1982bb9055a",
      "b0694bc41fa6b241"
    ]
}
```

The uid field is a unique identifier for the record you insert. If you do not specify a uid in the request, then Impira will generate one for you. You can use this uid in future requests (or just specify your own) to perform an upsert, which will insert a new record if the uid is not already present or update the record if it is. For example, if the insert above returned a uid of "b0694bc41fa6b241" for "Ammar Duffy," then you can run this to update their Hired field to true:


Example request

``` language=“js”
curl -X POST "https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/dc/337dd57d0c5948f9" \
  --header "X-Access-Token: YOUR_TOKEN_HERE" \
  --header "Content-Type: application/json" \
  --data '
{
  "data": [
    {
      "uid": "b0694bc41fa6b241"
      "Name": "Ammar Duffy"
      "Hired": true,
    }
  ]
}'
```

Example response

``` language="js"
{
  "uids": [
    "b0694bc41fa6b241"
  ]
}
```

NOTE: Unlike a PATCH request which updates a row, you must specify every field in an upsert. For example, if you omit the name field from the above request (e.g., {"uid": "b0694bc41fa6b241," "Hired": true}), then the request will overwrite Name with an empty value.

Updating data

When you insert a new record using a POST request, you can either specify your own uid or Impira will generate one for you if not. You can later use this uid to reference the record and PATCH commands to update it. This method works across both collections and datasets.

For each record in a PATCH request, you must specify the record's uid as well as any fields you'd like to update. This example updates the file we uploaded earlier to set its Inspected state to true.


Example request

``` language=“js”
curl -X PATCH "https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/collection/Applications" \
  --header "X-Access-Token: YOUR_TOKEN_HERE" \
  --header "Content-Type: application/json" \
  --data '
{
      "data": [
      {
        "uid": "8a4dc36aa82bf85b"
        "Inspected": true,
      },
    ...
  ]
}'
```

Example response

``` language="js"
{
  "uids": [
    "8a4dc36aa82bf85b"
  ]
}
```

Just like a POST request, you can specify multiple records to update in a PATCH. Each record can even contain different fields that you'd like to update.

Deleting files or data

Use the DELETE verb to delete a record from the global set of files, a collection, or a dataset. The payload is the same format as the response from POST and PATCH requests (e.g., a list of uid values that you'd like to delete). For example, to delete the file above globally, you can run:


Example request

``` language=“js”
curl -X DELETE "https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/files" \
  --header "X-Access-Token: YOUR_TOKEN_HERE" \
  --header "Content-Type: application/json" \
  --data '
{
  "uids": [
    "8a4dc36aa82bf85b"
  ]
}'
```

Example response

``` language="js"
{
  "uids": [
    "8a4dc36aa82bf85b"
 ]
}
```

Deleting from a collection

Just as in the application, if you issue a DELETE against a collection, Impira will remove the file record from the collection but not delete it globally. If you'd like to delete a file globally, then delete it from `/files` as shown above. This will also remove the file from any collections it belongs to.

Using integrations

Impira allows you to upload files through integrations with Amazon S3 and Dropbox over the API. Using the Integration API, you can programmatically manage your existing connections as well as setup new connections to supported sources. Currently, you can create S3 connections through the API, and you can refresh data from any supported source. Some connections can only be created through the Impira application due to external permission requirements.

Currently, the supported sources are:

  • Amazon S3
  • Dropbox

Connection creation

To create a new S3 connection, send a `POST` request to the S3 endpoint with the fields `integration_name`, `s3_bucket`, and `s3_prefix` specified in the body of the request. If you wish to upload an entire S3 bucket to Impira, then you may omit the `s3_prefix` field.

```language="js"
{"integration_name": "API Example", "s3_bucket": "examples", "s3_prefix": "documents/2020-01-01"}
```

Upon creation, the connection will begin to fetch all objects located within the specified bucket, at or beneath the specified prefix (recursively). For example, provided `{"s3_bucket": "examples", "s3_prefix": "documents/2020-01-01"}`, Impira will ingest objects `s3://examples/2020-01-01/onboarding.pdf` and `s3://examples/2020-01-01/marketing/ads/general_campaign.pdf`, but not `s3://examples/2019-12-01/starter.pdf`. If you want to modify the contents under the specified prefix after the initial ingest, simply update the source bucket to the desired state and trigger a 'Refresh' action on your existing connection.

Example request

``` language=“js"
curl -X POST https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/integrations/s3/sync \
  --header "X-Access-Token: YOUR_TOKEN_HERE" \
  --header "Content-Type: application/json" \
  --data '{"integration_name": "API Example", "s3_bucket": "examples", "s3_prefix": "documents/2020-01-01"}'
```

Connection Refresh

To refresh an existing connection, send a `POST` request to the source specific refresh endpoint with the `integration_name` field specified in the body of the request.

The URL is formatted in terms of the integration type:

``` language=“js”
`https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/integrations/s3/refresh`
```

```language="js"
`https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/integrations/dropbox/refresh`
```

And you specify the integration to refresh as `{"integration_name": "API Example"}`.

Refresh is a way to update the ingested files within Impira to match the files stored at the source. Impira will issue a series of inserts, updates, and deletes corresponding to actions that have taken place on the source files.

Using the example above, if `s3://examples/2020-01-01/onboarding.pdf` had been updated with new onboarding instructions, `s3://examples/2020-01-01/marketing/ads/general_campaign.pdf` had been deleted, and `s3://examples/2020-01-01/marketing/ads/food_campaign.pdf` had been added, upon refresh, Impira would update the 'onboarding.pdf' file to reflect the new contents, delete the row corresponding to 'general_campaign.pdf', and insert a new row for 'food_campaign.pdf'.

Example request

```language="js"
curl -X POST https://app.impira.com:443/o/<YOUR_ORG_HERE>/api/v2/integrations/s3/refresh \
  --header "X-Access-Token: YOUR_TOKEN_HERE" \
  --header "Content-Type: application/json" \
  --data '{"integration_name": "API Example"}'
```