REST
Now when you type in a new data object name, it will auto-suggest available GET endpoints on the service (type a "/").
Using the /pet/{petId} data object, the showed filter fields would be available, and the display fields will be automatically determined.
Authentication Options
Methods that use authentication tokens
Authentication tokens are cached for up to one day or until the token has expired, whichever comes first.
Basic Auth
Standard Authentication (HTTP header with the user and pass encoded in base64) is achieved by providing username and password under credentials.
Windows Authentication
Windows Authentication (Active Directory) has to be enabled in SmartIQ instance.
Access Token (OpenID Connect & SAML)
When SmartIQ is setup to use OpenID Connect authentication, the ResponseType can be configured to retrieve an access_token returned from the IDP. This access_token can be passed to external APIs as authentication as the current user.
OAuth 2.0 Client Credentials
The REST Action and REST Data Connection supports OAuth 2.0 Client Credentials Grant type. This allows the entry of the Client Id, Client Secret and Scope.
The Client Credentials grant is used when applications request an access token to access their own resources, not on behalf of a user. SmartIQ will automatically retrieve the access token from the token endpoint and cache it until expiry.
Each request to the target REST endpoint will include the access token from the cache (or retrieve a new one) in requests as a bearer token header.
Provide the following global configuration in Manage > Settings > REST:
- Default OAuth 2.0 Client Id
- Default OAuth 2.0 Client Secret
- Default OAuth 2.0 Scope
- Default OAuth 2.0 Token Url
The global Token Url can be overridden per datasource in the Connection Attributes or within the security Definitions section of the swagger or the openapi spec.
- In the Connection Attributes the Token Url is specified with the parameter oauthTokenUrl eg:
oauthTokenUrl=<https://mytokenurl> - In the Definitions section of the swagger or openapi spec the parameter clientCredentials should be set to the token url.
OAuth 2.0 Resource Owner Password Credentials
The REST Action, REST Escalation and REST Data Connection now support OAuth 2.0 Resource Owner Password Credentials Grant type.
Important
Use this function only as a last resort and only if more secure flows are not possible. This flow requires a very high degree of trust in the application, and carries risks which are not present in other flows as the resource owner’s password is exposed to the application.
The Resource Owner Password grant is used when trusted applications requests an access token to access their own resources, not on behalf of a user. SmartIQ will automatically retrieve the access token from the token endpoint and cache it until expiry.
Each request to the target REST endpoint will include the access token from the cache (or retrieve a new one) in requests as a bearer token.
If a username input on an Action of Escalation has a value then resource owner flow will be used.
Provide the following global configuration in Manage > Settings > REST:
- Default OAuth 2.0 Client Id
- Default OAuth 2.0 Client Secret
- Default OAuth 2.0 Scope
- Default OAuth 2.0 Token Url
- Default OAuth 2.0 Username
- Default OAuth 2.0 Password
The global Token Url can be overridden per datasource in the Connection Attributes or within the security Definitions section of the swagger or the openapi spec.
- In the Connection Attributes the Token Url is specified with the parameter oauthTokenUrl eg:
oauthTokenUrl=<https://mytokenurl> - In the Definitions section of the swagger or openapi spec the parameter password should be set to the token url.
No Credentials Required
Custom Headers can be used to specify Authorization Header values. See the section on using Custom Headers (below).
Custom Headers
Custom Headers can be used with any of the REST Data Connector authentication types. Header values can be supplied by the use of a custom header or within the connection string.
When creating a custom filter on a Data Object, names that are surrounded by square brackets are treated as headers and are added to the request.
For example, in Manage Data Object Edit click the Add Custom option and enter a name in brackets eg: [Authorization].
That value will appear in Design as a normal filter field. You can then type "Bearer [q1.AuthorizationToken]" in the filter field in design where [q1.AuthorizationToken], is a reference to a call that creates the actual token.
At runtime SmartIQ will pull out any filter names in square brackets and add them as a header.
The same pattern applies for connection string values. A connection of "schema=...;[Authorization]=Bearer bf17bd59-dc64-44e0-8f96-c175b27a6822" will be sent with a header of "Authorization: Bearer bf17bd59-dc64-44e0-8f96-c175b27a6822".
Connection URL Overrides
The swagger is always exported if it exists in the source environment. The swagger also gets overwritten if it exists in the package being imported.
The base URL for the REST connection can be defined in both an uploaded schema and in the Connection Attributes of the data source. As the schema can be exported between environments, the Connection Attributes definition is used as an environment-specific override of the schema value. If used, the "Export Connection Attributes" option should be left unchecked to avoid overrriding this value and the target value if present in the package being imported.
GET Method
The HTTP GET method is used to "read" (or retrieve) a representation of a resource. In the non-error path, GET returns a representation in XML or JSON and an HTTP response code of 200 (OK). In an error case, it most often returns a 404 (NOT FOUND) or 400 (BAD REQUEST).
For this particular scenario, we will use an open API provided by Open Weather Map via Current weather data.
By City Name
This method will return current weather conditions for a particular city, taking one or two parameters: by city name or city name and country code.
api.openweathermap.org/data/2.5/weather?q={city name}
Example:
api.openweathermap.org/data/2.5/weather?q=Canberra
Or
api.openweathermap.org/data/2.5/weather?q={city name},{country code}
Example:
api.openweathermap.org/data/2.5/weather?q=Canberra,au
Sandbox
To use Open Weather Map API, you'll need to create a sandbox and generate an API Key.
Similarly, as defining a JSON Data Source JSON, it is required to specify a JSON Schema in order to map Display fields.
How to create a JSON Schema?
The easiest way to generate a JSON schema is taking a sample JSON response from API and create a schema based on it.
There are a good number of online free tools and Integrated Development Environment (IDE) add-on.
SmartIQ Datasource Configuration
Navigate to SmartIQ Manage > Data Connection.
- Create a local copy of service schema and store it in a location accessible by SmartIQ Web App Server.
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json",
"properties": {
"base": {
"default": "stations",
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/base",
"title": "The Base schema.",
"type": "string"
},
"clouds": {
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/clouds",
"properties": {
"all": {
"default": 75,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/clouds/all",
"title": "The All schema.",
"type": "integer"
}
},
"type": "object"
},
"cod": {
"default": 200,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/cod",
"title": "The Cod schema.",
"type": "integer"
},
"coord": {
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/coord",
"properties": {
"lat": {
"default": -35.28,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/coord/lat",
"title": "The Lat schema.",
"type": "number"
},
"lon": {
"default": 149.13,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/coord/lon",
"title": "The Lon schema.",
"type": "number"
}
},
"type": "object"
},
"dt": {
"default": 1490322600,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/dt",
"title": "The Dt schema.",
"type": "integer"
},
"id": {
"default": 2172517,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/id",
"title": "The Id schema.",
"type": "integer"
},
"main": {
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/main",
"properties": {
"humidity": {
"default": 60,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/main/humidity",
"title": "The Humidity schema.",
"type": "integer"
},
"pressure": {
"default": 1021,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/main/pressure",
"title": "The Pressure schema.",
"type": "integer"
},
"temp": {
"default": 21.3,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/main/temp",
"title": "The Temp schema.",
"type": "number"
},
"temp_max": {
"default": 22,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/main/temp_max",
"title": "The Temp_max schema.",
"type": "integer"
},
"temp_min": {
"default": 20,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/main/temp_min",
"title": "The Temp_min schema.",
"type": "integer"
}
},
"type": "object"
},
"name": {
"default": "Canberra",
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/name",
"title": "The Name schema.",
"type": "string"
},
"sys": {
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/sys",
"properties": {
"country": {
"default": "AU",
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/sys/country",
"title": "The Country schema.",
"type": "string"
},
"id": {
"default": 8226,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/sys/id",
"title": "The Id schema.",
"type": "integer"
},
"message": {
"default": 0.0064,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/sys/message",
"title": "The Message schema.",
"type": "number"
},
"sunrise": {
"default": 1490299782,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/sys/sunrise",
"title": "The Sunrise schema.",
"type": "integer"
},
"sunset": {
"default": 1490342954,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/sys/sunset",
"title": "The Sunset schema.",
"type": "integer"
},
"type": {
"default": 1,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/sys/type",
"title": "The Type schema.",
"type": "integer"
}
},
"type": "object"
},
"visibility": {
"default": 10000,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/visibility",
"title": "The Visibility schema.",
"type": "integer"
},
"weather": {
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/weather",
"items": {
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/weather/0",
"properties": {
"description": {
"default": "broken clouds",
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/weather/0/description",
"title": "The Description schema.",
"type": "string"
},
"icon": {
"default": "04d",
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/weather/0/icon",
"title": "The Icon schema.",
"type": "string"
},
"id": {
"default": 803,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/weather/0/id",
"title": "The Id schema.",
"type": "integer"
},
"main": {
"default": "Clouds",
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/weather/0/main",
"title": "The Main schema.",
"type": "string"
}
},
"type": "object"
},
"type": "array"
},
"wind": {
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/wind",
"properties": {
"deg": {
"default": 110,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/wind/deg",
"title": "The Deg schema.",
"type": "integer"
},
"speed": {
"default": 4.1,
"description": "An explanation about the purpose of this instance.",
"id": "http://api.openweathermap.org/data/2.5/weather/root.json/wind/speed",
"title": "The Speed schema.",
"type": "number"
}
},
"type": "object"
}
},
"type": "object"
}
- Change Data Connection Name to REST GET Datasource.
- Select Connection Type as REST.
- Connection String requires two values:
- baseUrl = the first section of the service URL (eg baseUrl=http://server/)
- schema = the JSON schema definition of the result from the API call (schema=c:\temp\user.json)
baseurl=http://api.openweathermap.org;schema=C:\inetpub\wwwroot\SmartIQ\datasource\weatherService.schema.json
- Click Test Connection to validate that SmartIQ can establish a connection with JSON and Schema files.
- Click Save.
Good practice
It is recommended to enable "Allow Connection Export" as it will make the data source part of a project definition allowing to import it in a different environment.
After saving REST Data Connection in Manage, click the Data Objects button. This will bring a new window with all available Data Objects for this particular Data Connection.
The data object name is the rest of the API URL (e.g. api/{username}/invoice?id={invoiceid}). Key fields are extracted from the URL, template fields and display fields are read from the schema file (same as the JSON Data Connection JSON)
-
Click on New Data Object. Provide the following information:
Object Type: Method
Data Object Name / Definition: /data/2.5/weather?q={cityAndCode}&appid=yourAPIKey&units=metric
Display Name: CurrentByCity -
Add all filter fields clicking on Add All >> button.
-
Click the Save button.
http://api.openweathermap.org/data/2.5/weather?q={cityAndCode}&appid=yourAPIKey&units=metric
Simple Test
A straightforward way to test you have the correct endpoint, copy/paste the complete URL into a new tab in your browser, you should get a JSON response.
{
"coord": {
"lon": 149.13,
"lat": -35.28
},
"weather": [{
"id": 801,
"main": "Clouds",
"description": "few clouds",
"icon": "02d"
}],
"base": "stations",
"main": {
"temp": 11,
"pressure": 1020,
"humidity": 53,
"temp_min": 11,
"temp_max": 11
},
"visibility": 10000,
"wind": {
"speed": 1.5,
"deg": 310
},
"clouds": {
"all": 20
},
"dt": 1503363600,
"sys": {
"type": 1,
"id": 8226,
"message": 0.0115,
"country": "AU",
"sunrise": 1503347775,
"sunset": 1503387420
},
"id": 2172517,
"name": "Canberra",
"cod": 200
}
Post Method
The POST verb is most-often utilised to "create" new resources but also to retrieve data based on some parameters passed as in the body request. On successful, return HTTP status 201, returning a Location header with a link to the newly-created resource with the 201 HTTP status.
For this particular scenario, we will use Text Analytics API provided by Microsoft Azure Cognitive Services. The Text Analytics API is a suite of text analytics web services built with Azure Machine Learning. The API can be used to analyse unstructured text for tasks such as sentiment analysis, key phrase extraction and language detection. No training data is needed to use this API.
The Azure Text Analytics Quick Start provides all required information to configure this service in your Microsft Azure subscription.
Sandbox
To use Microsoft Azure Text Analytics, you'll need a Microsoft Azure account. (30 day trial with $200 free credit is available)
Note
Microsoft Azure Text Analytics provides a Swagger Definition that could be downloaded and used within SmartIQ containing request and response definitions for all methods (no need to create n number of Data Sources) however, in this example we will use JSON Schema to explain the difference between request and response.
POST method requires an extra parameter in the connection string pointing to the JSON Schema of the response.
- baseUrl = the first section of the service URL.
- schema = the JSON schema definition of the result from the API call.
- postSchema = the JSON schema definition of the request to the API.
- Create a new Data Connection using following information:
schema=C:\inetpub\wwwroot\SmartIQ\Datasource\TextAnalyticsLanguage.schema.json;postschema=C:\inetpub\wwwroot\SmartIQ\Datasource\TextAnalyticsLanguage.postschema.json;baseurl=https://westus.api.cognitive.microsoft.com
{
"type": "object",
"properties": {
"documents": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"description": "Unique document identifier.",
"type": "string"
},
"detectedLanguages": {
"description": "A list of extracted languages.",
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"description": "Long name of a detected language (e.g. English, French).",
"type": "string"
},
"iso6391Name": {
"description": "A two letter representation of the detected language according to the ISO 639-1 standard (e.g. en, fr).",
"type": "string"
},
"score": {
"format": "double",
"description": "A confidence score between 0 and 1. Scores close to 1 indicate 100% certainty that the identified language is true.",
"type": "number"
}
}
}
}
}
}
},
"errors": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"description": "Input document unique identifier the error refers to.",
"type": "string"
},
"message": {
"description": "Error message.",
"type": "string"
}
}
}
}
}
}
{
"type": "object",
"properties": {
"documents": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"description": "Unique, non-empty document identifier.",
"type": "string"
},
"text": {
"type": "string"
}
}
}
}
}
}
- Create a new Data Object, where:
Object Type: Method
Data Object Name / Definition: /text/analytics/v2.0/languages
Display Name: Languages
- Click the Save button.
Data Object is created with filter and display fields specified in schemas, however, for this particular example, an extra header for API Key requires to be defined.
- Click on Add Custom.
- Type: [Ocp-Apim-Subscription-Key] and click on Add.
An extra filter field will be added into this Data Object. Later Designer has to pass in the API Key to this custom header.
Array Data Filter
A REST data connector can use a repeat data filter to configure a JSON array.
- In your Data Object, add the following available fields as filter fields
- documents[]
- documents[].id
- documents[].text
- In Design, create a repeating section and within the repeating section, add two text fields called Id and Text.
- Below the repeating section, add your Datasource and select the Data Object.
- Add a repeat data filter
- Filter Field: $.documents[]
- Select Type: Repeat
- Repeat Source: Your repeating section
- We can now configure the array object property
- Select the repeat filter field created in the previous step
- Change Filter Field to $.documents[].id
- For comparison, select Equals
- Change Type to Parent Question
- Select the Id text field which is in the repeating section
- Repeat step 5 for the $documents[].text field
Your data filter tab should look like this:
Notice that the $.documents[].id and $.documents[].text fields are indented below the $.documents[] filter. This means that it will create a new object under each row of a repeating section. With 2 rows in a repeating section, your POST data will look something like this:
{
"documents":[
{
"id": "1",
"text": "First row"
},
{
"id": "2",
"text": "Second row"
}
]
}
Troubleshooting
An option "Enable advanced results display in Troubleshooting Mode" exists in the configuration for any REST data connector in Manage to allow the request and response for that data connector to be displayed in troubleshooting mode.
The option needs to be selected AND troubleshooting mode needs to be enabled for the project for this to appear.
Updated 23 days ago