Skip to main content
POST
/
v1
/
http
/
execute-request
{
  "jobs": [
    {}
  ],
  "data_object_id": "<string>",
  "400 Bad Request": {},
  "500 Internal Server Error": {}
}
This endpoint executes an HTTP request to an external API, creates data objects for both the request configuration and response data, establishes data lineage between them, and starts background jobs to analyze and prepare the data.

​
Request Body

​
project_id
string
required
ID of the project to associate with the request
​
datasource_id
string
required
ID of the datasource created for this HTTP connection
​
connect_spec
object
required
Connection specification object containing configuration details
​
url
string
required
The URL of the API endpoint
​
method
string
required
The HTTP method to use
​
headers
object
HTTP headers to include with the request
​
parameters
object
Query parameters with values and required flags
​
parameterGroups
array
Groups of related parameters
​
authType
string
Authentication type
​
auth
object
Authentication details
​
responsePathExtractor
string
JSON path to extract a specific key from the response to use as the main data source. For example, “value” will extract data from the “value” field of the response.

​
Response

​
jobs
array
Array of job objects created for processing the data
​
data_object_id
string
ID of the data object created for the response data

​
Request Example

{
  "url": "https://api.weather.com/v3/location/search",
  "method": "GET",
  "headers": {},
  "parameters": {
    "query": {
      "value": "London",
      "required": true
    },
    "apiKey": {
      "value": "abcdef123456",
      "required": true
    }
  },
  "parameterGroups": [],
  "authType": "None",
  "auth": {},
  "responsePathExtractor": "value",
  "project_id": "9fd3cedc-d303-4870-9635-c89e60c3f1b0",
  "datasource_id": "f4cfd57d-dcf0-4039-ab31-3e2e7178189e",
  "connect_spec": {
    "name": "Weather API Connection",
    "id": "http-requests",
    "config": {
      "url": "https://api.weather.com/v3/location/search",
      "method": "GET",
      "headers": {},
      "parameters": {
        "query": {
          "value": "London",
          "required": true
        },
        "apiKey": {
          "value": "abcdef123456",
          "required": true
        }
      },
      "parameterGroups": [],
      "authType": "None",
      "auth": {},
      "responsePathExtractor": "value"
    }
  }
}

​
Response Example

{
  "jobs": [
    {
      "id": "job-123456",
      "job_type": "build_dataline_from_connected_resource",
      "status": "queued",
      "project_id": "9fd3cedc-d303-4870-9635-c89e60c3f1b0",
      "source": "datasource",
      "source_id": "f4cfd57d-dcf0-4039-ab31-3e2e7178189e",
      "source_event_type": "http_api",
      "created_at": "2023-09-21T15:35:00Z"
    }
  ],
  "data_object_id": "abcdef-1234-5678-9abc-def123456789"
}

​
Error Responses

​
400 Bad Request
object
Returned when the request is malformed or invalid
​
500 Internal Server Error
object
Returned when there’s a server error processing the request

​
Under the Hood

When you call this endpoint, the system:
  1. Creates an HTTPEndpoint instance from your connect_spec
  2. Loads data from the API using endpoint.load_with_defaults()
  3. If a responsePathExtractor is provided, extracts data from the specified key in the response
  4. Stores the HTTP request configuration as a JSON file
  5. Converts the response data (or extracted data) to a Parquet file
  6. Creates data objects in the database for both files
  7. Establishes data lineage connections between the objects
  8. Creates and queues background jobs to analyze and prepare the data
The resulting data object is ready to be used as a datasource in your project.

​
Important Notes

  • The responsePathExtractor parameter should only be used in Step 1 (Test Connection) and Step 4 (Execute HTTP Request). It is not needed in Steps 2 and 3.
  • Use the responsePathExtractor when your API response contains a specific key that holds the main data you want to work with. For example, if your API returns JSON with a value field containing stringified JSON data, setting responsePathExtractor to “value” will extract and use that data.