Connector Details
Type
Virtual machines, Single VM , BYOL
Runs on
Google Compute Engine
Last Update
24 October, 2024
Category
Big data, Analytics, DevOps
Ajman Open Data Portal Connector
Overview
The Ajman Open Data Connector facilitates seamless integration with Ajman’s open data platform, enabling access to catalog and dataset-related data for applications such as data analytics, visualization tools, or geographic information systems. This connector acts as a proxy to streamline data retrieval, supporting actions for dataset enumeration, record queries, data exports, and facet-based navigation.
Integration Overview
This document provides a detailed guide for each integration point, its purpose, configuration, and workflow support using the Ajman Open Data Connector.
Supported Integration Action Points
- getDatasets: Retrieves a list of all datasets in the catalog.
- listExportFormats: Lists available export formats for the dataset catalog.
- exportDatasets: Exports the dataset catalog in a specified format (e.g., CSV, JSON).
- exportCatalogCSV: Exports the dataset catalog as CSV with customizable parameters.
- exportCatalogDCAT: Exports catalog metadata in DCAT-AP format (RDF/XML).
- getDatasetsFacets: Retrieves global catalog-level facets for guided navigation.
- getDataset: Retrieves metadata for a specific dataset.
- getRecords: Retrieves records for a specific dataset.
- getRecord: Retrieves a specific record from a dataset.
- listDatasetExportFormats: Lists supported export formats for a dataset.
- exportRecords: Exports a dataset in a specific format (e.g., CSV, GeoJSON).
- exportRecordsCSV: Exports a dataset in CSV format with specific parameters.
- exportRecordsParquet: Exports a dataset in Parquet format.
- exportRecordsGPX: Exports a dataset in GPX format.
- getRecordsFacets: Retrieves available facets for a dataset.
- getDatasetAttachments: Retrieves files or attachments related to a dataset.
Detailed Integration Documentation
Catalog Datasets Retrieval
Action
getDatasets
Purpose
Retrieves a comprehensive list of all datasets in the catalog. This serves as the primary entry point for exploring available data.
Parameters
- Required: None.
- Optional:
- select: Specify fields to include (e.g., dataset_id, fields).
- where: Filter datasets using ODSQL (e.g., publisher=”Opendatasoft”).
- order_by: Sort results (e.g., modified desc).
- limit: Number of items (default: 10, max: 100).
- offset: Starting index (default: 0).
- refine: Filter by facet (e.g., publisher:Opendatasoft).
- exclude: Exclude facet values (e.g., publisher:Opendatasoft).
- lang: Language for formatting (e.g., en).
- timezone: Timezone for datetime fields (e.g., Europe/Paris).
- group_by: Group results (e.g., publisher).
- include_links: Include HATEOAS links (boolean, default: false).
- include_app_metas: Include metadata (boolean, default: false).
- apikey: API key for authentication (if required).
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a JSON object with total_count, _links, and results (array of dataset objects containing dataset_id, fields, metas, etc.).
- Failure: Returns error details (e.g., invalid parameters).
Workflow Example
1. Configure the connector with the appropriate base URL.
2. Execute the getDatasets action to fetch a list of datasets.
3. Process the response to identify datasets (e.g., lands-sales) for further actions.
Catalog Exports Listing
Action
listExportFormats
Purpose
Lists all available export formats for the dataset catalog. This helps users choose an appropriate format for data export.
Parameters
- Required: None.
- Optional: None.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a JSON object with a links array listing export format options (e.g., csv, json).
- Failure: Returns error details (e.g., upstream API error).
Workflow Example
1. Execute the listExportFormats action to retrieve available export formats.
2. Review the response to select a format (e.g., csv) for catalog export.
3. Proceed to the exportDatasets action for export.
Catalog Export by Format
Action
exportDatasets
Purpose
Exports the entire dataset catalog in a user-specified format. Supported formats include JSON, CSV, and XLSX for versatile data use.
Parameters
- Required: format (e.g., csv, json).
- Optional: Same as getDatasets (e.g., select, where, limit).
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a file in the specified format.
- Failure: Returns error details (e.g., unsupported format).
Workflow Example
1. Use the listExportFormats action to identify supported formats.
2. Execute the exportDatasets action with format=csv and limit=10.
3. Save the exported CSV file for external analysis.
Catalog CSV Export
Action
exportCatalogCSV
Purpose
Exports the dataset catalog in CSV format with customizable parameters. This allows tailored data extraction for analysis or reporting.
Parameters
- Required: None.
- Optional:
- delimiter: Field delimiter (e.g., default: ;).
- list_separator: Multivalued string separator (default: ,).
- quote_all: Quote all strings (boolean, default: false).
- with_bom: Include Unicode BOM (boolean, default: true).
- Other parameters as in getDatasets.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a CSV file.
- Failure: Returns error details (e.g., invalid parameters).
Workflow Example
1. Execute the exportCatalogCSV action with delimiter=,.
2. Save the CSV file containing catalog metadata.
3. Use the file for data analysis or reporting.
Catalog DCAT Export
Action
exportCatalogDCAT
Purpose
Exports catalog metadata in RDF/XML (DCAT-AP) format. This enables integration with metadata systems supporting DCAT standards.
Parameters
- Required: dcat_ap_format (e.g., dcat_ap_ch, dcat_ap_de).
- Optional:
- include_exports: Export formats to include (e.g., csv, json).
- use_labels_in_exports: Use field labels (boolean, default: true).
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns an RDF/XML file.
- Failure: Returns error details (e.g., invalid DCAT format).
Workflow Example
1. Execute the exportCatalogDCAT action with dcat_ap_format=dcat_ap_ch.
2. Save the RDF/XML file.
3. Use the file for metadata integration with DCAT-compatible systems.
Catalog Facets Retrieval
Action
getDatasetsFacets
Purpose
Retrieves facet values for datasets to aid navigation. This helps users filter datasets based on specific attributes like publisher.
Parameters
- Required: None.
- Optional:
- facet: Specify facet to retrieve (e.g., publisher).
- Other parameters as in getDatasets (e.g., refine, exclude, where).
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a JSON object with links and facets arrays.
- Failure: Returns error details (e.g., invalid facet).
Workflow Example
1. Execute the getDatasetsFacets action with facet=publisher.
2. Review facet values (e.g., Opendatasoft) to refine dataset queries.
3. Use facets to filter results in subsequent actions.
Dataset Metadata Retrieval
Action
getDataset
Purpose
Retrieves detailed metadata for a specific dataset. This includes fields and endpoints to plan further data queries or exports.
Parameters
- Required: dataset_id (e.g., geonames-all-cities-with-a-population-1000).
- Optional:
- select, lang, timezone, include_links, include_app_metas, apikey.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a JSON object with dataset details (e.g., dataset_id, fields, metas).
- Failure: Returns error details (e.g., invalid dataset_id).
Workflow Example
1. Use the getDatasets action to identify a dataset (e.g., geonames-all-cities-with-a-population-1000).
2. Execute the getDataset action with the selected dataset_id.
3. Review metadata to plan record queries or exports.
Dataset Records Retrieval
Action
getRecords
Purpose
Queries records from a specific dataset. This allows users to access detailed data entries for analysis or visualization.
Parameters
- Required:dataset_id.
- Optional:
- select, where, group_by, order_by, limit (max: 100), offset, refine, exclude, lang, timezone, include_links, include_app_metas, apikey.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a JSON object with total_count, _links, and results (array of records).
- Failure:Returns error details (e.g., invalid dataset_id).
Workflow Example
1. Use the getDataset action to select a dataset (e.g., lands-sales).
2. Execute the getRecords action with dataset_id=lands-sales and where=transaction_type:Sale.
3. Process the response to display or analyze records.
Dataset Record Retrieval
Action
getRecord
Purpose
Retrieves a single record from a dataset. This provides detailed information for a specific data entry.
Parameters
- Required:dataset_id, record_id (e.g., sale_001).
- Optional:
- select, lang, timezone, apikey.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a JSON object representing the record.
- Failure: Returns error details (e.g., invalid record_id).
Workflow Example
1. Use the getRecords action to identify a record in lands-sales.
2. Execute the getRecord action with dataset_id=lands-sales and record_id=sale_001.
3. Review the record details for further processing.
Dataset Exports Listing
Action
listDatasetExportFormats
Purpose
Lists available export formats for a specific dataset. This helps users select a format for dataset export.
Parameters
- Required:dataset_id.
- Optional:
- None.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a JSON object with a links array listing export formats.
- Failure: Returns error details (e.g., invalid dataset_id).
Workflow Example
1. Use the getDataset action to select a dataset (e.g., lands-sales).
2. Execute the listDatasetExportFormats action with dataset_id=lands-sales.
3. Review available formats (e.g., csv, parquet) for export.
Dataset Export by Format
Action
exportRecords
Purpose
Exports a dataset in a user-specified format. Supported formats include CSV, GeoJSON, and others for diverse applications.
Parameters
- Required:dataset_id, format (e.g., csv, geojson).
- Optional:
- select, where, order_by, group_by, limit, refine, exclude, lang, timezone, use_labels, epsg, apikey.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful: Returns a file in the specified format.
- Failure: Returns error details (e.g., invalid format).
Workflow Example
1. Use the listDatasetExportFormats action to identify supported formats for lands-sales.
2. Execute the exportRecords action with dataset_id=lands-sales and format=csv.
3. Save the exported file for external use.
Dataset CSV Export
Action
exportRecordsCSV
Purpose
Exports a dataset in CSV format with customizable parameters. This enables tailored data extraction for specific needs.
Parameters
- Required:dataset_id.
- Optional:
- delimiter, list_separator, quote_all, with_bom, and others as in exportRecords.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful:Returns a CSV file.
- Failure:Returns error details (e.g., invalid parameters).
Workflow Example
1. Execute the exportRecordsCSV action with dataset_id=lands-sales and delimiter=,.
2. Save the CSV file containing dataset records.
3. Use the file for data analysis or reporting.
Dataset Parquet Export
Action
exportRecordsParquet
Purpose
Exports a dataset in Parquet format. This is ideal for efficient storage and processing in data analytics platforms.
Parameters
- Required:dataset_id.
- Optional:
- parquet_compression (snappy or zstd, default: snappy), and others as in exportRecords.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful:Returns a Parquet file.
- Failure:Returns error details (e.g., invalid dataset_id).
Workflow Example
1. Execute the exportRecordsParquet action with dataset_id=lands-sales.
2. Save the Parquet file.
3. Use the file for data processing in compatible systems.
Dataset GPX Export
Action
exportRecordsGPX
Purpose
Exports a dataset in GPX format. This is suitable for geographic data visualization and GPS applications.
Parameters
- Required:dataset_id.
- Optional:
- name_field, description_field_list, use_extension (boolean, default: true), and others as in exportRecords.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful:Returns a GPX file.
- Failure:Returns error details (e.g., invalid parameters).
Workflow Example
1. Execute the exportRecordsGPX action with dataset_id=lands-sales and name_field=name.
2. Save the GPX file.
3. Use the file for geographic data visualization.
Dataset Facets Retrieval
Action
getRecordsFacets
Purpose
Retrieves facet values for records in a specific dataset. This aids in filtering and navigating dataset records effectively.
Parameters
- Required:dataset_id.
- Optional:
- facet, where, refine, exclude, lang, timezone, apikey.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful:Returns a JSON object with links and facets arrays.
- Failure:Returns error details (e.g., invalid facet).
Workflow Example
1. Use the getDataset action to select lands-sales.
2. Execute the getRecordsFacets action with dataset_id=lands-sales and facet=transaction_type.
3. Use facet values to refine record queries.
Dataset Attachments Retrieval
Action
getDatasetAttachments
Purpose
Retrieves files or attachments related to a specific dataset. This provides access to supplementary data or resources.
Parameters
- Required:dataset_id.
- Optional:
- None.
Configuration
Ensure the connector is configured with the correct base URL.
Output
- Successful:Returns a JSON object with links and attachments arrays (e.g., href, mime-type, title).
- Failure:Returns error details (e.g., invalid dataset_id).
Workflow Example
1. Use the getDataset action to select lands-sales.
2. Execute the getDatasetAttachments action with dataset_id=lands-sales.
3. Download attachments (e.g., lands_sales_archive.zip) for further use.
Workflow Creation with the Connector
Example Workflow: Exploring and Exporting Land Sales
Retrieve Catalog Datasets:
- Use the getDatasets action to fetch a list of available datasets.
- Identify the target dataset (e.g., lands-sales).
Refine Dataset Exploration:
Execute the getRecordsFacets action with dataset_id=lands-sales and facet=transaction_type to filter records.
Query Dataset Records:
Use the getRecords action with dataset_id=lands-sales and where=transaction_type:Sale to fetch relevant records.
Export Dataset Data:
- Encode the dataset in CSV format using the exportRecordsCSV action with dataset_id=lands-sales.
- Export the dataset in Parquet format via the exportRecordsParquet action for analytical processing.
- Generate a GPX file for geographic visualization using the exportRecordsGPX action with name_field=name.
Retrieve Metadata and Attachments:
- Fetch metadata for lands-sales using the getDataset action.
- Download attachments (e.g., archives) via the getDatasetAttachments action.
Export Catalog Metadata
- Export the catalog as CSV using the exportCatalogCSV action.
- Generate DCAT-AP metadata via the exportCatalogDCAT action with dcat_ap_format=dcat_ap_ch.
Pricing
iSolution logo - white - transparent 250 px