> ## Documentation Index
> Fetch the complete documentation index at: https://learn.orbexa.io/llms.txt
> Use this file to discover all available pages before exploring further.

# ORBEXA API and database integration guide

> ORBEXA API and database integration guide — CSV import, ETL pipeline, visual scraping, and waterfall data ingestion for custom platforms.

# API and Database Integration

## Overview

For custom-built sites and non-standard e-commerce platforms, ORBEXA provides flexible data integration methods. When Shopify/WooCommerce/WordPress plugins are not applicable, merchants can bring product data into the platform through CSV import, ETL pipelines, or visual scraping.

## CSV Import

### Basic Flow

ORBEXA provides CSV file import functionality through the ETL Router:

1. **Upload CSV file** -- The merchant uploads a CSV file containing product data
2. **Field mapping** -- The platform provides a visual field mapping interface to map CSV column names to UCP/ACP standard fields
3. **Data validation** -- Automatically checks data format, required fields, and data types
4. **Import execution** -- After validation passes, product data is bulk imported
5. **AI cleaning** -- Imported data automatically enters the Refinery Pipeline for cleaning

### Field Mapping

Field mapping is the core of CSV import. Column names vary widely across different merchants' CSV files. ORBEXA's mapping engine supports:

* Manual mapping: Merchants specify the corresponding standard field for each column in the interface
* Smart suggestions: Automatically recommends mapping relationships based on column names and data content
* Saved mappings: Mapping rules are saved for reuse, eliminating the need to reconfigure on subsequent imports

## Visual Scraping

### Browser Automation Engine

For platforms that have no API and cannot export CSV files, ORBEXA provides visual scraping capabilities powered by a browser automation engine. It can browse web pages like a human and extract product information:

* Automatically identifies product listing pages and detail pages
* Extracts product name, price, images, description, and other fields
* Handles dynamically rendered page content
* Supports pagination and infinite scrolling

### Applicable Scenarios

* E-commerce sites built on traditional CMS platforms
* Legacy platforms without API interfaces
* Product data from third-party marketplaces or directories

## Waterfall Data Ingestion

ORBEXA implements a waterfall data ingestion strategy:

```
WooCommerce REST API
        |
        v
  Success? --Yes--> Data stored
        |
        No
        |
        v
  Visual Scraping Fallback
        |
        v
    Data stored
```

**API-first approach**: If the platform provides a REST API (such as WooCommerce), data is fetched through the API first.

**Automatic fallback to visual scraping**: If the API is unavailable or data is incomplete, visual scraping is automatically enabled as a fallback.

This strategy ensures ORBEXA can obtain product data regardless of the merchant's technical setup.

## Data Extraction Engine

ORBEXA's data extraction engine provides intelligent data processing:

* **Multi-source normalization**: Normalizes data from different sources (CSV, API, scraping) into a unified format
* **Smart field recognition**: Automatically identifies non-standard field names and maps them to UCP/ACP standard fields
* **Data quality scoring**: Evaluates the completeness and quality of each product record
* **Anomaly detection**: Flags price anomalies, missing descriptions, broken images, and other issues

The data extraction engine serves as the preprocessing stage before data enters the AI Refinery Pipeline for cleaning.

## Integration Method Comparison

| Method           | Applicable Scenario      | Technical Requirements | Data Freshness      |
| ---------------- | ------------------------ | ---------------------- | ------------------- |
| Shopify OAuth    | Shopify merchants        | None                   | Real-time (Webhook) |
| WooCommerce API  | WooCommerce merchants    | Generate API keys      | Real-time (Webhook) |
| WordPress Plugin | Non-e-commerce WordPress | Install plugin         | Low frequency       |
| CSV Import       | Any platform             | Prepare CSV file       | Manually triggered  |
| Visual Scraping  | Platforms without APIs   | Provide URL            | Periodic scraping   |
| API Integration  | Custom-built sites       | API development        | On demand           |

## Summary

API and database integration covers all scenarios beyond Shopify/WooCommerce/WordPress. CSV import provides the most universal bulk method, visual scraping solves the problem for platforms without APIs, the waterfall strategy ensures reliability of data acquisition, and the data extraction engine guarantees data quality.

***

Next chapter: [MCP Server](/en/book-7/ch6-mcp-server) -- 5 tools and 3 resources that let AI agents directly query products and inventory