Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions content/build/access/fetch-data.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,13 @@ Gateways are the most performant way to fetch data from Arweave, providing signi
- **Network Optimization** - Distributed infrastructure for better performance
- **Content Delivery** - Optimized serving with compression and CDN features

<Callout type="info">
For production planning before implementation, see [Gateway
Expectations](/build/access/gateway-expectations) for access-mode
recommendations, practical limits, and real-world `turbo-gateway.com`
scenarios.
</Callout>

## REST APIs for Fetching Data

Gateways support multiple API endpoints for accessing data:
Expand Down
152 changes: 152 additions & 0 deletions content/build/access/gateway-expectations.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
---
title: "Gateway Expectations"
description: "Production-first guidance for choosing gateway access modes, understanding limits, and planning reliable delivery"
---

import { Card, Cards } from "fumadocs-ui/components/card";
import { Network, Building2, Server, Terminal } from "lucide-react";

For conceptual background, start with [Data Availability](/learn/gateways/data-availability).

## Production Quick Start

If you need stronger uptime and performance guarantees:

1. Use a **managed or self-hosted gateway** as your primary path.
2. Keep the **open ar.io gateway network** as resilience and backup.
3. Use **Wayfinder** when you want client-side gateway failover.

This gives you both predictable primary delivery and decentralized fallback.

## Access Options and Recommended Uses

| Access mode | Recommended use | Guarantee profile |
| --- | --- | --- |
| **Open network** | Prototypes, low-risk public traffic, fast launch | Strong retrievability, variable per-gateway performance/limits |
| **Self-hosted gateway** | Teams that want direct control over infra and policy | You define and operate the guarantee level |
| **Managed gateway** | Teams that want high availability with less ops overhead | Provider-backed operational guarantees |
| **Hybrid (recommended for production)** | Critical apps and customer-facing workloads | Dedicated primary guarantees + decentralized backup path |

## How to Inspect Any Gateway

Use `GET /ar-io/info`:

```bash
curl -sS https://<gateway>/ar-io/info | jq
```

Focus on these fields:

- `rateLimiter.enabled`
- `rateLimiter.dataEgress.buckets.resource`
- `rateLimiter.dataEgress.buckets.ip`
- `x402.enabled`
- `x402.dataEgress.pricing`
- `x402.dataEgress.rateLimiterCapacityMultiplier`

Interpretation:

- `capacity` / `capacityBytes`: burst allowance.
- `refillRate` / `refillRateBytesPerSec`: sustained unpaid throughput.
- `x402` pricing + multiplier: how paid overflow extends access.

## turbo-gateway.com Snapshot (As of February 16, 2026)

Source: `https://turbo-gateway.com/ar-io/info` queried on February 16, 2026.

### Rate limiter buckets

| Bucket | Value |
| --- | --- |
| Resource `capacityBytes` | `1,024,000,000` bytes (~`976.56` MiB burst) |
| Resource `refillRateBytesPerSec` | `102,400` bytes/s (`100` KiB/s sustained) |
| IP `capacityBytes` | `102,400,000` bytes (~`97.66` MiB burst) |
| IP `refillRateBytesPerSec` | `20,480` bytes/s (`20` KiB/s sustained) |

### x402 data egress

| Field | Value |
| --- | --- |
| Enabled | `true` |
| Network | `base` |
| Per-byte price | `0.0000000001` USDC |
| Min price | `0.001000` USDC |
| Max price | `1.000000` USDC |
| Capacity multiplier | `10` |

<Callout type="warning">
This is a dated snapshot. Re-check the live endpoint before relying on exact
numbers.
</Callout>

## Real-World Scenarios (Illustrative)

These examples use current turbo-gateway bucket values and assume bytes are not already absorbed by downstream cache/CDN.

### 1) Small API payload pattern (10 KiB response)

- **Unpaid sustained pace (per client IP):** about **2 requests/second** (`20 KiB/s ÷ 10 KiB`).
- **Burst behavior:** IP burst bucket (~97.66 MiB) can absorb large short spikes before refill rate becomes the limiter.
- **With x402 paid path:** additional paid tokens extend access when free tokens are exhausted.

### 2) Static site page-load pattern (~1.5 MiB cold load)

- **Unpaid sustained pace (per client IP):** about **0.8 page loads/minute** if every load is fully uncached.
- **Burst behavior:** IP burst bucket supports roughly **65 cold loads** in a short window (`97.66 MiB ÷ 1.5 MiB`).
- **In practice:** browser cache, gateway cache, and CDN caching usually reduce byte pressure significantly.

### 3) Larger file delivery pattern (100 MiB object)

- **Unpaid behavior:** one request can quickly consume most/all IP burst allowance.
- **Refill reality:** recovering ~100 MiB of unpaid headroom at `20 KiB/s` is slow (~85 minutes).
- **With x402 paid path:** requests continue through paid tokens, which is important for sustained large-transfer workloads.

<Callout type="info">
These scenarios are directional planning examples, not hard guarantees. Real
behavior depends on caching, traffic shape, and your selected access mode.
</Callout>

## How Limits Surface to Clients

- **`402 Payment Required`**: request exceeded free limits and a payment path is available.
- **`429 Too Many Requests`**: request is rate limited without a usable payment path.

Treat both as control signals and retry/failover events in client logic.

## Client Checklist

- Use exponential backoff with jitter for retriable failures.
- Implement gateway failover (or use Wayfinder).
- Use `ETag`/`If-None-Match` and range requests where appropriate.
- Follow redirects (including sandbox redirects).
- Monitor `X-Cache`, `X-AR-IO-Verified`, `X-AR-IO-Trusted`, and `Content-Digest`.
- Re-check `/ar-io/info` regularly for policy/limit changes.

## Next Steps

<Cards>
<Card
title="Data Availability"
description="Guarantee levels and strategy selection by access mode"
href="/learn/gateways/data-availability"
icon={<Network className="w-8 h-8" />}
/>
<Card
title="Fetch Data"
description="REST retrieval patterns and implementation details"
href="/build/access/fetch-data"
icon={<Server className="w-8 h-8" />}
/>
<Card
title="Ar.io Gateway API"
description="`/ar-io/info` and other gateway endpoints"
href="/apis/ar-io-node/gateway"
icon={<Terminal className="w-8 h-8" />}
/>
<Card
title="Wayfinder"
description="Production routing and failover across gateway providers"
href="/build/access/wayfinder"
icon={<Building2 className="w-8 h-8" />}
/>
</Cards>
10 changes: 9 additions & 1 deletion content/build/access/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@ title: "Access Data"
description: "Learn how to retrieve and query data from Arweave's permanent storage network"
---

import { Globe, Search, Link, Compass } from 'lucide-react';
import { Globe, Search, Link, Compass, Gauge } from 'lucide-react';

Once data is stored on Arweave, it's permanently available. Here's how to access it efficiently for your applications.

For production planning, start with **Gateway Expectations** to choose an access mode and understand practical limits.

## Access Methods

Different methods serve different needs. Each provides unique capabilities for retrieving data from Arweave.
Expand Down Expand Up @@ -131,6 +133,12 @@ curl https://arweave.net/[transaction-id-from-above]
## Additional Access Options

<Cards>
<Card
title="Gateway Expectations"
description="Choose your production access mode and understand real-world limit behavior"
href="/build/access/gateway-expectations"
icon={<Gauge className="w-8 h-8" />}
/>
<Card
title="Wayfinder SDK"
description="Advanced gateway routing for production apps"
Expand Down
1 change: 1 addition & 0 deletions content/build/access/meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"defaultOpen": false,
"pages": [
"fetch-data",
"gateway-expectations",
"find-data",
"arns",
"wayfinder"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ The main Ar.io Gateway service that handles data retrieval, indexing, and servin

| Variable | Type | Default | Description |
| ---------------------------- | ------ | ------------------------------------ | -------------------------------------------- |
| `ON_DEMAND_RETRIEVAL_ORDER` | string | `s3,trusted-gateways,chunks,tx-data` | On-demand data retrieval priority |
| `BACKGROUND_RETRIEVAL_ORDER` | string | `chunks,s3,trusted-gateways,tx-data` | Background data retrieval priority |
| `ON_DEMAND_RETRIEVAL_ORDER` | string | `trusted-gateways,ar-io-network,chunks-offset-aware,tx-data` | On-demand data retrieval priority |
| `BACKGROUND_RETRIEVAL_ORDER` | string | `chunks` | Background data retrieval priority |
| `CHUNK_DATA_SOURCE_TYPE` | string | `fs` | Chunk data source type (fs, legacy-s3) |
| `CHUNK_METADATA_SOURCE_TYPE` | string | `fs` | Chunk metadata source type (fs, legacy-psql) |

Expand Down Expand Up @@ -274,9 +274,9 @@ The main Ar.io Gateway service that handles data retrieval, indexing, and servin
| Variable | Type | Default | Description |
| -------------------------------------- | ------- | -------------- | ------------------------------------------ |
| `ENABLE_X_402_USDC_DATA_EGRESS` | boolean | `false` | Enable x402 USDC payments |
| `X_402_USDC_NETWORK` | string | - | Base network (base-sepolia or base) |
| `X_402_USDC_NETWORK` | string | `base-sepolia` | Base network (base-sepolia or base) |
| `X_402_USDC_WALLET_ADDRESS` | string | - | USDC wallet address for receiving payments |
| `X_402_USDC_FACILITATOR_URL` | string | - | Payment facilitator URL |
| `X_402_USDC_FACILITATOR_URL` | string | `https://x402.org/facilitator` | Payment facilitator URL |
| `X_402_USDC_PER_BYTE_PRICE` | number | `0.0000000001` | Price per byte in USDC |
| `X_402_USDC_DATA_EGRESS_MIN_PRICE` | number | `0.001` | Minimum payment amount |
| `X_402_USDC_DATA_EGRESS_MAX_PRICE` | number | `1.00` | Maximum payment amount |
Expand Down
109 changes: 109 additions & 0 deletions content/learn/gateways/data-availability.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
---
title: "Data Availability"
description: "Choose an access strategy, understand guarantee levels, and run production-ready gateway access with network fallback"
---

import { Card, Cards } from "fumadocs-ui/components/card";
import { Compass, Shield, Network, Server } from "lucide-react";

Data availability has two parts:

- **Permanent storage** from Arweave.
- **Reliable delivery** from gateways.

For production workloads, the key decision is not whether data exists, but how you want to deliver it with the guarantee level you need.

## Choose Your Access Strategy (Quick Recommendation)

| Strategy | Best for | Recommendation |
| --- | --- | --- |
| **A. Open network only** | Fast startup, prototypes, low-risk public content | Good default for early-stage projects |
| **B. Self-hosted gateway + network fallback** | Teams wanting control and custom policy | Strong production option when you can operate infra |
| **C. Managed gateway + network fallback** | Teams needing high availability with less ops overhead | **Recommended for production guarantees** |

<Callout type="tip">
If uptime and performance guarantees matter, run a dedicated gateway path
(self-hosted or managed) and keep the decentralized network as resilience and
backup.
</Callout>

## What We Can Guarantee (Mode-Based)

### Network-level guarantees

With the ar.io gateway model, you get:

- **Retrievability through multiple paths**: requests can resolve through cache, trusted gateways, ar.io peers, chunk reconstruction, and tx-data fallback.
- **Open access across independent operators**: no single required public gateway.
- **Integrity/trust signals in responses**: headers such as `X-AR-IO-Verified`, `X-AR-IO-Trusted`, `X-AR-IO-Digest`, and `Content-Digest`.

### Network-level non-guarantees

At the public-network layer, there is no single global guarantee for:

- One universal latency target.
- One universal throughput target.
- One universal rate-limit profile across all operators.

### Dedicated gateway guarantees (managed or self-hosted)

When you run dedicated infrastructure (or have it managed for you), you can provide stronger guarantees, including:

- High-availability architecture and failover policy.
- Predictable performance targets for your workload profile.
- Explicit traffic policies and operational ownership.

In short: the network guarantees resilient access paths, while dedicated gateways are how you guarantee stricter production outcomes.

## Recommended Production Patterns

1. **Direct network access**: simplest path for non-critical or moderate workloads.
2. **Hybrid fallback**: dedicated primary gateway, network as backup.
3. **Managed HA primary + optional self-hosted secondary**: strongest balance of guarantees and independence.

## How Availability Works Under the Hood

Most gateways follow a layered read path:

1. Local cache.
2. Trusted gateways.
3. Ar.io network peers.
4. Chunk-based reconstruction.
5. Transaction-data fallback.

This layered path is why data stays reachable even when a single source is slow or unavailable.

## When to Use Wayfinder

Use [Wayfinder](/build/access/wayfinder) when you want client-side routing and failover across gateways without hard-coding a single endpoint.

Wayfinder is a strong default for production read paths that need resilience.

## Next Steps

<Cards>
<Card
title="Gateway Expectations"
description="Production expectations, turbo-gateway snapshot, and real-world traffic scenarios"
href="/build/access/gateway-expectations"
icon={<Server className="w-8 h-8" />}
/>
<Card
title="Data Retrieval"
description="Deep dive into fallback ordering and retrieval sources"
href="/learn/gateways/data-retrieval"
icon={<Network className="w-8 h-8" />}
/>
<Card
title="Data Verification"
description="How gateways validate integrity across retrieval paths"
href="/learn/gateways/data-verification"
icon={<Shield className="w-8 h-8" />}
/>
<Card
title="Wayfinder"
description="Route requests across gateways with failover and verification controls"
href="/build/access/wayfinder"
icon={<Compass className="w-8 h-8" />}
/>
</Cards>
15 changes: 14 additions & 1 deletion content/learn/gateways/data-retrieval.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,17 @@ title: "Data Retrieval"
description: "How ar.io gateways retrieve and share data from multiple sources including trusted peers and Arweave nodes"
---

import { Shield, Cpu, Globe, Settings } from 'lucide-react';
import { Shield, Cpu, Globe, Settings, Gauge } from 'lucide-react';

Ar.io gateways use a sophisticated multi-tier architecture to retrieve and serve Arweave data. This system ensures high availability, fast response times, and data integrity by leveraging multiple data sources with automatic fallback mechanisms.

<Callout type="info">
Next in the learning flow: [Data Availability](/learn/gateways/data-availability)
for guarantee levels and access-strategy recommendations, then [Gateway
Expectations](/build/access/gateway-expectations) for practical production
limits and scenarios.
</Callout>

## How Gateways Retrieve Data

When a gateway needs to serve data, it follows a hierarchical retrieval pattern, trying each source in order until the data is successfully retrieved:
Expand Down Expand Up @@ -170,6 +177,12 @@ The data retrieval system is fundamental to ar.io's mission of providing reliabl
href="/build/access"
icon={<Globe className="w-8 h-8" />}
/>
<Card
title="Gateway Expectations"
description="Understand production variability, limits, and client handling patterns"
href="/build/access/gateway-expectations"
icon={<Gauge className="w-8 h-8" />}
/>
<Card
title="Optimize Your Gateway"
description="Configure indexing and filtering to optimize gateway performance"
Expand Down
Loading