Skip to content

Commit c7c2228

Browse files
rodrigopavezirodrigopavezi
authored
feat: add TRON substreams for ERC20FeeProxy payment detection (#2)
- Move substreams-tron from requestNetwork monorepo - Add tron/ folder with Rust WASM module for indexing payments - Add GitHub Actions CI workflow with optional integration tests - Index TransferWithReferenceAndFee events from ERC20FeeProxy - Support both Nile testnet and mainnet contracts <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * TRON Substreams module to index ERC20FeeProxy payment events on TRON mainnet * GraphQL schema for payments to support subgraph/GraphQL integrations * SQL schema and sink for persistent storage, including energy/fee metadata * **Chores** * CI pipeline for build, test, integration, and package publishing * Docker and Docker Compose deployment configs and production compose * **Documentation** * Expanded README with setup, build, run, and deployment guides <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 87da60a commit c7c2228

29 files changed

Lines changed: 6665 additions & 2 deletions

‎.env.example‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Streamingfast API Token
2+
# Get from: https://app.streamingfast.io/
3+
SUBSTREAMS_API_TOKEN=your-streamingfast-api-token
4+
5+
# PostgreSQL Password (for local development)
6+
POSTGRES_PASSWORD=changeme
7+
8+
# PostgreSQL Port (optional, defaults to 5432)
9+
# Use a different port if 5432 is already in use
10+
POSTGRES_PORT=5433

‎.env.prod.example‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Production Environment Variables for Easypanel
2+
# Copy this to .env.prod and fill in the values
3+
4+
# PostgreSQL connection (from Easypanel's built-in PostgreSQL service)
5+
POSTGRES_HOST=your-project-postgres.internal
6+
POSTGRES_PORT=5432
7+
POSTGRES_DB=tron_payments
8+
POSTGRES_USER=postgres
9+
POSTGRES_PASSWORD=your-secure-password-here
10+
11+
# Streamingfast API Token
12+
# Get from: https://app.streamingfast.io/
13+
SUBSTREAMS_API_TOKEN=your-streamingfast-api-token

‎.github/workflows/deploy-to-production.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
name: Deploy to production
2+
3+
on:
4+
workflow_dispatch:
5+
release:
6+
types: [published]
7+
8+
jobs:
9+
deploy:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Webhook request
13+
run: |
14+
curl -X GET https://prod.request.network/api/deploy/${{ secrets.EASYPANEL_DEPLOY_KEY_PRODUCTION }}

‎.github/workflows/deploy-to-staging.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
name: Deploy to staging
2+
3+
on:
4+
workflow_dispatch:
5+
push:
6+
branches: [main]
7+
8+
jobs:
9+
deploy:
10+
runs-on: ubuntu-latest
11+
steps:
12+
- name: Webhook request
13+
run: |
14+
curl -X GET https://stage.request.network/api/deploy/${{ secrets.EASYPANEL_DEPLOY_KEY_STAGING }}

‎.github/workflows/tron-build.yml‎

Lines changed: 262 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,262 @@
1+
name: Tron Substreams Build
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- main
7+
paths:
8+
- 'tron/**'
9+
- '.github/workflows/tron-build.yml'
10+
push:
11+
branches:
12+
- main
13+
paths:
14+
- 'tron/**'
15+
- '.github/workflows/tron-build.yml'
16+
workflow_dispatch:
17+
inputs:
18+
run_integration_tests:
19+
description: 'Run integration tests against live endpoints'
20+
required: false
21+
default: false
22+
type: boolean
23+
publish_package:
24+
description: 'Build and publish package artifact'
25+
required: false
26+
default: false
27+
type: boolean
28+
29+
jobs:
30+
build-and-test:
31+
name: Build and Test Tron Substreams
32+
runs-on: ubuntu-latest
33+
34+
steps:
35+
- name: Checkout repository
36+
uses: actions/checkout@v4
37+
38+
- name: Install Rust toolchain
39+
uses: dtolnay/rust-toolchain@stable
40+
with:
41+
targets: wasm32-unknown-unknown
42+
43+
- name: Cache Cargo dependencies
44+
uses: actions/cache@v4
45+
with:
46+
path: |
47+
~/.cargo/bin/
48+
~/.cargo/registry/index/
49+
~/.cargo/registry/cache/
50+
~/.cargo/git/db/
51+
tron/target/
52+
key: ${{ runner.os }}-cargo-${{ hashFiles('tron/Cargo.toml') }}
53+
restore-keys: |
54+
${{ runner.os }}-cargo-
55+
56+
- name: Build WASM module
57+
working-directory: tron
58+
run: make build
59+
60+
- name: Run unit tests
61+
working-directory: tron
62+
run: make test
63+
64+
- name: Verify build artifacts
65+
working-directory: tron
66+
run: |
67+
echo "Checking build artifacts..."
68+
if [ ! -f "target/wasm32-unknown-unknown/release/request_network_tron.wasm" ]; then
69+
echo "ERROR: WASM file not found!"
70+
exit 1
71+
fi
72+
echo "Build artifacts verified successfully"
73+
74+
integration-test-mainnet:
75+
name: Integration Test (Mainnet)
76+
runs-on: ubuntu-latest
77+
needs: build-and-test
78+
# Run on: push to main (merge) or manual trigger
79+
# PRs only run unit tests (no API key needed)
80+
if: |
81+
github.event.inputs.run_integration_tests == 'true' ||
82+
github.event.inputs.publish_package == 'true' ||
83+
(github.event_name == 'push' && github.ref == 'refs/heads/main')
84+
85+
steps:
86+
- name: Checkout repository
87+
uses: actions/checkout@v4
88+
89+
- name: Install Rust toolchain
90+
uses: dtolnay/rust-toolchain@stable
91+
with:
92+
targets: wasm32-unknown-unknown
93+
94+
- name: Cache Cargo dependencies
95+
uses: actions/cache@v4
96+
with:
97+
path: |
98+
~/.cargo/bin/
99+
~/.cargo/registry/index/
100+
~/.cargo/registry/cache/
101+
~/.cargo/git/db/
102+
tron/target/
103+
key: ${{ runner.os }}-cargo-${{ hashFiles('tron/Cargo.toml') }}
104+
restore-keys: |
105+
${{ runner.os }}-cargo-
106+
107+
- name: Install Substreams CLI
108+
run: |
109+
curl -sSL https://github.com/streamingfast/substreams/releases/latest/download/substreams_linux_x86_64.tar.gz | tar xz
110+
sudo mv substreams /usr/local/bin/
111+
substreams --version
112+
113+
- name: Build WASM module
114+
working-directory: tron
115+
run: make build
116+
117+
- name: Package Substreams
118+
working-directory: tron
119+
run: make package
120+
121+
- name: Verify package created
122+
working-directory: tron
123+
run: |
124+
SPKG_FILE=$(ls -1 *.spkg 2>/dev/null | head -1)
125+
if [ -z "$SPKG_FILE" ]; then
126+
echo "ERROR: No .spkg package file found!"
127+
exit 1
128+
fi
129+
echo "✅ Package created: $SPKG_FILE"
130+
131+
- name: Deploy and test against Mainnet (Streamingfast)
132+
working-directory: tron
133+
env:
134+
SUBSTREAMS_API_TOKEN: ${{ secrets.SUBSTREAMS_API_TOKEN }}
135+
run: |
136+
echo "=== Deploying to TRON Mainnet (Streamingfast) ==="
137+
echo "Endpoint: mainnet.tron.streamingfast.io:443"
138+
echo "Block range: 79216121 to 79216221 (100 blocks from mainnet deployment)"
139+
echo ""
140+
141+
SPKG_FILE=$(ls -1 *.spkg | head -1)
142+
echo "Using package: $SPKG_FILE"
143+
144+
# Run substreams and capture JSON output
145+
if ! substreams run "$SPKG_FILE" map_erc20_fee_proxy_payments \
146+
-e mainnet.tron.streamingfast.io:443 \
147+
--start-block 79216121 \
148+
--stop-block +100 \
149+
-o json 2>&1 | tee output_mainnet.log; then
150+
echo ""
151+
echo "❌ FAILED: Substreams command returned non-zero exit code"
152+
echo "Output:"
153+
cat output_mainnet.log
154+
exit 1
155+
fi
156+
157+
echo ""
158+
echo "=== Validating Results ==="
159+
160+
# If any payments were found, validate the structure
161+
if grep -q '"payments"' output_mainnet.log; then
162+
echo "✅ Payments data structure detected in output"
163+
164+
# Extract and validate payment fields if present
165+
if grep -q '"token_address"' output_mainnet.log; then
166+
echo "✅ Payment fields present: token_address"
167+
fi
168+
if grep -q '"payment_reference"' output_mainnet.log; then
169+
echo "✅ Payment fields present: payment_reference"
170+
fi
171+
if grep -q '"amount"' output_mainnet.log; then
172+
echo "✅ Payment fields present: amount"
173+
fi
174+
else
175+
echo "ℹ️ No payments found in block range (expected if no transactions in these blocks)"
176+
fi
177+
178+
echo ""
179+
echo "=== Mainnet Integration Test PASSED ==="
180+
echo "The substreams module successfully:"
181+
echo " - Loaded WASM module"
182+
echo " - Connected to TRON Mainnet endpoint via Streamingfast"
183+
echo " - Processed 100 blocks without errors"
184+
echo " - Output valid JSON structure"
185+
186+
# Publish package artifact for download/deployment
187+
publish-package:
188+
name: Publish Mainnet Package
189+
runs-on: ubuntu-latest
190+
needs: [build-and-test, integration-test-mainnet]
191+
# Publish on: push to main (merge) or manual publish selection
192+
if: |
193+
(github.event_name == 'push' && github.ref == 'refs/heads/main') ||
194+
github.event.inputs.publish_package == 'true'
195+
196+
steps:
197+
- name: Checkout repository
198+
uses: actions/checkout@v4
199+
200+
- name: Install Rust toolchain
201+
uses: dtolnay/rust-toolchain@stable
202+
with:
203+
targets: wasm32-unknown-unknown
204+
205+
- name: Cache Cargo dependencies
206+
uses: actions/cache@v4
207+
with:
208+
path: |
209+
~/.cargo/bin/
210+
~/.cargo/registry/index/
211+
~/.cargo/registry/cache/
212+
~/.cargo/git/db/
213+
tron/target/
214+
key: ${{ runner.os }}-cargo-${{ hashFiles('tron/Cargo.toml') }}
215+
restore-keys: |
216+
${{ runner.os }}-cargo-
217+
218+
- name: Install Substreams CLI
219+
run: |
220+
curl -sSL https://github.com/streamingfast/substreams/releases/latest/download/substreams_linux_x86_64.tar.gz | tar xz
221+
sudo mv substreams /usr/local/bin/
222+
223+
- name: Build WASM module
224+
working-directory: tron
225+
run: make build
226+
227+
- name: Package Substreams
228+
working-directory: tron
229+
run: make package
230+
231+
- name: Get package info
232+
id: package_info
233+
working-directory: tron
234+
run: |
235+
SPKG_FILE=$(ls -1 *.spkg 2>/dev/null | head -1)
236+
if [ -z "$SPKG_FILE" ]; then
237+
echo "ERROR: No .spkg package file found!"
238+
exit 1
239+
fi
240+
echo "spkg_file=$SPKG_FILE" >> $GITHUB_OUTPUT
241+
242+
VERSION=$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)".*/\1/')
243+
echo "version=$VERSION" >> $GITHUB_OUTPUT
244+
245+
echo "Package created successfully: $SPKG_FILE (version: $VERSION)"
246+
247+
- name: Upload package artifact
248+
uses: actions/upload-artifact@v4
249+
with:
250+
name: request-network-tron-${{ steps.package_info.outputs.version }}
251+
path: tron/${{ steps.package_info.outputs.spkg_file }}
252+
retention-days: 90
253+
254+
- name: Package ready for deployment
255+
run: |
256+
echo "=============================================="
257+
echo "📦 Substreams Package Ready"
258+
echo "=============================================="
259+
echo ""
260+
echo "Package: ${{ steps.package_info.outputs.spkg_file }}"
261+
echo "Version: ${{ steps.package_info.outputs.version }}"
262+
echo "=============================================="

‎.gitignore‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Rust build artifacts
2+
target/
3+
4+
# Substreams packages (built during Docker/CI)
5+
*.spkg
6+
7+
# Substreams replay log
8+
replay.log
9+
10+
# IDE
11+
.idea/
12+
.vscode/
13+
*.swp
14+
*.swo
15+
16+
# OS
17+
.DS_Store
18+
Thumbs.db
19+
20+
# Environment
21+
.env
22+
.env.local

0 commit comments

Comments
 (0)