Skip to content

feat(plugin): fill section bloom using exist db #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
317787106 wants to merge 1 commit into
base: feature/release_v4.8.2_self
Choose a base branch
from
Open

feat(plugin): fill section bloom using exist db #80

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions plugins/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,36 @@ NOTE: large db may GC overhead limit exceeded.
- `--db`: db name.
- `-h | --help`: provide the help info

## DB Backfill-Bloom

DB backfill bloom provides the ability to backfill SectionBloom data for historical blocks to enable `eth_getLogs` address/topics filtering. This is useful when `isJsonRpcFilterEnabled` was disabled during block processing and later enabled, causing historical blocks to lack SectionBloom data.

### Available parameters:

- `-d | --database-directory`: Specify the database directory path, it is used to open the database to get the transaction log and write the SectionBloom data back, default: output-directory/database.
- `-s | --start-block`: Specify the start block number for backfill (required).
- `-e | --end-block`: Specify the end block number for backfill (optional, default: latest block).
- `-c | --max-concurrency`: Specify the maximum concurrency for processing, default: 8.
- `-h | --help`: Provide the help info.

### Examples:

```shell script
# full command
java -jar Toolkit.jar db backfill-bloom [-h] -s=<startBlock> [-e=<endBlock>] [-d=<databaseDirectory>] [-c=<maxConcurrency>]
# examples
java -jar Toolkit.jar db backfill-bloom -s 1000000 -e 2000000 #1. backfill blocks 1000000 to 2000000
java -jar Toolkit.jar db backfill-bloom -s 1000000 -d /path/to/database #2. specify custom database directory
java -jar Toolkit.jar db backfill-bloom -s 1000000 -c 8 #3. use higher concurrency (8 threads)
```

### Backfill speed

The time required to process different block ranges varies. It is recommended to increase `--max-concurrency` appropriately to speed up the backfill process.

- 0-10000000: It's done almost instantly because there are no logs inside.
- 10000000-70000000: Takes about 3-4 hours/10,000,000 blocks with `--max-concurrency` set to 32.

## Keystore

Keystore provides commands for managing account keystore files (Web3 Secret Storage format).
Expand Down
1 change: 1 addition & 0 deletions plugins/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ dependencies {
implementation 'io.github.tronprotocol:leveldb:1.18.2'
}
implementation project(":protocol")
implementation project(":chainbase")

@cubic-dev-ai cubic-dev-ai Bot Jun 7, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Missing project(':chainbase').jar in binaryRelease's explicit dependsOn list. The comment states all project dependency jars must be declared explicitly to avoid Gradle warnings and ensure correct build ordering for fat jar assembly.

Prompt for AI agents 
Check if this issue is valid — if so, understand the root cause and fix it. At plugins/build.gradle, line 73:

<comment>Missing `project(':chainbase').jar` in `binaryRelease`'s explicit `dependsOn` list. The comment states all project dependency jars must be declared explicitly to avoid Gradle warnings and ensure correct build ordering for fat jar assembly.</comment>

<file context>
@@ -70,6 +70,7 @@ dependencies {
         implementation 'io.github.tronprotocol:leveldb:1.18.2'
     }
     implementation project(":protocol")
+    implementation project(":chainbase")
 }
 
</file context>

}

check.dependsOn 'lint'
Expand Down
1 change: 1 addition & 0 deletions plugins/src/main/java/common/org/tron/plugins/Db.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
DbConvert.class,
DbLite.class,
DbCopy.class,
DbBackfillBloom.class,
DbRoot.class
},
commandListHeading = "%nCommands:%n%nThe most commonly used db commands are:%n"
Expand Down
Loading