-
Notifications
You must be signed in to change notification settings - Fork 615
fix(server): prevent graph clear API from clearing meta table in MySQL backend storage #2888
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: release-1.5.0
Are you sure you want to change the base?
Changes from 4 commits
e376b6c
45d95fe
22b2407
d7f35f1
4ad56cb
40e0314
ea75f05
e14eb0f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -54,7 +54,6 @@ public abstract class MysqlStore extends AbstractBackendStore<Session> { | |||||||||||||||
| private final BackendStoreProvider provider; | ||||||||||||||||
|
|
||||||||||||||||
| private final Map<HugeType, MysqlTable> tables; | ||||||||||||||||
|
|
||||||||||||||||
| private MysqlSessions sessions; | ||||||||||||||||
|
|
||||||||||||||||
| public MysqlStore(final BackendStoreProvider provider, | ||||||||||||||||
|
|
@@ -223,6 +222,7 @@ public void truncate() { | |||||||||||||||
| this.checkOpened(); | ||||||||||||||||
|
|
||||||||||||||||
| this.truncateTables(); | ||||||||||||||||
| this.init(); | ||||||||||||||||
|
imbajin marked this conversation as resolved.
Outdated
imbajin marked this conversation as resolved.
Outdated
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The PR description mentions that "HugeGraph only supports hstore, rocksdb, hbase, and memory" but this fix is only applied to MySQL backend. Questions to verify:
Recommendation:
this.truncateTables();
// MySQL-specific: Re-initialize tables to restore meta table
// Other backends handle meta table separately during truncation
this.init();
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Root Cause Analysis by Backend
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
@LYD031106 LGTM, we could add a basic test & comment for the above issue (and we could merge this PR soon) BTW, this week we prepare to release the new version (1.7.0)
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I have submitted a MySQL compatibility fix to the
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
refer ci steps: https://github.com/apache/incubator-hugegraph/actions/runs/18783126996/workflow?pr=2888#L49
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧹 Minor: Consider side effects of calling init() The
Observation: Potential issue: Suggestion:
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Adding
Please verify this is the correct approach and consider adding a comment explaining why re-initialization is needed after truncate. |
||||||||||||||||
| LOG.debug("Store truncated: {}", this.store); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
|
|
@@ -337,6 +337,9 @@ protected void clearTables() { | |||||||||||||||
| protected void truncateTables() { | ||||||||||||||||
| Session session = this.sessions.session(); | ||||||||||||||||
| for (MysqlTable table : this.tables()) { | ||||||||||||||||
| if (table instanceof MysqlTables.Meta || table instanceof MysqlTables.Counters){ | ||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The PR description mentions fixing Meta table truncation but doesn't explain why the Counters table should also be skipped. This needs clarification:
Please add a comment explaining the reasoning, or if this is a separate fix, consider splitting it into a separate PR for clarity.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry, this was my misjudgment: I mistakenly thought the Counters table was a system table similar to the Meta table, so I included it in the truncation-skip logic. |
||||||||||||||||
| continue; | ||||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The current fix uses
Suggested change
Alternatively, you could maintain a set of system tables or use table metadata to mark which tables should be preserved. |
||||||||||||||||
| } | ||||||||||||||||
| table.truncate(session); | ||||||||||||||||
| } | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -46,6 +46,8 @@ | |
| import org.apache.hugegraph.unit.id.IdTest; | ||
| import org.apache.hugegraph.unit.id.IdUtilTest; | ||
| import org.apache.hugegraph.unit.id.SplicingIdGeneratorTest; | ||
| import org.apache.hugegraph.unit.mysql.BaseMysqlUnitTest; | ||
| import org.apache.hugegraph.unit.mysql.MysqlTest; | ||
| import org.apache.hugegraph.unit.mysql.MysqlUtilTest; | ||
| import org.apache.hugegraph.unit.mysql.WhereBuilderTest; | ||
| import org.apache.hugegraph.unit.rocksdb.RocksDBCountersTest; | ||
|
|
@@ -130,6 +132,8 @@ | |
| /* mysql */ | ||
| MysqlUtilTest.class, | ||
| WhereBuilderTest.class, | ||
| BaseMysqlUnitTest.class, | ||
|
||
| MysqlTest.class, | ||
|
|
||
| /* rocksdb */ | ||
| RocksDBSessionsTest.class, | ||
|
|
||
|
imbajin marked this conversation as resolved.
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| /* | ||
| * | ||
| * * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * * contributor license agreements. See the NOTICE file distributed with | ||
| * * this work for additional information regarding copyright ownership. | ||
| * * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * * (the "License"); you may not use this file except in compliance with | ||
| * * the License. You may obtain a copy of the License at | ||
| * * | ||
| * * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * * | ||
| * * Unless required by applicable law or agreed to in writing, software | ||
| * * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * * See the License for the specific language governing permissions and | ||
| * * limitations under the License. | ||
| * | ||
| */ | ||
|
|
||
| package org.apache.hugegraph.unit.mysql; | ||
|
|
||
| import org.apache.commons.configuration2.Configuration; | ||
| import org.apache.hugegraph.backend.store.mysql.MysqlStoreProvider; | ||
| import org.apache.hugegraph.config.HugeConfig; | ||
| import org.apache.hugegraph.testutil.Utils; | ||
| import org.apache.hugegraph.unit.BaseUnitTest; | ||
| import org.apache.hugegraph.unit.FakeObjects; | ||
| import org.junit.After; | ||
|
|
||
| public class BaseMysqlUnitTest extends BaseUnitTest { | ||
|
|
||
| private static final String GRAPH_NAME = "test_graph"; | ||
|
|
||
| protected HugeConfig config; | ||
| protected MysqlStoreProvider provider; | ||
|
|
||
| public void setup() { | ||
| try { | ||
| Configuration conf = Utils.getConf(); | ||
| this.config = new HugeConfig(conf); | ||
| } catch (Exception e) { | ||
| this.config = FakeObjects.newConfig(); | ||
| } | ||
| this.provider = new MysqlStoreProvider(); | ||
| this.provider.open(GRAPH_NAME); | ||
| this.provider.loadSystemStore(config).open(config); | ||
| this.provider.loadGraphStore(config).open(config); | ||
| this.provider.loadSchemaStore(config).open(config); | ||
| this.provider.init(); | ||
| } | ||
|
|
||
| @After | ||
| public void down(){ | ||
| if (this.provider != null) { | ||
| try { | ||
| this.provider.close(); | ||
| } catch (Exception e) { | ||
| // pass | ||
| } | ||
| } | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,43 @@ | ||||||||||||||
| /* | ||||||||||||||
| * | ||||||||||||||
| * * Licensed to the Apache Software Foundation (ASF) under one or more | ||||||||||||||
| * * contributor license agreements. See the NOTICE file distributed with | ||||||||||||||
| * * this work for additional information regarding copyright ownership. | ||||||||||||||
| * * The ASF licenses this file to You under the Apache License, Version 2.0 | ||||||||||||||
| * * (the "License"); you may not use this file except in compliance with | ||||||||||||||
| * * the License. You may obtain a copy of the License at | ||||||||||||||
| * * | ||||||||||||||
| * * http://www.apache.org/licenses/LICENSE-2.0 | ||||||||||||||
| * * | ||||||||||||||
| * * Unless required by applicable law or agreed to in writing, software | ||||||||||||||
| * * distributed under the License is distributed on an "AS IS" BASIS, | ||||||||||||||
| * * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||||||||||||||
| * * See the License for the specific language governing permissions and | ||||||||||||||
| * * limitations under the License. | ||||||||||||||
| * | ||||||||||||||
| */ | ||||||||||||||
|
|
||||||||||||||
| package org.apache.hugegraph.unit.mysql; | ||||||||||||||
|
|
||||||||||||||
| import org.apache.hugegraph.backend.store.BackendStore; | ||||||||||||||
| import org.apache.hugegraph.testutil.Assert; | ||||||||||||||
| import org.junit.Before; | ||||||||||||||
| import org.junit.Test; | ||||||||||||||
|
|
||||||||||||||
| public class MysqlTest extends BaseMysqlUnitTest{ | ||||||||||||||
| @Before | ||||||||||||||
| public void setUp(){ | ||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The
Suggested change
|
||||||||||||||
| super.setup(); | ||||||||||||||
| } | ||||||||||||||
|
|
||||||||||||||
| @Test | ||||||||||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The test only verifies that the version is preserved, but doesn't verify:
Suggested additions: @Test
public void testMysqlTruncateClearsDataButPreservesMeta() {
// 1. Insert some test data into graph store
// 2. Record meta version before truncate
// 3. Call truncate
// 4. Verify meta version unchanged
// 5. Verify test data was actually cleared
// 6. Verify Counters table state (if needed)
} |
||||||||||||||
| public void testMysqlMetaVersion(){ | ||||||||||||||
| BackendStore systemStore = this.provider.loadSystemStore(config); | ||||||||||||||
| String beforeVersion = systemStore.storedVersion(); | ||||||||||||||
| this.provider.truncate(); | ||||||||||||||
| String afterInitVersion = systemStore.storedVersion(); | ||||||||||||||
| Assert.assertNotNull(beforeVersion); | ||||||||||||||
| Assert.assertNotNull(afterInitVersion); | ||||||||||||||
| Assert.assertEquals(beforeVersion, afterInitVersion); | ||||||||||||||
| } | ||||||||||||||
| } | ||||||||||||||

There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧹 Minor: Unnecessary whitespace change
This blank line removal is an unrelated formatting change. Please keep formatting changes separate from functional fixes to make code reviews easier.