Skip to content

Consolidate tserver tablet assignment state into TabletAssignmentTracker - #6621

Open
DomGarguilo wants to merge 2 commits into
apache:2.1from
DomGarguilo:unifiedLocking
Open

DomGarguilo wants to merge 2 commits into
apache:2.1from
DomGarguilo:unifiedLocking

Conversation

@DomGarguilo

Copy link
Copy Markdown
Member

Fixes #6587

Moves the tserver's unopened, opening, and online tablet collections into a new
TabletAssignmentTracker that owns them behind a single private lock. Callers use named
atomic transitions (addUnopenedIfNoOverlap, unopenedToOpening, openingToOnline,
awaitNotOpening, ...) instead of synchronizing on TabletServer fields. This resolves the
11 USO_UNSAFE_OBJECT_SYNCHRONIZATION findings from SpotBugs 4.10, removes the three-level
lock ordering, and makes the assignment state machine unit-testable. OnlineTablets is
folded into the tracker.

Most of the tracker is existing logic moved as-is. To see only the real changes:
git diff --color-moved=dimmed-zebra --color-moved-ws=allow-indentation-change upstream/2.1...unifiedLocking

Behavior changes

  • One lock instead of three, so there is no lock ordering.
  • Fixes the lock-order deadlock on the rolled-back split path, the same issue as Fixed possible deadlock issue during fixSplit #6588.
  • loadTablet overlap error logging runs after the lock is released, using tablets captured
    under the lock. Its output is unchanged.

Testing: new TabletAssignmentTrackerTest; tserver unit tests; -Psunny ITs; SpotBugs
4.9.8 and 4.10.5 (the 11 findings are gone).

Co-authored-by: Claude Opus 5.5

Comment on lines +119 to +120
waiter.start();
waiter.join(100);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's guaranteed that the thread has actually started running here. I typically handle this by waiting on a CountDownLatch, and call countDown in Thread.run.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I think you're right. A latch would still leave a gap though before wait() so I just opted for a polling loop to wait until the thread is WAITING. Also added a @Timeout to the test case to protect against that loop. I tried removing the notifyAll call temporarily and the test failed so I think things are correct here now.

@dlmarion dlmarion left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loogs good

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants