Uploaded image for project: 'Engineering'
  1. Engineering
  2. ENG-10353

Identical catalog updates sent at the same time to a cluster may all return success, even if only one should

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: V6.3
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
    • Team Backlog:
      RAM team
    • Sprint:
      RAM 17

      Description

      If the current catalog version is N, and you send 3 identical catalog or schema updates to the system at the same time, then you end up with 3 identical catalog/schema migration plans, 3 identical new catalogs and 3 calls to @UpdateApplicationCatalog.

      The first one will do the work and succeed, moving the cluster to catalog version N+1 globally. The second two will notice that they expect catalog version N, but see N+1. Since the desired catalog state is identical to the current N+1, we log a message and then silently succeed the transaction without doing any work.

      See: https://github.com/VoltDB/voltdb/blob/803b0cf50958e1368e927fe541ed4fd9ad0cea58/src/frontend/org/voltdb/sysprocs/UpdateApplicationCatalog.java#L295

      This code exists so we can restart catalog updates idempotently during a cluster failure event. It is not expected to be triggered by a user sending the same thing 3 times.

      The right thing to do is to differentiate between internal restarts and multiple requests for the same change at the same time. Perhaps we could look at the transaction id and verify it's the same on that made the pervious update before we return success?

      Note if you send work 1s apart, you're going to get an error that, for example, you can't create a table because that table already existed. If you hit this new check, 2/3 updates will fail, but the message will just say they failed because you sent them at the same time. We already have a message like this for when the updates requested are not hash-identical.

        Attachments

          Activity

            People

            • Assignee:
              jhugg John Hugg
              Reporter:
              jhugg John Hugg
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: