Skip to content

Avro adapter: setPosition doesn't work with BaseVariableWidthVectors #833

Description

@sap1ens

Describe the bug, including details regarding any error messages, version, and platform.

BaseVariableWidthVector-based vectors (VarBinaryVector, VarCharVector) expect the data to be written sequentially. This means Consumer.setPosition is not safe to use. Imagine the following scenario:

  • write index 0
  • write index 2
  • write index 1
  • write index 3

On the last step, BaseVariableWidthVector.fillHoles erases the data written for the index 2:

  protected final void fillHoles(int index) {
    for (int i = lastSet + 1; i < index; i++) {
      setBytes(i, emptyByteArray, 0, emptyByteArray.length);
    }
    lastSet = index - 1;
  }

Here, index would be 3, lastSet + 1 would be 2 (1 + 1), so setBytes(2, emptyByteArray, ...) is called.

Not sure what the right solution is here, but at a minimum, the library shouldn't allow calling setPosition for Consumers using BaseVariableWidthVector vectors.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type: bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions