Uploaded image for project: 'Engineering'
  1. Engineering
  2. ENG-10736

AAUIW VoltDB Hive Storage Handler to work with Beeline CLI

    XMLWordPrintable

    Details

    • Team Backlog:
      ECO team
    • Sprint:
      ECO-55

      Description

      A customer is trying to use the VoltDB Hive Storage Handler, appears to have configured it correctly and encountered an error after the CREATE TABLE command. Later reported that this was using Beeline CLI. When using the Hive CLI, it works. Would like to have it work also with Beeline CLI.

      About Beeline:

      Why Replace the Existing Hive CLI?

      Hive CLI is a legacy tool which had two main use cases. The first is that it served as a thick client for SQL on Hadoop and the second is that it served as a command line tool for Hive Server (the original Hive server, now often referred to as "HiveServer1"). Hive Server has been deprecated and removed from the Hive code base as of Hive 1.0.0 (HIVE-6977) and replaced with HiveServer2 (HIVE-2935), so the second use case no longer applies. For the first use case, Beeline provides or is supposed to provide equal functionality, yet is implemented differently from Hive CLI.

      Ideally, Hive CLI should be deprecated as the Hive community has long recommended using the Beeline plus HiveServer2 configuration; however, because of the wide use of Hive CLI, we instead are replacing Hive CLI's implementation with a new Hive CLI on top of Beeline plus embedded HiveServer2 (HIVE-10511) so that the Hive community only needs to maintain a single code path. In this way, the new Hive CLI is just an alias to Beeline at both the shell script level and the high code level. The goal is that no or minimal changes are required from existing user scripts using Hive CLI.

      See: https://cwiki.apache.org/confluence/display/Hive/Replacing+the+Implementation+of+Hive+CLI+Using+Beeline

      Using:
      Hive 1.1.0-cdh5.7.1
      Cloudera Distrubution cdh5.7.1
      Beeline version 1.1.0-cdh5.7.1 by Apache Hive
      0: jdbc:hive2://hive.prod.hd.some-customer.com:> 
      0: jdbc:hive2://hive.prod.hd.some-customer.com:> add jar hdfs:///share/lib/voltdb/voltdb-hadoop-1.1-SNAPSHOT.jar;
      No rows affected (0.108 seconds)
      0: jdbc:hive2://hive.prod.hd.some-customer.com:> add jar hdfs:///share/lib/voltdb/voltdb-hive-1.1-SNAPSHOT.jar;
      No rows affected (0.023 seconds)
      0: jdbc:hive2://hive.prod.hd.some-customer.com:> add jar hdfs:///share/lib/voltdb/voltdbclient-6.4.jar;
      No rows affected (0.047 seconds)
      0: jdbc:hive2://hive.prod.hd.some-customer.com:> CREATE TABLE HIVE_AUTOLOAD_TEST (
      0: jdbc:hive2://hive.prod.hd.some-customer.com:>    IFIELD INT,
      0: jdbc:hive2://hive.prod.hd.some-customer.com:>    LFIELD BIGINT,
      0: jdbc:hive2://hive.prod.hd.some-customer.com:>    FFIELD DOUBLE,
      0: jdbc:hive2://hive.prod.hd.some-customer.com:>    SFIELD STRING,
      0: jdbc:hive2://hive.prod.hd.some-customer.com:>    TFIELD TIMESTAMP
      0: jdbc:hive2://hive.prod.hd.some-customer.com:> ) STORED BY 'org.voltdb.hive.VoltStorageHandler' 
      0: jdbc:hive2://hive.prod.hd.some-customer.com:>   WITH SERDEPROPERTIES(
      0: jdbc:hive2://hive.prod.hd.some-customer.com:>       'voltdb.table'='HIVE_AUTOLOAD_TEST',
      0: jdbc:hive2://hive.prod.hd.some-customer.com:>       'voltdb.servers'='172.26.90.101,172.26.90.102,172.26.90.103,172.26.90.104,172.26.90.105'
      0: jdbc:hive2://hive.prod.hd.some-customer.com:> );
      INFO  : Compiling command(queryId=hive_20160708005050_32d2a961-be2d-4c57-bb30-ec9865b5fea9): CREATE TABLE HIVE_AUTOLOAD_TEST (
         IFIELD INT,
         LFIELD BIGINT,
         FFIELD DOUBLE,
         SFIELD STRING,
         TFIELD TIMESTAMP
      ) STORED BY 'org.voltdb.hive.VoltStorageHandler' 
        WITH SERDEPROPERTIES(
            'voltdb.table'='HIVE_AUTOLOAD_TEST',
            'voltdb.servers'='172.26.90.101,172.26.90.102,172.26.90.103,172.26.90.104,172.26.90.105'
      )
      INFO  : Semantic Analysis Completed
      INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
      INFO  : Completed compiling command(queryId=hive_20160708005050_32d2a961-be2d-4c57-bb30-ec9865b5fea9); Time taken: 0.018 seconds
      INFO  : Concurrency mode is disabled, not creating a lock manager
      INFO  : Executing command(queryId=hive_20160708005050_32d2a961-be2d-4c57-bb30-ec9865b5fea9): CREATE TABLE HIVE_AUTOLOAD_TEST (
         IFIELD INT,
         LFIELD BIGINT,
         FFIELD DOUBLE,
         SFIELD STRING,
         TFIELD TIMESTAMP
      ) STORED BY 'org.voltdb.hive.VoltStorageHandler' 
        WITH SERDEPROPERTIES(
            'voltdb.table'='HIVE_AUTOLOAD_TEST',
            'voltdb.servers'='172.26.90.101,172.26.90.102,172.26.90.103,172.26.90.104,172.26.90.105'
      )
      INFO  : Starting task [Stage-0:DDL] in serial mode
      ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
      INFO  : Completed executing command(queryId=hive_20160708005050_32d2a961-be2d-4c57-bb30-ec9865b5fea9); Time taken: 0.03 seconds
      Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: java.lang.reflect.InvocationTargetException (state=08S01,code=1)

        Attachments

          Activity

            People

            • Assignee:
              zhe Zeeman He
              Reporter:
              bballard Ben Ballard
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: